ratescan: add possibility to change fCounterMax and fResolution during runtime

[dneise]

I think it might be useful to be able to change the values of fResolution and fCounterMax also at runtime, so that long running rate scans can be executed, without the need to restart the program.

I think it is fine to let the user change these values in any state, even in "InProgress", similar to CHANGE_STEP_SIZE and the like.

fCounterMax is only used in HandleTriggerRates() and even if the user changes fCounterMax during a ratescan for every step in a crazy way, the results will never be wrong.

The same is true for fResolution. The results will never be wrong, just there will be nothing like a minimal resolution anymore. But since for each point in a ratescan, we do not only store the rate, but also the time, it took to take the point, we have the resolution available for every point. So there is no problem.

[tbretz]

We had discussed that many years ago and decided not to do that to avoid that (real) data can be taken with a wrong setup. If we now decide that this becomes more than "special test runs", we first need to find a reasonable solution which ensures that normal data taking can not be influenced because of somebody who by chance forgot to set values back. Re-starting a program is usually something you remember more easily and there is not much you can do wrong.

[dneise]

I did not mean to say, that the decision was wrong, when it was made years ago. Instead I think, the way software is used, changes and thus certain design decisions might be subject to change as well. One reason, why software needs maintenance, is imho exactly this: User requirements do change.
I'd be happy, if we could agree on the general justification of software maintenance.

If we now decide that this becomes more than "special test runs", we first need to find a reasonable solution which ensures that normal data taking can not be influenced because of somebody who by chance forgot to set values back.

On the contrary, nearly every measurement that is taken with FACT nowadays can be scripted. The scripts (be it JavaScript, the old DIM scripts, or python scripts via PyDim) can all be written such that, when the measurement is finished, the settings, that were changed in order to perform the measurement, are set back to their former values. Of course this is the responsibility of the person conducting the measurement (as it would be, when there are no scripts).

Quite some measurements are to be conducted during dark time, so by the shifter instead of the person, who devised the measurement. For long running measurements, people might want to share the work, so in some cases also during day time, a measurement might not be conducted by the person devising the experiment. Again in some cases, people might want to repeat a measurement, that has been performed months or years ago, in which case the script works at least as some kind of documentation (if it does not happen to be still functional). In these use cases conducting a measurement (also a very special once-and-never-again) measurement, is much easier when there is a script that can be invoked, instead of a manual that is to be followed. Because, as we all know, there is hardly any prose as precise and source code.

Therefore it is my opinion, that the programs should support scripting as much as possible.
If nobody shares this opinion (as it looks like atm), this ticket should be closed as "won't fix" or so...

[dneise]

--- mail by Adrian ---

Dear Colleagues,

I support the idea to be able to be able to do as much as possible without
the need for restarting any program, as well as agree it is important to
ensure that standard stuff is done with correct default values.

Is it possible to have a startup script (for begining of the night)
setting all parameters to default ? This would take care if somebody
changed something the previous shift or during daytime to nonstandard
without switching back, everything is correct before datataking starts.

Regards
Adrian

[dneise]

--- mail by Thomas Bretz ---

Dear Adrian,

this is clearly not the idea of our system. Daniela has spend a lot of time to create a database which stores exactly the configuration for each run taken so that it is reproducible.

There is a reason that so far we have had almost none inconsistencies in our data, simply because the system by construction doesn't allow for such inconsistencies. As soon as you allow inconsistencies there will be inconsistencies.

A startup script does not solve the problem, it makes it even worse, because everybody can easily change it and it's configuration is in the worst case lost forever.

The only solution is to have something (as in fadctrl or ftmctrl) which doesn't allow for inconsistencies. As in fadctrl and ftmctrl one could define run-types in the database and and use them to setup each and every resource of the configuration before a run is started.

What people do manually is their responsibility. What happens automatically must avoid inconsistencies by construction not by script&hope.

Thomas.

[dneise]

This ticket has been discussed quite a bit yesterday in the telcon.
The reason, for this topic to come up in the telcon was the fact, that it actually happened, that a shifter took a special ratescan with changed settings of fCounterMax and fResolution, but forgot to restart the ratescan program with the nominal values.

Daniela explained to us, how one can on the one hand stay consistent with the original design of FACT++, and still take these special ratescans. The solution she presented is based on the creation of dedicated "ratescan-types".

Currently, in order to take a rate scan, the command RATE_SCAN/START_THRESHOLD_SCAN is issued. The program ratescan will then according to the command parameters fThresholdMin, fThresholdMax and fThresholdStep conduct the measurement. During the ratescan, it will consider the start up parameters fCounterMax and fResolution, in order to decide how to proceed.

So in order to allow for different rate scan types, one could introduce a new command with this prototype:

    RATE_SCAN/NEW_THRESHOLD_SCAN <min> <max> <step> <TypeName>

In this sense, this command would look and behave similar to the command:

    MCP/START  <TimeMax> <NumMax> <Name>

---

So when this new command would be used to start a ratescan, it would work roughly like this:

• ratescan can look up the values of fCounterMax and fResolution, from the DB using TypeName.
• min, max and step are provided as usual directly as command parameters.
• It can not happen, that the next ratescan is affected by the settings made by a prior ratescan, since the TypeName must be given explicitely for each call of NEW_THRESHOLD_SCAN
• In order to inform the user of the rate scan results about the TypeName, the TypeName could be included into the service named PROCESS_DATA. Using this TypeName everybody can easily look up the corresponding values of fCounterMax and fResolution.
• The proposed types could be something like these (values to be saved in DB, not fix):
  • normal = {fResolution: 0.05, fCounterMax: 150}
  • closed_lid_short = {fResolution: 0.2, fCounterMax: 1000}
  • closed_lid_middle = {fResolution: 0.15, fCounterMax: 1000}
  • closed_lid_long = {fResolution: 0.1, fCounterMax: 1000}
• In order to make the look up of fResolution easier for the user, Daniela agreed to my request to put the information of the value fResolution into the PROCESS_DATA service in addition to the TypeName.

---

So to sum up the differences and similarities between my initial proposal and the refined solution presented by Daniela:

Similarities:

• Both fResolution and fCounterMax can be changed during run time of the ratescan program.

Differences:

• The user is not free to choose any values for fResolution and fCounterMax, but is given a choice from a number of different Types.
• Each ratescan implicitely defines the values for fResolution and fCounterMax via its Type, so it is excluded by design, that a ratescan is take with inapropriate settings. So nobody can accidentally start a ratescan with "old" settings for fResolution and fCounterMax.
• If a ratescan should be taken with settings, that are not covered by an already existing ratescantype, a new type might be created (if apropriate) in the DB. Only in this case, the program needs to be restarted for the DB changes to go into effect.

[dneise]

I wonder, what would happen, if we change that solution proposed by Daniela a little bit.If one could omit the entire introduction of new ratescan types altogether, but stick to the idea about the NEW_THRESHOLD_SCAN. One could implement it like this:

RATE_SCAN/NEW_THRESHOLD_SCAN <min> <max> <step> <fResolution> <fMaxCount>

This way, we still make sure, that no ratescan, that is conducted, suffers from strange settings, made in the past. Instead for each ratescan all 5 settings must be explicitely given.

The fact, that one has to write "0.05 150" instead of "normal" or "0.2 10000" instead of "closed_lid_no_bias", can be seen as a drawback, because one needs to memorize numbers instead of words, but I think this explicitness is in fact an advantage.

In my opinion this approach has the clear advantage, that neither database access is needed during normal operation, so there is one dependency less to be considered. And in case special ratescans are needed, there is no need to invent a new name and enter the ratescan into the DB prior to do the measurement, one simply specifies the settings.

When the later the results of the ratescans are to be looked at, all 5 settings are provided by the PROCESS_DATA service, and so misunderstandings about the meaning of the ratescans are unlikely, as was the case in the prior proposal.

[tbretz]

Can you imagine what had happened if not all setting for taking data were stored in the database but provided by the user from the commandline. Until we took data automatically, we would not have had two runs with two identical settings and even after that we had most probably taken runs with one setting the one day and runs with another setting the other day. Even worse... just to take a normal ratescan manually you will now need to read instructions first to find all the right settings.