The SHUTTLE

Author: A. Colla

Updated by: C. Zampolli

 

Last updated: June, 16th 2008

Preprocessor status table.
  Preprocessor implementation DAQ DCS HLT
  FXS Archive DB FXS FXS
  in cvs test failover test DA in production output files DP names DP implem. DP in prod. DA implem. output files DA implem. output files
ACORDE (ACO) Yes
    Missing Ok Ok Missing  Ok Not needed Not needed Not needed Not needed
EMCAL (EMC) Yes     Missing Ok Ok Missing Missing Not needed Not needed Unknown Unknown
FMD Yes     in progress Ok Not needed Not needed Not needed Not needed Not needed Not needed Not needed
HMPID (HMP) Yes     Ok Ok Ok Ok Ok Not needed Not needed Not needed Not needed
MUON tracker (MCH) Yes     Ok Ok Ok Ok Ok Missing Ok Not needed Not needed
MUON trigger (MTR) Yes     in progress Ok Ok Ok Ok Not needed Not needed Not needed Not needed
PHOS emc (PHS) Yes     in progress Ok Not needed Not needed Ok Unknown Unknown Missing Ok
PHOS cpv (CPV) Yes     Missing Ok Not needed Not needed Not needed Unknown Unknown Missing Missing
PMD Yes     Ok Ok Ok Ok in progress Not needed Not needed Not needed Not needed
 
  Preprocessor implementation DAQ DCS HLT
  FXS Archive DB FXS FXS
  in cvs test failover test DA in production output files DP names DP implem. DP in prod. DA implem. output files DA implem. output files
ITS spd (SPD) Yes     in progress (1 DA missing) in progress Not needed Not needed Not needed Not needed Not needed Not needed Not needed
ITS sdd (SDD) Yes     1 DA missing? (for INJECTOR triggers) in progress? Ok Ok Ok Not needed Not needed Not needed Not needed
ITS ssd (SSD) Yes     Ok Ok Not needed Not needed Not needed Not needed Not needed Not needed Not needed
TOF Yes     Ok Ok Ok Ok Ok Ok Ok Not needed Not needed
TPC Yes     Ok Ok Ok Ok Ok Ok Ok Ok Missing
TRD Yes     Ok Ok Ok Ok in progress Ok Ok Ok Ok
T0 (T00) Yes     Ok Ok Ok Ok in progress Not needed Not needed Not needed Not needed
VZERO (V00) Yes     Ok Ok Ok Ok Ok Not needed Not needed Not needed Not needed
ZDC Yes     Missing in DAQ Ok Ok Ok Ok Not needed Not needed Not needed Not needed
 
Global Run Parameters (GRP) Yes     Ok Ok Ok Ok Ok Not needed Ok Not needed Not needed

 

Table legenda

  • Detector column: in brackets is the "online" name of the subdetector (ref. ALICE-INT-2003-039 )
  • Preprocessor implementation:
    • In cvs: preprocessor is in AliROOT cvs (colour indicates the level of "completeness")
    • test: preprocessor successfully tested within the SHUTTLE full test setup, in "standard" mode (not fail-over test mode)
      • Status can be "green" (Passed), "red" (Failed) or "yellow" (if any external dependency is missing and the test cannot be performed, e.g. in case the DPs are still to be generated by the DCS team)
    • fail-over test: status of the "behaviour" of the preprocessor in fail-over test mode (e.g. disabling DCS query, FXS error, storage error, etc...)
  • DAQ, DCS, HLT File eXchange Server columns:
    • DA implem: detector algorithm implemented on the online system within the correspondent "DA framework" (e.g. reference: DAQ page -> Calibration, see DAQ for present status in DAQ), used in production. In case of red box, the " Missing in DAQ" label means that the DA code is present in AliRoot, but not in production, while " In Progress" means that the code is in AliRoot, and need some more adjustment in production.
    • output files: Calibration data files produced by the DA's and retrieved by the SHUTTLE through the FXS. The subdetector experts must provide the set of output files required by their preprocessor in order to run the SHUTTLE test campaign.
      • Ok: Link to DA output file sample(s) has been provided, together with the information needed to fill the FXS databases (system, file Id, source)
  • DCS Archive DB column:
    • DP names: The basic elements monitored by the DCS are called Data Points (DP). the SHUTTLE full test setup requires the simulation and archival of a sample of DP values. The "DP value generator" is provided by the DCS system. The parameters needed by the generator are:
      • Data Point or "alias" name;
      • Number of channels (-> N. of DPs);
      • Data type (bool, char, unsigned int, int, float);
      • Mean value;
      • Fluctuation around the mean value (in %)
      • Update frequency;

      The subdetector experts must provide a set of "realistic" parameters for the DPs required by their preprocessor.

    • DP implem.: indicates whether preprocessor already implemented the algorithms to process the retrieved DCS DP map, possibly reduce their data size, store them into OCDB.
  • Nota bene: The table is being filled with the information provided directly by the detector experts or, when not available, from the calibration requirements connected in the calibration readiness tables. Please verify what inserted and help us with the "unknown" cells!!

 

 

Preprocessor methods (to be implemented in the detector-specific preprocessor)

Base functions (to be implemented in the detector-specific preprocessor)

  • void Initialize (Int_t run, UInt_t startTime, UInt_t endTime)
    • Description: Called by Shuttle at each run, before Process. If needed, implement in this function the initialization of the preprocessor (declaration of histograms, arrays, etc)
  • UInt_t Process (TMap* dcsMap)
    • Description: Called by Shuttle at each run. Query of data from the online networks, processing and storage must be implemented here.
    • Return code: 0: success; > 0: failure
  • Bool_t ProcessDCS () 
    • Description: Called by Shuttle before DCS archive data query. If not implemented it returns TRUE. Preprocessors may disable DCS query (for instance in case of specific run types which do not need data from DCS) implementing this function.
    • Example:
         Bool_t ProcessDCS {
          TString runType = GetRunType();
          if(runType == "PEDESTALS")
            return kFALSE;
          return kTRUE;
        }

Helper functions (call delegated SHUTTLE functions)

  • TList* GetFileSources (Int_t system, const char* id)
    • Description: Retrieves the list of the online sources where the conditions data files have been created. To be used in conjunction with GetFile.
    • Example: TList* fileSources = GetFile(AliShuttleInterface::kDAQ, "pedestals");
    • Important: consider that GetFileSources may return:
      1. a null pointer to TList in case of problems connecting to the MySql database;
      2. an empty (but non-zero) TList in case the DA did not produce any files.

      The first two possibilities require different actions in the preprocessor:

      1. In this case the preprocessor should return error (>0) always, since this means that some technical problem happened in the Shuttle, which will be hopefully solvedsoon by the Shuttle team :);
      2. This case is more weird, because it may depend on how the online DA works:
        1. if the DA may or may not produce results from time to time (e.g. if it requires the collection of enough statistics over more runs) then if no entries are found the preprocessor should simply return success (0);
        2. if the DA is expected to always produce results, then pp should return error , since it means the DA failed.

      We suggest to implement the DA in a way that it *always* return something: this would make the error checking easier.

  • const char* GetFile (Int_t system, const char* id, const char* source)
    • Description: Retrieves a file from one of the online FXS. The file is copied to the local disk and the path to the local copy is returned.
    • Example: const char* fileName = GetFile(AliShuttleInterface::kDAQ, "pedestals", "LDC1");
  • const char* GetRunParameter (const char* param)
    • Description: Retrieves the current run logbook parameter.
    • Example: const char* time_start = GetRunParameter("time_start");
  • AliCDBEntry* GetFromOCDB (const char* pathLevel2, const char* pathLevel3)
    • Description: Retrieves object "/pathLevel2/pathLevel3" valid for the current run from OCDB.
    • Example: AliCDBEntry* entry = GetFromOCDB("Calib", "Data");
  • const char* GetRunType ()
    • Description: Retrieves current run type (PHYSICS, PEDESTALS, etc.) from logbook.
    • Example: const char* runType = GetRunType();
  • Bool_t GetHLTStatus ()
    • Description: Retrieves status of HLT for the current run, read from run logbook. This is useful to decide whether to retrieve files from the HLT FXS or not.
      The return code is: 1 = ON / 0 = OFF
    • Example: Bool_t hltStatus = GetHLTStatus();
  • Bool_t Store (const char* pathLevel2, const char* pathLevel3, TObject* object,
                         AliCDBMetaData* metaData, Int_t validityStart, Bool_t validityInfinite)
    • Description: Stores the object in the OCDB. The object's path will be "DET/pathLevel2/pathLevel3", where DET is the detector's "offline" name (e.g. "HMP" -> "HMPID").
      The object's validity starts from the current run minus validityStart. If validityInfinite = TRUE the object's validity is set up to infinity (run 999999999).
      The return code is: 
      • TRUE: Success
      • FALSE Failure
    • Example: Bool_t result = Store("Calib", "Data", metaData, object, 0, kTRUE);
  • Bool_t StoreReferenceData (const char* pathLevel2, const char* pathLevel3, TObject* object, AliCDBMetaData* metaData)
    • Description: Stores the object in the reference storage. See Store function for full description.
    • Example: Bool_t result = StoreReferenceData("Calib", "Data", metaData, object);
  • Bool_t StoreReferenceFile (const char* localFileName, const char* gridFileName) 
    • Description: Stores a file directly into the reference folder on the Grid, without opening it.
      The file will be saved in the following path:
      baseGridReferenceFolder/ DET >/ runNumber > _gridFileName
      example: /alice/data/2008/.../Reference/ITS/1000_CalibReferenceSPD.root
    • Example: Bool_t result = StoreReferenceFile(localFileName, "calib_reference.root");
  • void Log (const char* message)
    • Description: Writes a message in the detector's log file.
    • Example: Log("Preprocessor is done");

New! Functions to retrieve the time interval during which the query to the DCS db for the DPs was made (call delegated to SHUTTLE functions)

New functions have been introduced in the Shuttle code to follow the modifications in the query to the DCS db for the retrieval of the DPs. Instead of querying the database starting from the DAQ Start of Data Taking to the DAQ End of Data Taking, the time interval will start at the Time when the Run Number was created minus a certain offset, and will end at the so called Time Completed plus the same offset. Time Completed is the time when all the post-end-of-run processes have finished, which comes after the End of Data Taking, and corresponds to the time when a new run can start. Please note that startTime and endTime in the AliPreprocessor::Initialize(Int_t run, UInt_t startTime, UInt_t endTime) function will point to these values.

In order to easily access these new bounds, two new functions have been added to AliPreprocessor. Obviously, the timestamps corresponding to each entry in the TMap that is returned from the query will point at the time when the value was stored, so they can be used to track back the observed interval.

 

IMPORTANT! Since the new changes will imply that values outside the run will be provided to the preprocessors, the preprocessors themselves have to ensure to be able to handle them.

The new functions are:

  • UInt_t GetStartTimeDCSQuery ()
    • Description: Returns the timestamp corresponding to the time when the query to the DCS database for the DPs started.
    • Example: GetStartTimeDCSQuery();
  • UInt_t GetEndTimeDCSQuery ()
    • Description: Returns the timestamp corresponding to the time when the query to the DCS database for the DPs ended.
    • Example: GetEndTimeDCSQuery();

The timing relationship between the Start of the DCS Query, the time when the run number was created, the DAQ start time, the DAQ end time, and the End of the DCS Query are shown in the following picture.

Timing Schema