Official EventBuilding ToolChain (#253)

* Official toolchain for event building

EventBuilder ToolChain - very similar to DataDecoder toolchain, with some minor configuration differences:
- Updated trigger mask to include all triggers (1-64)
- Updated LoadGeometry file path for Gian's new gain calibration data
- Added Johann's SaveConfigInfo tool
- Excluded Stage1DataBuilder until bug fix
- Minor fix in ANNIEEventBuilderConfig (changing true/false to 1/0)
- Updated file paths for more recent data

* Update BeamFetcherConfig

Part of updated the BeamFetcher toolchain (needed prior to event building)

* Update LoadRunInfoConfig

Included a copy of the run info from the SQL webpage in the local toolanalysis directory

* Add files via upload

added local SQL run information for fetching beam info

* Update TriggerDataDecoderConfig

updated path for trigger mask to include all triggers

* Update my_files.txt

added new paths for event building

* Added the updated gains information from Gian

* Update LoadGeometryConfig

Updated gains file path within the LoadGeometryConfig file in the LoadGeometry toolchain - many tools call this config file (configfiles/LoadGeometry), instead of calling their own LoadGeometryConfig file that is present within that specific toolchain. Instead of replacing every path within all config files, replace this one.

* Update DefaultTriggerMask.txt

After testing the new EventBuilder toolchain, including all triggers in the triggermask file seems to slow the toolchain down by 2x (or so). Going back to default file.

* Update DefaultTriggerMask.txt

Added necessary trigwords for AmBe external trigger (15) and laser (46 or 47, dependent on settings. Through the current laser run campaign, various configurations will either lead to the trigword being 46 or 47... not sure why).

* Update ToolsConfig

For now, omit BeamDecoder until we have BeamFetcherv2 up and running

* Update DefaultTriggerMask.txt

Include the undelayed beam trigger for retroactive LAPPD time matching with other subsystems. If there is an LAPPD event, this triggerword can be used to check if the LAPPD event is correctly aligned, in case any hardware was broken (according to Yue)

* Update DefaultTriggerMask.txt

I swear this is the final change. Yue initially requested I put this trigword in, however he recently encountered errors when trying to pair the LAPPDs to the MRD. For now, omitting this trigword until he can find a suitable trigword for LAPPD timing alignment

* Update LoadRawDataConfig

Lowering the verbosity

* Update ANNIEEventBuilderConfig

Since we are omitting the beamfetcher tool for now, do not store the beam information in the final annieevent

* Update ANNIEEventBuilderConfig

change config file comments to use `#` instead of `//`

configfiles/BeamFetcher/BeamFetcherConfig

@@ -2,8 +2,8 @@
 verbose 5
 OutputFile ./2369_beamdb
 TimestampMode DB 	#Options: MSEC / LOCALDATE / DB
-DaylightSavings 0
-RunNumber 2280
+DaylightSavings 1
+RunNumber 4314
 StartDate ./configfiles/BeamFetcher/my_start_date.txt
 EndDate ./configfiles/BeamFetcher/my_end_date.txt
 #StartMillisecondsSinceEpoch 1498246528696

configfiles/BeamFetcher/LoadRunInfoConfig

@@ -1,4 +1,4 @@
 #LoadRunInfo Config
 
 verbosity 1
-RunInfoFile /pnfs/annie/persistent/users/mnieslon/data/runinfo/ANNIE_RunInformation_PSQL.txt
+RunInfoFile ./configfiles/BeamFetcher/ANNIE_RunInformation_PSQL.txt

configfiles/EventBuilder/ANNIEEventBuilderConfig

@@ -0,0 +1,20 @@
+verbosity 0
+
+BuildType TankAndMRDAndCTC
+ProcessedFilesBasename ProcessedRawData
+NumEventsPerPairing 200
+SavePath ./
+
+MinNumWavesInSet 134   # 1=Just throw any waveforms into ANNIE events if you got em
+
+ExecutesPerBuild 10
+OrphanOldTankTimestamps 1
+OldTimestampThreshold 1000
+OrphanFileBase OrphanStore
+MaxStreamMatchingTimeSeparation 60   # seconds. If one stream is ahead of the others, pause reading
+
+SaveRawData 0
+StoreBeamStatus 0
+
+BuildStage1Data 0 
+SaveSeparatePartfiles 1

configfiles/EventBuilder/BeamDecoderConfig

@@ -0,0 +1,17 @@
+# BeamDecoder config file
+verbosity 0
+# Names of devices needed for beam quality cuts
+HornCurrentDevice E:THCURR
+# The "first" toroid is the one farther upstream from the target
+FirstToroid E:TOR860
+SecondToroid E:TOR875
+# POT window
+CutPOTMin 5e11
+CutPOTMax 8e12
+# Peak horn current window (in kA)
+CutPeakHornCurrentMin 172
+CutPeakHornCurrentMax 176
+# Toroid agreement tolerance (fractional error)
+CutToroidAgreement 0.05
+# DB vs DAQ timestamp agreement tolerance (ms)
+CutTimestampAgreement 100

configfiles/EventBuilder/DefaultTriggerMask.txt

@@ -0,0 +1,10 @@
+#Triggers are given by Jonathan; place one number per line. Values are index + 1
+#4+1: Delayed beam.  32+1: LED trigger. 34+1: AmBe and Cosmic trigger.  35+1: MRD_CR_Trigger. 30+1: LED_Start. 45+1, 46+1: Laser. 14+1: AmBe external trigger
+5
+15
+36
+31
+33
+35
+46
+47

configfiles/EventBuilder/LoadGeometryConfig

@@ -0,0 +1,8 @@
+verbosity 0
+LAPPDChannelCount 60
+FACCMRDGeoFile ./configfiles/LoadGeometry/FullMRDGeometry.csv
+DetectorGeoFile ./configfiles/LoadGeometry/DetectorGeometrySpecs.csv
+LAPPDGeoFile ./configfiles/LoadGeometry/LAPPDGeometry.csv
+TankPMTGeoFile ./configfiles/LoadGeometry/FullTankPMTGeometry.csv
+TankPMTGainFile ./configfiles/LoadGeometry/ChannelSPEGains2023.csv
+AuxiliaryChannelFile ./configfiles/LoadGeometry/AuxChannels.csv

configfiles/EventBuilder/LoadRawDataConfig

@@ -0,0 +1,9 @@
+verbosity 0
+BuildType TankAndMRDAndCTC
+#BuildType CTC
+Mode FileList
+InputFile ./configfiles/EventBuilder/my_files.txt
+DummyRunInfo 1
+StoreTrigOverlap 0
+ReadTrigOverlap 1
+StoreRawData 0

configfiles/EventBuilder/MRDDataDecoderConfig

@@ -0,0 +1,2 @@
+verbosity 2
+DaylightSavingsSpring 1

configfiles/EventBuilder/PMTDataDecoderConfig

@@ -0,0 +1,5 @@
+verbosity 0
+ADCCountsToBuildWaves 0
+Mode Offline
+OffsetVME03 0
+OffsetVME01 0

configfiles/EventBuilder/PhaseIIADCCalibratorConfig

@@ -0,0 +1,15 @@
+# PhaseIIADCCalibrator config file
+
+verbosity 0
+
+BaselineEstimationType ze3ra_multi
+NumBaselineSamples 15
+NumSubWaveforms 10
+
+SamplesPerBaselineEstimate 2000
+BaselineUncertaintyTolerance 2
+PCritical 0.01
+MakeCalLEDWaveforms 0
+
+EventBuilding 1
+ExecutesPerBuild 10

configfiles/EventBuilder/PhaseIIADCHitFinderConfig

@@ -0,0 +1,10 @@
+verbosity 0
+
+UseLEDWaveforms 0
+
+PulseFindingApproach threshold
+PulseWindowType dynamic
+DefaultADCThreshold 7
+DefaultThresholdType relative
+
+EventBuilding 1

configfiles/EventBuilder/SaveConfigInfoConfig

@@ -0,0 +1,3 @@
+verbosity 5
+OutFileName config_info.txt
+FullDepth 1

configfiles/EventBuilder/ToolChainConfig

@@ -0,0 +1,23 @@
+#ToolChain dynamic setup file
+
+##### Runtime Paramiters #####
+verbose 1
+error_level 0 # 0= do not exit, 1= exit on unhandeled errors only, 2= exit on unhandeled errors and handeled errors
+attempt_recover 1
+
+###### Logging #####
+log_mode Interactive # Interactive=cout , Remote= remote logging system "serservice_name Remote_Logging" , Local = local file log;
+log_local_path ./log
+log_service LogStore
+
+###### Service discovery #####
+service_publish_sec -1
+service_kick_sec -1
+
+##### Tools To Add #####
+Tools_File ./configfiles/EventBuilder/ToolsConfig
+
+##### Run Type #####
+Inline -1
+Interactive 0
+

configfiles/EventBuilder/ToolsConfig

@@ -0,0 +1,9 @@
+LoadGeometry LoadGeometry ./configfiles/LoadGeometry/LoadGeometryConfig
+LoadRawData LoadRawData ./configfiles/EventBuilder/LoadRawDataConfig
+PMTDataDecoder PMTDataDecoder ./configfiles/EventBuilder/PMTDataDecoderConfig
+MRDDataDecoder MRDDataDecoder ./configfiles/EventBuilder/MRDDataDecoderConfig
+TriggerDataDecoder TriggerDataDecoder ./configfiles/EventBuilder/TriggerDataDecoderConfig
+PhaseIIADCCalibrator PhaseIIADCCalibrator ./configfiles/EventBuilder/PhaseIIADCCalibratorConfig
+PhaseIIADCHitFinder PhaseIIADCHitFinder ./configfiles/EventBuilder/PhaseIIADCHitFinderConfig
+SaveConfigInfo SaveConfigInfo ./configfiles/EventBuilder/SaveConfigInfoConfig
+ANNIEEventBuilder ANNIEEventBuilder ./configfiles/EventBuilder/ANNIEEventBuilderConfig

configfiles/EventBuilder/TriggerDataDecoderConfig

@@ -0,0 +1,4 @@
+verbosity 0
+TriggerMaskFile ./configfiles/EventBuilder/DefaultTriggerMask.txt
+StoreTrigOverlap 0
+ReadTrigOverlap 0

configfiles/EventBuilder/my_files.txt

@@ -0,0 +1,2 @@
+/pnfs/annie/persistent/raw/raw/4314/RAWDataR4314S0p0
+/pnfs/annie/persistent/raw/raw/4314/RAWDataR4314S0p1

configfiles/LoadGeometry/ChannelSPEGains2023.csv

@@ -0,0 +1,125 @@
+#Offline_Tubes: 333,346,349,352,431,444
+#Dead_Tubes: 337,342,345,416,445
+#Sinusoidal_Tubes
+#Channelkey,SPE_Gain
+332,0.001345
+334,0.001421
+335,0.001375
+336,0.001222
+338,0.001387
+339,0.001038
+340,0.001248
+341,0.001543
+343,0.000267
+344,0.001334
+347,0.001570
+348,0.001478
+350,0.001290
+351,0.001455
+353,0.000380
+354,0.001767
+355,0.001198
+356,0.000478
+357,0.001922
+358,0.000923
+359,0.000281
+360,0.001380
+361,0.000748
+362,0.001344
+363,0.001367
+364,0.001223
+365,0.001003
+366,0.000631
+367,0.001649
+368,0.001776
+369,0.000722
+370,0.001148
+371,0.001657
+372,0.001653
+373,0.001443
+374,0.001435
+375,0.001382
+376,0.001476
+377,0.001466
+378,0.001382
+379,0.001521
+380,0.001265
+381,0.001465
+382,0.001248
+383,0.001465
+384,0.001618
+385,0.001540
+386,0.001438
+387,0.001528
+388,0.001353
+389,0.001616
+390,0.001268
+391,0.001271
+392,0.001406
+393,0.001628
+394,0.001487
+395,0.001701
+396,0.001678
+397,0.001547
+398,0.001368
+399,0.001428
+400,0.001328
+401,0.001480
+402,0.001311
+403,0.001130
+404,0.001569
+405,0.000906
+406,0.001542
+407,0.001491
+408,0.001396
+409,0.001414
+410,0.001386
+411,0.001278
+412,0.001401
+413,0.001499
+414,0.001488
+415,0.001217
+417,0.001468
+418,0.001457
+419,0.001248
+420,0.001744
+421,0.001398
+422,0.001409
+423,0.001324
+424,0.001548
+425,0.001455
+426,0.001274
+427,0.001316
+428,0.001420
+429,0.001338
+430,0.001515
+432,0.001307
+433,0.001279
+434,0.001357
+435,0.001416
+436,0.001345
+437,0.001448
+438,0.001350
+439,0.001398
+440,0.001487
+441,0.001459
+442,0.001291
+443,0.001236
+446,0.001495
+447,0.001373
+448,0.001353
+449,0.001375
+450,0.001321
+451,0.001302
+452,0.001618
+453,0.001466
+454,0.001241
+455,0.001388
+456,0.001535
+457,0.001650
+458,0.001478
+459,0.001360
+460,0.001463
+461,0.001375
+462,0.001420
+463,0.001396

configfiles/LoadGeometry/LoadGeometryConfig

@@ -5,5 +5,5 @@ FACCMRDGeoFile ./configfiles/LoadGeometry/FullMRDGeometry_09_29_20.csv
 DetectorGeoFile ./configfiles/LoadGeometry/DetectorGeometrySpecs.csv
 LAPPDGeoFile ./configfiles/LoadGeometry/LAPPDGeometry.csv
 TankPMTGeoFile ./configfiles/LoadGeometry/FullTankPMTGeometry.csv
-TankPMTGainFile ./configfiles/LoadGeometry/ChannelSPEGains_BeamRun20192020.csv
+TankPMTGainFile ./configfiles/LoadGeometry/ChannelSPEGains2023.csv
 AuxiliaryChannelFile ./configfiles/LoadGeometry/AuxChannels.csv

configfiles/PreProcessTrigOverlap/TriggerDataDecoderConfig

@@ -1,5 +1,5 @@
 verbosity 2
-TriggerMaskFile ./configfiles/DataDecoder/DefaultTriggerMask.txt
+TriggerMaskFile ./configfiles/EventBuilder/DefaultTriggerMask.txt
 StoreTrigOverlap 1
 ReadTrigOverlap 0
 UseCStore 0