# # ChangeLog for trunk/DataCheck # # Generated by Trac 1.5.2 # 03/19/25 03:11:39 Wed, 04 Apr 2012 16:12:21 GMT Daniela Dorner [13313] * trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified) * trunk/DataCheck/Transfer/RsyncRawLP.sh (modified) improved logging Wed, 04 Apr 2012 15:28:17 GMT Daniela Dorner [13312] * trunk/DataCheck/Transfer/CheckTransfer.sh (modified) adapted parameters to run script via cron Wed, 04 Apr 2012 15:27:21 GMT Daniela Dorner [13311] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) added crons for CheckTransfer.sh and CheckDU.sh Wed, 04 Apr 2012 15:26:29 GMT Daniela Dorner [13310] * trunk/DataCheck/Sourcefile.sh (modified) warn only if lockfile older than 5 minutes Wed, 04 Apr 2012 15:25:44 GMT Daniela Dorner [13309] * trunk/DataCheck/Processing/CheckRawFilesAvail.sh (modified) added check for /archive/fact/fails/raw Wed, 04 Apr 2012 11:49:02 GMT Daniela Dorner [13308] * trunk/DataCheck/Setup/setup.lp.fact (deleted) replaced by new version Wed, 04 Apr 2012 11:48:41 GMT Daniela Dorner [13307] * trunk/DataCheck/Setup/setup.isdc.fact (deleted) replaced by new version Wed, 04 Apr 2012 11:45:27 GMT Daniela Dorner [13306] * trunk/DataCheck/Setup/setup.fact.isdc (added) * trunk/DataCheck/Setup/setup.fact.lp.data (added) * trunk/DataCheck/Setup/setup.fact.lp.gate (added) added (new versions of setup files to include all variables) Wed, 04 Apr 2012 11:44:47 GMT Daniela Dorner [13305] * trunk/DataCheck/Transfer/BackupDatabase.sh (added) added (script to backup databases) Wed, 04 Apr 2012 11:43:41 GMT Daniela Dorner [13304] * trunk/DataCheck/Cron/crontab.fact01 (modified) new version to use /users/fact instead of /home/fact Wed, 04 Apr 2012 11:42:59 GMT Daniela Dorner [13303] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) * trunk/DataCheck/Cron/crontab.fact-viewer (modified) * trunk/DataCheck/Cron/crontab.fact1 (modified) fixed typo Wed, 04 Apr 2012 11:41:46 GMT Daniela Dorner [13302] * trunk/DataCheck/Sourcefile.sh (modified) removed things which belong into setup files Wed, 04 Apr 2012 11:40:50 GMT Daniela Dorner [13301] * trunk/DataCheck/Monitoring/CheckDU.sh (modified) adapted script for more hosts Fri, 30 Mar 2012 17:07:31 GMT Daniela Dorner [13293] * trunk/DataCheck/Cron/crontab.fact1 (modified) fixed path Fri, 30 Mar 2012 17:00:33 GMT Daniela Dorner [13292] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) added nice and ionice for processes working with data Fri, 30 Mar 2012 16:58:02 GMT Daniela Dorner [13291] * trunk/DataCheck/Cron/crontab.fact1 (modified) added nice and ionice for processes working with data Thu, 29 Mar 2012 10:15:58 GMT Daniela Dorner [13289] * trunk/DataCheck/Setup/steps_fact_isdc.rc (modified) added DriveFileAvailISDC and changed requirement for Star from ... Thu, 29 Mar 2012 10:14:20 GMT Daniela Dorner [13288] * trunk/DataCheck/Processing/CheckDriveFileAvail.sh (added) added (script to check availability of ... Thu, 29 Mar 2012 10:09:21 GMT Daniela Dorner [13287] * trunk/DataCheck/Processing/CheckRawData.sh (modified) added nightly insert for new table DriveFileAvailISDCStatus Thu, 29 Mar 2012 09:03:53 GMT Daniela Dorner [13285] * trunk/DataCheck/Processing/CheckAuxFilesAvail.sh (modified) * trunk/DataCheck/Processing/CheckRawFilesAvail.sh (modified) give only warning if files older than some days are missing Thu, 29 Mar 2012 09:02:48 GMT Daniela Dorner [13284] * trunk/DataCheck/Processing/BuildSequences.sh (modified) fixed typo Thu, 29 Mar 2012 09:02:06 GMT Daniela Dorner [13283] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) run CheckStatus.sh only twice a day Thu, 29 Mar 2012 09:01:31 GMT Daniela Dorner [13282] * trunk/DataCheck/Cron/crontab.fact-viewer (modified) added jobmanager and SGE environment Thu, 29 Mar 2012 08:59:34 GMT Daniela Dorner [13281] * trunk/DataCheck/Cron/crontab.fact1 (modified) fixed typo Thu, 22 Mar 2012 16:57:10 GMT Daniela Dorner [13173] * trunk/DataCheck/Processing/BuildSequences.sh (added) added (script to build sequences) Thu, 22 Mar 2012 16:56:38 GMT Daniela Dorner [13172] * trunk/DataCheck/Cron/crontab.fact-viewer (added) added (crontab file for fact_opr on isdc-viewer00) Thu, 22 Mar 2012 16:55:48 GMT Daniela Dorner [13171] * trunk/DataCheck/Transfer/CheckTransfer.sh (modified) improved output, fixed bugs Wed, 21 Mar 2012 13:50:08 GMT Daniela Dorner [13158] * trunk/DataCheck/Processing/CheckRawData.sh (modified) added insert to nightly status tables, removed insert for status ... Mon, 19 Mar 2012 17:20:12 GMT Daniela Dorner [13154] * trunk/DataCheck/Transfer/CheckTransfer.sh (added) added (script to check status of transfer for deleting data) Mon, 19 Mar 2012 17:09:41 GMT Daniela Dorner [13153] * trunk/DataCheck/Processing/CheckRawFilesAvail.sh (modified) treat case that file is not yet there Mon, 19 Mar 2012 17:09:05 GMT Daniela Dorner [13152] * trunk/DataCheck/Transfer/BackupRawToWue.sh (modified) bugfix for drsfiles Wed, 14 Mar 2012 19:26:48 GMT Daniela Dorner [13108] * trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified) added check if directory exists Wed, 14 Mar 2012 19:25:37 GMT Daniela Dorner [13107] * trunk/DataCheck/Setup/setup.lp.fact (modified) * trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified) * trunk/DataCheck/Transfer/RsyncRawLP.sh (modified) added tmp path for rsyn Wed, 14 Mar 2012 10:12:07 GMT Daniela Dorner [13099] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) run CheckStatus.sh more often Wed, 14 Mar 2012 09:59:13 GMT Daniela Dorner [13098] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) adapted time for one job Wed, 14 Mar 2012 09:58:45 GMT Daniela Dorner [13097] * trunk/DataCheck/Cron/crontab.fact1 (modified) removed not needed lines Wed, 14 Mar 2012 09:03:44 GMT Daniela Dorner [13094] * trunk/DataCheck/Monitoring/CheckStatus.sh (modified) improved logging Wed, 14 Mar 2012 09:03:19 GMT Daniela Dorner [13093] * trunk/DataCheck/Transfer/BackupRawToWue.sh (modified) added drs file to transfer Tue, 13 Mar 2012 14:49:22 GMT Daniela Dorner [13082] * trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified) added drs-files to rsync and check for existance of links Tue, 13 Mar 2012 11:19:11 GMT Daniela Dorner [13079] * trunk/DataCheck/Processing/RunStar.sh (modified) extra log file for merpp Sun, 11 Mar 2012 17:41:57 GMT Daniela Dorner [13067] * trunk/DataCheck/Cron/crontab.fact1 (modified) added new monitoring script Sun, 11 Mar 2012 17:24:17 GMT Daniela Dorner [13066] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) added new monitoring scripts Sun, 11 Mar 2012 17:10:36 GMT Daniela Dorner [13065] * trunk/DataCheck/Setup/setup.isdc.fact (modified) * trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified) * trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified) added temp directory for rsync Sun, 11 Mar 2012 17:09:15 GMT Daniela Dorner [13064] * trunk/DataCheck/Monitoring/CheckStatus.sh (added) added (script to check database for failed and crashed jobs) Sun, 11 Mar 2012 17:08:45 GMT Daniela Dorner [13063] * trunk/DataCheck/Monitoring/CheckLogs.sh (added) added (script to check global logfile for errors and warnings) Sat, 10 Mar 2012 00:17:00 GMT Daniela Dorner [13059] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) adapted paths Fri, 09 Mar 2012 23:32:07 GMT Daniela Dorner [13058] * trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified) Fri, 09 Mar 2012 23:31:33 GMT Daniela Dorner [13057] * trunk/DataCheck/Processing/CheckRawData.sh (modified) added priority to status queries Fri, 09 Mar 2012 22:58:38 GMT Daniela Dorner [13056] * trunk/DataCheck/Cron/crontab.fact-transfer (modified) * trunk/DataCheck/Cron/crontab.fact01 (modified) * trunk/DataCheck/Cron/crontab.fact02 (modified) * trunk/DataCheck/Cron/crontab.fact04 (modified) * trunk/DataCheck/Cron/crontab.fact1 (modified) * trunk/DataCheck/Cron/crontab.fact2 (modified) adapted paths in crontab files Fri, 09 Mar 2012 22:27:53 GMT Daniela Dorner [13055] * trunk/DataCheck/Processing/CheckAuxFilesAvail.sh (modified) * trunk/DataCheck/Processing/CheckRawData.sh (modified) * trunk/DataCheck/Processing/CheckRawFilesAvail.sh (modified) * trunk/DataCheck/Processing/FillAuxData.sh (modified) * trunk/DataCheck/Setup/setup.lp.fact (modified) * trunk/DataCheck/Transfer/BackupRawToWue.sh (modified) * trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified) * trunk/DataCheck/Transfer/RsyncRawLP.sh (modified) * trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified) * trunk/DataCheck/Transfer/ZipRawData.sh (modified) adapted scripts to changes in directory structure Fri, 09 Mar 2012 22:18:35 GMT Daniela Dorner [13054] * trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified) * trunk/DataCheck/Transfer/BackupRawToWue.sh (modified) reworked version of scripts doing backup to Wuerzburg Fri, 09 Mar 2012 22:15:38 GMT Daniela Dorner [13053] * trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (added) * trunk/DataCheck/Transfer/RsyncRawToISDC.sh (added) added (scripts for transfer from LP to ISDC) Fri, 09 Mar 2012 22:13:25 GMT Daniela Dorner [13052] * trunk/DataCheck/Setup/setup.lp.fact (added) added (setup file for processing in La Palma) Fri, 09 Mar 2012 22:12:43 GMT Daniela Dorner [13051] * trunk/DataCheck/Setup/setup.isdc.fact (added) * trunk/DataCheck/Setup/steps_fact_isdc.rc (added) added (setup files for processing at ISDC) Fri, 09 Mar 2012 22:11:44 GMT Daniela Dorner [13050] * trunk/DataCheck/Processing/JobManager.sh (added) added (jobmanager to run callisto and star on a computing cluster Fri, 09 Mar 2012 22:09:54 GMT Daniela Dorner [13049] * trunk/DataCheck/Processing/RunCallisto.sh (added) * trunk/DataCheck/Processing/RunStar.sh (added) added (scripts for callisto and star) Fri, 09 Mar 2012 22:08:27 GMT Daniela Dorner [13048] * trunk/DataCheck/Processing/CheckAuxFilesAvail.sh (added) * trunk/DataCheck/Processing/CheckRawFilesAvail.sh (added) added (scripts to check availability of files) Fri, 09 Mar 2012 22:04:53 GMT Daniela Dorner [13047] * trunk/DataCheck/Transfer/RsyncDBToWue.sh (moved) moved db backup script to transfer directory Fri, 09 Mar 2012 22:03:57 GMT Daniela Dorner [13046] * trunk/DataCheck/Transfer/BackupAuxToWue.sh (moved) * trunk/DataCheck/Transfer/BackupRawToWue.sh (moved) moved rsync scripts to transfer directory Fri, 09 Mar 2012 22:01:08 GMT Daniela Dorner [13045] * trunk/DataCheck/Transfer/RsyncAuxLP.sh (moved) * trunk/DataCheck/Transfer/RsyncRawLP.sh (moved) moved La Palma rsync scripts to transfer directory Fri, 09 Mar 2012 21:28:22 GMT Daniela Dorner [13044] * trunk/DataCheck/Setup (added) added (directory for setup files) Fri, 09 Mar 2012 21:27:01 GMT Daniela Dorner [13043] * trunk/DataCheck/Monitoring/CheckDU.sh (moved) * trunk/DataCheck/Monitoring/CheckNTPDate.sh (moved) * trunk/DataCheck/Monitoring/CheckNetworkStatus.sh (moved) moved check scripts to monitoring directory Fri, 09 Mar 2012 21:23:58 GMT Daniela Dorner [13042] * trunk/DataCheck/CheckRawData.sh (deleted) moved to processing directory Fri, 09 Mar 2012 21:23:26 GMT Daniela Dorner [13041] * trunk/DataCheck/FillAuxData.sh (deleted) moved to processing directory Fri, 09 Mar 2012 21:19:11 GMT Daniela Dorner [13040] * trunk/DataCheck/Processing/FillAuxData.sh (copied) moved FillAuxData.sh to processing directory Fri, 09 Mar 2012 21:18:29 GMT Daniela Dorner [13039] * trunk/DataCheck/Processing/CheckRawData.sh (copied) added queries to fill status tables Fri, 09 Mar 2012 21:12:09 GMT Daniela Dorner [13038] * trunk/DataCheck/Transfer/ZipRawData.sh (moved) moved zipscript to transfer directory Fri, 09 Mar 2012 21:11:09 GMT Daniela Dorner [13037] * trunk/DataCheck/Monitoring (added) added (directory for monitoring scripts) Fri, 09 Mar 2012 21:08:33 GMT Daniela Dorner [13036] * trunk/DataCheck/Sourcefile.sh (modified) * trunk/DataCheck/crontab.coma (deleted) * trunk/DataCheck/crontab.fact-transfer (deleted) * trunk/DataCheck/crontab.fact01 (deleted) * trunk/DataCheck/crontab.fact02 (deleted) * trunk/DataCheck/crontab.fact04 (deleted) * trunk/DataCheck/crontab.fact1 (deleted) * trunk/DataCheck/crontab.fact2 (deleted) moved crontab files to cron directory Fri, 09 Mar 2012 21:07:19 GMT Daniela Dorner [13035] * trunk/DataCheck/Cron/crontab.coma (copied) * trunk/DataCheck/Cron/crontab.fact-transfer (copied) * trunk/DataCheck/Cron/crontab.fact01 (copied) * trunk/DataCheck/Cron/crontab.fact02 (copied) * trunk/DataCheck/Cron/crontab.fact04 (copied) * trunk/DataCheck/Cron/crontab.fact1 (copied) * trunk/DataCheck/Cron/crontab.fact2 (copied) moved crontab files to cron directory Fri, 09 Mar 2012 21:02:56 GMT Daniela Dorner [13034] * trunk/DataCheck/Processing (added) added (directory for processing scripts) Fri, 09 Mar 2012 20:53:36 GMT Daniela Dorner [13033] * trunk/DataCheck/Cron (added) added (directory for crontab files) Fri, 09 Mar 2012 20:50:22 GMT Daniela Dorner [13032] * trunk/DataCheck/Transfer (added) added (directory for transfer scripts) Thu, 01 Mar 2012 22:39:00 GMT Daniela Dorner [12988] * trunk/DataCheck/CheckRawData.sh (modified) fixed typo in mysql query Thu, 01 Mar 2012 15:02:57 GMT Daniela Dorner [12985] * trunk/DataCheck/CheckRawData.sh (modified) removed check for pw-file, adapted check for modification time, added ... Thu, 01 Mar 2012 15:01:43 GMT Daniela Dorner [12984] * trunk/DataCheck/FillAuxData.sh (modified) removed check for pw-file Thu, 01 Mar 2012 11:41:21 GMT Daniela Dorner [12980] * trunk/DataCheck/CheckRawData.sh (modified) * trunk/DataCheck/FillAuxData.sh (modified) * trunk/DataCheck/RsyncAux.sh (modified) * trunk/DataCheck/RsyncRaw.sh (modified) * trunk/DataCheck/ZipRawData.sh (modified) unified method to get dates to be processed Thu, 01 Mar 2012 11:32:40 GMT Daniela Dorner [12979] * trunk/DataCheck/FillAuxData.sh (modified) * trunk/DataCheck/RsyncRaw.sh (modified) small changes in logging Thu, 01 Mar 2012 11:27:46 GMT Daniela Dorner [12978] * trunk/DataCheck/CheckAuxData.sh (deleted) not needed as job is done by FillAuxData.sh Wed, 29 Feb 2012 13:16:26 GMT Daniela Dorner [12972] * trunk/DataCheck/CheckRawData.sh (modified) introduced usage of global logfile and function sendquery Wed, 29 Feb 2012 13:15:36 GMT Daniela Dorner [12971] * trunk/DataCheck/RsyncAux.sh (modified) * trunk/DataCheck/RsyncRaw.sh (modified) introduced usage of global logfile Wed, 29 Feb 2012 12:39:37 GMT Daniela Dorner [12970] * trunk/DataCheck/ZipRawData.sh (modified) introduced usage of global logfile Wed, 29 Feb 2012 12:30:32 GMT Daniela Dorner [12969] * trunk/DataCheck/FillAuxData.sh (modified) introduced logfile to get trace of fitsfile errors Wed, 29 Feb 2012 11:08:30 GMT Daniela Dorner [12968] * trunk/DataCheck/crontab.fact-transfer (modified) changed MAILTO Thu, 23 Feb 2012 16:55:37 GMT Daniela Dorner [12936] * trunk/DataCheck/Sourcefile.sh (modified) bugfix for ftcopy in cron; adapted paths Thu, 23 Feb 2012 16:54:46 GMT Daniela Dorner [12935] * trunk/DataCheck/FillAuxData.sh (modified) implemented function for getting statistics; usage of sendquery Thu, 23 Feb 2012 16:53:22 GMT Daniela Dorner [12934] * trunk/DataCheck/crontab.fact01 (modified) * trunk/DataCheck/crontab.fact02 (modified) * trunk/DataCheck/crontab.fact04 (modified) * trunk/DataCheck/crontab.fact1 (modified) * trunk/DataCheck/crontab.fact2 (modified) adapted path, changed mailto Thu, 23 Feb 2012 16:49:03 GMT Daniela Dorner [12933] * trunk/DataCheck/CheckRawData.sh (modified) added new variables and further checks Thu, 23 Feb 2012 16:42:14 GMT Daniela Dorner [12932] * trunk/DataCheck/CheckDU.sh (modified) changed limits Thu, 09 Feb 2012 13:37:44 GMT Daniela Dorner [12882] * trunk/DataCheck/crontab.fact1 (modified) adapted times for some scripts Wed, 08 Feb 2012 20:44:49 GMT Daniela Dorner [12876] * trunk/DataCheck/RsyncAux.sh (modified) * trunk/DataCheck/RsyncRaw.sh (modified) changed time range for bwlimit Wed, 08 Feb 2012 13:12:49 GMT lyard [12873] * trunk/DataCheck/Archive/README (modified) improved readme Wed, 08 Feb 2012 12:33:48 GMT lyard [12871] * trunk/DataCheck/Archive/ENDfixer.cpp (added) * trunk/DataCheck/Archive/MjDtoISO.cpp (added) * trunk/DataCheck/Archive/README (added) * trunk/DataCheck/Archive/RowChecker.cpp (added) * trunk/DataCheck/Archive/associateRunsAndCalibs.sh (added) * trunk/DataCheck/Archive/auxIngest.sh (modified) * trunk/DataCheck/Archive/checkForHeaderUpdatesAux.sh (added) * trunk/DataCheck/Archive/checkForHeaderUpdatesRaw.sh (added) * trunk/DataCheck/Archive/checkRawFileIntegrityAndContent.sh (added) * trunk/DataCheck/Archive/correctFileName.cpp (added) * trunk/DataCheck/Archive/fitsCompare.cpp (added) * trunk/DataCheck/Archive/fixRawKeyWords.sh (modified) * trunk/DataCheck/Archive/rawIngest.sh (modified) * trunk/DataCheck/Archive/repairRawFile.sh (modified) added more scripts Tue, 31 Jan 2012 11:27:05 GMT Daniela Dorner [12798] * trunk/DataCheck/Sourcefile.sh (added) added (file with common definitions etc for FillAuxData.sh and ... Mon, 30 Jan 2012 19:57:07 GMT Daniela Dorner [12789] * trunk/DataCheck/FillAuxData.sh (modified) included further checks and fill more information; common logfile Mon, 30 Jan 2012 19:56:29 GMT Daniela Dorner [12788] * trunk/DataCheck/CheckRawData.sh (modified) included further checks and fill more information; common logfile; ... Tue, 24 Jan 2012 15:42:25 GMT Daniela Dorner [12762] * trunk/DataCheck/RsyncAux.sh (modified) * trunk/DataCheck/RsyncRaw.sh (modified) adapted rsync options for running ftchecksum on data, but still commented Tue, 24 Jan 2012 15:39:34 GMT Daniela Dorner [12761] * trunk/DataCheck/CheckRawData.sh (modified) path to FACT++ as variable Tue, 24 Jan 2012 15:04:43 GMT Daniela Dorner [12760] * trunk/DataCheck/ZipRawData.sh (modified) implemented ftchecksum but left commented; implemented variable for ... Tue, 17 Jan 2012 11:45:35 GMT lyard [12750] * trunk/DataCheck/Archive (added) * trunk/DataCheck/Archive/ENDfixer (added) * trunk/DataCheck/Archive/ENDfixer.cxx (added) * trunk/DataCheck/Archive/RowChecker (added) * trunk/DataCheck/Archive/RowChecker.cxx (added) * trunk/DataCheck/Archive/auxIngest.sh (added) * trunk/DataCheck/Archive/correctFileName (added) * trunk/DataCheck/Archive/correctFileName.cxx (added) * trunk/DataCheck/Archive/fixAuxKeyWords.sh (added) * trunk/DataCheck/Archive/fixRawKeyWords.sh (added) * trunk/DataCheck/Archive/ingest.sh (added) * trunk/DataCheck/Archive/rawIngest.sh (added) * trunk/DataCheck/Archive/repairAuxFile.sh (added) * trunk/DataCheck/Archive/repairRawFile.sh (added) * trunk/DataCheck/Archive/testNaming.sh (added) added ingest stuff