#
# ChangeLog for trunk/DataCheck/Transfer
#
# Generated by Trac 1.5.2
# 12/22/24 09:47:20

Thu, 04 Apr 2019 17:12:05 GMT Daniela Dorner [19487]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	print only warning if the missing file is older than a few days


Sun, 21 May 2017 09:42:02 GMT Daniela Dorner [18875]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)

	adapted to use newdata instead of data for transfer


Sun, 21 May 2017 09:41:23 GMT Daniela Dorner [18874]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	added additional check for QLA results


Sun, 21 May 2017 09:40:55 GMT Daniela Dorner [18873]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	fixed small bug in logging


Sun, 21 May 2017 09:34:57 GMT Daniela Dorner [18872]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	implemented option to use /data1 instead of /scratch


Sun, 21 May 2017 09:33:52 GMT Daniela Dorner [18871]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	adapted to use proper rsync-temp-dir for each transfer


Sun, 14 May 2017 22:38:52 GMT Daniela Dorner [18867]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	adapted logging to usage of newdata


Sun, 14 May 2017 20:05:33 GMT Daniela Dorner [18863]
	* trunk/DataCheck/Transfer/CleanupNewdata.sh (added)

	added (script to clean up newdata)


Sun, 14 May 2017 20:01:03 GMT Daniela Dorner [18856]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	updated for newdata as download machine in LP


Sun, 14 May 2017 19:59:09 GMT Daniela Dorner [18854]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	updated to run on newdata and create a second copy


Sun, 14 May 2017 19:58:29 GMT Daniela Dorner [18853]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	updated to run on newdata and create a second copy


Sun, 14 May 2017 19:57:08 GMT Daniela Dorner [18852]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	updated to use newdata instead of data for transferring data


Sun, 14 May 2017 01:29:35 GMT Daniela Dorner [18851]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	adapted as discussed in telcon (implemented warning if files in ...


Sat, 08 Apr 2017 02:17:03 GMT Daniela Dorner [18807]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Thu, 02 Mar 2017 02:37:07 GMT Daniela Dorner [18778]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	fixed typo


Tue, 21 Feb 2017 21:21:56 GMT Daniela Dorner [18775]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	fixed bug and added check


Sat, 04 Feb 2017 19:51:45 GMT Daniela Dorner [18747]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	removed debugging output


Sat, 04 Feb 2017 16:41:48 GMT Daniela Dorner [18746]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	implemented speedup (rsync from/to ISDC took 1.5-2h each)


Sat, 04 Feb 2017 16:27:30 GMT Daniela Dorner [18744]
	* trunk/DataCheck/Transfer/BackupDatabase.sh (modified)

	fix in path


Wed, 25 Jan 2017 23:43:24 GMT Daniela Dorner [18739]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Sun, 08 Jan 2017 17:21:53 GMT Daniela Dorner [18723]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	files in the fails folder are considered now "fine" in the context of ...


Thu, 05 Jan 2017 00:56:47 GMT Daniela Dorner [18709]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	added backup of QLA to Wue


Wed, 21 Dec 2016 12:37:49 GMT Daniela Dorner [18701]
	* trunk/DataCheck/DataCheck/InfoForMWL.sh (modified)
	* trunk/DataCheck/QuickLook/SendData.sh (modified)
	* trunk/DataCheck/QuickLook/SendData_Internal.sh (modified)
	* trunk/DataCheck/Sourcefile.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	fixed bug (variable pwfile occured twice -> renamed)


Wed, 21 Dec 2016 12:36:32 GMT Daniela Dorner [18700]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Sun, 18 Dec 2016 20:54:38 GMT Daniela Dorner [18686]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	improved logging


Sun, 18 Dec 2016 20:50:56 GMT Daniela Dorner [18685]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Wed, 02 Nov 2016 03:33:54 GMT Daniela Dorner [18671]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	changed back to not using the rsync server which was gone after the ...


Wed, 02 Nov 2016 03:33:32 GMT Daniela Dorner [18670]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	changed back to not using the rsync server which was gone after the ...


Wed, 02 Nov 2016 03:32:52 GMT Daniela Dorner [18669]
	* trunk/DataCheck/Transfer/BackupDatabase.sh (modified)

	fixed syntax of mysqldump for new mysqlversion


Sat, 10 Sep 2016 17:31:04 GMT Daniela Dorner [18582]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved output; fixed regular expression for finding rawfiles


Sat, 13 Feb 2016 17:47:02 GMT Daniela Dorner [18435]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	added output for case of certaindate


Sat, 13 Feb 2016 17:46:02 GMT Daniela Dorner [18434]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging, additional check for DB (jobs that were reset while ...


Sat, 13 Feb 2016 17:42:50 GMT Daniela Dorner [18433]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	implemented usage of certaindate


Sat, 13 Feb 2016 17:41:36 GMT Daniela Dorner [18432]
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)

	implemented usage of variable certaindate


Sat, 02 Jan 2016 06:30:05 GMT Daniela Dorner [18397]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)

	removed wrong comment


Mon, 31 Aug 2015 00:53:15 GMT Daniela Dorner [18328]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	commented-out what is needed if daq is unavailable


Mon, 31 Aug 2015 00:23:29 GMT Daniela Dorner [18319]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	removed check of certaindate-variable (now done in getdate())


Mon, 31 Aug 2015 00:22:44 GMT Daniela Dorner [18318]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	added commented-out the paths in case daq is not available


Wed, 18 Feb 2015 09:28:14 GMT Daniela Dorner [18137]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)

	added usage of function getdata()


Wed, 18 Feb 2015 09:27:30 GMT Daniela Dorner [18136]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	ignore runs with a run number with more than 3 digits


Tue, 27 May 2014 21:15:04 GMT Daniela Dorner [17905]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	included and/or improved check of variable certaindate


Mon, 26 May 2014 15:10:34 GMT Daniela Dorner [17890]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	do not use function checklock anymore as script is only started when ...


Mon, 26 May 2014 15:09:30 GMT Daniela Dorner [17889]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Mon, 17 Mar 2014 13:18:25 GMT Daniela Dorner [17619]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	improved logging in case of full disk


Mon, 17 Mar 2014 12:40:04 GMT Daniela Dorner [17618]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging, re-enabled check if script is already running


Mon, 17 Mar 2014 12:38:48 GMT Daniela Dorner [17617]
	* trunk/DataCheck/Monitoring/CheckDU.sh (modified)
	* trunk/DataCheck/QuickLook/Step1.sh (modified)
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	improved logging in case of full disk


Sun, 16 Mar 2014 23:47:30 GMT Daniela Dorner [17616]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	added DISK tag to logging, improved logging, added certaindate and ...


Sun, 16 Mar 2014 23:42:56 GMT Daniela Dorner [17612]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	added check for full disk - at 100GB transfer is stopped


Sun, 02 Mar 2014 01:20:11 GMT Daniela Dorner [17594]
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)
	* trunk/DataCheck/Transfer/BackupDatabase.sh (modified)
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)
	* trunk/DataCheck/Transfer/RsyncDBToWue.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	changed logging for connection problems


Sun, 02 Mar 2014 00:35:31 GMT Daniela Dorner [17590]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	added copying of already compressed files


Sat, 01 Mar 2014 23:59:27 GMT Daniela Dorner [17576]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	added feature to run script manually for a certain night, added check ...


Sat, 01 Mar 2014 23:57:08 GMT Daniela Dorner [17575]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)

	updated path to qla at isdc


Sat, 01 Mar 2014 23:56:07 GMT Daniela Dorner [17574]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	added rsync of drs-file to avoid failing callistos


Sat, 01 Mar 2014 23:46:31 GMT Daniela Dorner [17573]
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)

	implemented usage of getdates function


Sat, 01 Mar 2014 23:41:53 GMT Daniela Dorner [17570]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	updated output with qlainfo, implemented fz files, following links in ...


Mon, 09 Sep 2013 14:55:29 GMT Daniela Dorner [17157]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	changed number of processes for pigz to be set from the general setup ...


Mon, 09 Sep 2013 13:24:03 GMT Daniela Dorner [17149]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	comments what changes needed to use rsync server


Fri, 06 Sep 2013 17:08:07 GMT Daniela Dorner [17114]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	implemented usage of getdates function


Fri, 06 Sep 2013 16:04:19 GMT Daniela Dorner [17103]
	* trunk/DataCheck/Transfer/BackupQLA.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	adapted for usage of rsync servers


Thu, 05 Sep 2013 13:25:27 GMT Daniela Dorner [17079]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	added check if QLA finished callisto for all data runs


Wed, 31 Jul 2013 16:31:10 GMT Daniela Dorner [16992]
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	added rsync-temp-dir to avoid temporary files from the rsync on the ...


Mon, 03 Jun 2013 21:42:07 GMT Daniela Dorner [16679]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	small fix in usage of lsof


Mon, 03 Jun 2013 21:40:40 GMT Daniela Dorner [16678]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	reworked output, added summary


Mon, 13 May 2013 09:57:16 GMT Daniela Dorner [15599]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	check files only in detail when number of files does agree


Mon, 13 May 2013 09:50:59 GMT Daniela Dorner [15596]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	allow more than one instance of script and moved control of number of ...


Mon, 13 May 2013 09:47:58 GMT Daniela Dorner [15595]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	changed order of dates for processing to make sure that older data ...


Mon, 13 May 2013 09:47:07 GMT Daniela Dorner [15594]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	changed limit to 5 MB/s during night


Wed, 08 May 2013 17:16:17 GMT Daniela Dorner [15559]
	* trunk/DataCheck/Transfer/BackupQLA.sh (added)

	added (script to backup results of QLA)


Tue, 07 May 2013 22:22:58 GMT Daniela Dorner [15517]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	added pv to control rate of zipping, limited pigz to 2 processes


Thu, 25 Apr 2013 12:12:54 GMT Daniela Dorner [15435]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)

	fixed bug which was caused by the introduction of the variable ...


Sat, 13 Apr 2013 23:26:27 GMT Daniela Dorner [15328]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	changed interval for crashed jobs to 5 hours


Mon, 04 Feb 2013 06:03:36 GMT Daniela Dorner [14841]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)

	adapted to new structure in LP (newdaq, new transfer scheme)


Mon, 04 Feb 2013 06:02:39 GMT Daniela Dorner [14840]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	adapted to new structure in LP (newdaq, new transfer scheme)


Mon, 04 Feb 2013 05:16:50 GMT Daniela Dorner [14824]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	updated to changes in LP (newdaq), implemented PhiDo (still ...


Mon, 17 Dec 2012 21:54:26 GMT Daniela Dorner [14757]
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	inverted order of dates (change from months ago)


Mon, 17 Dec 2012 20:23:38 GMT Daniela Dorner [14756]
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)

	inverted order of dates (change from months ago)


Tue, 19 Jun 2012 17:25:45 GMT Daniela Dorner [14192]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	bugfix for case file size is the same but file is in fails directory ...


Thu, 24 May 2012 12:13:34 GMT Daniela Dorner [13864]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging


Thu, 24 May 2012 12:09:22 GMT Daniela Dorner [13863]
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	reduced time span for transfer processes again


Thu, 24 May 2012 11:25:00 GMT Daniela Dorner [13862]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved logging, removed old code


Tue, 08 May 2012 12:41:49 GMT Daniela Dorner [13600]
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)

	changed time range for rsync of raw files on LP from 3 to 9 days


Tue, 08 May 2012 12:40:10 GMT Daniela Dorner [13599]
	* trunk/DataCheck/Processing/FillAuxData.sh (modified)
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)

	changed time range for rsync of aux files from 3 to 9 days


Mon, 07 May 2012 09:42:36 GMT Daniela Dorner [13580]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	run skript only for last three days or last X GB of data


Mon, 16 Apr 2012 11:47:39 GMT Daniela Dorner [13342]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	improved logging


Wed, 04 Apr 2012 16:12:21 GMT Daniela Dorner [13313]
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)

	improved logging


Wed, 04 Apr 2012 15:28:17 GMT Daniela Dorner [13312]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	adapted parameters to run script via cron


Wed, 04 Apr 2012 11:44:47 GMT Daniela Dorner [13305]
	* trunk/DataCheck/Transfer/BackupDatabase.sh (added)

	added (script to backup databases)


Thu, 22 Mar 2012 16:55:48 GMT Daniela Dorner [13171]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (modified)

	improved output, fixed bugs


Mon, 19 Mar 2012 17:20:12 GMT Daniela Dorner [13154]
	* trunk/DataCheck/Transfer/CheckTransfer.sh (added)

	added (script to check status of transfer for deleting data)


Mon, 19 Mar 2012 17:09:05 GMT Daniela Dorner [13152]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	bugfix for drsfiles


Wed, 14 Mar 2012 19:26:48 GMT Daniela Dorner [13108]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)

	added check if directory exists


Wed, 14 Mar 2012 19:25:37 GMT Daniela Dorner [13107]
	* trunk/DataCheck/Setup/setup.lp.fact (modified)
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)

	added tmp path for rsyn


Wed, 14 Mar 2012 09:03:19 GMT Daniela Dorner [13093]
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	added drs file to transfer


Tue, 13 Mar 2012 14:49:22 GMT Daniela Dorner [13082]
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	added drs-files to rsync and check for existance of links


Sun, 11 Mar 2012 17:10:36 GMT Daniela Dorner [13065]
	* trunk/DataCheck/Setup/setup.isdc.fact (modified)
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)

	added temp directory for rsync


Fri, 09 Mar 2012 23:32:07 GMT Daniela Dorner [13058]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (modified)




Fri, 09 Mar 2012 22:27:53 GMT Daniela Dorner [13055]
	* trunk/DataCheck/Processing/CheckAuxFilesAvail.sh (modified)
	* trunk/DataCheck/Processing/CheckRawData.sh (modified)
	* trunk/DataCheck/Processing/CheckRawFilesAvail.sh (modified)
	* trunk/DataCheck/Processing/FillAuxData.sh (modified)
	* trunk/DataCheck/Setup/setup.lp.fact (modified)
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)
	* trunk/DataCheck/Transfer/RsyncAuxLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawLP.sh (modified)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (modified)
	* trunk/DataCheck/Transfer/ZipRawData.sh (modified)

	adapted scripts to changes in directory structure


Fri, 09 Mar 2012 22:18:35 GMT Daniela Dorner [13054]
	* trunk/DataCheck/Transfer/BackupAuxToWue.sh (modified)
	* trunk/DataCheck/Transfer/BackupRawToWue.sh (modified)

	reworked version of scripts doing backup to Wuerzburg


Fri, 09 Mar 2012 22:15:38 GMT Daniela Dorner [13053]
	* trunk/DataCheck/Transfer/RsyncAuxToISDC.sh (added)
	* trunk/DataCheck/Transfer/RsyncRawToISDC.sh (added)

	added (scripts for transfer from LP to ISDC)


Fri, 09 Mar 2012 22:04:53 GMT Daniela Dorner [13047]
	* trunk/DataCheck/Transfer/RsyncDBToWue.sh (moved)

	moved db backup script to transfer directory