source: trunk/DataCheck/Transfer

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @19549   5 years Daniela Dorner add grb-run
(edit) @19487   6 years Daniela Dorner print only warning if the missing file is older than a few days
(edit) @18875   8 years Daniela Dorner adapted to use newdata instead of data for transfer
(edit) @18874   8 years Daniela Dorner added additional check for QLA results
(edit) @18873   8 years Daniela Dorner fixed small bug in logging
(edit) @18872   8 years Daniela Dorner implemented option to use /data1 instead of /scratch
(edit) @18871   8 years Daniela Dorner adapted to use proper rsync-temp-dir for each transfer
(edit) @18867   8 years Daniela Dorner adapted logging to usage of newdata
(edit) @18863   8 years Daniela Dorner added (script to clean up newdata)
(edit) @18856   8 years Daniela Dorner updated for newdata as download machine in LP
(edit) @18854   8 years Daniela Dorner updated to run on newdata and create a second copy
(edit) @18853   8 years Daniela Dorner updated to run on newdata and create a second copy
(edit) @18852   8 years Daniela Dorner updated to use newdata instead of data for transferring data
(edit) @18851   8 years Daniela Dorner adapted as discussed in telcon (implemented warning if files in ...
(edit) @18807   8 years Daniela Dorner improved logging
(edit) @18778   8 years Daniela Dorner fixed typo
(edit) @18775   8 years Daniela Dorner fixed bug and added check
(edit) @18747   8 years Daniela Dorner removed debugging output
(edit) @18746   8 years Daniela Dorner implemented speedup (rsync from/to ISDC took 1.5-2h each)
(edit) @18744   8 years Daniela Dorner fix in path
(edit) @18739   8 years Daniela Dorner improved logging
(edit) @18723   8 years Daniela Dorner files in the fails folder are considered now "fine" in the context of ...
(edit) @18709   8 years Daniela Dorner added backup of QLA to Wue
(edit) @18701   8 years Daniela Dorner fixed bug (variable pwfile occured twice -> renamed)
(edit) @18700   8 years Daniela Dorner improved logging
(edit) @18686   8 years Daniela Dorner improved logging
(edit) @18685   8 years Daniela Dorner improved logging
(edit) @18671   8 years Daniela Dorner changed back to not using the rsync server which was gone after the ...
(edit) @18670   8 years Daniela Dorner changed back to not using the rsync server which was gone after the ...
(edit) @18669   8 years Daniela Dorner fixed syntax of mysqldump for new mysqlversion
(edit) @18582   8 years Daniela Dorner improved output; fixed regular expression for finding rawfiles
(edit) @18435   9 years Daniela Dorner added output for case of certaindate
(edit) @18434   9 years Daniela Dorner improved logging, additional check for DB (jobs that were reset while ...
(edit) @18433   9 years Daniela Dorner implemented usage of certaindate
(edit) @18432   9 years Daniela Dorner implemented usage of variable certaindate
(edit) @18397   9 years Daniela Dorner removed wrong comment
(edit) @18328   9 years Daniela Dorner commented-out what is needed if daq is unavailable
(edit) @18319   9 years Daniela Dorner removed check of certaindate-variable (now done in getdate())
(edit) @18318   9 years Daniela Dorner added commented-out the paths in case daq is not available
(edit) @18137   10 years Daniela Dorner added usage of function getdata()
(edit) @18136   10 years Daniela Dorner ignore runs with a run number with more than 3 digits
(edit) @17905   10 years Daniela Dorner included and/or improved check of variable certaindate
(edit) @17890   10 years Daniela Dorner do not use function checklock anymore as script is only started when ...
(edit) @17889   10 years Daniela Dorner improved logging
(edit) @17619   11 years Daniela Dorner improved logging in case of full disk
(edit) @17618   11 years Daniela Dorner improved logging, re-enabled check if script is already running
(edit) @17617   11 years Daniela Dorner improved logging in case of full disk
(edit) @17616   11 years Daniela Dorner added DISK tag to logging, improved logging, added certaindate and ...
(edit) @17612   11 years Daniela Dorner added check for full disk - at 100GB transfer is stopped
(edit) @17594   11 years Daniela Dorner changed logging for connection problems
(edit) @17590   11 years Daniela Dorner added copying of already compressed files
(edit) @17576   11 years Daniela Dorner added feature to run script manually for a certain night, added check ...
(edit) @17575   11 years Daniela Dorner updated path to qla at isdc
(edit) @17574   11 years Daniela Dorner added rsync of drs-file to avoid failing callistos
(edit) @17573   11 years Daniela Dorner implemented usage of getdates function
(edit) @17570   11 years Daniela Dorner updated output with qlainfo, implemented fz files, following links in ...
(edit) @17157   11 years Daniela Dorner changed number of processes for pigz to be set from the general setup ...
(edit) @17149   11 years Daniela Dorner comments what changes needed to use rsync server
(edit) @17114   11 years Daniela Dorner implemented usage of getdates function
(edit) @17103   11 years Daniela Dorner adapted for usage of rsync servers
(edit) @17079   11 years Daniela Dorner added check if QLA finished callisto for all data runs
(edit) @16992   11 years Daniela Dorner added rsync-temp-dir to avoid temporary files from the rsync on the ...
(edit) @16679   11 years Daniela Dorner small fix in usage of lsof
(edit) @16678   11 years Daniela Dorner reworked output, added summary
(edit) @15599   12 years Daniela Dorner check files only in detail when number of files does agree
(edit) @15596   12 years Daniela Dorner allow more than one instance of script and moved control of number of ...
(edit) @15595   12 years Daniela Dorner changed order of dates for processing to make sure that older data are ...
(edit) @15594   12 years Daniela Dorner changed limit to 5 MB/s during night
(edit) @15559   12 years Daniela Dorner added (script to backup results of QLA)
(edit) @15517   12 years Daniela Dorner added pv to control rate of zipping, limited pigz to 2 processes
(edit) @15435   12 years Daniela Dorner fixed bug which was caused by the introduction of the variable ...
(edit) @15328   12 years Daniela Dorner changed interval for crashed jobs to 5 hours
(edit) @14841   12 years Daniela Dorner adapted to new structure in LP (newdaq, new transfer scheme)
(edit) @14840   12 years Daniela Dorner adapted to new structure in LP (newdaq, new transfer scheme)
(edit) @14824   12 years Daniela Dorner updated to changes in LP (newdaq), implemented PhiDo (still ...
(edit) @14757   12 years Daniela Dorner inverted order of dates (change from months ago)
(edit) @14756   12 years Daniela Dorner inverted order of dates (change from months ago)
(edit) @14192   12 years Daniela Dorner bugfix for case file size is the same but file is in fails directory ...
(edit) @13864   12 years Daniela Dorner improved logging
(edit) @13863   12 years Daniela Dorner reduced time span for transfer processes again
(edit) @13862   12 years Daniela Dorner improved logging, removed old code
(edit) @13600   13 years Daniela Dorner changed time range for rsync of raw files on LP from 3 to 9 days
(edit) @13599   13 years Daniela Dorner changed time range for rsync of aux files from 3 to 9 days
(edit) @13580   13 years Daniela Dorner run skript only for last three days or last X GB of data
(edit) @13342   13 years Daniela Dorner improved logging
(edit) @13313   13 years Daniela Dorner improved logging
(edit) @13312   13 years Daniela Dorner adapted parameters to run script via cron
(edit) @13305   13 years Daniela Dorner added (script to backup databases)
(edit) @13171   13 years Daniela Dorner improved output, fixed bugs
(edit) @13154   13 years Daniela Dorner added (script to check status of transfer for deleting data)
(edit) @13152   13 years Daniela Dorner bugfix for drsfiles
(edit) @13108   13 years Daniela Dorner added check if directory exists
(edit) @13107   13 years Daniela Dorner added tmp path for rsyn
(edit) @13093   13 years Daniela Dorner added drs file to transfer
(edit) @13082   13 years Daniela Dorner added drs-files to rsync and check for existance of links
(edit) @13065   13 years Daniela Dorner added temp directory for rsync
(edit) @13058   13 years Daniela Dorner
(edit) @13055   13 years Daniela Dorner adapted scripts to changes in directory structure
(edit) @13054   13 years Daniela Dorner reworked version of scripts doing backup to Wuerzburg
(edit) @13053   13 years Daniela Dorner added (scripts for transfer from LP to ISDC)
Note: See TracRevisionLog for help on using the revision log.