| Line | |
|---|
| 1 | This is the FACT construction data ingest and verification scripts README
|
|---|
| 2 |
|
|---|
| 3 | In order to ingest data, use either auxIngest.sh or rawIngest.sh appropriately.
|
|---|
| 4 | Raw Ingest:
|
|---|
| 5 | - if file is corrupted, a tentative repair is done
|
|---|
| 6 | - add missing END tag
|
|---|
| 7 | - reduce file size so that it re-become compliant to fits
|
|---|
| 8 | - Keywords are checked and added or updated if required.
|
|---|
| 9 | - checksum is added to the file
|
|---|
| 10 | - if file has been repaired successfully, then it is moved to the archive
|
|---|
| 11 |
|
|---|
| 12 | Aux Ingest:
|
|---|
| 13 | - if file is corrupted, a tentative repair is done
|
|---|
| 14 | - add missing END tag
|
|---|
| 15 | - reduce file size so that it re-become compliant to fits
|
|---|
| 16 | - if more rows seem to be present, restore extra rows while the time value seems correct (i.e. around the expected value)
|
|---|
| 17 | - If mjdref was not substracted to Time column, Time column is updated accordingly
|
|---|
| 18 | - Keywords are checked and added or updated if required.
|
|---|
| 19 | - checksum is added to the file
|
|---|
| 20 | - if file has been repaired successfully, then it is moved to the archive
|
|---|
| 21 |
|
|---|
| 22 |
|
|---|
| 23 | Checked keywords:
|
|---|
| 24 | - TSTARTI
|
|---|
| 25 | - TSTARTF
|
|---|
| 26 | - TSTOPI
|
|---|
| 27 | - TSTOPF
|
|---|
| 28 | - DATE-OBS
|
|---|
| 29 | - DATE-END
|
|---|
| 30 | - TELESCOP
|
|---|
| 31 | - PACKAGE
|
|---|
| 32 | - ORIGIN
|
|---|
| 33 | - TIMEUNIT
|
|---|
| 34 | - MJDREF
|
|---|
| 35 | - TIMESYS
|
|---|
| 36 |
|
|---|
| 37 | It takes source and dest folders as argument, the last arg being the string to append to log filenames.
|
|---|
| 38 |
|
|---|
| 39 | You will need the programs which source is included in this folder: please compile them appropriately
|
|---|
| 40 |
|
|---|
| 41 | The ingested data headers can be checked with checkForHeaderUpdatesAux/Raw.sh
|
|---|
| 42 |
|
|---|
| 43 | while the actual data can be compared with checkRawFioleIntegrityAndContent.sh
|
|---|
| 44 |
|
|---|
| 45 | This script will not work on aux data because the time column may have been updated. hence different files.
|
|---|
Note:
See
TracBrowser
for help on using the repository browser.
