source: fact/tools/pyscripts/doc/getting_started.rst@ 14098

===============
Getting Started
===============

1) Get an account at the FACT data center at ISDC
2) Setup your environment variables
3) Checkout the pyfact software repository
4) Try a few examples

1. Important links
=================================

**FACT data center at ISDC** <http://www.isdc.unige.ch/fact/datacenter>**
    here you find instructions how to get an account, how to set it up and how to use it

**SVN repository** <https://fact.isdc.unige.ch/svn/fact/>
    hosts all FACT software (including pyfact)

**FACT run database** <https://www.fact-project.org/run_db/db/fact_runinfo.php>
    here you find information about all available runs

**eLogbook** <https://www.fact-project.org/logbook/>
  telescope logbook used since Dec. 1, 2012; for logbook information before this date have a look here: <http://fact.ethz.ch/FACTelog/index.jsp>

**FACT La Palma pages** <https://www.fact-project.org/>
  plenty of information concerning the telescope in La Palma

2. Environment variables
========================
set the following env variables depending on the SHELL you are using, for instance in .bashrc or .tcshrc


* **ROOTSYS**: /swdev_nfs/root_v5.28.00
* **PATH**: add $ROOTSYS/bin
* **LD_LIBRARY_PATH**: add $ROOTSYS/lib
* **PYTHONPATH**: $ROOTSYS/lib
* **PYTHONPATH**: add all directories where python should search for modules. At least:
    * pyscripts/pyfact
* **LD_LIBRARY_PATH**: add the path to pyfits_h.so
    * pyscripts/pyfact in many cases

the absolute path depends on where you have checked (or will check) out the pyscripts repository

example .bashrc::

    # this is just a convenient shortcut
    export MY_PYFACT=/home_nfs/isdc/neise/py/pyscripts/pyfact

    # this is needed for ROOT only
    export ROOTSYS=/swdev_nfs/root_v5.28.00
    export PATH=$ROOTSYS/bin:$PATH
    export LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH

    # this is needed for Python to find the pyfact classes and PyROOT
    export PYTHONPATH=$MY_PYFACT
    export PYTHONPATH=$ROOTSYS/lib:$PYTHONPATH

    #this is one of many possible ways, for ROOT to find pyfits_h.so
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MY_PYFACT


3. Check out the repository
===========================
svn co https://fact.isdc.unige.ch/svn/fact/tools/pyscripts py


4a. Create the needed *.so libs @ ISDC
======================================

FACT data are stored in (gzipped) fits files, but the data files are too large to be read by the existing tools: 

`pyfits <http://www.stsci.edu/institute/software_hardware/pyfits>` which uses `cfitsio <http://heasarc.gsfc.nasa.gov/fitsio/>`_.

To mitigate this problem a C++ class defined in fits.h [#f1]_ is used. A simple possibility to create an interface of this C++ class is to use ROOT and later to import it using the
`pyroot <http://root.cern.ch/drupal/category/package-context/pyroot>`_ module.

example::
  neise@isdc-viewer00:~/py/pyfact$ root -l -q makelibs.C 
  root [0] 
  Processing makelibs.C...
  Info in <TUnixSystem::ACLiC>: creating shared library /home_nfs/isdc/neise/test_py/pyfact/./izstream_h.so
  Info in <TUnixSystem::ACLiC>: creating shared library /home_nfs/isdc/neise/test_py/pyfact/./factfits_h.so
  [... quite some warnings ...]
  
  neise@isdc-viewer00:~/py/pyfact$ ls *.so
  calfactfits_h.so  factfits_h.so  izstream_h.so

4b. Create pyfits_h.so library 
=============================

-----------This is a depricated section---------------

FACT data are stored in (gzipped) fits files, but the data files are too large to be read by the existing tools: 

`pyfits <http://www.stsci.edu/institute/software_hardware/pyfits>` which uses `cfitsio <http://heasarc.gsfc.nasa.gov/fitsio/>`_.

To mitigate this problem a C++ class defined in fits.h [#f1]_ is used. A simple possibility to create an interface of this C++ class is to use ROOT and later to import it using the
`pyroot <http://root.cern.ch/drupal/category/package-context/pyroot>`_ module.

example::

  neise@isdc-viewer00:~/py/pyscripts/pyfact$ root
  ROOT 5.28/00 (trunk@37585, Dec 14 2010, 15:20:27 on linuxx8664gcc)
  CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010
  Type ? for help. Commands must be C++ statements.
  Enclose multiple statements between { }.
  root [0] .L pyfits.h++
  Info in <TUnixSystem::ACLiC>: creating shared library /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits_h.so
  In file included from /home_nfs/isdc/neise/py/pyscripts/pyfact/pyfits_h_ACLiC_dict.h:34,
                   from /home_nfs/isdc/neise/py/pyscripts/pyfact/pyfits_h_ACLiC_dict.cxx:17:
  /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h: In member function »size_t std::fits::Table::GetN(const std::string&) const«:
  /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h:442: Warnung: Deklaration von »it« überdeckt einen vorhergehenden lokalen Bezeichner
  /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h:437: Warnung: Verdeckte Deklaration ist hier
  root [1] .q

check if it worked::

  neise@isdc-viewer00:~/py/pyscripts/pyfact$ ls pyfits*
  pyfits.h  pyfits_h.d  pyfits_h.so


This might not work for whatenver reason then you can try this::

  $rootcint -f my_dict.C -c pyfits.h izstream.h
  $g++ -fPIC -c -I$ROOTSYS/include my_dict.C -o my_dict.o
  $g++ -o pyfits_h.so -shared my_dict.o


In case one uses root 5.32 instead of root 5.28, one should do::

    neise@isdc-viewer00:~/py/pyscripts/pyfact$ root -b
      *******************************************
      *                                         *
      *        W E L C O M E  to  R O O T       *
      *                                         *
      *   Version   5.32/00   2 December 2011   *
      *                                         *
      *  You are welcome to visit our Web site  *
      *          http://root.cern.ch            *
      *                                         *
      *******************************************

    ROOT 5.32/00 (tags/v5-32-00@42375, Dec 02 2011, 12:42:25 on linuxx8664gcc)

    CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010
    Type ? for help. Commands must be C++ statements.
    Enclose multiple statements between { }.
    root [0] gSystem->Load("/usr/lib64/libz.so")
    (int)0
    root [1] .L pyfits.h++
    Info in <TUnixSystem::ACLiC>: creating shared library /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits_h.so
    In file included from /home_nfs/isdc/neise/py/pyscripts/pyfact/pyfits_h_ACLiC_dict.h:34,
                     from /home_nfs/isdc/neise/py/pyscripts/pyfact/pyfits_h_ACLiC_dict.cxx:17:
    /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h: In member function »size_t std::fits::Table::GetN(const std::string&) const«:
    /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h:442: Warnung: Deklaration von »it« überdeckt einen vorhergehenden lokalen Bezeichner
    /home_nfs/isdc/neise/py/pyscripts/pyfact/./pyfits.h:437: Warnung: Verdeckte Deklaration ist hier
    root [2] .q
    neise@isdc-viewer00:~/py/pyscripts/pyfact$ ls pyfits*
    pyfits.h  pyfits_h.d  pyfits_h.so

Now you can check if the reading rawdata is working::

  neise@isdc-viewer00:~/py/pyfact$ pyfact.py 
  showing test of iterator of RawData class
  in order to test the SlowData classe please use: ./pyfact.py fits-file-name
  ev  1 data[0,0] =  -7.68806196438 start_cell[0] =  426 trigger type =  416
  ev  2 data[0,0] =  0.792280672486 start_cell[0] =  437 trigger type =  416
  ev  3 data[0,0] =  54.9490855517 start_cell[0] =  449 trigger type =  416
  ev  4 data[0,0] =  3.9482592671 start_cell[0] =  461 trigger type =  416
  ev  5 data[0,0] =  -66.4329317281 start_cell[0] =  474 trigger type =  416
  ev  6 data[0,0] =  8.2917625181 start_cell[0] =  485 trigger type =  416
  ev  7 data[0,0] =  -41.6565116079 start_cell[0] =  498 trigger type =  416
  ev  8 data[0,0] =  33.4668984163 start_cell[0] =  509 trigger type =  416
  ev  9 data[0,0] =  -28.520090549 start_cell[0] =  522 trigger type =  416
  ev  10 data[0,0] =  -20.4925692795 start_cell[0] =  533 trigger type =  416


5. Run examples
===============

*datafilepath* and *calibfilepath* should be adjusted in case you are not working
on the ISDC cluster. 

explore class RawData::

  import pyfact
  datafilepath = '/data00/fact-construction/raw/2012/03/04/20120304_018.fits.gz'
  calibfilepath = '/data00/fact-construction/raw/2012/03/04/20120304_012.drs.fits.gz'
  run = pyfact.RawData( datafilepath, calibfilepath, return_dict = True)
  event = run.next()
  
  type(event)
  
  print 70*'*'
  for key in event:
    print 'key  :', key
    print 'value:', event[key]
    print 70*'*'

loop over RawData::

  import pyfact
  datafilepath = '/data00/fact-construction/raw/2012/03/04/20120304_018.fits.gz'
  calibfilepath = '/data00/fact-construction/raw/2012/03/04/20120304_012.drs.fits.gz'
  run = pyfact.RawData( datafilepath, calibfilepath, return_dict = True)
  
  for event in run:
    print 'event_id:', event['event_id'] 
    # the data can be found in event['acal_data']

.. rubric:: Footnotes
.. [#f1] fits.h (here named pyfits.h) is part of MARS and was written by T.Bretz, it was added to this repository for convienience