source: trunk/MagicSoft/TDAS-Extractor/Pedestal.tex@ 6376

\section{Pedestal Extraction \label{sec:pedestals}}

\subsection{Pedestal RMS}

The background $BG$ (Pedestal) 
can be completely described by the noise-autocorrelation matrix $\boldsymbol{B}$ 
(eq.~\ref{eq:autocorr}), 
where the diagonal elements give what is usually denoted as the ``Pedestal RMS''. 
\par

By definition, the noise autocorrelation matrix $B$ and thus the ``pedestal RMS'' 
is independent from the signal extractor.

\subsection{Bias and Error}

Consider a large number of signals (FADC spectra), all with the same
integrated charge $ST$ (true signal). By applying a signal extractor
we obtain a distribution of extracted signals $SE$ (for fixed $ST$ and
fixed background fluctuations $BG$). The distribution of the quantity

\begin{equation}
X = SE-ST 
\end{equation}

has the mean $B$ and the RMS $R$ defined by:

\begin{eqnarray}
   B    &=& <X> \\
   R    &=& \sqrt{<(X-B)^2>}
\end{eqnarray}

The parameter $B$ can be called the {\textit{\bf bias}} of the pedestal extractor and $R$ 
the RMS of the distribution of $X$ which 
depend generally on the size of $ST$ and the size of the background fluctuations $BG$.

\par

For the normal image cleaning, knowledge of $B$ is sufficient and the 
error $R$ should be known in order to calculate a correct background probability.
\par
Also for the model analysis, $B$ and $R$ are needed if one wants to keep small
signals. 

\subsection{Pedestal Fluctuations as Contribution to the Signal Fluctuations}

In case of the calibration with the F-Factor methoid, 
the basic relation is:

\begin{equation}
\frac{(\Delta ST)^2}{<ST>^2} = \frac{1}{<n_{phe}>} * F^2
\end{equation}

Here $\Delta ST$ is the fluctuation of the true signal $ST$ due to the
fluctuation of the number of photo-electrons. $ST$ is obtained from the
measured fluctuations of $SE$  ($RMS_{SE}$) subtracting those contributions to the 
fluctuations of the
extracted signal which are due to the fluctuation of the pedestal\footnote{%
A way to check whether the right RMS has been subtracted is to make the
``Razmick''-plot

\begin{equation}
    \frac{(\Delta ST)^2}{<ST>^2} \quad \textit{vs.} \quad \frac{1}{<ST>}
\end{equation}

This should give a straight line passing through the origin. The slope of
the line is equal to

\begin{equation}
    c * F^2
\end{equation}

where $c$ is the photon/ADC conversion factor  $<ST>/<m_{pe}>$.}.

\begin{equation}
 (\Delta ST)^2 = RMS_{SE}^2 - R^2 
\label{eq:rmssubtraction}
\end{equation}

If $R$ does not dependent on the signal height, (as it is the case 
for the digital filter, eq.~\ref{eq:of_noise}), then one can retrieve $R$ by 
applying the signal extractor on a {\textit{\bf fixed window}} of pedestal events.

\subsection{Methods to Retrieve Bias $B$ and Errors $R$}

$R$ is in general different from the pedestal RMS. It cannot be
obtained by applying the signal extractor to pedestal events, especially 
for large signals (e.g. calibration signals).
\par
In the case of the digital filter, $R$ is in theory independent from the 
signal amplitude $ST$ and depends only on the background $BG$ (eq.~\ref{eq:of_noise}).
It can be obtained from the
fitted error of the extracted signal ($\Delta(SE)_{fitted}$), 
which one can calculate for every event or by applying the extractor to a fixed window 
of pure background events (``pedestal events'').

\par

In order to get the missing information, we did the following investigations:
\begin{enumerate}
\item Determine $R$ by applying the signal extractor to a fixed window
    of pedestal events. The background fluctuations can be simulated with different 
    levels of night sky background and the continuous light source, but no signal size 
    dependency can be retrieved with the method. 
\item Determine bias $B$ and resolution $R$ from MC events with and without added noise. 
    Assuming that $R$ and $B$ are negligible for the events without noise, one can 
    get a dependency of both values from the size of the signal. 
\item Determine $R$ from the fitted error of $SE$, which is possible for the 
    fit and the digital filter (eq.~\ref{eq:of_noise}). 
    In prinicple, all dependencies can be retrieved with this method.
\end{enumerate}

\subsubsection{ \label{sec:determiner} Application of the Signal Extractor to a Fixed Window
of Pedestal Events}

By applying the signal extractor to a fixed window of pedestal events, we 
determine the parameter $R$ for the case of no signal ($ST = 0$). In the case of 
all extractors using a fixed window from the beginning (extractors nr. \#1 to \#22 
in section~\ref{sec:algorithms}), the results are by construction the same as calculating 
the pedestal RMS.
\par
In MARS, this possibility is implemented with a function-call to: \\ 

{\textit{\bf MJPedestal::SetExtractionWithExtractorRndm()}}. \\

In the case of the {\textit{\bf amplitude extracting spline}} (extractor nr. \#23), we placed the 
spline maximum value (which determines the exact extraction window) at a random place 
within the digitizing binning resolution of one central FADC slice. 
In the case of the {\textit{\bf digital filter}} (extractor nr. \#28), the time shift was  
randomized for each event within a fixed global extraction window.

\par

The following plots~\ref{fig:sw:distped} through~\ref{fig:amp:relrms:run38996} show results 
obtained with the second method for three background intensities: 
\begin{enumerate}
\item Closed camera and no (Poissonian) fluctuation due to photons from the night sky background
\item The camera pointing to an extra-galactic region with stars in the field of view
\item The camera illuminated by a continuous light source of high intensity causing much higher pedestal 
fluctuations than in usual observation conditions.
\end{enumerate}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{figure}[htp]
\centering
\includegraphics[height=0.43\textheight]{PedestalSpectrum-18-Run38993.eps}
\vspace{\floatsep}
\includegraphics[height=0.43\textheight]{PedestalSpectrum-18-Run38995.eps}
\caption{MExtractTimeAndChargeSlidingWindow with extraction window of 4 FADC slices: 
Distribution of extracted "pedestals"  from pedestal run with 
closed camera (top) and open camera observing an extra-galactic star field (bottom) for one channel 
(pixel 100). The result obtained from a simple addition of 4 FADC 
slice contents (``fundamental'') is displayed as red histogram, the one obtained from the application of
the algorithm on 
a fixed window of 4 FADC slices as blue histogram (``extractor random'') and the one obtained from the
full algorithm allowed to slide within a global window of 12 slices. The obtained histogram means and 
RMSs have been converted to equiv. photo-electrons.}
\label{fig:sw:distped}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.43\textheight]{PedestalSpectrum-23-Run38993.eps}
\vspace{\floatsep}
\includegraphics[height=0.43\textheight]{PedestalSpectrum-23-Run38995.eps}
\caption{MExtractTimeAndChargeSpline with amplitude extraction: 
Spectrum of extracted "pedestals"  from pedestal run with 
closed camera lids (top) and open lids observing an extra-galactic star field (bottom) for one channel 
(pixel 100). The result obtained from a simple addition of 2 FADC 
slice contents (``fundamental'') is displayed as red histogram, the one obtained from the application 
of the algorithm on a fixed window of 1 FADC slice as blue histogram  (``extractor random'')
and the one obtained from the
full algorithm allowed to slide within a global window of 12 slices. The obtained histogram means and 
RMSs have been converted to equiv. photo-electrons.}
\label{fig:amp:distped}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.43\textheight]{PedestalSpectrum-25-Run38993.eps}
\vspace{\floatsep}
\includegraphics[height=0.43\textheight]{PedestalSpectrum-25-Run38995.eps}
\caption{MExtractTimeAndChargeSpline with integral extraction over 2 FADC slices: 
Distribution of extracted "pedestals"  from pedestal run with 
closed camera lids (top) and open lids observing an extra-galactic star field (bottom) for one channel 
(pixel 100). The result obtained from a simple addition of 2 FADC 
slice contents (``fundamental'') is displayed as red histogram, the one obtained from the application 
of time-randomized weigths on a fixed window of 2 FADC slices as blue histogram and the one obtained from the
full algorithm allowed to slide within a global window of 12 slices. The obtained histogram means and 
RMSs have been converted to equiv. photo-electrons.}
\label{fig:int:distped}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.43\textheight]{PedestalSpectrum-28-Run38993.eps}
\vspace{\floatsep}
\includegraphics[height=0.43\textheight]{PedestalSpectrum-28-Run38995.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Spectrum of extracted "pedestals"  from pedestal run with 
closed camera lids (top) and open lids observing an extra-galactic star field (bottom) for one channel 
(pixel 100). The result obtained from a simple addition of 6 FADC 
slice contents (``fundamental'') is displayed as red histogram, the one obtained from the application 
of time-randomized weigths on a fixed window of 6 slices as blue histogram and the one obtained from the
full algorithm allowed to slide within a global window of 12 slices. The obtained histogram means and 
RMSs have been converted to equiv. photo-electrons.}
\label{fig:df:distped}
\end{figure}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38993_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38995_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with amplitude extraction: 
Difference in mean pedestal (per FADC slice) between extraction algorithm
applied on a fixed window of 1 FADC slice (``extractor random'') and a simple addition of 
2 FADC slices (``fundamental''). On the top, a run with closed camera has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:amp:relmean}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38993_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38995_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices: 
Difference in mean pedestal (per FADC slice) between extraction algorithm
applied on a fixed window of 2 FADC slices (``extractor random'') and a simple addition of 
2 FADC slices (``fundamental''). On the top, a run with closed camera has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:int:relmean}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38993_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38995_RelMean.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeDigitalFilter: 
Difference in mean pedestal (per FADC slice) between extraction algorithm
applied on a fixed window of 6 FADC slices and time-randomized weights (``extractor random'') 
and a simple addition of 
6 FADC slices (``fundamental''). On the top, a run with closed camera has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:df:relmean}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38993_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38995_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with amplitude:  
Difference in pedestal RMS (per FADC slice) between extraction algorithm
applied on a fixed window of 1 FADC slice (``extractor random'') and a simple addition of 
2 FADC slices (``fundamental''). On the top, a run with closed camera has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:amp:relrms}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38993_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38995_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices:  
Difference in pedestal RMS (per FADC slice) between extraction algorithm
applied on a fixed window of 2 FADC slices (``extractor random'') and a simple addition of 
2 FADC slices (``fundamental''). On the top, a run with closed camera has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:amp:relrms}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38993_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38995_RMSDiff.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeDigitalFilter:  
Difference in pedestal RMS (per FADC slice) between extraction algorithm
applied on a fixed window of 6 FADC slices and time-randomized weights (``extractor random'') 
and a simple addition of 6 FADC slices (``fundamental''). On the top, a run with closed camera 
has been taken, in the center
 an opened camera observing an extra-galactic star field and on the bottom, an open camera being 
illuminated by the continuous light of the calibration (level: 100). Every entry corresponds to one 
pixel.}
\label{fig:df:relrms}
\end{figure}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Figures~\ref{fig:df:distped}
and~\ref{fig:amp:distped} show the 
extracted pedestal distributions for the digital filter with cosmics weights (extractor~\#28) and the 
spline amplitude (extractor~\#27), respectively for one examplary channel (corresponding to pixel 200). 
One can see the (asymmetric) Poisson behaviour of the 
night sky background photons for the distributions with open camera and the cutoff at the lower egde 
for the distribution with high-intensity continuous light due to a limited pedestal offset and the cutoff 
to negative fluctuations.
\par
Figures~\ref{fig:df:relmean}
and~\ref{fig:amp:relmean} show the 
relative difference between the calculated pedestal mean and 
the one obtained by applying the extractor for 
all channels of the MAGIC camera. One can see that in all cases, the distribution is centered around zero, 
while its width is never larger than 0.01 which corresponds about to the precision of the extracted mean for 
the number of used events. (A very similar distribution is obtained by comparing the results 
of the same pedestal calculator applied to different ranges of FADC slices.)
\par
Figures~\ref{fig:df:relrms}
and~\ref{fig:amp:relrms} show the 
relative difference between the calculated pedestal RMS, normalized to an equivalent number of slices 
(2.5 for the digital filter and 1. for the amplitude of the spline) and 
the one obtained by applying the extractor for all channels of the MAGIC camera. 
One can see that in all cases, the distribution is not centered around zero, but shows an offset depending 
on the light intensity. The difference can be 10\% in the case of the digital filter and even 25\% for the 
spline. This big difference for the spline is partly explained by the fact that the pedestals have to be 
calculated from an even number of slices to account for the clock-noise. However, the (normalized) pedestal 
RMS depends critically on the number of summed FADC slices, especially at very low numbers. In general, 
the higher the number of summed FADC slices, the higher the (to the square root of the number of slices) 
normalized pedestal RMS.


\subsubsection{ \label{sec:determiner} Application of the Signal Extractor to a Sliding Window
of Pedestal Events}

In this section, we apply the signal extractor to a sliding window of pedestal events. 
\par
In MARS, this possibility can be used with a call to 
{\textit{\bf MJPedestal::SetExtractionWithExtractor()}}. 
\par
Because the background is determined by the single photo-electrons from the night-sky background,
the following possibilities can occur:

\begin{enumerate}
\item There is no ``signal'' (photo-electron) in the extraction window and the extractor 
finds only electronic noise. 
Usually, the returned signal charge is then negative.
\item The extractor finds the signal from one photo-electron
\item The extractor finds an overlap of two or more photo-electrons.
\end{enumerate}

Although the probability to find a certain number of photo-electrons in a fixed window follows a 
Poisson distribution, the one for employing the sliding window is {\textit{not}} Poissonian. The extractor 
will usually find one photo-electron even if more are present in the global search window, i.e. the 
probability for two or more photo-electrons to occur in the global search window is much higher than 
the probability for these photo-electrons to overlap in time such as to be recognized as a double 
or triple photo-electron pulse by the extractor. This is especially true for small extraction windows
and for the digital filter.

\par

Given a global extraction window of size $WS$ and an average rate of photo-electrons from the night-sky 
background $R$, we will now calculate the probability for the extractor to find zero photo-electrons in the 
$WS$. The probability to find $k$ photo-electrons can be written as:

\begin{equation}
P(k) = \frac{e^{-R\cdot WS} (R \cdot WS)^k}{k!}
\end{equation}

and thus:

\begin{equation}
P(0) = e^{-R\cdot WS} 
\end{equation}

The probability to find more than one photo-electron is then:

\begin{equation}
P(>0) = 1 - e^{-R\cdot WS} 
\end{equation}

Figures~\ref{fig:sphe:sphespectrum} show spectra 
obtained with the digital filter applied on two different global search windows.
One can clearly distinguish a pedestal peak (fitted to Gaussian with index 0), 
corresponding to the case of  $P(0)$ and further 
contributions of $P(1)$ and $P(2)$ (fitted to Gaussians with index 1 and 2). 
One can also see that the contribution of $P(0)$ dimishes 
with increasing global search window size.

\begin{figure}
\centering
\includegraphics[height=0.3\textheight]{SinglePheSpectrum-28-Run38995-WS2.5.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{SinglePheSpectrum-28-Run38995-WS4.5.eps}
\vspace{\floatsep}
\includegraphics[height=0.3\textheight]{SinglePheSpectrum-28-Run38995-WS8.5.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Spectrum obtained from the extraction 
of a pedestal run using a sliding window of 6 FADC slices allowed to move within a window of 
7 (top), 9 (center) and 13 slices.
A pedestal run with galactic star background has been taken and one exemplary pixel (Nr. 100).
One can clearly see the pedestal contribution and a further part corresponding to one or more 
photo-electrons.}
\label{fig:df:sphespectrum}
\end{figure}

In the following, we will make a short consistency test: Assuming that the spectral peaks are 
attributed correctly, one would expect the following relation:

\begin{equation}
P(0) / P(>0) = \frac{e^{-R\cdot WS}}{1-e^{-R\cdot WS}}
\end{equation}

We tested this relation assuming that the fitted area underneath the pedestal peak $Area_0$ is 
proportional to $P(0)$ and the sum of the fitted areas underneath the single photo-electron peak 
$Area_1$ and the double photo-electron peak $Area_2$ proportional to $P(>0)$. Thus, one expects:

\begin{equation}
Area_0 / (Area_1 + Area+2 ) = \frac{e^{-R\cdot WS}}{1-e^{-R\cdot WS}}
\end{equation}

We estimated the effective window size $WS$ as the sum of the range in which the digital filter 
amplitude weights are greater than 0.5 (1.6 FADC slices) and the global search window minus the 
size of the window size of the weights (which is 6 FADC slices). Figures~\ref{fig::df:ratiofit}
show the result for two different levels of night-sky background.

\par

\begin{figure}[htp]
\centering
\includegraphics[height=0.4\textheight]{SinglePheRatio-28-Run38995.eps}
\vspace{\floatsep}
\includegraphics[height=0.4\textheight]{SinglePheRatio-28-Run39258.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Fit to the ratio of the area beneath the pedestal peak and 
the single and double photo-electron(s) peak(s) with the extraction algorithm
applied on a sliding window of different sizes.
In the top plot, a pedestal run with extra-galactic star background has been taken and in the bottom, 
a galatic star background. An exemplary pixel (Nr. 100) has been used.
Above, a rate of 0.8 phe/ns and below, a rate of 1.0 phe/ns has been obtained.}
\label{fig:df:ratiofit}
\end{figure}



%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco"
%%% End: