source: trunk/MagicSoft/TDAS-Extractor/Pedestal.tex@ 5913

\section{Criteria for an Optimal Pedestal Extraction \label{sec:pedestals}}

\ldots {\it In this section, the distinction is made between: 
\begin{itemize}
\item Defining the pedestal RMS as contribution
    to the extracted signal fluctuations (later used in the calibration) 
\item Defining the Pedestal Mean and RMS as the result of distributions obtained by 
    applying the extractor to pedestal runs (yielding biases and modified widths).
\item Deriving the correct probability for background fluctuations based on the extracted signal height. 
  ( including biases and modified widths).
\end{itemize}
}

\subsection{Pedestal RMS}


\vspace{1cm}
\ldots {\it  Modified email by W. Wittek from 25 Oct 2004 and 10 Nov 2004}
\vspace{1cm}

The background $BG$ (Pedestal) 
can be completely described by the noise-autocorrelation matrix $\boldsymbol{B}$ 
(eq.~\ref{eq:autocorr}), 
where the diagonal elements give what is usually denoted as the ``Pedestal RMS''. 

\par

By definition, the noise autocorrelation matrix $B$ and thus the ``pedestal RMS'' 
is independent from the signal extractor.

\subsection{Bias and Error}

Consider a large number of signals (FADC spectra), all with the same
integrated charge $ST$ (true signal). By applying some signal extractor
we obtain a distribution of extracted signals $SE$ (for fixed $ST$ and
fixed background fluctuations $BG$). The distribution of the quantity

\begin{equation}
X = SE-ST 
\end{equation}

has the mean $B$ and the RMS $R$ defined by:

\begin{eqnarray}
   B    &=& <X> \\
   R^2  &=& <(X-B)^2>
\end{eqnarray}

One may also define

\begin{equation}
   D^2 = <(SE-ST)^2> = <(SE-ST-B + B)^2> = B^2 + R^2
\end{equation}

The parameter $B$ can be called the bias of the pedestal extractor and $R$ 
the RMS of the distribution of $X$ and $D$ is something
like the (asymmetric) error of $SE$. 
The distribution of $X$, and thus the parameters $B$ and $R$, 
depend generally on the size of $ST$ and the size of the background fluctuations $BG$.

\par

For the normal image cleaning, knowledge of $B$ is sufficient and the 
error $R$ should be known in order to calculate a correct background probability.

\par
\ldots {\textit{\bf THOMAS SCHWEIZER ???}}
\par
Also for the model analysis $B$ and $R$ are needed if one wants to keep small
signals. 
\par
In the case of the calibration with the F-Factor methoid, 
the basic relation is:

\begin{equation}
\frac{(\Delta ST)^2}{<ST>^2} = \frac{1}{<n_{phe}>} * F^2
\end{equation}

Here $\Delta ST$ is the fluctuation of the true signal $ST$ due to the
fluctuation of the number of photo electrons. $ST$ is obtained from the
measured fluctuations of $SE$  ($RMS_{SE}$) by subtracting those fluctuations of the
extracted signal which are due to the fluctuation of the pedestal ($R$)\footnote{%
A way to check whether the right RMS has been subtracted is to make the
Razmick plot

\begin{equation}
    \frac{(\Delta ST)^2}{<ST>^2} \quad \textit{vs.} \quad \frac{1}{<ST>}
\end{equation}

This should give a straight line passing through the origin. The slope of
the line is equal to

\begin{equation}
    c * F^2
\end{equation}

where $c$ is the photon/ADC conversion factor  $<ST>/<m_{pe}>$.}.

\begin{equation}
 (\Delta ST)^2 = RMS_{SE}^2 - R^2 
\label{eq:rmssubtraction}
\end{equation}

\subsection{How to Retrieve Bias $B$ and Error $R$}

$R$ is in general different from the pedestal RMS. It cannot be
obtained by applying the signal extractor to pedestal events, especially 
for large signals (e.g. calibration signals).
\par
In the case of the optimum filter, $R$ is in theory independent from the 
signal amplitude $ST$ and depends only on the background $BG$ (eq.~\ref{of_noise}).
It can be obtained from the
fitted error of the extracted signal ($\Delta(SE)_{fitted}$), 
which one can calculate for every event or by applying the extractor to a fixed window 
of pure background events (``pedestal events'').

\par

In order to get the missing information, we did the following investigations:
\begin{enumerate}
\item Determine $R$ by applying the signal extractor to a fixed window
    of pedestal events. The background fluctuations can be simulated with different 
    levels of night sky background and the continuous light, but no signal size 
    dependency can be retrieved with the method. 
\item Determine bias $B$ and resolution $R$ from MC events with and without added noise. 
    Assuming that $R$ and $B$ are negligible for the events without noise, one can 
    get a dependency of both values from the size of the signal. 
\item Determine $R$ from the fitted error of $SE$, which is possible for the 
    fit and the digital filter (eq.~\ref{of_noise}). 
    In prinicple, all dependencies can be retrieved with this method.
\end{enumerate}

\subsubsection{ \label{sec:determiner} Determine $R$ by Applying the Signal Extractor to a Fixed Window
of Pedestal Events}

By applying the signal extractor to a fixed window of pedestal events, we 
determined the parameter $R$ for the case of no signal ($ST = 0$). In the case of 
all extractors using a fixed window from the beginning (extractors nr. \#1 to \#22 
in section~\ref{sec:algorithms}), the results are thus by construction the same as calculating 
the mean and the RMS of a same (fixed) number of FADC slices (the conventional ``Pedestal 
Calculation'').

\par
In the case of the amplitude extracting spline (extractor nr. \#23), we placed the 
spline maximum value (which determines the exact extraction window) at a random place 
within the digitizing binning resolution (0.01 FADC slices) 
of one central FADC slice. 
In the case of the digital filter (extractor nr. \#28), the time shift was  
randomized for each event within one central FADC slice.

\par

The following plots~\ref{fig:df:distped:run38993} through~\ref{fig:amp:relrms:run38996} show results 
obtained with the second method for three background intensities: 
\begin{enumerate}
\item Closed camera and no (Poissonian) fluctuation due to photons from the night sky background
\item The camera pointing to a galactic region with stars in the field of view
\item The camera illuminated by a continuous light source of high intensity causing much higher pedestal 
fluctuations than in usual observation conditions.
\end{enumerate}

Figures~\ref{fig:df:distped:run38993},~\ref{fig:df:distped:run38995},~\ref{fig:df:distped:run38996},
and~\ref{fig:amp:distped:run38993},~\ref{fig:amp:distped:run38995},~\ref{fig:amp:distped:run38996} show the 
extracted pedestal distributions for the digital filter with cosmics weights (extractor~\#28) and the 
spline amplitude (extractor~\#27), respectively for one examplary channel (corresponding to pixel 200). 
One can see the (asymmetric) Poisson behaviour of the 
night sky background photons for the distributions with open camera and the cutoff at the lower egde 
for the distribution with high-intensity continuous light due to a limited pedestal offset and the cutoff 
to negative fluctuations.
\par
Figures~\ref{fig:df:relmean:run38993},~\ref{fig:df:relmean:run38995},~\ref{fig:df:relmean:run38996},
and~\ref{fig:amp:relmean:run38993},~\ref{fig:amp:relmean:run38995},~\ref{fig:amp:relmean:run38996} show the 
relative difference between the calculated pedestal mean and 
the one obtained by applying the extractor for 
all channels of the MAGIC camera. One can see that in all cases, the distribution is centered around zero, 
while its width is never larger than 0.01 which corresponds about to the precision of the extracted mean for 
the number of used events. (A very similar distribution is obtained by comparing the results 
of the same pedestal calculator applied to different ranges of FADC slices.)
\par
Figures~\ref{fig:df:relrms:run38993},~\ref{fig:df:relrms:run38995},~\ref{fig:df:relrms:run38996},
and~\ref{fig:amp:relrms:run38993},~\ref{fig:amp:relrms:run38995},~\ref{fig:amp:relrms:run38996} show the 
relative difference between the calculated pedestal RMS, normalized to an equivalent number of slices 
(2.5 for the digital filter and 1. for the amplitude of the spline) and 
the one obtained by applying the extractor for all channels of the MAGIC camera. 
One can see that in all cases, the distribution is not centered around zero, but shows an offset depending 
on the light intensity. The difference can be 10\% in the case of the digital filter and even 25\% for the 
spline. This big difference for the spline is partly explained by the fact that the pedestals have to be 
calculated from an even number of slices to account for the clock-noise. However, the (normalized) pedestal 
RMS depends critically on the number of summed FADC slices, especially at very low numbers. In general, 
the higher the number of summed FADC slices, the higher the (to the square root of the number of slices) 
normalized pedestal RMS.

\begin{figure}[htp]
\centering
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_00_18_02_14_Run_38993_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Distribution of extracted "pedestals"  from pedestal run with 
closed camera lids for one channel.}
\label{fig:df:distped:run38993}
\vspace{\floatsep}
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_00_18_02_14_Run_38995_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Distribution of extracted "pedestals"  from pedestal run with galactic star background for one channel.}
\label{fig:df:distped:run38995}
\vspace{\floatsep}
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_00_18_02_14_Run_38996_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Distribution of extracted "pedestals"  from run with 
continuous light level 100 for one channel.}
\label{fig:df:distped:run38996}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38993_RelMean.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Difference in mean pedestal (per FADC slice) from pedestal 
run with closed camera lids (in photo-electrons)}
\label{fig:df:relmean:run38993}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38995_RelMean.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Difference in mean pedestal (per FADC slice) from pedestal 
run with galactic star background (in photo-electrons)}
\label{fig:df:relmean:run38995}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeDigitalFilter: Difference in mean pedestal (per FADC slice) from run 
with continuous light level: 100 (in photo-electrons)}
\label{fig:df:relmean:run38996}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38993_RMSDiff.eps}
\caption{MExtractTimeAndChargeDigitalFilter:  Difference pedestal RMS (per FADC slice) with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices. 
Pedestal run 
with closed camera lids for inner (left) and outer (right) pixels. }
\label{fig:df:relrms:run38993}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38995_RMSDiff.eps}
\caption{MExtractTimeAndChargeDigitalFilter:  Difference pedestal RMS (per FADC slice)  with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices. 
 from pedestal run with galactic star background for inner (left) 
and outer (right) pixels. }
\label{fig:df:relrms:run38995}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeDigitalFilter_Weights_cosmics_weights.dat_Range_01_14_02_14_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeDigitalFilter:  Difference pedestal RMS (per FADC slice)  with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices. 
 from run with continuous light level: 100 for inner (left) 
and outer (right) pixels. } 
\label{fig:df:relrms:run38996}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeSpline_Amplitude_Range_00_10_04_11_Run_38993_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Distribution of extracted "pedestals"  from pedestal run 
with closed camera lids for one channel.}
\label{fig:amp:distped:run38993}
\vspace{\floatsep}
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeSpline_Amplitude_Range_00_10_04_11_Run_38995_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Distribution of extracted "pedestals"  from pedestal run 
with galactic star background for one channel.}
\label{fig:amp:distped:run38995}
\vspace{\floatsep}
\includegraphics[height=0.29\textheight]{MExtractTimeAndChargeSpline_Amplitude_Range_00_10_04_11_Run_38996_Signal_Pixel200.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Distribution of extracted "pedestals"  from run with 
continuous light level: 100 for one channel.}
\label{fig:amp:distped:run38996}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38993_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Difference in mean pedestal (per FADC slice)  with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices.
Pedestal run with closed camera lids.}
\label{fig:amp:relmean:run38993}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38995_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Difference in mean pedestal (per FADC slice)  with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices
Pedestal run with galactic star background.}
\label{fig:amp:relmean:run38995}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with amplitude: Difference in mean pedestal (per FADC slice)  with extraction algorithm
appied on a fixed window, and simply summing up the same number of FADC slices.
Pedestal run with continuous light level: 100}
\label{fig:amp:relmean:run38996}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38993_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with amplitude:  Difference pedestal RMS (per FADC slice)  with extraction 
 algorithm appied on a fixed window, and simply summing up the same number of FADC slices.
Pedestal run 
with closed camera lids for inner (left) and outer (right) pixels. }
\label{fig:amp:relrms:run38993}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38995_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with amplitude:  Difference pedestal RMS (per FADC slice)  with extraction 
algorithm appied on a fixed window, and simply summing up the same number of FADC slices.
Pedestal run with galactic star background for inner (left) 
and outer (right) pixels.}
\label{fig:amp:relrms:run38995}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Amplitude_Amplitude_Range_01_09_01_10_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with amplitude:  Difference pedestal RMS (per FADC slice)  with extraction 
algorithm appied on a fixed window, and simply summing up the same number of FADC slices.
Pedestal run with continuous light level: 100 for inner (left) 
and outer (right) pixels.} 
\label{fig:amp:relrms:run38996}
\end{figure}



\begin{figure}[htp]
\centering
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38993_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices: Difference in mean pedestal (per FADC slice) 
Pedestal run with closed camera lids.}
\label{fig:int:relmean:run38993}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38995_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices: Difference in mean pedestal (per FADC slice) 
Pedestal run with galactic star background.}
\label{fig:int:relmean:run38995}
\vspace{\floatsep}
\includegraphics[height=0.27\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38996_RelMean.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices: Difference in mean pedestal (per FADC slice) 
Pedestal run with continuous light level: 100}
\label{fig:int:relmean:run38996}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38993_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices:  Difference pedestal RMS (per FADC slice) 
  from pedestal run 
with closed camera lids for inner (left) and outer (right) pixels  (in photo-electrons). }
\label{fig:amp:relrms:run38993}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38995_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices:  Difference pedestal RMS (per FADC slice) 
  from pedestal run with galactic star background for inner (left) 
and outer (right) pixels  (in photo-electrons).}
\label{fig:amp:relrms:run38995}
\vspace{\floatsep}
\includegraphics[height=0.25\textheight]{MExtractTimeAndChargeSpline_Rise-and-Fall-Time_0.5_1.5_Range_01_10_02_12_Run_38996_RMSDiff.eps}
\caption{MExtractTimeAndChargeSpline with integral over 2 slices:  Difference pedestal RMS (per FADC slice) 
  from run with continuous light level: 100 for inner (left) 
and outer (right) pixels  (in photo-electrons).} 
\label{fig:amp:relrms:run38996}
\end{figure}




\vspace{1cm}
\ldots{\it More test plots can be found under: 
http://magic.ifae.es/$\sim$markus/ExtractorPedestals/ }
\vspace{1cm}

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco."
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "Pedestal"
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco."
%%% TeX-master: "MAGIC_signal_reco"
%%% End: