source: trunk/MagicSoft/TDAS-Extractor/Performance.tex@ 5637

\section{Performance}

\subsection{Calibration}

In this section, we describe the tests performed using light pulses of different colour, 
pulse shapes and intensities with the MAGIC calibration pulser box. 
\par
The LED pulser system is able to provide fast light pulses of 3--4\,ns FWHM 
with intensities ranging from 3--4 photo-electrons to more than 500 in one inner pixel of the 
camera. These pulses can be produced in three colours $green$, $blue$ and $UV$.

\begin{table}[htp]
\centering
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\hline
\multicolumn{6}{|c|}{The possible pulsed light colours} \\
\hline
\hline
Colour &  Wavelength & Spectral Width & Min. Nr. &  Max. Nr. & Secondary \\
      & [nm]         & [nm]           &  Phe's   &  Phe's    & Pulses  \\
\hline
Green &  520      & 40      & 6          &  120      & yes   \\
\hline
Blue &  460       & 30      & 6          &  500      & yes   \\
\hline
UV   &  375       & 12      & 3          &  50       & no  \\ 
\hline
\hline
\end{tabular}
\caption{The pulser colours available from the calibration system}
\label{tab:pulsercolours}
\end{table}

Table~\ref{tab:pulsercolours} lists the available colours and intensities. 
Whereas the UV-pulse is very stable, the green and blue pulses show sometimes smaller secondary 
pulses after about 10--40\,ns from the main pulse.
One can see that the very stable UV-pulses are unfortunately only available in such intensities as to 
not saturate the high-gain readout channel. However, the brightest combination of light pulses easily 
saturates all channels in the camera, but does not reach a saturation of the low-gain channel.
\par
Our tests can be classified into three subsections:

\begin{enumerate}
\item Un-calibrated pixels and events: These tests measure the percentage of failures of the extractor 
resulting either in a pixel declared as un-calibrated or in an event which produces a signal ouside 
of the expected Gaussian distribution.
\item Number of photo-electrons: These tests measure the reconstructed numbers of photo-electrons, their 
spread over the camera and the ratio of the obtained mean value for outer and inner pixels.
\item Linearity tests: These test the linearity of the extractor with respect to pulses of different intensity 
and colour.
\item Time resolution: These tests show the time resolution and stability obtained with different 
intensities and colours.
\end{enumerate}

We used data taken on the 7$^{th}$ of June, 2004 with different pulser LED combinations, each taken with 
16384 events. The corresponding run numbers range from nr. 31741 to 31772. This data was taken before the 
latest camera repair access which replaced about 2\% of the pixels known to be mal-functionning at that time.
Thus, there is a lower limit to the number of un-calibrated pixels of about 1.5--2\%.
\par
Although, we had looked at and tested all colour and extractor combinations resulting from these data, 
we refrain ourselves to show here only exemplary behaviour and results of extractors. 
All plots, including those which are not displayed in this TDAS, can be retrieved from the following 
locations:

\begin{verbatim}
http://www.magic.ifae.es/~markus/pheplots/
http://www.magic.ifae.es/~markus/timeplots/
\end{verbatim}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsubsection{Un-calibrated pixels and events}

The MAGIC calibration software incorporates a series of checks to sort out mal-functionning pixels. 
Except for the software bug searching criteria, the following exclusion reasons can apply:

\begin{enumerate}
\item The reconstructed mean signal is less than 2.5 times the extractor resolution $R$ from zero. 
(2.5 Pedestal RMS in the case of the simple fixed window extractors). This criterium cuts out 
dead pixels.
\item The reconstructed mean signal error is smaller than its value. This criterium cuts out 
signal distributions which fluctuate so much that their RMS is bigger than its mean value. This 
criterium cuts out ``ringing'' pixels or mal-functionning extractors. 
\item The reconstructed mean number of photo-electrons lies 4.5 sigma outside 
the distribution of photo-electrons obtained with the inner or outer pixels in the camera. 
\item All reconstructed negative mean signal, signal sigma's and mean numbers of photo-electrons 
smaller than one.
\end{enumerate}

Moreover, the number of events are counted which have been reconstructed outside a 5 sigma region 
from the mean signal. These events are called ``outliers''. Figure~\ref{fig:outlier} shows a typical 
outlier obtained with the digital filter.

\begin{figure}[htp]
\centering
\includegraphics[width=0.95\linewidth]{Outlier.eps}
\caption{Example of an event classified as ``un-calibrated''. The histogram has been obtained 
using the digital filter (extractor \#32) applied to a high-intensity blue pulse (run 31772). 
The event marked as ``outlier'' clearly has been mis-reconstructed. It lies outside the 5 sigma 
region from the fitted mean.}
\label{fig:outlier}
\end{figure}

The following figures~\ref{fig:unsuited:5ledsuv},~\ref{fig:unsuited:1leduv},~\ref{fig:unsuited:2ledsgreen}
and~\ref{fig:unsuited:23ledsblue} show the resulting numbers of un-calibrated pixels and events for 
different colours and intensities. 

\par

\begin{figure}[htp]
\centering
\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-5LedsUV-Colour-13.eps}
\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a typical calibration  
pulse of UV-light which does not saturate the high-gain readout.}
\label{fig:unsuited:5ledsuv}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-1LedUV-Colour-04.eps}
\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a very low 
intensity pulse.}
\label{fig:unsuited:1leduv}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-2LedsGreen-Colour-02.eps}
\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a typical green pulse.}
\label{fig:unsuited:2ledsgreen}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-23LedsBlue-Colour-00.eps}
\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a high-intensity blue pulse.}
\label{fig:unsuited:23ledsblue}
\end{figure}

One can see that in general, big extraction windows raise the 
number of un-calibrated pixels and are thus less stable. Especially for the very low-intensity 
$1LedUV$-pulse, the big extraction windows summing 8 or more slices, cannot calibrate more than 50\% 
of the inner pixels (fig.~\ref{fig:unsuited:1leduv}). This is an expected behavior since big windows 
add up more noise which in turn makes the for the small signal more difficult.
\par
In general, one can also say that all ``sliding window''-algorithms (extractors \#17-32) discard 
less pixels than the ``fixed window''-ones (extractors \#1--16). The digital filter with 
the correct weights (extractor \#32) discards the least number of pixels, but is also robust against 
slight modifications of its weights (extractors \#28--31). Also the ``spline'' algorithms on small  
windows (extractors \#23--25) discard less pixels than the previous extractors, although slightly more 
then the digital filter.
\par
Concerning the numbers of outliers, one can conclude that in general, the numbers are very low never exceeding
0.25\%. There seems to be the opposite trend of larger windows producing less 
outliers. However, one has to take into account that already more ``unsuited'' pixels have 
been excluded thus cleaning up the sample somewhat. It seems that the ``digital filter'' and a 
medium-sized ``spline'' (extractors \#25--26) yield the best result except for the outer pixels 
in fig~\ref{fig:unsuited:5ledsuv} where the digital filter produces a worse result than the rest 
of the extractors.
\par
In conclusion, one can say that this test excludes all extractors with too big window sizes because 
they are not able to extract small signals produced by about 4 photo-electrons. The excluded extractors 
are:
\begin{itemize}
\item: MExtractFixedWindow Nr. 3--5
\item: MExtractFixedWindowSpline Nr. 6--11
\item: MExtractFixedWindowPeakSearch Nr. 14--16
\item: MExtractTimeAndChargeSlidingWindow Nr. 21--22
\item: MExtractTimeAndChargeSpline Nr. 27
\end{itemize}

The best extractors after this test are:
\begin{itemize}
\item: MExtractFixedWindow Nr. 1--2
\item: MExtractFixedWindowPeakSearch Nr. 13
\item: MExtractTimeAndChargeSlidingWindow Nr. 17--19
\item: MExtractTimeAndChargeSpline Nr. 24--25
\item: MExtractTimeAndChargeDigitalFilter Nr. 28--32
\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsubsection{Number of photo-electrons}

Assuming that the readout chain is clean and adds only negligible noise with respect to the one 
introduced by the photo-multiplier itself, one can make the assumption that variance of the 
true (non-extracted) signal $ST$ is the amplified Poisson variance on the number of photo-electrons, 
multiplied with the excess noise of the photo-multiplier, characterized by the excess-noise factor $F$.

\begin{equation}
Var(ST) = F^2 \cdot Var(N_{phe}) \cdot \frac{<ST>^2}{<N_{phe}>^2}
\label{eq:excessnoise}
\end{equation}

After introducing the effect of the night-sky background (eq.~\ref{eq:rmssubtraction}) 
in formula~\ref{eq:excessnoise} and assuming that the number of photo-electrons per event follows a 
Poisson distribution, one can 
get an expression to retrieve the mean number of photo-electrons impinging on the pixel from the 
mean extracted signal $<SE>$, its variance $Var(SE)$ and the RMS of the extracted signal obtained from 
pure pedestal runs $R$ (see section~\ref{sec:determiner}):

\begin{equation}
<N_{phe}> \approx F^2 \cdot \frac{Var(SE) - R^2}{<SE>^2}
\label{eq:pheffactor}
\end{equation}

Equation~\ref{eq:pheffactor} must not depend on the extractor! Effectively, we will use it to test the 
quality of our extractors by requiring that a valid extractor yields the same number of photo-electrons 
for all pixels of a same type and does not deviate from the number obtained with other extractors. 
As the camera is flat-fielded, but the number of photo-electrons impinging on an inner and an outer pixel is 
different, we also use the ratio of the mean numbers of photo-electrons from the outer pixels to the one 
obtained from the inner pixels as a test variable. In the ideal case, it should always yield its central 
value of about 2.4--2.8.
\par
In our case, there is an additional complication due to the fact that the green and blue coloured pulses 
show secondary pulses which destroy the Poisson behaviour of the number of photo-electrons. We will thus 
have to split our sample of extractors into those being affected by the secondary pulses and those without
showing any effect. 
\par
Figures~\ref{fig:phe:5ledsuv},~\ref{fig:phe:1leduv},~\ref{fig:phe:23ledsblue}~and~\ref{fig:phe:2ledsgreen} show 
some of the obtained results. Although one can see an amazing stability for the standard 5Leds UV pulse, 
there is a considerable difference for all shown non-standard pulses. Especially the pulses from green 
and blue LEDs 
show a clear dependency on the extraction window of the number of photo-electrons. Only the largest 
extraction windows seem to catch the entire range of (jittering) secondary pulses and get also the ratio 
of outer vs. inner pixels right.
\par
The strongest discrepancy is observed in the low-gain extraction (fig.~\ref{fig:phe:23ledsblue}) where all 
fixed window extractors 


\begin{figure}[htp]
\centering
\includegraphics[height=0.92\textheight]{PheVsExtractor-5LedsUV-Colour-13.eps}
\caption{Number of photo-electrons from a typical, not saturating calibration pulse of colour UV, 
reconstructed with each of the tested signal extractors. 
The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
denote the mean of all not-excluded pixels, the error bars their RMS.}
\label{fig:phe:5ledsuv}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.92\textheight]{PheVsExtractor-1LedUV-Colour-04.eps}
\caption{Number of photo-electrons from a typical, very low-intensity calibration pulse of colour UV, 
reconstructed with each of the tested signal extractors. 
The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
denote the mean of all not-excluded pixels, the error bars their RMS.}
\label{fig:phe:1leduv}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[height=0.92\textheight]{PheVsExtractor-2LedsGreen-Colour-02.eps}
\caption{Number of photo-electrons from a typical, not saturating calibration pulse of colour green, 
reconstructed with each of the tested signal extractors. 
The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
denote the mean of all not-excluded pixels, the error bars their RMS.}
\label{fig:phe:2ledsgreen}
\end{figure}


\begin{figure}[htp]
\centering
\includegraphics[height=0.92\textheight]{PheVsExtractor-23LedsBlue-Colour-00.eps}
\caption{Number of photo-electrons from a typical, high-gain saturating calibration pulse of colour blue, 
reconstructed with each of the tested signal extractors. 
The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
denote the mean of all not-excluded pixels, the error bars their RMS.}
\label{fig:phe:23ledsblue}
\end{figure}

One can see that all extractor using a large window belong to the class of extractors being affected 
by the secondary pulses. The only exception to this rule is the digital filter which - despite of its 
6 slices extraction window - seems to filter out all the secondary pulses. 
\par
Moreover, one can see in fig.~\ref{fig:phe:1leduv} that all peak searching extractors show the influence of 
the bias at low numbers of photo-electrons. 
\par
The extractor MExtractFixedWindowPeakSearch at low extraction windows apparently yields chronically low 
numbers of photo-electrons. This is due to the fact that the decision to fix the extraction window is 
made sometimes by an inner pixel and sometimes by an outer one since the camera is flat-fielded and the 
pixel carrying the largest non-saturated peak-search window is more or found by a random signal 
fluctuation. However, inner and outer pixels have a systematic offset of about 0.5 to 1 FADC slices. 
Thus, the extraction fluctuates artificially for one given channel which results in a systematically 
large variance and thus in a systematically low reconstructed number of photo-electrons. This test thus 
excludes the extractors \#11--13.
\par
Moreover, one can see that the extractors applying a small fixed window do not get the ratio of 
photo-electrons from outer to inner pixels correctly for the green and blue pulses. 
\par
The extractor MExtractTimeAndChargeDigitalFilter seems to be veryu stable against modifications in the 
exact form of the weights since all applied weights yield about the same number of photo-electrons and the 
same ratio of outer vs. inner pixels. The last is also true for the extractor MExtractTimeAndChargeSpline, 
although the number of photo-electrons depends on the extraction window for green and blue pulses,  
(as with the other extractors).

\subsubsection{Linearity tests}

In this section, we test the lineary of the extractors. As the photo-multiplier is a linear device over a 
wide dynamic range, the number of photo-electrons per charge has to remain constant over the tested 
linearity region. We will show here only examples of extractors which were not already excluded in the 
previous section.
\par
A first test concerns the stability of the conversion factor photo-electrons per FADC counts over the 
tested intensity region.



\subsubsection{Time resolution}

\begin{figure}[htp]
\centering
\includegraphics[height=0.25\textheight]{RelArrTime_Pixel97_10LedUV_Extractor32.eps}
\includegraphics[height=0.25\textheight]{RelArrTime_Pixel97_10LedUV_Extractor23.eps}
\caption{Example of a two distributions of relative arrival times of an inner pixel with respect to 
the arrival time of the reference pixel Nr. 1. The left plot shows the result using the digital filter
 (extractor \#32), the right plot shows the result obtained with the half-maximum of the spline. A 
medium sized UV-pulse (10Leds UV) has been used which does not saturate the high-gain readout channel.}
\label{fig:reltimesinner}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel400_10LedUV_Extractor32.eps}
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel400_10LedUV_Extractor23.eps}
\caption{Example of a two distributions of relative arrival times of an outer pixel with respect to 
the arrival time of the reference pixel Nr. 1. The left plot shows the result using the digital filter
 (extractor \#32), the right plot shows the result obtained with the half-maximum of the spline. A 
medium sized UV-pulse (10Leds UV) has been used which does not saturate the high-gain readout channel.}
\label{fig:reltimesouter}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel97_10LedBlue_Extractor32.eps}
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel97_10LedBlue_Extractor23.eps}
\caption{Example of a two distributions of relative arrival times of an inner pixel with respect to 
the arrival time of the reference pixel Nr. 1. The left plot shows the result using the digital filter
 (extractor \#32), the right plot shows the result obtained with the half-maximum of the spline. A 
medium sized Blue-pulse (10Leds Blue) has been used which saturates the high-gain readout channel.}
\label{fig:reltimesinner}
\end{figure}

\begin{figure}[htp]
\centering
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel400_10LedBlue_Extractor32.eps}
\includegraphics[width=0.45\linewidth]{RelArrTime_Pixel400_10LedBlue_Extractor23.eps}
\caption{Example of a two distributions of relative arrival times of an outer pixel with respect to 
the arrival time of the reference pixel Nr. 1. The left plot shows the result using the digital filter
 (extractor \#32), the right plot shows the result obtained with the half-maximum of the spline. A 
medium sized Blue-pulse (10Leds Blue) has been used which saturates the high-gain readout channel.}
\label{fig:reltimesouter}
\end{figure}



\clearpage

\subsection{Pulpo Pulses}
\subsection{MC Data}
\subsection{Cosmics Data?}
The results of this subsection are based on the following runs taken
on the 21st of September 2004.
\begin{itemize}
\item{Run 39000}: OffCrab11 at 19.1 degrees zenith angle and 106.2
azimuth.
\item{Run 39182}: CrabNebula at 19.0 degrees zenith angle and 106.0 azimuth.
\end{itemize}

\subsection{Pedestals}


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco."
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco"
%%% End: