Changeset 5632 for trunk/MagicSoft/TDAS-Extractor

Timestamp:

12/19/04 22:34:38 (20 years ago)

Author:

gaug

Message:

*** empty log message ***

File:

• trunk/MagicSoft/TDAS-Extractor/Performance.tex (modified) (5 diffs)

trunk/MagicSoft/TDAS-Extractor/Performance.tex

@@ -1 +1 @@
 \section{Performance}
-\ldots {\textit This section will be written after the previous one}
 
 \subsection{Calibration}
@@ -44 +43 @@
 
 \begin{enumerate}
-\item Stability tests: These include numbers of excluded pixels by the calibration software, 
-numbers of reconstructed photo-electrons and counts of the numbers of outliers from the expected Gaussian 
-distributions of reconstructed charges. 
+\item Un-calibrated pixels and events: These tests measure the percentage of failures of the extractor 
+resulting either in a pixel declared as un-calibrated or in an event which produces a signal ouside 
+of the expected Gaussian distribution.
+\item Number of photo-electrons: These tests measure the reconstructed numbers of photo-electrons, their 
+spread over the camera and the ratio of the obtained mean value for outer and inner pixels.
 \item Linearity tests: These test the linearity of the extractor with respect to pulses of different intensity 
 and colour.
@@ -54 +55 @@
 
 We used data taken on the 7$^{th}$ of June, 2004 with different pulser LED combinations, each taken with 
-16384 events. The corresponding run numbers range from nr. 31741 to 31772.
-\par
-Although, we looked at and tested all colour and extractor combinations resulting from these data, 
-we will refrain ourselves to show here only exemplary behaviour and results of extractors. All taken  
-``control'' plots including those which are not displayed here, can be retrieved from the following 
+16384 events. The corresponding run numbers range from nr. 31741 to 31772. This data was taken before the 
+latest camera repair access which replaced about 2\% of the pixels known to be mal-functionning at that time.
+Thus, there is a lower limit to the number of un-calibrated pixels of about 1.5--2\%.
+\par
+Although, we had looked at and tested all colour and extractor combinations resulting from these data, 
+we refrain ourselves to show here only exemplary behaviour and results of extractors. 
+All plots, including those which are not displayed in this TDAS, can be retrieved from the following 
 locations:
 
@@ -66 +69 @@
 \end{verbatim}
 
-\subsubsection{Stability tests}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsection{Un-calibrated pixels and events}
+
+The MAGIC calibration software incorporates a series of checks to sort out mal-functionning pixels. 
+Except for the software bug searching criteria, the following exclusion reasons can apply:
+
+\begin{enumerate}
+\item The reconstructed mean signal is less than 2.5 times the extractor resolution $R$ from zero. 
+(2.5 Pedestal RMS in the case of the simple fixed window extractors). This criterium cuts out 
+dead pixels.
+\item The reconstructed mean signal error is smaller than its value. This criterium cuts out 
+signal distributions which fluctuate so much that their RMS is bigger than its mean value. This 
+criterium cuts out ``ringing'' pixels or mal-functionning extractors. 
+\item The reconstructed mean number of photo-electrons lies 4.5 sigma outside 
+the distribution of photo-electrons obtained with the inner or outer pixels in the camera. 
+\item All reconstructed negative mean signal, signal sigma's and mean numbers of photo-electrons 
+smaller than one.
+\end{enumerate}
+
+Moreover, the number of events are counted which have been reconstructed outside a 5 sigma region 
+from the mean signal. These events are called ``outliers''. Figure~\ref{fig:outlier} shows a typical 
+outlier obtained with the digital filter.
+
+\begin{figure}[htp]
+\centering
+\includegraphics[width=0.95\linewidth]{Outlier.eps}
+\caption{Example of an event classified as ``un-calibrated''. The histogram has been obtained 
+using the digital filter (extractor \#32) applied to a high-intensity blue pulse (run 31772). 
+The event marked as ``outlier'' clearly has been mis-reconstructed. It lies outside the 5 sigma 
+region from the fitted mean.}
+\label{fig:outlier}
+\end{figure}
+
+The following figures~\ref{fig:unsuited:5ledsuv},~\ref{fig:unsuited:1leduv},~\ref{fig:unsuited:2ledsgreen}
+and~\ref{fig:unsuited:23ledsblue} show the resulting numbers of un-calibrated pixels and events for 
+different colours and intensities. 
+
+\par
+
+\begin{figure}[htp]
+\centering
+\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-5LedsUV-Colour-13.eps}
+\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a typical calibration  
+pulse of UV-light which does not saturate the high-gain readout.}
+\label{fig:unsuited:5ledsuv}
+\end{figure}
 
 \begin{figure}[htp]
@@ -73 +122 @@
 \caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a very low 
 intensity pulse.}
-\end{figure}
+\label{fig:unsuited:1leduv}
+\end{figure}
+
+\begin{figure}[htp]
+\centering
+\includegraphics[height=0.95\textheight]{UnsuitVsExtractor-2LedsGreen-Colour-02.eps}
+\caption{Uncalibrated pixels and pixels outside of the Gaussian distribution for a typical green pulse.}
+\label{fig:unsuited:2ledsgreen}
+\end{figure}
+
+One can see that in general, big extraction windows raise the 
+number of un-calibrated pixels and are thus less stable. Especially for the very low-intensity 
+$1LedUV$-pulse, the big extraction windows summing 8 or more slices, cannot calibrate more than 50\% 
+of the inner pixels (fig.~\ref{fig:unsuited:1leduv}). This is an expected behavior since big windows 
+add up more noise which in turn makes the for the small signal more difficult.
+\par
+In general, one can also say that all ``sliding window''-algorithms (extractors \#17-32) discard 
+less pixels than the ``fixed window''-ones (extractors \#1--16). The digital filter with 
+the correct weights (extractor \#32) discards the least number of pixels, but is also robust against 
+slight modifications of its weights (extractors \#28--31). Also the ``spline'' algorithms on small  
+windows (extractors \#23--25) discard less pixels than the previous extractors, although slightly more 
+then the digital filter.
+\par
+Concerning the numbers of outliers, one can conclude that in general, the numbers are very low never exceeding
+0.25\%. There seems to be the opposite trend of larger windows producing less 
+outliers. However, one has to take into account that already more ``unsuited'' pixels have 
+been excluded thus cleaning up the sample somewhat. It seems that the ``digital filter'' and a 
+medium-sized ``spline'' (extractors \#25--26) yield the best result except for the outer pixels 
+in fig~\ref{fig:unsuited:5ledsuv} where the digital filter produces a worse result than the rest 
+of the extractors.
+\par
+In conclusion, one can say that this test excludes all extractors with too big window sizes because 
+they are not able to extract small signals produced by about 4 photo-electrons. The excluded extractors 
+are:
+\begin{itemize}
+\item: MExtractFixedWindow Nr. 3--5
+\item: MExtractFixedWindowSpline Nr. 6--11
+\item: MExtractFixedWindowPeakSearch Nr. 14--16
+\item: MExtractTimeAndChargeSlidingWindow Nr. 21--22
+\item: MExtractTimeAndChargeSpline Nr. 27
+\end{itemize}
+
+The best extractors after this test are:
+\begin{itemize}
+\item: MExtractFixedWindow Nr. 1--2
+\item: MExtractFixedWindowPeakSearch Nr. 13
+\item: MExtractTimeAndChargeSlidingWindow Nr. 17--19
+\item: MExtractTimeAndChargeSpline Nr. 24--25
+\item: MExtractTimeAndChargeDigitalFilter Nr. 28--32
+\end{itemize}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsubsection{Number of photo-electrons}
+
+Assuming that the readout chain is clean and adds only negligible noise with respect to the one 
+introduced by the photo-multiplier itself, one can make the assumption that variance of the 
+true (non-extracted) signal $ST$ is the amplified Poisson variance on the number of photo-electrons, 
+multiplied with the excess noise of the photo-multiplier, characterized by the excess-noise factor $F$.
+
+\begin{equation}
+Var(ST) = F^2 \cdot Var(N_{phe}) \cdot \frac{<ST>^2}{<N_{phe}>^2}
+\label{eq:excessnoise}
+\end{equation}
+
+After introducing the effect of the night-sky background (eq.~\ref{eq:rmssubtraction}) 
+in formula~\ref{eq:excessnoise} and assuming that the number of photo-electrons per event follows a 
+Poisson distribution, one can 
+get an expression to retrieve the mean number of photo-electrons impinging on the pixel from the 
+mean extracted signal $<SE>$, its variance $Var(SE)$ and the RMS of the extracted signal obtained from 
+pure pedestal runs $R$ (see section~\ref{sec:determiner}):
+
+\begin{equation}
+<N_{phe}> \approx F^2 \cdot \frac{Var(SE) - R^2}{<SE>^2}
+\label{eq:pheffactor}
+\end{equation}
+
+Equation~\ref{eq:pheffactor} must not depend on the extractor! Effectively, we will use it to test the 
+quality of our extractors by requiring that a valid extractor yields the same number of photo-electrons 
+for all pixels of a same type and does not deviate from the number obtained with other extractors. 
+As the camera is flat-fielded, but the number of photo-electrons impinging on an inner and an outer pixel is 
+different, we also use the ratio of the mean numbers of photo-electrons from the outer pixels to the one 
+obtained from the inner pixels as a test variable. In the ideal case, it should always yield its central 
+value of about 2.4--2.8.
+\par
+In our case, there is an additional complication due to the fact that the green and blue coloured pulses 
+show secondary pulses which destroy the Poisson behaviour of the number of photo-electrons. We will thus 
+have to split our sample of extractors into those being affected by the secondary pulses and those without
+showing any effect. 
+\par
+Figures~\ref{fig:phe:5ledsuv},~\ref{fig:phe:1leduv},~\ref{fig:phe:23ledsblue}~and~\ref{fig:phe:2ledsgreen} show 
+some of the obtained results. Although one can see an amazing stability for the standard 5Leds UV pulse, there 
+is a considerable difference for all shown non-standard pulses. Especially the pulses from green and blue LEDs 
+show a clear dependency on the extraction window of the number of photo-electrons. Only the largest 
+extraction windows seem to catch the entire range of (jittering) secondary pulses and get also the ratio 
+of outer vs. inner pixels right.
+
+\begin{figure}[htp]
+\centering
+\includegraphics[height=0.92\textheight]{PheVsExtractor-5LedsUV-Colour-13.eps}
+\caption{Number of photo-electrons from a typical, not saturating calibration pulse of colour UV, 
+reconstructed with each of the tested signal extractors. 
+The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
+for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
+outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
+denote the mean of all not-excluded pixels, the error bars their RMS.}
+\label{fig:phe:5ledsuv}
+\end{figure}
+
+\begin{figure}[htp]
+\centering
+\includegraphics[height=0.92\textheight]{PheVsExtractor-1LedUV-Colour-04.eps}
+\caption{Number of photo-electrons from a typical, very low-intensity calibration pulse of colour UV, 
+reconstructed with each of the tested signal extractors. 
+The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
+for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
+outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
+denote the mean of all not-excluded pixels, the error bars their RMS.}
+\label{fig:phe:1leduv}
+\end{figure}
+
+\begin{figure}[htp]
+\centering
+\includegraphics[height=0.92\textheight]{PheVsExtractor-2LedsGreen-Colour-02.eps}
+\caption{Number of photo-electrons from a typical, not saturating calibration pulse of colour green, 
+reconstructed with each of the tested signal extractors. 
+The first plots shows the number of photo-electrons obtained for the inner pixels, the second one 
+for the outer pixels and the third shows the ratio of the mean number of photo-electrons for the 
+outer pixels divided by the mean number of photo-electrons for the inner pixels. Points 
+denote the mean of all not-excluded pixels, the error bars their RMS.}
+\label{fig:phe:2ledsgreen}
+\end{figure}
+
+
+One can see that all extractor using a large window belong to the class of extractors being affected 
+by the secondary pulses. The only exception to this rule is the digital filter which - despite of its 
+6 slices extraction window - seems to filter out all the secondary pulses. 
+\par
+Moreover, one can see in fig.~\ref{fig:phe:1leduv} that all peak searching extractors show the influence of 
+the bias at low numbers of photo-electrons. 
+\par
+The extractor MExtractFixedWindowPeakSearch at low extraction windows apparently yields chronically low 
+numbers of photo-electrons. This is due to the fact that the decision to fix the extraction window is 
+made sometimes by an inner pixel and sometimes by an outer one since the camera is flat-fielded and the 
+pixel carrying the largest non-saturated peak-search window is more or found by a random signal 
+fluctuation. However, inner and outer pixels have a systematic offset of about 0.5 to 1 FADC slices. 
+Thus, the extraction fluctuates artificially for one given channel which results in a systematically 
+large variance and thus in a systematically low reconstructed number of photo-electrons. This test thus 
+excludes the extractors \#11--13.
+\par
+Moreover, one can see that the extractors applying a small fixed window do not get the ratio of 
+photo-electrons from outer to inner pixels correctly for the green and blue pulses. 
+\par
+The extractor MExtractTimeAndChargeDigitalFilter seems to be veryu stable against modifications in the 
+exact form of the weights since all applied weights yield about the same number of photo-electrons and the 
+same ratio of outer vs. inner pixels. The last is also true for the extractor MExtractTimeAndChargeSpline, 
+although the number of photo-electrons depends on the extraction window for green and blue pulses,  
+(as with the other extractors).
 
 \subsubsection{Linearity tests}
+
+In this section, we test the lineary of the extractors. As the photo-multiplier is a linear device over a 
+wide dynamic range, the number of photo-electrons per charge has to remain constant over the tested 
+linearity region. We will show here only examples of extractors which were not already excluded in the 
+previous section.
+\par
+A first test concerns the stability of the conversion factor photo-electrons per FADC counts over the 
+tested intensity region.