source: trunk/MagicSoft/TDAS-Extractor/MonteCarlo.tex@ 13657

\section{Monte Carlo \label{sec:mc}}

\subsection{Introduction \label{sec:mc:intro}}

Many characteristics of the extractor can only be investigated with the use of Monte-Carlo simulations~\cite{MC-Camera} 
of signal pulses and noise for the following reasons:

\begin{itemize}
\item While in real conditions, the signal can only be obtained in a Poisson distribution, simulated pulses of a specific 
number of photo-electrons can be generated.
\item The intrinsic arrival time spread can be chosen within the simulation.
\item The noise auto-correlation in the low-gain channel cannot be determined from data, 
but instead has to be retrieved from Monte-Carlo studies.
\item The same pulse can be studied with and without added noise, where the noise level can be deliberately adjusted.
\item The photo-multiplier and optical link gain fluctuations can be tuned or switched off completely.
\end{itemize}

Nevertheless, there are always systematic differences between the simulation and the real detector. In our case, especially the 
following short-comings are of concern:

\begin{itemize}
\item The low-gain pulse is not yet simulated with the correct pulse width, but instead the same pulse shape as the one of the 
high-gain channel has been used.
\item The low-gain pulse starts to saturate at already about 200 photo-electrons while in reality, 
this limit lies at more than 500 photo-electrons for an inner pixel. This is due to the wider low-gain pulse in real conditions.
\item The low-gain pulse is delayed by only 15 FADC slices in the Monte-Carlo simulations, while it arrives about 16.5 FADC slices 
after the high-gain pulse in real conditions.
\item No switching noise due to the low-gain switch has been simulated.
\item The intrinsic transit time spread of the photo-multipliers has not been simulated.
\item The pulses have been simulated in steps of 0.2\,ns before digitization. There is thus an artificial numerical time resolution 
limit of $0.2\,\mathrm{ns}/\sqrt{12} \approx 0.06\,\mathrm{ns}$.
\item The total dynamic range of the entire signal transmission chain was set to infinite, thus the detector has been simulated 
to be completely linear.
\end{itemize}

For the subsequent studies, the following settings have been used: 

\begin{itemize}
\item The gain fluctuations for signal pulses were switched off.
\item The gain fluctuations for the background noise of the light of night sky were instead fully simulated, i.e. very close to 
real conditions.
\item The intrinsic arrival time spread of the photons was set to be 1\,ns, as expected for gamma showers.
\item The conversion of total integrated charge to photo-electrons was set to be 7.8~FADC~counts 
per photo-electron, independent of the signal strength.
\item The trigger jitter was set to be uniformly distributed over 1~FADC slice only.
\item Only one inner pixel has been simulated.
\end{itemize}

The last point had the consequence that the extractor {\textit {\bf MExtractFixedWindowPeakSearch}} could not be tested since 
it was equivalent to the sliding window. 
In the following, we used the Monte-Carlo to determine especially the following quantities for each of the tested extractors:

\begin{itemize}
\item The charge resolution as a function of the input signal strength.
\item The charge extraction bias as a function of the input signal strength.
\item The time resolution as a function of the input signal strength.
\item The effect of adding or removing noise for the above quantities.
\end{itemize}

\subsection{Conversion Factors \label{sec:mc:convfactors}}

The following figures~\ref{fig:mc:ChargeDivNphe_FixW} through~\ref{fig:mc:ChargeDivNphe_DFSpline} show the conversion factors 
between reconstructed charge and the number of input photo-electrons for each of the tested extractors, with and without added noise 
and for the high-gain and low-gain channels, respectively. One can see that the conversion factors depend on the extraction window size and 
that the addition of noise raises the conversion factors uniformly for all fixed window extractors in the high-gain channel, 
while the sliding window extractors show a bias a low signal intensities.

\begin{figure}[htp]%%[t!]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_FixW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_FixW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_FixW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_FixW_WithNoise_LoGain.eps}
\caption[Charge per Number of photo-electrons Fixed Windows]{Extracted charge per photoelectron versus number of photoelectrons, 
for fixed window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeDivNphe_FixW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_SlidW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_SlidW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_SlidW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_SlidW_WithNoise_LoGain.eps}
\caption[Charge per Number of photo-electrons Sliding Windows]{Extracted charge per photoelectron versus number of photoelectrons, 
for sliding window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeDivNphe_SlidW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_DFSpline_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_DFSpline_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_DFSpline_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeDivNphevsNphe_DFSpline_WithNoise_LoGain.eps}
\caption[Charge per Number of photo-electrons Spline and Digital Filter]{Extracted charge per photoelectron versus number of photoelectrons, 
for spline and digital filter extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeDivNphe_DFSpline}
\end{figure}

\clearpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Measurement of the Biases \label{sec:mc:baises}}

We fitted the conversion factors obtained from the previous section in the constant region (above 10\,phe) and used 
them to convert the extracted charge back to equivalent photo-electrons. After subtracting the simulated number of photo-electrons, 
the bias (in units of photo-electrons) is obtained. 
\par
Figure~\ref{fig:mc:ConversionvsNphe_FixW} through~\ref{fig:mc:ChargeRes_DFSpline} show the results for the tested extractors, with and 
without added noise and for the high and low-gain regions separately.
\par
As expected, the fixed window extractor do not show any bias up to statistical precision. All sliding window extractor, however, do show
a bias. Usually, the bias vanishes for signals above 5--10~photo-electrons, except for the sliding windows with window sizes above 
8~FADC slices. There, the bias only vanishes for signals above 20~photo-electrons. The size of the bias as well as the minimum signal
strength above which the bias vanishes are clearly correlated with the extraction window size. Therefore, smaller window sizes yield
smaller biases and extend their linear range further downwards. The best extractors have a negligible bias above about 5 photo-electrons. 
This corresponds to the results found in section~\ref{sec:pedestals} where the lowest image cleaning threshold for extra-galactic 
noise levels yielded about 5 photo-electrons as well.
\par
All integrating spline extractors and all sliding window extractors with extraction windows above or equal 6 FADC slices
 yield the comparably smallest biases. The rest results to be about a factor 1.5 higher. The spline and digital filter biases fall 
down very steeply.


\begin{figure}[htp]%%[t!]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_FixW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_FixW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_FixW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_FixW_WithNoise_LoGain.eps}
\caption[Bias Fixed Windows]{The measured bias (extracted charge divided by the conversion factor minus the number of photoelectrons) 
versus number of photoelectrons, 
for fixed window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ConversionvsNphe_FixW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_SlidW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_SlidW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_SlidW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_SlidW_WithNoise_LoGain.eps}
\caption[Bias Sliding Windows]{The measured bias (extracted charge divided by the conversion factor minus the number of photoelectrons) 
versus number of photoelectrons, 
for sliding window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ConversionvsNphe_SlidW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_DFSpline_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_DFSpline_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_DFSpline_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ConversionvsNphe_DFSpline_WithNoise_LoGain.eps}
\caption[Bias Spline and Digital Filter]{The measured bias (extracted charge divided by the conversion factor minus the number of photoelectrons) 
versus number of photoelectrons, 
for spline and digital filter extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ConversionvsNphe_DFSpline}
\end{figure}

\clearpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Measurement of the Resolutions \label{sec:mc:resolutions}}

In order to obtain the resolution of a given extractor, we calculated the RMS of the distribution:

\begin{equation}
R_{\mathrm{MC}} \approx RMS(\widehat{Q}_{rec} - Q_{sim})
\end{equation}

where $\widehat{Q}_{rec}$ is the reconstructed charge, calibrated to photo-electrons with the conversion factors obtained in 
section~\ref{sec:mc:convfactors}.
\par
One can see that for small signals, small extraction windows yield better resolutions, but extractors which do not
entirely cover the whole pulse, show a clear dependency of the resolution with the signal strength. In the high-gain region, 
this is valid for all fixed window extractors up to 6~FADC slices integration region, all sliding window extractors up to 4~FADC 
slices and for all spline extractors and the digital filter. Among those extractors with a signal dependent resolution, the 
digital filter with 6~FADC slices extraction window shows the smallest dependency, namely 80\% per 50 photo-electrons. This 
finding is at first sight in contradiction with eq.~\ref{eq:of_noise} where the (theoretical) resolution depends only on the 
noise intensity, but not on the signal strength. Here, the input light distribution of the simulated light pulse introduces the 
amplitude dependency (the constancy is recovered for photon signals with no intrinsic input time spread). Here, the main 
difference between the spline and digital filter extractors is found: At all intensities, but especially very low intensities, the 
resolution of the digital filter is much better than the one for the spline.

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_FixW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_FixW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_FixW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_FixW_WithNoise_LoGain.eps}
\caption[Charge Resolution Fixed Windows]{The measured resolution (RMS of extracted charge divided by the conversion factor minus the number of photoelectrons) versus number of photoelectrons, 
for fixed window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeRes_FixW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_SlidW_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_SlidW_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_SlidW_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_SlidW_WithNoise_LoGain.eps}
\caption[Charge Resolution Sliding Windows]{The measured resolution (RMS of extracted charge divided by the conversion factor minus the number of photoelectrons) versus number of photoelectrons, 
for sliding window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeRes_SlidW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_DFSpline_NoNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_DFSpline_WithNoise_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_DFSpline_NoNoise_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_ChargeRes_DFSpline_WithNoise_LoGain.eps}
\caption[Charge Resolution Spline and Digital Filter]{The measured resolution 
(RMS of extracted charge divided by the conversion factor minus the number of photoelectrons) versus number of photoelectrons, 
for spline and digital filter extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:ChargeRes_DFSpline}
\end{figure}

\clearpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\subsection{Arrival Times \label{sec:mc:times}}

Like in the case of the charge resolution, we calculated the RMS of the distribution of the deviation of the 
reconstructed arrival time with respect to the simulated time:

\begin{equation}
\Delta T_{\mathrm{MC}} \approx RMS(\widehat{T}_{rec} - T_{sim})
\end{equation}

where $\widehat{T}_{rec}$ is the reconstructed arrival time and $T_{sim}$ the simulated one.
\par
Generally, the time resolutions $\Delta T_{\mathrm{MC}}$ are about  a factor 1.5 better than those obtained 
from the calibration (section~\ref{sec:cal:timeres}, figure~\ref{fig:time:dep}). This 
 is understandable since the Monte-Carlo pulses are smaller and 
further the intrinsic time spread of the photo-multiplier has not been simulated. Moreover, no time resolution offset was 
simulated, thus the reconstructed time resolutions follow about a $1/\sqrt{N_{\mathrm{phe}}}$\,--\,behaviour over the whole low-gain range.
The spline extractors level off in contradiction to what has been found with the calibration pulses.
\par
In figure~\ref{fig:mc:TimeRes_SlidW}, one can see nicely the effect of the addition of noise to the reconstructed time 
resolution: While without noise all sliding window extractors with a window size of at least 4~FADC slices show the same time 
resolution, with added noise, the resolution degrades with larger extraction window sizes. This can be understood by the fact that 
an extractor covers the whole pulse if integrating at least 4~FADC slices and each additional slice can only be affected by the noise.

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_SlidW_NoNoise_HiGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_SlidW_WithNoise_HiGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_SlidW_NoNoise_LoGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_SlidW_WithNoise_LoGain.eps}
\caption[Time Resolution Sliding Windows]{The measured time resolution (RMS of extracted time minus simulated time) 
versus number of photoelectrons, 
for sliding window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:TimeRes_SlidW}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_DFSpline_NoNoise_HiGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_DFSpline_WithNoise_HiGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_DFSpline_NoNoise_LoGain.eps}
\vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_TimeRes_DFSpline_WithNoise_LoGain.eps}
\caption[Time Resolution Spline and Digital Filter]{The measured time resolution (RMS of extracted time minus simulated time) 
versus number of photoelectrons, 
for spline and digital filter window extractors in different window sizes. The top plots show the high-gain and the bottom ones 
low-gain regions. Left: without noise, right: with simulated noise.}
\label{fig:mc:TimeRes_DFSpline}
\end{figure}


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "MAGIC_signal_reco"
%%% TeX-master: "MAGIC_signal_reco"
%%% End: 








%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\clearpage

\subsection{Charge Signals with and without Simulated Noise \label{fig:mc:sec:mc:chargenoise}}


\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_SlidW_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_FixW_HiGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_DFSpline_HiGain.eps}
\caption[Bias due to noise high-gain]{Bias due to noise: Difference of extracted charge of same events, with and without simulated noise, 
for different extractor methods in the high-gain region.}
\label{fig:mc:Bias_HiGain}
\end{figure}

\begin{figure}[htp]
\centering
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_SlidW_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_FixW_LoGain.eps}
  \vspace{\floatsep}
  \includegraphics[width=0.49\linewidth]{TimeAndChargePlots/TDAS_Bias_DFSpline_LoGain.eps} 
\caption[Bias due to noise low-gain]{Bias due to noise: Difference of extracted charge of same events, with and without simulated noise, 
for different extractor methods in the low-gain region.}
\label{fig:mc:Bias_LoGain}
\end{figure}