\documentclass[11pt]{article}
\usepackage{a4}
\usepackage{graphicx}

\oddsidemargin=0pt           % No extra space wasted after first inch.
\evensidemargin=0pt          % Ditto (for two-sided output).
\topmargin=0pt               % Same for top of page.
\headheight=0pt              % Don't waste any space on unused headers.
\headsep=0pt
\textwidth=16cm              % Effectively controls the right margin.
\textheight=24cm

\begin{document}
\title{\bf Discussion paper on the use of MAPS in a LC
electromagnetic calorimeter}
\input pdd.tex

\maketitle

\section{Introduction}
The CALICE collaboration is studying a tungsten-silicon electromagnetic
calorimeter (ECAL) for use at a future linear collider (LC). Although
the detector design is by no means fixed, the current ideas are
described in section~2. Section~3 describes the possible use of MAPS
as the sensitive detector. Section~4 lists the outstanding issues which
need to be answered.

The timescale for a LC is still very uncertain, but a working schedule
has the machine turning on for physics in 2015. The ECAL will need
several years to assemble before this date and, together with
prototyping, it is assumed the TDR must be ready around 2009. This
implies there is around 5 years of R\&D to define the basic concept of
the ECAL.

\section{Current CALICE design}
The physics of the LC requires accurate energy reconstruction for
hadronic jets, which in terms of the ECAL means tracking single
MIP particles through the ECAL. This requires high granularity
of the sensitive detectors in the ECAL and it needs to be
effectively a ``tracking calorimeter''.

The sensitive detectors being considered by CALICE
are silicon wafers with simple
$1 \times 1$ cm$^2$ diode pads and it is assumed that each wafer will
have something like $16 \times 16$ pads.
These wafers will be mounted directly onto a PCB, which will made as
wide an one wafer and as
long as feasible, ideally the full length required of 1.6m.
The signals from the silicon wafers will be AC-coupled to an ASIC,
also mounted on the PCB, which will probably provide shaping, digitisation 
and some level of buffering. Each wafer will be read by one or two
ASICs. At the end of the PCB will be circuitry to collect the data
from all the wafers and send them out by fibre to off-detector receivers.

The main requirements for the diode/ASIC system
are set by the shower densities
expected. In the core of the highest energy showers at the LC, the
density will be up to
100 MIPs/mm$^2$. Hence, over a $1 \times 1$ cm$^2$ diode
pad, the maximum expected signal corresponds to around $10^4$ MIPs.
The diode pads also have to be sensitive to single MIPs, which defines
the dynamic range as $10^4 \sim 2^{14}$. For reasonable efficiency,
the signal/noise for a single MIP needs to be at least 4/1, which means
an equivalent digitisation range of 16 bits is needed. In practise, it
is assumed this will be handled with a multi-gain amplifier.

To perform well, the ECAL must also minimise the gap between the
tungsten converter sheets. This constrains the thickness of the
wafer/ASIC/PCB structure and makes cooling a significant problem.
Hence, low power is a major aim for all the electronics on the PCB.

A very large amount of silicon wafers is needed for the ECAL.
The overall
design has a cylinder of length around 3m, an average radius of around 1m
and has 40 layers of silicon wafers. There are also endcaps. In
total, the area of silicon needed is around 3000 m$^2$. With each wafer
being 256 cm$^2$, this is approximately $10^5$ wafers. Ease of
fabrication and assembly are major issues but the biggest consequence
is the cost. Even with an optimistic price in the future of \pounds 1/cm$^2$,
this is still \pounds 30M for the silicon alone. Clearly, there is a major
incentive to improve the performance so as to be able to reduce the
required number of layers.

The radiation levels are very low compared with LHC, being estimated to
be at least a factor of $10^3$ lower. Correspondingly, the event rate
for interesting events is also much lower at around 1Hz or less.

One final complication is the LC technology choice since the beam
collision timing depends on this.
Currently there
are two main designs being considered. The superconducting (``cold'')
design, originating from DESY, has a bunch structure of up to 4886
beam crossings within 0.86ms, i.e. a 176ns spacing,
with a 199ms gap of no beam until the next bunch train.
The ``warm'' design, from SLAC/Japan, has much shorter bunch trains
and crossing times, with 192 beam crossings within a 269ns train,
i.e. only a 1.4ns spacing. The gap between bunch trains is then
correspondingly shorter, 8.3ms.
To obtain the 4/1 or better MIP signal/noise
required, the ASIC shaping times being considered are of the order of
100ns. This would not allow differentiation between crossings within
the warm machine bunch train and so would
require all background to be
integrated over the whole train and hence be superimposed on top of
the single event of interest.
It is not clear if there is a need for cosmics; if taken between
bunch trains then this could substantially reduce the amount of time
available to read out the data from each train.
A decision on the LC technology is expected by the end of 2004.

\section{Possible use of MAPs}
The issues which it would be worth considering if MAPS can improve are
the following:
\begin{enumerate}
\item Reliability of bulk fabrication and assembly.
\item Improved mechanical thinness and/or cooling.
\item Better timing resolution for the warm machine design.
\item Better tracking and hence reduction of cost.
\end{enumerate}
Clearly, one possible approach would be to have MAPS with pixels the
same size as the diode pads and then integrate the ASIC circuitry
into the MAPS. This would potentially benefit items~1 and 2 above:
\begin{itemize}
\item 
The electrical
connection of the wafers to the PCB in the CALICE design is
quite simple; the pads and power connections
are assumed to be bump-bonded directly to the PCB. The ASIC will probably 
require a more complicated connection and so would be a source
of assembly effort (and associated failure probability).
\item
The major heat load in the CALICE design comes from the ASICs, which
are small compared with the wafers they read out and so the heat
sources are lumped. This leads to local heating and large temperature
differentials across the PCB. With the readout circuitry spread more
evenly over the surface of the MAPS, this would be reduced.
In addition, the structure surface could be smoother as the ASICs
stick out by several mm from the PCB surface and make any cooling
pipework complicated. MAPS might make the mechnical integration of
the cooling and electronics easier and would potentially make the
whole structure thinner.
\end{itemize}
It would be interesting to know if such an approach would be
straightforward; an obvious concern is that
the ASIC design on the MAPS
could need significant effort. In addition, unless
the MAPS can be made to be a similar size to the diode wafers, any
improvements in assembly may be outweighed by the larger number needed
to be assembled.

However, it would be interesting to consider a conceptually simpler approach
which helps in terms of items~1 and 2 (as above) but also might improve
items~3 and 4 also.
Here, no analogue readout of a pixel is made, but each is read out as
one bit, indicating whether a signal of a MIP (or above) was seen in
the pixel. Each pixel would need a discriminator and good enough
signal/noise to make the output clean. Clearly, resolution is lost if
the probability of more than one MIP within one pixel is high, so the
pixel size must be small enough to handle the highest density expected,
i.e. 100 MIPs/mm$^2$. This indicates a size of $100 \times 100$ $\mu$m$^2$
for the pixels is the maximum and possibly something half the size in
each dimension, i.e. $50 \times 50$ $\mu$m$^2$ would keep the probability
of multiple MIPs in a pixel low enough.
With pixels of this size, a MIP signal/noise
of at least 10/1 might be achieved, giving a clean signal if the
discriminator threshold can be kept stable. This size is assumed in the
following. Possible benefits are then:
\begin{itemize}
\item 
An obvious consequence of the much finer pixel size is that the
tracking is correspondingly more accurate. This could reduce the
number of layers of silicon needed and hence the cost.
\item 
Retrieving the signal quickly could be possible, allowing
the possibility for the warm machine that beam crossings within the
bunch train can be distinguished, or at least than the integration
time can be reduced from the whole train of 300ns to something
significantly smaller, like 30ns or better.
\end{itemize}
Another obvious consequence of the much finer pixel size is that the
data volume would increase enormously. Each 
$1 \times 1$ cm$^2$ diode pad, which was read out as a 16 bit value,
has been replaced by a $200 \times 200$ array of pixels, each reading out
as one bit, which results in 40 kbits. The CALICE design has 30M diode
pads, so the raw data volume per ADC sample is 60 MBytes. The MAPS would
result in a raw data volume per sample of 150 GBytes. Clearly due to the
low event rate, most channels have no signal so
a huge
amount of zero suppression will be possible in both cases.
However, the
increase in raw data rate might make this simplest scheme unfeasible.
One approach then might be to sum the bits
over $2^n \times 2^n$ arrays of pixels and report out a digital sum of
the pixels in that area. The data size for each sum would be $2n$ bits,
rather than a single bit,
but the number of sums would be $1/2^{2n}$ of the number of pixels,
giving a volume reduction of $2n/2^{2n}$. E.g. summing over a
$32 \times 32 = 2^5 \times 2^5$ pixel array would give an effective
pixel size of 1.6 mm, have a data size of 10 bits and would reduce
the effective number of pixels by 1024, meaning a raw data volume per
sample of around 1.5 GBytes. The effective pixel size is still much
smaller than the diode pads.
(The advantage of this over a $1.6 \times 1.6$ mm$^2$ pixel is that
no ADC is required and the signal/noise and readout speed will be better.)
Such digital addition is presumably trivial compared with a 16 bit ADC
design. One could even think of performing sums only when required;
in cases where there are very few hits, individual pixels could be
reported out, while in dense showers, sums could be sent instead
and even the value of $n$ used could be density dependent.

Another issue is the threshold required. This would have to be stable
and, given the number of channels, would need to be uniform at least
for all channels on a single MAPS. Having trim DACs on the MAPS would
be operationally complicated given the number of wafers in the system.

\section{Outstanding issues}
\begin{enumerate}
\item What are feasible data rates output from MAPS? What
sparsification algorithms would be possible to implement?
How much buffering (in terms of bytes of memory) could be feasibly
built into MAPS?
\item How stable and uniform would a discriminator level be? What would
the rate of random noise hits be with a good efficiency ($ge 99\%$)?
\item What is the highest possible speed of readout of
$50 \times 50$ $\mu$m$^2$ pixels? What time resolution on the
energy deposits could be obtained?
\item How much improvement in jet reconstruction could be obtained
using $50 \times 50$ $\mu$m$^2$ digital pixels compared with
$1 \times 1$ cm$^2$ analogue pads? How does this translate into
reducing the silicon area without degrading the performance?
\item What is the likely power consumption of such MAPS? Is it
feasible to only power up during bunch trains, given the
need for pedestal, gain and discriminator stability?
\item What would be the cost per cm$^2$ of MAPS?
\item What component size could be made with MAPS?
\end{enumerate}

\end{document}



The TESLA linear collider will send beams in bunch trains. Each
train will consist of 2820 bunches which collide every 337 ns.
The whole train therefore lasts 0.95 ms. This structure sets the
timing with which the calorimeter will operate. The prototype
should therefore work at these speeds.
\item The analog parts of the system should be able to be selectively
powered down between bunch trains.
\item The fast control and timing system will need to synchronise the
different parts of both the ECAL and HCAL.
It should be kept as simple as possible.
Therefore, a minimum number of time-critical
signals should be required.
\item The system should be very flexible with regard to its operation.
It is not yet known where the beamtest will be and what timing structure
the beam will have. Hence, as much as possible should be software
configurable.
\item To learn about the operation of a calorimeter at the linear collider,
the prototype readout should mimic, to some extent, the final TESLA system.
\end{itemize}


\subsection{Overview of readout system}

An overview of the readout system in shown in Fig.~\ref{fig:overview}.
%
\begin{figure}
\begin{center}
\includegraphics[width=16cm]{overview.eps}
\caption{Conceptual overview of the readout system.}
\label{fig:overview}
\end{center}
\end{figure}
%
The prototype calorimeter will consist of 30 layers of silicon diodes.
Each layer comprises a $3 \times 3$ array of silicon diode wafers, each
containing a $6 \times 6$ array of diodes. Each diode needs to be read out
and so corresponds to a channel. Hence, each wafer contains 36 channels,
each layer contains 324 channels and the whole prototype is 9720 channels.

The diode signals are sent (without amplification) over kapton cables to the
very front end (VFE) chips which provide amplification and shaping.
These will be mounted on VFE daughterboards (d/b).
The VFE chip is assumed to handle 16 channels.
They will provide a sample-and-hold analog
signal at the fundamental beam crossing period of 337 ns, which
corresponds to a frequency of 2.97 MHz (refered to
here as ``3 MHz''). There will be four gain values for each channel.
The ``best'' gain range will not
be selected in hardware but all four are to be digitised. The four gain
signals for each channel will be multiplexed onto the same analog line.
The digitisation will therefore occur at four times the fundamental
frequency, namely 11.87 MHz (``12 MHz''), which has a period of 84 ns.

The diodes, VFE chips and VFE daughterboards will be provide by non-UK
groups, while the UK will provide all the electronics downstream of the
VFE daugtherboards. The VFE daughterboard will need to have a well-defined
connector interface to the UK electronics. 

The ADC's to perform the signal digitisation will be on a front end card (FEC).
There will be one 10-bit ADC per diode channel and each FEC will process
108 channels corresponding to a row of three silicon wafers in one layer.
There are therefore 90 FEC's needed in total for the prototype, 3 per layer.
The FEC's will be built to process 112 ($16\times 7$) channels.
Note, as the ADC's will be sampling at 12 MHz, then it would also
be usable as a signal shape multiple-sampling system for a single gain
preamplifier, with no changes.

The FEC's will store and reduce the data for a 
whole bunch train before being read out.
The FEC is controlled and read out using fibre-optic links; the ``uplink''
for control and the ``downlink'' for readout. The links will also distribute the
system clock at four times the digitisation speed, namely 47.48 MHz
(``48 MHz'').

The data from each FEC will be read out after the train
over a fibre-optic link to a back end card (BEC), which is 
possibly in a VME crate.
There are 6 BEC's needed for the prototype. These also supply the VME
interface to the control fibre-optic for the FEC's.

The data rates are such that one fibre link can be used to control and
readout 16 FEC's
with the 16-way optical-to-electronic conversion being done with a simple
link interface card (LIC). There are 6 LIC's needed for the prototype.

The system only needs to stay synchronised during a bunch train. It will
be resynchronised at the start of each train and a check on the time
taken during the train will be done at the end of the train.

Three modes of operation can be considered. If the test beam is effectively
continuous on the timescale of 1 ms, then bunch train commands can be
generated at any time. Hence, in this untriggered mode,
the operation of the system can be
asynchronous with the beam. However, if the beam structure is such that
this would be highly inefficient, then the fast control and timing
(FCT) system will need to generate the bunch train command from a beam
signal and distribute this to both the ECAL and HCAL readout. This
could be an accelerator signal indicating the start of a spill. Finally,
with a very low beam rate, then continuous bunch train could be sent
without generating data readout unless there was a coincidence with a
trigger, such as a simple scintillator in front of the calorimeters.
Again, this logic would need to be done centrally, i.e. in the FCT system.
Note, the FCT system is not part of the UK electronics.



\subsection{Nomenclature}
\begin{itemize}
\item Very front end (VFE) chip
\item Very front end (VFE) daughterboard
\item Front end card (FEC)
\item Back end card (BEC)
\item Link interface card (LIC)
\item Fast control and timing (FCT)
\end{itemize}



\section{Front end card (FEC)}

\subsection{Overview}

The FEC handles 112 channels and must digitise, store and readout the
data for a whole bunch train. A conceptual layout of the card is shown
in Fig.~\ref{fig:fec}.
%
\begin{figure}
\begin{center}
\includegraphics[width=16cm]{fec.eps}
\caption{Conceptual layout of the FEC.}
\label{fig:fec}
\end{center}
\end{figure}
%
The FEC is controlled and clocked through the uplink connector. The
protocol on the uplink is intepreted by the FPGA, which also controls
the FEC functions.

Seven VFE chips are required for each VFE daughterboard and the kapton
cable carrying the diode signals connects directly to the daughterboard.
There are 112 ADC's followed by approximately
2 MBytes of the ``large'' memory to
store the raw data during the bunch train. A 0.2 MByte ``small'' memory
is used to store the data above threshold (after data reduction) before
it is read out via the fibre-optic downlink.

The FEC will be configured to control the whole sequence related to
a bunch train after being sent a StartTrain signal from the FCT system
via the uplink.

The power to the analog sections can be switched on and off; this means
the ADC's and the power to the VFE chips and diodes.


\subsection{Data taking cycle}
\label{sec:cycle}
The processes which occur during a data run are as follows:
\begin{itemize}
\item The FEC is configured; if the power control bit is set, then analog
power is turned on at this point.
\item A StartTrain signal is received on the uplink.
This initiates the following:
\begin{itemize}
\item If the power control bit is not set, the power is turned on
immediately.
\item After a configurable count in 48 MHz clock ticks, the ADC
sampling is started.
\item If the calibration control bit is set,
a calibration timing signal is sent after a
configurable count of 48 MHz clock ticks.
\item After a configurable number of 3 MHz samples, the ADC's are
stopped.
%\item The first handshake is sent on the downlink.
\item If the power control bit is not set, the power is turned off.
\item The data reduction occurs, copying data above threshold from
the large to the small memory.
\item A handshake is sent on the downlink.
\end{itemize}
\item If running in triggered mode, then without a valid trigger, the
previous item is repeated. This can occur before the end of the data
reduction, which is then aborted. Otherwise
either or both of the large and small memories are read out
by sending the relevant InitiateRead commands.
\item Multiple further StartTrain signals are received and the data
read out, as above.
\item If required, the analog power is turned off by reconfiguring the
FEC with the power control bit set off.
\end{itemize}

\subsection{FPGA tasks}
The FPGA will load from a flash RAM on board the FEC on power up.
The tasks it has to perform are:
\begin{itemize}
\item Decode uplink command protocol (see Sec.~\ref{sec:command}).
\item Store configuration data and handle its read and write. Handle the
values specified (see Sec.~\ref{sec:configuration}).
\item Control the train sequence; count 48 MHz ticks in a 32 counter,
issue the 3 MHz and 12 MHz clocks, and issue the calibration timing signal if
requested (see Sec.~\ref{sec:cycle}).
\item Control on-board memory reads and writes, during the train
sequence, during data reduction and when requested by an uplink command.
\item Perform the data reduction algorithm,
transfering data from the large to small memory (see Sec.~\ref{sec:reduction}).
\item Allow the flash RAM to be updated via the uplink and reboot
itself using the new firmware.
\end{itemize}


\subsection{Configuration data}
\label{sec:configuration}
The list of required configuration data for the FEC is as follows:
\begin{itemize}
\item Control
\begin{itemize}
\item Power control; one bit. Meaning: 0 = power on and off during train,
1 = power on always.
Total number of bits: 1.
\item Memory control; two bits. Meaning: one bit for each of the large
and small memories, 0 = overwrite the memory during the train sequence
(normal operation), 1 = don't overwrite the memory (test operation).
The latter mode allows the memories to be loaded with known data so
check data reduction and readback.
Total number of bits: 2.
\end{itemize}
\item Timing
\begin{itemize}
\item Number of 3 MHz samples in train; maximum is 4096 to fit into large
memory, while the nominal TESLA value is 2820.
Total number of bits: 12.
\item 48 MHz counter values to start the 3 and 12 MHz clocks sent to the VFE.
This allows the sample timing to be adjusted relative to the StartTrain
command time. Two values are needed to allow a phase adjustment between
the 3 MHz clock (which is used by the VFE to perform the
sample-and-hold) and the 12 MHz clock (which is used to put the signals from
the different gains onto the output line).
\item 48 MHz counter value to start the 12 MHz ADC sampling clock;
ADC samples occur every
four 48 MHz clock ticks after this. The lowest two bits therefore allow the
phase of the ADC with respect to the VFE 12 MHz clock to be adjusted to
avoid switching transients from the analog multiplexing.
Total number of bits: 32.
\end{itemize}
\item Data reduction
\begin{itemize}
\item Threshold values; 10-bit word per channel and gain.
Total number of bits: $10 \times 4 \times 112 = 4480$.
\item Data reduction handling: for each channel and each gain, two
bits. Meaning: 0 = never read out, 1 = read out if above threshold, 
2 = read out if other gains above threshold, 3 = always read out.
Total number of bits: $2 \times 4 \times 112 = 896$.
%\item Channel mask to select on neighbours above threshold; 16-bit mask.
%Total number of bits: 16.
\end{itemize}
\item Calibration
\begin{itemize}
\item Calibration control; one bit. Meaning: 0 = don't send a calibration
timing signal, 1 = send a calibration timing signal.
Total number of bits: 1.
\item Calibration DAC setting; 16-bit value.
Total number of bits: 16.
\item Calibration channel selection; one bit per channel.
Total number of bits: 112.
\item 48 MHz counter value to send calibration signal.
Total number of bits: 32.
\end{itemize}
\end{itemize}
The total configuration data volume will be around 1 kByte per FEC.


\subsection{Command and data protocol}
\label{sec:command}
The commands required on uplink are as follows.
All require a handshake back to signal their
completion. All
further commands are ignored until the handshake is sent with one
exception; a StartTrain will be acted upon at any time, with any other
task in progress being stopped immediately.
\begin{itemize}
\item StartTrain (FCT distributed); initiates a train sequence,
i.e. power on (if relevant
configure bit set), reset memories, start ADC sampling,
send calibration signal (if relevant configure bit set),
take pre-configured number of ADC samples,
do data reduction and power off (if relevant
configure bit set).
\item ConfigurationWrite; send all configuration data in one packet.
\item InitiateConfigurationRead;
read back all configuration data in one packet. This could also have
a board serial number appended for identification purposes.
\item LargeWrite; write data directly into the large memory.
\item InitiateLargeRead;
read out contents of large memory, i.e. unreduced data.
Must be non-disruptive to memory contents.
\item SmallWrite; write data directly into the small memory.
\item InitiateSmallRead;
read out contents of small memory, i.e. reduced data.
Must be non-disruptive to memory contents.
\item FlashWrite; write firmware code into flash memory on FEC and
reboot FPGA to use the new code.
\end{itemize}
The XxxWrites return a simple handshake to indicate completion.
The InitiateXxxReads
require the data to be returned with the handshake.
The StartTrain
returns a handshake
indicate the end of the data reduction, which should be synchronous for
all FEC's. The amount of data in each of the two memories should be
reported back in the handshake.

A suggested command protocol is as follows. The command has to start
with the LSB being set to identify the start of the command. The
command then needs to use the next 31 bits to define which command
this corresponds to. 
The following 32 bits define the number of
4-byte words following; for all but the XxxWrite and StartTrain commands,
this word will be zero. For the XxxWrites, the
data follow contiguously. For the StartTrain, there is one 4-byte
word, specifically the train number.

The handshake format should similarly start by echoing the same command
number as the initiating command and giving the amount of data
following, both as 32-bit words.
For the XxxWrites, the amount of data will be zero.
For the InitiateXxxReads, the data follow contiguously.
For the StartTrain, three words of data should be sent; the train
number, the number of words in the large memory and the number of words
in the small memory.


\subsection{VFE chip daughterboard interface}
The signals which need to be exchanged across the interface to the VFE
chip daughterboard are as follows:
\begin{itemize}
\item Analog signals for each channel. Differential; voltage levels, bipolar?
Two pins per channel. 
Total number of pins = 224 (32 per VFE chip).
\item Calibration selection for each channel. Digital LVDS. Fixed during
train. One pin per channel. 
Total number of pins = 112 (16 per VFE chip).
\item Power for daughterboard and diodes. Need $+5$ V, few hundred (?)
V for diodes, and others? Also ground.
Total number of pins $\sim$ 10.
\item The 3 MHz sample-and-hold clock. Differential digital, voltage levels?
Total number of pins = 2.
\item The 12 MHz gain-switching clock. Differential digital, voltage levels?
Total number of pins = 2.
\item Calibration DAC analog signal. Differential, constant during
train, voltage levels?
Total number of pins = 2.
\item Calibration pulse time, on 48 MHz clock. Differential digital,
voltage levels?
Total number of pins = 2.
\end{itemize}
The total interface requires around 350 pins (and needs further definition).
This could be subdivided mechanically into several (seven?)
daughterboards per FEC, with 64 pins per daughterboard connector.

\subsection{Data reduction}
\label{sec:reduction}
The data reduction is assumed to be mainly reformatting and copying
data from the large memory to the small memory when the sample is above
a configurable threshold. The data in the large memory are simply the
10-bit ADC values, stored in time order.
Each sample above threshold could be
stored in the small memory as a 32-bit word, consisting of the channel
number (0-111, requiring 7 bits), gain number (0-3, requiring 2 bits),
sample number (0-4095, requiring 12 bits) and ADC value (requiring 10 bits).

Configurable options allow any channel to be selectively never or
always written into the small memory. Otherwise it is assumed it would
be written when the sample is above the threshold for that particular
gain, or (more usually) when any gain for the channel and sample is
above threshold.

One option which should be considered is keeping samples, which are
below threshold, either side in time of ones which are above threshold. This
would allow the pedestal to be determined and also check for noise
activity close to the signal which might distort the shape of the
signal pulse. Ideally, the number of time samples away from the one above
threshold which results in then still being retained would be configurable.


The data reduction is assumed to result in at least an order of
magnitude reduction, which sets the small memory overall size.
If the FPGA dealt with one channel at a time and the large memory is
also read at 12 MHz, then the data rate would be 10 bits at 12 MHz,
i.e., 114 Mbits/s
or 14 MBytes/s. This would then take around 150 ms to do the
data reduction, so parallelising this process in the FPGA would be very
desirable. To access the data,
each channel needs at least 10 pins input and 31 output to the FPGA.
Assuming four channels can be handled in parallel (which will
take around 160 pins of the FPGA), then the time taken is around 38 ms.
Running the memories at 24 MHz would clearly halve this time.


\subsection{Mechanical FEC crate structure}
There are a total of 90 FEC's in the system and these need to be
mechanically held and cooled. Is a custom mechnical structure required?
Power will come via the LIC cable so no power distribution is needed.

The physical board width of the FEC's should be kept so they do not
exceed the silicon wafer size, i.e. 7 cm. Given the numbers of ADC and
memory components, it is likely that the FEC length will be at least 30 cm.



\subsection{Data volumes and rates}
For 10 bits per ADC sample and 112 channels, then each 12 MHz sample
results in 1120 bits of data input to the large memories. This is a
total rate of 1600 MBytes/s.

The maximum
number of 3 MHz samples which can be taken in one burst must be at least
the TESLA value of 2820, so take 4096 as the maximum. Each has four
gain range samples per 3 MHz clock tick, so the depth of the large memories
must be 16k samples deep. This fixes the specifications for the large memory,
i.e., 1120 bits wide, 16k deep, filling at 12 MHz. This is 17.5 Mbits, or
2.2 MBytes, in total. Note that if there would be a significant cost
saving for memories of half the depth, then 2048 samples could be taken
as the maximum.
The readout rate on the downlink will be one bit per FEC at 48 MHz, i.e.,
46 Mbits/s or 5.7 Mbytes/s. Hence, to read the full
large memory will take around 380 ms.


For small memory reads; assume an average shower (e.g., for a 10 GeV
electron) has around 150 channels above threshold.
Assuming the signal
is above threshold for three time samples and that all four gains are
always read out if any is above threshold, then the total number
is around 1800 samples of 32 bits each, corresponding to a
volume of data
for each shower of around 60 kbits or 7 kBytes.
At high beam rates, there could be hundreds of showers per train
without suffering from significant overlap. In addition, there will be
random noise for all channels. Assuming a 0.1\% noise rate above
threshold, then each 3 MHz sample will have around 10 noise channels.
A full train of around 1 ms will contain around 30000 such channels
with each resulting in all four gains being read, this is 120k samples.
For one hundred showers per train, the total count is
around 300k samples, corresponding to a data volume of 1.1 MBytes.

This is for the full
system; these channels are spread over all 90 FEC's.
Each FEC is read out with a 5.7 MBytes/s rate, so the total
readout speed will be limited by the highest occupancy and fluctuations
around the mean may be important. For the average, which is a data
volume of 13 kBytes, which takes around 2.2 ms to read out
over the downlink.


\subsection{Calibration}
Each FEC has a 16-bit DAC controlled by the FPGA and configured before
the bunch train. A constant (within a bunch train) analog
signal is sent and switched as close as possible to the point at which
it is used so as to minimise transmission line effects, etc. The time
at which the signal is switched is controlled by a digital
calibration timing signal on the 48 MHz clock so as to allow adjustment of
the signal peaking time within the analog sample-and-hold 3 MHz time.
The timing signal will be able to be switched to any combination of the 112
channels within the VFE daughterboard so 112 calibration channel selection
bits are required to determine which channels are to be calibrated.


\subsection{Power, grounding and shielding}

The power for the FEC's (and hence also the VFE chip daughterboards and
diodes) will come through the uplink connector from the LIC
which also supplies the 48 MHz clock and uplink data.

Care is needed to not inject noise from the FEC digital components into
the analog VFE daughterboard. The digital components may need to be mounted
on the opposite side of the FEC from the VFE daughterboard mounts, with
good ground planes between. In addition, plastic mechanical daughterboard
mounts and screws should be used.

An estimate of the FEC power consumption is 24W for the analog parts
of the board (which can be powered down when not needed)
and 1W for the digital part (which is always on). The average power
consumption then depends on the duty cycle for the analog power, which
is needed during the train sequence. At the final TESLA collider, this
is 1 ms in 200 ms, or 0.5\%. The duty factor at the beam test will
depend heavily on the mode of operation. For untriggered mode, the
time between trains is unlikely to be less than 50 ms, which implies
a duty factor of 2\% and hence an average power of 1.5W per FEC, or
130W for all FEC's. However, in triggered mode with a low rate, then
the system may be continuously running through the train sequence
without being read out, with a duty factor which might be around 50\%.
Safety considerations would imply the cooling for the system should be
able to deal with the analog power being on continuously, which 
corresponds to a 2.2kW load for all the FEC's.


\subsection{Testability}
One feature which would be useful for testing is the ability to
load known data into the large or small memory. This would test
both the memory write and read back are functional as well as allowing
tests of the data reduction using the data in the large memory. This
latter requires the memory to not be overwritten during a train
sequence; hence this needs to be made a (configurable) option.

It would also be sensible to make a ``fake'' VFE daughterboard which
generates known signals on the analog lines to allow the FEC's to be
tested when VFE boards are not available. This could use the
calibration select lines to give a significant amount of selectivity
to the faked signals.

Other desirable features for the prototype and debugging phase:
\begin{itemize}
\item Status LED's to diagnose errors; FPGA loaded, link locked, analog
power on, train occuring, data on downlink, data on uplink, downlink laser
power.
\item Scope probe hooks on the 48 MHz clock, downlink and uplink.
\item Unique and data-readable (within configuration data) serial number
for each FEC, to detect cabling errors.
\item Ability to generate a StartTrain signal on board the FEC ``by hand'',
if a BEC and/or LIC are not available. This would also require the FPGA to
boot up into with usable default configuration data.
\end{itemize}


\subsection{Implementation, costs, timescale, effort}
A possible implementation would use 10-bit dual ADC packages, requiring a
total of 56 components per FEC, each costed at around \pounds 5. These
can multiplex their output to give output at 24 MHz.

The large memory
would then obviously need to run at 24 MHz and be 32k samples deep,
but would only need a physical input width of 560 bits. With 32-bit
wide, 32k deep, 24 MHz packages and packing 3 channels (i.e. 30 bits)
per component, then 19 components are needed. These may be around
\pounds 20 each. The total large memory is then 2.4 MBytes.

The small memory should also run at 24 MHz to speed up data reduction.
Assuming four channels are handled at once, then four 32-bit words need
to be written at one time. Using the same memories as for the large
memory, then four components are needed, giving a total of 0.5 MBytes.

A very rough costing of the FEC is as follows:
\begin{itemize}
\item ADC's; \pounds 5 per component $\times$ 56 components = \pounds 280.
\item FPGA; \pounds 100 per FPGA = \pounds 100.
\item Large memory; \pounds 20 per component $\times$ 19 components
= \pounds 380.
\item Small memory; \pounds 20 per component $\times$ 4 components
= \pounds 80.
\item PCB manufacturing, mounting and minor components; 
\pounds 400 per board = \pounds 400.
\end{itemize}
The total cost per FEC is therefore approximately \pounds 1140.

\section{Link interface card (LIC)}

\subsection{Overview}
A conceptual layout of the LIC is shown in Fig.~\ref{fig:LIC}.
%
\begin{figure}
\begin{center}
\includegraphics[width=16cm]{LIC.eps}
\caption{Conceptual layout of the LIC}
\label{fig:LIC}
\end{center}
\end{figure}
%
The LIC takes one fibre-optic uplink and converts the data into 16
parallel 48 MHz single bit
data streams. These data are sent to 15 FEC's (rather than 16 as this
matches the physical FEC distribution)
along with a copy of the 48 MHz
clock on a short cable.
In addition, the power for the FEC's, VFE daughterboards and diodes is
distributed through this board on the same cables.
The downlink data from the FEC is returned on the same cable as a 48
MHz single bit stream.

It is assumed that the skew on the returning downlink signals from
the different FEC's can be controlled well enough that no explicit
resynchronisation is required on the LIC and hence no return clock
is needed. This may require some care in picking the component on the
FEC which handles the clock and link data i/o; 48 MHz has a 21 ns
period so skews of 5-10 ns might not be tolerable. 


\subsection{Cables}
The cables therefore carry
48 MHz clock and one signal bit per link direction per FEC, totalling
six pins. Differential digital; what voltage?

The power and ground will require further pins. 

\subsection{Power, grounding and shielding}
The grounding should go through the LIC in a tree topology to the FEC's,
VFE daughterboards and diodes. This requires the mechanical structures
holding these components to be electrically ioslated from the grounds.

Fuses on the power are needed on the LIC's.

\subsection{Options}
The number of LIC's is set by the number of FEC's (i.e. 90) divided
by the width of the words on the fibre interface (assumed 16). If
fibre sets with a 32-bit wide input could be used, then the number
of LIC's required would be reduced to only three, with a subsequent
saving in numbers (and complexity) of BEC's.

If such links are not available, at least which run 48 MHz, then an
alternative might be to direct every other 16-bit word on the link
to a different set of FEC's. Specifically using 1 of the 16 bits
to flag which set the data are for, then the remaining 15 bits could
alternatively service 30 FEC's. This matches precisely the number
of FEC's which would be required per LIC. Clearly, this implementation
would reduce the overall data rate per FEC by a factor of two, which
might be acceptable. Otherwise, it might be possible to run the links
at twice the fundamental rate, i.e. 96 MHz. The main downside of such
schemes is that this would require some intelligence (i.e. an FPGA)
on the LIC, which is otherwise a very simple board.

\subsection{Costs, timescale, effort}
A very rough costing of the LIC is as follows:
\begin{itemize}
\item Uplink; \pounds 100 per fibre-optic RX = \pounds 100.
\item Downlink; \pounds 100 per fibre-optic TX = \pounds 100.
\item PCB manufacturing, mounting, minor components and FEC cables; 
\pounds 200 per board = \pounds 200.
\end{itemize}
The total cost per LIC is therefore approximately \pounds 400.


\section{Back end card (BEC)}

\subsection{Overview}
The specification of the BEC is much less well-defined. Its principle
requirement is to send configuration and control data on the uplink
and receive event data on the downlink. An obvious implementation would
be to use VME but this and other options are considered below.

The BEC's need to handle six uplink/downlink fibre pairs to the six
LIC's. Conceptually, there are two options. The first is to
have cards which send
uplink data and receive downlink data for the same LIC.
The alternative is one card which
only sends uplink data and another card (or cards)
which only receive downlink data.
The advantage of the first option is that only one card needs to be
designed and prototyping can be done with only one such card. The
second option makes clock and control synchronisation easier as all
uplink signals emerge from a single board; in fact, the uplink card
becomes something like a FCT distribution board.

For each option, the number of fibres each card would handle also needs
to be decided. Clearly a single board could
physically have six fibre pairs mounted on it and this might, in fact,
combine the best of both options. However, the resulting card would
be highly complex and have a lot of components, making spares more
expensive and failure more likely. Other feasible combinations would
be having one, two or three fibre pairs per card for the first option,
resulting in six, three or two cards in total, respectively. For the
second option, the main gain is lost if all uplink fibres do not
originate from one board, although the downlinks could be collected by
two or three cards, making a total of three or four in the system,
respectively.

To be concrete, the following fleshes out the option with one BEC
uplink driver card and three BEC downlink receiver cards, each taking
two fibres, giving a total of four cards. These are assumed to be 6U
VME cards. An overview is shown in Fig.~\ref{fig:bec}.
%
\begin{figure}
\begin{center}
\includegraphics[width=16cm]{bec.eps}
\caption{Conceptual layout of the BEC.}
\label{fig:bec}
\end{center}
\end{figure}
%
The system 48 MHz clock and StartTrain commands are sent to the BEC
driver via the FCT system. These are sent synchronously on
all six fibre-optic uplink TX's to
the LIC's and then on to the FEC's. In addition, the
other uplink commands can be
loaded into RAM through the VME interface and sent down the same path.
For prototyping, the BEC driver has a 48 MHz internal clock which
can be used to run the BEC stand-alone; the StartTrain commands can
then be ``faked'' via the VME interface.

The BEC receiver takes two downlink fibres from LIC's.
The data packets from the downlinks are all written directly to memory
and then read out via VME at a later time. The fibre link RX is
continuously monitored for non-zero data indicating the start of a
data packet.

Synchronisation check of FEC's need to happen on BEC; first need to
synchronise BEC's using the FCT system. Do this on receipt of the
StartTrain command.


\subsection{FPGA tasks}

HOW DO TO SYNCH CHECK???


The tasks the BEC driver FPGA needs to perform are as follows:
\begin{itemize}
\item Interface to VME.
\item Select whether to receive 48 MHz clock from FCT or 
internal clock and send on uplink.
\item Interpret FCT StartTrain command, distribute to downlink FPGA's,
convert to uplink protocol and send on uplink.
\item Control RAM memory reads to the uplink and writes from VME.
\end{itemize}

The tasks the BEC receiver FPGA needs to perform is as follows:
\begin{itemize}
\item Interface to VME.
\item Keep 32-bit 48 MHz counter to all FEC synchronisation check.
This counter is zeroed by a StartTrain command.
\item Detect and decode handshakes on the downlinks and write the data,
together with the counter timestamp, into the large memory.
\item Control large memory reads from VME and writes from downlink.
\end{itemize}


\subsection{Power, grounding and shielding}
The power for the BEC's is from the VME crate which houses them. Since
they only connect optically to the rest of the system, grounding and
shielding should be straightforward.

What is the power consumption of the BEC's?


\subsection{Data rates}
Each downlink has a rate of 5.7 MBytes per second and will take around
380 ms to read out the full large memory, corresponding to 2.2 MBytes
per FEC.
Each BEC receiver handles 30 FEC's and therefore 
therefore stores up to 66 Mbytes of data. The total amount of
data for all six boards is 200 MBytes.
With a VME backplane speed of around 25 Mbytes per sec, this will
take a total of around 8 seconds to fully read out. The subsequent
write to disk should be faster than this and can happen in parallel
with the next bunch train sequence.
The VME read is
clearly the bottleneck for large memory readout and will limit the
bunch train repetition rate to around 0.1 Hz. 

FIX UP!!!

When running standardly, only data above threshold in the small memory 
are read out. This takes around 16 $\mu$s per shower to read on the downlinks
and gives a total on average of around 15 kBytes per shower. This
will take around 0.6 ms per shower to read out over the VME backplane.

The bottleneck
for small memory readout is therefore the fixed FEC data reduction speed
of around 20 ms for low numbers of showers per bunch train, but increases
linearly with the average number of showers per train $N$ as $0.6N$ ms.
To be conservative, assume with overheads (VME polling, disk access, etc.)
that the fixed component is
50 ms per bunch train and the linear component is $1.0N$ ms.
For a 1 ms train length and a beam particle
rate $R$, then the average number of showers per train would be given by
$N = 10^{-3}R$. To avoid pile-up effects of overlapping showers, the
beam rate needs to be significantly less than the sample speed of 3 MHz.
Ignoring any pile-up effects, the average 
rate $R^\prime$ of showers which can be read out is then
\begin{equation}
R^\prime = \frac{N}{\left(50 + N \right)\times 10^{-3}}
= \frac{R}{50 + 10^{-3}R}
\end{equation}
for $R$ and $R^\prime$ in Hz. This
is shown in Fig.~\ref{fig:rate} for input beam rates up to 1 MHz.
%
\begin{figure}
\begin{center}
\includegraphics[width=10cm]{rate.eps}
\caption{Readout shower rate as a function of beam particle rate.}
\label{fig:rate}
\end{center}
\end{figure}
%
For low rates, the readout speed goes proportionately to the beam rate
$R^\prime \approx 0.02R$, while for high rates, the readout rate limits
at around $R^\prime \approx 1$ kHz.

\subsection{Open questions}
\begin{itemize}
\item Does the BEC need to monitor the laser power on the uplink TX?
\end{itemize}


\subsection{Costs, timescale, effort}
A very rough costing of the BEC is as follows:
\begin{itemize}
\item FPGA; \pounds 100 per FPGA $\times$ 17 FPGA's = \pounds 1700.
\item Uplink; \pounds 100 per fibre-optic TX = \pounds 100.
\item Downlinks; \pounds 100 per fibre-optic RX $\times$ 16 RX's = \pounds 1600.
\item Memory; \pounds 20 per MByte $\times$ 40 MBytes = \pounds 800.
\item PCB manufacturing, mounting and minor components; 
\pounds 1500 per board = \pounds 1500.
\end{itemize}
The total cost per BEC is therefore approximately \pounds 6200.


\subsection{Options for BEC's}
Four different options for the physical realisation of the BEC's have
been suggested:
\begin{itemize}
\item VME; As above.
\item CompacPCI; This would be conceptually similar to a VME
implementation except in a PCI crate, which can contain the PC and disk
directly. The readout bus speed may be around 3 times faster (85 MBytes/s)
and the crate could be considerably cheaper.
\item PCI card; Here, the BEC's are implemented directly onto PCI cards
which attach to the PCI bus in a readout PC. This saves a substantial
amount on crate costs but the card area available in this scheme is highly
restricted. This might restrict the memory it is possible to contain
and so require a more complex readout. Either serialised LIC readout
(which lengthens the time to read out the data) or unbuffered readout
directly to the PCI bus have been suggested.
\item Ethernet interface; Getting the BEC's to send their data directly
onto a network using IP would again bypass the need for any crate.
However, this would lead to more complex syncronisation problems in the
event build.
\end{itemize}


\section{Fast control and timing (FCT)}

\subsection{Requirements}
The FCT system needs to distribute the 48 MHz clock and StartTrain command
to all BEC's (as well as the HCAL).
The StartTrain should include a bunch train number so the
event build can identify the different pieces of data to put together.

In addition, any triggering or beam timing signals needed for
synchronous operation need to also be handled and distributed by the
FCT.


\section{Total costs, timescale, effort}

\subsection{Costs}
We assume that each of the three boards needs 2 prototypes.
For production, we need 8 spare FEC's in addition to the 90 for the
system and 2 spare BEC's and LIC's in addition to the 6 for the system.
We also assume the FEC's are housed in custom (?) crates with 15 FEC's
per crate (i.e. 5 layers, corresponding to one LIC) and hence 6 crates.
These supply cooling but not power.

Therefore, a very rough costing of the project is as follows:
\begin{itemize}
\item FEC's; \pounds 1140 per FEC $\times$ 100 FEC's = \pounds 114k.
\item BEC's; \pounds 6200 per BEC $\times$ 10 BEC's = \pounds 62k.???
\item LIC's; \pounds 400 per LIC $\times$ 10 LIC's = \pounds 4k.
\item FEC crates; \pounds 2k per crate $\times$ 6 crates = \pounds 12k.
\item BEC crate, 6U VME; \pounds 5k per crate = \pounds 5k.
\item Readout PC, PCI interface, disk; \pounds 10k total = \pounds 10k.
\item Power supplies; \pounds 10k total = \pounds 10k.
\end{itemize}
The total cost of the project is therefore approximately \pounds 205k.

\subsection{Timescale}
The system needs to be ready for a beamtest in early 2004. It therefore
needs to be completed by at latest end 2003. Production and testing
of the full system is estimated to require around 6 months, which
sets the end of the prototype phase to be mid 2003.

Hence, prototyping will take all of FY02/03 and part of FY03/04, while
the rest of FY03/04 will be taken up with production. The prototype
costs will be for the infrastructure (VME crate, PC etc. and power supplies)
and two of each of the boards, giving a total of \pounds 55k, spread
over the two FY's. The remaining production cost of \pounds 150k will
be all in FY03/04.

\subsection{Effort}
Each of the FEC and BEC will take a full engineer for the time up until
the system is complete, i.e. end 2003. It might be possible to have one
RAL TD system engineer to take an overview of the project and have two
half-time engineers from Universities doing part of each of the board
designs.
The LIC is much simpler and can
be produced in the Universities.
This gives a total RAL TD requirement of 1.0SY for FY02/03 and 0.7SY
for FY03/04, making a total of 1.7SY, with cost equivalent roughly
\pounds 170k.
