sinara-hw
/
datasheets
7
1
Fork 3
Code Issues 8 Pull Requests Projects Releases Wiki Activity

1124 Carrier Kasli 2.0 update

This commit is contained in:
architeuthidae 2024-10-22 21:38:45 +02:00
parent f3858552b6
commit 86d8e871dd
1 changed files with 96 additions and 79 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

175
1124.tex
View File

@ -23,8 +23,8 @@
\title{1124 Carrier Kasli 2.0}
\author{M-Labs Limited}
\date{January 2022}
\revision{Revision 1}
\date{October 2024}
\revision{Revision 2}
\companylogo{\includegraphics[height=0.73in]{artiq_sinara.pdf}}
\begin{document}
@ -33,33 +33,27 @@
\section{Features}
\begin{itemize}
\item{4 SFP 6Gb/s slots for Ethernet or DRTIO.}
\item{12 EEM Connectors.}
\item{4 MMCX clock outputs.}
\item{FPGA core device.}
\item{4 SFP 6Gb/s slots for Ethernet and DRTIO}
\item{12 EEM ports for daughtercards}
\item{4 MMCX clock outputs}
\item{Xilinx Artix-7 FPGA core}
\item{DDR3 SDRAM}
\end{itemize}
\section{Applications}
\begin{itemize}
\item{Runs ARTIQ kernels.}
\item{Control the EEMs.}
\item{Communication with the host.}
\item{Run ARTIQ kernels}
\item{Communicate with the host}
\item{Control other Sinara EEM cards}
\end{itemize}
\section{General Description}
The 1124 Carrier Kasli 2.0 card is a 8hp EEM module.
It controls the EEMs by running ARTIQ kernels sent from the host.
The 1124 Kasli 2.0 Carrier card is an 8hp EEM module, designed to run ARTIQ kernels sent from a host machine over the network. It supports up to 12 EEM connections to other EEM cards in the ARTIQ-Sinara family and up to three SFP connections to satellite carriers.
It supports 12 EEM connections to other EEM cards in the ARTIQ Sinara family.
Real-time control of the EEMs are implemented using the RTIO system.
1ns temporal resolution can be achieved for TTL events.
Real-time control of EEM daughtercards is implemented using the ARTIQ RTIO system. 1ns temporal resolution can be achieved for TTL events.
4 SFP 6Gb/s slots are supported for Ethernet or DRTIO.
Communication with the host is supported by the Ethernet, while the
Distributed Real Time Input/Output (DRTIO) system allows inclusion of
additional core devices (e.g. Kasli 2.0) as DRTIO satellites,
indirectly controlled by the DRTIO Master.
4 SFP Gb/s slots are provided for Ethernet or DRTIO communications. Communication with a host machine is supported over Ethernet, while the Distributed Real-Time Input/Output (DRTIO) system allows for the use of additional core devices (e.g. Kasli 2.0, Kasli-SoC) as satellite cards, capable of running subkernels or distributing commands from the DRTIO master.
% Switch to next column
\vfill\break
@ -186,19 +180,30 @@ indirectly controlled by the DRTIO Master.
\begin{figure}[hbt!]
\centering
\includegraphics[height=2in]{Kasli_FP.pdf}
\includegraphics[height=2in]{photo1124.jpg}
\caption{Kasli 2.0 Card photo}
\caption{Kasli 2.0 card}
\end{figure}
\begin{figure}[hbt!]
\centering
\includegraphics[angle=90,height=0.9in]{Kasli_FP.pdf}
\caption{Kasli 2.0 front panel}
\end{figure}
% For wide tables, a single column layout is better. It can be switched
% page-by-page.
\onecolumn
\section{Source}
Kasli 2.0, like all the Sinara hardware family, is open-source hardware, and design files (schematics, PCB layouts, BOMs) can be found in detail at the repository \url{https://github.com/sinara-hw/Kasli}.
\section{Electrical Specifications}
External clock parameters are derived based on the internal termination specified in UG471\footnote{\label{ug471}https://docs.xilinx.com/v/u/en-US/ug471\_7Series\_SelectIO},
and the voltage range specified in DS181\footnote{\label{ds181}https://docs.xilinx.com/v/u/en-US/ds181\_Artix\_7\_Data\_Sheet}.
The figure had accounted for the insertion loss of the RF transformer (TC2-1TX+\footnote{\label{rf_trans}https://www.minicircuits.com/pdfs/TC2-1TX+.pdf}).
External clock parameters are derived based on the internal termination specified in
UG471\footnote{\label{ug471}\url{https://docs.xilinx.com/v/u/en-US/ug471\_7Series\_SelectIO}}
and the voltage range specified in
DS181\footnote{\label{ds181}\url{https://docs.xilinx.com/v/u/en-US/ds181\_Artix\_7\_Data\_Sheet}}. These figures account for the insertion loss of the RF transformer (TC2-1TX+\footnote{\label{rf_trans}\url{https://www.minicircuits.com/pdfs/TC2-1TX+.pdf}}).
\begin{table}[h]
\centering
\begin{threeparttable}
@ -212,7 +217,7 @@ The figure had accounted for the insertion loss of the RF transformer (TC2-1TX+\
\hspace{3mm} Input frequency & & 125 & & MHz & Si5324 synthesizer bypassed \\
\cline{2-6}
% 100R termination & 100/350/600 mV differential input after the transformer.
& \multicolumn{4}{c|}{10/100/125 MHz} & RTIO clock synthesized from input \\
& \multicolumn{3} {c|}{10/100/125} & MHz & RTIO clock synthesized from input \\
\cline{2-6}
\hspace{3mm} Power & -9 & 1.5 & 5.5 & dBm & \\
\hline
@ -222,102 +227,114 @@ The figure had accounted for the insertion loss of the RF transformer (TC2-1TX+\
\end{threeparttable}
\end{table}
\section{Distributed RTIO (DRTIO)}
DRTIO is a time and data transfer system that allows ARTIQ RTIO channels to be distributed among several satellite devices synchronized and controlled by a central core device.
Multiple core devices (e.g. Kasli 2.0) can be interconnected through DRTIO. All core devices in the DRTIO system are classified as 1 of the 2 roles:
Power is to be supplied through the barrel connector in the front panel, size 5.5 mm OD, 2.5 mm ID, and is passed on to daughtercards through the EEM connections. Locking barrel connectors are supported.
\section{FPGA}
Kasli 2.0 features an XC7A100T-2FGG484I Xilink Artix-7 FPGA to facilitate reconfigurable high-speed real-time control of inputs and outputs. Most commonly, this FPGA is flashed with binaries compiled from the ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) control system, which equips the carrier board with specialized gateware for handling other Sinara EEMs and an on-FPGA CPU for running ARTIQ experiment \mbox{kernels}.
ARTIQ is open-source and can be found in the repository \url{https://github.com/m-labs/artiq}. Orders of Sinara hardware are normally accompanied with precompiled binaries. Long-term support for ARTIQ systems can also be purchased.
A micro-USB located at the back of the Kasli 2.0 board is equipped for JTAG, I2C, and UART serial output.
\subsection{Note on distributed RTIO (DRTIO)}
DRTIO is the time and data transfer system that allows ARTIQ RTIO channels to be distributed among several core device carrier boards, synchronized and controlled by a central core device. The system itself is more fully described in the ARTIQ documentation\footnote{\label{manual-drtio}\url{https://m-labs.hk/artiq/manual/drtio.html}}. With ARTIQ firmware/gateware, supported core devices, including Kasli 2.0, can take one of three roles:
\begin{enumerate}
\item DRTIO Master \\
The DRTIO master is unique in a DRTIO system. It controls the DRTIO satellites(s) and local RTIO channels.
\item DRTIO Satellite \\
The rest of the core devices are DRTIO satellites. DRTIO satellites need an upstream connection to one other core devices (master or satellite).
It may provide downstream conenction to other DRTIO satellties.
\item \textbf{Master} \\
The DRTIO master is unique in a DRTIO system. It requires a direct network connection to the host machine. It may make downstream connections to satellites. It controls its own local RTIO channels and the downstream DRTIO satellite(s).
\item \textbf{Satellite} \\
Any other core devices in a DRTIO system are DRTIO satellites. They require an upstream connection to one other core device, master or satellite, through which communications will ultimately be chained to the master. They may make further downstream connections to other satellites. They may control RTIO channels through subkernels or simply pass on communications from the master.
\end{enumerate}
When run in a non-distributed ARTIQ configuration, with a single central core device but no satellites, that core device is instead known as \textbf{standalone.}
\section{Network Interface}
Communication between the host and the core device(s) is implemented using small form-factor pluggable (SFP) interfaces.
Approprate SFP transceivers must be plugged inside the corresponding SFP cages to enable communication between core devices.
\section{Communication interfaces}
Communication between devices is implemented using 10000Base-X small form-factor pluggable (SFP) interfaces. Four are available on the Kasli 2.0. Appropriate SFP transceivers must be plugged inside the corresponding SFP cages. Each SFP connector possesses an indicator LED.
\subsection{Upstream connection}
A Kasli 2.0 board must acquire an upstream connection through the \texttt{SFP0} slot.
\subsection{Upstream Connection}
A core device (e.g. Kasli 2.0) must acquire an upstream network connection through the \texttt{SFP0} slot.
\begin{itemize}
\item Standalone/DRTIO master \\
An Ethernet capable SFP transceiver must be inserted to the \texttt{SFP0} slot.
Typically, a RJ45 SFP module is inserted to the slot with an Ethernet cable with network connection attached to the module.
\item DRTIO Satellite \\
The \texttt{SFP0} port of DRTIO satellite should be connected to an appropriate SFP slot of the upstream core device (DRTIO master or satellite) with cable connection with SFP transceivers.
\item \textbf{Standalone/Master} \\
An Ethernet-capable SFP transceiver should be inserted into the \texttt{SFP0} slot. Typically, a 10000Base-X RJ45 SFP module is used, with an network-connected Ethernet cable attached to the module.
\item \textbf{Satellite} \\
The \texttt{SFP0} port should be connected to one of the free SFP slots on an upstream core device, using a cable connection with SFP transceivers.
\end{itemize}
\subsection{Downstream Connection}
The 1124 Carrier Kasli 2.0 supports up to 3 DRTIO satellite connections per device.
DRTIO satellites can be connected using any of the 3 downstream SFP ports (i.e. \texttt{SFP1}, \texttt{SFP2}, \texttt{SFP3}) through cable connections with SFP transceivers.
Kasli 2.0 supports up to 3 DRTIO satellite connections per device. Any of the 3 downstream SFP ports (i.e. \texttt{SFP1}, \texttt{SFP2}, \texttt{SFP3}) may be used.
\section{Clock Routing}
\subsection{DRTIO Master/Standalone}
The RTIO clock is typically synthesized by the Si5324 clock multiplier, and distributed by the ADCLK948 clock fanout buffer to both the FPGA and the MMCX connectors.
An external reference can be supplied to synthesize the clock, which is supplied to the SMA connector. It is then buffered in the FPGA and sent to the Si5324 for clock synthesis.
Kasli 2.0 supports a set of clock systhesizing options for the (D)RTIO system:
\subsection{Standalone/Master}
The RTIO clock is typically synthesized by the Si5324 clock multiplier and distributed by the ADCLK948 clock fanout buffer to both the FPGA and the MMCX connectors. Alternatively, an external reference can be supplied through the front panel SMA connector. It is then buffered in the PFGA and sent to the Si5324 for clock synthesis. Kasli 2.0 supports a set of RTIO clock options:
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|}
\hline
RTIO frequency & Configuration & Clock generation \\ \hline
100 MHz & \texttt{int\char`_100} & internal crystal oscillator using PLL, 100 MHz output \\ \hline
\multirow{4}{*}{125 MHz} & \texttt{int\char`_125} & internal crystal oscillator using PLL, 125 MHz output (default) \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_10to125} & external 10 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_100to125} & external 100 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
\multirow{4}{*}{125 MHz} & \texttt{int\char`_125} & internal crystal oscillator using PLL, 125 MHz output (default) \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_10to125} & external 10 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_100to125} & external 100 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_125to125} & external 125 MHz reference using PLL, 125 MHz output \\ \hline
150 MHz & \texttt{int\char`_150} & internal crystal oscillator using PLL, 150 MHz output \\ \hline
\end{tabular}
\end{table}
Alternatively, the clock synthesizer can be bypassed using the \texttt{ext0\char`_bypass} clocking option, where the RTIO clock is directly supplied to the SMA connector.
The resulting clock signal is then routed to both the RTIO system and downstream DRTIO satellites.
Clocking options should be configured by setting the value of the \texttt{rtio} key to the desired configuration through \texttt{artiq\char`_coremgmt}.
For example, the RTIO frequency is synthesized from the external 10 MHz from the SMA connector after issuing the following command.
The clock synthesizer may also be bypassed, using the \texttt{ext0\char`_bypass} option, which will accept a RTIO clock directly supplied to the SMA connector. The resulting signal is then routed to both the RTIO system and downstream satellites.
Clocking options in a running system should be configured by setting the value of the \texttt{rtio\char`_clock} key to the desired configuration through \texttt{artiq\char`_coremgmt}. For example, a RTIO frequency of 125MHz will be synthesized from an external 10 MHz signal after issuing the following command:
\begin{minted}{bash}
artiq_coremgmt config write -s rtio ext0_synth0_10to125
artiq_coremgmt config write -s rtio_clock ext0_synth0_10to125
\end{minted}
\subsection{DRTIO Satellite}
The RTIO clock is first recovered from the SFP transceiver connected to the upstream device. The signal is then cleaned by Si5324 clock synthesizer.
The resulting clock signal is then routed to the RTIO system and downstream DRTIO satellties.
and rebooting.
\subsection{Satellite}
The RTIO clock is recovered from the SFP transceiver connected to the upstream device. The resulting signal is then cleaned by the Si5324 and routed to both the RTIO system and downstream satellites.
\subsection{WRPLL}
Kasli 2.0 can be configured to use WRPLL, a clock recovery method making use of White Rabbit's DDMTD (Digital Dual Mixer Time Difference) and the card's Si549 oscillator, both to lock the main RTIO clock and to lock satellite clocks to master.
\section{User LEDs}
Kasli 2.0 supplies three user LEDs for debugging purposes. Two are located on the front panel. The third is located on the PCB itself, beside the SFP cage. An additional ERR LED on the front panel is used by ARTIQ firmware to indicate a runtime panic.
\newpage
\section{Example ARTIQ code}
The sections below demonstrate simple usage scenarios of the system extensions of the ARTIQ control system.
These extensions make use of the resources on 1124 Carrier Kasli 2.0.
They do not exhaustively demonstrate all the features of the ARTIQ system.
The full documentation for the ARTIQ software and gateware is available at \url{https://m-labs.hk}.
The sections below demonstrate simple usage scenarios of extensions on the ARTIQ control system. These extensions make use of the resources on the Kasli 2.0 1124 carrier board. They do not exhaustively demonstrate all the features of the ARTIQ system.
The full documentation for ARTIQ software and gateware, including the guide for its use, is available at \url{https://m-labs.hk/artiq/manual/}. Please consult the manual for details and reference material on the functions and structures used here.
\subsection{Direct Memory Access (DMA)}
Instead of directly emitting RTIO events, a sequence of RTIO events can be recorded in advance and stored in the local SDRAM.
The event sequence can be replayed at another specified timestamp at a higher speed compared to the CPU alone.
The following example records an LED blinking sequence, and replayed twice consecutively using \texttt{CoreDMA}.
\texttt{led0} blinked twice in this example.
Instead of directly emitting RTIO events, sequences of RTIO events can be recorded in advance and stored in the local SDRAM. The event sequence can then be replayed at a specified timestamp. This is of special advantage in cases where RTIO events are too closely placed to be generated as they are executed, as events can be replayed at a higher speed than the on-FPGA CPU alone is capable of.
The following example records an LED event sequence and replays it twice consecutively using \texttt{CoreDMA}. When run, the LED should blink twice.
\inputcolorboxminted{firstline=10,lastline=29}{examples/dma.py}
The stored waveform can be referenced and replayed in different kernels.
However, the waveform is no longer retrievable once core device is rebooted.
Stored waveforms can be referenced and replayed in different kernels, but cannot be retrieved and must be regenerated if the core device is rebooted.
\newpage
\subsection{Dataset Manipulation with Core Device Cache}
Experiments may require values computed/found in previously executed kernels.
To avoid invoking an RPC/sacrificing the pre-computation in \texttt{prepare()} stage, data can be cached in the core device cache.
The following code snippets consists of 2 experiments, where the data from the first experiement is cached.
The same data is retrieved and printed as hexadecimal in the second experiment.
\subsection{Dataset manipulation with core device cache}
Experiments may require values computed or found in previously executed kernels. To avoid invoking an RPC or sacrificing the pre-computation in \texttt{prepare()} stage, data can be stored in the core device cache.
The following code snippets describe two experiments, in which the data from the first experiment is cached. The data is then retrieved and printed in hexadecimal form in the second experiment.
\inputcolorboxminted{firstline=9,lastline=16}{examples/cache.py}
\inputcolorboxminted{firstline=24,lastline=35}{examples/cache.py}
Similar to DMA, the cached data is no longer retrievable once the core device is rebooted.
Similar to DMA, cached data is no longer retrievable once the core device has been rebooted.
\section{Ordering Information}
To order, please visit \url{https://m-labs.hk} and select the 1124 Carrier Kasli 2.0 in the ARTIQ Sinara crate configuration tool.
The cards may also be ordered separately by writing to \url{mailto:sales@m-labs.hk}.
\section{Ordering information}
To order, please visit \url{https://m-labs.hk} and select 1124 Carrier Kasli 2.0 in the ARTIQ/Sinara crate configuration tool. Cards may also be ordered separately by writing to \url{mailto:sales@m-labs.hk}.
\section*{}
\vspace*{\fill}