Create file for shared coredevice sections

This commit is contained in:
architeuthis 2024-12-12 20:48:25 +01:00
parent 0e67fa4da0
commit e97877467a
2 changed files with 161 additions and 114 deletions

View File

@ -1,4 +1,5 @@
\include{preamble.tex} \input{preamble.tex}
\input{shared/coredevice.tex}
\graphicspath{{images/1124}{images}} \graphicspath{{images/1124}{images}}
\title{1124 Carrier Kasli 2.0} \title{1124 Carrier Kasli 2.0}
@ -30,7 +31,7 @@
\end{itemize} \end{itemize}
\section{General Description} \section{General Description}
The 1124 Kasli 2.0 Carrier card is an 8hp EEM module, designed to run ARTIQ kernels sent from a host machine over the network. It supports up to 12 EEM connections to other EEM cards in the ARTIQ-Sinara family and up four SFP connections, used for comunications with other carriers and/or Ethernet. The 1124 Kasli 2.0 Carrier card is an 8hp EEM module, designed to run ARTIQ kernels sent from a host machine over the network. It supports up to 12 EEM connections to other EEM cards in the ARTIQ/Sinara family and up four SFP connections, used for comunications with other carriers and/or Ethernet.
Real-time control of EEM daughtercards is implemented using the ARTIQ RTIO system. 1ns temporal resolution can be achieved for TTL events. Real-time control of EEM daughtercards is implemented using the ARTIQ RTIO system. 1ns temporal resolution can be achieved for TTL events.
@ -163,13 +164,14 @@ Real-time control of EEM daughtercards is implemented using the ARTIQ RTIO syste
\caption{Kasli 2.0 front panel} \caption{Kasli 2.0 front panel}
\end{figure} \end{figure}
% For wide tables, a single column layout is better. It can be switched % END PAGE ONE (for wide pages a single-column layout is preferable)
% page-by-page.
\onecolumn \onecolumn
\sourcesection{Kasli 2.0}{https://github.com/sinara-hw/Kasli} \sourcesection{Kasli 2.0}{https://github.com/sinara-hw/Kasli}
\section{Electrical Specifications} \section{Electrical Specifications}
External clock parameters are derived based on the internal termination specified in External clock parameters are derived based on the internal termination specified in
UG471\footnote{\label{ug471}\url{https://docs.xilinx.com/v/u/en-US/ug471\_7Series\_SelectIO}} UG471\footnote{\label{ug471}\url{https://docs.xilinx.com/v/u/en-US/ug471\_7Series\_SelectIO}}
and the voltage range specified in and the voltage range specified in
@ -203,21 +205,11 @@ Power is to be supplied through the barrel connector in the front panel, size 5.
Kasli 2.0 features an XC7A100T-3FGG484E Xilinx Artix-7 FPGA to facilitate reconfigurable high-speed real-time control of inputs and outputs. Most commonly, this FPGA is flashed with binaries compiled from the ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) control system, which equips the carrier board with specialized gateware for handling other Sinara EEMs and an on-FPGA CPU for running ARTIQ experiment \mbox{kernels}. Kasli 2.0 features an XC7A100T-3FGG484E Xilinx Artix-7 FPGA to facilitate reconfigurable high-speed real-time control of inputs and outputs. Most commonly, this FPGA is flashed with binaries compiled from the ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) control system, which equips the carrier board with specialized gateware for handling other Sinara EEMs and an on-FPGA CPU for running ARTIQ experiment \mbox{kernels}.
ARTIQ is open-source and can be found in the repository \url{https://github.com/m-labs/artiq}. Orders of Sinara hardware are normally accompanied with precompiled binaries. Long-term support for ARTIQ systems can also be purchased.
A micro-USB located on the front panel is equipped for JTAG, I2C, and UART serial output. The serial interface runs at 115200bps 8-N-1. A micro-USB located on the front panel is equipped for JTAG, I2C, and UART serial output. The serial interface runs at 115200bps 8-N-1.
\subsection{Note on distributed RTIO (DRTIO)} \artiqsection
DRTIO is the time and data transfer system that allows ARTIQ RTIO channels to be distributed among several core device carrier boards, synchronized and controlled by a central core device. The system itself is more fully described in the ARTIQ documentation\footnote{\label{manual-drtio}\url{https://m-labs.hk/artiq/manual/drtio.html}}. With ARTIQ firmware/gateware, supported core devices, including Kasli 2.0, can take one of three roles: \noteondrtio{Kasli 2.0}
\begin{enumerate}
\item \textbf{Master} \\
The DRTIO master is unique in a DRTIO system. It requires a direct network connection to the host machine. It may make downstream connections to satellites. It controls its own local RTIO channels and the downstream DRTIO satellite(s).
\item \textbf{Satellite} \\
Any other core devices in a DRTIO system are DRTIO satellites. They require an upstream connection to one other core device, master or satellite, through which communications will ultimately be chained to the master. They may make further downstream connections to other satellites. They may control RTIO channels through subkernels or simply pass on communications from the master.
\item \textbf{Standalone}\\
When run in a non-distributed ARTIQ configuration, with a single central core device but no satellites, that core device is instead known as standalone.
\end{enumerate}
\section{Communication Interfaces} \section{Communication Interfaces}
@ -237,68 +229,31 @@ A Kasli 2.0 board must acquire an upstream connection through the \texttt{SFP0}
\subsection{Downstream connection} \subsection{Downstream connection}
Kasli 2.0 supports up to 3 DRTIO satellite connections per device. Any of the 3 downstream SFP ports (i.e. \texttt{SFP1}, \texttt{SFP2}, \texttt{SFP3}) may be used. Kasli 2.0 supports up to 3 DRTIO satellite connections per device. Any of the 3 downstream SFP ports (i.e. \texttt{SFP1}, \texttt{SFP2}, \texttt{SFP3}) may be used.
\section{Clock Routing} \clockingsection{Kasli 2.0}{FPGA}
\subsection{Standalone/Master}
The RTIO clock is typically synthesized by the Si5324 clock multiplier and distributed by the ADCLK948 clock fanout buffer to both the FPGA and the MMCX connectors. Alternatively, an external reference can be supplied through the front panel SMA connector. It is then buffered in the FPGA and sent to the Si5324 for clock synthesis. Kasli 2.0 supports a set of RTIO clock options:
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|}
\hline
RTIO frequency & Configuration & Clock generation \\ \hline
100 MHz & \texttt{int\char`_100} & internal crystal oscillator using PLL, 100 MHz output \\ \hline
\multirow{4}{*}{125 MHz} & \texttt{int\char`_125} & internal crystal oscillator using PLL, 125 MHz output (default) \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_10to125} & external 10 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_100to125} & external 100 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_125to125} & external 125 MHz reference using PLL, 125 MHz output \\ \hline
150 MHz & \texttt{int\char`_150} & internal crystal oscillator using PLL, 150 MHz output \\ \hline
\end{tabular}
\end{table}
The clock synthesizer may also be bypassed, using the \texttt{ext0\char`_bypass} option, which will accept a RTIO clock directly supplied to the SMA connector. The resulting signal is then routed to both the RTIO system and downstream satellites.
Clocking options in a running system should be configured by setting the value of the \texttt{rtio\char`_clock} key to the desired configuration through \texttt{artiq\char`_coremgmt}. For example, a RTIO frequency of 125MHz will be synthesized from an external 10 MHz signal after issuing the following command:
\begin{minted}{bash}
artiq_coremgmt config write -s rtio_clock ext0_synth0_10to125
\end{minted}
and rebooting.
\subsection{Satellite}
The RTIO clock is recovered from the SFP transceiver connected to the upstream device. The resulting signal is then cleaned up by the Si5324 and routed to both the RTIO system and downstream satellites.
\subsection{WRPLL}
Kasli 2.0 can be configured to use WRPLL, a clock recovery method making use of White Rabbit's DDMTD (Digital Dual Mixer Time Difference) and the card's Si549 oscillators, both to lock the main RTIO clock and to lock satellite clocks to master.
\section{User LEDs} \section{User LEDs}
Kasli 2.0 supplies three user LEDs for debugging purposes. Two are located on the front panel. The third is located on the PCB itself, beside the SFP cage. An additional ERR LED on the front panel is used by ARTIQ firmware to indicate a runtime panic. Kasli 2.0 supplies three user LEDs for debugging purposes. Two are located on the front panel. The third is located on the PCB itself, beside the SFP cage. An additional ERR LED on the front panel is used by ARTIQ firmware to indicate a runtime panic.
\newpage \sysdescsection
\codesection{Kasli 2.0 1124 carrier} An example description file for a system using 1124 Kasli 2.0 as a master core device might begin:
\subsection{Direct Memory Access (DMA)} \begin{tcolorbox}[colback=white]
Instead of directly emitting RTIO events, sequences of RTIO events can be recorded in advance and stored in the local SDRAM. The event sequence can then be replayed at a specified timestamp. This is of special advantage in cases where RTIO events are too closely placed to be generated as they are executed, as events can be replayed at a higher speed than the on-FPGA CPU alone is capable of. \begin{minted}{json}
"target": "kasli",
"variant": "my_variant",
"hw_rev": "v2.0",
"base": "master",
"peripherals": [ ]
\end{minted}
\end{tcolorbox}
The following example records an LED event sequence and replays it twice consecutively using \texttt{CoreDMA}. When run, the LED should blink twice. \coresysdesc
\inputcolorboxminted{firstline=10,lastline=29}{examples/dma.py}
Stored waveforms can be referenced and replayed in different kernels, but cannot be retrieved and must be regenerated if the core device is rebooted.
\newpage \newpage
\subsection{Dataset manipulation with core device cache} \coredevicecode{Kasli 2.0 1124 carrier}
Experiments may require values computed or found in previously executed kernels. To avoid invoking an RPC or sacrificing the pre-computation in \texttt{prepare()} stage, data can be stored in the core device cache.
The following code snippets describe two experiments, in which the data from the first experiment is cached. The data is then retrieved and printed in hexadecimal form in the second experiment.
\inputcolorboxminted{firstline=9,lastline=16}{examples/cache.py}
\inputcolorboxminted{firstline=24,lastline=35}{examples/cache.py}
Similar to DMA, cached data is no longer retrievable once the core device has been rebooted.
\ordersection{1124 Carrier Kasli 2.0} \ordersection{1124 Carrier Kasli 2.0}

92
shared/coredevice.tex Normal file
View File

@ -0,0 +1,92 @@
\newcommand{\artiqsection} {
\section{Firmware/ARTIQ}
ARTIQ is open-source and can be found in the repository \url{https://github.com/m-labs/artiq}. Orders of Sinara hardware are normally preflashed with suitable firmware and gateware binaries. Long-term support for ARTIQ systems can also be purchased, including updated binaries through AFWS (the ARTIQ Firmware Service).
}
\newcommand{\noteondrtio}[1]{
\subsection{Note on distributed RTIO (DRTIO)}
DRTIO is the time and data transfer system which allows ARTIQ RTIO channels to be distributed among several core device carrier boards, synchronized and controlled by a central master device. The system itself is described in more detail in the ARTIQ documentation\footnote{\label{manual-drtio}\url{https://m-labs.hk/artiq/manual/drtio.html}}. Within ARTIQ, core devices, including #1, can take one of three roles:
\begin{enumerate}%[topsep=2pt, itemsep=2pt]
\item \textbf{Master} \\
A DRTIO system must contain one DRTIO master. It controls its own local RTIO channels and the downstream DRTIO satellite(s). It requires a direct network connection to the host machine. It may make downstream connections to satellites.
\item \textbf{Satellite} \\
Other core devices in a DRTIO system are DRTIO satellites. They require an upstream connection to one other core device, master or satellite, through which communications are carried to the master. They may make further downstream connections to other satellites. They may control their local RTIO channels directly through subkernels or simply pass on communications from the master.
\item \textbf{Standalone}\\
When run in a non-distributed ARTIQ configuration, with a single central core device but without satellites, that core device is known as standalone.
\end{enumerate}
}
\newcommand{\clockingsection}[2]{
\section{Clock Routing}
\subsection{Standalone/Master}
The RTIO clock is typically synthesized by the Si5324 clock multiplier and distributed by the ADCLK948 clock fanout buffer to both the #2 and the MMCX connectors. Alternatively, an external reference can be supplied through the front panel SMA connector. It is then buffered in the #2 and sent to the Si5324 for clock synthesis. #1 supports a set of RTIO clock options:
\begin{table}[H]
\centering
\begin{tabular}{|c|c|c|}
\hline
RTIO frequency & Configuration & Clock generation \\ \hline
100 MHz & \texttt{int\char`_100} & internal crystal oscillator using PLL, 100 MHz output \\ \hline
\multirow{4}{*}{125 MHz} & \texttt{int\char`_125} & internal crystal oscillator using PLL, 125 MHz output (default) \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_10to125} & external 10 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_100to125} & external 100 MHz reference using PLL, 125 MHz output \\ \cline{2-3}
& \texttt{ext0\char`_synth0\char`_125to125} & external 125 MHz reference using PLL, 125 MHz output \\ \hline
150 MHz & \texttt{int\char`_150} & internal crystal oscillator using PLL, 150 MHz output \\ \hline
\end{tabular}
\end{table}
The clock synthesizer may also be bypassed, using the \texttt{ext0\char`_bypass} option, which will accept a RTIO clock directly supplied to the SMA connector. The resulting signal is then routed to both the RTIO system and downstream satellites.
Clocking options in a running system should be configured by setting the value of the \texttt{rtio\char`_clock} key to the desired configuration through the ARTIQ \texttt{artiq\char`_coremgmt} command. For example, a RTIO frequency of 125MHz will be synthesized from an external 10 MHz signal after issuing the following command:
\begin{center}
\texttt{artiq\_coremgmt config write -s rtio\_clock ext0\_synth0\_10to125}
\end{center}
and rebooting.
\subsection{Satellite}
The RTIO clock is recovered from the SFP transceiver connected to the upstream device. The resulting signal is then cleaned up by the Si5324 and routed to both the RTIO system and downstream satellites.
\subsection{WRPLL}
#1 can be configured to use WRPLL, a clock recovery method making use of White Rabbit's DDMTD (Digital Dual Mixer Time Difference) and the card's Si549 oscillators, both to lock the main RTIO clock and to lock satellite clocks to master.
}
\newcommand{\coresysdesc}{ % again including the minted JSON snippet through a macro isn't practical
where the \texttt{peripherals} list contains the corresponding entries for peripherals (daughtercards) in use.
For all accepted keys and values, see the JSON schema \texttt{coredevice\_generic.schema.json} in the ARTIQ repository.\footnote{\url{https://github.com/m-labs/artiq/blob/release-8/artiq/coredevice/coredevice_generic.schema.json}}.
}
\newcommand{\coredevicecode}[1] {
\codesection{#1}
\subsection{Direct Memory Access (DMA)}
Instead of directly emitting RTIO events, sequences of RTIO events can be recorded in advance and stored in the local SDRAM. The event sequence can then be replayed at a specified timestamp. This is of special advantage in cases where RTIO events are too closely placed to be generated as they are executed, as events can be replayed at a higher speed than the on-FPGA CPU alone is capable of.
The following example records an LED event sequence and replays it twice consecutively using \texttt{CoreDMA}. When run, the LED should blink twice.
\inputcolorboxminted{firstline=10,lastline=29}{examples/dma.py}
Stored waveforms can be referenced and replayed in different kernels, but cannot be retrieved and must be regenerated if the core device is rebooted.
\subsection{Dataset manipulation with core device cache}
Experiments may require values computed or found in previously executed kernels. To avoid invoking an RPC or sacrificing the pre-computation in \texttt{prepare()} stage, data can be stored in the core device cache.
The following code snippets describe two experiments, in which the data from the first experiment is cached. The data is then retrieved and printed in hexadecimal form in the second experiment.
\inputcolorboxminted{firstline=9,lastline=16}{examples/cache.py}
\inputcolorboxminted{firstline=24,lastline=35}{examples/cache.py}
Similar to DMA, cached data is no longer retrievable once the core device has been rebooted.
}