From d7ef07a0c256f9aeb7d0fc43a5f3c20ed58117e1 Mon Sep 17 00:00:00 2001 From: Sebastien Bourdeauducq Date: Tue, 26 Sep 2017 16:44:23 +0800 Subject: [PATCH] rtio/sed: document architecture --- artiq/gateware/rtio/sed/__init__.py | 57 +++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/artiq/gateware/rtio/sed/__init__.py b/artiq/gateware/rtio/sed/__init__.py index e69de29bb..d0055b52a 100644 --- a/artiq/gateware/rtio/sed/__init__.py +++ b/artiq/gateware/rtio/sed/__init__.py @@ -0,0 +1,57 @@ +""" +The traditional RTIO system used one dedicated FIFO per output channel. While this architecture +is simple and appropriate for ARTIQ systems that were rather small and simple, it shows limitations +on more complex ones. By decreasing importance: +* with DRTIO, the master needed to keep track, for each FIFO in each satellite, a lower bound on +the number of available entries plus the last timestamp written. The timestamp is stored in order +to detect sequence errors rapidly (and allow precise exceptions without compromising performance). +When many satellites are involved, especially with DRTIO switches, the storage requirements become +prohibitive. +* with many channels in one device, the large muxes and the error detection logic that +can handle all the FIFOs make timing closure problematic. +* with many channels in one device, the FIFOs waste FPGA space, as they are never all filled at the +same time. + +The scalable event dispatcher (SED) addresses those issues: +* only one lower bound on the available entries needs to be stored per satellite device for flow +control purposes (called "buffer space"). Most sequence errors no longer exist (non-increasing +timestamps into one channel are permitted to an extent) so rapid detection of them is no longer +required. +* the events can be demultiplexed to the different channels using pipeline stages that ease timing. +* only a few FIFOs are required and they are shared between the channels. + +The SED core contains a configurable number of FIFOs that hold the usual information about RTIO +events (timestamp, address, data), the channel number, and a sequence number. The sequence number is +increased for each event submitted. + +When an event is submitted, it is written into the current FIFO if its timestamp is strictly +increasing. Otherwise, the current FIFO number is incremented by one (and wraps around, if the +current FIFO was the last) and the event is written there, unless that FIFO already contains an +event with a greater timestamp. In that case, an asynchronous error is reported. If the destination +FIFO is full, the submitter is blocked. + +In order to help spreading events among FIFOs and maximize buffering, the SED core may optionally +also switch to the next FIFO after the current FIFO has been full. + +At the output of the FIFOs, the events are distributed to the channels and simultaneous events on +the same channel are handled using a structure similar to a odd-even merge-sort network that sorts +by channel. When there are simultaneous events on the same channel, the event with the highest +sequence number is kept and a flag is raised to indicate that a replacement occured on that +channel. If a replacement was made on a channel that has replacements disabled, the final +event is dropped and a collision error is reported asynchronously. + +Underflow errors are detected as before by comparing the event timestamp with the current value of +the counter, and dropping events that do not have enough time to make it through the system. + +The sequence number is sized to be able to represent the combined capacity of all FIFOs, plus +2 bits that allow the detection of wrap-arounds. + +The maximum number of simultaneous events (on different channels), and the maximum number of active +timeline "rewinds", are equal to the number of FIFOs. + +The SED logic support both synchronous and asynchronous FIFOs, which are used respectively for local +RTIO and DRTIO. + +To implement flow control in DRTIO, the master queries the satellite for buffer space. The satellite +uses as buffer space the space available in its fullest FIFO. +"""