cxp router: init routing 50Gbps somehow

router:

.pre-commit-config.yaml

@@ -0,0 +1,8 @@
+repos: 
+  - repo: local
+    hooks:
+      - id: cargo_format
+        name: cargo_format
+        entry: cargofmt.sh
+        language: script
+        pass_filenames: false

README.md

@@ -4,60 +4,102 @@ ARTIQ on Zynq
 How to use
 ----------
 
-1. Install the ARTIQ version that corresponds to the artiq-zynq version you are targeting.
-2. To obtain firmware binaries, select the latest successful build on [Hydra](https://nixbld.m-labs.hk/) for the targeted artiq-zynq version, or use AFWS. If using Hydra, search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``).
-3. Place the ``boot.bin`` file, obtained from Hydra's "binary distribution" download link or from AFWS, at the root of a FAT-formatted SD card.
-4. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network.
-5. Insert the SD card into the board and set up the board to boot from the SD card. For the ZC706, this is achieved by placing the large DIP switch SW11 in the 00110 position.
-6. Power up the board. After the firmware starts successfully, it should respond to ping at its IP addresses, and boot messages can be observed from its UART at 115200bps.
-7. Create and use an ARTIQ device database as usual, but set ``"target": "cortexa9"`` in the arguments of the core device.
+1. [Install ARTIQ](https://m-labs.hk/artiq/manual/installing.html). Get the corresponding version to the ``artiq-zynq`` version you are targeting.
+2. To obtain firmware binaries, use AFWS or build your own; see [the ARTIQ manual](https://m-labs.hk/artiq/manual/building_developing.html) for detailed instructions or skip to "Development" below. ZC706 variants only can also be downloaded from latest successful build on [Hydra](https://nixbld.m-labs.hk/).
+3. Place ``boot.bin`` file at the root ``/`` of a FAT-formatted SD card.
+4. Optionally, create a ``config.txt`` configuration file containing ``key=value`` pairs on each line and place it at the root of the SD card. See below for valid keys. The ``ip``, ``ip6`` and ``mac`` keys can be used to set networking information. If these keys are not found, the firmware will use default values which may or may not be compatible with your network.
+5. Insert the SD card into the board and set the board to boot from the SD card. For ZC706, this is achieved by placing the large DIP switch SW11 into the 00110 position. On Kasli-SoC, place the BOOT MODE switches to SD.
+6. Power up the board. After successful boot the firmware should respond to ping at its IP addresses. Boot output can be observed from UART at 115200bps 8-N-1.
+7. Create and use an ARTIQ device database as usual.
 
 Configuration
 -------------
 
-Configuring the device is done using the ``config.txt`` text file at the root of the SD card, plus the contents of the ``config`` folder. When searching for a configuration key, the firmware first looks for a file named ``/config/[key].bin`` and, if it exists, returns the contents of that file. If not, it looks into ``/config.txt``, which contains a list of ``key=value`` pairs, one per line. The ``config`` folder allows configuration values that consist in binary data, such as the startup kernel.
+Configuring the device is done using the ``config.txt`` text file at the root of the SD card plus optionally a ``config`` folder. When searching for a configuration key, the firmware first looks for a file named ``/config/[key].bin`` and, if it exists, returns the contents of that file. If not, it looks into ``/config.txt``, which should contain a list of ``key=value`` pairs, one per line. ``config.txt`` should be used for most keys but the ``config`` folder allows for setting configuration values which consist of binary data, such as the startup kernel.
 
-The following configuration keys are available:
+The following configuration keys are available among others:
 
 - ``mac``: Ethernet MAC address.
 - ``ip``: IPv4 address.
 - ``ip6``: IPv6 address.
-- ``startup``: startup kernel in ELF format (as produced by ``artiq_compile``).
+- ``idle_kernel``: idle kernel in ELF format (as produced by ``artiq_compile``).
+- ``startup_kernel``: startup kernel in ELF format (as produced by ``artiq_compile``).
 - ``rtio_clock``: source of RTIO clock; valid values are ``ext0_bypass`` and ``int_125``.
-- ``boot``: SD card "boot.bin" file, for replacing the boot firmware/gateware. Write only.
 
-Configurations can be read/written/removed via ``artiq_coremgmt``. Config erase is
-not implemented as it seems not very useful.
+See [ARTIQ manual](https://m-labs.hk/artiq/manual-beta/core_device.html#configuration-storage) for full list. Configurations can be read/written/removed with ``artiq_coremgmt``. Config erase is not implemented, as it isn't particularly useful.
+
+For convenience, the ``boot`` key can be used with ``artiq_coremgmt`` and a ``boot.bin`` file to replace firmware/gateware in a running system. This key is read-only. When loading ``boot.bin`` onto the SD card directly, place it at the root and not in the ``config`` folder.
 
 Development instructions
 ------------------------
 
-ARTIQ on Zynq is packaged using the [Nix](https://nixos.org) Flakes system. Install Nix 2.8+ and enable flakes by adding ``experimental-features = nix-command flakes`` to ``nix.conf`` (e.g. ``~/.config/nix/nix.conf``).
+ARTIQ on Zynq is packaged using [Nix](https://nixos.org) Flakes. Install Nix 2.8+ and enable flakes by adding ``experimental-features = nix-command flakes`` to ``nix.conf`` (e.g. ``~/.config/nix/nix.conf``).
 
-Pure build with Nix and execution on a remote JTAG server:
+**Pure build with Nix:**
 
 ```shell
-nix build .#zc706-nist_clock-jtag  # or zc706-nist_qc2-jtag or zc706-nist_clock_satellite-jtag etc.
-./remote_run.sh
+nix build .#zc706-nist_clock-jtag  # or zc706-nist_qc2-jtag or zc706-nist_clock-sd or etc
 ```
 
-Impure incremental build and execution on a remote JTAG server:
+Run ``nix flake show`` to see all valid build targets. Targets suffixed with ``-jtag`` produce separate firmware and gateware files, intended for use in booting via JTAG server/Ethernet, e.g. ``./remote_run.sh -i`` with a remote JTAG server. Targets suffixed with ``-sd`` will produce ``boot.bin`` file suitable for SD card boot. ``-firmware`` and ``-gateware`` respectively build firmware and gateware only.
+
+The Kasli-SoC target requires a system description file as input. See ARTIQ manual for exact instructions or use incremental build.
+
+**Impure incremental build:**
+
+For boards with fixed variants, i.e. ZC706, etc. :
 
 ```shell
 nix develop
 cd src
-gateware/zc706.py -g ../build/gateware -V <variant> # build gateware
-make GWARGS="-V <variant>" <runtime/satman>    # build firmware
-cd ..
-./remote_run.sh -i
+gateware/<board>.py -g ../build/gateware -V <variant> # gateware
+make GWARGS="-V <variant>" <runtime/satman>    # firmware
+```
+
+For boards with system descriptions, i.e. Kasli-SoC, etc. :
+
+```shell
+nix develop
+cd src
+gateware/<board>.py -g ../build/gateware <description.json> # gateware
+make TARGET=<board> GWARGS="path/to/description.json" <runtime/satman> # firmware
+```
+
+``szl.elf`` can be obtained with:
+
+```shell
+nix build git+https://git.m-labs.hk/m-labs/zynq-rs#<board>-szl
+```
+
+To generate ``boot.bin`` use ``mkbootimage``, e.g.:
+
+```shell
+echo "the_ROM_image:
+    {
+        [bootloader]result/szl.elf
+        gateware/top.bit
+        firmware/armv7-none-eabihf/release/<runtime/satman>
+    }
+    EOF" >> boot.bif
+mkbootimage boot.bif boot.bin
 ```
 
 Notes:
 
 - The impure build process is also compatible with non-Nix systems.
-- When calling make, you need to specify both the variant and firmware type.
 - Firmware type must be either ``runtime`` for DRTIO-less or DRTIO master variants, or ``satman`` for DRTIO satellite.
-- If the board is connected to the local machine, use the ``local_run.sh`` script.
+- If the board is connected to the local machine by JTAG, use the ``local_run.sh`` script.
+- A known Xilinx hardware bug prevents repeatedly loading the bootloader over JTAG without a POR reset. If booting over JTAG, install a jumper on ``PS_POR_B`` and use the POR reset script [here](https://git.m-labs.hk/M-Labs/zynq-rs/src/branch/master/kasli_soc_por.py).
+
+Pre-Commit Hooks
+----------------
+
+You are strongly recommended to use the provided pre-commit hooks to automatically reformat files and check for non-optimal Rust/C/C++ practices. Run `pre-commit install` to install the hook and `pre-commit` will automatically run `cargo fmt`, `cargo clippy`, and `clang-format` for you.
+
+Several things to note:
+
+- If `cargo fmt`, `cargo clippy`, or `clang-format` returns an error, the pre-commit hook will fail. You should fix all errors before trying to commit again.
+- If `cargo fmt` or `clang-format` reformats some files, the pre-commit hook will also fail. You should review the changes and, if satisfied, try to commit again.
 
 License
 -------

build_CI.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+set -e
+
+if [ -z "$OPENOCD_ZYNQ" ]; then
+    echo "OPENOCD_ZYNQ environment variable must be set"
+    exit 1
+fi
+if [ -z "$SZL" ]; then
+    echo "SZL environment variable must be set"
+    exit 1
+fi
+
+
+# variant="firmware"
+# variant="gateware"
+# variant="jtag"
+variant="sd"
+
+nix build .#kasli_soc-demo-$variant -L
+nix build .#kasli_soc-master-$variant -L
+nix build .#kasli_soc-satellite-$variant -L
+
+# nix build .#zc706-acpki_nist_clock-$variant -L
+# nix build .#zc706-acpki_nist_clock_master-$variant -L
+# nix build .#zc706-acpki_nist_clock_satellite-$variant -L
+
+# nix build .#zc706-acpki_nist_qc2-$variant -L
+# nix build .#zc706-acpki_nist_qc2_master-$variant -L
+# nix build .#zc706-acpki_nist_qc2_satellite-$variant -L
+
+# nix build .#zc706-nist_clock-$variant -L
+# nix build .#zc706-nist_clock_master-$variant -L
+# nix build .#zc706-nist_clock_satellite-$variant -L
+
+# nix build .#zc706-nist_qc2-$variant -L
+# nix build .#zc706-nist_qc2_master-$variant -L
+# nix build .#zc706-nist_qc2_satellite-$variant -L

cargofmt.sh

@@ -0,0 +1,5 @@
+#!/usr/bin/env bash
+
+nix-shell -p gnumake --command 'make manifests -B'
+cd src
+cargo fmt -- --check

coaxpress.drawio

@@ -0,0 +1,338 @@
+<mxfile host="65bd71144e">
+    <diagram id="en7HUHNV3kVsTTCxeEt8" name="Page-1">
+        <mxGraphModel dx="1155" dy="1481" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="0" pageScale="1" pageWidth="850" pageHeight="1100" background="none" math="0" shadow="0">
+            <root>
+                <mxCell id="0"/>
+                <mxCell id="1" parent="0"/>
+                <mxCell id="7" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;rounded=0;" parent="1" edge="1">
+                    <mxGeometry relative="1" as="geometry">
+                        <mxPoint x="320" y="280" as="targetPoint"/>
+                        <mxPoint x="240" y="280.0000000000001" as="sourcePoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="109" value="32" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="7" vertex="1" connectable="0">
+                    <mxGeometry x="-0.3138" y="2" relative="1" as="geometry">
+                        <mxPoint x="12" y="-8" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="14" value="" style="endArrow=classic;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;rounded=0;dashed=1;" parent="1" edge="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="390" y="600" as="sourcePoint"/>
+                        <mxPoint x="240" y="599.76" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="15" value="CTRL/Trig" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];rounded=0;" parent="14" vertex="1" connectable="0">
+                    <mxGeometry x="-0.375" y="4" relative="1" as="geometry">
+                        <mxPoint x="-15" y="-14" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="21" value="TX&lt;br style=&quot;border-color: var(--border-color);&quot;&gt;Low speed" style="shape=offPageConnector;whiteSpace=wrap;html=1;rotation=0;size=0.3333333333333333;direction=south;fillColor=#DAE8FC;strokeColor=#6c8ebf;rounded=0;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="400" width="120" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="24" value="RX GTX&lt;br style=&quot;border-color: var(--border-color);&quot;&gt;High Speed&lt;br&gt;Master" style="shape=offPageConnector;whiteSpace=wrap;html=1;rotation=0;size=0.3333333333333333;direction=north;fillColor=#f8cecc;strokeColor=#b85450;rounded=0;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="240" width="120" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="25" value="TX High Speed&lt;br&gt;(optional)" style="shape=offPageConnector;whiteSpace=wrap;html=1;rotation=0;size=0.3333333333333333;direction=south;fillColor=#f8cecc;strokeColor=#b85450;dashed=1;rounded=0;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="560" width="120" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="26" value="RX GTX&lt;br style=&quot;border-color: var(--border-color);&quot;&gt;High Speed&amp;nbsp;&lt;br&gt;Extension&lt;br&gt;(Optional)" style="shape=offPageConnector;whiteSpace=wrap;html=1;rotation=0;size=0.4166666666666667;direction=north;fillColor=#f8cecc;strokeColor=#b85450;rounded=0;dashed=1;" parent="1" vertex="1">
+                    <mxGeometry x="120" width="120" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="30" value="" style="endArrow=classic;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;rounded=0;" parent="1" edge="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="400" y="439.58" as="sourcePoint"/>
+                        <mxPoint x="240" y="440" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="31" value="CTRL/Trig &lt;br&gt;DATA PACKET" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];rounded=0;" parent="30" vertex="1" connectable="0">
+                    <mxGeometry x="-0.375" y="4" relative="1" as="geometry">
+                        <mxPoint x="-30" y="-24" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="33" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;TX pipeline&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- priority transmission&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- IDLE&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;align=center;" parent="1" vertex="1">
+                    <mxGeometry x="400" y="400" width="160" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="38" value="Red: clocked by cxp_gtx_rx/tx (31.25MHz - 312.5MHz)&amp;nbsp;" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;strokeColor=#b85450;fillColor=#f8cecc;align=left;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="680" width="360" height="40" as="geometry"/>
+                </mxCell>
+                <mxCell id="39" value="Blue: clocked by sys with CEInserter (20.83MHz - 41.66MHz)&amp;nbsp;" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;strokeColor=#6c8ebf;fillColor=#dae8fc;align=left;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="720" width="360" height="40" as="geometry"/>
+                </mxCell>
+                <mxCell id="40" value="White: clocked by sys (125MHz)" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;strokeColor=default;fontColor=#000000;fillColor=default;align=left;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="800" width="360" height="40" as="geometry"/>
+                </mxCell>
+                <mxCell id="43" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&lt;b&gt;&lt;u&gt;CXP Bootstrap FW&lt;/u&gt;&lt;/b&gt;&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- handle GTX speed&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- test connection&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- hand over to RTIO after init&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&lt;span style=&quot;white-space: pre;&quot;&gt;&#9;&lt;/span&gt;- cannot access CTRL PAK&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- with camera specific .rs file&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- compare heatbeat to check connection status&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- handle event ack??&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- don't use tag?&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;align=center;fillColor=#fff2cc;strokeColor=#d6b656;" parent="1" vertex="1">
+                    <mxGeometry x="800" y="240" width="240" height="240" as="geometry"/>
+                </mxCell>
+                <mxCell id="53" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&lt;u&gt;&lt;b&gt;CXP Camera specific prog&lt;/b&gt;&lt;/u&gt;&lt;br&gt;GenICam interface @&amp;nbsp;&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;RTIO coredevice&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- handle frame programming&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- handle event as well??&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- two interface&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&amp;nbsp; &amp;nbsp;- IO CTRL packet via API!&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&amp;nbsp; &amp;nbsp;- on master ch only&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;&amp;nbsp; &amp;nbsp;- O Frame data&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1">
+                    <mxGeometry x="1120" y="80" width="160" height="480" as="geometry"/>
+                </mxCell>
+                <mxCell id="54" value="RX pipeline" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="400" width="160" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="55" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;rounded=0;" parent="1" edge="1">
+                    <mxGeometry relative="1" as="geometry">
+                        <mxPoint x="400" y="39.660000000000025" as="targetPoint"/>
+                        <mxPoint x="240" y="39.660000000000025" as="sourcePoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="63" value="Green: clocked by rio / rio_phy" style="rounded=0;whiteSpace=wrap;html=1;labelBackgroundColor=none;strokeColor=#82b366;fillColor=#d5e8d4;align=left;" parent="1" vertex="1">
+                    <mxGeometry x="120" y="760" width="360" height="40" as="geometry"/>
+                </mxCell>
+                <mxCell id="64" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1040" y="279.65999999999997" as="sourcePoint"/>
+                        <mxPoint x="1120" y="280.03999999999996" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="69" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="640" y="439.71" as="sourcePoint"/>
+                        <mxPoint x="560" y="439.71" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="71" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;TX Bootstrap&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- testseq&lt;/span&gt;&lt;br&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- DMA&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;align=center;" parent="1" vertex="1">
+                    <mxGeometry x="640" y="400" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="72" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="800" y="439.7100000000002" as="sourcePoint"/>
+                        <mxPoint x="720" y="439.7100000000002" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="93" value="CTRL/Event&lt;br&gt;Packet (DMA)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="72" vertex="1" connectable="0">
+                    <mxGeometry x="-0.4276" y="-1" relative="1" as="geometry">
+                        <mxPoint x="-17" y="-19" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="77" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;RX Bootstrap&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- testseq&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- DMA&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;align=center;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="640" y="240" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="78" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="720" y="279.71" as="sourcePoint"/>
+                        <mxPoint x="800" y="279.71" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="82" value="CTRL/Event&lt;br style=&quot;border-color: var(--border-color);&quot;&gt;Packet (DMA)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="78" vertex="1" connectable="0">
+                    <mxGeometry x="-0.4143" y="2" relative="1" as="geometry">
+                        <mxPoint x="17" y="-18" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="85" value="32+8" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry y="10" width="100" relative="1" as="geometry">
+                        <mxPoint x="560" y="279.71000000000004" as="sourcePoint"/>
+                        <mxPoint x="640" y="279.71000000000004" as="targetPoint"/>
+                        <Array as="points"/>
+                        <mxPoint as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="94" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;" parent="1" source="77" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="720" y="119.66000000000003" as="sourcePoint"/>
+                        <mxPoint x="800" y="119.66000000000003" as="targetPoint"/>
+                        <Array as="points">
+                            <mxPoint x="680" y="120"/>
+                        </Array>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="97" value="4x Frame data" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="94" vertex="1" connectable="0">
+                    <mxGeometry x="0.5972" y="-1" relative="1" as="geometry">
+                        <mxPoint x="-1" y="-11" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="95" value="CTRL Packet" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry y="-10" width="100" relative="1" as="geometry">
+                        <mxPoint x="1120" y="439.66000000000025" as="sourcePoint"/>
+                        <mxPoint x="1040" y="439.66000000000025" as="targetPoint"/>
+                        <Array as="points"/>
+                        <mxPoint as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="96" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1040" y="119.57999999999984" as="sourcePoint"/>
+                        <mxPoint x="1120" y="119.95999999999995" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="98" value="&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;RX Bootstrap&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- testseq&lt;/span&gt;&lt;/div&gt;&lt;div style=&quot;text-align: justify;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- DMA&lt;/span&gt;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;align=center;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="640" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="99" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="560" y="39.58000000000004" as="sourcePoint"/>
+                        <mxPoint x="640" y="39.58000000000004" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="100" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="1" target="33" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1120" y="520.0000000000003" as="sourcePoint"/>
+                        <mxPoint x="1040" y="520.0000000000003" as="targetPoint"/>
+                        <Array as="points">
+                            <mxPoint x="480" y="520"/>
+                        </Array>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="101" value="Trigger" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="100" vertex="1" connectable="0">
+                    <mxGeometry x="0.7255" y="-2" relative="1" as="geometry">
+                        <mxPoint x="-53" y="-18" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="103" value="RX EC" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="320" y="240" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="104" value="RX pipeline" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="480" y="240" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="105" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;rounded=0;" parent="1" edge="1">
+                    <mxGeometry relative="1" as="geometry">
+                        <mxPoint x="480" y="310" as="targetPoint"/>
+                        <mxPoint x="400" y="310" as="sourcePoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="107" value="32" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="105" vertex="1" connectable="0">
+                    <mxGeometry x="-0.2448" y="-2" relative="1" as="geometry">
+                        <mxPoint x="10" y="-12" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="106" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;rounded=0;" parent="1" edge="1">
+                    <mxGeometry relative="1" as="geometry">
+                        <mxPoint x="480" y="250" as="targetPoint"/>
+                        <mxPoint x="400" y="250" as="sourcePoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="108" value="8 dchar" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" parent="106" vertex="1" connectable="0">
+                    <mxGeometry x="-0.5034" y="1" relative="1" as="geometry">
+                        <mxPoint x="20" y="-9" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="180" value="Streams&lt;br&gt;Crossbar" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="760" y="-480" width="80" height="320" as="geometry"/>
+                </mxCell>
+                <mxCell id="181" value="ROI Engine" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" parent="1" vertex="1">
+                    <mxGeometry x="1560" y="-360" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="182" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1480" y="-320.0000000000001" as="sourcePoint"/>
+                        <mxPoint x="1560" y="-319.62" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="185" value="&lt;div style=&quot;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- 32bit pixel data&lt;/span&gt;&lt;/div&gt;- frame valid (new frame)&lt;br&gt;- line break" style="text;html=1;align=left;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" parent="1" vertex="1">
+                    <mxGeometry x="1520" y="-430" width="160" height="60" as="geometry"/>
+                </mxCell>
+                <mxCell id="186" value="CRC Checker" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="920" y="-360.42" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="187" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1000" y="-321.25000000000017" as="sourcePoint"/>
+                        <mxPoint x="1080" y="-320.87000000000006" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="188" value="Double&lt;br&gt;Buffered&lt;br&gt;Memory" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="1080" y="-360.42" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="194" value="Pixel Decoder" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#fad9d5;strokeColor=#ae4132;" parent="1" vertex="1">
+                    <mxGeometry x="1400" y="-360" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="195" value="&lt;div style=&quot;&quot;&gt;&lt;span style=&quot;background-color: initial;&quot;&gt;- extract line break &amp;amp; new frame&lt;/span&gt;&lt;/div&gt;" style="text;html=1;align=left;verticalAlign=middle;resizable=0;points=[];autosize=1;strokeColor=none;fillColor=none;" parent="1" vertex="1">
+                    <mxGeometry x="1400" y="-270" width="190" height="30" as="geometry"/>
+                </mxCell>
+                <mxCell id="203" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1160" y="-319.9999999999999" as="sourcePoint"/>
+                        <mxPoint x="1240" y="-319.6199999999998" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="205" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="840" y="-320.4200000000002" as="sourcePoint"/>
+                        <mxPoint x="920" y="-320.0400000000001" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="206" value="CRC Checker" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;dashed=1;" parent="1" vertex="1">
+                    <mxGeometry x="920" y="-480" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="207" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1000" y="-440.83000000000027" as="sourcePoint"/>
+                        <mxPoint x="1080" y="-440.45000000000016" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="208" value="Double&lt;br&gt;Buffered&lt;br&gt;Memory" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;dashed=1;" parent="1" vertex="1">
+                    <mxGeometry x="1080" y="-480" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="209" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="680" y="-440.4200000000003" as="sourcePoint"/>
+                        <mxPoint x="760" y="-440.0400000000002" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="210" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="680" y="-280.00000000000017" as="sourcePoint"/>
+                        <mxPoint x="760" y="-279.62000000000006" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="211" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="680" y="-360.00000000000017" as="sourcePoint"/>
+                        <mxPoint x="760" y="-359.62000000000006" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="212" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="680" y="-200.49000000000018" as="sourcePoint"/>
+                        <mxPoint x="760" y="-200.11000000000007" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="213" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;dashed=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="840" y="-440.42000000000036" as="sourcePoint"/>
+                        <mxPoint x="920" y="-440.04000000000025" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="214" value="Stream&lt;br&gt;Parser" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex="1">
+                    <mxGeometry x="1240" y="-360.00000000000006" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="215" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" edge="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="1320" y="-320.41999999999996" as="sourcePoint"/>
+                        <mxPoint x="1400" y="-320.03999999999985" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="216" value="Streams&lt;br&gt;Crossbar" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
+                    <mxGeometry x="800" y="-40" width="80" height="200" as="geometry"/>
+                </mxCell>
+                <mxCell id="217" value="Stream pipeline #1" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
+                    <mxGeometry x="960" y="80" width="80" height="80" as="geometry"/>
+                </mxCell>
+                <mxCell id="218" value="" style="edgeStyle=none;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="1">
+                    <mxGeometry width="100" relative="1" as="geometry">
+                        <mxPoint x="880" y="119.57999999999993" as="sourcePoint"/>
+                        <mxPoint x="960" y="119.96000000000004" as="targetPoint"/>
+                        <Array as="points"/>
+                    </mxGeometry>
+                </mxCell>
+            </root>
+        </mxGraphModel>
+    </diagram>
+</mxfile>

cxp_kernel.py

@@ -0,0 +1,390 @@
+"""
+Non-realtime drivers for CXP.
+"""
+
+# TODO: add api calls for CTRL packet similar i2c
+# TODO: add timing critical trigger ack
+
+
+from artiq.language.core import syscall, kernel
+from artiq.language.types import TBool, TInt32, TNone
+from artiq.coredevice.rtio import rtio_output,rtio_input_timestamped_data
+from artiq.experiment import *
+import numpy as np
+import math
+
+# TODO: change this to read bytes and accept TBytearray 
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_read_words(addr: TInt32, val: TList(TInt32), with_tag: TBool) -> TInt32:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_readu32(addr: TInt32, with_tag: TBool) -> TInt32:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_readu64(addr: TInt32, with_tag: TBool) -> TInt64:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_writeu32(addr: TInt32, val: TInt32, with_tag: TBool) -> TNone:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_writeu64(addr: TInt32, val: TInt64, with_tag: TBool) -> TNone:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_setup() -> TBool:
+    raise NotImplementedError("syscall not simulated")
+
+
+@syscall(flags={"nounwind", "nowrite"})
+def cxp_debug_frame_print() -> TNone:
+    raise NotImplementedError("syscall not simulated")
+
+
+# Bootstrap register
+_XML_MANIFEST_SIZE = 0x0008
+_XML_MANIFEST_SEL = 0x000C
+_XML_VER = 0x0010
+_XML_SCHEMA_VER = 0x0014
+_XML_URL_ADDR = 0x0018
+_WIDTH_ADDR = 0x3000
+_HEIGHT_ADDR = 0x3004
+_ACQUISITION_MODE_ADDR = 0x3008
+_ACQUISITION_START_ADDR = 0x300C
+_ACQUISITION_STOP_ADDR = 0x3010
+_PIXEL_FORMAT_ADDR = 0x3014
+_DEVICE_TAP_GEOG_ADDR = 0x3018
+_IMG_1_STREAMID_ADDR = 0x301C
+
+_MAX_BYTE_SIZE = 100  # in bytes
+_MAX_WORD_SIZE = _MAX_BYTE_SIZE // 4  # in bytes
+
+
+class CoaXPress:
+    def __init__(self, channel_base, core_device="core", xml_url_len=_MAX_WORD_SIZE):
+        # __device_mgr is private
+        # self.core = dmgr.get(core_device)
+
+        # you can get the channel via `print(len(rtio_channels))` before calling
+        # `rtio_channels.append(rtio.Channel.from_phy(cxp_interface))`
+        self.channel_base = channel_base
+        # the first 8 bits is reserved for the rtlink.OInterface.addr not for channel no.
+        self.target_o = channel_base << 8
+
+        self.with_tag = False
+
+        self.xml_addr = 0
+        self.width_addr = 0
+        self.height_addr = 0
+        self.acq_mode_addr = 0
+        self.acq_start_addr = 0
+        self.acq_stop_addr = 0
+        self.pixel_fmt_addr = 0
+        self.device_tap_geog_addr = 0
+        self.img_1_streamid_addr = 0
+
+        self.xml_url = [0] * xml_url_len
+
+        self.econ_roi = False
+
+    @staticmethod
+    def get_rtio_channels(channel, **kwargs):
+        return [(channel, None)]
+
+    @kernel
+    def trigger(self, linktrig, trigdelay):
+        rtio_output(self.channel_base << 8, linktrig | trigdelay << 1)
+
+    @kernel
+    def setup_roi(self, n, x0, y0, x1, y1):
+        if self.econ_roi:
+            assert x1-x0 >= 4
+        # DEBUG:
+        # c = int64(self.core.ref_multiplier)
+        c = int64(8)
+        rtio_output(((self.channel_base + 1) << 8) | (4*n+0), x0)
+        delay_mu(c)
+        rtio_output(((self.channel_base + 1) << 8) | (4*n+1), y0)
+        delay_mu(c)
+        rtio_output(((self.channel_base + 1) << 8) | (4*n+2), x1)
+        delay_mu(c)
+        rtio_output(((self.channel_base + 1) << 8) | (4*n+3), y1)
+        delay_mu(c)
+
+    # TODO: add gate
+
+    @kernel
+    def input_mu(self, data, tt, timeout_mu=-1):
+        assert len(data) == len(tt)
+        channel = self.channel_base + 2
+
+        for i in range(len(data)):
+            timestamp, roi_output = rtio_input_timestamped_data(timeout_mu, channel)
+            data[i] = roi_output
+            tt[i] = timestamp
+
+    @kernel
+    def init(self):
+        self.with_tag = cxp_setup()
+
+        self.xml_addr = self.read_u32(_XML_URL_ADDR)
+        # self.width_addr = self.read_u32(_WIDTH_ADDR)
+        # self.height_addr = self.read_u32(_HEIGHT_ADDR)
+        self.acq_mode_addr = self.read_u32(_ACQUISITION_MODE_ADDR)
+        self.acq_start_addr = self.read_u32(_ACQUISITION_START_ADDR)
+        self.acq_stop_addr = self.read_u32(_ACQUISITION_STOP_ADDR)
+        self.pixel_fmt_addr = self.read_u32(_PIXEL_FORMAT_ADDR)
+        # self.device_tap_geog_addr = self.read_u32(_DEVICE_TAP_GEOG_ADDR)
+        self.img_1_streamid_addr = self.read_u32(_IMG_1_STREAMID_ADDR)
+
+        self.read_words(self.xml_addr, self.xml_url)
+
+    @kernel
+    def read_u32(self, addr: TInt32) -> TInt32:
+        return cxp_readu32(addr, self.with_tag)
+
+    @kernel
+    def read_u64(self, addr: TInt32) -> TInt64:
+        return cxp_readu64(addr, self.with_tag)
+
+    @kernel
+    def read_words(self, addr: TInt32, val: TList(TInt32)):
+        cxp_read_words(addr, val, self.with_tag)
+
+    @kernel
+    def write_u32(self, addr: TInt32, val: TInt32):
+        cxp_writeu32(addr, val, self.with_tag)
+
+    @kernel
+    def write_u64(self, addr: TInt32, val: TInt64):
+        cxp_writeu64(addr, val, self.with_tag)
+
+    @kernel
+    def write_wide(self, addr: TInt32, vals: TList(TInt32)):
+        for i in range(len(vals)):
+            cxp_writeu32(addr + i * 4, vals[i], self.with_tag)
+
+    @kernel
+    def read_width(self) -> TInt32:
+        return self.read_u32(self.width_addr)
+
+    @kernel
+    def read_height(self) -> TInt32:
+        return self.read_u32(self.height_addr)
+
+    @kernel
+    def read_acq_mode(self) -> TInt64:
+        return self.read_u64(self.acq_mode_addr)
+
+    @kernel
+    def write_acq_mode(self, val: TInt64):
+        self.write_u64(self.acq_mode_addr, val)
+
+    @kernel
+    def start(self):
+        self.write_u32(self.acq_start_addr, 0x00000001)
+
+    @kernel
+    def stop(self):
+        self.write_u32(self.acq_stop_addr, 0x00000001)
+
+    @kernel
+    def get_frameid(self) -> TInt32:
+        return self.read_u32(self.img_1_streamid_addr)
+
+    @kernel
+    def get_pixel_format(self) -> TInt32:
+        return self.read_u32(self.pixel_fmt_addr)
+
+    @host_only
+    def print_xml_url(self):
+        url = ""
+        for x in self.xml_url:
+            url += x.to_bytes(4, byteorder="big").decode("ascii")
+        print(f"url = {url}")
+        if "Local:" in url:
+            file_name, start_addr, size = url.split(";", 2)
+            print(
+                f"file name: {file_name.replace("Local:", "")}, start addr: 0x{start_addr}, size; 0x{size.split("?", 1)[0]} bytes"
+            )
+
+    @kernel
+    def get_xml_data(self, xml_start_addr, xml_data):
+        i = -1
+        addr_offset = 0
+        for i in range(len(xml_data) // _MAX_WORD_SIZE):
+            buf = [0] * _MAX_WORD_SIZE
+            self.read_words(xml_start_addr + addr_offset, buf)
+            for j in range(len(buf)):
+                xml_data[j+i*_MAX_WORD_SIZE] = buf[j]
+
+            addr_offset += _MAX_WORD_SIZE * 4
+
+        buf = [0]*(len(xml_data) % _MAX_WORD_SIZE)
+        self.read_words(xml_start_addr + addr_offset, buf)
+        for j in range(len(buf)):
+            xml_data[j+(i+1)*_MAX_WORD_SIZE] = buf[j]
+
+    @host_only
+    def write_xml_data(self, xml_data, file_path):
+        byte_arr = bytearray()
+        for d in xml_data:
+            byte_arr += d.to_bytes(4, "big", signed=True)
+        with open(file_path, "wb") as binary_file:
+            binary_file.write(byte_arr)
+
+
+# From the camera XML files
+_USER_SET_SELECTOR = 0x10000050
+_REAL_ACQ_MODE = 0x10000bb4
+_REAL_ACQ_START = 0x10000498
+_REAL_ACQ_STOP = 0x100004a4
+_REAL_ACQ_ABORT = 0x100004b0 #stop all acq immediately
+_BSL_SENSOR_ON = 0x100004d4 # power on sensor
+_BSL_SENSOR_STAND_BY = 0x100004c8 # put sensor in standby mode, certain parameter can only be change during standby
+_BSL_SENSOR_OFF = 0x100004bc # power down sensor
+_BSL_POWER_MODE = 0x100000b4
+# strange d_470 -> d_469 (= 3) perhaps a obscure security trick? So I guess looking @ Index"3" is fine?
+_TRIG_MODE_INDEX_3 = 0x10001424 # d_87
+_TRIG_SRC_INDEX_3 = 0x100081ac # d_479
+_TRIG_ACT_INDEX_3 = 0x1000293c # d_502
+_TRIG_SOFTWARE_INDEX_3 = 0x10000c34 # d_525
+# https://docs.baslerweb.com/test-patterns useful for testing out ROI
+_TEST_PATTERN = 0x10003500 # d_1431
+_PIXEL_FORMAT = 0x100078b4 #d_17
+
+CXP_TRIG = True
+PIXELFORMAT = "MONO8"
+FORMAT_DICT = {
+    "MONO8": 17301505,
+    "MONO10": 17825795,
+    "MONO12": 17825797,
+}
+TESTPATTERN = "WHITE"
+PATTERN_DICT = {
+    "OFF" : 0, # normal operation
+    "BLACK" : 1, # all pixels set to 0
+    "WHITE" : 2, # all pixels set to (2^N)-1
+}
+
+class IdleKernel(EnvExperiment):
+    def build(self):
+        self.setattr_device("core")
+        self.setattr_device("led0")
+
+        # declare the class before using it in kernel
+        self.cxp = CoaXPress(0x0)
+        # self.vals = [0]*0x11ab3
+
+        xml_word_size = math.ceil(0x11ab3/4)
+        self.vals = [0] * xml_word_size
+
+        self.cnt = [0]
+        self.timestamp = [np.int64(0)]
+
+    @kernel
+    def extract_camera_xml(self):
+        # DEBUG: download xml data
+        self.cxp.get_xml_data(0xc0000000, self.vals)
+        self.cxp.write_xml_data(self.vals, "genicam_16e13898.zip")
+
+    @kernel
+    def camera_init(self):
+        self.cxp.init()
+
+    @kernel
+    def camera_trigger_setup(self, pixel_format, test_pattern):
+        # DEBUG: Try to setup trigger
+        # All address below is from the XML, what's the point of bootstrap anyway?
+        # NOTE: setting is persistent over ConnectionReset but NOT power cycle
+        # self.cxp.write_u32(_REAL_ACQ_MODE, 1) # single frame mode
+        # self.cxp.write_u32(_REAL_ACQ_START, 1) # single acq start
+        # self.cxp.write_u32(_REAL_ACQ_STOP, 1) # single acq end
+        # self.cxp.write_u32(_REAL_ACQ_ABORT, 1) # single acq ABORT
+
+        # boA2448-250cm is area scan camera:
+        # see https://docs.baslerweb.com/triggered-image-acquisition#hardware-and-software-triggering to setup triggering properly
+        
+        self.cxp.write_u32(_PIXEL_FORMAT, pixel_format)
+        
+        # TEST parttern setup 
+        self.cxp.write_u32(_TEST_PATTERN, test_pattern)
+
+        # TRIGGER: setup
+        # self.cxp.write_u32(_TRIG_SELECTOR, 3) # FrameStart by default, boA xml b_469 don't have an address for some reason
+        self.cxp.write_u32(_TRIG_MODE_INDEX_3, 1) # ON
+        if CXP_TRIG:
+            self.cxp.write_u32(_TRIG_SRC_INDEX_3, 7) # CXPTrigger0
+        else:
+            self.cxp.write_u32(_TRIG_SRC_INDEX_3, 0) # use software trigger 
+        self.cxp.write_u32(_TRIG_ACT_INDEX_3, 2) # trig on anyedge
+        
+        cxp_debug_frame_print()
+
+        # TAKING PICTURE
+        self.cxp.write_u32(_REAL_ACQ_MODE, 1) # single frame mode
+        self.cxp.write_u32(_REAL_ACQ_START, 1) # single acq start
+
+        # STOP acq
+        # self.cxp.write_u32(_REAL_ACQ_STOP, 1) # single acq end
+        # self.cxp.write_u32(_REAL_ACQ_ABORT, 1) # single acq ABORT
+        
+        # self.cxp.write_u32(_BSL_SENSOR_STAND_BY, 1)
+        return self.cxp.read_u32(_BSL_POWER_MODE)
+
+    @kernel
+    def camera_trigger(self):
+        # reset mu for rtio
+        self.core.reset()
+        self.core.break_realtime()
+
+        self.cxp.setup_roi(0, 1, 1, 5, 5)
+        delay_mu(1000000)
+        
+        if CXP_TRIG:
+            self.cxp.trigger(0 ,0x00)
+        else:
+            self.cxp.write_u32(_TRIG_SOFTWARE_INDEX_3, 0) # software trigger via register write
+
+        delay_mu(100)
+        
+        # self.cxp.write_u32(_REAL_ACQ_STOP, 1) # single acq end
+        # self.cxp.write_u32(_REAL_ACQ_ABORT, 1) # single acq ABORT
+        # self.cxp.write_u32(_BSL_SENSOR_STAND_BY, 1)
+         
+        # NOTE: This may not print when using CXP hardware TRIG
+        # As the write_u32 trigger a packet printout that delays the CPU enough that the frame arrive
+        # But using hw trigger, the print is not necessory i.e. not enough time delay for the zc706 to receive the frame
+        # cxp_debug_frame_print()
+
+        self.cxp.input_mu(self.cnt, self.timestamp, 10000)
+
+        for _ in range(4):
+            cxp_debug_frame_print()
+        
+        # reduce power draw & temperature
+        # overtemperature can cause unstable connection
+        self.cxp.write_u32(_BSL_SENSOR_OFF, 1) 
+
+        return self.cxp.read_u32(_BSL_POWER_MODE)
+
+    def print_hex(self, arr: TList(TInt32)):
+        print("[{}]".format(", ".join(hex(np.uint32(x)) for x in arr)))
+
+    def run(self):
+        self.camera_init()
+        print(f"power mode before trigger = {self.camera_trigger_setup(FORMAT_DICT[PIXELFORMAT], PATTERN_DICT[TESTPATTERN])}")
+        print(f"power mode after trigger = {self.camera_trigger()}")
+        print(f"count = {np.uint32(self.cnt)} | timestamp = {self.timestamp}")
+
+        
+        # self.cxp.print_xml_url()
+        # self.extract_camera_xml()

cxp_note.md

@@ -0,0 +1,78 @@
+# CXP
+
+## Finished
+- Upconn - Low speed serial 
+  [x] Low speed serial PHY
+    [x] 20.833Mbps & 41.666Mbps change
+    [x] 8b10b encoder
+  [x] TX Pipeline with priority transmission
+    [x] Trigger
+    [x] Trigger ack
+    [x] Test & Ctrl packet with DMA
+  [x] CTRL Packet serialize firmware
+    [x] follow DRTIO DMA
+    [x] check crc
+- Downconn - GTX
+  [x] GTX serial PHY
+    [x] QPLL & GTX DRP to config linerate
+    [x] Comma checker & restart rx
+  [x] RX Pipeline with priority decoder
+    [x] Trigger ack 
+    [x] CTRL packet DMA with extra buffer
+    [x] Connection test sequence checker
+  [x] CTRL Packet deserialize firmware
+    [x] follow DRTIO DMA
+    [x] check crc
+  [x] GTX Multilane setup
+- Camera boostrap
+  [x] get the CXP version
+  [x] test connection
+  [x] discovery other extension (links)
+  [x] set bitrate
+- Camera frame pipeline
+  [x] CXP frame packet routing (maybe no need to routing non zero streaming id (we have ROI buildin anyways)?)
+  [x] CXP CRC32 detection
+
+
+## TODO
+[] remove ALL debug tools
+  [] flake.nix mod
+  [] local_run.sh mod
+### Gateware
+[x] rename word_dw to word_width
+[x] Test out CXP trigger
+[] add __init__.py for cxp??
+[] Try to fix tight s/h time pins
+[] refactor error_cnt 
+[] Heartbeat (is it useful?? lol)
+[] rename circular buffer to slots
+[] remove pmod/debug_sma in fns args
+[] add enum for gtx mode (e.g. tx only, rx only, both)
+   [] use if tx_mode / rx_mode instead
+[] Region of interest engine
+   [x] pixel gearbox
+   [] pixel parser (xy pos)
+[] rtio to getting the frame
+  - O: trigger
+  - I: frame
+[] add a packet parser module that mux the packet?
+
+### Firmware (design with driver)
+[] Camera linkdown detection
+[] Camera auto linkup/linkdown using threads
+[] API programming
+[] add tag handling for api calls
+  - support line reset in kernel using syscall
+[] add heartbeat checking
+
+### Coredevice Driver
+[] support simple camera programming interface (Not real time)
+  - basic i2c-like interface with read/write u32
+[] add grabber like fns & docs
+[] use camera test pattern black/white to verify roi https://docs.baslerweb.com/test-patterns
+
+
+### PR
+1. push the gtx init fix
+2. push the cxp core to misoc
+3. push the cxp rtio core to artiq-zynq & artiq

device_db.py

@@ -0,0 +1,30 @@
+device_db = {
+    "core": {
+        "type": "local",
+        "module": "artiq.coredevice.core",
+        "class": "Core",
+        "arguments": {
+            "host": "192.168.1.14",
+            "ref_period": 1e-9,
+            "ref_multiplier": 8,
+            "target": "cortexa9"
+        }
+    },
+    "core_cache": {
+        "type": "local",
+        "module": "artiq.coredevice.cache",
+        "class": "CoreCache"
+    },
+    "core_dma": {
+        "type": "local",
+        "module": "artiq.coredevice.dma",
+        "class": "CoreDMA"
+    },
+
+    "led0": {
+        "type": "local",
+        "module": "artiq.coredevice.ttl",
+        "class": "TTLOut",
+        "arguments": {"channel": 0x000001},
+    },
+}

examples/ebaz4205/device_db.py

@@ -0,0 +1,78 @@
+core_addr = "192.168.1.57"
+
+device_db = {
+    "core": {
+        "type": "local",
+        "module": "artiq.coredevice.core",
+        "class": "Core",
+        "arguments": {
+            "host": core_addr,
+            "ref_period": 1e-9,
+            "target": "cortexa9",
+        },
+    },
+    "core_log": {
+        "type": "controller",
+        "host": "::1",
+        "port": 1068,
+        "command": "aqctl_corelog -p {port} --bind {bind} " + core_addr,
+    },
+    "core_moninj": {
+        "type": "controller",
+        "host": "::1",
+        "port_proxy": 1383,
+        "port": 1384,
+        "command": "aqctl_moninj_proxy --port-proxy {port_proxy} --port-control {port} --bind {bind} "
+        + core_addr,
+    },
+    "core_analyzer": {
+        "type": "controller",
+        "host": "::1",
+        "port_proxy": 1385,
+        "port": 1386,
+        "command": "aqctl_coreanalyzer_proxy --port-proxy {port_proxy} --port-control {port} --bind {bind} "
+        + core_addr,
+    },
+    "core_cache": {
+        "type": "local",
+        "module": "artiq.coredevice.cache",
+        "class": "CoreCache",
+    },
+    "core_dma": {"type": "local", "module": "artiq.coredevice.dma", "class": "CoreDMA"},
+    "led0": {
+        "type": "local",
+        "module": "artiq.coredevice.ttl",
+        "class": "TTLOut",
+        "arguments": {"channel": 0},
+    },
+    "led1": {
+        "type": "local",
+        "module": "artiq.coredevice.ttl",
+        "class": "TTLOut",
+        "arguments": {"channel": 1},
+    },
+}
+
+# TTLs starting at RTIO channel 2, ending at RTIO channel 15
+for i in range(2, 16):
+    device_db["ttl" + str(i)] = {
+        "type": "local",
+        "module": "artiq.coredevice.ttl",
+        "class": "TTLInOut",
+        "arguments": {"channel": i},
+    }
+
+device_db.update(
+    spi0={
+        "type": "local",
+        "module": "artiq.coredevice.spi2",
+        "class": "SPIMaster",
+        "arguments": {"channel": 16},
+    },
+    dds0={
+        "type": "local",
+        "module": "artiq.coredevice.ad9834",
+        "class": "AD9834",
+        "arguments": {"spi_device": "spi0"},
+    },
+)

flake.lock

@@ -3,19 +3,19 @@
     "artiq": {
       "inputs": {
         "artiq-comtools": "artiq-comtools",
-        "mozilla-overlay": "mozilla-overlay",
         "nixpkgs": "nixpkgs",
+        "rust-overlay": "rust-overlay",
         "sipyco": "sipyco",
         "src-migen": "src-migen",
         "src-misoc": "src-misoc",
         "src-pythonparser": "src-pythonparser"
       },
       "locked": {
-        "lastModified": 1716972728,
-        "narHash": "sha256-88J+eckZamtwhcCQkPpKLu6R1hmgj5+C9n2U5i+sHUE=",
+        "lastModified": 1736946744,
+        "narHash": "sha256-RKqrWcJfkLlm5JYVfz46KOVg1FYch1pNkKDpW5VzehU=",
         "ref": "refs/heads/master",
-        "rev": "49e402780bebba437c6098047ab1dc68eaf5a17c",
-        "revCount": 8808,
+        "rev": "33c91d73bb768a06fa427c237b124916261c5ab9",
+        "revCount": 9135,
         "type": "git",
         "url": "https://github.com/m-labs/artiq.git"
       },
@@ -37,11 +37,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1707216368,
-        "narHash": "sha256-ZXoqzG2QsVsybALLYXs473avXcyKSZNh2kIgcPo60XQ=",
+        "lastModified": 1734270714,
+        "narHash": "sha256-7bzGn/hXLIsLQHGQsvo+uoIFUrw9DjXSlMC449BY4ME=",
         "owner": "m-labs",
         "repo": "artiq-comtools",
-        "rev": "e5d0204490bccc07ef9141b0d7c405ab01cb8273",
+        "rev": "7e3152314af8f5987370e33b347b2ec2697567ed",
         "type": "github"
       },
       "original": {
@@ -55,11 +55,11 @@
         "systems": "systems"
       },
       "locked": {
-        "lastModified": 1694529238,
-        "narHash": "sha256-zsNZZGTGnMOf9YpHKJqMSsa0dXbfmxeoJ7xHlrt+xmY=",
+        "lastModified": 1710146030,
+        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
         "owner": "numtide",
         "repo": "flake-utils",
-        "rev": "ff7b65b44d01cf9ba6a71320833626af21126384",
+        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
         "type": "github"
       },
       "original": {
@@ -68,66 +68,18 @@
         "type": "github"
       }
     },
-    "mozilla-overlay": {
-      "flake": false,
-      "locked": {
-        "lastModified": 1704373101,
-        "narHash": "sha256-+gi59LRWRQmwROrmE1E2b3mtocwueCQqZ60CwLG+gbg=",
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "rev": "9b11a87c0cc54e308fa83aac5b4ee1816d5418a2",
-        "type": "github"
-      },
-      "original": {
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "type": "github"
-      }
-    },
-    "mozilla-overlay_2": {
-      "flake": false,
-      "locked": {
-        "lastModified": 1704373101,
-        "narHash": "sha256-+gi59LRWRQmwROrmE1E2b3mtocwueCQqZ60CwLG+gbg=",
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "rev": "9b11a87c0cc54e308fa83aac5b4ee1816d5418a2",
-        "type": "github"
-      },
-      "original": {
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "type": "github"
-      }
-    },
-    "mozilla-overlay_3": {
-      "flake": false,
-      "locked": {
-        "lastModified": 1704373101,
-        "narHash": "sha256-+gi59LRWRQmwROrmE1E2b3mtocwueCQqZ60CwLG+gbg=",
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "rev": "9b11a87c0cc54e308fa83aac5b4ee1816d5418a2",
-        "type": "github"
-      },
-      "original": {
-        "owner": "mozilla",
-        "repo": "nixpkgs-mozilla",
-        "type": "github"
-      }
-    },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1716542732,
-        "narHash": "sha256-0Y9fRr0CUqWT4KgBITmaGwlnNIGMYuydu2L8iLTfHU4=",
+        "lastModified": 1736798957,
+        "narHash": "sha256-qwpCtZhSsSNQtK4xYGzMiyEDhkNzOCz/Vfu4oL2ETsQ=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "d12251ef6e8e6a46e05689eeccd595bdbd3c9e60",
+        "rev": "9abb87b552b7f55ac8916b6fc9e5cb486656a2f3",
         "type": "github"
       },
       "original": {
         "owner": "NixOS",
-        "ref": "nixos-24.05",
+        "ref": "nixos-unstable",
         "repo": "nixpkgs",
         "type": "github"
       }
@@ -135,10 +87,53 @@
     "root": {
       "inputs": {
         "artiq": "artiq",
-        "mozilla-overlay": "mozilla-overlay_2",
         "zynq-rs": "zynq-rs"
       }
     },
+    "rust-overlay": {
+      "inputs": {
+        "nixpkgs": [
+          "artiq",
+          "nixpkgs"
+        ]
+      },
+      "locked": {
+        "lastModified": 1719454714,
+        "narHash": "sha256-MojqG0lyUINkEk0b3kM2drsU5vyaF8DFZe/FAlZVOGs=",
+        "owner": "oxalica",
+        "repo": "rust-overlay",
+        "rev": "d1c527659cf076ecc4b96a91c702d080b213801e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "oxalica",
+        "ref": "snapshot/2024-08-01",
+        "repo": "rust-overlay",
+        "type": "github"
+      }
+    },
+    "rust-overlay_2": {
+      "inputs": {
+        "nixpkgs": [
+          "zynq-rs",
+          "nixpkgs"
+        ]
+      },
+      "locked": {
+        "lastModified": 1719454714,
+        "narHash": "sha256-MojqG0lyUINkEk0b3kM2drsU5vyaF8DFZe/FAlZVOGs=",
+        "owner": "oxalica",
+        "repo": "rust-overlay",
+        "rev": "d1c527659cf076ecc4b96a91c702d080b213801e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "oxalica",
+        "ref": "snapshot/2024-08-01",
+        "repo": "rust-overlay",
+        "type": "github"
+      }
+    },
     "sipyco": {
       "inputs": {
         "nixpkgs": [
@@ -147,11 +142,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1708937641,
-        "narHash": "sha256-Hkb9VYFzFgkYxfbh4kYcDSn7DbMUYehoQDeTALrxo2Q=",
+        "lastModified": 1734267097,
+        "narHash": "sha256-aWg7XDiOlWnkXfDbKrBn9ITR46/JXfndvYHxFJ1vN78=",
         "owner": "m-labs",
         "repo": "sipyco",
-        "rev": "4a28b311ce0069454b4e8fe1e6049db11b9f1296",
+        "rev": "430978ada3fefe32de01f1b884b3031e48aaef96",
         "type": "github"
       },
       "original": {
@@ -163,11 +158,11 @@
     "src-migen": {
       "flake": false,
       "locked": {
-        "lastModified": 1715484909,
-        "narHash": "sha256-4DCHBUBfc/VA+7NW2Hr0+JP4NnKPru2uVJyZjCCk0Ws=",
+        "lastModified": 1735131698,
+        "narHash": "sha256-P4vaF+9iVekRAC2/mc9G7IwI6baBpPAxiDQ8uye4sAs=",
         "owner": "m-labs",
         "repo": "migen",
-        "rev": "4790bb577681a8c3a8d226bc196a4e5deb39e4df",
+        "rev": "4c2ae8dfeea37f235b52acb8166f12acaaae4f7c",
         "type": "github"
       },
       "original": {
@@ -179,11 +174,11 @@
     "src-misoc": {
       "flake": false,
       "locked": {
-        "lastModified": 1715647536,
-        "narHash": "sha256-q+USDcaKHABwW56Jzq8u94iGPWlyLXMyVt0j/Gyg+IE=",
+        "lastModified": 1736416570,
+        "narHash": "sha256-tbcN/fzejZIaYbTbwk8Ir1glYevESqMinMeDB3z8oxg=",
         "ref": "refs/heads/master",
-        "rev": "fea9de558c730bc394a5936094ae95bb9d6fa726",
-        "revCount": 2455,
+        "rev": "1f5318e9edc1085ac77e9b85b8f5e03371dba54c",
+        "revCount": 2464,
         "submodules": true,
         "type": "git",
         "url": "https://github.com/m-labs/misoc.git"
@@ -227,18 +222,18 @@
     },
     "zynq-rs": {
       "inputs": {
-        "mozilla-overlay": "mozilla-overlay_3",
         "nixpkgs": [
           "artiq",
           "nixpkgs"
-        ]
+        ],
+        "rust-overlay": "rust-overlay_2"
       },
       "locked": {
-        "lastModified": 1716519432,
-        "narHash": "sha256-vgKBJCQRPCutJ4n+FtJNczMZULWW7J3B8icf/PUothw=",
+        "lastModified": 1734668221,
+        "narHash": "sha256-X0U2yPmlsD3VLBZQyfWv8qw04Qn0qFWIONJUPPigB0U=",
         "ref": "refs/heads/master",
-        "rev": "46dc25b89e46b9043129d77e3c9348916748e325",
-        "revCount": 645,
+        "rev": "213529cf7a50aa1b2d9ffdf575e3e38202ff9bd6",
+        "revCount": 666,
         "type": "git",
         "url": "https://git.m-labs.hk/m-labs/zynq-rs"
       },

flake.nix

@@ -2,27 +2,26 @@
   description = "ARTIQ port to the Zynq-7000 platform";
 
   inputs.artiq.url = git+https://github.com/m-labs/artiq.git;
-  inputs.mozilla-overlay = { url = github:mozilla/nixpkgs-mozilla; flake = false; };
   inputs.zynq-rs.url = git+https://git.m-labs.hk/m-labs/zynq-rs;
   inputs.zynq-rs.inputs.nixpkgs.follows = "artiq/nixpkgs";
 
-  outputs = { self, mozilla-overlay, zynq-rs, artiq }:
+  outputs = { self, zynq-rs, artiq }:
   let
-    pkgs = import artiq.inputs.nixpkgs { system = "x86_64-linux"; overlays = [ (import mozilla-overlay) ]; };
+    pkgs = import artiq.inputs.nixpkgs { system = "x86_64-linux"; overlays = [ (import zynq-rs.inputs.rust-overlay) ]; };
     zynqpkgs = zynq-rs.packages.x86_64-linux;
     artiqpkgs = artiq.packages.x86_64-linux;
-    llvmPackages_11 = zynq-rs.llvmPackages_11;
+    zynqRev = self.sourceInfo.rev or "unknown";
 
     rust = zynq-rs.rust;
     rustPlatform = zynq-rs.rustPlatform;
 
     fastnumbers = pkgs.python3Packages.buildPythonPackage rec {
       pname = "fastnumbers";
-      version = "2.2.1";
+      version = "5.1.0";
 
       src = pkgs.python3Packages.fetchPypi {
         inherit pname version;
-        sha256 = "0j15i54p7nri6hkzn1wal9pxri4pgql01wgjccig6ar0v5jjbvsy";
+        sha256 = "sha256-4JLTP4uVwxcaL7NOV57+DFSwKQ3X+W/6onYkN2AdkKc=";
       };
     };
 
@@ -75,7 +74,7 @@
 
       propagatedBuildInputs = with pkgs.python3Packages; [ setuptools click numpy toolz jinja2 ramda artiqpkgs.migen artiqpkgs.misoc ];
 
-      checkInputs = with pkgs.python3Packages; [ pytest-runner pytestCheckHook pytest-timeout ];
+      checkInputs = with pkgs.python3Packages; [ pytestCheckHook pytest-timeout ];
 
       # migen/misoc version checks are broken with pyproject for some reason
       postPatch = ''
@@ -126,6 +125,9 @@
           lockFile = src/Cargo.lock;
           outputHashes = {
               "tar-no-std-0.1.8" = "sha256-xm17108v4smXOqxdLvHl9CxTCJslmeogjm4Y87IXFuM=";
+              "nalgebra-0.32.6" = "sha256-ZbQQZbM3A5cJ4QbujtUxkrI0/qGlI4UzfahtyQnvMZA=";
+              "core_io-0.1.0" = "sha256-0HINFWRiJx8pjMgUOL/CS336ih7SENSRh3Kah9LPRrw=";
+              "fatfs-0.3.6" = "sha256-Nz9hCq/1YgSXF8ltJ5ZawV0Hc8WV44KNK0tJdVnNb4U=";
             };
         };
 
@@ -133,12 +135,13 @@
           pkgs.gnumake
           (pkgs.python3.withPackages(ps: [ ps.jsonschema artiqpkgs.migen migen-axi artiqpkgs.misoc artiqpkgs.artiq ]))
           zynqpkgs.cargo-xbuild
-          llvmPackages_11.llvm
-          llvmPackages_11.clang-unwrapped
+          pkgs.llvmPackages_13.llvm
+          pkgs.llvmPackages_13.clang-unwrapped
         ];
         buildPhase = ''
+          export ZYNQ_REV=${zynqRev}
           export XARGO_RUST_SRC="${rust}/lib/rustlib/src/rust/library"
-          export CLANG_EXTRA_INCLUDE_DIR="${llvmPackages_11.clang-unwrapped.lib}/lib/clang/11.1.0/include"
+          export CLANG_EXTRA_INCLUDE_DIR="${pkgs.llvmPackages_13.clang-unwrapped.lib}/lib/clang/13.0.1/include"
           export CARGO_HOME=$(mktemp -d cargo-home.XXX)
           export ZYNQ_RS=${zynq-rs}
           make TARGET=${target} GWARGS="${if json == null then "-V ${variant}" else json}" ${fwtype}
@@ -164,6 +167,7 @@
           ];
         }
         ''
+          export ZYNQ_REV=${zynqRev}
           python ${./src/gateware}/${target}.py -g build ${if json == null then "-V ${variant}" else json}
           mkdir -p $out $out/nix-support
           cp build/top.bit $out
@@ -341,6 +345,7 @@
       {
         inherit fastnumbers artiq-netboot ramda migen-axi binutils-arm;
       } //
+      (board-package-set { target = "zc706"; variant = "cxp_demo"; }) //
       (board-package-set { target = "zc706"; variant = "nist_clock"; }) //
       (board-package-set { target = "zc706"; variant = "nist_clock_master"; }) //
       (board-package-set { target = "zc706"; variant = "nist_clock_master_100mhz"; }) //
@@ -363,7 +368,8 @@
       (board-package-set { target = "zc706"; variant = "acpki_nist_qc2_satellite_100mhz"; }) //
       (board-package-set { target = "kasli_soc"; variant = "demo"; json = ./demo.json; }) //
       (board-package-set { target = "kasli_soc"; variant = "master"; json = ./kasli-soc-master.json; }) //
-      (board-package-set { target = "kasli_soc"; variant = "satellite"; json = ./kasli-soc-satellite.json; });
+      (board-package-set { target = "kasli_soc"; variant = "satellite"; json = ./kasli-soc-satellite.json; }) //
+      (board-package-set { target = "ebaz4205"; variant = "base"; });
 
     hydraJobs = packages.x86_64-linux // { inherit zc706-hitl-tests; inherit gateware-sim; inherit fmt-check; };
 
@@ -371,8 +377,8 @@
       name = "artiq-zynq-dev-shell";
       buildInputs = with pkgs; [
         rust
-        llvmPackages_11.llvm
-        llvmPackages_11.clang-unwrapped
+        llvmPackages_13.llvm
+        llvmPackages_13.clang-unwrapped
         gnumake
         cacert
         zynqpkgs.cargo-xbuild
@@ -383,9 +389,11 @@
         artiqpkgs.artiq
         artiqpkgs.vivado
         binutils-arm
+        pre-commit
       ];
+      ZYNQ_REV="${zynqRev}";
       XARGO_RUST_SRC = "${rust}/lib/rustlib/src/rust/library";
-      CLANG_EXTRA_INCLUDE_DIR = "${llvmPackages_11.clang-unwrapped.lib}/lib/clang/11.1.0/include";
+      CLANG_EXTRA_INCLUDE_DIR = "${pkgs.llvmPackages_13.clang-unwrapped.lib}/lib/clang/13.0.1/include";
       ZYNQ_RS = "${zynq-rs}";
       OPENOCD_ZYNQ = "${zynq-rs}/openocd";
       SZL = "${zynqpkgs.szl}";

local_run.sh

@@ -13,7 +13,7 @@ fi
 
 impure=0
 load_bitstream=1
-board_type="kasli_soc"
+board_type="zc706"
 fw_type="runtime"
 
 while getopts "ilb:t:f:" opt; do
@@ -36,7 +36,7 @@ done
 if [ -z "$board_host" ]; then
     case $board_type in
     kasli_soc) board_host="192.168.1.56";;
-    zc706) board_host="192.168.1.52";;
+    zc706) board_host="192.168.1.14";;
     *) echo "Unknown board type"; exit 1;;
     esac
 fi
@@ -58,4 +58,4 @@ else
         load_bitstream_cmd="-g $result_dir/top.bit"
     fi
     artiq_netboot $load_bitstream_cmd -f $result_dir/$fw_type.bin -b $board_host
-fi
+fi

notes.drawio

@@ -0,0 +1,82 @@
+<mxfile host="65bd71144e">
+    <diagram id="en7HUHNV3kVsTTCxeEt8" name="Page-1">
+        <mxGraphModel dx="924" dy="545" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="0" pageScale="1" pageWidth="850" pageHeight="1100" background="none" math="0" shadow="0">
+            <root>
+                <mxCell id="0"/>
+                <mxCell id="1" parent="0"/>
+                <mxCell id="50" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="280" y="200" as="sourcePoint"/>
+                        <mxPoint x="440" y="200" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="52" value="Payload (layout)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="50">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="-63" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="53" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="280" y="239.81" as="sourcePoint"/>
+                        <mxPoint x="440" y="239.81" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="54" value="stb (valid/readable)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="53">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="-63" y="-1" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="55" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="440" y="279.80999999999995" as="sourcePoint"/>
+                        <mxPoint x="280" y="279.80999999999995" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="56" value="ack (ready)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="55">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="-197" y="1" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="48" value="Source" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
+                    <mxGeometry x="320" y="160" width="80" height="160" as="geometry"/>
+                </mxCell>
+                <mxCell id="57" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="520" y="200" as="sourcePoint"/>
+                        <mxPoint x="680" y="200" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="58" value="Payload (layout)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="57">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="197" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="59" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="520" y="239.80999999999995" as="sourcePoint"/>
+                        <mxPoint x="680" y="239.80999999999995" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="60" value="stb (valid)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="59">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="197" y="-1" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="61" value="" style="shape=flexArrow;endArrow=classic;html=1;fillColor=#647687;strokeColor=#314354;" edge="1" parent="1">
+                    <mxGeometry width="50" height="50" relative="1" as="geometry">
+                        <mxPoint x="680" y="279.99999999999994" as="sourcePoint"/>
+                        <mxPoint x="520" y="279.99999999999994" as="targetPoint"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="62" value="ack (ready/writeable)" style="edgeLabel;html=1;align=center;verticalAlign=middle;resizable=0;points=[];" vertex="1" connectable="0" parent="61">
+                    <mxGeometry x="-0.8404" y="-1" relative="1" as="geometry">
+                        <mxPoint x="73" y="1" as="offset"/>
+                    </mxGeometry>
+                </mxCell>
+                <mxCell id="49" value="Sink" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="1">
+                    <mxGeometry x="560" y="160" width="80" height="160" as="geometry"/>
+                </mxCell>
+            </root>
+        </mxGraphModel>
+    </diagram>
+</mxfile>

reset.py

@@ -0,0 +1,17 @@
+from time import sleep
+from pyftdi.ftdi import Ftdi
+
+POR = 1 << 7
+
+def main():
+    dev = Ftdi()
+    dev.open_bitbang_from_url("ftdi://ftdi:4232h/0")
+    dev.set_bitmode(POR, Ftdi.BitMode.BITBANG)
+    dev.write_data(bytes([0]))
+    sleep(0.1)
+    dev.write_data(bytes([POR]))
+    sleep(0.1)
+    dev.close()
+
+if __name__ == "__main__":
+    main()

sat_run.sh

@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+
+python reset.py
+
+set -e
+
+if [ -z "$OPENOCD_ZYNQ" ]; then
+    echo "OPENOCD_ZYNQ environment variable must be set"
+    exit 1
+fi
+if [ -z "$SZL" ]; then
+    echo "SZL environment variable must be set"
+    exit 1
+fi
+
+impure=0
+load_bitstream=1
+board_type="kasli_soc"
+fw_type="satman"
+
+while getopts "ilb:t:f:" opt; do
+    case "$opt" in
+    \?) exit 1
+        ;;
+    i)  impure=1
+        ;;
+    l)  load_bitstream=0
+        ;;
+    b)  board_host=$OPTARG
+        ;;
+    t)  board_type=$OPTARG
+        ;;
+    f)  fw_type=$OPTARG
+        ;;
+    esac
+done
+
+if [ -z "$board_host" ]; then
+    case $board_type in
+    kasli_soc) board_host="192.168.1.56";;
+    zc706) board_host="192.168.1.52";;
+    *) echo "Unknown board type"; exit 1;;
+    esac
+fi
+
+load_bitstream_cmd=""
+
+build_dir=`pwd`/build
+result_dir=`pwd`/result
+cd $OPENOCD_ZYNQ
+openocd -f $board_type.cfg -c "load_image $SZL/szl-$board_type.elf; resume 0; exit"
+sleep 5
+if [ $impure -eq 1 ]; then
+    if [ $load_bitstream -eq 1 ]; then
+        load_bitstream_cmd="-g $build_dir/gateware/top.bit"
+    fi
+    artiq_netboot $load_bitstream_cmd -f $build_dir/$fw_type.bin -b $board_host
+else
+    if [ $load_bitstream -eq 1 ]; then
+        load_bitstream_cmd="-g $result_dir/top.bit"
+    fi
+    artiq_netboot $load_bitstream_cmd -f $result_dir/$fw_type.bin -b $board_host
+fi

sim.py

@@ -0,0 +1,55 @@
+from migen import *
+from misoc.interconnect.csr import *
+
+from functools import reduce
+from itertools import combinations
+from operator import or_, and_
+
+class Voter(Module):
+    def __init__(self):
+        self.data_4x = Signal(32)
+        self.k_4x = Signal(4)
+        
+        # Section 9.2.2.1 (CXP-001-2021)
+        # decoder should immune to single bit errors when handling duplicated characters
+        self.char = Signal(8)
+        self.k = Signal()
+
+
+        # majority voting
+        char = [[self.data_4x[i*8:(i+1)*8], self.k_4x[i]] for i in range(4)]
+        voter = [Record([("data", 8), ("k", 1)]) for _ in range(4)]
+
+        # stage 1
+        for i, code in enumerate(combinations(char, 3)):
+            self.sync += [
+                voter[i].data.eq(reduce(and_, [c[0] for c in code])),
+                voter[i].k.eq(reduce(and_, [c[1] for c in code])),
+            ]
+
+        # stage 2
+        self.sync += [
+            self.char.eq(reduce(or_, [v.data for v in voter])),
+            self.k.eq(reduce(or_, [v.k for v in voter])),
+        ]
+
+        
+
+
+dut = Voter()
+def check_case(data_4x, k_4x, char, k):
+  yield dut.data_4x.eq(data_4x)
+  yield dut.k_4x.eq(k_4x)
+  yield
+  yield
+  yield
+  print(f"char = {yield dut.char:#X} k = {yield dut.k:#X}")
+  assert (yield dut.char) == char and (yield dut.k) == k
+
+def testbench():
+  yield from check_case(0xFFFFFFFF, 0b1111, 0xFF, 1)
+  yield from check_case(0xFFFFFF00, 0b1110, 0xFF, 1)
+  yield from check_case(0xFFFFF00f, 0b0001, 0xFF, 0)
+  yield from check_case(0xFFFFFFFF, 0b1111, 0xFF, 1)
+
+run_simulation(dut, testbench())

sim_4xgearbox.py

@@ -0,0 +1,92 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_router import *
+
+class DUT(Module):
+    def __init__(self):
+        self.submodules.gearbox = gearbox = Stream_Packet_Gearbox()
+        self.sink, self.source = gearbox.sink, gearbox.source
+
+        self.comb += self.source.ack.eq(1)
+
+
+dut = DUT()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+    source = dut.source
+
+    for i, p in enumerate(packets):
+        yield sink.data.eq(p["data"])
+        yield sink.k.eq(p["k"])
+
+        if "stb_break" in p:
+            yield sink.stb.eq(0)
+        else:
+            yield sink.stb.eq(1)
+
+        if "eop" in p:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+        yield
+
+    for _ in range(10):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+
+        yield source.ack.eq(1)
+        yield
+
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # stream id
+        {"data": C(0x00000001, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x6AEFACF6, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0, word_width), "k" : Replicate(0, 4), "stb_break":0}, # cyc break
+
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)},  
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # Xsize[23:16]
+        {"data": C(0x09090909, word_width), "k" : Replicate(0, 4)}, # Xsize[15:8] 
+        {"data": C(0x90909090, word_width), "k" : Replicate(0, 4)}, # Xsize[7:0] 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x8EE1DAA1, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0, word_width), "k" : Replicate(0, 4), "stb_break":0}, # cyc break
+
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # DsizeL[23:16] 
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)}, # DsizeL[15:8] 
+        {"data": C(0x64646464, word_width), "k" : Replicate(0, 4)}, # DsizeL[7:0] 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x51C243EA, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+    ]
+
+    
+    yield from packet_sim(paks)
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_arbiter.py

@@ -0,0 +1,149 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from sim_generator import CXPCRC32Inserter
+from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+CXP_CHANNELS = 2
+
+
+class Frame_Pipeline(Module):
+    def __init__(self, n_downconn):
+        # to construct correct crc and ack/stb signal
+        self.submodules.arbiter = arbiter = Stream_Arbiter(n_downconn)
+        self.submodules.broadcaster = broadcaster = Stream_Broadcaster(1)
+        self.submodules.buffer = buffer = Buffer(word_layout_dchar)
+        self.sinks = []
+
+        for i in range(n_downconn):
+            # generating the packet
+            dchar_decoder = Duplicated_Char_Decoder()
+            eop_marker = EOP_Marker()
+
+            pipeline = [dchar_decoder, eop_marker]
+            self.submodules += pipeline
+
+            for s, d in zip(pipeline, pipeline[1:]):
+                self.comb += s.source.connect(d.sink)
+            self.sinks.append(pipeline[0].sink)
+
+            self.comb += pipeline[-1].source.connect(arbiter.sinks[i])
+
+        # self.comb += arbiter.source.ack.eq(1)
+         
+        self.comb += [
+            arbiter.source.connect(broadcaster.sink),
+            broadcaster.sources[0].connect(buffer.sink),
+        ]
+        self.comb += buffer.source.ack.eq(1)
+
+
+
+dut = Frame_Pipeline(CXP_CHANNELS)
+
+
+def packet_sim(packets=[], active_ch=2):
+    assert active_ch <= CXP_CHANNELS
+
+    print("=================TEST========================")
+    # yield dut.arbiter.active_channels.eq((2**active_ch) - 1)
+    yield dut.arbiter.active_channels.eq((2**active_ch) - 1)
+    sinks = dut.sinks
+
+    for p in packets:
+        for i in range(CXP_CHANNELS):
+            if p["sink_id"] == i:
+                yield sinks[p["sink_id"]].data.eq(p["data"])
+                yield sinks[p["sink_id"]].k.eq(p["k"])
+                yield sinks[p["sink_id"]].stb.eq(1)
+                if "eop" in p:
+                    yield sinks[p["sink_id"]].eop.eq(1)
+                else:
+                    yield sinks[p["sink_id"]].eop.eq(0)
+            else:
+                yield sinks[i].data.eq(0)
+                yield sinks[i].k.eq(0)
+                yield sinks[i].stb.eq(0)
+                yield sinks[i].eop.eq(0)
+        yield
+
+    # extra clk cycles
+    for _ in range(100):
+        for i in range(CXP_CHANNELS):
+            yield sinks[i].data.eq(0)
+            yield sinks[i].k.eq(0)
+            yield sinks[i].stb.eq(0)
+            yield sinks[i].eop.eq(0)
+            yield
+    assert True
+
+
+def testbench(n_downconn, with_header=True, with_crc=True):
+    paks = []
+
+    stream_id = 0
+    pix_len = 10
+    for p in range(20):
+        n_conn = p % n_downconn
+        if with_header:
+            frame_pak_len = pix_len if with_crc else pix_len - 1
+            frame = [
+                {
+                    "sink_id": n_conn,
+                    "data": Replicate(C(stream_id, char_width), 4),
+                    "k": Replicate(0, 4),
+                },
+                {
+                    "sink_id": n_conn,
+                    "data": Replicate(C(0, char_width), 4),
+                    "k": Replicate(0, 4),
+                },
+                {
+                    "sink_id": n_conn,
+                    "data": Replicate(C(frame_pak_len, 2 * char_width)[8:], 4),
+                    "k": Replicate(0, 4),
+                },
+                {
+                    "sink_id": n_conn,
+                    "data": Replicate(C(frame_pak_len, 2 * char_width)[:8], 4),
+                    "k": Replicate(0, 4),
+                },
+            ]
+        else:
+            frame = []
+
+        for n_pix in range(pix_len):
+            if n_pix < pix_len - 1:
+                frame.append(
+                    {
+                        "sink_id": n_conn,
+                        # "data": C((n_conn + 1) << 28 | p << 24 | n_pix, word_width),
+                        "data": C( p << 24 | n_pix, word_width),
+                        "k": Replicate(0, 4),
+                    }
+                )
+            else:
+                frame.append(
+                    {
+                        "sink_id": n_conn,
+                        # "data": C((n_conn + 1) << 28 | p << 24 | n_pix, word_width),
+                        "data": C( p << 24 | n_pix, word_width),
+                        "k": Replicate(0, 4),
+                    }
+                )
+                frame.append(
+                    {
+                        "sink_id": n_conn,
+                        "data": Replicate(KCode["pak_end"], 4),
+                        "k": Replicate(1, 4),
+                    }
+                )
+
+        paks += frame
+
+    yield from packet_sim(paks)
+
+
+run_simulation(dut, testbench(CXP_CHANNELS,with_crc=False), vcd_name="sim-cxp.vcd")

sim_broadcaster.py

@@ -0,0 +1,91 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from sim_generator import CXPCRC32Inserter
+from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+class double_buffer_pipeline(Module):
+    def __init__(self):
+        fifo = stream.SyncFIFO(word_layout, 32)
+        dchar_decoder = Duplicated_Char_Decoder()
+        broadcaster = Stream_Broadcaster(1)
+        pipeline = [fifo, dchar_decoder, broadcaster]
+        self.submodules += pipeline
+
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        self.sink = pipeline[0].sink
+
+        self.submodules.buffer = buffer = Buffer(word_layout_dchar)
+        self.comb += broadcaster.sources[0].connect(buffer.sink)
+        self.source = buffer.source
+       
+
+        # for sim, no backpressure
+        self.comb += self.source.ack.eq(1)
+
+
+
+dut = double_buffer_pipeline()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+
+    cyc = len(packets)
+    pak = packets
+    for c in range(cyc):
+        yield sink.data.eq(pak[c]["data"])
+        yield sink.k.eq(pak[c]["k"])
+        yield sink.stb.eq(1)
+        if "eop" in pak[c]:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+        yield
+
+    # extra clk cycles
+    for _ in range(cyc, cyc + 20):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+        yield
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": Replicate(C(0, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(0, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(4, 2 * char_width)[8:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(4, 2 * char_width)[:8], 4), "k": Replicate(0, 4)}, # CRC doesn't count
+        {"data": C(0x7C7C7C7C, word_width), "k": Replicate(1, 4)},
+        {"data": C(0x01010101, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0xF6ACEF6A, word_width), "k": Replicate(0, 4)},
+
+
+        {"data": Replicate(C(0, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(1, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(8, 2 * char_width)[8:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(8, 2 * char_width)[:8], 4), "k": Replicate(0, 4)}, # CRC doesn't count
+        {"data": C(0x19191919, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x09090909, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x90909090, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x985EFDB2, word_width), "k": Replicate(0, 4)},
+    ]
+    yield from packet_sim(paks)
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_buffer.py

@@ -0,0 +1,93 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+# from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+from types import SimpleNamespace
+
+class DUT(Module):
+    def __init__(self):
+        # PHY
+        phy = SimpleNamespace()
+        phy.sink = stream.Endpoint(word_layout)
+        phy.source = stream.Endpoint(word_layout)
+
+        self.sync += [
+            phy.source.stb.eq(0),
+            If(~((phy.sink.data[:8] == 0xBC) & (phy.sink.k[0] == 1)),
+                phy.source.stb.eq(1),
+                phy.source.data.eq(phy.sink.data),
+                phy.source.k.eq(phy.sink.k),
+            ),
+        ]
+
+        # # #
+        
+        dchar_decoder = Duplicated_Char_Decoder()
+        eop_marker = EOP_Marker()
+
+        pipeline = [phy, dchar_decoder, eop_marker]
+        self.submodules += pipeline[1:] #phy is not a submodules
+
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        self.sink, self.source = pipeline[0].sink, pipeline[-1].source
+
+
+
+dut = DUT()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+    source = dut.source
+
+    for i, p in enumerate(packets):
+        yield sink.data.eq(p["data"])
+        yield sink.k.eq(p["k"])
+        yield sink.stb.eq(p["stb"])
+
+        yield source.ack.eq(1)
+        # if i % 2 == 0:
+        #     yield source.ack.eq(1)
+        # else:
+        #     yield source.ack.eq(0)
+        
+        yield
+
+    for _ in range(10):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+
+        yield source.ack.eq(1)
+        yield
+
+    assert True
+
+
+def testbench():
+    paks = []
+    for i in range(1, 10):
+        paks += [
+            {"data": C(i << 8 | 1, word_width), "k" : Replicate(0, 4), "stb":1},
+            {"data": C(i << 8 | 2, word_width), "k" : Replicate(0, 4), "stb":1},
+            {"data": C(i << 8 | 3, word_width), "k" : Replicate(0, 4), "stb":1},
+            {"data": C(i << 8 | 4, word_width), "k" : Replicate(0, 4), "stb":1},
+            {"data": C(0xB53C3CBC, word_width), "k" : C(0b0111, 4), "stb":0},
+            {
+                "data": Replicate(KCode["pak_end"], 4),
+                "k": Replicate(1, 4),
+                "stb": 1,
+            },
+        ]
+
+    
+    yield from packet_sim(paks)
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_comb.py

@@ -0,0 +1,28 @@
+from migen import *
+from misoc.interconnect import stream
+
+class Frame(Module):
+    def __init__(self):
+        self.a = Signal()
+        self.b = Signal()
+        self.comb += [
+            self.a.eq(self.b),
+            # self.b.eq(self.a),
+        ]
+
+dut = Frame()
+
+def check_case():
+    yield dut.a.eq(1)
+    yield 
+    yield dut.a.eq(0)
+    yield 
+    for i in range(10):
+        yield
+
+
+def testbench():
+    yield from check_case()
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_crc.py

@@ -0,0 +1,58 @@
+from migen import *
+from misoc.interconnect import stream
+from sim_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+dut = StreamData_Generator()
+
+
+def check_case(packet=[], ack=0):
+    print("=================TEST========================")
+    for i, p in enumerate(packet):
+        yield dut.sink.data.eq(p["data"])
+        yield dut.sink.k.eq(p["k"])
+        yield dut.sink.stb.eq(1)
+        if "eop" in p:
+            yield dut.sink.eop.eq(1)
+
+        # CLK 
+        yield
+
+        sink = dut.sink
+        source = dut.source
+        crc = dut.crc_inserter.crc
+        print(
+            # f"\n CYCLE#{i} : sink char = {yield sink.data:#X} k = {yield sink.k:#X}"
+            f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X}"
+            f" stb = {yield source.stb} eop = {yield source.eop} ack = {yield source.ack} "
+            f"\nCYCLE#{i} : crc error = {yield crc.error:#X} crc value = {yield crc.value:#X}"
+            f" crc data = {yield crc.data:#X} engine next = {yield crc.engine.next:#X}"
+            f"\nCYCLE#{i} : crc ce = {yield crc.ce:#X} "
+        )
+    # extra clk cycles
+    cyc = i + 1
+    for i in range(cyc, cyc + 11):
+        # yield has memory for some reason
+        yield dut.sink.stb.eq(0)
+        yield dut.source.ack.eq(1)
+        yield
+        print(
+            # f"\n CYCLE#{i} : sink char = {yield sink.data:#X} k = {yield sink.k:#X}"
+            f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X}"
+            f" stb = {yield source.stb} eop = {yield source.eop} ack = {yield source.ack} "
+            f"\nCYCLE#{i} : crc error = {yield crc.error:#X} crc value = {yield crc.value:#X}"
+            f" crc data = {yield crc.data:#X} engine next = {yield crc.engine.next:#X}"
+            f"\nCYCLE#{i} : crc ce = {yield crc.ce:#X} "
+        )
+    assert True
+
+
+def testbench():
+    packet = [
+        {"data": 0x0000_0004, "k": Replicate(0, 4)},
+        {"data": 0x0000_0000, "k": Replicate(0, 4), "eop":1},
+    ]
+    yield from check_case(packet)
+
+
+run_simulation(dut, testbench())

sim_crc_checker.py

@@ -0,0 +1,92 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_frame_pipeline import *
+
+class DUT(Module):
+    def __init__(self):
+        self.submodules.crc_checker = crc_checker = CXPCRC32_Checker()
+        self.sink, self.source = crc_checker.sink, crc_checker.source
+
+        self.comb += self.source.ack.eq(1)
+
+
+dut = DUT()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+    source = dut.source
+
+    for i, p in enumerate(packets):
+        yield sink.data.eq(p["data"])
+        yield sink.k.eq(p["k"])
+
+        if "stb_break" in p:
+            yield sink.stb.eq(0)
+        else:
+            yield sink.stb.eq(1)
+
+        if "eop" in p:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+        yield
+
+    for _ in range(10):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+
+        yield source.ack.eq(1)
+        yield
+
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # stream id
+        {"data": C(0x00000001, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x6AEFACF6, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0, word_width), "k" : Replicate(0, 4), "stb_break":0}, # cyc break
+
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)},  
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # Xsize[23:16]
+        {"data": C(0x09090909, word_width), "k" : Replicate(0, 4)}, # Xsize[15:8] 
+        {"data": C(0x90909090, word_width), "k" : Replicate(0, 4)}, # Xsize[7:0] 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x8EE1DAA1, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0, word_width), "k" : Replicate(0, 4), "stb_break":0}, # cyc break
+
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # DsizeL[23:16] 
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)}, # DsizeL[15:8] 
+        {"data": C(0x64646464, word_width), "k" : Replicate(0, 4)}, # DsizeL[7:0] 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x51C243EA, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+    ]
+
+    
+    yield from packet_sim(paks)
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_double_buffer.py

@@ -0,0 +1,80 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from sim_generator import CXPCRC32Inserter
+from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+class double_buffer_pipeline(Module):
+    def __init__(self):
+        fifo = stream.SyncFIFO(word_layout_dchar, 32)
+        double_buffer = CXPCRC32_Checker(0x100)
+        dchar_dropper = DChar_Dropper()
+        pipeline = [double_buffer, dchar_dropper]
+        self.submodules += pipeline
+
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source
+
+        # for sim, no backpressure
+        self.comb += self.source.ack.eq(1)
+
+
+
+dut = double_buffer_pipeline()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+
+    cyc = len(packets)
+    pak = packets
+    for c in range(cyc):
+        yield sink.data.eq(pak[c]["data"])
+        yield sink.k.eq(pak[c]["k"])
+        yield sink.stb.eq(1)
+        if "eop" in pak[c]:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+        yield
+
+    # extra clk cycles
+    for _ in range(cyc, cyc + 20):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+        yield
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": C(0x7C7C7C7C, word_width), "k": Replicate(1, 4)},
+        {"data": C(0x01010101, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        # {"data": C(0xF6ACEF6B, word_dw), "k": Replicate(0, 4), "eop":1},
+        {"data": C(0x6AEFACF6, word_width), "k": Replicate(0, 4), "eop":1},
+
+
+        {"data": C(0x19191919, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x09090909, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x90909090, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x00000000, word_width), "k": Replicate(0, 4)},
+        {"data": C(0xB2FD5E98, word_width), "k": Replicate(0, 4), "eop":1},
+    ]
+    yield from packet_sim(paks)
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_eop.py

@@ -0,0 +1,72 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from sim_generator import CXPCRC32Inserter
+from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+class EOP_Pipeline(Module):
+    def __init__(self):
+        dchar_decoder = Duplicated_Char_Decoder()
+        eop_inserter = EOP_Marker()
+        buffer = stream.SyncFIFO(word_layout_dchar, 32)
+        pipeline = [dchar_decoder, eop_inserter, buffer]
+        self.submodules += pipeline
+
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source
+
+        # for sim, no backpressure
+        self.comb += self.source.ack.eq(1)
+
+
+
+dut = EOP_Pipeline()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+
+    cyc = len(packets)
+    pak = packets
+    for c in range(cyc):
+        yield sink.data.eq(pak[c]["data"])
+        yield sink.k.eq(pak[c]["k"])
+        yield sink.stb.eq(1)
+        yield
+
+    # extra clk cycles
+    for _ in range(cyc, cyc + 10):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+        yield
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": Replicate(KCode["pak_start"], 4), "k": Replicate(1, 4)},
+        {"data": C(0x1111, word_width), "k": Replicate(1, 4)},
+        {"data": C(0x2222, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x3333, word_width), "k": Replicate(0, 4)},
+        {"data": C(0x4444, word_width), "k": Replicate(0, 4)},
+        {"data": Replicate(KCode["pak_end"], 4), "k": Replicate(1, 4)},
+
+        {"data": Replicate(KCode["pak_start"], 4), "k": Replicate(1, 4)},
+        {"data": C(0xAAAA, word_width), "k": Replicate(1, 4)},
+        {"data": C(0xBBBB, word_width), "k": Replicate(0, 4)},
+        {"data": C(0xCCCC, word_width), "k": Replicate(0, 4)},
+        {"data": C(0xDDDD, word_width), "k": Replicate(0, 4)},
+        {"data": Replicate(KCode["pak_end"], 4), "k": Replicate(1, 4)},
+    ]
+    yield from packet_sim(paks)
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_frame.py

@@ -0,0 +1,134 @@
+from migen import *
+from misoc.interconnect import stream
+
+from sim_generator import CXPCRC32Inserter
+from sim_frame_gen import get_frame_packet
+
+from src.gateware.cxp_pipeline import *
+from src.gateware.cxp_frame_pipeline import *
+
+import numpy as np
+from PIL import Image
+
+
+class Frame(Module):
+    def __init__(self):
+        # to construct correct crc and ack/stb signal
+        self.submodules.buffer = buffer = stream.SyncFIFO(word_layout, 32)
+        self.submodules.crc_inserter = crc_inserter = CXPCRC32Inserter()
+        self.submodules.dchar_decoder = dchar_decoder = Duplicated_Char_Decoder()
+
+        # NOTE: eop is needed for crc to work correctly and RX_Bootstrap need to be followed by a EOP marker anyway
+        self.submodules.eop_marker = eop_marker = EOP_Marker()
+
+        self.submodules.stream_pipe = stream_pipe = Pixel_Pipeline()
+
+        pipeline = [buffer, crc_inserter, dchar_decoder, eop_marker, stream_pipe]
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+        self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source
+
+        # no backpressure for sim
+        self.sync += self.source.ack.eq(1)
+
+
+dut = Frame()
+
+
+def check_case(packet=[]):
+    print("=================TEST========================")
+
+    sink = dut.sink
+    stream_pipe = dut.stream_pipe
+
+    for i, p in enumerate(packet):
+        yield sink.data.eq(p["data"])
+        yield sink.k.eq(p["k"])
+        yield sink.stb.eq(1)
+        if "eop" in p:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+
+        # check cycle result
+        yield
+        # source = dut.dchar_decoder.source
+        # source = dut.stream_pipe.frame_extractor.sink
+        source = dut.sink
+        # print(
+        #     f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X} stb = {yield source.stb} ack = {yield source.ack} eop = {yield source.eop}"
+        # )
+
+    # extra clk cycles
+    cyc = i + 1
+    img = []
+    line = -1
+    total_pixel = 1000
+    for i in range(cyc, cyc + total_pixel):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+        yield
+        # print(
+        #     f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X} stb = {yield source.stb} ack = {yield source.ack} eop = {yield source.eop}"
+        # )
+
+        frame_extractoer = dut.stream_pipe.frame_extractor
+        new_line = yield frame_extractoer.new_line
+        if new_line:
+            img.append([])
+            line += 1
+            
+        stb = yield frame_extractoer.source.stb 
+        data = yield frame_extractoer.source.data
+        if stb:
+            # CXP use MSB
+            img[line].append(np.uint16(data & 0xFFFF))
+            img[line].append(np.uint16(data >> 16))
+
+        # metadata = dut.stream_pipe.frame_extractor.metadata
+        # img_header_layout = [
+        #     "stream_id",
+        #     "source_tag",
+        #     "x_size",
+        #     "x_offset",
+        #     "y_size",
+        #     "y_offset",
+        #     "l_size",  # number of data words per image line
+        #     "pixel_format",
+        #     "tap_geo",
+        #     "flag",
+        # ]
+        # for name in img_header_layout:
+        #     print(f"{name} = {yield getattr(metadata, name):#04X} ", end="")
+        # print()
+    Image.fromarray(np.array(img, dtype=np.uint8)).show()
+    assert True
+
+
+def testbench():
+    stream_id = 0x69
+    packet_tag = 0
+    frame_packet = get_frame_packet(stream_id)
+    packet = [
+        {"data": Replicate(C(stream_id, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(C(packet_tag, char_width), 4), "k": Replicate(0, 4)},
+        {
+            "data": Replicate(C(len(frame_packet), 2*char_width)[8:], 4),
+            "k": Replicate(0, 4),
+        },
+        {
+            "data": Replicate(C(len(frame_packet), 2*char_width)[:8], 4),
+            "k": Replicate(0, 4),
+        },
+    ]
+    packet += frame_packet
+    # NOTE: for crc inserter!!!!
+    packet[-1]["eop"] = 0
+
+    yield from check_case(packet)
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_frame_gen.py

@@ -0,0 +1,148 @@
+from migen import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_pipeline import *
+from src.gateware.cxp_frame_pipeline import *
+
+from PIL import Image
+import numpy as np
+
+
+def get_image_header(
+    stream_id, source_tag, xsize, xoffset, ysize, yoffset, dsize, pixelF, tag_geo, flag
+):
+    stream_id = C(stream_id, char_width)
+    source_tag = C(source_tag, 2 * char_width)
+    xsize = C(xsize, 3 * char_width)
+    xoffset = C(xoffset, 3 * char_width)
+    ysize = C(ysize, 3 * char_width)
+    yoffset = C(yoffset, 3 * char_width)
+    dsize = C(dsize, 3 * char_width)
+    pixelF = C(pixelF, 2 * char_width)
+    tag_geo = C(tag_geo, 2 * char_width)
+    flag = C(flag, char_width)
+
+    assert len(stream_id) == len(flag) == char_width
+    assert len(source_tag) == len(pixelF) == len(tag_geo) == 2 * char_width
+    assert len(xsize) == len(xoffset) == len(ysize) == len(yoffset) == 3 * char_width
+    return [
+        {"data": Replicate(KCode["stream_marker"], 4), "k": Replicate(1, 4)},
+        {"data": Replicate(C(0x01, char_width), 4), "k": Replicate(0, 4)},
+        {"data": Replicate(stream_id, 4), "k": Replicate(0, 4)},
+        {"data": Replicate(source_tag[8:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(source_tag[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xsize[16:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xsize[8:16], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xsize[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xoffset[16:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xoffset[8:16], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(xoffset[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(ysize[16:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(ysize[8:16], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(ysize[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(yoffset[16:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(yoffset[8:16], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(yoffset[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(dsize[16:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(dsize[8:16], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(dsize[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(pixelF[8:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(pixelF[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(tag_geo[8:], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(tag_geo[:8], 4), "k": Replicate(0, 4)},
+        {"data": Replicate(flag, 4), "k": Replicate(0, 4)},
+    ]
+
+
+def get_line_marker():
+    return [
+        {"data": Replicate(KCode["stream_marker"], 4), "k": Replicate(1, 4)},
+        {"data": Replicate(C(0x02, char_width), 4), "k": Replicate(0, 4)},
+    ]
+
+
+def get_frame_packet(stream_id, pixel_format="mono16"):
+    assert pixel_format in ["mono16"]
+
+    arr = [
+        [204, 200, 203, 205, 190, 187, 189, 205, 214, 197, 188, 185, 181, 178, 193, 209, 211, 207, 211, 192, 168, 168, 171, 199, 210, 212, 203, 196],
+        [218, 205, 199, 190, 192, 197, 196, 195, 184, 178, 182, 173, 166, 132, 122, 114, 154, 184, 187, 188, 171, 168, 170, 180, 192, 196, 202, 198],
+        [223, 222, 222, 224, 216, 199, 199, 207, 205, 189, 183, 182, 144, 66,  61,  66, 80, 148, 181, 175, 169, 170, 174, 177, 196, 206, 223, 218],
+        [221, 226, 225, 222, 211, 200, 202, 208, 215, 201, 187, 180, 133, 116, 113, 118, 96, 111, 206, 193, 170, 169, 186, 211, 218, 224, 231, 223],
+        [219, 216, 206, 197, 210, 201, 206, 203, 191, 190, 185, 145, 134, 140, 159, 170, 150, 116, 180, 173, 179, 170, 172, 185, 201, 218, 227, 227],
+        [203, 198, 194, 208, 227, 201, 201, 201, 215, 221, 209, 170, 136, 113, 141, 139, 141, 145, 188, 170, 180, 169, 184, 173, 174, 192, 215, 230],
+        [206, 224, 213, 213, 233, 207, 204, 226, 233, 227, 214, 166, 145, 123, 145, 155, 147, 186, 213, 187, 171, 169, 193, 193, 171, 178, 186, 207],
+        [212, 228, 216, 205, 214, 205, 204, 230, 235, 225, 219, 187, 143, 122, 146, 163, 158, 195, 209, 203, 174, 168, 190, 185, 187, 202, 180, 174],
+        [197, 206, 201, 223, 213, 201, 203, 231, 234, 225, 218, 206, 147, 125, 149, 155, 190, 208, 206, 203, 175, 168, 171, 179, 184, 206, 189, 176],
+        [213, 202, 209, 235, 223, 200, 202, 202, 227, 227, 202, 176, 138, 122, 144, 153, 190, 209, 207, 191, 172, 167, 179, 204, 190, 191, 180, 193],
+        [225, 225, 207, 231, 219, 197, 215, 200, 194, 199, 181, 172, 131, 129, 147, 159, 113, 175, 196, 179, 184, 169, 181, 210, 202, 204, 200, 177],
+        [208, 222, 204, 223, 210, 191, 195, 198, 203, 167, 171, 168, 135, 129, 149, 175,  66,  57,  90, 121, 147, 165, 181, 205, 195, 217, 209, 173],
+        [188, 216, 201, 206, 199, 180, 185, 180, 129,  75, 139, 166, 124, 146, 189, 135,  51,  41,  38,  40,  45,  63, 131, 201, 189, 215, 193, 170],
+        [188, 194, 195, 192, 182, 180, 134,  68,  45,  41,  96, 130, 116, 156, 163,  64,  46,  41,  43,  41,  42,  42,  74, 181, 177, 198, 175, 193],
+        [179, 179, 209, 224, 198, 182,  99,  42,  44,  41,  44, 100, 116, 125, 100,  46,  45,  42,  42,  37,  44,  43,  49, 150, 183, 170, 172, 198],
+        [175, 177, 208, 223, 197, 180,  94,  40,  42,  40,  41,  99, 134, 117,  80,  43,  46,  43,  37,  37,  44,  42,  35, 129, 195, 170, 170, 180],
+        [179, 181, 187, 217, 193, 175,  91,  38,  41,  41,  42, 106, 151, 107,  62,  43,  45,  41,  33,  38,  42,  34,  33,  77, 188, 175, 173, 208],
+        [190, 191, 180, 213, 194, 175,  78,  38,  40,  40,  40,  98, 134, 97,  51,  44,  59,  50,  37,  40,  36,  26,  36,  44, 100, 178, 192, 206],
+        [199, 191, 184, 204, 196, 176,  78,  33,  38,  38,  39,  80, 102, 83,  43,  44, 112, 130, 122,  63,  33,  24,  29,  34,  33,  74, 162, 195],
+        [191, 170, 196, 193, 186, 177,  88,  27,  34,  37,  36,  74, 101, 70,  36,  37,  81, 127, 137, 113,  40,  28,  30,  32,  36,  29, 69, 173],
+        [164, 189, 190, 180, 176, 172,  83,  26,  28,  33,  32,  68,  97, 62,  32,  30,  44,  97, 123, 136,  58,  42,  44,  43,  43,  40, 58, 162],
+        [177, 202, 205, 181, 174, 163,  78,  38,  35,  47,  54,  67,  92, 51,  28,  29,  26,  21,  39,  85,  47,  46,  52,  47,  46,  45, 48, 141],
+        [181, 193, 199, 192, 171, 163,  91,  67, 121, 123,  91,  63,  89, 45,  25,  25,  23,  20,  15,  13,  20,  48,  54,  35,  34,  34, 68, 146],
+        [175, 192, 195, 179, 165, 163, 100,  64,  99,  94,  82,  58,  83, 37,  23,  22,  22,  27,  21,  15,  14,  44,  98,  83,  94, 118, 164, 157],
+        [153, 184, 171, 163, 161, 157, 140,  70,  58,  89,  61,  53,  76, 30,  20,  20,  20,  31,  24,  19,  16,  47, 159, 163, 160, 171, 160, 142],
+        [142, 150, 161, 168, 154, 154, 164, 138,  76,  55,  26,  37,  62, 24,  19,  19,  20,  21,  23,  27,  31,  46, 142, 156, 151, 153, 147, 145],
+        [153, 147, 174, 171, 151, 150, 169, 158, 142,  92,  28,  60,  59, 20,  20,  18,  20,  26,  27,  29,  33,  38, 125, 153, 150, 147, 147, 148],
+        [138, 141, 166, 164, 146, 144, 164, 149, 132,  72,  34,  88,  72, 24,  19,  18,  18,  23,  25,  28,  31,  30,  98, 150, 146, 144, 146, 144]
+    ]
+    source_tag = 0
+    xsize, ysize = len(arr[0]), len(arr)
+
+    xoffset, yoffset = 0, 0
+    if pixel_format == "mono16":
+        dsize = xsize // 2
+        pixelF = 0x0105
+    tag_geo = 0
+    flag = 0
+
+    packet = []
+    # Image header
+    packet += get_image_header(
+        stream_id,
+        source_tag,
+        xsize,
+        xoffset,
+        ysize,
+        yoffset,
+        dsize,
+        pixelF,
+        tag_geo,
+        flag,
+    )
+
+    for line in arr:
+        packet += get_line_marker()
+        if pixel_format == "mono16":
+            for i in range(len(line)):
+                if (i % 2) == 0:
+                    if i == len(line) - 1:
+                        # print(C(line[i]))
+                        packet += [
+                            {
+                                "data": C(line[i], 4 * char_width),
+                                "k": Replicate(0, 4),
+                            },
+                        ]
+                    else:
+                        # print(C(line[i], 2 * char_width), C(line[i + 1]))
+                        # CXP use MSB
+                        packet += [
+                            {
+                                "data": Cat(
+                                    C(line[i], 2 * char_width),
+                                    C(line[i + 1], 2 * char_width),
+                                ),
+                                "k": Replicate(0, 4),
+                            },
+                        ]
+
+    return packet

sim_generator.py

@@ -0,0 +1,122 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_frame_pipeline import CXPCRC32
+from src.gateware.cxp_pipeline import *
+
+class CXPCRC32Inserter(Module):
+    def __init__(self, insert_eop=True):
+        self.sink = stream.Endpoint(word_layout)
+        self.source = stream.Endpoint(word_layout)
+
+        # # #
+
+        self.submodules.crc = crc = CXPCRC32(word_width)
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+
+        # WARNING: this will eat data if the source don't care about ack
+        # NOTE: need one cycle to turn itself on
+        fsm.act(
+            "IDLE",
+            crc.reset.eq(1),
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                self.sink.ack.eq(0),
+                NextState("COPY"),
+            ),
+        )
+        fsm.act(
+            "COPY",
+            crc.ce.eq(self.sink.stb & self.source.ack),
+            crc.data.eq(self.sink.data),
+            self.sink.connect(self.source),
+            self.source.eop.eq(0),
+            If(
+                self.sink.stb & self.sink.eop & self.source.ack,
+                NextState("INSERT"),
+            ),
+        )
+        fsm.act(
+            "INSERT",
+            self.source.stb.eq(1),
+            self.source.eop.eq(1) if insert_eop else self.source.eop.eq(0),
+            self.source.data.eq(crc.value),
+            If(self.source.ack, NextState("IDLE")),
+        )
+
+
+class StreamPacket_Wrapper(Module):
+    def __init__(self):
+        self.sink = stream.Endpoint(word_layout)
+        self.source = stream.Endpoint(word_layout)
+
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+
+        fsm.act(
+            "IDLE",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                self.sink.ack.eq(0),
+                NextState("INSERT_HEADER_0"),
+            ),
+        )
+
+        fsm.act(
+            "INSERT_HEADER_0",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(KCode["pak_start"], 4)),
+            self.source.k.eq(Replicate(1, 4)),
+            If(self.source.ack, NextState("INSERT_HEADER_1")),
+        )
+
+        fsm.act(
+            "INSERT_HEADER_1",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(C(0x01, char_width), 4)),
+            self.source.k.eq(Replicate(0, 4)),
+            If(self.source.ack, NextState("COPY")),
+        )
+
+        fsm.act(
+            "COPY",
+            self.sink.connect(self.source),
+            self.source.eop.eq(0),
+            If(
+                self.sink.stb & self.sink.eop & self.source.ack,
+                NextState("INSERT_FOOTER"),
+            ),
+        )
+
+        fsm.act(
+            "INSERT_FOOTER",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(KCode["pak_end"], 4)),
+            self.source.k.eq(Replicate(1, 4)),
+            # Simulate RX don't have eop tagged
+            # self.source.eop.eq(1),
+            If(self.source.ack, NextState("IDLE")),
+        )
+
+
+# With KCode & 0x01*4
+class StreamData_Generator(Module):
+    def __init__(self):
+        # should be big enough for all test
+        self.submodules.buffer = buffer = stream.SyncFIFO(word_layout, 32)
+        self.submodules.crc_inserter = crc_inserter = CXPCRC32Inserter()
+        self.submodules.wrapper = wrapper = StreamPacket_Wrapper()
+
+        # # #
+        pipeline = [buffer, crc_inserter, wrapper]
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+        self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source

sim_idle.py

@@ -0,0 +1,149 @@
+from migen import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_pipeline import Packet_Wrapper
+
+char_width = 8
+word_dw = 32
+word_layout = [("data", word_dw), ("k", word_dw//8)]
+
+
+def K(x, y):
+    return ((y << 5) | x)
+
+KCode = {
+    "pak_start" : C(K(27, 7), char_width),
+    "io_ack" : C(K(28, 6), char_width), 
+    "trig_indic_28_2" : C(K(28, 2), char_width),
+    "stream_marker" : C(K(28, 3), char_width),
+    "trig_indic_28_4" : C(K(28, 4), char_width),
+    "pak_end" : C(K(29, 7), char_width),
+    "idle_comma" : C(K(28, 5), char_width),
+    "idle_alignment" : C(K(28, 1), char_width),
+}
+
+class TX_Bootstrap(Module):
+    def __init__(self):
+        self.tx_testseq = Signal()
+
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+        self.source = stream.Endpoint(word_layout)
+
+        self.cnt = Signal(max=0xFFF)
+        fsm.act("IDLE",
+            If(self.tx_testseq, 
+                NextValue(self.cnt, self.cnt.reset),
+                NextState("WRITE_TEST_PACKET_TYPE"),
+            )
+        )
+
+        fsm.act("WRITE_TEST_PACKET_TYPE",
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(C(0x04, char_width), 4)),
+            self.source.k.eq(Replicate(0, 4)),
+            If(self.source.ack,NextState("WRITE_TEST_COUNTER"))
+        )
+
+        # testword = Signal(word_dw)
+        # self.comb += [
+        #     testword[:8].eq(self.cnt[:8]),
+        #     testword[8:16].eq(self.cnt[:8]+1),
+        #     testword[16:24].eq(self.cnt[:8]+2),
+        #     testword[24:].eq(self.cnt[:8]+3),
+        # ]
+
+        fsm.act("WRITE_TEST_COUNTER",
+            self.source.stb.eq(1),
+            self.source.data[:8].eq(self.cnt[:8]),
+            self.source.data[8:16].eq(self.cnt[:8]+1),
+            self.source.data[16:24].eq(self.cnt[:8]+2),
+            self.source.data[24:].eq(self.cnt[:8]+3),
+            self.source.k.eq(Cat(0, 0, 0, 0)),
+            If(self.source.ack,
+                If(self.cnt == 0x0FF-3,
+                    self.source.eop.eq(1),
+                    NextState("IDLE")
+                ).Else(
+                    NextValue(self.cnt, self.cnt + 4),
+                )
+               
+            )
+        )
+
+class Idle_Word_Inserter(Module):
+    def __init__(self):
+        # Section 9.2.5 (CXP-001-2021)
+        # Send K28.5, K28.1, K28.1, D21.5  as idle word
+        self.submodules.fsm = fsm = FSM(reset_state="WRITE_IDLE")
+        
+        self.sink = stream.Endpoint(word_layout)
+        self.source = stream.Endpoint(word_layout)
+
+        cnt = Signal(max=0x10, reset=0xF)
+        
+        fsm.act("WRITE_IDLE",
+            self.source.stb.eq(1),
+            self.source.data.eq(Cat(KCode["idle_comma"], KCode["idle_alignment"], KCode["idle_alignment"], C(0xB5, char_width))),
+            self.source.k.eq(Cat(1, 1, 1, 0)),
+
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                self.sink.ack.eq(0),
+                If(self.source.ack,
+                    NextValue(cnt, cnt.reset),
+                    NextState("COPY"),
+                )
+            ),
+        )
+
+        fsm.act("COPY",
+            self.sink.connect(self.source),
+            # increment when upstream has data and got ack
+            If(self.sink.stb & self.source.ack, NextValue(cnt, cnt - 1)),
+            If((((~self.sink.stb) | (self.sink.eop) | (cnt == 0) ) & self.source.ack), NextState("WRITE_IDLE"))
+        )
+
+class Pipeline(Module):
+    def __init__(self):
+        self.submodules.bootstrap = boostrap = TX_Bootstrap()
+        self.submodules.wrapper = wrapper = Packet_Wrapper()
+        # self.submodules.buffer = buffer = stream.SyncFIFO(word_layout, 32)
+        self.submodules.idle_inserter = idle_inserter = Idle_Word_Inserter()
+
+        # # #
+
+        pipeline = [boostrap, wrapper, idle_inserter]
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+        # self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source
+
+        # no backpressure
+        # self.comb += self.source.ack.eq(1)
+
+dut = Pipeline()
+
+def check_case():
+    source = dut.source
+    # sink = dut.sink
+    # for i in range(1, 30):
+    #     yield sink.data.eq(i)
+    #     yield sink.stb.eq(1)
+    #     yield 
+    yield dut.bootstrap.tx_testseq.eq(1)
+    yield
+    yield dut.bootstrap.tx_testseq.eq(0)
+    for i in range(10):
+        yield
+    for _ in range(100):
+        yield source.ack.eq(1)
+        yield
+
+
+def testbench():
+    yield from check_case()
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_roi.py

@@ -0,0 +1,323 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from src.gateware.cxp_frame_pipeline import *
+from src.gateware.cxp_pipeline import *
+
+from math import ceil
+
+class ROI(Module):
+    def __init__(self):
+        fifo = stream.SyncFIFO(word_layout, 32) # to avoid data getting eaten and act as delay between eop
+        dchar_decoder = Duplicated_Char_Decoder()
+        # self.crc = CXPCRC32_Checker()
+        self.pipeline = Pixel_Pipeline(24, 32)
+
+        pipeline = [fifo, dchar_decoder, self.pipeline]
+        self.submodules += pipeline
+
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        self.sink = pipeline[0].sink
+        # self.source = pipeline[-1].source
+         
+        # DEBUG: test roi
+        cfg = self.pipeline.roi.cfg
+        self.comb += [
+            cfg.x0.eq(1),
+            cfg.x1.eq(2),
+            cfg.y0.eq(1),
+            cfg.y1.eq(2),
+        ]
+
+
+
+dut = ROI()
+
+
+def packet_sim(packets=[]):
+    print("=================TEST========================")
+    sink = dut.sink
+
+    cyc = len(packets)
+    pak = packets
+    for c in range(cyc):
+        yield sink.data.eq(pak[c]["data"])
+        yield sink.k.eq(pak[c]["k"])
+        yield sink.stb.eq(1)
+        if "eop" in pak[c]:
+            yield sink.eop.eq(1)
+        else:
+            yield sink.eop.eq(0)
+        yield
+
+    # extra clk cycles
+    for _ in range(cyc, cyc + 20):
+        yield sink.data.eq(0)
+        yield sink.k.eq(0)
+        yield sink.stb.eq(0)
+        yield sink.eop.eq(0)
+        yield
+    metadata = dut.pipeline.header_decoder.metadata
+    img_header_layout = [
+        "stream_id",
+        "source_tag", # image index since powering on
+        "x_size",
+        "x_offset",
+        "y_size",
+        "y_offset",
+        "l_size",  # number of data words per image line
+        "pixel_format",
+        "tap_geo",
+        "flag",
+    ]
+    for name in img_header_layout:
+        print(f"{name} = {yield getattr(metadata, name):#04X} ", end="")
+    print()
+
+    roi = dut.pipeline.roi
+    print(f"x0 = {yield roi.cfg.x0} y0 = {yield roi.cfg.y0} | x1 = {yield roi.cfg.x1} y1 = {yield roi.cfg.y1}")
+    print(f"out count = {yield roi.out.count}")
+    assert True
+
+
+def testbench():
+    paks = [
+        {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # stream id
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x6AEFACF6, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)},  
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # Xsize[23:16]
+        {"data": C(0x09090909, word_width), "k" : Replicate(0, 4)}, # Xsize[15:8] 
+        {"data": C(0x90909090, word_width), "k" : Replicate(0, 4)}, # Xsize[7:0] 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x8EE1DAA1, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # DsizeL[23:16] 
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)}, # DsizeL[15:8] 
+        {"data": C(0x64646464, word_width), "k" : Replicate(0, 4)}, # DsizeL[7:0] 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x51C243EA, word_width), "k" : Replicate(0, 4), "eop":0}, # crc
+
+        # the new line + pixel data
+        {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0D0D0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0B0D, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0D0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0C0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0A0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0C0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0C0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0C0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0C0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0D0A, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0A0B0D, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0D0C0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0A0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0D0D0B0A, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0C0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0C0B0B0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0C0C0B, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x0B0B0B0C, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0xCB5DCDD6, word_width), "k" : Replicate(0, 4), "eop": 0}, # crc
+
+        # {"data": C(0x0C0B0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0B0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0B0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0D0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0A, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0D0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0D, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0A0C0C0D, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0B0A, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0D0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0B0D0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0D0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0B0A, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0A0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0D0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0D0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0D0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0D0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0D0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0C0D0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0D0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0D0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0B0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0B0C0B, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0B0C0C0D, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0D0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0B0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0C0C0C0C, word_dw), "k" : Replicate(0, 4)},  
+        # {"data": C(0x0BADF020, word_dw), "k" : Replicate(0, 4), "eop":0},  # crc
+    ]
+    yield from packet_sim(paks)
+
+
+def testbench_fake_data():
+    # config
+    pix_size = 8
+    # x_size = 2448
+    x_size = 10
+    y_size = 2
+
+    
+    l_size = ceil(x_size*pix_size/32)
+    pix_fmt = {
+        8: 0x0101,
+        10: 0x0102,
+        12: 0x0103,
+        14: 0x0104,
+        16: 0x0105,
+    }
+    # frame header
+    paks = [
+        {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+        {"data": C(0x01010101, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # stream id
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x6AEFACF6, word_width), "k" : Replicate(0, 4), "eop":0}, # fake crc
+
+        {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)},  
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # Xsize[23:16]
+        {"data": Replicate(C(x_size >> 8, char_width), 4), "k" : Replicate(0, 4)}, # Xsize[15:8] 
+        {"data": Replicate(C(x_size & 0xFF, char_width), 4), "k" : Replicate(0, 4)}, # Xsize[7:0] 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # Ysize[23:16]
+        {"data": C(0x8EE1DAA1, word_width), "k" : Replicate(0, 4), "eop":0}, # fake crc
+
+        {"data": Replicate(C(y_size >> 8, char_width), 4), "k" : Replicate(0, 4)}, # Ysize[15:8]
+        {"data": Replicate(C(y_size & 0xFF, char_width), 4), "k" : Replicate(0, 4)}, # Ysize[7:0]
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x08080808, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, # DsizeL[23:16] 
+        {"data": Replicate(C(l_size >> 8, char_width), 4), "k" : Replicate(0, 4)}, # DsizeL[15:8] 
+        {"data": Replicate(C(l_size & 0xFF, char_width), 4), "k" : Replicate(0, 4)}, # DsizeL[7:0] 
+        {"data": Replicate(C(pix_fmt[pix_size] >> 8, char_width), 4), "k" : Replicate(0, 4)}, # PixelF[15:8]
+        {"data": Replicate(C(pix_fmt[pix_size] & 0xFF, char_width), 4), "k" : Replicate(0, 4)}, # PixelF[7:0]
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x00000000, word_width), "k" : Replicate(0, 4)}, 
+        {"data": C(0x51C243EA, word_width), "k" : Replicate(0, 4), "eop":0}, # fake crc
+    ]
+
+    # pixel data
+    base = 0
+    for _ in range(y_size):
+        pix = []
+        for _ in range(x_size):
+            base += 1
+            pix.append(base)
+        print(pix)
+
+        packed = 0
+        for i, p in enumerate(pix):
+            packed += p << i*pix_size
+        # print(f"{packed:08X}")
+        
+        # new line indicator
+        paks += [
+            {"data": C(0x7C7C7C7C, word_width), "k" : Replicate(1, 4)}, 
+            {"data": C(0x02020202, word_width), "k" : Replicate(0, 4)}, 
+        ]
+        for i in range(l_size):
+            serialized = (packed & (0xFFFF_FFFF << i*word_width)) >> i*word_width
+            print(f"{serialized:08X}")
+            paks.append({"data": C(serialized, word_width), "k" : Replicate(0, 4)})
+
+    paks.append({"data": C(0xCB5DCDD6, word_width), "k" : Replicate(0, 4), "eop": 0}) # fake crc
+    
+    yield from packet_sim(paks)
+
+run_simulation(dut, testbench_fake_data(), vcd_name="sim-cxp.vcd")

sim_stream.py

@@ -0,0 +1,154 @@
+from migen import *
+from misoc.interconnect import stream
+
+from sim_pipeline import *
+from sim_generator import StreamData_Generator
+
+from src.gateware.cxp_pipeline import *
+
+class CXP_Links(Module):
+    def __init__(self):
+        # TODO: select the correct buffer to read from
+        # NOTE: although there are double buffer in each connect, the reading must be faster than writing to avoid data loss
+
+        self.downconn_sources = []
+        self.stream_sinks = []
+        for i in range(2):
+            downconn = Pipeline()
+            setattr(self.submodules, "cxp_conn"+str(i), downconn)
+            self.downconn_sources.append(downconn)
+
+            stream_pipeline = Stream_Pipeline()
+            setattr(self.submodules, "stream_pipeline"+str(i), stream_pipeline)
+            self.stream_sinks.append(stream_pipeline)
+
+        self.submodules.crossbar = Streams_Crossbar(self.downconn_sources, self.stream_sinks)
+        
+class Pipeline(Module):
+    def __init__(self):
+        self.submodules.generator = generator = StreamData_Generator()
+        self.submodules.dchar_decoder = dchar_decoder = Duplicated_Char_Decoder()
+        self.submodules.data_decoder = data_decoder = Control_Packet_Reader()
+        self.submodules.eop_marker = eop_marker = EOP_Marker()
+
+        # # #
+
+        pipeline = [generator, dchar_decoder, data_decoder, eop_marker]
+        for s, d in zip(pipeline, pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+        self.sink = pipeline[0].sink
+        self.source = pipeline[-1].source
+
+        # self.comb += self.source.ack.eq(1)
+
+
+dut = CXP_Links()
+
+def check_case(packet=[]):
+    print("=================TEST========================")
+
+    downconns = dut.downconn_sources
+    stream_buffers = dut.stream_sinks
+    ch = 0
+    
+    for i, p in enumerate(packet):
+        for x in range(len(downconns)):
+            if x == ch:
+                yield downconns[x].sink.data.eq(p["data"])
+                yield downconns[x].sink.k.eq(p["k"])
+                yield downconns[x].sink.stb.eq(1)
+            else:
+                yield downconns[x].sink.data.eq(0)
+                yield downconns[x].sink.k.eq(0)
+                yield downconns[x].sink.stb.eq(0)
+                yield downconns[x].sink.eop.eq(0)
+        if "eop" in p:
+            yield downconns[ch].sink.eop.eq(1)
+            # compensate for delay
+            # yield
+            # yield downconns[ch].sink.data.eq(0)
+            # yield downconns[ch].sink.k.eq(0)
+            # yield downconns[ch].sink.stb.eq(0)
+            # yield downconns[ch].sink.eop.eq(0)
+            # yield
+            # yield
+            # yield
+            ch = (ch + 1) % len(downconns)
+        else:
+            yield downconns[ch].sink.eop.eq(0)
+
+        # check cycle result
+        yield
+        # source = dut.stream_pipeline_sinks[0].source
+        source = dut.stream_sinks[0].double_buffer.source
+        print(
+            f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X} stb = {yield source.stb} ack = {yield source.ack} eop = {yield source.eop}"
+            # f" source dchar = {yield source.dchar:#X} dchar_k = {yield source.dchar_k:#X}"
+            f"\nCYCLE#{i} : read mask = {yield dut.crossbar.mux.sel}"
+            # f"\nCYCLE#{i} : stream id = {yield decoder.stream_id:#X} pak_tag = {yield decoder.pak_tag:#X}"
+            # f" stream_pak_size = {yield decoder.stream_pak_size:#X}"
+        )
+        # crc = downconns[1].generator.crc_inserter.crc
+        # crc = dut.double_buffer.crc
+        # print(
+        #     f"CYCLE#{i} : crc error = {yield crc.error:#X} crc value = {yield crc.value:#X}"
+        #     f" crc data = {yield crc.data:#X} engine next = {yield crc.engine.next:#X} ce = {yield crc.ce}"
+        # )
+
+            
+    # extra clk cycles
+    cyc = i + 1
+    for i in range(cyc, cyc + 30):
+        for x in range(len(downconns)):
+            # yield won't reset every cycle
+            yield downconns[x].sink.data.eq(0)
+            yield downconns[x].sink.k.eq(0)
+            yield downconns[x].sink.stb.eq(0)
+            yield downconns[x].sink.eop.eq(0)
+        yield
+        print(
+            f"\nCYCLE#{i} : source char = {yield source.data:#X} k = {yield source.k:#X} stb = {yield source.stb} ack = {yield source.ack} eop = {yield source.eop}"
+            # f" source dchar = {yield source.dchar:#X} dchar_k = {yield source.dchar_k:#X}"
+            f"\nCYCLE#{i} : read mask = {yield dut.crossbar .mux.sel}"
+            # f"\nCYCLE#{i} : stream id = {yield decoder.stream_id:#X} pak_tag = {yield decoder.pak_tag:#X}"
+            # f" stream_pak_size = {yield decoder.stream_pak_size:#X}"
+        )
+    assert True
+
+
+def testbench():
+    # stream_id = 0x01
+    streams = [
+        [
+            {"data": 0x11111111, "k": Replicate(0, 4)},
+            {"data": 0xB105F00D, "k": Replicate(0, 4)},
+        ],
+        [
+            {"data": 0x22222222, "k": Replicate(0, 4)},
+            {"data": 0xC001BEA0, "k": Replicate(0, 4)},
+        ],
+        [
+            {"data": 0x33333333, "k": Replicate(0, 4)},
+            {"data": 0xC0A79AE5, "k": Replicate(0, 4)},
+        ],
+    ]
+
+    packet = []
+    for i, s in enumerate(streams):
+        s[-1]["eop"] = 0
+        packet += [
+            {"data": Replicate(C(i % 2, char_width), 4), "k": Replicate(0, 4)},
+            {"data": Replicate(C(i, char_width), 4), "k": Replicate(0, 4)},
+            {
+                "data": Replicate(C(len(s) >> 8 & 0xFF, char_width), 4),
+                "k": Replicate(0, 4),
+            },
+            {"data": Replicate(C(len(s) & 0xFF, char_width), 4), "k": Replicate(0, 4)},
+            *s,
+        ]
+
+
+    yield from check_case(packet)
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

sim_test.py

@@ -0,0 +1,34 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+class test(Module):
+    def __init__(self):
+
+        self.test_cnt = Signal(32, reset=0x03020100)
+
+        self.sync += [
+            self.test_cnt[:8].eq(self.test_cnt[:8] + 4),
+            self.test_cnt[8:16].eq(self.test_cnt[8:16] + 4),
+            self.test_cnt[16:24].eq(self.test_cnt[16:24] + 4),
+            self.test_cnt[24:].eq(self.test_cnt[24:] + 4),
+        ]
+
+
+
+dut = test()
+
+
+def packet_sim(packets=[]):
+
+    for _ in range(0x100):
+        yield
+    
+    assert True
+
+
+def testbench():
+    yield from packet_sim()
+
+
+run_simulation(dut, testbench(), vcd_name="sim-cxp.vcd")

src/.clang-format

@@ -0,0 +1,32 @@
+BasedOnStyle: LLVM
+
+Language: Cpp
+Standard: Cpp11
+
+AccessModifierOffset: -1
+AlignEscapedNewlines: Left
+AlwaysBreakAfterReturnType: None
+AlwaysBreakTemplateDeclarations: Yes
+AllowAllParametersOfDeclarationOnNextLine: false
+AllowShortFunctionsOnASingleLine: Inline
+BinPackParameters: false
+BreakBeforeBinaryOperators: NonAssignment
+BreakBeforeTernaryOperators: true
+BreakConstructorInitializers: AfterColon
+BreakInheritanceList: AfterColon
+ColumnLimit: 120
+ConstructorInitializerAllOnOneLineOrOnePerLine: true
+ContinuationIndentWidth: 4
+DerivePointerAlignment: false
+IndentCaseLabels: true
+IndentPPDirectives: None
+IndentWidth: 4
+MaxEmptyLinesToKeep: 1
+PointerAlignment: Left
+ReflowComments: true
+SortIncludes: false
+SortUsingDeclarations: true
+SpaceAfterTemplateKeyword: false
+SpacesBeforeTrailingComments: 2
+TabWidth: 4
+UseTab: Never

src/.clippy.toml

@@ -0,0 +1 @@
+doc-valid-idents = ["CPython", "NumPy", ".."]

src/.pre-commit-config.yaml

@@ -0,0 +1,32 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+
+default_stages: [commit]
+
+repos:
+  - repo: local
+    hooks:
+      - id: cargo-fmt
+        name: artiq-zynq cargo format
+        entry: nix
+        language: system
+        types: [file, rust]
+        pass_filenames: false
+        description: Runs cargo fmt on the codebase.
+        args: [develop, -c, cargo, fmt, --manifest-path, src/Cargo.toml, --all]
+      - id: cargo-clippy
+        name: artiq-zynq cargo clippy
+        entry: nix
+        language: system
+        types: [file, rust]
+        pass_filenames: false
+        description: Runs cargo clippy on the codebase.
+        args: [develop, -c, cargo, clippy, --manifest-path, src/Cargo.toml, --tests]
+  - repo: https://github.com/pre-commit/mirrors-clang-format
+    rev: v19.1.0
+    hooks:
+      - id: clang-format
+        name: artiq-zynq clang-format
+        description: Runs clang-format on the codebase.
+        files: \.(cpp|h|hpp|c)$
+        args: [-style=file, -fallback-style=none, -assume-filename=src/.clang-format]

src/Cargo.lock

@@ -2,6 +2,15 @@
 # It is not intended for manual editing.
 version = 3
 
+[[package]]
+name = "approx"
+version = "0.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cab112f0a86d568ea0e627cc1d6be74a1e9cd55214684db5561995f6dad897c6"
+dependencies = [
+ "num-traits",
+]
+
 [[package]]
 name = "arrayvec"
 version = "0.7.4"
@@ -49,9 +58,9 @@ version = "0.0.0"
 
 [[package]]
 name = "byteorder"
-version = "1.4.3"
+version = "1.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
+checksum = "60f0b0d4c0a382d2734228fd12b5a6b5dac185c60e938026fd31b265b94f9bd2"
 
 [[package]]
 name = "cc"
@@ -73,18 +82,14 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
 
 [[package]]
 name = "compiler_builtins"
-version = "0.1.39"
+version = "0.1.49"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3748f82c7d366a0b4950257d19db685d4958d2fa27c6d164a3f069fec42b748b"
+checksum = "20b1438ef42c655665a8ab2c1c6d605a305f031d38d9be689ddfef41a20f3aa2"
 
 [[package]]
 name = "core_io"
-version = "0.1.20210325"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "97f8932064288cc79feb4d343a399d353a6f6f001e586ece47fe518a9e8507df"
-dependencies = [
- "rustc_version",
-]
+version = "0.1.0"
+source = "git+https://git.m-labs.hk/M-Labs/rs-core_io.git?rev=e9d3edf027#e9d3edf0272502b0dd6c26e8a4869c2912657615"
 
 [[package]]
 name = "crc"
@@ -132,9 +137,8 @@ dependencies = [
 
 [[package]]
 name = "fatfs"
-version = "0.3.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e18f80a87439240dac45d927fd8f8081b6f1e34c03e97271189fa8a8c2e96c8f"
+version = "0.3.6"
+source = "git+https://git.m-labs.hk/M-Labs/rust-fatfs.git?rev=4b5e420084#4b5e420084fd1c4a9c105680b687523909b6469c"
 dependencies = [
  "bitflags",
  "byteorder",
@@ -246,6 +250,7 @@ dependencies = [
  "libsupport_zynq",
  "log",
  "log_buffer",
+ "nalgebra",
  "nb 0.1.3",
  "unwind",
  "vcell",
@@ -268,6 +273,7 @@ name = "libboard_artiq"
 version = "0.0.0"
 dependencies = [
  "build_zynq",
+ "byteorder",
  "core_io",
  "crc",
  "embedded-hal",
@@ -363,9 +369,9 @@ checksum = "822add9edb1860698b79522510da17bef885171f75aa395cff099d770c609c24"
 
 [[package]]
 name = "log"
-version = "0.4.17"
+version = "0.4.14"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "abb12e687cfb44aa40f41fc3978ef76448f9b6038cad6aef4259d3c095a2382e"
+checksum = "51b9bbe6c47d51fc3e1a9b945965946b4c44142ab8792c50835a980d362c2710"
 dependencies = [
  "cfg-if 1.0.0",
 ]
@@ -382,6 +388,19 @@ version = "0.7.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "c75de51135344a4f8ed3cfe2720dc27736f7711989703a0b43aadf3753c55577"
 
+[[package]]
+name = "nalgebra"
+version = "0.32.6"
+source = "git+https://git.m-labs.hk/M-Labs/nalgebra.git?rev=ad42410ab0#ad42410ab0abb014229e3ff6bc6ccd39ca92d5d1"
+dependencies = [
+ "approx",
+ "num-complex",
+ "num-rational",
+ "num-traits",
+ "simba",
+ "typenum",
+]
+
 [[package]]
 name = "nb"
 version = "0.1.3"
@@ -397,6 +416,15 @@ version = "1.0.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "546c37ac5d9e56f55e73b677106873d9d9f5190605e41a856503623648488cae"
 
+[[package]]
+name = "num-complex"
+version = "0.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "26873667bbbb7c5182d4a37c1add32cdf09f841af72da53318fdb81543c15085"
+dependencies = [
+ "num-traits",
+]
+
 [[package]]
 name = "num-derive"
 version = "0.3.3"
@@ -408,6 +436,26 @@ dependencies = [
  "syn",
 ]
 
+[[package]]
+name = "num-integer"
+version = "0.1.46"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f"
+dependencies = [
+ "num-traits",
+]
+
+[[package]]
+name = "num-rational"
+version = "0.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d41702bd167c2df5520b384281bc111a4b5efcf7fbc4c9c222c815b07e0a6a6a"
+dependencies = [
+ "autocfg",
+ "num-integer",
+ "num-traits",
+]
+
 [[package]]
 name = "num-traits"
 version = "0.2.15"
@@ -415,8 +463,15 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "578ede34cf02f8924ab9447f50c28075b4d3e5b269972345e7e0372b38c6cdcd"
 dependencies = [
  "autocfg",
+ "libm",
 ]
 
+[[package]]
+name = "paste"
+version = "1.0.15"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
+
 [[package]]
 name = "pin-project-lite"
 version = "0.2.9"
@@ -461,6 +516,7 @@ dependencies = [
  "build_zynq",
  "byteorder",
  "core_io",
+ "crc",
  "cslice",
  "dwarf",
  "dyld",
@@ -486,15 +542,6 @@ dependencies = [
  "void",
 ]
 
-[[package]]
-name = "rustc_version"
-version = "0.1.7"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c5f5376ea5e30ce23c03eb77cbe4962b988deead10910c372b226388b594c084"
-dependencies = [
- "semver",
-]
-
 [[package]]
 name = "satman"
 version = "0.0.0"
@@ -518,10 +565,16 @@ dependencies = [
 ]
 
 [[package]]
-name = "semver"
-version = "0.1.20"
+name = "simba"
+version = "0.8.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d4f410fedcf71af0345d7607d246e7ad15faaadd49d240ee3b24e5dc21a820ac"
+checksum = "50582927ed6f77e4ac020c057f37a268fc6aebc29225050365aacbb9deeeddc4"
+dependencies = [
+ "approx",
+ "num-complex",
+ "num-traits",
+ "paste",
+]
 
 [[package]]
 name = "smoltcp"
@@ -555,6 +608,12 @@ dependencies = [
  "log",
 ]
 
+[[package]]
+name = "typenum"
+version = "1.17.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "42ff0bf0c66b8238c6f3b578df37d0b7848e55df8577b3f74f92a69acceeb825"
+
 [[package]]
 name = "unicode-ident"
 version = "1.0.5"

src/armv7-none-eabihf.json

@@ -1,12 +1,4 @@
 {
-  "abi-blacklist": [
-    "stdcall",
-    "fastcall",
-    "vectorcall",
-    "thiscall",
-    "win64",
-    "sysv64"
-  ],
   "arch": "arm",
   "data-layout": "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64",
   "emit-debug-gdb-scripts": false,
@@ -21,7 +13,6 @@
   "os": "none",
   "panic-strategy": "abort",
   "requires-uwtable": true,
-  "force-unwind-tables": "yes",
   "relocation-model": "static",
   "target-c-int-width": "32",
   "target-endian": "little",

src/gateware/config.py

@@ -1,5 +1,15 @@
+import os
+from artiq._version import get_version
 from misoc.integration import cpu_interface
 
+
+def generate_ident(variant):
+    return "{}+{};{}".format(
+        get_version().split(".")[0],
+        os.getenv("ZYNQ_REV", default="unknown")[:8],
+        variant,
+    )
+
 def write_csr_file(soc, filename):
     with open(filename, "w") as f:
         f.write(cpu_interface.get_csr_rust(

src/gateware/cxp.py

@@ -0,0 +1,602 @@
+from migen import *
+from migen.genlib.cdc import MultiReg, PulseSynchronizer
+from misoc.interconnect.csr import *
+from misoc.interconnect.stream import StrideConverter
+
+from artiq.gateware.rtio import rtlink
+
+from cxp_downconn import CXP_RXPHYs
+from cxp_upconn import CXP_TXPHYs
+from cxp_pipeline import (
+    Command_Packet_Reader,
+    Command_Test_Packet_Writer,
+    Duplicated_Char_Decoder,
+    Heartbeat_Packet_Reader,
+    Idle_Word_Inserter,
+    Packet_Arbiter,
+    Packet_Wrapper,
+    Test_Sequence_Checker,
+    Trigger_ACK_Inserter,
+    Trigger_ACK_Reader,
+    Trigger_Inserter,
+    Trigger_Reader,
+)
+from cxp_frame_pipeline import *
+
+import cxp_router
+
+from types import SimpleNamespace
+
+class CXP_PHYS(Module, AutoCSR):
+    def __init__(self, refclk, upconn_pads, downconn_pads, sys_clk_freq, master=0):
+        assert len(upconn_pads) == len(downconn_pads)
+
+        self.submodules.tx = CXP_TXPHYs(upconn_pads, sys_clk_freq)
+        self.submodules.rx = CXP_RXPHYs(refclk, downconn_pads, sys_clk_freq, master)
+
+        self.phys = []
+        for tx, rx in zip(self.tx.phys, self.rx.phys):
+            phy = SimpleNamespace()
+            phy.tx, phy.rx = tx, rx
+            self.phys.append(phy)
+        
+
+class CXP_Core(Module, AutoCSR):
+    def __init__(self, phy, command_buffer_depth=32, nrxslot=4):
+        # control buffer is only 32 words (128 bytes) for compatibility with version1.x compliant Devices
+        # Section 12.1.6 (CXP-001-2021)
+        self.buffer_depth, self.nslots = command_buffer_depth, nrxslot
+        
+        self.submodules.tx = TX_Pipeline(phy.tx, command_buffer_depth, False)
+        self.submodules.rx = RX_Pipeline(phy.rx, command_buffer_depth, nrxslot, False)
+
+    def get_tx_port(self):
+        return self.tx.writer.mem.get_port(write_capable=True)
+
+    def get_rx_port(self):
+        return self.rx.command_reader.mem.get_port(write_capable=False)
+
+    def get_mem_size(self):
+        return word_width * self.buffer_depth * self.nslots // 8
+
+class RX_Pipeline(Module, AutoCSR):
+    def __init__(self, phy, command_buffer_depth, nslot, with_trigger):
+        self.ready = CSRStatus()
+
+        self.trigger_ack = CSR()
+
+        self.pending_packet = CSR()
+        self.read_ptr = CSRStatus(log2_int(nslot))
+        self.reader_buffer_err = CSR()
+
+        self.reader_decode_err = CSR()
+        self.test_error_counter = CSRStatus(16)
+        self.test_packet_counter = CSRStatus(16)
+        self.test_counts_reset = CSR()
+
+        self.heartbeat = CSR()
+        self.host_id = CSRStatus(32)
+        self.device_time = CSRStatus(64)
+
+        # for multilane router
+        self.active = Signal()
+
+        if with_trigger:
+            self.trig = Signal()
+            self.trig_delay = Signal(char_width)
+            self.trig_linktrigger_n = Signal(char_width)
+        
+        # # #
+
+        gtx = phy.gtx
+        self.sync += [
+            self.ready.status.eq(gtx.rx_ready),
+            self.active.eq(gtx.rx_ready),
+        ]
+
+        # Host rx pipeline 
+        #
+        #        32             32+8(dchar)
+        # PHY ───/───> dchar   ─────/─────> trigger  ─────> trigger ack ─────>  packet parser  ─────> EOP Marker ─────> stream data packet
+        #              decoder              reader          reader               │  │  │                                with CRC
+        #                                 (optional)                             │  │  └──────> test sequence checker
+        #                                                                        │  │
+        #                                                                        │  └─────────> heartbeat packet reader
+        #                                                                        │   
+        #                                                                        └────────────> command packet reader
+        #  
+        cdr = ClockDomainsRenamer("cxp_gtx_rx")
+
+        # decode all incoming data as duplicate char and inject the result into the bus for downstream modules
+        self.submodules.dchar_decoder = dchar_decoder = cdr(Duplicated_Char_Decoder())
+
+        # Priority level 0 packet - Trigger packet
+        if with_trigger:
+            self.submodules.trig_reader = trig_reader = cdr(Trigger_Reader())
+            self.sync.cxp_gtx_rx += [
+                self.trig.eq(trig_reader.trig),
+                self.trig_delay.eq(trig_reader.delay),
+                self.trig_linktrigger_n.eq(trig_reader.linktrigger_n),
+            ]
+
+        # Priority level 1 packet - Trigger ack packet
+        self.submodules.trig_ack_reader= trig_ack_reader = cdr(Trigger_ACK_Reader())
+        
+        self.submodules.trig_ack_ps = trig_ack_ps = PulseSynchronizer("cxp_gtx_rx", "sys")
+        self.sync.cxp_gtx_rx += trig_ack_ps.i.eq(trig_ack_reader.ack)
+        self.sync += [
+            If(trig_ack_ps.o,
+                self.trigger_ack.w.eq(1),
+            ).Elif(self.trigger_ack.re,
+                self.trigger_ack.w.eq(0),
+            ),
+        ]
+
+        # Priority level 2 packet - stream, test, heartbeat and command packets
+        self.submodules.arbiter = arbiter = cdr(Packet_Arbiter())
+
+        self.submodules.decode_err_ps = decode_err_ps = PulseSynchronizer("cxp_gtx_rx", "sys")
+        self.sync.cxp_gtx_rx += decode_err_ps.i.eq(arbiter.decode_err)
+        self.sync += [
+            If(decode_err_ps.o,
+                self.reader_decode_err.w.eq(1),
+            ).Elif(self.reader_decode_err.re,
+                self.reader_decode_err.w.eq(0),
+            ),
+        ]
+
+        if with_trigger:
+            rx_pipeline = [phy, dchar_decoder, trig_reader, trig_ack_reader, arbiter]
+        else:
+            rx_pipeline = [phy, dchar_decoder, trig_ack_reader, arbiter]
+        for s, d in zip(rx_pipeline, rx_pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+        # Stream packet
+        # Drop the K29.7 and mark the EOP for downstream
+        self.submodules.eop_marker = eop_marker = cdr(EOP_Marker())
+        self.comb += arbiter.source_stream.connect(eop_marker.sink)
+
+        # set pipeline source to output stream packet 
+        self.source = eop_marker.source
+
+
+        # Test packet 
+        self.submodules.test_seq_checker = test_seq_checker = cdr(Test_Sequence_Checker())
+        self.comb += arbiter.source_test.connect(test_seq_checker.sink)
+        
+        self.submodules.test_reset_ps = test_reset_ps = PulseSynchronizer("sys", "cxp_gtx_rx")
+        self.comb += test_reset_ps.i.eq(self.test_counts_reset.re),
+
+        test_err_cnt_rx = Signal.like(self.test_error_counter.status)
+        test_pak_cnt_rx = Signal.like(self.test_packet_counter.status)
+        test_err_r, test_pak_r = Signal(), Signal()
+        self.sync.cxp_gtx_rx += [ 
+            test_err_r.eq(test_seq_checker.error),
+            test_pak_r.eq(arbiter.recv_test_pak),
+
+            If(test_reset_ps.o,
+                test_err_cnt_rx.eq(test_err_cnt_rx.reset),
+            ).Elif(test_err_r,
+                test_err_cnt_rx.eq(test_err_cnt_rx + 1),
+            ),
+            If(test_reset_ps.o,
+                test_pak_cnt_rx.eq(test_pak_cnt_rx.reset),
+            ).Elif(test_pak_r,
+                test_pak_cnt_rx.eq(test_pak_cnt_rx + 1),
+            ),
+        ]
+        self.specials += [
+            MultiReg(test_err_cnt_rx, self.test_error_counter.status),
+            MultiReg(test_pak_cnt_rx, self.test_packet_counter.status),
+        ]
+
+        # Command packet
+        self.submodules.command_reader = command_reader = cdr(Command_Packet_Reader(command_buffer_depth, nslot))
+        self.comb += arbiter.source_command.connect(command_reader.sink)
+
+        # nslot buffers control interface
+        write_ptr_sys = Signal.like(command_reader.write_ptr)
+        
+        self.specials += [
+            MultiReg(self.read_ptr.status, command_reader.read_ptr, odomain="cxp_gtx_rx"),
+            MultiReg(command_reader.write_ptr, write_ptr_sys)
+        ]
+        self.sync += [
+            self.pending_packet.w.eq(self.read_ptr.status != write_ptr_sys),
+            If(~gtx.rx_ready,
+                self.read_ptr.status.eq(0),
+            ).Elif(self.pending_packet.re & self.pending_packet.w, 
+                self.read_ptr.status.eq(self.read_ptr.status + 1),
+            )
+        ]
+
+        self.submodules.buffer_err_ps = buffer_err_ps = PulseSynchronizer("cxp_gtx_rx", "sys")
+        self.sync.cxp_gtx_rx += buffer_err_ps.i.eq(command_reader.buffer_err),
+        self.sync += [
+            If(buffer_err_ps.o,
+                self.reader_buffer_err.w.eq(1),
+            ).Elif(self.reader_buffer_err.re,
+                self.reader_buffer_err.w.eq(0),
+            ),
+        ]
+
+        # Heartbeat packet
+        self.submodules.heartbeat_reader = heartbeat_reader = cdr(Heartbeat_Packet_Reader())
+        self.comb += arbiter.source_heartbeat.connect(heartbeat_reader.sink)
+
+        self.specials += [
+            MultiReg(heartbeat_reader.host_id, self.host_id.status),
+            MultiReg(heartbeat_reader.heartbeat, self.device_time.status),
+        ]
+        
+        self.submodules.heartbeat_ps = heartbeat_ps = PulseSynchronizer("cxp_gtx_rx", "sys")
+        self.sync.cxp_gtx_rx += heartbeat_ps.i.eq(arbiter.recv_heartbeat)
+        self.sync += [
+            If(heartbeat_ps.o,
+                self.heartbeat.w.eq(1),
+            ).Elif(self.heartbeat.re,
+                self.heartbeat.w.eq(0),
+            ),
+        ]
+        
+
+
+
+        
+class TX_Pipeline(Module, AutoCSR):
+    def __init__(self, phy, command_buffer_depth, with_trigger_ack):
+        self.trig_stb = Signal()
+        self.trig_delay = Signal(char_width)
+        self.trig_linktrigger_mode = Signal()
+
+        if with_trigger_ack:
+            self.trig_ack_stb = Signal()
+        
+        # # #
+        
+        # Host tx pipeline
+        #
+        #                 32                                                  32            8
+        # command/test ───/───> packet ─────> idle word ─────> trigger ack ───/───> conv ───/───> trigger ─────> PHY
+        # packet writer         wrapper       inserter         inserter                           inserter
+        #                                                      (optional)
+        #
+        # Equivalent transmission priority:
+        # trigger > tigger ack > idle word > command/test packet
+        #
+        # The pipeline is splited into 32 and 8 bits section to handle the word and char boundary priority insertion requirement:
+        # Insertion @ char boundary: trigger packets
+        # Insertion @ word boundary: idle packets and trigger ack packet
+        # - Section 9.2.4 (CXP-001-2021)
+        # 
+        # The idle inserter is placed between the trigger ack inserter and command/test packet writer to maintain the trigger performance,
+        # as idle word should not be inserted into trigger and trigger ack packet - Section 9.2.5.1 (CXP-001-2021) 
+        # 
+        
+
+        # Priority level 0 packet - Trigger packet
+        self.submodules.trig = trig = Trigger_Inserter()
+        self.comb += [
+            trig.stb.eq(self.trig_stb),
+            trig.delay.eq(self.trig_delay),
+            trig.linktrig_mode.eq(self.trig_linktrigger_mode)
+        ]
+
+        # Priority level 1 packet - Trigger ack
+        if with_trigger_ack:
+            self.submodules.trig_ack = trig_ack = Trigger_ACK_Inserter()
+            self.comb += self.trig_ack_stb.eq(trig_ack.stb)
+        
+        # Priority level 2 packet - command and test packet
+        # Control is not timing dependent, all the data packets are handled in firmware
+        self.submodules.writer = writer = Command_Test_Packet_Writer(command_buffer_depth)
+
+        # writer memory control interface
+        self.writer_word_len = CSRStorage(log2_int(command_buffer_depth))
+        self.writer_stb = CSR()
+        self.writer_stb_testseq = CSR()
+        self.writer_busy = CSRStatus()
+
+        self.sync += [
+            writer.word_len.eq(self.writer_word_len.storage),
+            writer.stb.eq(self.writer_stb.re),
+            writer.stb_testseq.eq(self.writer_stb_testseq.re),
+            self.writer_busy.status.eq(writer.busy),
+        ]
+
+        # Misc
+        self.submodules.pak_wrp = pak_wrp = Packet_Wrapper()
+        self.submodules.idle = idle = Idle_Word_Inserter()
+        self.submodules.converter = converter = StrideConverter(word_layout, char_layout)
+
+        if with_trigger_ack:
+            tx_pipeline = [writer, pak_wrp, idle, trig_ack, converter, trig, phy]
+        else:
+            tx_pipeline = [writer, pak_wrp, idle, converter, trig, phy]
+
+        for s, d in zip(tx_pipeline, tx_pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+
+class CXP_Frame_Pipeline(Module, AutoCSR):
+    # optimal stream packet size is 2 KiB - Section 9.5.2 (CXP-001-2021)
+    def __init__(self, pipelines, pmod_pads, roi_engine_count=1, res_width=16, count_width=31, master=0, packet_size=16384):
+        n_channels = len(pipelines)
+        assert n_channels > 0
+        assert count_width <= 31
+
+        # Trigger rtio
+        nbit_trigdelay = 8
+        nbit_linktrig = 1
+        self.trigger = rtlink.Interface(
+            rtlink.OInterface(nbit_trigdelay + nbit_linktrig),
+            rtlink.IInterface(word_width, timestamped=False)
+        )
+
+        self.sync.rio += [
+            If(self.trigger.o.stb,
+                pipelines[master].tx.trig_delay.eq(self.trigger.o.data[nbit_linktrig:]),
+                pipelines[master].tx.trig_linktrigger_mode.eq(self.trigger.o.data[:nbit_linktrig]),
+            ),
+            pipelines[master].tx.trig_stb.eq(self.trigger.o.stb),
+        ]
+
+
+        # ROI rtio
+         
+        # 4 cfg (x0, y0, x1, y1) per roi_engine
+        self.config = rtlink.Interface(rtlink.OInterface(res_width, bits_for(4*roi_engine_count-1)))
+
+        # select which roi engine can output rtio_input signal
+        self.gate_data = rtlink.Interface(
+            rtlink.OInterface(roi_engine_count),
+            # the 32th bits is for sentinel (gate detection)
+            rtlink.IInterface(count_width+1, timestamped=False)
+        )
+
+        self.crc_error_cnt = CSRStatus(16)
+        self.crc_error_reset = CSR()
+
+        # DEBUG: csr
+        self.arbiter_active_ch = CSRStatus(n_channels)
+
+        self.roi_counter = CSRStatus(count_width)
+        self.roi_update = CSR()
+        self.pix_y = CSRStatus(res_width)
+
+        self.header_l_size = CSRStatus(3*char_width)
+        self.header_x_size = CSRStatus(3*char_width)
+        self.header_y_size = CSRStatus(3*char_width)
+        self.header_new_line = CSRStatus(3*char_width)
+        
+        # # #
+        
+        cdr = ClockDomainsRenamer("cxp_gtx_rx")
+    
+        debug_out = False
+
+        if not debug_out:
+            self.submodules.pixel_pipeline = pixel_pipeline = cdr(Pixel_Pipeline(res_width, count_width, packet_size))
+
+            # CRC error counter
+            self.submodules.crc_reset_ps = crc_reset_ps = PulseSynchronizer("sys", "cxp_gtx_rx")
+            self.comb += crc_reset_ps.i.eq(self.crc_error_reset.re)
+
+            crc_error_cnt_rx = Signal.like(self.crc_error_cnt.status)
+
+            crc_error_r = Signal()
+            self.sync.cxp_gtx_rx += [
+                # to improve timinig
+                crc_error_r.eq(pixel_pipeline.crc_checker.error),
+
+                If(crc_reset_ps.o,
+                    crc_error_cnt_rx.eq(crc_error_cnt_rx.reset),
+                ).Elif(crc_error_r,
+                    crc_error_cnt_rx.eq(crc_error_cnt_rx + 1),
+                ),
+            ]
+            self.specials += MultiReg(crc_error_cnt_rx, self.crc_error_cnt.status)
+
+            # RTIO interface
+
+            n = 0
+            cfg = pixel_pipeline.roi.cfg
+            for offset, target in enumerate([cfg.x0, cfg.y0, cfg.x1, cfg.y1]):
+                roi_boundary = Signal.like(target)
+                self.sync.rio += If(self.config.o.stb & (self.config.o.address == 4*n+offset),
+                                roi_boundary.eq(self.config.o.data))
+                self.specials += MultiReg(roi_boundary, target, "cxp_gtx_rx")
+
+
+
+            roi_out = pixel_pipeline.roi.out
+            update = Signal()
+            self.submodules.ps = ps = PulseSynchronizer("cxp_gtx_rx", "sys")
+            self.sync.cxp_gtx_rx += ps.i.eq(roi_out.update)
+            self.sync += update.eq(ps.o)
+
+            sentinel = 2**count_width
+            count_sys = Signal.like(roi_out.count)
+
+            self.specials += MultiReg(roi_out.count, count_sys),
+            self.sync.rio += [
+                # TODO: add gating
+                self.gate_data.i.stb.eq(update),
+                # without the slice, unspecified bits will be 1 for some reason
+                # i.e. data[count_wdith:] = 0b111111... when using data.eq(count_sys)
+                self.gate_data.i.data[:count_width].eq(count_sys),
+            ]
+
+            
+
+            # TODO: fix count_sys seems like end of frame is broken
+            # BUG: it maybe related to the KCode bug that only happens after frameheader decoder COPY (i.e. when sending pixel data)
+            # DEBUG:
+
+            new_line_cnt_rx, new_line_cnt_sys = Signal(3*char_width), Signal(3*char_width)
+            l_size_rx, l_size_sys = Signal(3*char_width), Signal(3*char_width)
+            x_size_rx, x_size_sys = Signal(3*char_width), Signal(3*char_width)
+            y_size_rx, y_size_sys = Signal(3*char_width), Signal(3*char_width)
+            y_pix_rx, y_pix_sys = Signal(res_width), Signal(res_width)
+            self.sync.cxp_gtx_rx += [
+                If(pixel_pipeline.header_decoder.new_line,
+                    new_line_cnt_rx.eq(new_line_cnt_rx + 1),
+                ),
+
+                l_size_rx.eq(pixel_pipeline.header_decoder.metadata.l_size),
+                x_size_rx.eq(pixel_pipeline.header_decoder.metadata.x_size),
+                y_size_rx.eq(pixel_pipeline.header_decoder.metadata.y_size),                
+
+                y_pix_rx.eq(pixel_pipeline.parser.pixel4x[0].y),
+            ]
+            self.specials += [
+                MultiReg(new_line_cnt_rx, new_line_cnt_sys),
+                MultiReg(l_size_rx, l_size_sys),
+                MultiReg(x_size_rx, x_size_sys),
+                MultiReg(y_size_rx, y_size_sys),
+                MultiReg(y_pix_rx, y_pix_sys),
+            ]
+            self.sync += [
+                self.header_new_line.status.eq(new_line_cnt_sys),
+                self.pix_y.status.eq(y_pix_sys),
+                self.header_l_size.status.eq(l_size_sys),
+                self.header_x_size.status.eq(x_size_sys),
+                self.header_y_size.status.eq(y_size_sys),
+                self.roi_counter.status.eq(count_sys),
+                If(update,
+                    self.roi_update.w.eq(1),
+                ).Elif(self.roi_update.re,
+                    self.roi_update.w.eq(0),
+                ),
+            ]
+
+        else:
+            # DEBUG:
+            crc_checker = cdr(CXPCRC32_Checker())
+
+            # TODO: handle full buffer gracefully
+            # TODO: investigate why there is a heartbeat message in the middle of the frame with k27.7 code too???
+            # NOTE: sometimes there are 0xFBFBFBFB K=0b1111
+            # perhaps the buffer is full overflowing and doing strange stuff             
+
+            # it should be mem block not "cycle buffer"
+            # self.submodules.dropper = dropper = cdr(DChar_Dropper())
+            buffer = cdr(Buffer(word_layout_dchar)) # crcchecker timinig is bad
+            buffer_cdc_fifo = cdr(Buffer(word_layout_dchar)) # to improve timing
+            cdc_fifo = stream.AsyncFIFO(word_layout_dchar, 2**log2_int(packet_size//word_width))
+            self.submodules += buffer, crc_checker, buffer_cdc_fifo
+            self.submodules += ClockDomainsRenamer({"write": "cxp_gtx_rx", "read": "sys"})(cdc_fifo)
+            self.submodules.debug_out = debug_out = RX_Debug_Buffer(word_layout_dchar, 2**log2_int(packet_size//word_width))
+
+            pipeline = [buffer, crc_checker, buffer_cdc_fifo, cdc_fifo, debug_out]
+            for s, d in zip(pipeline, pipeline[1:]):
+                self.comb += s.source.connect(d.sink)
+
+
+
+        # Routing WIP
+        #                        +---------+     +-------------+
+        # downconn pipline ----->|         |     |             |------> crc checker ------> raw stream data 
+        #                        | arbiter |---->| broadcaster |
+        # downconn pipline ----->|         |     |             |------> crc checker ------> raw stream data 
+        #                        +---------+     +-------------+
+        #
+
+
+        self.submodules.arbiter = arbiter = cdr(Stream_Arbiter(n_channels))
+        self.submodules.broadcaster = broadcaster = cdr(Stream_Broadcaster())
+
+        # Connect pipeline
+        for i, p in enumerate(pipelines):
+            # Assume downconns pipeline already marks the eop
+            self.comb += p.rx.source.connect(arbiter.sinks[i])
+
+        self.comb += arbiter.source.connect(broadcaster.sink)
+
+        if not debug_out:
+            self.comb += broadcaster.sources[0].connect(pixel_pipeline.sink),
+        else:
+            self.comb += broadcaster.sources[0].connect(pipeline[0].sink),
+
+        # Control interface
+        # only the simple topology MASTER:ch0, extension:ch1,2,3 is supported right now
+        active_channels_sys = Signal(n_channels)
+        for i, p in enumerate(pipelines):
+            # TODO: change this to non csr signal?
+            self.sync += active_channels_sys[i].eq(p.rx.active)
+        self.specials += MultiReg(active_channels_sys, arbiter.active_channels, odomain="cxp_gtx_rx"),
+
+        # DEBUG:
+        self.sync += self.arbiter_active_ch.status.eq(active_channels_sys)
+
+        for i, p in enumerate(pipelines):
+            # self.comb += p.rx.source.ack.eq(1)
+
+            rx_stb = Signal()
+            self.sync.cxp_gtx_rx += rx_stb.eq(p.rx.source.stb)
+            self.specials += [
+                # Instance("OBUF", i_I=rx_stb, o_O=pmod_pads[i]),
+                # Instance("OBUF", i_I=arbiter.sinks[i].stb, o_O=pmod_pads[i]),
+            ]
+
+class NEO_CXP_Frame_pipeline(Module):
+    # optimal stream packet size is 2 KiB - Section 9.5.2 (CXP-001-2021)
+    def __init__(self, pipelines, pmod_pads, roi_engine_count=1, res_width=16, count_width=31, master=0, packet_size=16384):
+        n_channels = len(pipelines)
+        assert n_channels > 0
+        assert count_width <= 31
+
+        # Trigger rtio
+        nbit_trigdelay = 8
+        nbit_linktrig = 1
+        self.trigger = rtlink.Interface(
+            rtlink.OInterface(nbit_trigdelay + nbit_linktrig),
+            rtlink.IInterface(word_width, timestamped=False)
+        )
+
+        self.sync.rio += [
+            If(self.trigger.o.stb,
+                pipelines[master].tx.trig.delay.eq(self.trigger.o.data[nbit_linktrig:]),
+                pipelines[master].tx.trig.linktrig_mode.eq(self.trigger.o.data[:nbit_linktrig]),
+            ),
+            pipelines[master].tx.trig.stb.eq(self.trigger.o.stb),
+        ]
+
+        
+        # ROI rtio
+         
+        # 4 cfg (x0, y0, x1, y1) per roi_engine
+        self.config = rtlink.Interface(rtlink.OInterface(res_width, bits_for(4*roi_engine_count-1)))
+
+        # select which roi engine can output rtio_input signal
+        self.gate_data = rtlink.Interface(
+            rtlink.OInterface(roi_engine_count),
+            # the 32th bits is for sentinel (gate detection)
+            rtlink.IInterface(count_width+1, timestamped=False)
+        )
+
+       
+        # # #
+        
+        cdr = ClockDomainsRenamer("cxp_gtx_rx")
+    
+        debug_out = False
+
+        if debug_out:
+            pass
+
+
+        #
+        # downconn pipline -----> router -----> arbiter ------> crc checker ------> 4x converters
+        #
+
+        # Connect pipeline
+        for i, p in enumerate(pipelines):
+            broadcaster = cdr(cxp_router.Stream_Router()) # strip the packet id, tag & packet size
+            crc_checker = cdr(CXPCRC32_Checker())
+            self.submodules += broadcaster, crc_checker
+
+            self.comb += [
+                p.rx.source.connect(broadcaster.sink),
+                broadcaster.sources[0].connect(crc_checker),
+            ]

src/gateware/cxp_4r_fmc.py

@@ -0,0 +1,82 @@
+from migen.build.generic_platform import *
+
+fmc_adapter_io = [
+    # CoaXPress high speed link
+    ("CXP_HS", 0,
+        Subsignal("txp", Pins("HPC:DP0_C2M_P")),
+        Subsignal("txn", Pins("HPC:DP0_C2M_N")),
+        Subsignal("rxp", Pins("HPC:DP0_M2C_P")),
+        Subsignal("rxn", Pins("HPC:DP0_M2C_N")),
+    ),
+    ("CXP_HS", 1,
+        Subsignal("txp", Pins("HPC:DP1_C2M_P")),
+        Subsignal("txn", Pins("HPC:DP1_C2M_N")),
+        Subsignal("rxp", Pins("HPC:DP1_M2C_P")),
+        Subsignal("rxn", Pins("HPC:DP1_M2C_N")),
+    ),
+    ("CXP_HS", 2,
+        Subsignal("txp", Pins("HPC:DP2_C2M_P")),
+        Subsignal("txn", Pins("HPC:DP2_C2M_N")),
+        Subsignal("rxp", Pins("HPC:DP2_M2C_P")),
+        Subsignal("rxn", Pins("HPC:DP2_M2C_N")),
+    ),
+    ("CXP_HS", 3,
+        Subsignal("txp", Pins("HPC:DP3_C2M_P")),
+        Subsignal("txn", Pins("HPC:DP3_C2M_N")),
+        Subsignal("rxp", Pins("HPC:DP3_M2C_P")),
+        Subsignal("rxn", Pins("HPC:DP3_M2C_N")),
+    ),
+
+    # CoaXPress low speed link
+    ("CXP_LS", 0, Pins("HPC:LA00_CC_P"), IOStandard("LVCMOS33")),
+    ("CXP_LS", 1, Pins("HPC:LA01_CC_N"), IOStandard("LVCMOS33")),
+    ("CXP_LS", 2, Pins("HPC:LA01_CC_P"), IOStandard("LVCMOS33")),
+    ("CXP_LS", 3, Pins("HPC:LA02_N"), IOStandard("LVCMOS33")),
+
+    # CoaXPress green and red LED
+    ("CXP_LED", 0,
+        Subsignal("green", Pins("HPC:LA11_P"), IOStandard("LVCMOS33")),
+        Subsignal("red", Pins("HPC:LA11_N"), IOStandard("LVCMOS33")),
+    ),
+    ("CXP_LED", 1,
+        Subsignal("green", Pins("HPC:LA12_P"), IOStandard("LVCMOS33")),
+        Subsignal("red", Pins("HPC:LA12_N"), IOStandard("LVCMOS33")),
+    ),
+    ("CXP_LED", 2,
+        Subsignal("green", Pins("HPC:LA13_P"), IOStandard("LVCMOS33")),
+        Subsignal("red", Pins("HPC:LA13_N"), IOStandard("LVCMOS33")),
+    ),
+    ("CXP_LED", 3,
+        Subsignal("green", Pins("HPC:LA14_P"), IOStandard("LVCMOS33")),
+        Subsignal("red", Pins("HPC:LA14_N"), IOStandard("LVCMOS33")),
+    ),
+
+    # Power over CoaXPress 
+    ("PoCXP", 0,
+        Subsignal("enable", Pins("HPC:LA21_N"), IOStandard("LVCMOS33")),
+        Subsignal("alert", Pins("HPC:LA18_CC_P"), IOStandard("LVCMOS33")),
+    ),
+    ("PoCXP", 1,
+        Subsignal("enable", Pins("HPC:LA21_P"), IOStandard("LVCMOS33")),
+        Subsignal("alert", Pins("HPC:LA19_N"), IOStandard("LVCMOS33")),
+    ),
+    ("PoCXP", 2,
+        Subsignal("enable", Pins("HPC:LA22_N"), IOStandard("LVCMOS33")),
+        Subsignal("alert", Pins("HPC:LA19_P"), IOStandard("LVCMOS33")),
+    ),
+    ("PoCXP", 3,
+        Subsignal("enable", Pins("HPC:LA22_P"), IOStandard("LVCMOS33")),
+        Subsignal("alert", Pins("HPC:LA20_N"), IOStandard("LVCMOS33")),
+    ),
+    ("i2c", 0,
+        Subsignal("scl", Pins("HPC:IIC_SCL")),
+        Subsignal("sda", Pins("HPC:IIC_SDA")),
+        IOStandard("LVCMOS33")
+    ),
+
+    # On board 125MHz (25ppm) reference 
+    ("clk125", 0,
+        Subsignal("p", Pins("HPC:GBTCLK0_M2C_P")),
+        Subsignal("n", Pins("HPC:GBTCLK0_M2C_N")),
+    ),
+]

src/gateware/cxp_downconn.py

@@ -0,0 +1,688 @@
+from migen import *
+from migen.genlib.cdc import MultiReg
+from migen.genlib.resetsync import AsyncResetSynchronizer
+
+from misoc.cores.code_8b10b import Encoder, Decoder
+from misoc.interconnect.csr import *
+from misoc.interconnect import stream
+
+from artiq.gateware.drtio.transceiver.gtx_7series_init import *
+from cxp_pipeline import word_layout
+
+from functools import reduce
+from operator import add
+
+class CXP_RXPHYs(Module, AutoCSR):
+    def __init__(self, gt_refclk, pads, sys_clk_freq, master):
+        self.qpll_reset = CSR()
+        self.qpll_locked = CSRStatus()
+        self.gtx_start_init = CSRStorage()
+        self.gtx_restart = CSR()
+
+        # DRP port
+        self.gtx_daddr = CSRStorage(9)
+        self.gtx_dread = CSR()
+        self.gtx_din_stb = CSR()
+        self.gtx_din = CSRStorage(16)
+        self.gtx_dout = CSRStatus(16)
+        self.gtx_dready = CSR()
+
+        self.phys = []
+        # # #
+
+        # For speed higher than 6.6Gbps, QPLL need to be used instead of CPLL - DS191 (v1.18.1) Table 9.1
+        self.submodules.qpll = qpll = QPLL(gt_refclk, sys_clk_freq)
+        self.sync += [
+            qpll.reset.eq(self.qpll_reset.re),
+            self.qpll_locked.status.eq(qpll.lock),
+        ]
+
+        self.submodules.rx_resetter = rx_resetter = RX_Resetter()
+
+        for i, pad in enumerate(pads):
+            if len(pads) == 1:
+                rx_mode = "single"
+            else:
+                rx_mode = "master" if i == master else "slave"
+            rx = Receiver(qpll, pad, sys_clk_freq, rx_mode)
+            self.phys.append(rx)
+            setattr(self.submodules, "rx"+str(i), rx)
+
+        for i, phy in enumerate(self.phys):
+            if i == master:
+                self.comb += rx_resetter.rx_ready.eq(phy.gtx.rx_ready)
+                # Connect master GTX connections' output DRP
+                self.sync += [
+                    If(phy.gtx.dready,
+                        self.gtx_dready.w.eq(1),
+                        self.gtx_dout.status.eq(phy.gtx.dout),
+                    ).Elif(self.gtx_dready.re,
+                        self.gtx_dready.w.eq(0),
+                    ),
+                ]
+
+            # Connect all GTX connections' input DRP
+            self.sync += [
+                phy.gtx.daddr.eq(self.gtx_daddr.storage),
+                phy.gtx.den.eq(self.gtx_dread.re | self.gtx_din_stb.re),
+                phy.gtx.dwen.eq(self.gtx_din_stb.re),
+                phy.gtx.din.eq(self.gtx_din.storage),
+            ]
+            self.comb += [
+                phy.gtx.dclk.eq(ClockSignal("sys")),
+                phy.gtx.rx_manual_restart.eq(self.gtx_restart.re | rx_resetter.rx_reset),
+                phy.gtx.rx_init.clk_path_ready.eq(self.gtx_start_init.storage),
+            ]
+
+        # master rx_init will lock up when slaves_phaligndone signal is not connected
+        self.submodules.rx_phase_alignment = GTXInitPhaseAlignment([rx_phy.gtx.rx_init for rx_phy in self.phys])
+
+class Receiver(Module):
+    def __init__(self, qpll, pad, sys_clk_freq, rx_mode):
+        self.submodules.gtx = gtx = GTX(qpll, pad, sys_clk_freq, None, rx_mode)
+       
+        self.source = stream.Endpoint(word_layout)
+
+        data_valid = Signal()
+        self.sync.cxp_gtx_rx += [
+            data_valid.eq(gtx.comma_aligner.rxfsm.ongoing("READY")),
+
+            self.source.stb.eq(0),
+            If(data_valid & ~((gtx.decoders[0].d == 0xBC) & (gtx.decoders[0].k == 1)),
+                self.source.stb.eq(1),
+                self.source.data.eq(Cat(gtx.decoders[i].d for i in range(4))),
+                self.source.k.eq(Cat(gtx.decoders[i].k for i in range(4))),
+            )
+        ]
+
+class QPLL(Module, AutoCSR):
+    def __init__(self, gt_refclk, sys_clk_freq):
+        self.clk = Signal()
+        self.refclk = Signal()
+        self.lock = Signal()
+        self.reset = Signal()
+
+        self.daddr = CSRStorage(8)
+        self.dread = CSR()
+        self.din_stb = CSR()
+        self.din = CSRStorage(16)
+
+        self.dout = CSRStatus(16)
+        self.dready = CSR()
+
+        # # #
+
+        # VCO @ 10GHz, linerate = 1.25Gbps
+        # feedback divider = 80
+        qpll_fbdiv = 0b0100100000
+        qpll_fbdiv_ratio = 1
+        refclk_div = 1
+        self.Xxout_div = 8
+
+        # used for txuserclk pll
+        fbdiv_real = 80
+        self.tx_usrclk_freq = (sys_clk_freq*fbdiv_real/self.Xxout_div)/40
+        
+        dready = Signal()
+        self.specials += [
+            Instance("GTXE2_COMMON",
+                i_QPLLREFCLKSEL=0b001,
+                i_GTREFCLK0=gt_refclk,         
+
+                i_QPLLPD=0,
+                i_QPLLRESET=self.reset,
+                i_QPLLLOCKEN=1,
+                o_QPLLLOCK=self.lock,
+                o_QPLLOUTCLK=self.clk,
+                o_QPLLOUTREFCLK=self.refclk,
+
+                # See UG476 (v1.12.1) Table 2-16
+                p_QPLL_FBDIV=qpll_fbdiv,
+                p_QPLL_FBDIV_RATIO=qpll_fbdiv_ratio,
+                p_QPLL_REFCLK_DIV=refclk_div,
+
+                # From 7 Series FPGAs Transceivers Wizard
+                p_BIAS_CFG=0x0000040000001000,
+                p_COMMON_CFG=0x00000000,
+                p_QPLL_CFG=0x0680181,
+                p_QPLL_CLKOUT_CFG=0b0000,
+                p_QPLL_COARSE_FREQ_OVRD=0b010000,
+                p_QPLL_COARSE_FREQ_OVRD_EN=0b0,
+                p_QPLL_CP=0b0000011111,
+                p_QPLL_CP_MONITOR_EN=0b0,
+                p_QPLL_DMONITOR_SEL=0b0,
+                p_QPLL_FBDIV_MONITOR_EN= 0b0,
+                p_QPLL_INIT_CFG=0x000006,
+                p_QPLL_LOCK_CFG=0x21E8,
+                p_QPLL_LPF=0b1111,
+             
+                # Reserved, values cannot be modified
+                i_BGBYPASSB=0b1,
+                i_BGMONITORENB=0b1,
+                i_BGPDB=0b1,
+                i_BGRCALOVRD=0b11111,
+                i_RCALENB=0b1,
+                i_QPLLRSVD1=0b0,
+                i_QPLLRSVD2=0b11111,
+
+                # Dynamic Reconfiguration Ports
+                i_DRPADDR=self.daddr.storage,
+                i_DRPCLK=ClockSignal("sys"),
+                i_DRPEN=(self.dread.re | self.din_stb.re),
+                i_DRPWE=self.din_stb.re,
+                i_DRPDI=self.din.storage,
+                o_DRPDO=self.dout.status,
+                o_DRPRDY=dready,
+            )
+        ]
+
+        self.sync += [
+            If(dready,
+               self.dready.w.eq(1),
+            ),
+            If(self.dready.re,
+               self.dready.w.eq(0),
+            ),
+        ]
+
+
+class RX_Resetter(Module):
+    def __init__(self, reset_period=10_000_000):
+        self.rx_ready = Signal()
+        self.rx_reset = Signal()
+
+        # # #
+
+        # periodically reset rx until rx is connected and receiving valid data
+        # as after connecting RXP/RXN, the whole RX need to be reset 
+        
+        reset_counter = Signal(reset=reset_period-1, max=reset_period)
+        self.sync += [
+            self.rx_reset.eq(0),
+            If(~self.rx_ready,
+                If(reset_counter == 0,
+                    reset_counter.eq(reset_counter.reset),
+                    self.rx_reset.eq(1),
+                ).Else(
+                    reset_counter.eq(reset_counter - 1),
+                )
+            )
+        ]
+
+# Warning: Xilinx transceivers are LSB first, and comma needs to be flipped
+# compared to the usual 8b10b binary representation.
+class Comma_Aligner(Module):
+    def __init__(self, comma, reset_period=10_000_000):
+        self.data = Signal(20)
+        self.comma_aligned = Signal()
+        self.comma_realigned = Signal()
+        self.comma_det = Signal()
+
+        self.aligner_en = Signal()
+        self.ready_sys = Signal()
+
+        # # #
+
+        # Data and comma checker
+        # From UG476 (v1.12.1) p.228
+        # The built-in RXBYTEISALIGNED can be falsely asserted at linerate higher than 5Gbps
+        # The validity of data and comma needed to be checked externally
+
+        comma_n = ~comma & 0b1111111111
+
+        comma_seen = Signal()
+        error_seen = Signal()
+        one_counts = Signal(max=11)
+
+        # From CXP-001-2021 section 9.2.5.1
+        # For high speed connection an IDLE word shall be transmitted at least once every 100 words
+        counter_period = 200
+
+        counter = Signal(reset=counter_period-1, max=counter_period)
+        check_reset = Signal()
+        check = Signal()
+
+        self.sync.cxp_gtx_rx += [
+            If(check_reset,
+                counter.eq(counter.reset),   
+                check.eq(0),
+            ).Elif(counter == 0,
+                check.eq(1),
+            ).Else(
+                counter.eq(counter - 1),   
+            ),
+
+            If(check_reset,
+                comma_seen.eq(0),
+            ).Elif((self.data[:10] == comma) | (self.data[:10] == comma_n),
+                comma_seen.eq(1)
+            ),
+
+            one_counts.eq(reduce(add, [self.data[i] for i in range(10)])),
+            If(check_reset,
+                error_seen.eq(0),
+            ).Elif((one_counts != 4) & (one_counts != 5) & (one_counts != 6),
+                error_seen.eq(1),
+            ),
+
+        ]
+
+        self.submodules.rxfsm = rxfsm = ClockDomainsRenamer("cxp_gtx_rx")(FSM(reset_state="WAIT_COMMA"))
+
+        rxfsm.act("WAIT_COMMA",
+            If(self.comma_det,
+                NextState("ALIGNING"),
+            )
+        )
+
+        rxfsm.act("ALIGNING",
+            If(self.comma_aligned & (~self.comma_realigned),
+                NextState("WAIT_ALIGNED_DATA"),               
+            ).Else(
+                self.aligner_en.eq(1),
+            )
+        )
+
+        # wait for the aligned data to arrive at the FPGA RX interface 
+        # as there is a delay before the data is avaiable after RXBYTEISALIGNED is asserted
+        self.submodules.timer = timer = ClockDomainsRenamer("cxp_gtx_rx")(WaitTimer(10_000))
+
+        rxfsm.act("WAIT_ALIGNED_DATA",
+            timer.wait.eq(1),
+            If(timer.done,
+                check_reset.eq(1),
+                NextState("CHECKING"),
+            )
+        )
+
+        rxfsm.act("CHECKING",
+            If(check,
+                check_reset.eq(1),
+                If(comma_seen & (~error_seen),
+                    NextState("READY"),
+                ).Else(
+                    NextState("WAIT_COMMA")
+                )
+            )
+        )
+
+        ready = Signal()
+        self.specials += MultiReg(ready, self.ready_sys)
+        rxfsm.act("READY",
+            ready.eq(1),
+            If(check,
+                check_reset.eq(1),
+                If(~(comma_seen & (~error_seen)),
+                    NextState("WAIT_COMMA"),
+                )
+            )
+        )
+
+
+class GTX(Module):
+    """
+    A linerate reconfigurable 40bit width GTX with QPLL
+    Designed for 12.5, 10, 6.25, 5, 3.125, 2.5, 1.25Gpbs
+
+    To change the linerate:
+    1) Change the QPLL VCO frequency
+        - 12.5, 6.25, 3.125Gbps: VCO @ 12.5GHz,  
+        - 10, 6.25, 5, 2.5, 1.25Gbps: VCO @ 10GHz 
+    2) Update the xXOUT_DIV and TXUSRCLK frequency if using tx
+    3) Reset the entire rx and tx
+    """
+    def __init__(self, qpll, pads, sys_clk_freq, tx_mode="single", rx_mode="single"):
+        assert tx_mode in ["single", "master", "slave", None]
+        assert rx_mode in ["single", "master", "slave", None]
+
+
+        self.tx_restart = Signal()
+        self.rx_manual_restart = Signal()
+        self.loopback_mode = Signal(3)
+
+        self.txenable = Signal()
+        self.rx_ready = Signal()
+
+        # Dynamic Reconfiguration Ports
+        self.daddr = Signal(9)
+        self.dclk = Signal()
+        self.den = Signal()
+        self.dwen = Signal()
+        self.din = Signal(16)
+        self.dout = Signal(16)
+        self.dready = Signal()
+
+
+
+        # transceiver direct clock outputs
+        # useful to specify clock constraints in a way palatable to Vivado
+        self.txoutclk = Signal()
+        self.rxoutclk = Signal()
+
+        # # #
+
+        txdata = Signal(40)
+        rxdata = Signal(40)
+
+        if tx_mode:
+            self.submodules.tx_init = tx_init = GTXInit(sys_clk_freq, False, mode=tx_mode)
+            self.comb += tx_init.cplllock.eq(qpll.lock),
+
+            self.submodules.encoder = ClockDomainsRenamer("cxp_gtx_tx")(Encoder(4, True))
+            self.comb += txdata.eq(Cat(self.encoder.output[0], self.encoder.output[1], self.encoder.output[2], self.encoder.output[3])),
+
+        if rx_mode:
+            self.submodules.rx_init = rx_init = GTXInit(sys_clk_freq, True, mode=rx_mode)
+            self.comb += rx_init.cplllock.eq(qpll.lock)
+
+            self.submodules.decoders = [ClockDomainsRenamer("cxp_gtx_rx")(
+                (Decoder(True))) for _ in range(4)]
+            self.comb += [
+                self.decoders[0].input.eq(rxdata[:10]),
+                self.decoders[1].input.eq(rxdata[10:20]),
+                self.decoders[2].input.eq(rxdata[20:30]),
+                self.decoders[3].input.eq(rxdata[30:]),
+            ]
+
+        comma_aligned = Signal()
+        comma_realigned = Signal()
+        comma_det = Signal()
+        comma_aligner_en = Signal()
+        # Note: the following parameters were set after consulting AR45360
+        self.specials += \
+            Instance("GTXE2_CHANNEL",
+                # PMA Attributes
+                p_PMA_RSV=0x001E7080,
+                p_PMA_RSV2=0x2050,              # PMA_RSV2[5] = 0: Eye scan feature disabled
+                p_PMA_RSV3=0,
+                p_PMA_RSV4=1,                   # PMA_RSV[4],RX_CM_TRIM[2:0] = 0b1010: Common mode 800mV
+                p_RX_BIAS_CFG=0b000000000100,
+                p_RX_OS_CFG=0b0000010000000,
+                p_RX_CLK25_DIV=5,
+                p_TX_CLK25_DIV=5,
+
+                # Power-Down Attributes
+                p_PD_TRANS_TIME_FROM_P2=0x3c,
+                p_PD_TRANS_TIME_NONE_P2=0x3c,
+                p_PD_TRANS_TIME_TO_P2=0x64,
+                i_CPLLPD=1,
+
+                # QPLL
+                i_QPLLCLK=qpll.clk,
+                i_QPLLREFCLK=qpll.refclk,
+                p_RXOUT_DIV=qpll.Xxout_div,
+                p_TXOUT_DIV=qpll.Xxout_div,
+                i_RXSYSCLKSEL=0b11, # use QPLL & QPLL's REFCLK
+                i_TXSYSCLKSEL=0b11, # use QPLL & CPLL's REFCLK
+
+                # TX clock
+                p_TXBUF_EN="FALSE",
+                p_TX_XCLK_SEL="TXUSR",
+                o_TXOUTCLK=self.txoutclk,
+                # i_TXSYSCLKSEL=0b00,
+                i_TXOUTCLKSEL=0b11,
+
+                # TX Startup/Reset
+                i_TXPHDLYRESET=0,
+                i_TXDLYBYPASS=0,
+                i_TXPHALIGNEN=1 if tx_mode in ["master", "slave"] else 0,
+                i_GTTXRESET=tx_init.gtXxreset if tx_mode else 0,
+                o_TXRESETDONE=tx_init.Xxresetdone if tx_mode else 0,
+                i_TXDLYSRESET=tx_init.Xxdlysreset if tx_mode else 0,
+                o_TXDLYSRESETDONE=tx_init.Xxdlysresetdone if tx_mode else 0,
+                i_TXPHINIT=tx_init.txphinit if tx_mode in ["master", "slave"] else 0,
+                o_TXPHINITDONE=tx_init.txphinitdone if tx_mode in ["master", "slave"] else Signal(),
+                i_TXPHALIGN=tx_init.Xxphalign if tx_mode in ["master", "slave"] else 0,
+                i_TXDLYEN=tx_init.Xxdlyen if tx_mode in ["master", "slave"] else 0,
+                o_TXPHALIGNDONE=tx_init.Xxphaligndone if tx_mode else 0,
+                i_TXUSERRDY=tx_init.Xxuserrdy if tx_mode else 0,
+                p_TXPMARESET_TIME=1,
+                p_TXPCSRESET_TIME=1,
+                i_TXINHIBIT=~self.txenable,
+
+                # TX data
+                p_TX_DATA_WIDTH=40,
+                p_TX_INT_DATAWIDTH=1, # 1 if a line rate is greater than 6.6 Gbps
+                i_TXCHARDISPMODE=Cat(txdata[9], txdata[19], txdata[29], txdata[39]),
+                i_TXCHARDISPVAL=Cat(txdata[8], txdata[18], txdata[28], txdata[38]),
+                i_TXDATA=Cat(txdata[:8], txdata[10:18], txdata[20:28], txdata[30:38]),
+                i_TXUSRCLK=ClockSignal("cxp_gtx_tx") if tx_mode else 0,
+                i_TXUSRCLK2=ClockSignal("cxp_gtx_tx") if tx_mode else 0,
+
+                # TX electrical
+                i_TXBUFDIFFCTRL=0b100,
+                i_TXDIFFCTRL=0b1000,
+
+                # RX Startup/Reset
+                i_RXPHDLYRESET=0,
+                i_RXDLYBYPASS=0,
+                i_RXPHALIGNEN=1 if rx_mode in ["master", "slave"] else 0,
+                i_GTRXRESET=rx_init.gtXxreset if rx_mode else 0,
+                o_RXRESETDONE=rx_init.Xxresetdone if rx_mode else 0,
+                i_RXDLYSRESET=rx_init.Xxdlysreset if rx_mode else 0,
+                o_RXDLYSRESETDONE=rx_init.Xxdlysresetdone if rx_mode else 0,
+                i_RXPHALIGN=rx_init.Xxphalign if rx_mode in ["master", "slave"] else 0,
+                i_RXDLYEN=rx_init.Xxdlyen if rx_mode in ["master", "slave"] else 0,
+                o_RXPHALIGNDONE=rx_init.Xxphaligndone if rx_mode else 0,
+                i_RXUSERRDY=rx_init.Xxuserrdy if rx_mode else 0,
+                p_RXPMARESET_TIME=1,
+                p_RXPCSRESET_TIME=1,
+
+                # RX AFE
+                p_RX_DFE_XYD_CFG=0,
+                p_RX_CM_SEL=0b11,               # RX_CM_SEL = 0b11: Common mode is programmable
+                p_RX_CM_TRIM=0b010,             # PMA_RSV[4],RX_CM_TRIM[2:0] = 0b1010: Common mode 800mV
+                i_RXDFEXYDEN=1,
+                i_RXDFEXYDHOLD=0,
+                i_RXDFEXYDOVRDEN=0,
+                i_RXLPMEN=1,                    # RXLPMEN = 1: LPM mode is enable for non scramble 8b10b data
+                p_RXLPM_HF_CFG=0b00000011110000,
+                p_RXLPM_LF_CFG=0b00000011110000,
+
+                p_RX_DFE_GAIN_CFG=0x0207EA,
+                p_RX_DFE_VP_CFG=0b00011111100000011,
+                p_RX_DFE_UT_CFG=0b10001000000000000,
+                p_RX_DFE_KL_CFG=0b0000011111110,
+                p_RX_DFE_KL_CFG2=0x3788140A,
+                p_RX_DFE_H2_CFG=0b000110000000,
+                p_RX_DFE_H3_CFG=0b000110000000,
+                p_RX_DFE_H4_CFG=0b00011100000,
+                p_RX_DFE_H5_CFG=0b00011100000,
+                p_RX_DFE_LPM_CFG=0x0904,        # RX_DFE_LPM_CFG = 0x0904: linerate <= 6.6Gb/s
+                                                #                = 0x0104: linerate > 6.6Gb/s
+
+                # RX clock
+                i_RXDDIEN=1,
+                # i_RXSYSCLKSEL=0b00,
+                i_RXOUTCLKSEL=0b010,
+                o_RXOUTCLK=self.rxoutclk,
+                i_RXUSRCLK=ClockSignal("cxp_gtx_rx") if rx_mode else 0,
+                i_RXUSRCLK2=ClockSignal("cxp_gtx_rx") if rx_mode else 0,
+
+                # RX Clock Correction Attributes
+                p_CLK_CORRECT_USE="FALSE",
+                p_CLK_COR_SEQ_1_1=0b0100000000,
+                p_CLK_COR_SEQ_2_1=0b0100000000,
+                p_CLK_COR_SEQ_1_ENABLE=0b1111,
+                p_CLK_COR_SEQ_2_ENABLE=0b1111,
+
+                # RX data
+                p_RX_DATA_WIDTH=40,
+                p_RX_INT_DATAWIDTH=1,   # 1 if a line rate is greater than 6.6 Gbps
+                o_RXDISPERR=Cat(rxdata[9], rxdata[19], rxdata[29], rxdata[39]),
+                o_RXCHARISK=Cat(rxdata[8], rxdata[18], rxdata[28], rxdata[38]),
+                o_RXDATA=Cat(rxdata[:8], rxdata[10:18], rxdata[20:28], rxdata[30:38]),
+
+                # RX Byte and Word Alignment Attributes
+                p_ALIGN_COMMA_DOUBLE="FALSE",
+                p_ALIGN_COMMA_ENABLE=0b1111111111,
+                p_ALIGN_COMMA_WORD=4, # align comma to rxdata[:10] only
+                p_ALIGN_MCOMMA_DET="TRUE",
+                p_ALIGN_MCOMMA_VALUE=0b1010000011,
+                p_ALIGN_PCOMMA_DET="TRUE",
+                p_ALIGN_PCOMMA_VALUE=0b0101111100,
+                p_SHOW_REALIGN_COMMA="FALSE",
+                p_RXSLIDE_AUTO_WAIT=7,
+                p_RXSLIDE_MODE="OFF",
+                p_RX_SIG_VALID_DLY=10,
+                i_RXPCOMMAALIGNEN=comma_aligner_en,
+                i_RXMCOMMAALIGNEN=comma_aligner_en,
+                i_RXCOMMADETEN=1,
+                i_RXSLIDE=0,
+                o_RXBYTEISALIGNED=comma_aligned,
+                o_RXBYTEREALIGN=comma_realigned,
+                o_RXCOMMADET=comma_det,
+
+                # RX 8B/10B Decoder Attributes
+                p_RX_DISPERR_SEQ_MATCH="FALSE",
+                p_DEC_MCOMMA_DETECT="TRUE",
+                p_DEC_PCOMMA_DETECT="TRUE",
+                p_DEC_VALID_COMMA_ONLY="FALSE",
+
+                # RX Buffer Attributes
+                p_RXBUF_ADDR_MODE="FAST",
+                p_RXBUF_EIDLE_HI_CNT=0b1000,
+                p_RXBUF_EIDLE_LO_CNT=0b0000,
+                p_RXBUF_EN="FALSE",
+                p_RX_BUFFER_CFG=0b000000,
+                p_RXBUF_RESET_ON_CB_CHANGE="TRUE",
+                p_RXBUF_RESET_ON_COMMAALIGN="FALSE",
+                p_RXBUF_RESET_ON_EIDLE="FALSE",     # RXBUF_RESET_ON_EIDLE = FALSE: OOB is disabled
+                p_RXBUF_RESET_ON_RATE_CHANGE="TRUE",
+                p_RXBUFRESET_TIME=0b00001,
+                p_RXBUF_THRESH_OVFLW=61,
+                p_RXBUF_THRESH_OVRD="FALSE",
+                p_RXBUF_THRESH_UNDFLW=4,
+                p_RXDLY_CFG=0x001F,
+                p_RXDLY_LCFG=0x030,
+                p_RXDLY_TAP_CFG=0x0000,
+                p_RXPH_CFG=0xC00002,
+                p_RXPHDLY_CFG=0x084020,
+                p_RXPH_MONITOR_SEL=0b00000,
+                p_RX_XCLK_SEL="RXUSR",
+                p_RX_DDI_SEL=0b000000,
+                p_RX_DEFER_RESET_BUF_EN="TRUE",
+
+                # CDR Attributes
+                p_RXCDR_CFG=0x03_0000_23FF_1008_0020,   # LPM @ 0.5G-1.5625G , 8B/10B encoded data, CDR setting < +/- 200ppm
+                                                        # (See UG476 (v1.12.1), p.206)
+                p_RXCDR_FR_RESET_ON_EIDLE=0b0,
+                p_RXCDR_HOLD_DURING_EIDLE=0b0,
+                p_RXCDR_PH_RESET_ON_EIDLE=0b0,
+                p_RXCDR_LOCK_CFG=0b010101,
+
+                # Pads
+                i_GTXRXP=pads.rxp,
+                i_GTXRXN=pads.rxn,
+                o_GTXTXP=pads.txp,
+                o_GTXTXN=pads.txn,
+
+                # Dynamic Reconfiguration Ports
+                p_IS_DRPCLK_INVERTED=0b0,
+                i_DRPADDR=self.daddr,
+                i_DRPCLK=self.dclk,
+                i_DRPEN=self.den,
+                i_DRPWE=self.dwen,
+                i_DRPDI=self.din,
+                o_DRPDO=self.dout,
+                o_DRPRDY=self.dready,
+
+                # Nearend Loopback
+                i_LOOPBACK = self.loopback_mode,
+                p_TX_LOOPBACK_DRIVE_HIZ = "FALSE",
+                p_RXPRBS_ERR_LOOPBACK = 0b0,
+
+                # Other parameters
+                p_PCS_RSVD_ATTR=(
+                    (tx_mode != "single") << 1 |    # PCS_RSVD_ATTR[1] = 0: TX Single Lane Auto Mode
+                                                    #                  = 1: TX Manual Mode
+                    (rx_mode != "single") << 2 |    #              [2] = 0: RX Single Lane Auto Mode
+                                                    #                  = 1: RX Manual Mode
+                    0 << 8                          #              [8] = 0: OOB is disabled
+                ),
+                i_RXELECIDLEMODE=0b11,              # RXELECIDLEMODE = 0b11: OOB is disabled
+                p_RX_DFE_LPM_HOLD_DURING_EIDLE=0b0,
+                p_ES_EYE_SCAN_EN="TRUE",            # Must be TRUE for GTX
+            )
+
+
+        # TX clocking
+        # When bypassing the TX buffer and changing frequency of VCO of QPLL/CPLL,
+        # TXUSRCLK rate will always be the refclk rate.
+        # To match the required TXUSRCLK rate = linerate/datewidth (UG476 (v1.12.1) Equation 3-1), a DRP PLL is used.
+         
+        # Slave TX will use cxp_gtx_tx from master
+        if tx_mode == "single" or tx_mode == "master":
+            self.clock_domains.cd_cxp_gtx_tx = ClockDomain()
+            txpll_fb_clk = Signal()
+            txoutclk_buf = Signal()
+            txpll_clkout = Signal()
+
+            self.txpll_reset = Signal()
+            self.pll_daddr = Signal(7)
+            self.pll_dclk = Signal()
+            self.pll_den = Signal()
+            self.pll_din = Signal(16)
+            self.pll_dwen = Signal()
+
+            self.txpll_locked = Signal()
+            self.pll_dout = Signal(16)
+            self.pll_dready = Signal()
+
+            pll_fbout_mult = 8
+            txusr_pll_div = pll_fbout_mult*sys_clk_freq/qpll.tx_usrclk_freq
+            self.specials += [
+                Instance("PLLE2_ADV",
+                    p_BANDWIDTH="HIGH",
+                    o_LOCKED=self.txpll_locked,
+                    i_RST=self.txpll_reset,
+                
+                    p_CLKIN1_PERIOD=1e9/sys_clk_freq, # ns
+                    i_CLKIN1=txoutclk_buf,
+
+                    # VCO @ 1.25GHz 
+                    p_CLKFBOUT_MULT=pll_fbout_mult, p_DIVCLK_DIVIDE=1,
+                    i_CLKFBIN=txpll_fb_clk, o_CLKFBOUT=txpll_fb_clk, 
+
+                    # frequency = linerate/40
+                    p_CLKOUT0_DIVIDE=txusr_pll_div, p_CLKOUT0_PHASE=0.0, o_CLKOUT0=txpll_clkout,
+
+                    # Dynamic Reconfiguration Ports
+                    i_DADDR = self.pll_daddr,
+                    i_DCLK = self.pll_dclk,
+                    i_DEN = self.pll_den,
+                    i_DI = self.pll_din,
+                    i_DWE = self.pll_dwen,
+                    o_DO = self.pll_dout,
+                    o_DRDY = self.pll_dready,
+                ),
+                Instance("BUFG", i_I=self.txoutclk, o_O=txoutclk_buf),
+                Instance("BUFG", i_I=txpll_clkout, o_O=self.cd_cxp_gtx_tx.clk),
+                AsyncResetSynchronizer(self.cd_cxp_gtx_tx, ~self.txpll_locked & ~tx_init.done)
+            ]
+            self.comb += tx_init.restart.eq(self.tx_restart)
+
+        # RX clocking
+        # When frequency of VCO of QPLL/CPLL is changed, RXUSRCLK will match the required frequency
+        # RXUSRCLK rate = linerate/datewidth (UG476 (v1.12.1) Equation 4-2). And PLL is not needed.
+        
+        # Slave RX will use cxp_gtx_rx from master
+        if rx_mode == "single" or rx_mode == "master":
+            self.clock_domains.cd_cxp_gtx_rx = ClockDomain()
+            self.specials += [
+                Instance("BUFG", i_I=self.rxoutclk, o_O=self.cd_cxp_gtx_rx.clk),
+                AsyncResetSynchronizer(self.cd_cxp_gtx_rx, ~rx_init.done)
+            ]
+
+        if rx_mode:
+            self.comb += rx_init.restart.eq(self.rx_manual_restart)
+
+            self.submodules.comma_aligner = comma_aligner = Comma_Aligner(0b0101111100)
+            self.comb += [
+                comma_aligner.data.eq(rxdata),
+                comma_aligner.comma_aligned.eq(comma_aligned),
+                comma_aligner.comma_realigned.eq(comma_realigned),
+                comma_aligner.comma_det.eq(comma_det),
+                comma_aligner_en.eq(comma_aligner.aligner_en),
+                self.rx_ready.eq(comma_aligner.ready_sys),
+            ]

src/gateware/cxp_frame_pipeline.py

@@ -0,0 +1,898 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect.stream import Endpoint, SyncFIFO
+from misoc.cores.liteeth_mini.mac.crc import LiteEthMACCRCEngine
+
+from cxp_pipeline import *
+# from src.gateware.cxp_pipeline import * # for sim only
+
+from types import SimpleNamespace
+from math import lcm
+from operator import or_, add
+
+pixel_width = 16
+pixel4x_layout = [
+    ("data", pixel_width*4),
+    ("valid", 4), 
+]
+
+def switch_endianness(s):
+    assert len(s) % 8 == 0
+    char = [s[i*8:(i+1)*8] for i in range(len(s)//8)]
+    return Cat(char[::-1])
+
+class End_Of_Line_Inserter(Module):
+    """
+    Insert eop to indicate end of line
+    And drop the K codes and Duplicate char
+
+    """
+    def __init__(self):
+        self.l_size = Signal(3*char_width)
+
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint([("data", word_width)]) # pixel data don't need k code
+        
+        # # #
+
+        # TODO: there maybe a reset bug where cxp_gtx_rx is not reset but frame size is changed
+        # cnt will be reset to last l_size instead of the new l_size resulting in wrong eop tag
+
+        # NOTE: because the self.sink.stb is only active after new_frame, the cnt is changed after the new_frame is high
+        # Also, after transmitting the last word, cnt = 1, so cnt will update to the correct self.l_size regardless 
+
+        cnt = Signal.like(self.l_size, reset=1)
+        self.sync += [
+            If((~self.source.stb | self.source.ack),
+                self.sink.connect(self.source, omit={"ack", "eop", "k", "dchar", "dchar_k"}),
+                If(self.sink.stb,
+                    If(cnt == 1,
+                        cnt.eq(self.l_size)
+                    ).Else(
+                        cnt.eq(cnt - 1),
+                    )
+                ),
+            ),
+        ]
+        self.comb += [
+            self.sink.ack.eq(~self.source.stb | self.source.ack),
+            # repurpose eop as end of line
+            self.source.eop.eq(cnt == 1),
+        ]
+
+class Stream_Arbiter(Module):
+    def __init__(self, n_channels):
+        assert n_channels > 1 # don't need a arbiter if there is only one channel
+
+        self.active_channels = Signal(n_channels) 
+
+        self.sinks = [Endpoint(word_layout_dchar) for _ in range(n_channels)]
+        self.source = Endpoint(word_layout_dchar)
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="0")
+
+        # Section 9.5.5 (CXP-001-2021)
+        # When Multiple connections are active, stream packets are transmitted in
+        # ascending order of Connection ID
+        # Support ch0->1->2->4 topology only
+        for n, sink in enumerate(self.sinks):
+            if n < n_channels - 1:
+                fsm.act(str(n),
+                    sink.connect(self.source),
+                    If(sink.stb & sink.eop & self.source.ack,
+                        If(self.active_channels[n+1],
+                            NextState(str(n+1)),
+                        ). Else(
+                            NextState(str(0)),
+                        ),
+                    )
+                )
+            else:
+                fsm.act(str(n),
+                    sink.connect(self.source),
+                    If(sink.stb & sink.eop & self.source.ack,
+                       NextState(str(0))
+                    ),
+                )
+
+@ResetInserter()
+@CEInserter()
+class CXPCRC32(Module):
+    # Section 9.2.2.2 (CXP-001-2021)
+    width = 32
+    polynom = 0x04C11DB7
+    seed = 2**width - 1
+    check = 0x00000000
+
+    def __init__(self, data_width):
+        self.data = Signal(data_width)
+        self.value = Signal(self.width)
+        self.error = Signal()
+
+        # # #
+
+        self.submodules.engine = LiteEthMACCRCEngine(
+            data_width, self.width, self.polynom
+        )
+        reg = Signal(self.width, reset=self.seed)
+        self.sync += reg.eq(self.engine.next)
+        self.comb += [
+            self.engine.data.eq(self.data),
+            self.engine.last.eq(reg),
+            self.value.eq(reg[::-1]),
+            self.error.eq(reg != self.check),
+        ]
+
+class CXPCRC32_Checker(Module):
+    """Verify crc in stream data packet"""
+    def __init__(self):
+        # TODO: handle the error into a counter
+        self.error = Signal()
+
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar)
+
+        # # #
+
+        self.submodules.crc = crc = CXPCRC32(word_width)
+        self.comb += crc.data.eq(self.sink.data),
+
+        self.submodules.fsm = fsm = FSM(reset_state="INIT")
+        fsm.act("INIT",
+            crc.reset.eq(1),
+            NextState("CHECKING"),
+        )
+
+        fsm.act("RESET",
+            crc.reset.eq(1),
+            self.error.eq(crc.error),
+            NextState("CHECKING"),
+        )
+
+        fsm.act("CHECKING",
+            If(self.sink.stb & self.sink.eop,
+                # discard the crc
+                self.sink.ack.eq(1),
+                NextState("RESET"),
+            ).Else(
+                self.sink.connect(self.source),
+            ),
+            crc.ce.eq(self.sink.stb),
+        )
+    
+        
+class Stream_Broadcaster(Module):
+    def __init__(self, layout=word_layout_dchar, routing_ids=[0]):
+        n_id = len(routing_ids)
+        assert n_id > 0
+
+        self.sources = [Endpoint(layout) for _ in range(n_id)]
+        self.sink = Endpoint(layout)
+
+        # # #
+
+        
+        stream_id = Signal(char_width)
+        pak_tag = Signal(char_width)
+        stream_pak_size = Signal(char_width * 2)
+
+        self.submodules.fsm = fsm = FSM(reset_state="WAIT_HEADER")
+
+        fsm.act(
+            "WAIT_HEADER",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_id, self.sink.dchar),
+                NextState("GET_PAK_TAG"),
+            ),
+        )
+
+        fsm.act(
+            "GET_PAK_TAG",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(pak_tag, self.sink.dchar),
+                NextState("GET_PAK_SIZE_0"),
+            ),
+        )
+
+        fsm.act(
+            "GET_PAK_SIZE_0",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_pak_size[8:], self.sink.dchar),
+                NextState("GET_PAK_SIZE_1"),
+            ),
+        )
+
+        routing_case = {"default": NextState("DISCARD")}
+        for id in (routing_ids):
+            routing_case[id] = [NextState(f"COPY_TO_BUFFER_{id}")]
+
+        fsm.act(
+            "GET_PAK_SIZE_1",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_pak_size[:8], self.sink.dchar),
+                Case(stream_id, routing_case),
+            ),
+        )
+
+        for key in routing_case:
+            if key == "default":
+                fsm.act(
+                    "DISCARD",
+                    self.sink.ack.eq(1),
+                    If(self.sink.stb,
+                        NextValue(stream_pak_size, stream_pak_size - 1),
+                        If(stream_pak_size == 0,
+                            NextValue(stream_id, stream_id.reset),
+                            NextValue(pak_tag, pak_tag.reset),
+                            NextValue(stream_pak_size, stream_pak_size.reset),
+                            NextState("WAIT_HEADER"),
+                        )
+                    ),
+                )
+            else:
+                fsm.act(
+                    f"COPY_TO_BUFFER_{key}",
+                    self.sink.connect(self.sources[key]),
+
+                    # assume downstream is not blocked
+                    If(self.sink.stb,
+                        NextValue(stream_pak_size, stream_pak_size - 1),
+                        If(stream_pak_size == 0,
+                            NextValue(stream_id, stream_id.reset),
+                            NextValue(pak_tag, pak_tag.reset),
+                            NextValue(stream_pak_size, stream_pak_size.reset),
+                            NextState("WAIT_HEADER"),
+                        )
+                    ),
+                
+                )
+
+class Frame_Header_Decoder(Module):
+    def __init__(self):
+        self.decode_err = Signal()
+
+        self.new_frame = Signal()
+        self.new_line = Signal()
+
+        # Table 47 (CXP-001-2021)
+        n_metadata_chars = 23
+        img_header_layout = [
+            ("stream_id", char_width),
+            ("source_tag", 2*char_width), 
+            ("x_size", 3*char_width), 
+            ("x_offset", 3*char_width), 
+            ("y_size", 3*char_width), 
+            ("y_offset", 3*char_width), 
+            ("l_size", 3*char_width), # number of data words per image line 
+            ("pixel_format", 2*char_width), 
+            ("tap_geo", 2*char_width), 
+            ("flag", char_width), 
+        ]
+        assert layout_len(img_header_layout) == n_metadata_chars*char_width  
+
+
+        # # #
+
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar)
+
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+
+        fsm.act("IDLE",
+            self.sink.ack.eq(1),
+            If((self.sink.stb & (self.sink.dchar == KCode["stream_marker"]) & (self.sink.dchar_k == 1)),
+                NextState("DECODE"),
+            )
+        )
+
+        fsm.act("COPY",
+            # until for new line or new frame
+            If((self.sink.stb & (self.sink.dchar == KCode["stream_marker"]) & (self.sink.dchar_k == 1)),
+                self.sink.ack.eq(1),
+                NextState("DECODE"),
+            ).Else(
+                self.sink.connect(self.source),
+            )
+        )
+
+        type = {
+            "new_frame": 0x01,
+            "line_break": 0x02,
+        }
+
+        cnt = Signal(max=n_metadata_chars)
+        fsm.act("DECODE",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                Case(self.sink.dchar, {
+                    type["new_frame"]: [
+                        self.new_frame.eq(1),
+                        NextValue(cnt, cnt.reset),
+                        NextState("GET_FRAME_DATA"),
+                    ],
+                    type["line_break"]: [
+                        self.new_line.eq(1),
+                        NextState("COPY"),
+                    ],
+                    "default": [
+                         self.decode_err.eq(1),
+                         # discard all data until valid frame header
+                         NextState("IDLE"),
+                    ],
+                }),
+            )
+        )
+
+        packet_buffer = Signal(layout_len(img_header_layout))
+        case = dict(
+            (i, NextValue(packet_buffer[8*i:8*(i+1)], self.sink.dchar))
+            for i in range(n_metadata_chars)
+        )
+        fsm.act("GET_FRAME_DATA",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                Case(cnt, case),
+                If(cnt == n_metadata_chars - 1,
+                    NextState("COPY"),
+                    NextValue(cnt, cnt.reset),
+                ).Else(
+                    NextValue(cnt, cnt + 1),
+                ),
+            ),
+        )
+
+        # dissect packet 
+        self.metadata = SimpleNamespace()
+        idx = 0
+        for name, size in img_header_layout:
+            # CXP use MSB even when sending duplicate chars
+            setattr(self.metadata, name, switch_endianness(packet_buffer[idx:idx+size]))
+            idx += size
+
+class Pixel_Gearbox(Module):
+    """Convert 32 bits word into 4x pixel"""
+    def __init__(self, size):
+        assert size <= pixel_width
+        assert size in [8, 10, 12, 14, 16]
+
+        self.x_size = Signal(3*char_width)
+
+        sink_dw, source_dw = word_width, size*4
+        self.sink = Endpoint([("data", sink_dw)])
+        self.source = Endpoint(pixel4x_layout)
+
+        # # #
+
+        ring_buf_size = lcm(sink_dw, source_dw)
+        # ensure the shift register is at least twice the size of sink/source dw
+        if (ring_buf_size//sink_dw) < 2:
+            ring_buf_size = ring_buf_size * 2
+        if (ring_buf_size//source_dw) < 2:
+            ring_buf_size = ring_buf_size * 2
+
+        # Control interface
+
+        reset_reg = Signal()
+        we = Signal()
+        re = Signal()
+        level = Signal(max=ring_buf_size)
+        w_cnt = Signal(max=ring_buf_size//sink_dw)
+        r_cnt = Signal(max=ring_buf_size//source_dw)
+
+        self.sync += [
+            If(reset_reg,
+                level.eq(level.reset),
+            ).Else(
+                If(we & ~re, level.eq(level + sink_dw)),
+                If(~we & re, level.eq(level - source_dw)),
+                If(we & re, level.eq(level + sink_dw - source_dw)),
+            ),
+
+            If(reset_reg,
+                w_cnt.eq(w_cnt.reset),
+                r_cnt.eq(r_cnt.reset),
+            ).Else(
+                If(we, 
+                    If(w_cnt == ((ring_buf_size//sink_dw) - 1),
+                        w_cnt.eq(w_cnt.reset),
+                    ).Else(
+                        w_cnt.eq(w_cnt + 1),
+                    )
+                ),
+                If(re, 
+                    If(r_cnt == ((ring_buf_size//source_dw) - 1),
+                        r_cnt.eq(r_cnt.reset),
+                    ).Else(
+                        r_cnt.eq(r_cnt + 1),
+                    )
+                ),
+            )
+        ]
+         
+        extra_eol_handling = size in [10, 12, 14]
+        if extra_eol_handling:
+            # the source need to be stb twice
+            # (one for level >= source_dw and the other for the remaining pixels)
+            # when last word of each line packet satisfied the following condition:
+            # 
+            # if there exist an integers j such that
+            # sink_dw * i > size * j > source_dw * k  
+            # where i,k are postive integers and source_dw * k - sink_dw * (i-1) > 0
+            # 
+            stb_aligned = Signal()
+            match size: 
+                case 10:
+                    # For example size == 10
+                    # 32 * 2 > 10 * (5) > 40 * 1
+                    # 32 * 2 > 10 * (6) > 40 * 1
+                    # 32 * 3 > 10 * (9) > 40 * 2
+                    # ...
+                    # 
+                    # the packing pattern for size == 10 repeat every 16 pixels
+                    # the remaining special case can be taken care off using modulo operation
+                    stb_cases = {
+                        5: stb_aligned.eq(1),
+                        6: stb_aligned.eq(1),
+                        9: stb_aligned.eq(1),
+                    }
+                    self.sync += Case(self.x_size[:4], stb_cases) # mod 16
+                case 12:
+                    stb_cases = {
+                        5: stb_aligned.eq(1),
+                    }
+                    self.sync += Case(self.x_size[:3], stb_cases) # mod 8
+                case 14:
+                    stb_cases = {
+                        9: stb_aligned.eq(1),
+                        13: stb_aligned.eq(1),
+                    }
+                    self.sync += Case(self.x_size[:4], stb_cases) # mod 16
+
+
+
+        self.submodules.fsm = fsm = FSM(reset_state="SHIFTING")
+        fsm.act(
+            "SHIFTING",
+            self.sink.ack.eq(1),
+            self.source.stb.eq(level >= source_dw),
+            we.eq(self.sink.stb),
+            re.eq((self.source.stb & self.source.ack)),
+            If(self.sink.stb & self.sink.eop,
+                (If(stb_aligned,
+                    NextState("MOVE_ALIGNED_PIX"),
+                ).Else(
+                    NextState("MOVE_REMAINING_PIX"),
+                ) if extra_eol_handling else
+                    NextState("MOVE_REMAINING_PIX"),
+                )
+            ),
+        )
+
+        if extra_eol_handling:
+            fsm.act(
+                "MOVE_ALIGNED_PIX",
+                self.source.stb.eq(1),
+                re.eq((self.source.stb & self.source.ack)),
+                NextState("MOVE_REMAINING_PIX"),
+            )
+
+        stb_remaining_pix = Signal()
+        fsm.act(
+            "MOVE_REMAINING_PIX",
+            reset_reg.eq(1),
+            self.source.stb.eq(1),
+            stb_remaining_pix.eq(1),
+            NextState("SHIFTING"),
+        )
+
+        # Data path
+         
+        ring_buf = Signal(ring_buf_size, reset_less=True)
+
+        sink_cases = {}
+        for i in range(ring_buf_size//sink_dw):
+            sink_cases[i] = [
+                ring_buf[sink_dw*i:sink_dw*(i+1)].eq(self.sink.data),
+            ]
+        self.sync += If(self.sink.stb, Case(w_cnt, sink_cases))
+
+        source_cases = {}
+        for i in range(ring_buf_size//source_dw):
+            source_cases[i] = []
+            for j in range(4):
+                source_cases[i].append(
+                    self.source.data[pixel_width * j : pixel_width * (j + 1)].eq(
+                        ring_buf[(source_dw * i) + (size * j) : (source_dw * i) + (size * (j + 1))]
+                    )
+                )
+
+        # calcule which last pixels are valid
+        valid = Signal(4)
+        bit_cases = {
+            0: valid.eq(0b1111),
+            1: valid.eq(0b0001),
+            2: valid.eq(0b0011),
+            3: valid.eq(0b0111),
+        }
+        self.sync += Case(self.x_size[:2], bit_cases)
+
+        self.comb += [
+            Case(r_cnt, source_cases),
+            If(stb_remaining_pix,
+                self.source.valid.eq(valid),
+                self.source.eop.eq(1),
+            ).Else(
+                self.source.valid.eq(0b1111),
+            ),
+        ]
+
+class Pixel_Coordinate_Tracker(Module):
+    """
+    Track pixel coordinates
+
+    Assume
+    - camera is in area scan mode
+    - 1X-1Y Tap geometry
+    """
+    def __init__(self, res_width):
+        # largest x/y pixel size supported by frame header are 24 bits
+        assert res_width <= 3*char_width
+
+        # line scaning frame will have y_size = 0 and won't trigger the end of frame bit
+        self.y_size = Signal(3*char_width)
+        self.sink = Endpoint(pixel4x_layout)
+
+        # # #
+        
+        # NOTE: no need for last_x/last_y csr which is use to indicate how big is the frame
+        # layout = Record([
+        #    ("x", res_width),
+        #    ("y", res_width),
+        #    ("d", pixel_width),
+        #    ("stb", 1),
+        #    ("eof", 1), # end of frame
+        # ])
+        # self.pixel4x = [layout for _ in range(4)]
+        
+        # DEBUG: for sim only, to show all record in sim
+        self.pixel4x = []
+        for _ in range(4):
+            self.pixel4x.append(Record([
+                ("x", res_width),
+                ("y", res_width),
+                ("gray", pixel_width),
+                ("stb", 1),
+                ("eof", 1), # end of frame
+            ]))
+
+        x_4x = [Signal(len(self.pixel4x[0].x), reset=i) for i in range(4)]
+        y_r = Signal(len(self.pixel4x[0].y))
+
+        y_max = Signal.like(self.y_size)
+        self.sync += [
+            self.sink.ack.eq(1),
+            y_max.eq(self.y_size - 1),
+        ]
+        for i, (x_r, pix) in enumerate(zip(x_4x, self.pixel4x)):
+            self.sync += [
+                pix.stb.eq(0),
+                pix.eof.eq(0),
+                If(self.sink.stb,
+                    If(self.sink.eop,
+                        # new line
+                        x_r.eq(x_r.reset),
+
+                        If(y_r == y_max,
+                            pix.eof.eq(1),
+                            y_r.eq(y_r.reset),
+                        ).Else(
+                            y_r.eq(y_r + 1),
+                        )
+                    ).Else(
+                        x_r.eq(x_r + 4),
+                    ),
+                    pix.stb.eq(self.sink.valid[i]),
+                    pix.x.eq(x_r),
+                    pix.y.eq(y_r),
+                    pix.gray.eq(self.sink.data[pixel_width*i:pixel_width*(i+1)]),
+                )
+            ]
+
+class ROI(Module):
+    """
+    ROI Engine. For each frame, accumulates pixels values within a
+    rectangular region of interest, and reports the total.
+    """
+    def __init__(self, pixel_4x, count_width):
+        assert len(pixel_4x) == 4
+
+        self.cfg = Record([
+            ("x0", len(pixel_4x[0].x)),
+            ("y0", len(pixel_4x[0].y)),
+            ("x1", len(pixel_4x[0].x)),
+            ("y1", len(pixel_4x[0].y)),
+        ])
+
+        self.out = Record([
+            ("update", 1),
+            # registered output - can be used as CDC input
+            ("count", count_width),
+        ])
+
+        # # #
+
+        roi_4x = [
+            Record([
+                ("x_good", 1),
+                ("y_good", 1),
+                ("gray", len(pixel_4x[0].gray)),
+                ("stb", 1),
+                ("count", count_width),
+            ]) for _ in range(4)
+            
+        ]
+
+        for pix, roi in zip(pixel_4x, roi_4x):
+            self.sync += [
+                # stage 1 - generate "good" (in-ROI) signals
+                roi.x_good.eq(0),
+                If((self.cfg.x0 <= pix.x) & (pix.x < self.cfg.x1),
+                    roi.x_good.eq(1)
+                ),
+
+                # the 4 pixels are on the same y level, no need for extra calculation
+                If(pix.y == self.cfg.y0,
+                    roi.y_good.eq(1)
+                ),
+                If(pix.y == self.cfg.y1,
+                    roi.y_good.eq(0)
+                ),
+                If(pix.eof,
+                    roi.x_good.eq(0),
+                    roi.y_good.eq(0)
+                ),
+                roi.gray.eq(pix.gray),
+                roi.stb.eq(pix.stb),
+
+                # stage 2 - accumulate
+                If((roi.stb & roi.x_good & roi.y_good),
+                    roi.count.eq(roi.count + roi.gray)
+                )
+            ]
+
+        eof = Signal()
+        eof_buf = Signal()
+        count_buf = [Signal(count_width), Signal(count_width)]
+        
+        # stage 3 - update
+        self.sync += [
+            eof.eq(reduce(or_, [pix.eof for pix in pixel_4x])),
+            eof_buf.eq(eof),
+            count_buf[0].eq(roi_4x[0].count + roi_4x[1].count),
+            count_buf[1].eq(roi_4x[2].count + roi_4x[3].count),
+
+            self.out.update.eq(0),
+            If(eof_buf,
+                [roi.count.eq(0) for roi in roi_4x],
+                self.out.update.eq(1),
+                self.out.count.eq(reduce(add, count_buf))
+            ),
+        ]
+
+class Economical_ROI(Module):
+    """
+    ROI Engine. For each frame, accumulates pixels values within a
+    rectangular region of interest, and reports the total.
+
+    This econ version limits the distance between x0 and x1 need to be at least 4 pixel long
+    """
+    def __init__(self, pixel_4x, count_width):
+        assert len(pixel_4x) == 4
+
+        self.cfg = Record([
+            ("x0", len(pixel_4x[0].x)),
+            ("y0", len(pixel_4x[0].y)),
+            ("x1", len(pixel_4x[0].x)),
+            ("y1", len(pixel_4x[0].y)),
+        ])
+
+        self.out = Record([
+            ("update", 1),
+            # registered output - can be used as CDC input
+            ("count", count_width),
+        ])
+
+        # # #
+
+        roi_4x = [
+            Record([
+                ("x_good", 1),
+                ("y_good", 1),
+                ("gray", len(pixel_4x[0].gray)),
+                ("stb", 1),
+                ("count", count_width),
+            ]) for _ in range(4)
+            
+        ]
+
+        # Pipeline the offset calculation
+        x0_offset = [Signal.like(self.cfg.x0) for _ in range(4)]
+        x1_offset = [Signal.like(self.cfg.x1) for _ in range(4)]
+
+        for offset, (x0, x1) in enumerate(zip(x0_offset, x1_offset)):
+            self.sync += [
+                x0.eq(self.cfg.x0 + offset),
+                x1.eq(self.cfg.x1 + offset),
+            ]
+
+        for pix, roi in zip(pixel_4x, roi_4x):
+            # stage 1 - generate "good" (in-ROI) signals
+             
+            # if p in [x0, x0+3] => x_good == 1
+            # if p in [x1, x1+3] => x_good == 0
+            # when [x0, x0+3] overlap with [x1, x1+3]:
+            # some x_good won't have the chance to fall and keep accmulated extra data
+            # Thus, x1-x0 >= 4 need to be match for pixel to accmulated correctly
+            for x0, x1 in zip(x0_offset, x1_offset):
+                self.sync += [
+                    If(pix.x == x0,
+                       roi.x_good.eq(1),
+                    ),
+                    If(pix.x == x1,
+                       roi.x_good.eq(0),
+                    ),
+                ]
+            self.sync += [
+                # the 4 pixels are on the same y level, no need for extra calculation
+                If(pix.y == self.cfg.y0,
+                    roi.y_good.eq(1)
+                ),
+                If(pix.y == self.cfg.y1,
+                    roi.y_good.eq(0)
+                ),
+                If(pix.eof,
+                    roi.x_good.eq(0),
+                    roi.y_good.eq(0)
+                ),
+                roi.gray.eq(pix.gray),
+                roi.stb.eq(pix.stb),
+
+                # stage 2 - accumulate
+                If((roi.stb & roi.x_good & roi.y_good),
+                    roi.count.eq(roi.count + roi.gray)
+                )
+            ]
+
+        eof = Signal()
+        eof_buf = Signal()
+        count_buf = [Signal(count_width), Signal(count_width)]
+        
+        # stage 3 - update
+        self.sync += [
+            eof.eq(reduce(or_, [pix.eof for pix in pixel_4x])),
+            eof_buf.eq(eof),
+            count_buf[0].eq(roi_4x[0].count + roi_4x[1].count),
+            count_buf[1].eq(roi_4x[2].count + roi_4x[3].count),
+
+            self.out.update.eq(0),
+            If(eof_buf,
+                [roi.count.eq(0) for roi in roi_4x],
+                self.out.update.eq(1),
+                self.out.count.eq(reduce(add, count_buf))
+            ),
+        ]
+
+        
+
+        
+
+class Pixel_Parser(Module):
+    """
+    Convert the raw frame data into pixel data
+
+    Currently only support:
+    - Pixel format: mono8, mono10, mono12, mono14, mono16
+    - Tap geometry: 1X-1Y
+    - Scaning mode: area scanning
+
+    """
+    def __init__(self, res_width):
+        self.l_size = Signal(3*char_width)
+        self.x_size = Signal(3*char_width)
+        self.y_size = Signal(3*char_width)
+        self.pixel_format = Signal(2*char_width)
+
+        # # #
+
+        #                                                                            
+        #  32+8(dchar)              32                      pixel 4x                    
+        # ----/----> end of line ---/---> Pixel Gearboxes ----/----> Pixel Coordinate ------> pixel 4x
+        #            inserter                                       Tracker                  w/ coord
+        # 
+         
+        self.submodules.eol_inserter = eol_inserter = End_Of_Line_Inserter()
+        self.sync += eol_inserter.l_size.eq(self.l_size)
+        self.sink = eol_inserter.sink
+
+
+        gearboxes = {}
+        for s in [8, 10, 12, 14, 16]:
+            gearbox = Pixel_Gearbox(s)
+            gearboxes["mono"+str(s)] = gearbox
+            self.submodules += gearbox
+            self.sync += gearbox.x_size.eq(self.x_size),
+
+        # From Table 34 (CXP-001-2021)
+        pix_fmt = {
+            "mono8": 0x0101,
+            "mono10": 0x0102,
+            "mono12": 0x0103,
+            "mono14": 0x0104,
+            "mono16": 0x0105,
+        }
+
+        self.submodules.tracker = tracker = Pixel_Coordinate_Tracker(res_width)
+        self.sync += tracker.y_size.eq(self.y_size)
+        self.pixel4x = tracker.pixel4x
+
+        mux_cases = {
+            "default": [
+                # discard unknown pixel format
+                eol_inserter.source.ack.eq(1),
+            ],
+        }
+        for fmt in pix_fmt:
+            mux_cases[pix_fmt[fmt]] = [
+                eol_inserter.source.connect(gearboxes[fmt].sink),
+                gearboxes[fmt].source.connect(tracker.sink),
+            ]
+
+        self.comb += Case(self.pixel_format, mux_cases)
+
+
+
+class Pixel_Pipeline(Module):
+    def __init__(self, res_width, count_width, packet_size):
+
+        # NOTE: csr need to stay outside since this module need to be cdr in the CXP_FRAME_Pipeline module
+        # NOTE: TapGeo other than 1X-1Y are not supported
+        # TODO: match pixel and see whether the it matches the supported ones (via csr perhaps?)
+
+        #  32+8(dchar)                                                      
+        # ----/----> crc checker ------> frame header ------> Pixel Parser ------> pixel 4x
+        #                                decoder                                   w/ coord
+         
+        # DEBUG: adding fifo doesn't help
+        self.submodules.buffer = buffer = SyncFIFO(word_layout_dchar, 32, True)
+        # self.submodules.buffer = buffer = Buffer(word_layout_dchar) # to improve timing from broadcaster
+        self.submodules.crc_checker = crc_checker = CXPCRC32_Checker()
+        self.submodules.header_decoder = header_decoder = Frame_Header_Decoder()
+        self.submodules.parser = parser = Pixel_Parser(res_width)
+
+        self.submodules.roi = ROI(parser.pixel4x, count_width)
+
+        self.comb += [
+            parser.l_size.eq(header_decoder.metadata.l_size),
+            parser.x_size.eq(header_decoder.metadata.x_size),
+            parser.y_size.eq(header_decoder.metadata.y_size),
+            parser.pixel_format.eq(header_decoder.metadata.pixel_format),
+        ]
+
+        self.pipeline = [buffer, crc_checker, header_decoder, parser]
+        for s, d in zip(self.pipeline, self.pipeline[1:]):
+            self.comb += s.source.connect(d.sink)
+        self.sink = self.pipeline[0].sink
+
+        # DEBUG
+        # self.pix = self.pipeline[-1].pix
+        # self.source = self.pipeline[-1].source
+        # self.comb += self.source.ack.eq(1) # simulated a proper consumer, idk why but without this it will destory timing

src/gateware/cxp_pipeline.py

@@ -0,0 +1,726 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect.stream import Endpoint, SyncFIFO
+
+from functools import reduce
+from itertools import combinations
+from operator import or_, and_
+
+char_width = 8
+char_layout = [("data", char_width), ("k", char_width//8)]
+
+word_width = 32
+word_layout = [("data", word_width), ("k", word_width//8)]
+
+word_layout_dchar = [
+    ("data", word_width),
+    ("k", word_width//8), 
+    ("dchar", char_width), 
+    ("dchar_k", char_width//8),
+]
+
+def K(x, y):
+    return ((y << 5) | x)
+
+def switch_endianness(s):
+    assert len(s) % 8 == 0
+    char = [s[i*8:(i+1)*8] for i in range(len(s)//8)]
+    return Cat(char[::-1])
+
+KCode = {
+    "pak_start" : C(K(27, 7), char_width),
+    "io_ack" : C(K(28, 6), char_width), 
+    "trig_indic_28_2" : C(K(28, 2), char_width),
+    "stream_marker" : C(K(28, 3), char_width),
+    "trig_indic_28_4" : C(K(28, 4), char_width),
+    "pak_end" : C(K(29, 7), char_width),
+    "idle_comma" : C(K(28, 5), char_width),
+    "idle_alignment" : C(K(28, 1), char_width),
+}
+
+class Packet_Wrapper(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout)
+        self.source = Endpoint(word_layout)
+
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+        
+        fsm.act("IDLE",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                self.sink.ack.eq(0),
+                NextState("INSERT_HEADER"),
+            )
+        )
+
+        fsm.act("INSERT_HEADER",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(KCode["pak_start"], 4)),
+            self.source.k.eq(Replicate(1, 4)),
+            If(self.source.ack, NextState("COPY")),
+        )
+
+        fsm.act("COPY",
+            self.sink.connect(self.source),
+            self.source.eop.eq(0),
+            If(self.sink.stb & self.sink.eop & self.source.ack,
+                NextState("INSERT_FOOTER"),
+            ),
+        )
+
+        fsm.act("INSERT_FOOTER",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(KCode["pak_end"], 4)),
+            self.source.k.eq(Replicate(1, 4)),
+            self.source.eop.eq(1),
+            If(self.source.ack, NextState("IDLE")),
+        )
+        
+class Trigger_Inserter(Module):
+    def __init__(self):
+        self.stb = Signal()
+        self.delay = Signal(char_width) 
+        self.linktrig_mode = Signal()
+
+        # # #
+
+        self.sink = Endpoint(char_layout)
+        self.source = Endpoint(char_layout)
+
+        # Table 15 & 16 (CXP-001-2021)
+        # Send [K28.2, K28.4, K28.4] or [K28.4, K28.2, K28.2] and 3x delay as trigger packet 
+
+        trig_packet = [Signal(char_width), Signal(char_width), Signal(char_width), self.delay, self.delay, self.delay]
+        trig_packet_k = [1, 1, 1, 0, 0, 0]
+        self.comb += [
+            If(self.linktrig_mode,
+                trig_packet[0].eq(KCode["trig_indic_28_4"]),
+                trig_packet[1].eq(KCode["trig_indic_28_2"]),
+                trig_packet[2].eq(KCode["trig_indic_28_2"]),
+            ).Else(
+                trig_packet[0].eq(KCode["trig_indic_28_2"]),
+                trig_packet[1].eq(KCode["trig_indic_28_4"]),
+                trig_packet[2].eq(KCode["trig_indic_28_4"]),
+            ),
+        ]
+        
+        self.submodules.fsm = fsm = FSM(reset_state="COPY")
+        
+        cnt = Signal(max=6)
+        fsm.act("COPY",
+            NextValue(cnt, cnt.reset),
+            self.sink.connect(self.source),
+            If(self.stb, NextState("WRITE_TRIG"))
+        )
+
+        fsm.act("WRITE_TRIG",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Array(trig_packet)[cnt]),
+            self.source.k.eq(Array(trig_packet_k)[cnt]),
+            If(self.source.ack,
+                If(cnt == 5,
+                    NextState("COPY"),
+                ).Else(
+                    NextValue(cnt, cnt + 1),
+                )
+            )
+        )
+
+class Idle_Word_Inserter(Module):
+    def __init__(self):
+        # Section 9.2.5 (CXP-001-2021)
+        # Send K28.5, K28.1, K28.1, D21.5  as idle word
+        self.submodules.fsm = fsm = FSM(reset_state="WRITE_IDLE")
+        
+        self.sink = Endpoint(word_layout)
+        self.source = Endpoint(word_layout)
+
+        # Section 9.2.5.1 (CXP-001-2021)
+        # IDLE should be transmitter every 10000 words
+        cnt = Signal(max=10000, reset=9999)
+        
+        fsm.act("WRITE_IDLE",
+            self.source.stb.eq(1),
+            self.source.data.eq(Cat(KCode["idle_comma"], KCode["idle_alignment"], KCode["idle_alignment"], C(0xB5, char_width))),
+            self.source.k.eq(Cat(1, 1, 1, 0)),
+
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                self.sink.ack.eq(0),
+                If(self.source.ack,
+                    NextValue(cnt, cnt.reset),
+                    NextState("COPY"),
+                )
+            ),
+        )
+
+        fsm.act("COPY",
+            self.sink.connect(self.source),
+            # increment when uphas data and got ack
+            If(self.sink.stb & self.source.ack, NextValue(cnt, cnt - 1)),
+            If((( (~self.sink.stb) | (self.sink.eop) | (cnt == 0) ) & self.source.ack), NextState("WRITE_IDLE"))
+        )
+
+
+class Trigger_ACK_Inserter(Module):
+    def __init__(self):
+        self.stb = Signal()
+
+        # # #
+
+        # Section 9.3.2 (CXP-001-2021)
+        # Send 4x K28.6 and 4x 0x01 as trigger packet ack
+        self.submodules.fsm = fsm = FSM(reset_state="COPY")
+        
+        self.sink = Endpoint(word_layout)
+        self.source = Endpoint(word_layout)
+        fsm.act("COPY",
+            self.sink.connect(self.source),
+            If(self.stb, NextState("WRITE_ACK0"))
+        )
+
+        fsm.act("WRITE_ACK0",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(KCode["io_ack"], 4)),
+            self.source.k.eq(Replicate(1, 4)),
+            If(self.source.ack, NextState("WRITE_ACK1")),
+        )
+
+        fsm.act("WRITE_ACK1",
+            self.sink.ack.eq(0),
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(C(0x01, char_width), 4)),
+            self.source.k.eq(Replicate(0, 4)),
+            If(self.source.ack, NextState("COPY")),
+        )
+
+
+@FullMemoryWE()
+class Command_Test_Packet_Writer(Module):
+    def __init__(self, buffer_depth):
+        self.word_len = Signal(log2_int(buffer_depth))
+        self.stb = Signal()
+        self.stb_testseq = Signal()
+
+        self.busy = Signal()
+
+        # # #
+        
+        self.specials.mem = mem = Memory(word_width, buffer_depth)
+        self.specials.mem_port = mem_port = mem.get_port()
+        self.source = Endpoint(word_layout)
+
+        # increment addr in the same cycle the moment addr_inc is high
+        # as memory takes one cycle to shift to the correct addr
+        addr_next = Signal(log2_int(buffer_depth))
+        addr = Signal.like(addr_next)
+        addr_rst = Signal()
+        addr_inc = Signal()
+        self.sync += addr.eq(addr_next),
+
+        self.comb += [
+            addr_next.eq(addr),
+            If(addr_rst,
+                addr_next.eq(addr_next.reset),
+            ).Elif(addr_inc,
+                addr_next.eq(addr + 1),
+            ),
+            mem_port.adr.eq(addr_next),
+        ]
+
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+        self.comb += self.busy.eq(~fsm.ongoing("IDLE"))
+
+        cnt = Signal(max=0xFFF)
+        fsm.act("IDLE",
+            addr_rst.eq(1),
+            If(self.stb, NextState("TRANSMIT")),
+            If(self.stb_testseq, 
+                NextValue(cnt, cnt.reset),
+                NextState("WRITE_TEST_PACKET_TYPE"),
+            )
+        )
+
+        fsm.act("TRANSMIT",
+            self.source.stb.eq(1),
+            self.source.data.eq(mem_port.dat_r),
+            If(self.source.ack,
+                addr_inc.eq(1),
+            ),
+            If(addr_next == self.word_len,
+                self.source.eop.eq(1),
+                NextState("IDLE")
+            )
+        )
+
+        fsm.act("WRITE_TEST_PACKET_TYPE",
+            self.source.stb.eq(1),
+            self.source.data.eq(Replicate(C(0x04, char_width), 4)),
+            self.source.k.eq(Replicate(0, 4)),
+            If(self.source.ack,NextState("WRITE_TEST_COUNTER"))
+        )
+
+        fsm.act("WRITE_TEST_COUNTER",
+            self.source.stb.eq(1),
+            self.source.data[:8].eq(cnt[:8]),
+            self.source.data[8:16].eq(cnt[:8]+1),
+            self.source.data[16:24].eq(cnt[:8]+2),
+            self.source.data[24:].eq(cnt[:8]+3),
+            self.source.k.eq(Cat(0, 0, 0, 0)),
+            If(self.source.ack,
+                If(cnt == 0xFFC,
+                    self.source.eop.eq(1),
+                    NextState("IDLE")
+                ).Else(
+                    NextValue(cnt, cnt + 4),
+                )
+               
+            )
+        )
+
+class RX_Debug_Buffer(Module,AutoCSR):
+    def __init__(self, layout, size):
+        self.submodules.buf_out = buf_out = SyncFIFO(layout, size, True)
+        self.sink = buf_out.sink
+
+        self.inc = CSR()
+        self.dout_valid = CSRStatus()
+
+        self.dout_pak = CSRStatus(word_width)
+        self.kout_pak = CSRStatus(word_width//8)
+        self.crc_error = CSRStatus()
+        self.eop = CSRStatus()
+
+        self.sync += [
+            buf_out.source.ack.eq(self.inc.re),
+            self.dout_valid.status.eq(buf_out.source.stb),
+            # output
+            self.eop.status.eq(buf_out.source.eop),
+            self.dout_pak.status.eq(buf_out.source.data),
+            self.kout_pak.status.eq(buf_out.source.k),
+            # self.crc_error.status.eq(buf_out.source.error)
+        ]
+
+class Duplicated_Char_Decoder(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout)
+        self.source = Endpoint(word_layout_dchar)
+
+        # # #
+
+
+        # For duplicated characters, an error correction method (e.g. majority voting) is required to meet the CXP spec:
+        # RX decoder should immune to single bit errors when handling duplicated characters - Section 9.2.2.1 (CXP-001-2021)
+        # 
+        #
+        #                                 32
+        #             +--->  buffer  -----/-----+
+        #         32  |                         |            32+8(dchar)
+        # sink ---/---+                         ---> source -----/-----> down
+        #             |                8(dchar) |                        decoders
+        #             +---> majority -----/-----+
+        #                   voting
+        # 
+        #
+        # Due to the tight setup/hold time requiremnt for 12.5Gbps CXP, the voting logic cannot be implemented as combinational logic
+        # Hence, a pipeline approach is needed to avoid any s/h violation, where the majority voting result are pre-calculate and injected into the bus immediate after the PHY.
+        # And any downstream modules can access the voting result anytime
+
+        # cycle 1 - buffer data & calculate intermediate result  
+        buffer = Endpoint(word_layout)
+        self.sync += [
+            If(buffer.ack,
+                self.sink.connect(buffer, omit={"ack"}),
+            )
+        ]
+        self.comb += self.sink.ack.eq(buffer.ack)
+        
+        # calculate ABC, ABD, ACD, BCD  
+        char = [[self.sink.data[i*8:(i+1)*8], self.sink.k[i]] for i in range(4)]
+        voters = [Record([("data", 8), ("k", 1)]) for _ in range(4)]
+
+        for i, comb in enumerate(combinations(char, 3)):
+            self.sync += [
+                If(buffer.ack,
+                    voters[i].data.eq(reduce(and_, [code[0] for code in comb])),
+                    voters[i].k.eq(reduce(and_, [code[1] for code in comb])),
+                )
+            ]
+
+
+        # cycle 2 - inject the voting result 
+        self.sync += [
+            If(self.source.ack,
+                buffer.connect(self.source, omit={"ack", "dchar", "dchar_k"}),
+                self.source.dchar.eq(Replicate(reduce(or_, [v.data for v in voters]), 4)),
+                self.source.dchar_k.eq(Replicate(reduce(or_, [v.k for v in voters]), 4)),
+            )
+        ]
+        self.comb += buffer.ack.eq(self.source.ack)
+
+
+class Packet_Arbiter(Module):
+    def __init__(self):
+        self.decode_err = Signal()
+        self.recv_test_pak = Signal()
+        self.recv_heartbeat = Signal()
+
+        self.sink = Endpoint(word_layout_dchar)
+        self.source_stream = Endpoint(word_layout_dchar)
+        self.source_test = Endpoint(word_layout_dchar)
+        self.source_heartbeat = Endpoint(word_layout_dchar)
+        self.source_command = Endpoint(word_layout_dchar)
+
+        # # #
+
+        type = {
+            "data_stream": 0x01,
+            "control_ack_no_tag": 0x03,
+            "test_packet": 0x04,
+            "control_ack_with_tag": 0x06,
+            "event": 0x07,
+            "heartbeat": 0x09,
+        }
+
+
+        # Data packet parser
+        self.submodules.fsm = fsm = FSM(reset_state="IDLE")
+
+        fsm.act("IDLE",
+            self.sink.ack.eq(1),
+            If((self.sink.stb & (self.sink.dchar == KCode["pak_start"]) & (self.sink.dchar_k == 1)),
+                NextState("DECODE"),
+            )
+        )
+
+        fsm.act("DECODE",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                Case(self.sink.dchar, {
+                    type["data_stream"]: NextState("COPY_STREAM_PACKET"),
+                    type["test_packet"]: [
+                        self.recv_test_pak.eq(1),
+                        NextState("COPY_TEST_PACKET"),
+                    ],
+                    type["control_ack_no_tag"]:[
+                        # pass packet type for downstream decoder 
+                        self.source_command.stb.eq(1),
+                        self.source_command.data.eq(self.sink.data),
+                        NextState("COPY_COMMAND_PACKET"),
+                    ],
+                    type["control_ack_with_tag"]:[
+                        # pass packet type for downstream decoder 
+                        self.source_command.stb.eq(1),
+                        self.source_command.data.eq(self.sink.data),
+                        NextState("COPY_COMMAND_PACKET"),
+                    ],
+                    type["event"]: [
+                        # pass packet type for downstream decoder 
+                        self.source_command.stb.eq(1),
+                        self.source_command.data.eq(self.sink.data),
+                        NextState("COPY_COMMAND_PACKET"),
+                    ],
+                    type["heartbeat"] : [
+                        self.recv_heartbeat.eq(1),
+                         NextState("COPY_HEARTBEAT_PACKET"),
+                    ],
+                    "default": [
+                         self.decode_err.eq(1),
+                         # wait till next valid packet
+                         NextState("IDLE"),
+                    ],
+                }),
+            )
+        )
+
+        # copy stream data packet with K29.7 
+        fsm.act("COPY_STREAM_PACKET",
+            self.sink.connect(self.source_stream),
+            If((self.sink.stb & self.source_stream.ack & (self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                NextState("IDLE")
+            )      
+        )
+
+        # copy test sequence packet with K29.7 
+        fsm.act("COPY_TEST_PACKET",
+            self.sink.connect(self.source_test),
+            If((self.sink.stb & self.source_test.ack & (self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                NextState("IDLE")
+            )      
+        )
+
+        # copy command packet with K29.7 
+        fsm.act("COPY_COMMAND_PACKET",
+            self.sink.connect(self.source_command),
+            If((self.sink.stb & self.source_command.ack & (self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                NextState("IDLE")
+            )      
+        )
+
+        # copy heartbeat packet with K29.7 
+        fsm.act("COPY_HEARTBEAT_PACKET",
+            self.sink.connect(self.source_heartbeat),
+            If((self.sink.stb & self.source_heartbeat.ack & (self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                NextState("IDLE")
+            )      
+        )
+
+@FullMemoryWE()
+class Command_Packet_Reader(Module):
+    def __init__(self, buffer_depth, nslot):
+        self.write_ptr = Signal(log2_int(nslot))
+        self.read_ptr = Signal.like(self.write_ptr)
+        self.buffer_err = Signal()
+
+        self.sink = Endpoint(word_layout_dchar)
+
+        # # #
+         
+        # N buffers for firmware to read packet from
+         
+        self.specials.mem = mem = Memory(word_width, nslot*buffer_depth)
+        self.specials.mem_port = mem_port = mem.get_port(write_capable=True)
+        buf_mem_we = Signal.like(mem_port.we)
+        buf_mem_dat_w = Signal.like(mem_port.dat_w)
+        buf_mem_adr = Signal.like(mem_port.adr)
+
+        # buffered mem_port to improve timing
+        self.sync += [
+            mem_port.we.eq(buf_mem_we),
+            mem_port.dat_w.eq(buf_mem_dat_w),
+            mem_port.adr.eq(buf_mem_adr)
+        ]
+
+        addr_nbits = log2_int(buffer_depth)
+        addr = Signal(addr_nbits)
+        self.comb += [
+            buf_mem_adr[:addr_nbits].eq(addr),
+            buf_mem_adr[addr_nbits:].eq(self.write_ptr),
+        ]
+
+        # Data packet parser
+        self.submodules.fsm = fsm = FSM(reset_state="LOAD_BUFFER")
+
+        fsm.act("LOAD_BUFFER",
+            buf_mem_we.eq(0),
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                If(((self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                    NextState("MOVE_BUFFER_PTR"),
+                ).Else(
+                    buf_mem_we.eq(1),
+                    buf_mem_dat_w.eq(self.sink.data),
+                    NextValue(addr, addr + 1),
+                    If(addr == buffer_depth - 1,
+                        # discard the packet
+                        self.buffer_err.eq(1),
+                        NextValue(addr, addr.reset),
+                    )
+                )
+            )
+        )
+
+        fsm.act("MOVE_BUFFER_PTR",
+            self.sink.ack.eq(0),
+            If(self.write_ptr + 1 == self.read_ptr,
+                # if next one hasn't been read, overwrite the current buffer when new packet comes in
+                self.buffer_err.eq(1),
+            ).Else(
+                NextValue(self.write_ptr, self.write_ptr + 1),
+            ),
+            NextValue(addr, addr.reset),
+            NextState("LOAD_BUFFER"),
+        )
+
+
+class Heartbeat_Packet_Reader(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout_dchar)
+
+        self.host_id = Signal(4*char_width)
+        self.heartbeat = Signal(8*char_width)
+
+        # # #
+
+        n_chars = 12
+        packet_layout = [
+            ("stream_id", len(self.host_id)),
+            ("source_tag", len(self.heartbeat)), 
+        ]
+        assert layout_len(packet_layout) == n_chars*char_width  
+
+
+        cnt = Signal(max=n_chars)
+        packet_buffer = Signal(layout_len(packet_layout))
+        case = dict(
+            (i, packet_buffer[8*i:8*(i+1)].eq(self.sink.dchar))
+            for i in range(n_chars)
+        )
+        self.sync += [
+            self.host_id.eq(switch_endianness(packet_buffer[:4*char_width])),
+            self.heartbeat.eq(switch_endianness(packet_buffer[4*char_width:])),
+
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                Case(cnt, case),
+                If(((self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                    cnt.eq(cnt.reset),
+                ).Else(
+                    cnt.eq(cnt + 1),
+                ),
+            ),
+        ]
+        
+
+class Test_Sequence_Checker(Module):
+    def __init__(self):
+        self.error = Signal()
+
+        self.sink = Endpoint(word_layout_dchar)
+
+        # # #
+
+        # Section 9.9.1 (CXP-001-2021)
+        # the received test data packet (0x00, 0x01 ... 0xFF) 
+        # need to be compared against the local test sequence generator
+        cnt_bytes = [Signal(char_width, reset=i) for i in range(4)]
+        test_errors = [Signal() for _ in range(4)]
+
+        self.sync += [
+            self.sink.ack.eq(1),
+            self.error.eq(reduce(or_, test_errors))
+        ]
+        for i, (cnt, err) in enumerate(zip(cnt_bytes,test_errors)):
+            self.sync += [
+                err.eq(0),
+                If(self.sink.stb,
+                    If(((self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                        cnt.eq(cnt.reset)
+                    ).Else(
+                        If(self.sink.data[8 * i : 8 * (i + 1)] != cnt,
+                            err.eq(1),
+                        ),
+                        If(cnt == 0xFC + i,
+                            cnt.eq(cnt.reset),
+                        ).Else(
+                            cnt.eq(cnt + 4)
+                        ),
+                    )
+                )
+            ]
+
+class Trigger_Reader(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar)
+
+        self.trig = Signal()
+        self.delay = Signal(char_width)
+        self.linktrigger_n = Signal(char_width)
+
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="COPY")
+
+        fsm.act("COPY",
+            If((self.sink.stb & (self.sink.dchar == KCode["trig_indic_28_2"]) & (self.sink.dchar_k == 1)),
+                # discard K28,2
+                self.sink.ack.eq(1),
+                NextState("READ_DELAY")
+            ).Else(
+                self.sink.connect(self.source),
+            )
+        )
+
+        fsm.act("READ_DELAY",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                NextValue(self.delay, self.sink.dchar),
+                NextState("READ_LINKTRIGGER"),
+            )
+        )
+
+        fsm.act("READ_LINKTRIGGER",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                NextValue(self.linktrigger_n, self.sink.dchar),
+                self.trig.eq(1),
+                NextState("COPY"),
+            )
+        )
+
+class Trigger_ACK_Reader(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar)
+
+        self.ack = Signal()
+
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="COPY")
+
+        fsm.act("COPY",
+            If((self.sink.stb & (self.sink.dchar == KCode["io_ack"]) & (self.sink.dchar_k == 1)),
+                # discard K28,6
+                self.sink.ack.eq(1),
+                NextState("READ_ACK")
+            ).Else(
+                self.sink.connect(self.source),
+            )
+        )
+
+        fsm.act("READ_ACK",
+            self.sink.ack.eq(1),
+            If(self.sink.stb,
+                NextState("COPY"),
+                # discard the word after K28,6
+                If((self.sink.dchar == 0x01) & (self.sink.dchar_k == 0),
+                    self.ack.eq(1),
+                )
+            )
+        )
+
+class Buffer(Module):
+    def __init__(self, layout):
+        self.sink = Endpoint(layout)
+        self.source = Endpoint(layout)
+
+        # # #
+
+        self.sync += [
+            If(self.source.ack,
+                self.sink.connect(self.source, omit={"ack"}),
+            ),
+        ]
+        self.comb += [
+            self.sink.ack.eq(self.source.ack),
+        ]
+
+class EOP_Marker(Module):
+    def __init__(self):
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar)
+
+        # # #
+
+        self.sync += [
+            If(self.source.ack,
+                If(~((self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1)),
+                    self.sink.connect(self.source, omit={"ack", "eop"}),
+                ).Else(
+                    # don't pass K29.7 to downstream
+                    self.source.stb.eq(0),
+                )
+            ),
+        ]
+        self.comb += [
+            self.sink.ack.eq(self.source.ack),
+            self.source.eop.eq(((self.sink.dchar == KCode["pak_end"]) & (self.sink.dchar_k == 1))),
+        ]

src/gateware/cxp_router.py

@@ -0,0 +1,297 @@
+from migen import *
+from misoc.interconnect.csr import *
+from misoc.interconnect.stream import Endpoint
+
+from cxp_pipeline import *
+# from src.gateware.cxp_pipeline import * # for sim only
+
+from types import SimpleNamespace
+from math import lcm
+from operator import or_, add
+
+word_layout_dchar_4x = [
+    ("data", 4*word_width),
+    ("k", 4*word_width//8), 
+    ("dchar", 4*char_width), 
+    ("dchar_k", 4*char_width//8),
+    ("valid", 4),
+]
+
+class Stream_Router(Module):
+    """
+    Match the id and route stream packet to the correct downstream and strip the packet header   
+    """
+    def __init__(self, routing_ids=[0]):
+        n_id = len(routing_ids)
+        assert n_id > 0
+
+        self.sources = [stream.Endpoint(word_layout_dchar) for _ in range(n_id)]
+        self.sink = stream.Endpoint(word_layout_dchar)
+
+        # # #
+
+        
+        stream_id = Signal(char_width)
+        pak_tag = Signal(char_width)
+        stream_pak_size = Signal(char_width * 2)
+
+        self.submodules.fsm = fsm = FSM(reset_state="WAIT_HEADER")
+
+        fsm.act(
+            "WAIT_HEADER",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_id, self.sink.dchar),
+                NextState("GET_PAK_TAG"),
+            ),
+        )
+
+        fsm.act(
+            "GET_PAK_TAG",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(pak_tag, self.sink.dchar),
+                NextState("GET_PAK_SIZE_0"),
+            ),
+        )
+
+        fsm.act(
+            "GET_PAK_SIZE_0",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_pak_size[8:], self.sink.dchar),
+                NextState("GET_PAK_SIZE_1"),
+            ),
+        )
+
+        routing_case = {"default": NextState("DISCARD")}
+        for id in (routing_ids):
+            routing_case[id] = [NextState(f"COPY_TO_BUFFER_{id}")]
+
+        fsm.act(
+            "GET_PAK_SIZE_1",
+            self.sink.ack.eq(1),
+            If(
+                self.sink.stb,
+                NextValue(stream_pak_size[:8], self.sink.dchar),
+                Case(stream_id, routing_case),
+            ),
+        )
+
+        for key in routing_case:
+            if key == "default":
+                fsm.act(
+                    "DISCARD",
+                    self.sink.ack.eq(1),
+                    If(self.sink.stb,
+                        NextValue(stream_pak_size, stream_pak_size - 1),
+                        If(stream_pak_size == 0,
+                            NextValue(stream_id, stream_id.reset),
+                            NextValue(pak_tag, pak_tag.reset),
+                            NextValue(stream_pak_size, stream_pak_size.reset),
+                            NextState("WAIT_HEADER"),
+                        )
+                    ),
+                )
+            else:
+                fsm.act(
+                    f"COPY_TO_BUFFER_{key}",
+                    self.sink.connect(self.sources[key]),
+
+                    # assume downstream is not blocked
+                    If(self.sink.stb,
+                        NextValue(stream_pak_size, stream_pak_size - 1),
+                        If(stream_pak_size == 0,
+                            NextValue(stream_id, stream_id.reset),
+                            NextValue(pak_tag, pak_tag.reset),
+                            NextValue(stream_pak_size, stream_pak_size.reset),
+                            NextState("WAIT_HEADER"),
+                        )
+                    ),
+                
+                )
+
+class Stream_Packet_Gearbox(Module):
+    """
+
+    1:4 gearbox
+
+    """
+    def __init__(self):
+        self.sink = Endpoint(word_layout_dchar)
+        self.source = Endpoint(word_layout_dchar_4x)
+
+        # # #
+
+        # TODO: take into account of stbs
+        sink_bits = len(self.sink.payload.raw_bits())
+        source_bits = len(self.source.payload.raw_bits()) - 4 # 4 extra "valid" bits
+        print(sink_bits, source_bits)
+        assert source_bits/sink_bits == 4
+
+        self.submodules.fsm = fsm = FSM(reset_state="0")
+
+
+        ring_buf_size = lcm(sink_bits, source_bits)
+        # ensure the shift register is at least twice the size of sink/source dw
+        if (ring_buf_size//sink_bits) < 2:
+            ring_buf_size = ring_buf_size * 2
+        if (ring_buf_size//source_bits) < 2:
+            ring_buf_size = ring_buf_size * 2
+
+        ring_buffer_layout = []
+        for name, width in word_layout_dchar:
+            ring_buffer_layout.append(
+                (name, width*ring_buf_size)
+            )
+        ring_buffer = Record(ring_buffer_layout)
+
+
+        # Control interface
+
+        reset_reg = Signal()
+        we = Signal()
+        re = Signal()
+        level = Signal(max=ring_buf_size)
+        w_cnt = Signal(max=ring_buf_size//sink_bits)
+        r_cnt = Signal(max=ring_buf_size//source_bits)
+
+        self.sync += [
+            If(reset_reg,
+                level.eq(level.reset),
+            ).Else(
+                If(we & ~re, level.eq(level + sink_bits)),
+                If(~we & re, level.eq(level - source_bits)),
+                If(we & re, level.eq(level + sink_bits - source_bits)),
+            ),
+
+            If(reset_reg,
+                w_cnt.eq(w_cnt.reset),
+                r_cnt.eq(r_cnt.reset),
+            ).Else(
+                If(we, 
+                    If(w_cnt == ((ring_buf_size//sink_bits) - 1),
+                        w_cnt.eq(w_cnt.reset),
+                    ).Else(
+                        w_cnt.eq(w_cnt + 1),
+                    )
+                ),
+                If(re, 
+                    If(r_cnt == ((ring_buf_size//source_bits) - 1),
+                        r_cnt.eq(r_cnt.reset),
+                    ).Else(
+                        r_cnt.eq(r_cnt + 1),
+                    )
+                ),
+            )
+        ]
+
+        # IO
+        sink_cases = {}
+        for i in range(ring_buf_size//sink_bits):
+            sink_cases[i] = []
+            for name, width in word_layout_dchar:
+                src = getattr(self.sink, name)
+                dst = getattr(ring_buffer, name)[width*i: width*(i+1)]
+                sink_cases[i].append(dst.eq(src))
+        self.sync += If(self.sink.stb, Case(w_cnt, sink_cases))
+
+        source_cases = {}
+        for i in range(ring_buf_size//source_dw):
+            source_cases[i] = []
+            for name, width in word_layout_dchar_4x:
+                src = getattr(ring_buffer, name)[width*i: width*(i+1)]
+                dst = getattr(self.source, name)
+                source_cases[i].append(dst.eq(src))
+        
+
+        
+class CXPCRC32_Checker(Module):
+    """
+    Verify crc in stream data packet and stream crc less output
+    """
+    def __init__(self):
+        self.error = Signal()
+
+        # TODO: change to fifo style like in LiteEthMACCRCChecker to improve timinig?
+        self.sink = stream.Endpoint(word_layout_dchar)
+        self.submodules.buf = buf = Buffer(word_layout_dchar)
+        self.source = buf.source
+
+        # # #
+
+        self.submodules.crc = crc = CXPCRC32(word_width)
+        self.comb += crc.data.eq(self.sink.data),
+
+        self.submodules.fsm = fsm = FSM(reset_state="INIT")
+        fsm.act("INIT",
+            crc.reset.eq(1),
+            NextState("CHECKING"),
+        )
+
+        fsm.act("RESET",
+            crc.reset.eq(1),
+            self.error.eq(crc.error),
+            NextState("CHECKING"),
+        )
+
+        fsm.act("CHECKING",
+            If(self.sink.stb & self.sink.eop,
+                # discard the crc
+                self.sink.ack.eq(1),
+                buf.source.eop.eq(1),
+                NextState("RESET"),
+            ).Else(
+                self.sink.connect(buf.sink),
+            ),
+            crc.ce.eq(self.sink.stb),
+        )
+    
+
+
+
+#
+# 4x word pipeline
+# 
+
+class Stream_Merger(Module):
+    """
+    Merge n channels stream packet into one sequentially 
+    """
+    def __init__(self, layout, n_channels):
+        assert n_channels > 1 # don't need a arbiter if there is only one channel
+
+        self.active_channels = Signal(n_channels) 
+
+        self.sinks = [stream.Endpoint(layout) for _ in range(n_channels)]
+        self.source = stream.Endpoint(layout)
+        # # #
+
+        self.submodules.fsm = fsm = FSM(reset_state="0")
+
+        # Section 9.5.5 (CXP-001-2021)
+        # When Multiple connections are active, stream packets are transmitted in
+        # ascending order of Connection ID
+        # Support ch0->1->2->4 topology only
+        for n, sink in enumerate(self.sinks):
+            if n < n_channels - 1:
+                fsm.act(str(n),
+                    sink.connect(self.source),
+                    If(sink.stb & sink.eop & self.source.ack,
+                        If(self.active_channels[n+1],
+                            NextState(str(n+1)),
+                        ). Else(
+                            NextState(str(0)),
+                        ),
+                    )
+                )
+            else:
+                fsm.act(str(n),
+                    sink.connect(self.source),
+                    If(sink.stb & sink.eop & self.source.ack,
+                       NextState(str(0))
+                    ),
+                )

src/gateware/cxp_upconn.py

@@ -0,0 +1,142 @@
+from math import ceil
+
+from migen import *
+
+from misoc.cores.code_8b10b import SingleEncoder
+from misoc.interconnect import stream
+from misoc.interconnect.csr import *
+
+from cxp_pipeline import char_layout
+
+@ResetInserter()
+class ClockGen(Module):
+    def __init__(self, sys_clk_freq):
+        self.clk = Signal()
+        self.clk_10x = Signal() # 20.83MHz 48ns or 41.66MHz 24ns
+
+        self.freq2x_enable = Signal()
+        # # #
+
+        period = 1e9/sys_clk_freq
+        max_count = ceil(48/period)
+        counter = Signal(max=max_count, reset=max_count-1)
+
+        clk_div = Signal(max=10, reset=9)
+
+        self.sync += [
+            self.clk.eq(0),
+            self.clk_10x.eq(0),
+
+            If(counter == 0,
+                self.clk_10x.eq(1),
+                If(self.freq2x_enable,
+                    counter.eq(int(max_count/2)-1),
+                ).Else(
+                    counter.eq(counter.reset),
+                ),
+            ).Else(
+                counter.eq(counter-1),
+            ),
+
+            If(counter == 0,
+                If(clk_div == 0,
+                    self.clk.eq(1),
+                    clk_div.eq(clk_div.reset),
+                ).Else(
+                    clk_div.eq(clk_div-1),
+                )
+            )
+
+        ]
+
+@ResetInserter()
+@CEInserter()
+class SERDES_10bits(Module):
+    def __init__(self, pad):
+        self.oe = Signal()
+        self.d = Signal(10)
+
+        # # #
+
+        tx_bitcount = Signal(max=10)
+        tx_reg = Signal(10)
+
+        self.sync += [
+            If(self.oe,
+                # send LSB first
+                pad.eq(tx_reg[0]),
+                tx_reg.eq(Cat(tx_reg[1:], 0)),
+                tx_bitcount.eq(tx_bitcount + 1),   
+
+                If(tx_bitcount == 9,
+                    tx_bitcount.eq(0),
+                    tx_reg.eq(self.d),
+                ),
+            ).Else(
+                pad.eq(0),
+                tx_bitcount.eq(0),
+            )
+        ]
+
+class Transmitter(Module, AutoCSR):
+    def __init__(self, pad, sys_clk_freq):
+        self.bitrate2x_enable = Signal()
+        self.clk_reset = Signal()
+        self.enable = Signal()
+
+        # # #
+
+        self.sink = stream.Endpoint(char_layout)
+
+        self.submodules.cg = cg = ClockGen(sys_clk_freq)
+        self.submodules.encoder = encoder = SingleEncoder(True)
+
+        oe = Signal()
+        self.sync += [
+            If(self.enable,
+                self.sink.ack.eq(0),
+                If(cg.clk,
+                    oe.eq(1),
+                    encoder.disp_in.eq(encoder.disp_out),
+                    self.sink.ack.eq(1),
+                    encoder.d.eq(self.sink.data),
+                    encoder.k.eq(self.sink.k),
+                )
+            ).Else(
+                # discard packets until tx is enabled
+                self.sink.ack.eq(1),
+                oe.eq(0),
+            )
+        ]
+
+        self.submodules.serdes = serdes = SERDES_10bits(pad)
+
+        self.comb += [
+            cg.reset.eq(self.clk_reset),
+            cg.freq2x_enable.eq(self.bitrate2x_enable),
+
+            serdes.reset.eq(self.clk_reset),
+            serdes.ce.eq(cg.clk_10x),
+            serdes.d.eq(encoder.output),
+            serdes.oe.eq(oe),
+        ]
+
+class CXP_TXPHYs(Module, AutoCSR):
+    def __init__(self, pads, sys_clk_freq):
+        self.clk_reset = CSR()
+        self.bitrate2x_enable = CSRStorage()
+        self.enable = CSRStorage()
+
+        # # #
+
+
+        self.phys = []
+        for i, pad in enumerate(pads):
+            tx = Transmitter(pad, sys_clk_freq)
+            self.phys.append(tx)
+            setattr(self.submodules, "tx"+str(i), tx)
+            self.sync += [
+                tx.clk_reset.eq(self.clk_reset.re),
+                tx.bitrate2x_enable.eq(self.bitrate2x_enable.storage),
+                tx.enable.eq(self.enable.storage),
+            ]

src/gateware/ebaz4205.py

@@ -0,0 +1,307 @@
+#!/usr/bin/env python
+
+import argparse
+
+import analyzer
+import dma
+from artiq.gateware import rtio
+from artiq.gateware.rtio.phy import spi2, ttl_simple
+from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
+from config import generate_ident, write_csr_file, write_mem_file, write_rustc_cfg_file
+from migen import *
+from migen.build.generic_platform import IOStandard, Misc, Pins, Subsignal
+from migen.build.platforms import ebaz4205
+from migen_axi.integration.soc_core import SoCCore
+from misoc.interconnect.csr import *
+
+_ps = [
+    (
+        "ps",
+        0,
+        Subsignal("clk", Pins("E7"), IOStandard("LVCMOS33"), Misc("SLEW=FAST")),
+        Subsignal("por_b", Pins("C7"), IOStandard("LVCMOS33"), Misc("SLEW=FAST")),
+        Subsignal("srst_b", Pins("B10"), IOStandard("LVCMOS18"), Misc("SLEW=FAST")),
+    )
+]
+
+_ddr = [
+    (
+        "ddr",
+        0,
+        Subsignal(
+            "a",
+            Pins("N2 K2 M3 K3 M4 L1 L4 K4 K1 J4 F5 G4 E4 D4 F4"),
+            IOStandard("SSTL15"),
+        ),
+        Subsignal("ba", Pins("L5 R4 J5"), IOStandard("SSTL15")),
+        Subsignal("cas_n", Pins("P5"), IOStandard("SSTL15")),
+        Subsignal("cke", Pins("N3"), IOStandard("SSTL15")),
+        Subsignal("cs_n", Pins("N1"), IOStandard("SSTL15")),
+        Subsignal("ck_n", Pins("M2"), IOStandard("DIFF_SSTL15"), Misc("SLEW=FAST")),
+        Subsignal("ck_p", Pins("L2"), IOStandard("DIFF_SSTL15"), Misc("SLEW=FAST")),
+        # Pins "T1 Y1" not connected
+        Subsignal("dm", Pins("A1 F1"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
+        Subsignal(
+            "dq",
+            Pins("C3 B3 A2 A4 D3 D1 C1 E1 E2 E3 G3 H3 J3 H2 H1 J1"),
+            # Pins "P1 P3 R3 R1 T4 U4 U2 U3 V1 Y3 W1 Y4 Y2 W3 V2 V3" not connected
+            IOStandard("SSTL15_T_DCI"),
+            Misc("SLEW=FAST"),
+        ),
+        Subsignal(
+            "dqs_n",
+            Pins("B2 F2"),  # Pins "T2 W4" not connected
+            IOStandard("DIFF_SSTL15_T_DCI"),
+            Misc("SLEW=FAST"),
+        ),
+        Subsignal(
+            "dqs_p",
+            Pins("C2 G2"),  # Pins "R2 W5" not connected
+            IOStandard("DIFF_SSTL15_T_DCI"),
+            Misc("SLEW=FAST"),
+        ),
+        Subsignal("vrn", Pins("G5"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
+        Subsignal("vrp", Pins("H5"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
+        Subsignal("drst_n", Pins("B4"), IOStandard("SSTL15"), Misc("SLEW=FAST")),
+        Subsignal("odt", Pins("N5"), IOStandard("SSTL15")),
+        Subsignal("ras_n", Pins("P4"), IOStandard("SSTL15")),
+        Subsignal("we_n", Pins("M5"), IOStandard("SSTL15")),
+    )
+]
+
+# Connector J3
+_i2c = [
+    (
+        "i2c",
+        0,
+        Subsignal("scl", Pins("U12"), IOStandard("LVCMOS33")),
+        Subsignal("sda", Pins("V13"), IOStandard("LVCMOS33")),
+    )
+]
+
+_spi = [
+    (
+        "spi",
+        0,
+        Subsignal("clk", Pins("V20")),
+        Subsignal("mosi", Pins("U20")),
+        Subsignal("cs_n", Pins("P19")),
+        IOStandard("LVCMOS33"),
+    )
+]
+
+
+# Connector DATA1
+def _create_ttl():
+    _ttl = []
+
+    for idx, elem in enumerate([x for x in range(5, 21) if x not in (10, 12)]):
+        _ttl.append(
+            ("ttl", idx, Pins("DATA1:DATA1-{}".format(elem)), IOStandard("LVCMOS33")),
+        )
+    return _ttl
+
+
+class EBAZ4205(SoCCore):
+    def __init__(self, rtio_clk=125e6, acpki=False):
+        self.acpki = acpki
+
+        platform = ebaz4205.Platform()
+        platform.toolchain.bitstream_commands.extend(
+            [
+                "set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
+            ]
+        )
+        platform.add_extension(_ps)
+        platform.add_extension(_ddr)
+        platform.add_extension(_i2c)
+        platform.add_extension(_spi)
+        platform.add_extension(_create_ttl())
+
+        gmii = platform.request("gmii")
+        platform.add_period_constraint(gmii.rx_clk, 10)
+        platform.add_period_constraint(gmii.tx_clk, 10)
+        platform.add_platform_command(
+            "set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets gmii_tx_clk_IBUF]"
+        )
+
+        ident = generate_ident(self.__class__.__name__)
+        if self.acpki:
+            ident = "acpki_" + ident
+        SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident)
+        fix_serdes_timing_path(platform)
+        self.config["RTIO_FREQUENCY"] = str(rtio_clk / 1e6)
+        platform.add_period_constraint(self.ps7.cd_sys.clk, 10)
+
+        self.comb += [
+            self.ps7.enet0.enet.gmii.tx_clk.eq(gmii.tx_clk),
+            self.ps7.enet0.enet.gmii.rx_clk.eq(gmii.rx_clk),
+        ]
+        self.clock_domains.cd_eth_rx = ClockDomain(reset_less=False)
+        self.clock_domains.cd_eth_tx = ClockDomain(reset_less=False)
+        self.comb += [
+            ClockSignal("eth_rx").eq(gmii.rx_clk),
+            ClockSignal("eth_tx").eq(gmii.tx_clk),
+        ]
+        self.sync.eth_tx += [
+            gmii.txd.eq(self.ps7.enet0.enet.gmii.txd),
+            gmii.tx_en.eq(self.ps7.enet0.enet.gmii.tx_en),
+        ]
+        self.sync.eth_rx += [
+            self.ps7.enet0.enet.gmii.rxd.eq(gmii.rxd),
+            self.ps7.enet0.enet.gmii.rx_dv.eq(gmii.rx_dv),
+        ]
+
+        # MDIO
+        mdio = platform.request("mdio")
+        self.comb += mdio.mdc.eq(self.ps7.enet0.enet.mdio.mdc)
+        self.specials += Instance(
+            "IOBUF",
+            i_I=self.ps7.enet0.enet.mdio.o,
+            io_IO=mdio.mdio,
+            o_O=self.ps7.enet0.enet.mdio.i,
+            i_T=~self.ps7.enet0.enet.mdio.t_n,
+        )
+
+        # I2C
+        i2c = self.platform.request("i2c")
+        self.specials += [
+            # SCL
+            Instance(
+                "IOBUF",
+                i_I=self.ps7.i2c0.scl.o,
+                io_IO=i2c.scl,
+                o_O=self.ps7.i2c0.scl.i,
+                i_T=~self.ps7.i2c0.scl.t_n,
+            ),
+            # SDA
+            Instance(
+                "IOBUF",
+                i_I=self.ps7.i2c0.sda.o,
+                io_IO=i2c.sda,
+                o_O=self.ps7.i2c0.sda.i,
+                i_T=~self.ps7.i2c0.sda.t_n,
+            ),
+        ]
+
+        self.rtio_channels = []
+        for i in (0, 1):
+            print("USER LED at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
+            user_led = self.platform.request("user_led", i)
+            phy = ttl_simple.Output(user_led)
+            self.submodules += phy
+            self.rtio_channels.append(rtio.Channel.from_phy(phy))
+
+        for i in range(14):
+            print("TTL at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
+            ttl = self.platform.request("ttl", i)
+            phy = ttl_simple.InOut(ttl)
+            self.submodules += phy
+            self.rtio_channels.append(rtio.Channel.from_phy(phy))
+
+        print("SPI at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
+        spi_phy = spi2.SPIMaster(platform.request("spi"))
+        self.submodules += spi_phy
+        self.rtio_channels.append(rtio.Channel.from_phy(spi_phy, ififo_depth=4))
+        self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
+        self.rtio_channels.append(rtio.LogChannel())
+
+        self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
+        self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
+        self.csr_devices.append("rtio_core")
+        if self.acpki:
+            import acpki
+
+            self.config["KI_IMPL"] = "acp"
+            self.submodules.rtio = acpki.KernelInitiator(
+                self.rtio_tsc,
+                bus=self.ps7.s_axi_acp,
+                user=self.ps7.s_axi_acp_user,
+                evento=self.ps7.event.o,
+            )
+            self.csr_devices.append("rtio")
+        else:
+            self.config["KI_IMPL"] = "csr"
+            self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
+            self.csr_devices.append("rtio")
+
+        self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
+        self.csr_devices.append("rtio_dma")
+
+        self.submodules.cri_con = rtio.CRIInterconnectShared(
+            [self.rtio.cri, self.rtio_dma.cri],
+            [self.rtio_core.cri],
+            enable_routing=True,
+        )
+        self.csr_devices.append("cri_con")
+
+        self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
+        self.csr_devices.append("rtio_moninj")
+
+        self.submodules.rtio_analyzer = analyzer.Analyzer(
+            self.rtio_tsc, self.rtio_core.cri, self.ps7.s_axi_hp1
+        )
+        self.csr_devices.append("rtio_analyzer")
+
+
+class BASE(EBAZ4205):
+    def __init__(self, rtio_clk, acpki):
+        EBAZ4205.__init__(self, rtio_clk, acpki)
+
+
+VARIANTS = {cls.__name__.lower(): cls for cls in [BASE]}
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="ARTIQ port to the EBAZ4205 control card of Ebit E9+ BTC miner"
+    )
+    parser.add_argument(
+        "-r", default=None, help="build Rust interface into the specified file"
+    )
+    parser.add_argument(
+        "-m", default=None, help="build Rust memory interface into the specified file"
+    )
+    parser.add_argument(
+        "-c",
+        default=None,
+        help="build Rust compiler configuration into the specified file",
+    )
+    parser.add_argument(
+        "-g", default=None, help="build gateware into the specified directory"
+    )
+    parser.add_argument("--rtio-clk", default=125e6, help="RTIO Clock Frequency (Hz)")
+    parser.add_argument(
+        "-V",
+        "--variant",
+        default="base",
+        help="variant: " "[acpki_]base" "(default: %(default)s)",
+    )
+    args = parser.parse_args()
+
+    rtio_clk = int(args.rtio_clk)
+    variant = args.variant.lower()
+    acpki = variant.startswith("acpki_")
+    if acpki:
+        variant = variant[6:]
+
+    try:
+        cls = VARIANTS[variant]
+    except KeyError:
+        raise SystemExit("Invalid variant (-V/--variant)")
+
+    soc = cls(rtio_clk=rtio_clk, acpki=acpki)
+    soc.finalize()
+
+    if args.r is not None:
+        write_csr_file(soc, args.r)
+    if args.m is not None:
+        write_mem_file(soc, args.m)
+    if args.c is not None:
+        write_rustc_cfg_file(soc, args.c)
+    if args.g is not None:
+        soc.build(build_dir=args.g)
+
+
+if __name__ == "__main__":
+    main()

src/gateware/kasli_soc.py

@@ -24,10 +24,10 @@ from artiq.gateware.wrpll import wrpll
 
 import dma
 import analyzer
-import acpki
+import acpki as acpki_lib
 import drtio_aux_controller
 import zynq_clocking
-from config import write_csr_file, write_mem_file, write_rustc_cfg_file
+from config import generate_ident, write_csr_file, write_mem_file, write_rustc_cfg_file
 
 eem_iostandard_dict = {
      0: "LVDS_25",
@@ -115,7 +115,7 @@ class GenericStandalone(SoCCore):
         platform.toolchain.bitstream_commands.extend([
             "set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
         ])
-        ident = description["variant"]
+        ident = generate_ident(description["variant"])
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -184,10 +184,10 @@ class GenericStandalone(SoCCore):
 
         if self.acpki:
             self.config["KI_IMPL"] = "acp"
-            self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
-                                                         bus=self.ps7.s_axi_acp,
-                                                         user=self.ps7.s_axi_acp_user,
-                                                         evento=self.ps7.event.o)
+            self.submodules.rtio = acpki_lib.KernelInitiator(self.rtio_tsc,
+                                                             bus=self.ps7.s_axi_acp,
+                                                             user=self.ps7.s_axi_acp_user,
+                                                             evento=self.ps7.event.o)
             self.csr_devices.append("rtio")
         else:
             self.config["KI_IMPL"] = "csr"
@@ -229,7 +229,7 @@ class GenericMaster(SoCCore):
         platform.toolchain.bitstream_commands.extend([
             "set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
         ])
-        ident = description["variant"]
+        ident = generate_ident(description["variant"])
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -349,10 +349,10 @@ class GenericMaster(SoCCore):
 
         if self.acpki:
             self.config["KI_IMPL"] = "acp"
-            self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
-                                                         bus=self.ps7.s_axi_acp,
-                                                         user=self.ps7.s_axi_acp_user,
-                                                         evento=self.ps7.event.o)
+            self.submodules.rtio = acpki_lib.KernelInitiator(self.rtio_tsc,
+                                                             bus=self.ps7.s_axi_acp,
+                                                             user=self.ps7.s_axi_acp_user,
+                                                             evento=self.ps7.event.o)
             self.csr_devices.append("rtio")
         else:
             self.config["KI_IMPL"] = "csr"
@@ -438,7 +438,7 @@ class GenericSatellite(SoCCore):
         platform.toolchain.bitstream_commands.extend([
             "set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
         ])
-        ident = description["variant"]
+        ident = generate_ident(description["variant"])
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -544,10 +544,10 @@ class GenericSatellite(SoCCore):
 
         if self.acpki:
             self.config["KI_IMPL"] = "acp"
-            self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
-                                                         bus=self.ps7.s_axi_acp,
-                                                         user=self.ps7.s_axi_acp_user,
-                                                         evento=self.ps7.event.o)
+            self.submodules.rtio = acpki_lib.KernelInitiator(self.rtio_tsc,
+                                                             bus=self.ps7.s_axi_acp,
+                                                             user=self.ps7.s_axi_acp_user,
+                                                             evento=self.ps7.event.o)
             self.csr_devices.append("rtio")
         else:
             self.config["KI_IMPL"] = "csr"
@@ -560,7 +560,10 @@ class GenericSatellite(SoCCore):
         self.submodules.local_io = SyncRTIO(
             self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
         )
-        self.comb += self.drtiosat.async_errors.eq(self.local_io.async_errors)
+        self.comb += [
+            self.drtiosat.async_errors.eq(self.local_io.async_errors),
+            self.local_io.sed_spread_enable.eq(self.drtiosat.sed_spread_enable.storage)
+        ]
 
         self.submodules.cri_con = rtio.CRIInterconnectShared(
             [self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri],

src/gateware/zc706.py

@@ -25,7 +25,9 @@ import analyzer
 import acpki
 import drtio_aux_controller
 import zynq_clocking
-from config import write_csr_file, write_mem_file, write_rustc_cfg_file
+import cxp
+import cxp_4r_fmc
+from config import generate_ident, write_csr_file, write_mem_file, write_rustc_cfg_file
 
 class SMAClkinForward(Module):
     def __init__(self, platform):
@@ -130,7 +132,7 @@ class ZC706(SoCCore):
         platform = zc706.Platform()
         prepare_zc706_platform(platform)
 
-        ident = self.__class__.__name__
+        ident = generate_ident(self.__class__.__name__)
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -138,7 +140,7 @@ class ZC706(SoCCore):
         platform.add_extension(si5324_fmc33)
         self.comb += platform.request("si5324_33").rst_n.eq(1)
 
-        cdr_clk = Signal()
+        self.cdr_clk = Signal()
         cdr_clk_buf = Signal()
         si5324_out = platform.request("si5324_clkout")
         platform.add_period_constraint(si5324_out.p, 8.0)
@@ -146,11 +148,11 @@ class ZC706(SoCCore):
             Instance("IBUFDS_GTE2",
                 i_CEB=0,
                 i_I=si5324_out.p, i_IB=si5324_out.n,
-                o_O=cdr_clk,
+                o_O=self.cdr_clk,
                 p_CLKCM_CFG="TRUE",
                 p_CLKRCV_TRST="TRUE",
                 p_CLKSWING_CFG=3),
-            Instance("BUFG", i_I=cdr_clk, o_O=cdr_clk_buf)
+            Instance("BUFG", i_I=self.cdr_clk, o_O=cdr_clk_buf)
         ]
         self.config["HAS_SI5324"] = None
         self.config["SI5324_AS_SYNTHESIZER"] = None
@@ -203,7 +205,7 @@ class _MasterBase(SoCCore):
 
         platform = zc706.Platform()
         prepare_zc706_platform(platform)
-        ident = self.__class__.__name__
+        ident = generate_ident(self.__class__.__name__)
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -344,7 +346,7 @@ class _SatelliteBase(SoCCore):
 
         platform = zc706.Platform()
         prepare_zc706_platform(platform)
-        ident = self.__class__.__name__
+        ident = generate_ident(self.__class__.__name__)
         if self.acpki:
             ident = "acpki_" + ident
         SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
@@ -487,6 +489,10 @@ class _SatelliteBase(SoCCore):
         self.csr_devices.append("rtio_dma")
 
         self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels)
+        self.comb += [
+            self.drtiosat.async_errors.eq(self.local_io.async_errors),
+            self.local_io.sed_spread_enable.eq(self.drtiosat.sed_spread_enable.storage)
+        ]
         self.submodules.cri_con = rtio.CRIInterconnectShared(
             [self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri],
             [self.local_io.cri] + self.drtio_cri,
@@ -648,6 +654,119 @@ class _NIST_QC2_RTIO:
         self.add_rtio(rtio_channels)
 
 
+class CXP_FMC():
+    """
+    CoaXpress FMC with 4 CXP channel and 1 SMA trigger
+    """
+    def __init__(self):
+        platform = self.platform
+        platform.add_extension(cxp_4r_fmc.fmc_adapter_io)
+        platform.add_extension(leds_fmc33)
+
+        debug_sma = [
+            ("user_sma_clock_33", 0,
+                Subsignal("p_tx", Pins("AD18"), IOStandard("LVCMOS33")),
+                Subsignal("n_rx", Pins("AD19"), IOStandard("LVCMOS33")),
+            ),
+        ]
+
+        pmod1_33 = [
+            ("pmod1_33", 0, Pins("AJ21"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 1, Pins("AK21"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 2, Pins("AB21"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 3, Pins("AB16"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 4, Pins("Y20"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 5, Pins("AA20"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 6, Pins("AC18"), IOStandard("LVCMOS33")),
+            ("pmod1_33", 7, Pins("AC19"), IOStandard("LVCMOS33")),
+        ]
+
+        platform.add_extension(debug_sma)
+        platform.add_extension(pmod1_33)
+        debug_sma_pad = platform.request("user_sma_clock_33") 
+        pmod_pads = [platform.request("pmod1_33", i) for i in range(8)]
+
+        clk_freq = 125e6
+
+        links = 4
+        master_ch = 0
+        cxp_downconn_pads = [platform.request("CXP_HS", i) for i in range(links)]
+        cxp_upconn_pads = [platform.request("CXP_LS", i) for i in range(links)]
+
+
+        self.submodules.cxp_phys = cxp_phys = cxp.CXP_PHYS(
+            refclk=self.cdr_clk,
+            upconn_pads=cxp_upconn_pads,
+            downconn_pads=cxp_downconn_pads,
+            sys_clk_freq=clk_freq,
+            master=master_ch,
+        )
+        self.csr_devices.append("cxp_phys")
+
+
+        rtio_channels = []
+        cxp_csr_group = []
+        cxp_mem_group = []
+        cxp_core_pipelines = []
+        for i, phy in enumerate(cxp_phys.phys):
+            cxp_name = "cxp" + str(i)
+
+            core = cxp.CXP_Core(phy)
+
+            setattr(self.submodules, cxp_name, core)
+            self.csr_devices.append(cxp_name)
+            cxp_csr_group.append(cxp_name)
+
+            cxp_core_pipelines.append(core)
+
+            # Add memory group
+
+            mem_name = "cxp_" + str(i) + "_mem"
+            cxp_mem_group.append(mem_name)
+            mem_size = core.get_mem_size()
+            # upper half is tx while lower half is rx
+            memory_address = self.axi2csr.register_port(core.get_tx_port(), mem_size)
+            self.axi2csr.register_port(core.get_rx_port(), mem_size)
+            self.add_memory_region(mem_name, self.mem_map["csr"] + memory_address, mem_size * 2)
+
+        self.add_memory_group("cxp_mem", cxp_mem_group)
+        self.add_csr_group("cxp", cxp_csr_group)
+
+        self.submodules.cxp_frame_pipeline = cxp_frame_pipeline = cxp.CXP_Frame_Pipeline(cxp_core_pipelines, pmod_pads, master=master_ch)
+        self.csr_devices.append("cxp_frame_pipeline")
+
+        print("CoaXPress at RTIO channel 0x{:06x}".format(len(rtio_channels)))
+        # rtio_channels.append(rtio.Channel.from_phy(cxp_frame_pipeline ))
+        rtio_channels += [
+            rtio.Channel(cxp_frame_pipeline.trigger),
+            rtio.Channel(cxp_frame_pipeline.config),
+            rtio.Channel(cxp_frame_pipeline.gate_data),
+        ]
+            
+
+       
+
+        # max freq of cxp_gtx_rx = linerate/internal_datawidth = 12.5Gbps/40 = 312.5MHz
+        # zc706 use speed grade 2 which only support up to 10.3125Gbps (4ns)
+        # pushing to 12.5Gbps (3.2ns) will result in Pulse width violation but setup/hold times are met
+        rx = cxp_phys.phys[0].rx
+        platform.add_period_constraint(rx.gtx.cd_cxp_gtx_rx.clk, 3.2)
+        # constraint the CLK path 
+        platform.add_false_path_constraints(self.sys_crg.cd_sys.clk, rx.gtx.cd_cxp_gtx_rx.clk)
+
+        # FIXME remove this placeholder RTIO channel
+        # There are too few RTIO channels and cannot be compiled (adr width issue of the lane distributor)
+        # see https://github.com/m-labs/artiq/pull/2158 for similar issue
+        print("USER LED at RTIO channel 0x{:06x}".format(len(rtio_channels)))
+        phy = ttl_simple.Output(self.platform.request("user_led_33", 0))
+        self.submodules += phy
+        rtio_channels.append(rtio.Channel.from_phy(phy))
+        self.config["HAS_RTIO_LOG"] = None
+        rtio_channels.append(rtio.LogChannel())
+
+        self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels)
+        self.add_rtio(rtio_channels)
+
 class NIST_CLOCK(ZC706, _NIST_CLOCK_RTIO):
     def __init__(self, acpki, drtio100mhz):
         ZC706.__init__(self, acpki)
@@ -680,8 +799,13 @@ class NIST_QC2_Satellite(_SatelliteBase, _NIST_QC2_RTIO):
         _SatelliteBase.__init__(self, acpki, drtio100mhz)
         _NIST_QC2_RTIO.__init__(self)
 
+class CXP_Demo(ZC706, CXP_FMC):
+    def __init__(self, acpki, drtio100mhz):
+        ZC706.__init__(self, acpki)
+        CXP_FMC.__init__(self)
+
 VARIANTS = {cls.__name__.lower(): cls for cls in [NIST_CLOCK, NIST_CLOCK_Master, NIST_CLOCK_Satellite,
-                                                  NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite]}
+                                                  NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite, CXP_Demo]}
 
 def main():
     parser = argparse.ArgumentParser(

src/libboard_artiq/Cargo.toml.tpl

@@ -10,6 +10,7 @@ name = "libboard_artiq"
 [features]
 target_zc706 = ["libboard_zynq/target_zc706", "libconfig/target_zc706"]
 target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libconfig/target_kasli_soc"]
+target_ebaz4205 = ["libboard_zynq/target_ebaz4205", "libconfig/target_ebaz4205"]
 calibrate_wrpll_skew = []
 
 [build-dependencies]
@@ -19,10 +20,11 @@ build_zynq = { path = "../libbuild_zynq" }
 log = "0.4"
 log_buffer = { version = "1.2" }
 crc = { version = "1.7", default-features = false }
-core_io = { version = "0.1", features = ["collections"] }
+core_io = { git = "https://git.m-labs.hk/M-Labs/rs-core_io.git", rev = "e9d3edf027", features = ["collections"] }
 embedded-hal = "0.2"
 nb = "1.0"
 void = { version = "1", default-features = false }
+byteorder = { version = "1.3", default-features = false }
 
 io = { path = "../libio", features = ["byteorder"] }
 libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq" }

src/libboard_artiq/src/cxp.rs

@@ -0,0 +1,388 @@
+use core::{fmt, result::Result};
+
+use embedded_hal::blocking::delay::DelayMs;
+use libboard_zynq::timer::GlobalTimer;
+use log::{info, warn};
+
+use crate::{cxp_ctrl::{read_u32, read_u64, reset_tag, send_test_packet, write_bytes_no_ack, write_u32, write_u64},
+            cxp_phys::{change_linerate, CXP_SPEED},
+            cxp_proto::Error as ProtoError,
+            pl::csr::{CXP, CXP_LEN}};
+
+// Bootstrap registers address
+const STANDARD: u32 = 0x0000;
+const REVISION: u32 = 0x0004;
+const CONNECTION_RESET: u32 = 0x4000;
+const DEVICE_CONNECTION_ID: u32 = 0x4004;
+const MASTER_HOST_CONNECTION_ID: u32 = 0x4008;
+
+const CONTROL_PACKET_SIZE_MAX: u32 = 0x400C;
+const STREAM_PACKET_SIZE_MAX: u32 = 0x4010;
+const CONNECTION_CFG: u32 = 0x4014;
+const CONNECTION_CFG_DEFAULT: u32 = 0x4018;
+
+const TESTMODE: u32 = 0x401C;
+const TEST_ERROR_COUNT_SELECTOR: u32 = 0x4020;
+const TEST_ERROR_COUNT: u32 = 0x4024;
+const TEST_PACKET_COUNT_TX: u32 = 0x4028;
+const TEST_PACKET_COUNT_RX: u32 = 0x4030;
+
+const VERSION_SUPPORTED: u32 = 0x4044;
+const VERSION_USED: u32 = 0x4048;
+
+// Setup const
+// TEST: result
+// Currently, multilane/channel is not working properly in gateware due to buffer overflow issue
+// only single lane @ 0x800 max pak size or dual lane 256 max pak size is working
+// tho the small lane size only masked the issue as it just create more overhead for the camera
+// 0x800/16 OK for fifo size 256 or 32
+// 0x800/8 NG for fifo size 256 + pixel fifo 32
+// TODO: upgrate it to 2 KiB (0x800) after implementing the correct stream merger in gateware
+// const MAX_STREAM_PAK_SIZE: u32 = 0x800; // 2 KiB
+const MAX_STREAM_PAK_SIZE: u32 = 0x800 / 16;
+const TX_TEST_CNT: u8 = 10;
+pub const MASTER_CHANNEL: u8 = 0;
+const HOST_CONNECTION_ID: u32 = 0xC001C0DE; // TODO: rename this
+
+#[derive(Debug)]
+pub enum Error {
+    CameraNotDetected,
+    ConnectionLost,
+    UnstableRX,
+    UnstableTX,
+    UnsupportedSpeed(u32),
+    UnsupportedTopology,
+    UnsupportedVersion,
+    Protocol(ProtoError),
+}
+
+impl From<ProtoError> for Error {
+    fn from(value: ProtoError) -> Error {
+        Error::Protocol(value)
+    }
+}
+
+impl fmt::Display for Error {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        match self {
+            &Error::CameraNotDetected => write!(f, "CameraNotDetected"),
+            &Error::ConnectionLost => write!(f, "ConnectionLost Some active channels cannot be detected"),
+            &Error::UnstableRX => write!(f, "UnstableRX RX connection test failed"),
+            &Error::UnstableTX => write!(f, "UnstableTX TX connection test failed"),
+            &Error::UnsupportedSpeed(linerate_code) => write!(
+                f,
+                "UnsupportedSpeed {:#X} linerate code is not supported",
+                linerate_code
+            ),
+            &Error::UnsupportedTopology => write!(
+                f,
+                "UnsupportedTopology CH#1 should be the master channel while CH#2-4 should be connected to extension \
+                 #1-3 respectively"
+            ),
+            &Error::UnsupportedVersion => write!(
+                f,
+                "UnsupportedVersion Cannot find a compatible protocol version between the grabber & camera"
+            ),
+            &Error::Protocol(ref err) => write!(f, "ProtocolError {}", err),
+        }
+    }
+}
+
+pub fn setup() -> Result<bool, Error> {
+    // assume timer was initialized successfully
+    let mut timer = unsafe { GlobalTimer::get() };
+    camera_setup(&mut timer)
+}
+
+fn scan_active_channels() -> u8 {
+    let mut active_channels: u8 = 0;
+    for ch in 0..CXP_LEN {
+        if unsafe { (CXP[ch].rx_ready_read)() } == 1 {
+            active_channels += 1;
+        }
+    }
+    active_channels
+}
+
+fn discover_camera(timer: &mut GlobalTimer) -> Result<(), Error> {
+    // Section 7.6 (CXP-001-2021)
+    // 1.25Gbps (CXP_1) and 3.125Gbps (CXP_3) are the discovery rate
+    // both linerate need to be checked as camera only support ONE of discovery rates
+    for speed in [CXP_SPEED::CXP_1, CXP_SPEED::CXP_3].iter() {
+        change_linerate(*speed);
+
+        // Section 12.1.2 (CXP-001-2021)
+        // send ConnectionReset on all channels -> wait 200ms -> scan for active channels
+        for ch in 0..CXP_LEN {
+            write_bytes_no_ack(ch as u8, CONNECTION_RESET, &1_u32.to_be_bytes(), false)?;
+        }
+        timer.delay_ms(200);
+
+        if scan_active_channels() > 0 {
+            return Ok(());
+        }
+    }
+    Err(Error::CameraNotDetected)
+}
+
+fn check_master_channel() -> Result<(), Error> {
+    if read_u32(MASTER_CHANNEL, DEVICE_CONNECTION_ID, false)? == 0 {
+        Ok(())
+    } else {
+        warn!(
+            "Channel #{} is not connected to master channel of the camera",
+            MASTER_CHANNEL
+        );
+        Err(Error::UnsupportedTopology)
+    }
+}
+
+fn check_connection_topology(active_channels: u8) -> Result<(), Error> {
+    // only the simple topology MASTER:ch0, extension:ch1,2,3 is supported right now
+    for ch in 0..active_channels {
+        if read_u32(ch, DEVICE_CONNECTION_ID, false)? != ch as u32 {
+            warn!("Channel #{} is not connected to the right port of the camera", ch);
+            return Err(Error::UnsupportedTopology);
+        };
+        match ch {
+            0 => info!("CHANNEL #{} is active - master connection", ch),
+            _ => info!("CHANNEL #{} is active - extension connection", ch),
+        }
+    }
+    Ok(())
+}
+
+fn negotiate_active_channels(timer: &mut GlobalTimer) -> Result<u8, Error> {
+    let max_camera_chs = read_u32(MASTER_CHANNEL, CONNECTION_CFG_DEFAULT, false)? >> 16;
+    let available_chs = max_camera_chs.min(CXP_LEN as u32);
+
+    // activate channels on camera but preserve the discovery linerate
+    let current_cfg = read_u32(MASTER_CHANNEL, CONNECTION_CFG, false)?;
+    write_u32(
+        MASTER_CHANNEL,
+        CONNECTION_CFG,
+        current_cfg & 0xFFFF | available_chs << 16,
+        false,
+    )?;
+
+    timer.delay_ms(200);
+    let active_chs = scan_active_channels();
+    check_connection_topology(active_chs)?;
+
+    if available_chs > active_chs as u32 {
+        info!(
+            "Only detected {} channel(s), disabling excess channels on camera",
+            active_chs
+        );
+        write_u32(
+            MASTER_CHANNEL,
+            CONNECTION_CFG,
+            current_cfg & 0xFFFF | ((active_chs as u32) << 16),
+            false,
+        )?;
+
+        timer.delay_ms(200);
+        // check no active channels are down after the cfg change
+        if active_chs != scan_active_channels() {
+            return Err(Error::ConnectionLost);
+        }
+    }
+    Ok(active_chs)
+}
+
+fn set_host_connection_id() -> Result<(), Error> {
+    info!("setting host connection id to = {}", HOST_CONNECTION_ID);
+    write_u32(MASTER_CHANNEL, MASTER_HOST_CONNECTION_ID, HOST_CONNECTION_ID, false)?;
+    Ok(())
+}
+
+fn negotiate_cxp_version() -> Result<bool, Error> {
+    let rev = read_u32(MASTER_CHANNEL, REVISION, false)?;
+
+    let mut major_rev: u32 = rev >> 16;
+    let mut minor_rev: u32 = rev & 0xFF;
+    info!("Camera's CXP revision is {}.{}", major_rev, minor_rev);
+
+    // Section 12.1.4 (CXP-001-2021)
+    // For CXP 2.0 and onward, Host need to check the VersionSupported register to determine
+    // the highest common version that supported by both device & host
+    if major_rev >= 2 {
+        let reg = read_u32(MASTER_CHANNEL, VERSION_SUPPORTED, false)?;
+
+        if ((reg >> 3) & 1) == 1 {
+            major_rev = 2;
+            minor_rev = 1;
+        } else if ((reg >> 2) & 1) == 1 {
+            major_rev = 2;
+            minor_rev = 0;
+        } else {
+            return Err(Error::UnsupportedVersion);
+        }
+
+        write_u32(MASTER_CHANNEL, VERSION_USED, major_rev << 16 | minor_rev, false)?;
+    }
+    info!(
+        "Camera supports CXP {}.{}, using CXP {}.{} protcol now",
+        major_rev, minor_rev, major_rev, minor_rev
+    );
+
+    Ok(major_rev >= 2)
+}
+
+fn negotiate_pak_max_size(with_tag: bool) -> Result<(), Error> {
+    // DEBUG: print control & stream packet size
+    let reg = read_u32(MASTER_CHANNEL, CONTROL_PACKET_SIZE_MAX, with_tag)?;
+    info!("Max CTRL PAK size = {:#010X}", reg);
+
+    let reg = read_u32(MASTER_CHANNEL, STREAM_PACKET_SIZE_MAX, with_tag)?;
+    info!("Max STREAM PAK size = {:#010X}", reg);
+
+    write_u32(MASTER_CHANNEL, STREAM_PACKET_SIZE_MAX, MAX_STREAM_PAK_SIZE, with_tag)?;
+    let reg = read_u32(MASTER_CHANNEL, STREAM_PACKET_SIZE_MAX, with_tag)?;
+    info!("Max STREAM PAK size = {:#010X}", reg);
+    Ok(())
+}
+
+fn check_magic_word() -> Result<(), Error> {
+    let reg = read_u32(MASTER_CHANNEL, STANDARD, true)?;
+    info!("CoaXPress magic WORD = {:#010X} !!!", reg);
+    Ok(())
+}
+
+fn decode_cxp_speed(linerate_code: u32) -> Option<CXP_SPEED> {
+    match linerate_code {
+        0x28 => Some(CXP_SPEED::CXP_1),
+        0x30 => Some(CXP_SPEED::CXP_2),
+        0x38 => Some(CXP_SPEED::CXP_3),
+        0x40 => Some(CXP_SPEED::CXP_5),
+        0x48 => Some(CXP_SPEED::CXP_6),
+        0x50 => Some(CXP_SPEED::CXP_10),
+        0x58 => Some(CXP_SPEED::CXP_12),
+        _ => None,
+    }
+}
+
+fn set_operation_linerate(active_channels: u8, with_tag: bool, timer: &mut GlobalTimer) -> Result<(), Error> {
+    let current_cfg = read_u32(MASTER_CHANNEL, CONNECTION_CFG, with_tag)?;
+    info!("Current connection cfg = {:#010X}", current_cfg);
+
+    let recommended_linerate_code = read_u32(MASTER_CHANNEL, CONNECTION_CFG_DEFAULT, with_tag)? & 0xFFFF;
+    info!("recommended_linerate_code = {:#010X}", recommended_linerate_code);
+
+    if let Some(speed) = decode_cxp_speed(recommended_linerate_code) {
+        // preserve the number of active channels
+        write_u32(
+            MASTER_CHANNEL,
+            CONNECTION_CFG,
+            current_cfg & 0xFFFF0000 | recommended_linerate_code,
+            with_tag,
+        )?;
+
+        change_linerate(speed);
+        timer.delay_ms(200);
+
+        // check no active channels are down after linerate change
+        if scan_active_channels() == active_channels {
+            Ok(())
+        } else {
+            Err(Error::ConnectionLost)
+        }
+    } else {
+        Err(Error::UnsupportedSpeed(recommended_linerate_code))
+    }
+}
+
+fn test_counter_reset(channel: u8, with_tag: bool) -> Result<(), Error> {
+    unsafe { (CXP[channel as usize].rx_test_counts_reset_write)(1) };
+    write_u32(MASTER_CHANNEL, TEST_ERROR_COUNT_SELECTOR, channel as u32, with_tag)?;
+    write_u32(MASTER_CHANNEL, TEST_ERROR_COUNT, 0, with_tag)?;
+    write_u64(MASTER_CHANNEL, TEST_PACKET_COUNT_TX, 0, with_tag)?;
+    write_u64(MASTER_CHANNEL, TEST_PACKET_COUNT_RX, 0, with_tag)?;
+    Ok(())
+}
+
+fn verify_test_result(channel: u8, with_tag: bool) -> Result<(), Error> {
+    write_u32(MASTER_CHANNEL, TEST_ERROR_COUNT_SELECTOR, channel as u32, with_tag)?;
+
+    // Section 9.9.3 (CXP-001-2021)
+    // verify grabber -> camera connection test result
+    if read_u64(MASTER_CHANNEL, TEST_PACKET_COUNT_RX, with_tag)? != TX_TEST_CNT as u64 {
+        return Err(Error::UnstableTX);
+    };
+    if read_u32(MASTER_CHANNEL, TEST_ERROR_COUNT, with_tag)? > 0 {
+        return Err(Error::UnstableTX);
+    };
+
+    // Section 9.9.4 (CXP-001-2021)
+    // verify camera -> grabber connection test result
+    let camera_test_pak_cnt = read_u64(MASTER_CHANNEL, TEST_PACKET_COUNT_TX, true)?;
+    unsafe {
+        if (CXP[channel as usize].rx_test_packet_counter_read)() != camera_test_pak_cnt as u16 {
+            info!(
+                "CHANNEL #{} test packet cnt = {}",
+                channel,
+                (CXP[channel as usize].rx_test_packet_counter_read)()
+            );
+            return Err(Error::UnstableRX);
+        };
+        if (CXP[channel as usize].rx_test_error_counter_read)() > 0 {
+            info!(
+                "CHANNEL #{} test packet error cnt = {}",
+                channel,
+                (CXP[channel as usize].rx_test_error_counter_read)()
+            );
+            return Err(Error::UnstableRX);
+        };
+    };
+    info!("CHANNEL #{} pass testing", channel);
+    Ok(())
+}
+
+fn test_channels_stability(active_channels: u8, with_tag: bool, timer: &mut GlobalTimer) -> Result<(), Error> {
+    // let active_channels = active_channels as usize;
+    for ch in 0..active_channels {
+        test_counter_reset(ch, with_tag)?;
+    }
+
+    // grabber -> camera connection test
+    for ch in 0..active_channels {
+        for _ in 0..TX_TEST_CNT {
+            send_test_packet(ch)?;
+            // sending the whole test sequence @ 20.833Mbps will take a minimum of 1.972ms
+            // and leave some room to send IDLE word
+            timer.delay_ms(2);
+        }
+    }
+
+    // camera -> grabber connection test
+    // enabling the TESTMODE on master channel will send test packets on all channels
+    // and ctrl packet write overhead is used as a delay
+    write_u32(MASTER_CHANNEL, TESTMODE, 1, with_tag)?;
+    write_u32(MASTER_CHANNEL, TESTMODE, 0, with_tag)?;
+
+    for ch in 0..active_channels {
+        verify_test_result(ch, with_tag)?;
+    }
+
+    Ok(())
+}
+
+fn camera_setup(timer: &mut GlobalTimer) -> Result<bool, Error> {
+    reset_tag();
+    discover_camera(timer)?;
+    check_master_channel()?;
+
+    let active_channels = negotiate_active_channels(timer)?;
+    set_host_connection_id()?;
+    let cxp_v2_or_greater = negotiate_cxp_version()?;
+
+    negotiate_pak_max_size(cxp_v2_or_greater)?;
+    set_operation_linerate(active_channels, cxp_v2_or_greater, timer)?;
+
+    // DEBUG: print
+    check_magic_word()?;
+
+    test_channels_stability(active_channels, cxp_v2_or_greater, timer)?;
+
+    Ok(cxp_v2_or_greater)
+}

src/libboard_artiq/src/cxp_ctrl.rs

@@ -0,0 +1,260 @@
+use core::slice;
+
+use byteorder::{ByteOrder, NetworkEndian};
+use io::Cursor;
+use libboard_zynq::{println, time::Milliseconds, timer::GlobalTimer};
+
+use crate::{cxp_proto::{print_packet, Error, RXPacket, TXPacket, CTRL_PACKET_MAXSIZE, DATA_MAXSIZE},
+            mem::mem::CXP_MEM,
+            pl::csr::CXP};
+
+const TRANSMISSION_TIMEOUT: u64 = 200;
+
+fn packet_pending(channel: u8) -> bool {
+    unsafe { (CXP[channel as usize].rx_pending_packet_read)() == 1 }
+}
+
+fn receive(channel: u8) -> Result<Option<RXPacket>, Error> {
+    if packet_pending(channel) {
+        let channel = channel as usize;
+        unsafe {
+            let read_buffer_ptr = (CXP[channel].rx_read_ptr_read)() as usize;
+            println!("buffer ptr = {}", read_buffer_ptr);
+            let ptr =
+                (CXP_MEM[channel].base + CXP_MEM[channel].size / 2 + read_buffer_ptr * CTRL_PACKET_MAXSIZE) as *mut u32;
+
+            let mut reader = Cursor::new(slice::from_raw_parts_mut(ptr as *mut u8, CTRL_PACKET_MAXSIZE));
+            // DEBUG:
+            // println!("RX MEM before reading");
+            // print_packet(&reader.get_ref()[0..40]);
+
+            let packet = RXPacket::read_from(&mut reader);
+            // DEBUG:
+            println!("{:X?}", packet);
+
+            (CXP[channel].rx_pending_packet_write)(1);
+            Ok(Some(packet?))
+        }
+    } else {
+        Ok(None)
+    }
+}
+
+fn receive_timeout(channel: u8, timeout_ms: u64) -> Result<RXPacket, Error> {
+    // assume timer was initialized successfully
+    let timer = unsafe { GlobalTimer::get() };
+    let limit = timer.get_time() + Milliseconds(timeout_ms);
+    while timer.get_time() < limit {
+        match receive(channel)? {
+            None => (),
+            Some(packet) => return Ok(packet),
+        }
+    }
+    Err(Error::TimedOut)
+}
+
+fn send_data_packet(channel: u8, packet: &TXPacket) -> Result<(), Error> {
+    // assume tx is enabled
+    let channel = channel as usize;
+    unsafe {
+        while (CXP[channel].tx_writer_busy_read)() == 1 {}
+        let ptr = CXP_MEM[channel].base as *mut u32;
+        let mut writer = Cursor::new(slice::from_raw_parts_mut(ptr as *mut u8, CTRL_PACKET_MAXSIZE));
+
+        packet.write_to(&mut writer)?;
+        // DEBUG:
+        // println!("TX MEM after writing");
+        // print_packet(&writer.get_ref()[0..40]);
+
+        (CXP[channel].tx_writer_word_len_write)((writer.position() / 4) as u8);
+        (CXP[channel].tx_writer_stb_write)(1);
+    }
+
+    Ok(())
+}
+
+pub fn send_test_packet(channel: u8) -> Result<(), Error> {
+    // assume tx is enabled
+    let channel = channel as usize;
+    unsafe {
+        while (CXP[channel].tx_writer_busy_read)() == 1 {}
+        (CXP[channel].tx_writer_stb_testseq_write)(1);
+    }
+    Ok(())
+}
+
+//
+//
+// DEBUG:
+//
+//
+pub fn rx_debug_mem_print(channel: u8) {
+    unsafe {
+        let ptr = CXP_MEM[channel as usize].base as *mut u32;
+        let arr = slice::from_raw_parts_mut(ptr as *mut u8, CTRL_PACKET_MAXSIZE * 4);
+        print_packet(arr);
+    }
+}
+
+pub fn print_decode_error(channel: u8) {
+    unsafe {
+        println!(
+            "CH#{} Decode error = {}",
+            channel,
+            (CXP[channel as usize].rx_reader_decode_err_read)()
+        );
+    }
+}
+
+//
+//
+// CTRL packet
+//
+//
+
+// Section 9.6.1.2 (CXP-001-2021)
+// CTRL packet need to be tagged for CXP 2.0 or greater
+static mut TAG: u8 = 0;
+
+pub fn reset_tag() {
+    unsafe { TAG = 0 }
+}
+
+fn increment_tag() {
+    unsafe { TAG = TAG.wrapping_add(1) };
+}
+
+fn check_tag(tag: Option<u8>) -> Result<(), Error> {
+    unsafe {
+        if tag.is_some() && tag != Some(TAG) {
+            Err(Error::TagMismatch)
+        } else {
+            Ok(())
+        }
+    }
+}
+
+fn check_length(length: u32) -> Result<(), Error> {
+    if length > DATA_MAXSIZE as u32 || length == 0 {
+        Err(Error::LengthOutOfRange)
+    } else {
+        Ok(())
+    }
+}
+
+fn process_ack_packet(channel: u8, timeout: u64) -> Result<(), Error> {
+    match receive_timeout(channel, timeout) {
+        Ok(RXPacket::CtrlAck { tag }) => {
+            check_tag(tag)?;
+            Ok(())
+        }
+        Ok(RXPacket::CtrlDelay { tag, time }) => {
+            check_tag(tag)?;
+
+            // info!("delaying by {} ms ....", time);
+            process_ack_packet(channel, time as u64)
+        }
+        Ok(_) => Err(Error::UnexpectedReply),
+        Err(e) => Err(e),
+    }
+}
+
+fn process_reply_packet(channel: u8, timeout: u64, expected_length: u32) -> Result<[u8; DATA_MAXSIZE], Error> {
+    match receive_timeout(channel, timeout) {
+        Ok(RXPacket::CtrlReply { tag, length, data }) => {
+            check_tag(tag)?;
+
+            if length != expected_length {
+                return Err(Error::UnexpectedReply);
+            };
+
+            Ok(data)
+        }
+        Ok(RXPacket::CtrlDelay { tag, time }) => {
+            check_tag(tag)?;
+
+            // info!("delaying by {} ms ....", time);
+            process_reply_packet(channel, time as u64, expected_length)
+        }
+        Ok(_) => Err(Error::UnexpectedReply),
+        Err(e) => Err(e),
+    }
+}
+
+pub fn write_bytes_no_ack(channel: u8, addr: u32, val: &[u8], with_tag: bool) -> Result<(), Error> {
+    let length = val.len() as u32;
+    check_length(length)?;
+
+    let mut data: [u8; DATA_MAXSIZE] = [0; DATA_MAXSIZE];
+    data[..length as usize].clone_from_slice(val);
+
+    let tag: Option<u8> = if with_tag { Some(unsafe { TAG }) } else { None };
+    send_data_packet(
+        channel,
+        &TXPacket::CtrlWrite {
+            tag,
+            addr,
+            length,
+            data,
+        },
+    )
+}
+
+pub fn write_bytes(channel: u8, addr: u32, val: &[u8], with_tag: bool) -> Result<(), Error> {
+    write_bytes_no_ack(channel, addr, val, with_tag)?;
+    process_ack_packet(channel, TRANSMISSION_TIMEOUT)?;
+
+    // DEBUG
+    print_decode_error(channel);
+
+    if with_tag {
+        increment_tag();
+    };
+    Ok(())
+}
+
+pub fn write_u32(channel: u8, addr: u32, val: u32, with_tag: bool) -> Result<(), Error> {
+    write_bytes(channel, addr, &val.to_be_bytes(), with_tag)
+}
+
+pub fn write_u64(channel: u8, addr: u32, val: u64, with_tag: bool) -> Result<(), Error> {
+    write_bytes(channel, addr, &val.to_be_bytes(), with_tag)
+}
+
+fn read(channel: u8, addr: u32, length: u32, with_tag: bool) -> Result<(), Error> {
+    check_length(length)?;
+    let tag: Option<u8> = if with_tag { Some(unsafe { TAG }) } else { None };
+    send_data_packet(channel, &TXPacket::CtrlRead { tag, addr, length })
+}
+
+pub fn read_bytes(channel: u8, addr: u32, bytes: &mut [u8], with_tag: bool) -> Result<(), Error> {
+    let length = bytes.len() as u32;
+    read(channel, addr, length, with_tag)?;
+
+    let data = process_reply_packet(channel, TRANSMISSION_TIMEOUT, length)?;
+    bytes.clone_from_slice(&data[..length as usize]);
+
+    // DEBUG
+    print_decode_error(channel);
+
+    if with_tag {
+        increment_tag();
+    };
+    Ok(())
+}
+
+pub fn read_u32(channel: u8, addr: u32, with_tag: bool) -> Result<u32, Error> {
+    let mut bytes: [u8; 4] = [0; 4];
+    read_bytes(channel, addr, &mut bytes, with_tag)?;
+    let val = NetworkEndian::read_u32(&bytes);
+
+    Ok(val)
+}
+
+pub fn read_u64(channel: u8, addr: u32, with_tag: bool) -> Result<u64, Error> {
+    let mut bytes: [u8; 8] = [0; 8];
+    read_bytes(channel, addr, &mut bytes, with_tag)?;
+    let val = NetworkEndian::read_u64(&bytes);
+
+    Ok(val)
+}

src/libboard_artiq/src/cxp_phys.rs

@@ -0,0 +1,197 @@
+use log::info;
+
+use crate::pl::csr;
+
+#[derive(Clone, Copy, Debug)]
+#[allow(non_camel_case_types)]
+pub enum CXP_SPEED {
+    CXP_1,
+    CXP_2,
+    CXP_3,
+    CXP_5,
+    CXP_6,
+    CXP_10,
+    CXP_12,
+}
+
+pub fn setup() {
+    rx::setup();
+    tx::setup();
+    change_linerate(CXP_SPEED::CXP_1);
+}
+
+pub fn change_linerate(speed: CXP_SPEED) {
+    info!("Changing all channels datarate to {:?}", speed);
+    rx::change_linerate(speed);
+    tx::change_linerate(speed);
+}
+
+mod tx {
+    use super::*;
+
+    pub fn setup() {
+        unsafe {
+            csr::cxp_phys::tx_enable_write(1);
+        }
+    }
+
+    pub fn change_linerate(speed: CXP_SPEED) {
+        unsafe {
+            match speed {
+                CXP_SPEED::CXP_1 | CXP_SPEED::CXP_2 | CXP_SPEED::CXP_3 | CXP_SPEED::CXP_5 | CXP_SPEED::CXP_6 => {
+                    csr::cxp_phys::tx_bitrate2x_enable_write(0);
+                }
+                CXP_SPEED::CXP_10 | CXP_SPEED::CXP_12 => {
+                    csr::cxp_phys::tx_bitrate2x_enable_write(1);
+                }
+            };
+            csr::cxp_phys::tx_clk_reset_write(1);
+        }
+    }
+}
+
+mod rx {
+    use super::*;
+
+    pub fn setup() {
+        unsafe {
+            csr::cxp_phys::rx_qpll_reset_write(1);
+            info!("waiting for QPLL/CPLL to lock...");
+            while csr::cxp_phys::rx_qpll_locked_read() != 1 {}
+            info!("QPLL locked");
+            csr::cxp_phys::rx_gtx_start_init_write(1);
+        }
+    }
+
+    pub fn change_linerate(speed: CXP_SPEED) {
+        change_qpll_fb_divider(speed);
+        change_gtx_divider(speed);
+        change_cdr_cfg(speed);
+
+        unsafe {
+            csr::cxp_phys::rx_qpll_reset_write(1);
+            info!("waiting for QPLL/CPLL to lock...");
+            while csr::cxp_phys::rx_qpll_locked_read() != 1 {}
+            info!("QPLL locked");
+
+            csr::cxp_phys::rx_gtx_restart_write(1);
+        }
+    }
+
+    fn change_qpll_fb_divider(speed: CXP_SPEED) {
+        let qpll_div_reg = match speed {
+            CXP_SPEED::CXP_1 | CXP_SPEED::CXP_2 | CXP_SPEED::CXP_5 | CXP_SPEED::CXP_10 => 0x0120, // FB_Divider = 80, QPLL VCO @ 10GHz
+            CXP_SPEED::CXP_3 | CXP_SPEED::CXP_6 | CXP_SPEED::CXP_12 => 0x0170, // FB_Divider = 100, QPLL VCO @ 12.5GHz
+        };
+
+        // DEBUG:
+        // println!("QPLL DRP:");
+        // println!("0x36 = {:#06x}", qpll_read(0x36));
+        qpll_write(0x36, qpll_div_reg);
+        // println!("0x36 = {:#06x}", qpll_read(0x36));
+    }
+
+    fn change_gtx_divider(speed: CXP_SPEED) {
+        let div_reg = match speed {
+            CXP_SPEED::CXP_1 => 0x33,                      // RXOUT_DIV = 8
+            CXP_SPEED::CXP_2 | CXP_SPEED::CXP_3 => 0x22,   // RXOUT_DIV = 4
+            CXP_SPEED::CXP_5 | CXP_SPEED::CXP_6 => 0x11,   // RXOUT_DIV = 2
+            CXP_SPEED::CXP_10 | CXP_SPEED::CXP_12 => 0x00, // RXOUT_DIV = 1
+        };
+
+        // DEBUG:
+        // println!("RX GTX DRP:");
+        // println!("channel {}, 0x88 = {:#06x}", channel, gtx_read(channel, 0x88));
+        gtx_write(0x88, div_reg);
+        // println!("channel {}, 0x88 = {:#06x}", channel, gtx_read(channel, 0x88));
+    }
+
+    fn change_cdr_cfg(speed: CXP_SPEED) {
+        struct CdrConfig {
+            pub cfg_reg0: u16, // addr = 0xA8
+            pub cfg_reg1: u16, // addr = 0xA9
+            pub cfg_reg2: u16, // addr = 0xAA
+            pub cfg_reg3: u16, // addr = 0xAB
+            pub cfg_reg4: u16, // addr = 0xAC
+        }
+
+        let cdr_cfg = match speed {
+            // when RXOUT_DIV = 8
+            CXP_SPEED::CXP_1 => CdrConfig {
+                cfg_reg0: 0x0020,
+                cfg_reg1: 0x1008,
+                cfg_reg2: 0x23FF,
+                cfg_reg3: 0x0000,
+                cfg_reg4: 0x0003,
+            },
+            // when RXOUT_DIV = 4
+            CXP_SPEED::CXP_2 | CXP_SPEED::CXP_5 => CdrConfig {
+                cfg_reg0: 0x0020,
+                cfg_reg1: 0x1010,
+                cfg_reg2: 0x23FF,
+                cfg_reg3: 0x0000,
+                cfg_reg4: 0x0003,
+            },
+            // when RXOUT_DIV= 2
+            CXP_SPEED::CXP_3 | CXP_SPEED::CXP_6 => CdrConfig {
+                cfg_reg0: 0x0020,
+                cfg_reg1: 0x1020,
+                cfg_reg2: 0x23FF,
+                cfg_reg3: 0x0000,
+                cfg_reg4: 0x0003,
+            },
+            // when RXOUT_DIV= 1
+            CXP_SPEED::CXP_10 | CXP_SPEED::CXP_12 => CdrConfig {
+                cfg_reg0: 0x0020,
+                cfg_reg1: 0x1040,
+                cfg_reg2: 0x23FF,
+                cfg_reg3: 0x0000,
+                cfg_reg4: 0x000B,
+            },
+        };
+
+        gtx_write(0x0A8, cdr_cfg.cfg_reg0);
+        gtx_write(0x0A9, cdr_cfg.cfg_reg1);
+        gtx_write(0x0AA, cdr_cfg.cfg_reg2);
+        gtx_write(0x0AB, cdr_cfg.cfg_reg3);
+        gtx_write(0x0AC, cdr_cfg.cfg_reg4);
+    }
+
+    #[allow(dead_code)]
+    fn gtx_read(address: u16) -> u16 {
+        unsafe {
+            csr::cxp_phys::rx_gtx_daddr_write(address);
+            csr::cxp_phys::rx_gtx_dread_write(1);
+            while csr::cxp_phys::rx_gtx_dready_read() != 1 {}
+            csr::cxp_phys::rx_gtx_dout_read()
+        }
+    }
+
+    fn gtx_write(address: u16, value: u16) {
+        unsafe {
+            csr::cxp_phys::rx_gtx_daddr_write(address);
+            csr::cxp_phys::rx_gtx_din_write(value);
+            csr::cxp_phys::rx_gtx_din_stb_write(1);
+            while csr::cxp_phys::rx_gtx_dready_read() != 1 {}
+        }
+    }
+
+    #[allow(dead_code)]
+    fn qpll_read(address: u8) -> u16 {
+        unsafe {
+            csr::cxp_phys::rx_qpll_daddr_write(address);
+            csr::cxp_phys::rx_qpll_dread_write(1);
+            while csr::cxp_phys::rx_qpll_dready_read() != 1 {}
+            csr::cxp_phys::rx_qpll_dout_read()
+        }
+    }
+
+    fn qpll_write(address: u8, value: u16) {
+        unsafe {
+            csr::cxp_phys::rx_qpll_daddr_write(address);
+            csr::cxp_phys::rx_qpll_din_write(value);
+            csr::cxp_phys::rx_qpll_din_stb_write(1);
+            while csr::cxp_phys::rx_qpll_dready_read() != 1 {}
+        }
+    }
+}

src/libboard_artiq/src/cxp_proto.rs

@@ -0,0 +1,424 @@
+use core::fmt;
+
+use byteorder::{ByteOrder, NetworkEndian};
+use core_io::{Error as IoError, Read, Write};
+use crc::crc32::checksum_ieee;
+use io::Cursor;
+use libboard_zynq::println;
+
+pub const CTRL_PACKET_MAXSIZE: usize = 128; // for compatibility with version1.x compliant Devices - Section 12.1.6 (CXP-001-2021)
+pub const DATA_MAXSIZE: usize =
+    CTRL_PACKET_MAXSIZE - /*packet start KCodes, data packet types, CMD, Tag, Addr, CRC, packet end KCode*/4*7;
+
+#[derive(Debug)]
+pub enum Error {
+    CorruptedPacket,
+    CtrlAckError(u8),
+    Io(IoError),
+    LengthOutOfRange,
+    TagMismatch,
+    TimedOut,
+    UnexpectedReply,
+    UnknownPacket(u8),
+}
+
+impl fmt::Display for Error {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        match self {
+            &Error::CorruptedPacket => write!(f, "CorruptedPacket Received packet fail CRC test"),
+            &Error::CtrlAckError(ref ack_code) => match ack_code {
+                0x40 => write!(f, "CtrlAckError Invalid Address"),
+                0x41 => write!(f, "CtrlAckError Invalid data for the address"),
+                0x42 => write!(f, "CtrlAckError Invalid operation code"),
+                0x43 => write!(f, "CtrlAckError Write attempted to a read-only address"),
+                0x44 => write!(f, "CtrlAckError Read attempted from a write-only address"),
+                0x45 => write!(f, "CtrlAckError Size field too large, exceed packet size limit"),
+                0x46 => write!(f, "CtrlAckError Message size is inconsistent with size field"),
+                0x47 => write!(f, "CtrlAckError Malformed packet"),
+                0x80 => write!(f, "CtrlAckError Failed CRC test in last received command"),
+                _ => write!(f, "CtrlAckError Unknown ack code {:#X}", ack_code),
+            },
+            &Error::Io(ref err) => write!(f, "IoError {:?}", err),
+            &Error::LengthOutOfRange => write!(f, "LengthOutOfRange Message Length is too long"),
+            &Error::TagMismatch => write!(f, "TagMismatch Received tag is different from the transmitted tag"),
+            &Error::TimedOut => write!(f, "MessageTimedOut"),
+            &Error::UnexpectedReply => write!(f, "UnexpectedReply"),
+            &Error::UnknownPacket(packet_type) => write!(f, "UnknownPacket with type id {:#X} ", packet_type),
+        }
+    }
+}
+
+impl From<IoError> for Error {
+    fn from(value: IoError) -> Error {
+        Error::Io(value)
+    }
+}
+
+fn get_cxp_crc(bytes: &[u8]) -> u32 {
+    // Section 9.2.2.2 (CXP-001-2021)
+    // Only Control packet need CRC32 appended in the end of the packet
+    // CoaXpress use the polynomial of IEEE-802.3 (Ethernet) CRC but the checksum calculation is different
+    (!checksum_ieee(bytes)).swap_bytes()
+}
+
+trait CxpRead {
+    fn read_u8(&mut self) -> Result<u8, Error>;
+
+    fn read_u16(&mut self) -> Result<u16, Error>;
+
+    fn read_u32(&mut self) -> Result<u32, Error>;
+
+    fn read_u64(&mut self) -> Result<u64, Error>;
+
+    fn read_exact_4x(&mut self, buf: &mut [u8]) -> Result<(), Error>;
+
+    fn read_4x_u8(&mut self) -> Result<u8, Error>;
+
+    fn read_4x_u16(&mut self) -> Result<u16, Error>;
+
+    fn read_4x_u32(&mut self) -> Result<u32, Error>;
+}
+impl<Cursor: Read> CxpRead for Cursor {
+    fn read_u8(&mut self) -> Result<u8, Error> {
+        let mut bytes = [0; 1];
+        self.read_exact(&mut bytes)?;
+        Ok(bytes[0])
+    }
+
+    fn read_u16(&mut self) -> Result<u16, Error> {
+        let mut bytes = [0; 2];
+        self.read_exact(&mut bytes)?;
+        Ok(NetworkEndian::read_u16(&bytes))
+    }
+
+    fn read_u32(&mut self) -> Result<u32, Error> {
+        let mut bytes = [0; 4];
+        self.read_exact(&mut bytes)?;
+        Ok(NetworkEndian::read_u32(&bytes))
+    }
+
+    fn read_u64(&mut self) -> Result<u64, Error> {
+        let mut bytes = [0; 8];
+        self.read_exact(&mut bytes)?;
+        Ok(NetworkEndian::read_u64(&bytes))
+    }
+
+    fn read_exact_4x(&mut self, buf: &mut [u8]) -> Result<(), Error> {
+        for byte in buf {
+            // Section 9.2.2.1 (CXP-001-2021)
+            // decoder should immune to single bit errors when handling 4x duplicated characters
+            let a = self.read_u8()?;
+            let b = self.read_u8()?;
+            let c = self.read_u8()?;
+            let d = self.read_u8()?;
+            // vote and return majority
+            *byte = a & b & c | a & b & d | a & c & d | b & c & d;
+        }
+        Ok(())
+    }
+
+    fn read_4x_u8(&mut self) -> Result<u8, Error> {
+        let mut bytes = [0; 1];
+        self.read_exact_4x(&mut bytes)?;
+        Ok(bytes[0])
+    }
+
+    fn read_4x_u16(&mut self) -> Result<u16, Error> {
+        let mut bytes = [0; 2];
+        self.read_exact_4x(&mut bytes)?;
+        Ok(NetworkEndian::read_u16(&bytes))
+    }
+
+    fn read_4x_u32(&mut self) -> Result<u32, Error> {
+        let mut bytes = [0; 4];
+        self.read_exact_4x(&mut bytes)?;
+        Ok(NetworkEndian::read_u32(&bytes))
+    }
+}
+
+#[derive(Debug)]
+pub enum NameSpace {
+    GenICam,
+    DeviceSpecific,
+}
+
+#[derive(Debug)]
+pub enum RXPacket {
+    CtrlReply {
+        tag: Option<u8>,
+        length: u32,
+        data: [u8; DATA_MAXSIZE],
+    },
+    CtrlDelay {
+        tag: Option<u8>,
+        time: u32,
+    },
+    CtrlAck {
+        tag: Option<u8>,
+    },
+    Event {
+        conn_id: u32,
+        packet_tag: u8,
+        length: u16,
+        ev_size: u16,
+        namespace: NameSpace,
+        event_id: u16,
+        timestamp: u64,
+        ev: [u8; DATA_MAXSIZE],
+    },
+}
+
+impl RXPacket {
+    pub fn read_from(reader: &mut Cursor<&mut [u8]>) -> Result<Self, Error> {
+        match reader.read_4x_u8()? {
+            0x03 => RXPacket::get_ctrl_packet(reader, false),
+            0x06 => RXPacket::get_ctrl_packet(reader, true),
+            0x07 => RXPacket::get_event_packet(reader),
+            ty => Err(Error::UnknownPacket(ty)),
+        }
+    }
+
+    fn get_ctrl_packet(reader: &mut Cursor<&mut [u8]>, with_tag: bool) -> Result<Self, Error> {
+        let mut tag: Option<u8> = None;
+        if with_tag {
+            tag = Some(reader.read_4x_u8()?);
+        }
+
+        let ackcode = reader.read_4x_u8()?;
+
+        match ackcode {
+            0x00 | 0x04 => {
+                let length = reader.read_u32()?;
+                let mut data: [u8; DATA_MAXSIZE] = [0; DATA_MAXSIZE];
+                reader.read(&mut data[0..length as usize])?;
+
+                // Section 9.6.3 (CXP-001-2021)
+                // only bytes after the first 4 are used in calculating the checksum
+                let checksum = get_cxp_crc(&reader.get_ref()[4..reader.position()]);
+                if reader.read_u32()? != checksum {
+                    return Err(Error::CorruptedPacket);
+                }
+
+                if ackcode == 0x00 {
+                    return Ok(RXPacket::CtrlReply { tag, length, data });
+                } else {
+                    return Ok(RXPacket::CtrlDelay {
+                        tag,
+                        time: NetworkEndian::read_u32(&data[..4]),
+                    });
+                }
+            }
+            0x01 => return Ok(RXPacket::CtrlAck { tag }),
+            _ => return Err(Error::CtrlAckError(ackcode)),
+        }
+    }
+
+    fn get_event_packet(reader: &mut Cursor<&mut [u8]>) -> Result<Self, Error> {
+        let conn_id = reader.read_4x_u32()?;
+        let packet_tag = reader.read_4x_u8()?;
+        let length = reader.read_4x_u16()?;
+
+        let ev_size = reader.read_u16()?;
+        if ev_size + 3 != length {
+            println!("length mismatch");
+            return Err(Error::CorruptedPacket);
+        }
+
+        let mut bytes = [0; 2];
+        reader.read_exact(&mut bytes)?;
+        let namespace_bits = (bytes[0] & 0xC0) >> 6;
+        let namespace = match namespace_bits {
+            0 => NameSpace::GenICam,
+            2 => NameSpace::DeviceSpecific,
+            _ => {
+                println!("namespace = {} error", namespace_bits);
+                return Err(Error::CorruptedPacket);
+            }
+        };
+
+        let event_id = (bytes[0] & 0xF) as u16 | (bytes[1] as u16);
+
+        let timestamp = reader.read_u64()?;
+
+        let mut ev: [u8; DATA_MAXSIZE] = [0; DATA_MAXSIZE];
+        reader.read(&mut ev[0..ev_size as usize])?;
+
+        let checksum = get_cxp_crc(&reader.get_ref()[4..reader.position()]);
+        if reader.read_u32()? != checksum {
+            println!("crc error");
+            return Err(Error::CorruptedPacket);
+        }
+
+        Ok(RXPacket::Event {
+            conn_id,
+            packet_tag,
+            length,
+            ev_size,
+            namespace,
+            event_id,
+            timestamp,
+            ev,
+        })
+    }
+}
+
+trait CxpWrite {
+    fn write_all_4x(&mut self, buf: &[u8]) -> Result<(), Error>;
+
+    fn write_4x_u8(&mut self, value: u8) -> Result<(), Error>;
+
+    fn write_4x_u16(&mut self, value: u16) -> Result<(), Error>;
+
+    fn write_4x_u32(&mut self, value: u32) -> Result<(), Error>;
+
+    fn write_u32(&mut self, value: u32) -> Result<(), Error>;
+}
+impl<Cursor: Write> CxpWrite for Cursor {
+    fn write_all_4x(&mut self, buf: &[u8]) -> Result<(), Error> {
+        for byte in buf {
+            self.write_all(&[*byte; 4])?;
+        }
+        Ok(())
+    }
+
+    fn write_4x_u8(&mut self, value: u8) -> Result<(), Error> {
+        self.write_all_4x(&[value])
+    }
+
+    fn write_4x_u16(&mut self, value: u16) -> Result<(), Error> {
+        let mut bytes = [0; 2];
+        NetworkEndian::write_u16(&mut bytes, value);
+        self.write_all_4x(&bytes)
+    }
+
+    fn write_4x_u32(&mut self, value: u32) -> Result<(), Error> {
+        let mut bytes = [0; 4];
+        NetworkEndian::write_u32(&mut bytes, value);
+        self.write_all_4x(&bytes)
+    }
+
+    fn write_u32(&mut self, value: u32) -> Result<(), Error> {
+        let mut bytes = [0; 4];
+        NetworkEndian::write_u32(&mut bytes, value);
+        self.write_all(&bytes)?;
+        Ok(())
+    }
+}
+
+#[derive(Debug)]
+pub enum TXPacket {
+    CtrlRead {
+        tag: Option<u8>,
+        addr: u32,
+        length: u32,
+    },
+    CtrlWrite {
+        tag: Option<u8>,
+        addr: u32,
+        length: u32,
+        data: [u8; DATA_MAXSIZE],
+    },
+    EventAck {
+        packet_tag: u8,
+    },
+}
+
+impl TXPacket {
+    pub fn write_to(&self, writer: &mut Cursor<&mut [u8]>) -> Result<(), Error> {
+        match *self {
+            TXPacket::CtrlRead { tag, addr, length } => {
+                match tag {
+                    Some(t) => {
+                        writer.write_4x_u8(0x05)?;
+                        writer.write_4x_u8(t)?;
+                    }
+                    None => {
+                        writer.write_4x_u8(0x02)?;
+                    }
+                }
+
+                let mut bytes = [0; 3];
+                NetworkEndian::write_u24(&mut bytes, length);
+                writer.write_all(&[0x00, bytes[0], bytes[1], bytes[2]])?;
+
+                writer.write_u32(addr)?;
+
+                // Section 9.6.2 (CXP-001-2021)
+                // only bytes after the first 4 are used in calculating the checksum
+                let checksum = get_cxp_crc(&writer.get_ref()[4..writer.position()]);
+                writer.write_u32(checksum)?;
+            }
+            TXPacket::CtrlWrite {
+                tag,
+                addr,
+                length,
+                data,
+            } => {
+                match tag {
+                    Some(t) => {
+                        writer.write_4x_u8(0x05)?;
+                        writer.write_4x_u8(t)?;
+                    }
+                    None => {
+                        writer.write_4x_u8(0x02)?;
+                    }
+                }
+
+                let mut bytes = [0; 3];
+                NetworkEndian::write_u24(&mut bytes, length);
+                writer.write_all(&[0x01, bytes[0], bytes[1], bytes[2]])?;
+
+                writer.write_u32(addr)?;
+                writer.write_all(&data[0..length as usize])?;
+
+                // Section 9.6.2 (CXP-001-2021)
+                // only bytes after the first 4 are used in calculating the checksum
+                let checksum = get_cxp_crc(&writer.get_ref()[4..writer.position()]);
+                writer.write_u32(checksum)?;
+            }
+            TXPacket::EventAck { packet_tag } => {
+                writer.write_4x_u8(0x08)?;
+                writer.write_4x_u8(packet_tag)?;
+            }
+        }
+        Ok(())
+    }
+}
+
+// DEBUG: use only
+//
+//
+//
+pub fn print_packet(pak: &[u8]) {
+    println!("pak = [");
+    for i in 0..(pak.len() / 4) {
+        println!(
+            "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X},",
+            i + 1,
+            pak[i * 4],
+            pak[i * 4 + 1],
+            pak[i * 4 + 2],
+            pak[i * 4 + 3]
+        )
+    }
+    println!("]");
+    println!("============================================");
+}
+
+pub fn print_packetu32(pak: &[u32], k: &[u8]) {
+    println!("pak = [");
+    for i in 0..(pak.len()) {
+        let data: [u8; 4] = pak[i].to_le_bytes();
+        println!(
+            "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b},",
+            i + 1,
+            data[0],
+            data[1],
+            data[2],
+            data[3],
+            k[i],
+        )
+    }
+    println!("]");
+    println!("============================================");
+}

src/libboard_artiq/src/drtio_eem.rs

@@ -185,6 +185,24 @@ unsafe fn align_comma(timer: &mut GlobalTimer) {
     }
 }
 
+pub unsafe fn align_wordslip(timer: &mut GlobalTimer, trx_no: u8) -> bool {
+    pl::csr::eem_transceiver::transceiver_sel_write(trx_no);
+
+    for slip in 0..=1 {
+        pl::csr::eem_transceiver::wordslip_write(slip as u8);
+        timer.delay_us(1);
+        pl::csr::eem_transceiver::comma_align_reset_write(1);
+        timer.delay_us(100);
+
+        if pl::csr::eem_transceiver::comma_read() == 1 {
+            debug!("comma alignment completed with {} wordslip", slip);
+            return true;
+        }
+    }
+
+    false
+}
+
 pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
     for trx_no in 0..pl::csr::CONFIG_EEM_DRTIO_COUNT {
         unsafe {
@@ -222,7 +240,6 @@ pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
 
         unsafe {
             align_comma(timer);
-            pl::csr::eem_transceiver::rx_ready_write(1);
         }
     }
 }

src/libboard_artiq/src/drtioaux.rs

@@ -6,7 +6,7 @@ use io::{proto::{ProtoRead, ProtoWrite},
          Cursor};
 use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
 
-pub use crate::drtioaux_proto::Packet;
+pub use crate::drtioaux_proto::{Packet, MAX_PACKET};
 use crate::{drtioaux_proto::Error as ProtocolError, mem::mem::DRTIOAUX_MEM, pl::csr::DRTIOAUX};
 
 #[derive(Debug)]
@@ -35,6 +35,15 @@ impl From<IoError> for Error {
     }
 }
 
+pub fn copy_work_buffer(src: *mut u32, dst: *mut u32, len: isize) {
+    // fix for artiq-zynq#344
+    unsafe {
+        for i in 0..(len / 4) {
+            *dst.offset(i) = *src.offset(i);
+        }
+    }
+}
+
 pub fn reset(linkno: u8) {
     let linkno = linkno as usize;
     unsafe {
@@ -115,7 +124,9 @@ where F: FnOnce(&mut [u8]) -> Result<usize, Error> {
     unsafe {
         while (DRTIOAUX[linkno].aux_tx_read)() != 0 {}
         let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
-        let len = f(slice::from_raw_parts_mut(ptr as *mut u8, 0x400 as usize))?;
+        let mut buf: [u8; MAX_PACKET] = [0; MAX_PACKET];
+        let len = f(&mut buf)?;
+        copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
         (DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
         (DRTIOAUX[linkno].aux_tx_write)(1);
         Ok(())

src/libboard_artiq/src/drtioaux_async.rs

@@ -9,8 +9,8 @@ use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
 use nb;
 use void::Void;
 
-pub use crate::drtioaux_proto::Packet;
-use crate::{drtioaux::{has_rx_error, Error},
+pub use crate::drtioaux_proto::{Packet, MAX_PACKET};
+use crate::{drtioaux::{copy_work_buffer, has_rx_error, Error},
             mem::mem::DRTIOAUX_MEM,
             pl::csr::DRTIOAUX};
 
@@ -102,7 +102,9 @@ where F: FnOnce(&mut [u8]) -> Result<usize, Error> {
     unsafe {
         let _ = block_async!(tx_ready(linkno)).await;
         let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
-        let len = f(slice::from_raw_parts_mut(ptr as *mut u8, 0x400 as usize))?;
+        let mut buf: [u8; MAX_PACKET] = [0; MAX_PACKET];
+        let len = f(&mut buf)?;
+        copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
         (DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
         (DRTIOAUX[linkno].aux_tx_write)(1);
         Ok(())

src/libboard_artiq/src/drtioaux_proto.rs

@@ -1,11 +1,12 @@
 use core_io::{Error as IoError, Read, Write};
 use io::proto::{ProtoRead, ProtoWrite};
 
-const MAX_PACKET: usize = 1024;
+pub const MAX_PACKET: usize = 1024;
 
 // maximum size of arbitrary payloads
 // used by satellite -> master analyzer, subkernel exceptions
-pub const SAT_PAYLOAD_MAX_SIZE: usize  = /*max size*/MAX_PACKET - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
+pub const SAT_PAYLOAD_MAX_SIZE: usize = /*max size*/
+    MAX_PACKET - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
 // used by DDMA, subkernel program data (need to provide extra ID and destination)
 pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*source*/1 - /*destination*/1 - /*ID*/4;
 
@@ -255,6 +256,7 @@ pub enum Packet {
         destination: u8,
         id: u32,
         run: bool,
+        timestamp: u64,
     },
     SubkernelLoadRunReply {
         destination: u8,
@@ -267,12 +269,14 @@ pub enum Packet {
         exception_src: u8,
     },
     SubkernelExceptionRequest {
+        source: u8,
         destination: u8,
     },
     SubkernelException {
+        destination: u8,
         last: bool,
         length: u16,
-        data: [u8; SAT_PAYLOAD_MAX_SIZE],
+        data: [u8; MASTER_PAYLOAD_MAX_SIZE],
     },
     SubkernelMessage {
         source: u8,
@@ -285,6 +289,77 @@ pub enum Packet {
     SubkernelMessageAck {
         destination: u8,
     },
+
+    CoreMgmtGetLogRequest {
+        destination: u8,
+        clear: bool,
+    },
+    CoreMgmtClearLogRequest {
+        destination: u8,
+    },
+    CoreMgmtSetLogLevelRequest {
+        destination: u8,
+        log_level: u8,
+    },
+    CoreMgmtSetUartLogLevelRequest {
+        destination: u8,
+        log_level: u8,
+    },
+    CoreMgmtConfigReadRequest {
+        destination: u8,
+        length: u16,
+        key: [u8; MASTER_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtConfigReadContinue {
+        destination: u8,
+    },
+    CoreMgmtConfigWriteRequest {
+        destination: u8,
+        last: bool,
+        length: u16,
+        data: [u8; MASTER_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtConfigRemoveRequest {
+        destination: u8,
+        length: u16,
+        key: [u8; MASTER_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtConfigEraseRequest {
+        destination: u8,
+    },
+    CoreMgmtRebootRequest {
+        destination: u8,
+    },
+    CoreMgmtAllocatorDebugRequest {
+        destination: u8,
+    },
+    CoreMgmtFlashRequest {
+        destination: u8,
+        payload_length: u32,
+    },
+    CoreMgmtFlashAddDataRequest {
+        destination: u8,
+        last: bool,
+        length: u16,
+        data: [u8; MASTER_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtDropLinkAck {
+        destination: u8,
+    },
+    CoreMgmtDropLink,
+    CoreMgmtGetLogReply {
+        last: bool,
+        length: u16,
+        data: [u8; SAT_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtConfigReadReply {
+        last: bool,
+        length: u16,
+        value: [u8; SAT_PAYLOAD_MAX_SIZE],
+    },
+    CoreMgmtReply {
+        succeeded: bool,
+    },
 }
 
 impl Packet {
@@ -512,6 +587,7 @@ impl Packet {
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 run: reader.read_bool()?,
+                timestamp: reader.read_u64()?,
             },
             0xc5 => Packet::SubkernelLoadRunReply {
                 destination: reader.read_u8()?,
@@ -524,14 +600,17 @@ impl Packet {
                 exception_src: reader.read_u8()?,
             },
             0xc9 => Packet::SubkernelExceptionRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
             },
             0xca => {
+                let destination = reader.read_u8()?;
                 let last = reader.read_bool()?;
                 let length = reader.read_u16()?;
-                let mut data: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
+                let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 reader.read_exact(&mut data[0..length as usize])?;
                 Packet::SubkernelException {
+                    destination: destination,
                     last: last,
                     length: length,
                     data: data,
@@ -558,6 +637,115 @@ impl Packet {
                 destination: reader.read_u8()?,
             },
 
+            0xd0 => Packet::CoreMgmtGetLogRequest {
+                destination: reader.read_u8()?,
+                clear: reader.read_bool()?,
+            },
+            0xd1 => Packet::CoreMgmtClearLogRequest {
+                destination: reader.read_u8()?,
+            },
+            0xd2 => Packet::CoreMgmtSetLogLevelRequest {
+                destination: reader.read_u8()?,
+                log_level: reader.read_u8()?,
+            },
+            0xd3 => Packet::CoreMgmtSetUartLogLevelRequest {
+                destination: reader.read_u8()?,
+                log_level: reader.read_u8()?,
+            },
+            0xd4 => {
+                let destination = reader.read_u8()?;
+                let length = reader.read_u16()?;
+                let mut key: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut key[0..length as usize])?;
+                Packet::CoreMgmtConfigReadRequest {
+                    destination: destination,
+                    length: length,
+                    key: key,
+                }
+            }
+            0xd5 => Packet::CoreMgmtConfigReadContinue {
+                destination: reader.read_u8()?,
+            },
+            0xd6 => {
+                let destination = reader.read_u8()?;
+                let last = reader.read_bool()?;
+                let length = reader.read_u16()?;
+                let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut data[0..length as usize])?;
+                Packet::CoreMgmtConfigWriteRequest {
+                    destination: destination,
+                    last: last,
+                    length: length,
+                    data: data,
+                }
+            }
+            0xd7 => {
+                let destination = reader.read_u8()?;
+                let length = reader.read_u16()?;
+                let mut key: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut key[0..length as usize])?;
+                Packet::CoreMgmtConfigRemoveRequest {
+                    destination: destination,
+                    length: length,
+                    key: key,
+                }
+            }
+            0xd8 => Packet::CoreMgmtConfigEraseRequest {
+                destination: reader.read_u8()?,
+            },
+            0xd9 => Packet::CoreMgmtRebootRequest {
+                destination: reader.read_u8()?,
+            },
+            0xda => Packet::CoreMgmtAllocatorDebugRequest {
+                destination: reader.read_u8()?,
+            },
+            0xdb => Packet::CoreMgmtFlashRequest {
+                destination: reader.read_u8()?,
+                payload_length: reader.read_u32()?,
+            },
+            0xdc => {
+                let destination = reader.read_u8()?;
+                let last = reader.read_bool()?;
+                let length = reader.read_u16()?;
+                let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut data[0..length as usize])?;
+                Packet::CoreMgmtFlashAddDataRequest {
+                    destination: destination,
+                    last: last,
+                    length: length,
+                    data: data,
+                }
+            }
+            0xdd => Packet::CoreMgmtDropLinkAck {
+                destination: reader.read_u8()?,
+            },
+            0xde => Packet::CoreMgmtDropLink,
+            0xdf => {
+                let last = reader.read_bool()?;
+                let length = reader.read_u16()?;
+                let mut data: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut data[0..length as usize])?;
+                Packet::CoreMgmtGetLogReply {
+                    last: last,
+                    length: length,
+                    data: data,
+                }
+            }
+            0xe0 => {
+                let last = reader.read_bool()?;
+                let length = reader.read_u16()?;
+                let mut value: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
+                reader.read_exact(&mut value[0..length as usize])?;
+                Packet::CoreMgmtConfigReadReply {
+                    last: last,
+                    length: length,
+                    value: value,
+                }
+            }
+            0xe1 => Packet::CoreMgmtReply {
+                succeeded: reader.read_bool()?,
+            },
+
             ty => return Err(Error::UnknownPacket(ty)),
         })
     }
@@ -872,12 +1060,14 @@ impl Packet {
                 destination,
                 id,
                 run,
+                timestamp,
             } => {
                 writer.write_u8(0xc4)?;
                 writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_bool(run)?;
+                writer.write_u64(timestamp)?;
             }
             Packet::SubkernelLoadRunReply { destination, succeeded } => {
                 writer.write_u8(0xc5)?;
@@ -896,12 +1086,19 @@ impl Packet {
                 writer.write_bool(with_exception)?;
                 writer.write_u8(exception_src)?;
             }
-            Packet::SubkernelExceptionRequest { destination } => {
+            Packet::SubkernelExceptionRequest { source, destination } => {
                 writer.write_u8(0xc9)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
             }
-            Packet::SubkernelException { last, length, data } => {
+            Packet::SubkernelException {
+                destination,
+                last,
+                length,
+                data,
+            } => {
                 writer.write_u8(0xca)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(last)?;
                 writer.write_u16(length)?;
                 writer.write_all(&data[0..length as usize])?;
@@ -926,6 +1123,115 @@ impl Packet {
                 writer.write_u8(0xcc)?;
                 writer.write_u8(destination)?;
             }
+
+            Packet::CoreMgmtGetLogRequest { destination, clear } => {
+                writer.write_u8(0xd0)?;
+                writer.write_u8(destination)?;
+                writer.write_bool(clear)?;
+            }
+            Packet::CoreMgmtClearLogRequest { destination } => {
+                writer.write_u8(0xd1)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtSetLogLevelRequest { destination, log_level } => {
+                writer.write_u8(0xd2)?;
+                writer.write_u8(destination)?;
+                writer.write_u8(log_level)?;
+            }
+            Packet::CoreMgmtSetUartLogLevelRequest { destination, log_level } => {
+                writer.write_u8(0xd3)?;
+                writer.write_u8(destination)?;
+                writer.write_u8(log_level)?;
+            }
+            Packet::CoreMgmtConfigReadRequest {
+                destination,
+                length,
+                key,
+            } => {
+                writer.write_u8(0xd4)?;
+                writer.write_u8(destination)?;
+                writer.write_u16(length)?;
+                writer.write_all(&key[0..length as usize])?;
+            }
+            Packet::CoreMgmtConfigReadContinue { destination } => {
+                writer.write_u8(0xd5)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtConfigWriteRequest {
+                destination,
+                last,
+                length,
+                data,
+            } => {
+                writer.write_u8(0xd6)?;
+                writer.write_u8(destination)?;
+                writer.write_bool(last)?;
+                writer.write_u16(length)?;
+                writer.write_all(&data[0..length as usize])?;
+            }
+            Packet::CoreMgmtConfigRemoveRequest {
+                destination,
+                length,
+                key,
+            } => {
+                writer.write_u8(0xd7)?;
+                writer.write_u8(destination)?;
+                writer.write_u16(length)?;
+                writer.write_all(&key[0..length as usize])?;
+            }
+            Packet::CoreMgmtConfigEraseRequest { destination } => {
+                writer.write_u8(0xd8)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtRebootRequest { destination } => {
+                writer.write_u8(0xd9)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtAllocatorDebugRequest { destination } => {
+                writer.write_u8(0xda)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtFlashRequest {
+                destination,
+                payload_length,
+            } => {
+                writer.write_u8(0xdb)?;
+                writer.write_u8(destination)?;
+                writer.write_u32(payload_length)?;
+            }
+            Packet::CoreMgmtFlashAddDataRequest {
+                destination,
+                last,
+                length,
+                data,
+            } => {
+                writer.write_u8(0xdc)?;
+                writer.write_u8(destination)?;
+                writer.write_bool(last)?;
+                writer.write_u16(length)?;
+                writer.write_all(&data[..length as usize])?;
+            }
+            Packet::CoreMgmtDropLinkAck { destination } => {
+                writer.write_u8(0xdd)?;
+                writer.write_u8(destination)?;
+            }
+            Packet::CoreMgmtDropLink => writer.write_u8(0xde)?,
+            Packet::CoreMgmtGetLogReply { last, length, data } => {
+                writer.write_u8(0xdf)?;
+                writer.write_bool(last)?;
+                writer.write_u16(length)?;
+                writer.write_all(&data[0..length as usize])?;
+            }
+            Packet::CoreMgmtConfigReadReply { last, length, value } => {
+                writer.write_u8(0xe0)?;
+                writer.write_bool(last)?;
+                writer.write_u16(length)?;
+                writer.write_all(&value[0..length as usize])?;
+            }
+            Packet::CoreMgmtReply { succeeded } => {
+                writer.write_u8(0xe1)?;
+                writer.write_bool(succeeded)?;
+            }
         }
         Ok(())
     }
@@ -943,6 +1249,8 @@ impl Packet {
             Packet::SubkernelLoadRunReply { destination, .. } => Some(*destination),
             Packet::SubkernelMessage { destination, .. } => Some(*destination),
             Packet::SubkernelMessageAck { destination } => Some(*destination),
+            Packet::SubkernelExceptionRequest { destination, .. } => Some(*destination),
+            Packet::SubkernelException { destination, .. } => Some(*destination),
             Packet::DmaPlaybackStatus { destination, .. } => Some(*destination),
             Packet::SubkernelFinished { destination, .. } => Some(*destination),
             _ => None,
@@ -960,7 +1268,9 @@ impl Packet {
             | Packet::SubkernelLoadRunReply { .. }
             | Packet::SubkernelMessageAck { .. }
             | Packet::DmaPlaybackStatus { .. }
-            | Packet::SubkernelFinished { .. } => false,
+            | Packet::SubkernelFinished { .. }
+            | Packet::CoreMgmtDropLinkAck { .. }
+            | Packet::InjectionRequest { .. } => false,
             _ => true,
         }
     }

src/libboard_artiq/src/lib.rs

@@ -3,6 +3,7 @@
 #![feature(naked_functions)]
 #![feature(asm)]
 
+extern crate byteorder;
 extern crate core_io;
 extern crate crc;
 extern crate embedded_hal;
@@ -25,7 +26,7 @@ pub mod fiq;
 #[cfg(feature = "target_kasli_soc")]
 pub mod io_expander;
 pub mod logger;
-#[cfg(has_drtio)]
+#[cfg(any(has_drtio, has_cxp_phys))]
 #[rustfmt::skip]
 #[path = "../../../build/mem.rs"]
 pub mod mem;
@@ -42,6 +43,14 @@ pub mod si5324;
 pub mod si549;
 use core::{cmp, str};
 
+pub mod cxp_proto;
+
+pub mod cxp_phys;
+
+pub mod cxp;
+
+pub mod cxp_ctrl;
+
 pub fn identifier_read(buf: &mut [u8]) -> &str {
     unsafe {
         pl::csr::identifier::address_write(0);

src/libdwarf/src/eh.rs

@@ -85,10 +85,7 @@ unsafe fn get_ttype_entry(
         encoding | DW_EH_PE_pcrel,
         ttype_base,
     )
-    .map(|v| match v {
-        ttype_base => None,
-        ttype_entry => Some(ttype_entry as *const u8),
-    })
+    .map(|v| (v != ttype_base).then(|| v as *const u8))
 }
 
 pub unsafe fn find_eh_action(

src/libio/Cargo.toml.tpl

@@ -8,7 +8,7 @@ name = "io"
 path = "lib.rs"
 
 [dependencies]
-core_io = { version = "0.1", features = ["collections"] }
+core_io = { git = "https://git.m-labs.hk/M-Labs/rs-core_io.git", rev = "e9d3edf027", features = ["collections"] }
 byteorder = { version = "1.0", default-features = false, optional = true }
 
 libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }

src/libio/cursor.rs

@@ -1,5 +1,6 @@
 #[cfg(feature = "alloc")]
 use alloc::vec::Vec;
+use core::arch::asm;
 
 use core_io::{Error as IoError, Read, Write};
 
@@ -47,6 +48,9 @@ impl<T: AsRef<[u8]>> Read for Cursor<T> {
         let len = buf.len().min(data.len());
         // ``copy_from_slice`` generates AXI bursts, use a regular loop instead
         for i in 0..len {
+            unsafe {
+                asm!("", options(preserves_flags, nostack, readonly));
+            }
             buf[i] = data[i];
         }
         self.pos += len;
@@ -59,6 +63,9 @@ impl Write for Cursor<&mut [u8]> {
         let data = &mut self.inner[self.pos..];
         let len = buf.len().min(data.len());
         for i in 0..len {
+            unsafe {
+                asm!("", options(preserves_flags, nostack, readonly));
+            }
             data[i] = buf[i];
         }
         self.pos += len;

src/libio/lib.rs

@@ -1,5 +1,6 @@
 #![no_std]
 #![feature(never_type)]
+#![feature(asm)]
 
 #[cfg(feature = "alloc")]
 extern crate alloc;

src/libksupport/Cargo.toml.tpl

@@ -12,7 +12,7 @@ build_zynq = { path = "../libbuild_zynq" }
 cslice = "0.3"
 log = "0.4"
 nb = "0.1"
-core_io = { version = "0.1", features = ["collections"] }
+core_io = { git = "https://git.m-labs.hk/M-Labs/rs-core_io.git", rev = "e9d3edf027", features = ["collections"] }
 byteorder = { version = "1.3", default-features = false }
 void = { version = "1", default-features = false }
 log_buffer = { version = "1.2" }
@@ -32,3 +32,9 @@ unwind = { path = "../libunwind" }
 libc = { path = "../libc" }
 io = { path = "../libio" }
 libboard_artiq = { path = "../libboard_artiq" }
+
+[dependencies.nalgebra]
+git = "https://git.m-labs.hk/M-Labs/nalgebra.git"
+rev = "ad42410ab0"
+default-features = false
+features = ["libm", "alloc"]

src/libksupport/src/cxp.rs

@@ -0,0 +1,271 @@
+use byteorder::{ByteOrder, NetworkEndian};
+use cslice::CMutSlice;
+use libboard_artiq::{cxp::{setup, MASTER_CHANNEL},
+                     cxp_ctrl,
+                     cxp_proto::DATA_MAXSIZE};
+
+// for downloading the XML files
+// TODO: change this to read bytes and accept TBytearray
+pub extern "C" fn cxp_read_words(addr: i32, val: &mut CMutSlice<i32>, with_tag: bool) {
+    let mut bytes: [u8; DATA_MAXSIZE] = [0; DATA_MAXSIZE];
+    cxp_ctrl::read_bytes(MASTER_CHANNEL, addr as u32, &mut bytes[..val.len() * 4], with_tag)
+        .unwrap_or_else(|e| panic!("CXP readu64 failure: {}", e));
+
+    for i in 0..val.len() {
+        val.as_mut_slice()[i] = NetworkEndian::read_u32(&bytes[i * 4..(i + 1) * 4]) as i32;
+    }
+}
+
+pub extern "C" fn cxp_readu32(addr: i32, with_tag: bool) -> i32 {
+    // TODO: use artiq_raise like i2c?
+    cxp_ctrl::read_u32(MASTER_CHANNEL, addr as u32, with_tag).unwrap_or_else(|e| panic!("CXP readu64 failure: {}", e))
+        as i32
+}
+
+pub extern "C" fn cxp_readu64(addr: i32, with_tag: bool) -> i64 {
+    // TODO: use artiq_raise like i2c?
+    cxp_ctrl::read_u64(MASTER_CHANNEL, addr as u32, with_tag).unwrap_or_else(|e| panic!("CXP read u64 failure: {}", e))
+        as i64
+}
+
+pub extern "C" fn cxp_writeu32(addr: i32, val: i32, with_tag: bool) {
+    // TODO: use artiq_raise like i2c?
+    cxp_ctrl::write_u32(MASTER_CHANNEL, addr as u32, val as u32, with_tag)
+        .unwrap_or_else(|e| panic!("CXP write u32 failure: {}", e));
+}
+
+pub extern "C" fn cxp_writeu64(addr: i32, val: i64, with_tag: bool) {
+    // TODO: use artiq_raise like i2c?
+    cxp_ctrl::write_u64(MASTER_CHANNEL, addr as u32, val as u64, with_tag)
+        .unwrap_or_else(|e| panic!("CXP write u64 failure: {}", e));
+}
+
+pub extern "C" fn cxp_setup() -> bool {
+    setup().unwrap()
+}
+
+// DEBUG: ONLY
+pub extern "C" fn cxp_debug_frame_print() {
+    use libboard_zynq::println;
+
+    use crate::pl::csr::CXP;
+    const LEN: usize = 512;
+    let mut eop: [u8; LEN] = [0; LEN];
+    let mut pak_arr: [u32; LEN] = [0; LEN];
+    let mut k_arr: [u8; LEN] = [0; LEN];
+    let mut err: [u8; LEN] = [0; LEN];
+    let mut i: usize = 0;
+    unsafe {
+        use crate::pl::csr::cxp_frame_pipeline;
+        println!(
+            "arbiter active ch = {:#06b} | CRC error count = {} ",
+            cxp_frame_pipeline::arbiter_active_ch_read(),
+            cxp_frame_pipeline::crc_error_cnt_read(),
+        );
+        println!(
+            "roi counter = {} | roi update = {}",
+            cxp_frame_pipeline::roi_counter_read(),
+            cxp_frame_pipeline::roi_update_read()
+        );
+        if cxp_frame_pipeline::roi_update_read() == 1 {
+            println!("roi update clear");
+            cxp_frame_pipeline::roi_update_write(1);
+        };
+        println!("pixel4x[0] y = {}", cxp_frame_pipeline::pix_y_read(),);
+        println!(
+            "header decode l_size = {} | x_size = {} | y_size = {} | new lines count = {}",
+            cxp_frame_pipeline::header_l_size_read(),
+            cxp_frame_pipeline::header_x_size_read(),
+            cxp_frame_pipeline::header_y_size_read(),
+            cxp_frame_pipeline::header_new_line_read(),
+        );
+
+        if (CXP[0].rx_heartbeat_read)() == 1 {
+            println!(
+                "host id = {:#X} | heartbeat = {:#X}",
+                (CXP[0].rx_host_id_read)(),
+                (CXP[0].rx_device_time_read)(),
+            );
+            (CXP[0].rx_heartbeat_write)(1);
+        };
+
+        // while cxp_frame_pipeline::debug_out_dout_valid_read() == 1 {
+        //     pak_arr[i] = cxp_frame_pipeline::debug_out_dout_pak_read();
+        //     k_arr[i] = cxp_frame_pipeline::debug_out_kout_pak_read();
+        //     eop[i] = cxp_frame_pipeline::debug_out_eop_read();
+        //     err[i] = cxp_frame_pipeline::debug_out_crc_error_read();
+        //     // println!("received {:#04X}", pak_arr[i]);
+        //     cxp_frame_pipeline::debug_out_inc_write(1);
+        //     i += 1;
+        //     if i == LEN {
+        //         break;
+        //     }
+        // }
+        // let channel: usize = 0;
+        // while (CXP[channel].rx_debug_out_dout_valid_read)() == 1 {
+        //     pak_arr[i] = (CXP[channel].rx_debug_out_dout_pak_read)();
+        //     k_arr[i] = (CXP[channel].rx_debug_out_kout_pak_read)();
+        //     // println!("received {:#04X}", pak_arr[i]);
+        //     (CXP[channel].rx_debug_out_inc_write)(1);
+        //     i += 1;
+        //     if i == LEN {
+        //         break;
+        //     }
+        // }
+        // println!(
+        //     "Decode error = {}",
+        //     (CXP[MASTER_CHANNEL as usize].rx_reader_decode_err_read)()
+        // );
+        if (CXP[MASTER_CHANNEL as usize].rx_trigger_ack_read)() == 1 {
+            println!("Trigger ack recv and clear");
+            (CXP[MASTER_CHANNEL as usize].rx_trigger_ack_write)(1);
+        } else {
+            println!("NO Trigger ack");
+        }
+    }
+    cxp_ctrl::print_decode_error(MASTER_CHANNEL);
+    if i > 0 {
+        print_packetu32(&pak_arr, &k_arr, &err, &eop, i as usize);
+    } else {
+        println!("No frame data received");
+    }
+}
+
+pub fn print_packetu32(pak: &[u32], k: &[u8], err: &[u8], eop: &[u8], len: usize) {
+    use libboard_zynq::{print, println};
+    println!("pak = [");
+    let mut i_stream_id: usize = len + 1;
+    let mut i_pak_tag: usize = len + 1;
+    let mut i_size_0: usize = len + 1;
+    let mut i_size_1: usize = len + 1;
+    for i in 0..len {
+        let data: [u8; 4] = pak[i].to_be_bytes();
+        // if eop[i] == 1 {
+        //     if err[i] == 1 {
+        //         println!(
+        //             "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | eop crc error",
+        //             i + 1,
+        //             data[0],
+        //             data[1],
+        //             data[2],
+        //             data[3],
+        //             k[i],
+        //         );
+        //     } else {
+        //         println!(
+        //             "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | eop",
+        //             i + 1,
+        //             data[0],
+        //             data[1],
+        //             data[2],
+        //             data[3],
+        //             k[i],
+        //         );
+        //     }
+        //     i_stream_id = i + 1;
+        //     i_pak_tag = i + 2;
+        //     i_size_0 = i + 3;
+        //     i_size_1 = i + 4
+        // } else if i == i_stream_id {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | stream id",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // } else if i == i_pak_tag {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | packet tag",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // } else if i == i_size_0 {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | DsizeP[15:8]",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // } else if i == i_size_1 {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | DsizeP[7:0]",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // } else {
+        //     if err[i] == 1 {
+        //         println!(
+        //             "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | crc error",
+        //             i + 1,
+        //             data[0],
+        //             data[1],
+        //             data[2],
+        //             data[3],
+        //             k[i],
+        //         );
+        //     } else {
+        //         println!(
+        //             "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} |",
+        //             i + 1,
+        //             data[0],
+        //             data[1],
+        //             data[2],
+        //             data[3],
+        //             k[i],
+        //         );
+        //     }
+        // if err[i] == 1 {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} | crc error",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // } else {
+        //     println!(
+        //         "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} |",
+        //         i + 1,
+        //         data[0],
+        //         data[1],
+        //         data[2],
+        //         data[3],
+        //         k[i],
+        //     );
+        // };
+        print!(
+            "{:#03}     {:#04X} {:#04X} {:#04X} {:#04X} | K {:04b} |",
+            i + 1,
+            data[0],
+            data[1],
+            data[2],
+            data[3],
+            k[i],
+        );
+        if err[i] == 1 {
+            print!(" crc error |")
+        };
+        if eop[i] == 1 {
+            print!(" eop |")
+        };
+        println!("")
+    }
+    println!("]");
+    println!("============================================");
+}

src/libksupport/src/eh_artiq.rs

@@ -14,8 +14,10 @@
 
 use core::mem;
 
-use cslice::CSlice;
+use core_io::Error as ReadError;
+use cslice::{AsCSlice, CSlice};
 use dwarf::eh::{self, EHAction, EHContext};
+use io::{Cursor, ProtoRead};
 use libc::{c_int, c_void, uintptr_t};
 use log::{error, trace};
 use unwind as uw;
@@ -94,29 +96,35 @@ struct ExceptionBuffer {
 }
 
 static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
-    uw_exceptions: [uw::_Unwind_Exception {
-        exception_class: EXCEPTION_CLASS,
-        exception_cleanup: cleanup,
-        private: [0; uw::unwinder_private_data_size],
+    uw_exceptions: [const {
+        uw::_Unwind_Exception {
+            exception_class: EXCEPTION_CLASS,
+            exception_cleanup: cleanup,
+            private: [0; uw::unwinder_private_data_size],
+        }
     }; MAX_INFLIGHT_EXCEPTIONS],
     exceptions: [None; MAX_INFLIGHT_EXCEPTIONS + 1],
     exception_stack: [-1; MAX_INFLIGHT_EXCEPTIONS + 1],
     backtrace: [(0, 0); MAX_BACKTRACE_SIZE],
     backtrace_size: 0,
-    stack_pointers: [StackPointerBacktrace {
-        stack_pointer: 0,
-        initial_backtrace_size: 0,
-        current_backtrace_size: 0,
+    stack_pointers: [const {
+        StackPointerBacktrace {
+            stack_pointer: 0,
+            initial_backtrace_size: 0,
+            current_backtrace_size: 0,
+        }
     }; MAX_INFLIGHT_EXCEPTIONS + 1],
     exception_count: 0,
 };
 
 pub unsafe extern "C" fn reset_exception_buffer() {
     trace!("reset exception buffer");
-    EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception {
-        exception_class: EXCEPTION_CLASS,
-        exception_cleanup: cleanup,
-        private: [0; uw::unwinder_private_data_size],
+    EXCEPTION_BUFFER.uw_exceptions = [const {
+        uw::_Unwind_Exception {
+            exception_class: EXCEPTION_CLASS,
+            exception_cleanup: cleanup,
+            private: [0; uw::unwinder_private_data_size],
+        }
     }; MAX_INFLIGHT_EXCEPTIONS];
     EXCEPTION_BUFFER.exceptions = [None; MAX_INFLIGHT_EXCEPTIONS + 1];
     EXCEPTION_BUFFER.exception_stack = [-1; MAX_INFLIGHT_EXCEPTIONS + 1];
@@ -220,8 +228,6 @@ pub unsafe fn artiq_personality(
 }
 
 pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
-    use cslice::AsCSlice;
-
     let count = EXCEPTION_BUFFER.exception_count;
     let stack = &mut EXCEPTION_BUFFER.exception_stack;
     let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize;
@@ -295,6 +301,60 @@ pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
     unreachable!();
 }
 
+fn read_exception_string<'a>(reader: &mut Cursor<&[u8]>) -> Result<CSlice<'a, u8>, ReadError> {
+    let len = reader.read_u32()? as usize;
+    if len == usize::MAX {
+        let data = reader.read_u32()?;
+        Ok(unsafe { CSlice::new(data as *const u8, len) })
+    } else {
+        let pos = reader.position();
+        let slice = unsafe {
+            let ptr = reader.get_ref().as_ptr().offset(pos as isize);
+            CSlice::new(ptr, len)
+        };
+        reader.set_position(pos + len);
+        Ok(slice)
+    }
+}
+
+fn read_exception(raw_exception: &[u8]) -> Result<Exception, ReadError> {
+    let mut reader = Cursor::new(raw_exception);
+
+    let mut byte = reader.read_u8()?;
+    // to sync
+    while byte != 0x5a {
+        byte = reader.read_u8()?;
+    }
+    // skip sync bytes, 0x09 indicates exception
+    while byte != 0x09 {
+        byte = reader.read_u8()?;
+    }
+    let _len = reader.read_u32()?;
+    // ignore the remaining exceptions, stack traces etc. - unwinding from another device would be unwise anyway
+    Ok(Exception {
+        id: reader.read_u32()?,
+        message: read_exception_string(&mut reader)?,
+        param: [
+            reader.read_u64()? as i64,
+            reader.read_u64()? as i64,
+            reader.read_u64()? as i64,
+        ],
+        file: read_exception_string(&mut reader)?,
+        line: reader.read_u32()?,
+        column: reader.read_u32()?,
+        function: read_exception_string(&mut reader)?,
+    })
+}
+
+pub fn raise_raw(raw_exception: &[u8]) -> ! {
+    use crate::artiq_raise;
+    if let Ok(exception) = read_exception(raw_exception) {
+        unsafe { raise(&exception) };
+    } else {
+        artiq_raise!("SubkernelError", "Error passing exception");
+    }
+}
+
 pub unsafe extern "C" fn resume() -> ! {
     trace!("resume");
     assert!(EXCEPTION_BUFFER.exception_count != 0);
@@ -421,20 +481,30 @@ extern "C" fn stop_fn(
     }
 }
 
-// Must be kept in sync with preallocate_runtime_exception_names() in artiq/language/embedding_map.py
-static EXCEPTION_ID_LOOKUP: [(&str, u32); 12] = [
-    ("RuntimeError", 0),
-    ("RTIOUnderflow", 1),
-    ("RTIOOverflow", 2),
-    ("RTIODestinationUnreachable", 3),
-    ("DMAError", 4),
-    ("I2CError", 5),
-    ("CacheError", 6),
-    ("SPIError", 7),
-    ("ZeroDivisionError", 8),
-    ("IndexError", 9),
-    ("UnwrapNoneError", 10),
-    ("SubkernelError", 11),
+// Must be kept in sync with preallocate_runtime_exception_names() in `artiq.compiler.embedding`
+static EXCEPTION_ID_LOOKUP: [(&str, u32); 22] = [
+    ("RTIOUnderflow", 0),
+    ("RTIOOverflow", 1),
+    ("RTIODestinationUnreachable", 2),
+    ("DMAError", 3),
+    ("I2CError", 4),
+    ("CacheError", 5),
+    ("SPIError", 6),
+    ("SubkernelError", 7),
+    ("AssertionError", 8),
+    ("AttributeError", 9),
+    ("IndexError", 10),
+    ("IOError", 11),
+    ("KeyError", 12),
+    ("NotImplementedError", 13),
+    ("OverflowError", 14),
+    ("RuntimeError", 15),
+    ("TimeoutError", 16),
+    ("TypeError", 17),
+    ("ValueError", 18),
+    ("ZeroDivisionError", 19),
+    ("LinAlgError", 20),
+    ("UnwrapNoneError", 21),
 ];
 
 pub fn get_exception_id(name: &str) -> u32 {
@@ -469,3 +539,29 @@ macro_rules! artiq_raise {
     }};
     ($name:expr, $message:expr) => {{ artiq_raise!($name, $message, 0, 0, 0) }};
 }
+
+/// Takes as input exception id from host
+/// Generates a new exception with:
+///   * `id` set to `exn_id`
+///   * `message` set to corresponding exception name from `EXCEPTION_ID_LOOKUP`
+///
+/// The message is matched on host to ensure correct exception is being referred
+/// This test checks the synchronization of exception ids for runtime errors
+#[no_mangle]
+pub extern "C" fn test_exception_id_sync(exn_id: u32) {
+    let message = EXCEPTION_ID_LOOKUP
+        .iter()
+        .find_map(|&(name, id)| if id == exn_id { Some(name) } else { None })
+        .unwrap_or("unallocated internal exception id");
+
+    let exn = Exception {
+        id: exn_id,
+        file: file!().as_c_slice(),
+        line: 0,
+        column: 0,
+        function: "test_exception_id_sync".as_c_slice(),
+        message: message.as_c_slice(),
+        param: [0, 0, 0],
+    };
+    unsafe { raise(&exn) };
+}

src/libksupport/src/kernel/api.rs

@@ -9,8 +9,10 @@ use log::{info, warn};
 use super::subkernel;
 use super::{cache,
             core1::rtio_get_destination_status,
-            dma,
+            dma, linalg,
             rpc::{rpc_recv, rpc_send, rpc_send_async}};
+#[cfg(has_cxp_phys)]
+use crate::cxp;
 use crate::{eh_artiq, i2c, rtio};
 
 extern "C" {
@@ -126,6 +128,22 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
         #[cfg(has_drtio)]
         api!(subkernel_await_message = subkernel::await_message),
 
+        // CoaXPress
+        #[cfg(has_cxp_phys)]
+        api!(cxp_read_words = cxp::cxp_read_words),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_readu32 = cxp::cxp_readu32),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_readu64 = cxp::cxp_readu64),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_writeu32 = cxp::cxp_writeu32),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_writeu64 = cxp::cxp_writeu64),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_setup = cxp::cxp_setup),
+        #[cfg(has_cxp_phys)]
+        api!(cxp_debug_frame_print = cxp::cxp_debug_frame_print),
+
         // Double-precision floating-point arithmetic helper functions
         // RTABI chapter 4.1.2, Table 2
         api!(__aeabi_dadd),
@@ -303,6 +321,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
         api_libm_f64f64f64!(nextafter),
         api_libm_f64f64f64!(pow),
         api_libm_f64f64!(round),
+        api_libm_f64f64!(rint),
         api_libm_f64f64!(sin),
         api_libm_f64f64!(sinh),
         api_libm_f64f64!(sqrt),
@@ -318,6 +337,26 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
             }
             api!(yn = yn)
         },
+
+        // linalg
+        api!(np_linalg_cholesky = linalg::np_linalg_cholesky),
+        api!(np_linalg_qr = linalg::np_linalg_qr),
+        api!(np_linalg_svd = linalg::np_linalg_svd),
+        api!(np_linalg_inv = linalg::np_linalg_inv),
+        api!(np_linalg_pinv = linalg::np_linalg_pinv),
+        api!(np_linalg_matrix_power = linalg::np_linalg_matrix_power),
+        api!(np_linalg_det = linalg::np_linalg_det),
+        api!(sp_linalg_lu = linalg::sp_linalg_lu),
+        api!(sp_linalg_schur = linalg::sp_linalg_schur),
+        api!(sp_linalg_hessenberg = linalg::sp_linalg_hessenberg),
+
+        /*
+         * syscall for unit tests
+         * Used in `artiq.tests.coredevice.test_exceptions.ExceptionTest.test_raise_exceptions_kernel`
+         * This syscall checks that the exception IDs used in the Python `EmbeddingMap` (in `artiq.language.embedding`)
+         * match the `EXCEPTION_ID_LOOKUP` defined in the firmware (`libksupport::src::eh_artiq`)
+         */
+        api!(test_exception_id_sync = eh_artiq::test_exception_id_sync)
     ];
     api.iter()
         .find(|&&(exported, _)| exported.as_bytes() == required)

src/libksupport/src/kernel/dma.rs

@@ -82,7 +82,7 @@ pub extern "C" fn dma_record_stop(duration: i64, enable_ddma: bool) {
 #[inline(always)]
 unsafe fn dma_record_output_prepare(timestamp: i64, target: i32, words: usize) {
     // See gateware/rtio/dma.py.
-    const HEADER_LENGTH: usize = /*length*/1 + /*channel*/3 + /*timestamp*/8 + /*address*/1;
+    const HEADER_LENGTH: usize = /*length*/ 1 + /*channel*/3 + /*timestamp*/8 + /*address*/1;
     let length = HEADER_LENGTH + /*data*/words * 4;
 
     let buffer = &mut RECORDER.as_mut().unwrap().buffer;

src/libksupport/src/kernel/linalg.rs

@@ -0,0 +1,440 @@
+// Uses `nalgebra` crate to invoke `np_linalg` and `sp_linalg` functions
+// When converting between `nalgebra::Matrix` and `NDArray` following considerations are necessary
+//
+// * Both `nalgebra::Matrix` and `NDArray` require their content to be stored in row-major order
+// * `NDArray` data pointer can be directly read and converted to `nalgebra::Matrix` (row and column number must be known)
+// * `nalgebra::Matrix::as_slice` returns the content of matrix in column-major order and initial data needs to be transposed before storing it in `NDArray` data pointer
+
+use alloc::vec::Vec;
+use core::slice;
+
+use nalgebra::DMatrix;
+
+use crate::artiq_raise;
+
+pub struct InputMatrix {
+    pub ndims: usize,
+    pub dims: *const usize,
+    pub data: *mut f64,
+}
+
+impl InputMatrix {
+    fn get_dims(&mut self) -> Vec<usize> {
+        let dims = unsafe { slice::from_raw_parts(self.dims, self.ndims) };
+        dims.to_vec()
+    }
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_cholesky(mat1: *mut InputMatrix, out: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out = out.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+    if dim1[0] != dim1[1] {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+
+    let outdim = out.get_dims();
+    let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+
+    let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let result = matrix1.cholesky();
+    match result {
+        Some(res) => {
+            out_slice.copy_from_slice(res.unpack().transpose().as_slice());
+        }
+        None => {
+            artiq_raise!("LinAlgError", "Matrix is not positive definite");
+        }
+    };
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_qr(mat1: *mut InputMatrix, out_q: *mut InputMatrix, out_r: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out_q = out_q.as_mut().unwrap();
+    let out_r = out_r.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+    let outq_dim = (*out_q).get_dims();
+    let outr_dim = (*out_r).get_dims();
+
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let out_q_slice = slice::from_raw_parts_mut(out_q.data, outq_dim[0] * outq_dim[1]);
+    let out_r_slice = slice::from_raw_parts_mut(out_r.data, outr_dim[0] * outr_dim[1]);
+
+    // Refer to https://github.com/dimforge/nalgebra/issues/735
+    let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+
+    let res = matrix1.qr();
+    let (q, r) = res.unpack();
+
+    // Uses different algo need to match numpy
+    out_q_slice.copy_from_slice(q.transpose().as_slice());
+    out_r_slice.copy_from_slice(r.transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_svd(
+    mat1: *mut InputMatrix,
+    outu: *mut InputMatrix,
+    outs: *mut InputMatrix,
+    outvh: *mut InputMatrix,
+) {
+    let mat1 = mat1.as_mut().unwrap();
+    let outu = outu.as_mut().unwrap();
+    let outs = outs.as_mut().unwrap();
+    let outvh = outvh.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+    let outu_dim = (*outu).get_dims();
+    let outs_dim = (*outs).get_dims();
+    let outvh_dim = (*outvh).get_dims();
+
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let out_u_slice = slice::from_raw_parts_mut(outu.data, outu_dim[0] * outu_dim[1]);
+    let out_s_slice = slice::from_raw_parts_mut(outs.data, outs_dim[0]);
+    let out_vh_slice = slice::from_raw_parts_mut(outvh.data, outvh_dim[0] * outvh_dim[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let result = matrix.svd(true, true);
+    out_u_slice.copy_from_slice(result.u.unwrap().transpose().as_slice());
+    out_s_slice.copy_from_slice(result.singular_values.as_slice());
+    out_vh_slice.copy_from_slice(result.v_t.unwrap().transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_inv(mat1: *mut InputMatrix, out: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out = out.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+    let dim1 = (*mat1).get_dims();
+
+    if dim1[0] != dim1[1] {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+
+    let outdim = out.get_dims();
+    let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    if !matrix.is_invertible() {
+        artiq_raise!("LinAlgError", "no inverse for Singular Matrix");
+    }
+    let inv = matrix.try_inverse().unwrap();
+    out_slice.copy_from_slice(inv.transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_pinv(mat1: *mut InputMatrix, out: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out = out.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+    let dim1 = (*mat1).get_dims();
+    let outdim = out.get_dims();
+    let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let svd = matrix.svd(true, true);
+    let inv = svd.pseudo_inverse(1e-15);
+
+    match inv {
+        Ok(m) => {
+            out_slice.copy_from_slice(m.transpose().as_slice());
+        }
+        Err(_) => {
+            artiq_raise!("LinAlgError", "SVD computation does not converge");
+        }
+    }
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_matrix_power(mat1: *mut InputMatrix, mat2: *mut InputMatrix, out: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let mat2 = mat2.as_mut().unwrap();
+    let out = out.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+    let power = slice::from_raw_parts_mut(mat2.data, 1);
+    let power = power[0];
+    let outdim = out.get_dims();
+    let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let mut abs_power = power;
+    if abs_power < 0.0 {
+        abs_power = abs_power * -1.0;
+    }
+    let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    if !matrix1.is_square() {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+    let mut result = matrix1.pow(abs_power as u32);
+
+    if power < 0.0 {
+        if !matrix1.is_invertible() {
+            artiq_raise!("LinAlgError", "no inverse for Singular Matrix");
+        }
+        result = result.try_inverse().unwrap();
+    }
+    out_slice.copy_from_slice(result.transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn np_linalg_det(mat1: *mut InputMatrix, out: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out = out.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+    let dim1 = (*mat1).get_dims();
+    let out_slice = slice::from_raw_parts_mut(out.data, 1);
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    if !matrix.is_square() {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+    out_slice[0] = matrix.determinant();
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn sp_linalg_lu(mat1: *mut InputMatrix, out_l: *mut InputMatrix, out_u: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out_l = out_l.as_mut().unwrap();
+    let out_u = out_u.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+    let outl_dim = (*out_l).get_dims();
+    let outu_dim = (*out_u).get_dims();
+
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let out_l_slice = slice::from_raw_parts_mut(out_l.data, outl_dim[0] * outl_dim[1]);
+    let out_u_slice = slice::from_raw_parts_mut(out_u.data, outu_dim[0] * outu_dim[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let (_, l, u) = matrix.lu().unpack();
+
+    out_l_slice.copy_from_slice(l.transpose().as_slice());
+    out_u_slice.copy_from_slice(u.transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn sp_linalg_schur(mat1: *mut InputMatrix, out_t: *mut InputMatrix, out_z: *mut InputMatrix) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out_t = out_t.as_mut().unwrap();
+    let out_z = out_z.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+
+    if dim1[0] != dim1[1] {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+
+    let out_t_dim = (*out_t).get_dims();
+    let out_z_dim = (*out_z).get_dims();
+
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let out_t_slice = slice::from_raw_parts_mut(out_t.data, out_t_dim[0] * out_t_dim[1]);
+    let out_z_slice = slice::from_raw_parts_mut(out_z.data, out_z_dim[0] * out_z_dim[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let (z, t) = matrix.schur().unpack();
+
+    out_t_slice.copy_from_slice(t.transpose().as_slice());
+    out_z_slice.copy_from_slice(z.transpose().as_slice());
+}
+
+/// # Safety
+///
+/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
+#[no_mangle]
+pub unsafe extern "C" fn sp_linalg_hessenberg(
+    mat1: *mut InputMatrix,
+    out_h: *mut InputMatrix,
+    out_q: *mut InputMatrix,
+) {
+    let mat1 = mat1.as_mut().unwrap();
+    let out_h = out_h.as_mut().unwrap();
+    let out_q = out_q.as_mut().unwrap();
+
+    if mat1.ndims != 2 {
+        artiq_raise!(
+            "ValueError",
+            "expected 2D Vector Input, but received {1}D input)",
+            0,
+            mat1.ndims as i64,
+            0
+        );
+    }
+
+    let dim1 = (*mat1).get_dims();
+
+    if dim1[0] != dim1[1] {
+        artiq_raise!(
+            "ValueError",
+            "last 2 dimensions of the array must be square: {1} != {2}",
+            0,
+            dim1[0] as i64,
+            dim1[1] as i64
+        );
+    }
+
+    let out_h_dim = (*out_h).get_dims();
+    let out_q_dim = (*out_q).get_dims();
+
+    let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
+    let out_h_slice = slice::from_raw_parts_mut(out_h.data, out_h_dim[0] * out_h_dim[1]);
+    let out_q_slice = slice::from_raw_parts_mut(out_q.data, out_q_dim[0] * out_q_dim[1]);
+
+    let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
+    let (q, h) = matrix.hessenberg().unpack();
+
+    out_h_slice.copy_from_slice(h.transpose().as_slice());
+    out_q_slice.copy_from_slice(q.transpose().as_slice());
+}

src/libksupport/src/kernel/mod.rs

@@ -13,6 +13,7 @@ mod dma;
 mod rpc;
 pub use dma::DmaRecorder;
 mod cache;
+mod linalg;
 #[cfg(has_drtio)]
 mod subkernel;
 
@@ -23,6 +24,7 @@ pub enum SubkernelStatus {
     Timeout,
     IncorrectState,
     CommLost,
+    Exception(Vec<u8>),
     OtherError,
 }
 
@@ -79,6 +81,7 @@ pub enum Message {
         id: u32,
         destination: u8,
         run: bool,
+        timestamp: u64,
     },
     #[cfg(has_drtio)]
     SubkernelLoadRunReply {
@@ -90,9 +93,7 @@ pub enum Message {
         timeout: i64,
     },
     #[cfg(has_drtio)]
-    SubkernelAwaitFinishReply {
-        status: SubkernelStatus,
-    },
+    SubkernelAwaitFinishReply,
     #[cfg(has_drtio)]
     SubkernelMsgSend {
         id: u32,
@@ -109,9 +110,10 @@ pub enum Message {
     },
     #[cfg(has_drtio)]
     SubkernelMsgRecvReply {
-        status: SubkernelStatus,
         count: u8,
     },
+    #[cfg(has_drtio)]
+    SubkernelError(SubkernelStatus),
 }
 
 static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None);

src/libksupport/src/kernel/subkernel.rs

@@ -3,7 +3,7 @@ use alloc::vec::Vec;
 use cslice::CSlice;
 
 use super::{Message, SubkernelStatus, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0};
-use crate::{artiq_raise, rpc::send_args};
+use crate::{artiq_raise, eh_artiq, rpc::send_args, rtio::now_mu};
 
 pub extern "C" fn load_run(id: u32, destination: u8, run: bool) {
     unsafe {
@@ -14,6 +14,7 @@ pub extern "C" fn load_run(id: u32, destination: u8, run: bool) {
                 id: id,
                 destination: destination,
                 run: run,
+                timestamp: now_mu() as u64,
             });
     }
     match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
@@ -36,21 +37,18 @@ pub extern "C" fn await_finish(id: u32, timeout: i64) {
             });
     }
     match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
-        Message::SubkernelAwaitFinishReply {
-            status: SubkernelStatus::NoError,
-        } => (),
-        Message::SubkernelAwaitFinishReply {
-            status: SubkernelStatus::IncorrectState,
-        } => artiq_raise!("SubkernelError", "Subkernel not running"),
-        Message::SubkernelAwaitFinishReply {
-            status: SubkernelStatus::Timeout,
-        } => artiq_raise!("SubkernelError", "Subkernel timed out"),
-        Message::SubkernelAwaitFinishReply {
-            status: SubkernelStatus::CommLost,
-        } => artiq_raise!("SubkernelError", "Lost communication with satellite"),
-        Message::SubkernelAwaitFinishReply {
-            status: SubkernelStatus::OtherError,
-        } => artiq_raise!("SubkernelError", "An error occurred during subkernel operation"),
+        Message::SubkernelAwaitFinishReply => (),
+        Message::SubkernelError(SubkernelStatus::IncorrectState) => {
+            artiq_raise!("SubkernelError", "Subkernel not running")
+        }
+        Message::SubkernelError(SubkernelStatus::Timeout) => artiq_raise!("SubkernelError", "Subkernel timed out"),
+        Message::SubkernelError(SubkernelStatus::CommLost) => {
+            artiq_raise!("SubkernelError", "Lost communication with satellite")
+        }
+        Message::SubkernelError(SubkernelStatus::OtherError) => {
+            artiq_raise!("SubkernelError", "An error occurred during subkernel operation")
+        }
+        Message::SubkernelError(SubkernelStatus::Exception(raw_exception)) => eh_artiq::raise_raw(&raw_exception),
         _ => panic!("expected SubkernelAwaitFinishReply after SubkernelAwaitFinishRequest"),
     }
 }
@@ -92,30 +90,22 @@ pub extern "C" fn await_message(id: i32, timeout: i64, tags: &CSlice<u8>, min: u
             });
     }
     match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
-        Message::SubkernelMsgRecvReply {
-            status: SubkernelStatus::NoError,
-            count,
-        } => {
+        Message::SubkernelMsgRecvReply { count } => {
             if min > count || count > max {
                 artiq_raise!("SubkernelError", "Received more or less arguments than required")
             }
         }
-        Message::SubkernelMsgRecvReply {
-            status: SubkernelStatus::IncorrectState,
-            ..
-        } => artiq_raise!("SubkernelError", "Subkernel not running"),
-        Message::SubkernelMsgRecvReply {
-            status: SubkernelStatus::Timeout,
-            ..
-        } => artiq_raise!("SubkernelError", "Subkernel timed out"),
-        Message::SubkernelMsgRecvReply {
-            status: SubkernelStatus::CommLost,
-            ..
-        } => artiq_raise!("SubkernelError", "Lost communication with satellite"),
-        Message::SubkernelMsgRecvReply {
-            status: SubkernelStatus::OtherError,
-            ..
-        } => artiq_raise!("SubkernelError", "An error occurred during subkernel operation"),
+        Message::SubkernelError(SubkernelStatus::IncorrectState) => {
+            artiq_raise!("SubkernelError", "Subkernel not running")
+        }
+        Message::SubkernelError(SubkernelStatus::Timeout) => artiq_raise!("SubkernelError", "Subkernel timed out"),
+        Message::SubkernelError(SubkernelStatus::CommLost) => {
+            artiq_raise!("SubkernelError", "Lost communication with satellite")
+        }
+        Message::SubkernelError(SubkernelStatus::OtherError) => {
+            artiq_raise!("SubkernelError", "An error occurred during subkernel operation")
+        }
+        Message::SubkernelError(SubkernelStatus::Exception(raw_exception)) => eh_artiq::raise_raw(&raw_exception),
         _ => panic!("expected SubkernelMsgRecvReply after SubkernelMsgRecvRequest"),
     }
     // RpcRecvRequest should be called after this to receive message data

src/libksupport/src/lib.rs

@@ -1,7 +1,8 @@
 #![no_std]
+#![allow(incomplete_features)]
 #![feature(c_variadic)]
 #![feature(const_btree_new)]
-#![feature(const_in_array_repeat_expressions)]
+#![feature(inline_const)]
 #![feature(naked_functions)]
 #![feature(asm)]
 
@@ -35,6 +36,8 @@ pub mod rtio;
 #[path = "../../../build/pl.rs"]
 pub mod pl;
 
+#[cfg(has_cxp_phys)]
+pub mod cxp;
 
 #[derive(Debug, Clone)]
 pub struct RPCException {

src/libunwind/build.rs

@@ -23,6 +23,7 @@ mod llvm_libunwind {
         cfg.flag("--target=armv7-none-eabihf");
         cfg.flag("-O2");
         cfg.flag("-flto");
+        cfg.flag("-Wno-everything");
 
         cfg.flag("-std=c99");
         cfg.flag("-fstrict-aliasing");

src/libunwind/lib.rs

@@ -1,7 +1,7 @@
 #![no_std]
 #![feature(link_cfg)]
 #![feature(nll)]
-#![feature(unwind_attributes)]
+#![feature(c_unwind)]
 #![feature(static_nobundle)]
 #![cfg_attr(not(target_env = "msvc"), feature(libc))]
 

src/libunwind/libunwind.rs

@@ -77,8 +77,7 @@ pub type _Unwind_Exception_Cleanup_Fn =
     all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
     link(name = "unwind", kind = "static")
 )]
-extern "C" {
-    #[unwind(allowed)]
+extern "C-unwind" {
     pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !;
     pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception);
     pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void;
@@ -226,8 +225,7 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
         pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn,
                                  trace_argument: *mut c_void)
@@ -238,8 +236,7 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
     }
 

src/runtime/Cargo.toml.tpl

@@ -8,6 +8,7 @@ edition = "2018"
 [features]
 target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"]
 target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"]
+target_ebaz4205 = ["libboard_zynq/target_ebaz4205", "libsupport_zynq/target_ebaz4205", "libconfig/target_ebaz4205", "libboard_artiq/target_ebaz4205"]
 default = ["target_zc706"]
 
 [build-dependencies]
@@ -19,7 +20,8 @@ num-derive = "0.3"
 cslice = "0.3"
 log = "0.4"
 embedded-hal = "0.2"
-core_io = { version = "0.1", features = ["collections"] }
+core_io = { git = "https://git.m-labs.hk/M-Labs/rs-core_io.git", rev = "e9d3edf027", features = ["collections"] }
+crc = { version = "1.7", default-features = false }
 byteorder = { version = "1.3", default-features = false }
 void = { version = "1", default-features = false }
 futures = { version = "0.3", default-features = false, features = ["async-await"] }
@@ -44,4 +46,4 @@ libboard_artiq = { path = "../libboard_artiq" }
 
 [dependencies.tar-no-std]
 git = "https://git.m-labs.hk/M-Labs/tar-no-std"
-rev = "2ab6dc5"
+rev = "2ab6dc5"

src/runtime/src/comms.rs

@@ -10,15 +10,11 @@ use io::Cursor;
 #[cfg(has_drtio)]
 use ksupport::rpc;
 use ksupport::{kernel, resolve_channel_name};
-#[cfg(has_drtio)]
-use libasync::delay;
 use libasync::{smoltcp::{Sockets, TcpStream},
                task};
 use libboard_artiq::drtio_routing;
 #[cfg(feature = "target_kasli_soc")]
 use libboard_zynq::error_led::ErrorLED;
-#[cfg(has_drtio)]
-use libboard_zynq::time::Milliseconds;
 use libboard_zynq::{self as zynq,
                     smoltcp::{self,
                               iface::{EthernetInterfaceBuilder, NeighborCache},
@@ -405,8 +401,9 @@ async fn handle_run_kernel(
                 id,
                 destination: _,
                 run,
+                timestamp,
             } => {
-                let succeeded = match subkernel::load(aux_mutex, routing_table, timer, id, run).await {
+                let succeeded = match subkernel::load(aux_mutex, routing_table, timer, id, run, timestamp).await {
                     Ok(()) => true,
                     Err(e) => {
                         error!("Error loading subkernel: {:?}", e);
@@ -422,33 +419,23 @@ async fn handle_run_kernel(
             #[cfg(has_drtio)]
             kernel::Message::SubkernelAwaitFinishRequest { id, timeout } => {
                 let res = subkernel::await_finish(aux_mutex, routing_table, timer, id, timeout).await;
-                let status = match res {
-                    Ok(ref res) => {
+                let response = match res {
+                    Ok(res) => {
                         if res.status == subkernel::FinishStatus::CommLost {
-                            kernel::SubkernelStatus::CommLost
-                        } else if let Some(exception) = &res.exception {
-                            error!("Exception in subkernel");
-                            match stream {
-                                None => (),
-                                Some(stream) => {
-                                    write_chunk(stream, exception).await?;
-                                }
-                            }
-                            // will not be called after exception is served
-                            kernel::SubkernelStatus::OtherError
+                            kernel::Message::SubkernelError(kernel::SubkernelStatus::CommLost)
+                        } else if let Some(exception) = res.exception {
+                            kernel::Message::SubkernelError(kernel::SubkernelStatus::Exception(exception))
                         } else {
-                            kernel::SubkernelStatus::NoError
+                            kernel::Message::SubkernelAwaitFinishReply
                         }
                     }
-                    Err(SubkernelError::Timeout) => kernel::SubkernelStatus::Timeout,
-                    Err(SubkernelError::IncorrectState) => kernel::SubkernelStatus::IncorrectState,
-                    Err(_) => kernel::SubkernelStatus::OtherError,
+                    Err(SubkernelError::Timeout) => kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout),
+                    Err(SubkernelError::IncorrectState) => {
+                        kernel::Message::SubkernelError(kernel::SubkernelStatus::IncorrectState)
+                    }
+                    Err(_) => kernel::Message::SubkernelError(kernel::SubkernelStatus::OtherError),
                 };
-                control
-                    .borrow_mut()
-                    .tx
-                    .async_send(kernel::Message::SubkernelAwaitFinishReply { status: status })
-                    .await;
+                control.borrow_mut().tx.async_send(response).await;
             }
             #[cfg(has_drtio)]
             kernel::Message::SubkernelMsgSend { id, destination, data } => {
@@ -469,35 +456,23 @@ async fn handle_run_kernel(
             #[cfg(has_drtio)]
             kernel::Message::SubkernelMsgRecvRequest { id, timeout, tags } => {
                 let message_received = subkernel::message_await(id as u32, timeout, timer).await;
-                let (status, count) = match message_received {
-                    Ok(ref message) => (kernel::SubkernelStatus::NoError, message.count),
-                    Err(SubkernelError::Timeout) => (kernel::SubkernelStatus::Timeout, 0),
-                    Err(SubkernelError::IncorrectState) => (kernel::SubkernelStatus::IncorrectState, 0),
-                    Err(SubkernelError::CommLost) => (kernel::SubkernelStatus::CommLost, 0),
+                let response = match message_received {
+                    Ok(ref message) => kernel::Message::SubkernelMsgRecvReply { count: message.count },
+                    Err(SubkernelError::Timeout) => kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout),
+                    Err(SubkernelError::IncorrectState) => {
+                        kernel::Message::SubkernelError(kernel::SubkernelStatus::IncorrectState)
+                    }
+                    Err(SubkernelError::CommLost) => kernel::Message::SubkernelError(kernel::SubkernelStatus::CommLost),
                     Err(SubkernelError::SubkernelException) => {
-                        error!("Exception in subkernel");
                         // just retrieve the exception
                         let status = subkernel::await_finish(aux_mutex, routing_table, timer, id as u32, timeout)
                             .await
                             .unwrap();
-                        match stream {
-                            None => (),
-                            Some(stream) => {
-                                write_chunk(stream, &status.exception.unwrap()).await?;
-                            }
-                        }
-                        (kernel::SubkernelStatus::OtherError, 0)
+                        kernel::Message::SubkernelError(kernel::SubkernelStatus::Exception(status.exception.unwrap()))
                     }
-                    Err(_) => (kernel::SubkernelStatus::OtherError, 0),
+                    Err(_) => kernel::Message::SubkernelError(kernel::SubkernelStatus::OtherError),
                 };
-                control
-                    .borrow_mut()
-                    .tx
-                    .async_send(kernel::Message::SubkernelMsgRecvReply {
-                        status: status,
-                        count: count,
-                    })
-                    .await;
+                control.borrow_mut().tx.async_send(response).await;
                 if let Ok(message) = message_received {
                     // receive code almost identical to RPC recv, except we are not reading from a stream
                     let mut reader = Cursor::new(message.data);
@@ -529,7 +504,7 @@ async fn handle_run_kernel(
                             .async_send(kernel::Message::RpcRecvReply(Ok(0)))
                             .await;
                         i += 1;
-                        if i < count {
+                        if i < message.count {
                             current_tags = remaining_tags;
                         } else {
                             break;
@@ -771,14 +746,13 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
     #[cfg(has_drtio_routing)]
     drtio_routing::interconnect_disable_all();
 
-    rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, timer);
+    rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, &cfg, timer);
     ksupport::setup_device_map(&cfg);
 
     analyzer::start(&aux_mutex, &drtio_routing_table, &up_destinations, timer);
     moninj::start(timer, &aux_mutex, &drtio_routing_table);
 
     let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start()));
-    let idle_kernel = Rc::new(cfg.read("idle_kernel").ok());
     if let Ok(buffer) = cfg.read("startup_kernel") {
         info!("Loading startup kernel...");
         let routing_table = drtio_routing_table.borrow();
@@ -805,13 +779,34 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
         }
     }
 
-    mgmt::start(cfg);
+    let cfg = Rc::new(cfg);
+    let restart_idle = Rc::new(Semaphore::new(1, 1));
+    mgmt::start(
+        cfg.clone(),
+        restart_idle.clone(),
+        Some(mgmt::DrtioContext(
+            aux_mutex.clone(),
+            drtio_routing_table.clone(),
+            timer,
+        )),
+    );
 
     task::spawn(async move {
         let connection = Rc::new(Semaphore::new(1, 1));
         let terminate = Rc::new(Semaphore::new(0, 1));
+        let can_restart_idle = Rc::new(Semaphore::new(1, 1));
+        let restart_idle = restart_idle.clone();
         loop {
-            let mut stream = TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap();
+            let control = control.clone();
+            let mut maybe_stream = select_biased! {
+                s = (async {
+                        TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap()
+                    }).fuse() => Some(s),
+                _ = (async {
+                        restart_idle.async_wait().await;
+                        can_restart_idle.async_wait().await;
+                    }).fuse() => None
+            };
 
             if connection.try_wait().is_none() {
                 // there is an existing connection
@@ -819,47 +814,58 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
                 connection.async_wait().await;
             }
 
+            let maybe_idle_kernel = cfg.read("idle_kernel").ok();
+            if maybe_idle_kernel.is_none() && maybe_stream.is_none() {
+                control.borrow_mut().restart(); // terminate idle kernel if running
+            }
+
             let control = control.clone();
-            let idle_kernel = idle_kernel.clone();
             let connection = connection.clone();
             let terminate = terminate.clone();
+            let can_restart_idle = can_restart_idle.clone();
             let up_destinations = up_destinations.clone();
             let aux_mutex = aux_mutex.clone();
             let routing_table = drtio_routing_table.clone();
 
             // we make sure the value of terminate is 0 before we start
             let _ = terminate.try_wait();
+            let _ = can_restart_idle.try_wait();
             task::spawn(async move {
                 let routing_table = routing_table.borrow();
                 select_biased! {
                     _ = (async {
-                        let _ = handle_connection(&mut stream, control.clone(), &up_destinations, &aux_mutex, &routing_table, timer)
-                            .await
-                            .map_err(|e| warn!("connection terminated: {}", e));
-                        if let Some(buffer) = &*idle_kernel {
-                            info!("Loading idle kernel");
-                            let res = handle_flash_kernel(&buffer, &control, &up_destinations,  &aux_mutex, &routing_table, timer)
-                                .await;
-                            match res {
-                                #[cfg(has_drtio)]
-                                Err(Error::DestinationDown) => {
-                                    let mut countdown = timer.countdown();
-                                    delay(&mut countdown, Milliseconds(500)).await;
+                        if let Some(stream) = &mut maybe_stream {
+                            let _ = handle_connection(stream, control.clone(), &up_destinations, &aux_mutex, &routing_table, timer)
+                                .await
+                                .map_err(|e| warn!("connection terminated: {}", e));
+                        }
+                        can_restart_idle.signal();
+                        match maybe_idle_kernel {
+                            Some(buffer) => {
+                                loop {
+                                    info!("loading idle kernel");
+                                    match handle_flash_kernel(&buffer, &control, &up_destinations, &aux_mutex, &routing_table, timer).await {
+                                        Ok(_) => {
+                                            info!("running idle kernel");
+                                            match handle_run_kernel(None, &control, &up_destinations, &aux_mutex, &routing_table, timer).await {
+                                                Ok(_) => info!("idle kernel finished"),
+                                                Err(_) => warn!("idle kernel running error")
+                                            }
+                                        },
+                                        Err(_) => warn!("idle kernel loading error")
+                                    }
                                 }
-                                Err(_) => warn!("error loading idle kernel"),
-                                _ => (),
-                            }
-                            info!("Running idle kernel");
-                            let _ = handle_run_kernel(None, &control, &up_destinations, &aux_mutex, &routing_table, timer)
-                                .await.map_err(|_| warn!("error running idle kernel"));
-                            info!("Idle kernel terminated");
+                            },
+                            None => info!("no idle kernel found")
                         }
                     }).fuse() => (),
                     _ = terminate.async_wait().fuse() => ()
                 }
                 connection.signal();
-                let _ = stream.flush().await;
-                let _ = stream.abort().await;
+                if let Some(stream) = maybe_stream {
+                    let _ = stream.flush().await;
+                    let _ = stream.abort().await;
+                }
             });
         }
     });
@@ -908,7 +914,8 @@ pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
 
     Sockets::init(32);
 
-    mgmt::start(cfg);
+    let dummy = Rc::new(Semaphore::new(0, 1));
+    mgmt::start(Rc::new(cfg), dummy, None);
 
     // getting eth settings disables the LED as it resets GPIO
     // need to re-enable it here

src/runtime/src/main.rs

@@ -13,6 +13,8 @@ use core::cell::RefCell;
 
 use ksupport;
 use libasync::task;
+#[cfg(has_cxp_phys)]
+use libboard_artiq::cxp_phys;
 #[cfg(has_drtio_eem)]
 use libboard_artiq::drtio_eem;
 #[cfg(feature = "target_kasli_soc")]
@@ -150,5 +152,8 @@ pub fn main_core0() {
 
     task::spawn(ksupport::report_async_rtio_errors());
 
+    #[cfg(has_cxp_phys)]
+    cxp_phys::setup();
+
     comms::main(timer, cfg);
 }

src/runtime/src/moninj.rs

@@ -102,7 +102,7 @@ mod remote_moninj {
         overrd: i8,
         value: i8,
     ) {
-        let _lock = aux_mutex.lock();
+        let _lock = aux_mutex.async_lock().await;
         drtioaux_async::send(
             linkno,
             &drtioaux_async::Packet::InjectionRequest {

src/runtime/src/rtio_clocking.rs

@@ -1,6 +1,8 @@
+#[cfg(not(feature = "target_ebaz4205"))]
 use embedded_hal::blocking::delay::DelayMs;
 #[cfg(has_si5324)]
 use ksupport::i2c;
+#[cfg(not(feature = "target_ebaz4205"))]
 use libboard_artiq::pl;
 #[cfg(has_si5324)]
 use libboard_artiq::si5324;
@@ -11,6 +13,8 @@ use libboard_zynq::i2c::I2c;
 use libboard_zynq::timer::GlobalTimer;
 use libconfig::Config;
 use log::{info, warn};
+#[cfg(feature = "target_ebaz4205")]
+use {libboard_zynq::slcr, libregister::RegisterRW};
 
 #[derive(Debug, PartialEq, Copy, Clone)]
 #[allow(non_camel_case_types)]
@@ -69,7 +73,7 @@ fn get_rtio_clock_cfg(cfg: &Config) -> RtioClock {
     res
 }
 
-#[cfg(not(has_drtio))]
+#[cfg(not(any(has_drtio, feature = "target_ebaz4205")))]
 fn init_rtio(timer: &mut GlobalTimer) {
     info!("Switching SYS clocks...");
     unsafe {
@@ -406,6 +410,38 @@ fn get_si549_setting(clk: RtioClock) -> si549::FrequencySetting {
     }
 }
 
+#[cfg(feature = "target_ebaz4205")]
+fn set_fclk0_freq(clk: RtioClock, cfg: &Config) {
+    let io_pll_freq: u32 = 1_000_000_000; // Hardcoded in zynq-rs
+    let mut target_freq = 0;
+    let mut divisor0 = 1u8;
+
+    match clk {
+        RtioClock::Int_100 => {
+            target_freq = 100_000_000;
+            divisor0 = 10;
+        }
+        RtioClock::Int_125 => {
+            target_freq = 125_000_000;
+            divisor0 = 8;
+        }
+        _ => {
+            warn!("Unsupported RTIO Clock: '{:?}'", clk);
+            return;
+        }
+    }
+
+    slcr::RegisterBlock::unlocked(|slcr| {
+        slcr.fpga0_clk_ctrl.modify(|_, w| w.divisor0(divisor0));
+    });
+
+    info!(
+        "Set FCLK0 to {:.2} MHz (target: {} MHz).",
+        io_pll_freq as f64 / divisor0 as f64,
+        target_freq / 1_000_000
+    );
+}
+
 pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
     let clk = get_rtio_clock_cfg(cfg);
     #[cfg(has_si5324)]
@@ -429,9 +465,19 @@ pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
     #[cfg(has_drtio)]
     init_drtio(timer);
 
-    #[cfg(not(has_drtio))]
+    #[cfg(not(any(has_drtio, feature = "target_ebaz4205")))]
     init_rtio(timer);
 
+    #[cfg(feature = "target_ebaz4205")]
+    {
+        match clk {
+            RtioClock::Int_100 | RtioClock::Int_125 => {
+                set_fclk0_freq(clk, cfg);
+            }
+            _ => {} // Not set for external clocks
+        }
+    }
+
     #[cfg(all(has_si549, has_wrpll))]
     {
         // SYS CLK switch will reset CSRs that are used by WRPLL

src/runtime/src/rtio_mgt.rs

@@ -3,7 +3,9 @@ use core::cell::RefCell;
 
 use libboard_artiq::{drtio_routing, drtio_routing::RoutingTable, pl::csr};
 use libboard_zynq::timer::GlobalTimer;
+use libconfig::Config;
 use libcortex_a9::mutex::Mutex;
+use log::{info, warn};
 
 #[cfg(has_drtio)]
 pub mod drtio {
@@ -11,9 +13,13 @@ pub mod drtio {
     use core::fmt;
 
     use embedded_hal::blocking::delay::DelayMs;
+    #[cfg(has_drtio_eem)]
+    use embedded_hal::blocking::delay::DelayUs;
     use ksupport::{resolve_channel_name, ASYNC_ERROR_BUSY, ASYNC_ERROR_COLLISION, ASYNC_ERROR_SEQUENCE_ERROR,
                    SEEN_ASYNC_ERRORS};
     use libasync::{delay, task};
+    #[cfg(has_drtio_eem)]
+    use libboard_artiq::drtio_eem;
     use libboard_artiq::{drtioaux::Error as DrtioError,
                          drtioaux_async,
                          drtioaux_async::Packet,
@@ -24,6 +30,10 @@ pub mod drtio {
     use super::*;
     use crate::{analyzer::remote_analyzer::RemoteBuffer, rtio_dma::remote_dma, subkernel};
 
+    #[cfg(has_drtio_eem)]
+    const DRTIO_EEM_LINKNOS: core::ops::Range<usize> =
+        (csr::DRTIO.len() - csr::CONFIG_EEM_DRTIO_COUNT as usize)..csr::DRTIO.len();
+
     #[derive(Debug, PartialEq, Eq, Clone, Copy)]
     pub enum Error {
         Timeout,
@@ -74,8 +84,18 @@ pub mod drtio {
         });
     }
 
-    async fn link_rx_up(linkno: u8) -> bool {
+    async fn link_rx_up(linkno: u8, _timer: &mut GlobalTimer) -> bool {
         let linkno = linkno as usize;
+        #[cfg(has_drtio_eem)]
+        if DRTIO_EEM_LINKNOS.contains(&linkno) {
+            let eem_trx_no = linkno - DRTIO_EEM_LINKNOS.start;
+            unsafe {
+                csr::eem_transceiver::transceiver_sel_write(eem_trx_no as u8);
+                csr::eem_transceiver::comma_align_reset_write(1);
+            }
+            _timer.delay_us(100);
+            return unsafe { csr::eem_transceiver::comma_read() == 1 };
+        }
         unsafe { (csr::DRTIO[linkno].rx_up_read)() == 1 }
     }
 
@@ -127,6 +147,8 @@ pub mod drtio {
             | Packet::SubkernelLoadRunReply { destination, .. }
             | Packet::SubkernelMessage { destination, .. }
             | Packet::SubkernelMessageAck { destination, .. }
+            | Packet::SubkernelException { destination, .. }
+            | Packet::SubkernelExceptionRequest { destination, .. }
             | Packet::DmaPlaybackStatus { destination, .. }
             | Packet::SubkernelFinished { destination, .. } => {
                 if destination == 0 {
@@ -148,8 +170,8 @@ pub mod drtio {
         }
     }
 
-    async fn recv_aux_timeout(linkno: u8, timeout: u64, timer: GlobalTimer) -> Result<Packet, Error> {
-        if !link_rx_up(linkno).await {
+    async fn recv_aux_timeout(linkno: u8, timeout: u64, mut timer: GlobalTimer) -> Result<Packet, Error> {
+        if !link_rx_up(linkno, &mut timer).await {
             return Err(Error::LinkDown);
         }
         match drtioaux_async::recv_timeout(linkno, Some(timeout), timer).await {
@@ -164,9 +186,9 @@ pub mod drtio {
         linkno: u8,
         routing_table: &RoutingTable,
         request: &Packet,
-        timer: GlobalTimer,
+        mut timer: GlobalTimer,
     ) -> Result<Packet, Error> {
-        if !link_rx_up(linkno).await {
+        if !link_rx_up(linkno, &mut timer).await {
             return Err(Error::LinkDown);
         }
         let _lock = aux_mutex.async_lock().await;
@@ -181,10 +203,7 @@ pub mod drtio {
 
     async fn drain_buffer(linkno: u8, draining_time: Milliseconds, timer: GlobalTimer) {
         let max_time = timer.get_time() + draining_time;
-        loop {
-            if timer.get_time() > max_time {
-                return;
-            }
+        while timer.get_time() < max_time {
             let _ = drtioaux_async::recv(linkno).await;
         }
     }
@@ -193,11 +212,11 @@ pub mod drtio {
         aux_mutex: &Rc<Mutex<bool>>,
         linkno: u8,
         routing_table: &RoutingTable,
-        timer: GlobalTimer,
+        mut timer: GlobalTimer,
     ) -> u32 {
         let mut count = 0;
         loop {
-            if !link_rx_up(linkno).await {
+            if !link_rx_up(linkno, &mut timer).await {
                 return 0;
             }
             count += 1;
@@ -392,29 +411,32 @@ pub mod drtio {
                             }
                             Ok(Packet::DestinationOkReply) => (),
                             Ok(Packet::DestinationSequenceErrorReply { channel }) => {
+                                let global_ch = ((destination as u32) << 16) | channel as u32;
                                 error!(
                                     "[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}",
                                     destination,
                                     channel,
-                                    resolve_channel_name(channel as u32)
+                                    resolve_channel_name(global_ch)
                                 );
                                 unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
                             }
                             Ok(Packet::DestinationCollisionReply { channel }) => {
+                                let global_ch = ((destination as u32) << 16) | channel as u32;
                                 error!(
                                     "[DEST#{}] RTIO collision involving channel 0x{:04x}:{}",
                                     destination,
                                     channel,
-                                    resolve_channel_name(channel as u32)
+                                    resolve_channel_name(global_ch)
                                 );
                                 unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
                             }
                             Ok(Packet::DestinationBusyReply { channel }) => {
+                                let global_ch = ((destination as u32) << 16) | channel as u32;
                                 error!(
                                     "[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}",
                                     destination,
                                     channel,
-                                    resolve_channel_name(channel as u32)
+                                    resolve_channel_name(global_ch)
                                 );
                                 unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
                             }
@@ -461,7 +483,7 @@ pub mod drtio {
         aux_mutex: &Rc<Mutex<bool>>,
         routing_table: &RoutingTable,
         up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
-        timer: GlobalTimer,
+        mut timer: GlobalTimer,
     ) {
         let mut up_links = [false; csr::DRTIO.len()];
         loop {
@@ -469,16 +491,35 @@ pub mod drtio {
                 let linkno = linkno as u8;
                 if up_links[linkno as usize] {
                     /* link was previously up */
-                    if link_rx_up(linkno).await {
+                    if link_rx_up(linkno, &mut timer).await {
                         process_unsolicited_aux(aux_mutex, linkno, routing_table).await;
                         process_local_errors(linkno).await;
                     } else {
                         info!("[LINK#{}] link is down", linkno);
                         up_links[linkno as usize] = false;
+
+                        #[cfg(has_drtio_eem)]
+                        if DRTIO_EEM_LINKNOS.contains(&(linkno as usize)) {
+                            unsafe {
+                                csr::eem_transceiver::rx_ready_write(0);
+                            }
+                            while !matches!(drtioaux_async::recv(linkno).await, Ok(None)) {}
+                        }
                     }
                 } else {
                     /* link was previously down */
-                    if link_rx_up(linkno).await {
+                    #[cfg(has_drtio_eem)]
+                    if DRTIO_EEM_LINKNOS.contains(&(linkno as usize)) {
+                        let eem_trx_no = linkno - DRTIO_EEM_LINKNOS.start as u8;
+                        if !unsafe { drtio_eem::align_wordslip(&mut timer, eem_trx_no) } {
+                            continue;
+                        }
+                        unsafe {
+                            csr::eem_transceiver::rx_ready_write(1);
+                        }
+                    }
+
+                    if link_rx_up(linkno, &mut timer).await {
                         info!("[LINK#{}] link RX became up, pinging", linkno);
                         let ping_count = ping_remote(aux_mutex, linkno, routing_table, timer).await;
                         if ping_count > 0 {
@@ -522,7 +563,7 @@ pub mod drtio {
 
         for linkno in 0..csr::DRTIO.len() {
             let linkno = linkno as u8;
-            if task::block_on(link_rx_up(linkno)) {
+            if task::block_on(link_rx_up(linkno, &mut timer)) {
                 let reply = task::block_on(aux_transact(
                     &aux_mutex,
                     linkno,
@@ -539,7 +580,7 @@ pub mod drtio {
         }
     }
 
-    async fn partition_data<PacketF, HandlerF>(
+    pub async fn partition_data<PacketF, HandlerF>(
         linkno: u8,
         aux_mutex: &Rc<Mutex<bool>>,
         routing_table: &RoutingTable,
@@ -791,6 +832,7 @@ pub mod drtio {
         id: u32,
         destination: u8,
         run: bool,
+        timestamp: u64,
     ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         let reply = aux_transact(
@@ -802,6 +844,7 @@ pub mod drtio {
                 source: 0,
                 destination: destination,
                 run: run,
+                timestamp,
             },
             timer,
         )
@@ -833,13 +876,19 @@ pub mod drtio {
                 linkno,
                 routing_table,
                 &Packet::SubkernelExceptionRequest {
+                    source: 0,
                     destination: destination,
                 },
                 timer,
             )
             .await?;
             match reply {
-                Packet::SubkernelException { last, length, data } => {
+                Packet::SubkernelException {
+                    destination: 0,
+                    last,
+                    length,
+                    data,
+                } => {
                     remote_data.extend(&data[0..length as usize]);
                     if last {
                         return Ok(remote_data);
@@ -898,12 +947,36 @@ pub mod drtio {
     pub fn reset(_aux_mutex: Rc<Mutex<bool>>, _routing_table: &RoutingTable, mut _timer: GlobalTimer) {}
 }
 
+fn toggle_sed_spread(val: u8) {
+    unsafe {
+        csr::rtio_core::sed_spread_enable_write(val);
+    }
+}
+
+fn setup_sed_spread(cfg: &Config) {
+    if let Ok(spread_enable) = cfg.read_str("sed_spread_enable") {
+        match spread_enable.as_ref() {
+            "1" => toggle_sed_spread(1),
+            "0" => toggle_sed_spread(0),
+            _ => {
+                warn!("sed_spread_enable value not supported (only 1, 0 allowed), disabling by default");
+                toggle_sed_spread(0)
+            }
+        };
+    } else {
+        info!("SED spreading disabled by default");
+        toggle_sed_spread(0);
+    }
+}
+
 pub fn startup(
     aux_mutex: &Rc<Mutex<bool>>,
     routing_table: &Rc<RefCell<RoutingTable>>,
     up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
+    cfg: &Config,
     timer: GlobalTimer,
 ) {
+    setup_sed_spread(cfg);
     drtio::startup(aux_mutex, routing_table, up_destinations, timer);
     unsafe {
         csr::rtio_core::reset_phy_write(1);

src/runtime/src/subkernel.rs

@@ -100,12 +100,22 @@ pub async fn load(
     timer: GlobalTimer,
     id: u32,
     run: bool,
+    timestamp: u64,
 ) -> Result<(), Error> {
     if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
         if subkernel.state != SubkernelState::Uploaded {
             return Err(Error::IncorrectState);
         }
-        drtio::subkernel_load(aux_mutex, routing_table, timer, id, subkernel.destination, run).await?;
+        drtio::subkernel_load(
+            aux_mutex,
+            routing_table,
+            timer,
+            id,
+            subkernel.destination,
+            run,
+            timestamp,
+        )
+        .await?;
         if run {
             subkernel.state = SubkernelState::Running;
         }

src/rustfmt.toml

@@ -54,7 +54,7 @@ use_field_init_shorthand = false
 force_explicit_abi = true
 condense_wildcard_suffixes = false
 color = "Auto"
-required_version = "1.4.32"
+required_version = "1.4.37"
 unstable_features = false
 disable_all_formatting = false
 skip_children = false
@@ -65,4 +65,4 @@ report_todo = "Never"
 report_fixme = "Never"
 ignore = []
 emit_mode = "Files"
-make_backup = false
+make_backup = false

src/satman/Cargo.toml.tpl

@@ -15,7 +15,9 @@ build_zynq = { path = "../libbuild_zynq" }
 
 [dependencies]
 log = { version = "0.4", default-features = false }
-core_io = { version = "0.1", features = ["collections"] }
+byteorder = { version = "1.3", default-features = false }
+core_io = { git = "https://git.m-labs.hk/M-Labs/rs-core_io.git", rev = "e9d3edf027", features = ["collections"] }
+crc = { version = "1.7", default-features = false }
 cslice = "0.3"
 embedded-hal = "0.2"
 

src/satman/src/mgmt.rs

@@ -0,0 +1,149 @@
+use alloc::vec::Vec;
+
+use byteorder::{ByteOrder, NativeEndian};
+use crc::crc32;
+use io::{ProtoRead, ProtoWrite};
+use libboard_artiq::{drtioaux_proto::SAT_PAYLOAD_MAX_SIZE,
+                     logger::{BufferLogger, LogBufferRef}};
+use libconfig::Config;
+use log::{debug, error, info, warn, LevelFilter};
+
+use crate::routing::{SliceMeta, Sliceable};
+
+type Result<T> = core::result::Result<T, ()>;
+
+pub fn byte_to_level_filter(level_byte: u8) -> Result<LevelFilter> {
+    Ok(match level_byte {
+        0 => LevelFilter::Off,
+        1 => LevelFilter::Error,
+        2 => LevelFilter::Warn,
+        3 => LevelFilter::Info,
+        4 => LevelFilter::Debug,
+        5 => LevelFilter::Trace,
+        lv => {
+            error!("unknown log level: {}", lv);
+            return Err(());
+        }
+    })
+}
+
+fn get_logger_buffer() -> LogBufferRef<'static> {
+    let logger = unsafe { BufferLogger::get_logger().as_mut().unwrap() };
+    loop {
+        if let Some(buffer_ref) = logger.buffer() {
+            return buffer_ref;
+        }
+    }
+}
+
+pub fn clear_log() {
+    let mut buffer = get_logger_buffer();
+    buffer.clear();
+}
+
+pub struct Manager<'a> {
+    cfg: &'a mut Config,
+    last_log: Sliceable,
+    config_payload: Vec<u8>,
+    last_value: Sliceable,
+    image_payload: Vec<u8>,
+}
+
+impl<'a> Manager<'_> {
+    pub fn new(cfg: &mut Config) -> Manager {
+        Manager {
+            cfg: cfg,
+            last_log: Sliceable::new(0, Vec::new()),
+            config_payload: Vec::new(),
+            last_value: Sliceable::new(0, Vec::new()),
+            image_payload: Vec::new(),
+        }
+    }
+
+    pub fn log_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE], consume: bool) -> SliceMeta {
+        // Populate buffer if depleted
+        if self.last_log.at_end() {
+            let mut buffer = get_logger_buffer();
+            self.last_log.extend(buffer.extract().as_bytes());
+            if consume {
+                buffer.clear();
+            }
+        }
+
+        self.last_log.get_slice_satellite(data_slice)
+    }
+
+    pub fn fetch_config_value(&mut self, key: &str) -> Result<()> {
+        self.cfg
+            .read(&key)
+            .map(|value| {
+                debug!("got value");
+                self.last_value = Sliceable::new(0, value)
+            })
+            .map_err(|_| warn!("read error: no such key"))
+    }
+
+    pub fn get_config_value_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
+        self.last_value.get_slice_satellite(data_slice)
+    }
+
+    pub fn add_config_data(&mut self, data: &[u8], data_len: usize) {
+        self.config_payload.write_all(&data[..data_len]).unwrap();
+    }
+
+    pub fn clear_config_data(&mut self) {
+        self.config_payload.clear();
+    }
+
+    pub fn write_config(&mut self) -> Result<()> {
+        let mut payload = &self.config_payload[..];
+        let key = payload.read_string().map_err(|_err| error!("error on reading key"))?;
+        debug!("write key: {}", key);
+        let value = payload.read_bytes().unwrap();
+
+        self.cfg
+            .write(&key, value)
+            .map(|()| debug!("write success"))
+            .map_err(|err| error!("failed to write: {:?}", err))
+    }
+
+    pub fn remove_config(&mut self, key: &str) -> Result<()> {
+        debug!("erase key: {}", key);
+        self.cfg
+            .remove(&key)
+            .map(|()| debug!("erase success"))
+            .map_err(|err| warn!("failed to erase: {:?}", err))
+    }
+
+    pub fn allocate_image_buffer(&mut self, image_size: usize) {
+        self.image_payload = Vec::with_capacity(image_size);
+    }
+
+    pub fn add_image_data(&mut self, data: &[u8], data_len: usize) {
+        self.image_payload.extend(&data[..data_len]);
+    }
+
+    pub fn write_image(&self) {
+        let mut image = self.image_payload.clone();
+        let image_ref = &image[..];
+        let bin_len = image.len() - 4;
+
+        let (image_ref, expected_crc) = {
+            let (image_ref, crc_slice) = image_ref.split_at(bin_len);
+            (image_ref, NativeEndian::read_u32(crc_slice))
+        };
+
+        let actual_crc = crc32::checksum_ieee(image_ref);
+
+        if actual_crc == expected_crc {
+            info!("CRC passed. Writing boot image to SD card...");
+            image.truncate(bin_len);
+            self.cfg.write("boot", image).expect("failed to write boot image");
+        } else {
+            panic!(
+                "CRC failed, images have not been written to flash.\n(actual {:08x}, expected {:08x})",
+                actual_crc, expected_crc
+            );
+        }
+    }
+}

src/satman/src/repeater.rs

@@ -87,6 +87,10 @@ impl Repeater {
                 if rep_link_rx_up(self.repno) {
                     if let Ok(Some(drtioaux::Packet::EchoReply)) = drtioaux::recv(self.auxno) {
                         info!("[REP#{}] remote replied after {} packets", self.repno, ping_count);
+                        let max_time = timer.get_time() + Milliseconds(200);
+                        while timer.get_time() < max_time {
+                            let _ = drtioaux::recv(self.auxno);
+                        }
                         self.state = RepeaterState::Up;
                         if let Err(e) = self.sync_tsc(timer) {
                             error!("[REP#{}] failed to sync TSC ({:?})", self.repno, e);
@@ -204,10 +208,37 @@ impl Repeater {
         }
     }
 
-    pub fn aux_forward(&self, request: &drtioaux::Packet, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
+    pub fn aux_forward(
+        &self,
+        request: &drtioaux::Packet,
+        router: &mut Router,
+        routing_table: &drtio_routing::RoutingTable,
+        rank: u8,
+        self_destination: u8,
+        timer: &mut GlobalTimer,
+    ) -> Result<(), drtioaux::Error> {
         self.aux_send(request)?;
-        let reply = self.recv_aux_timeout(200, timer)?;
-        drtioaux::send(0, &reply).unwrap();
+        loop {
+            let reply = self.recv_aux_timeout(200, timer)?;
+            match reply {
+                // async/locally requested packets to be consumed or routed
+                // these may come while a packet would be forwarded
+                drtioaux::Packet::DmaPlaybackStatus { .. }
+                | drtioaux::Packet::SubkernelFinished { .. }
+                | drtioaux::Packet::SubkernelMessage { .. }
+                | drtioaux::Packet::SubkernelMessageAck { .. }
+                | drtioaux::Packet::SubkernelLoadRunReply { .. }
+                | drtioaux::Packet::SubkernelException { .. }
+                | drtioaux::Packet::DmaAddTraceReply { .. }
+                | drtioaux::Packet::DmaPlaybackReply { .. } => {
+                    router.route(reply, routing_table, rank, self_destination);
+                }
+                _ => {
+                    drtioaux::send(0, &reply).unwrap();
+                    break;
+                }
+            }
+        }
         Ok(())
     }
 

src/satman/src/routing.rs

@@ -57,8 +57,8 @@ impl Sliceable {
         self.data.extend(data);
     }
 
-    get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
     get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
+    get_slice_fn!(get_slice_satellite, SAT_PAYLOAD_MAX_SIZE);
 }
 
 // Packets from downstream (further satellites) are received and routed appropriately.

src/satman/src/subkernel.rs

@@ -2,16 +2,16 @@ use alloc::{collections::BTreeMap,
             format,
             string::{String, ToString},
             vec::Vec};
-use core::{option::NoneError, slice, str};
+use core::{slice, str};
 
 use core_io::{Error as IoError, Write};
 use cslice::AsCSlice;
 use dma::{Error as DmaError, Manager as DmaManager};
 use io::{Cursor, ProtoWrite};
-use ksupport::{eh_artiq, kernel, rpc};
+use ksupport::{eh_artiq, kernel, rpc, rtio};
 use libboard_artiq::{drtio_routing::RoutingTable,
                      drtioaux,
-                     drtioaux_proto::{PayloadStatus, MASTER_PAYLOAD_MAX_SIZE, SAT_PAYLOAD_MAX_SIZE},
+                     drtioaux_proto::{PayloadStatus, MASTER_PAYLOAD_MAX_SIZE},
                      pl::csr};
 use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
 use libcortex_a9::sync_channel::Receiver;
@@ -47,6 +47,9 @@ enum KernelState {
     DmaAwait {
         max_time: Milliseconds,
     },
+    SubkernelRetrievingException {
+        destination: u8,
+    },
 }
 
 #[derive(Debug)]
@@ -62,12 +65,6 @@ pub enum Error {
     DmaError(DmaError),
 }
 
-impl From<NoneError> for Error {
-    fn from(_: NoneError) -> Error {
-        Error::KernelNotFound
-    }
-}
-
 impl From<IoError> for Error {
     fn from(_value: IoError) -> Error {
         Error::SubkernelIoError
@@ -123,10 +120,11 @@ struct MessageManager {
 struct Session {
     id: u32,
     kernel_state: KernelState,
-    last_exception: Option<Sliceable>,
+    last_exception: Option<Sliceable>,   // exceptions raised locally
+    external_exception: Option<Vec<u8>>, // exceptions from sub-subkernels
     messages: MessageManager,
     source: u8, // which destination requested running the kernel
-    subkernels_finished: Vec<u32>,
+    subkernels_finished: Vec<(u32, Option<u8>)>,
 }
 
 impl Session {
@@ -135,6 +133,7 @@ impl Session {
             id: id,
             kernel_state: KernelState::Absent,
             last_exception: None,
+            external_exception: None,
             messages: MessageManager::new(),
             source: 0,
             subkernels_finished: Vec::new(),
@@ -311,7 +310,7 @@ impl<'a> Manager<'_> {
                             complete: false,
                         },
                     );
-                    self.kernels.get_mut(&id)?
+                    self.kernels.get_mut(&id).ok_or_else(|| Error::KernelNotFound)?
                 } else {
                     kernel
                 }
@@ -324,7 +323,7 @@ impl<'a> Manager<'_> {
                         complete: false,
                     },
                 );
-                self.kernels.get_mut(&id)?
+                self.kernels.get_mut(&id).ok_or_else(|| Error::KernelNotFound)?
             }
         };
         kernel.library.extend(&data[0..data_len]);
@@ -344,7 +343,7 @@ impl<'a> Manager<'_> {
         }
     }
 
-    pub fn run(&mut self, source: u8, id: u32) -> Result<(), Error> {
+    pub fn run(&mut self, source: u8, id: u32, timestamp: u64) -> Result<(), Error> {
         if self.session.kernel_state != KernelState::Loaded || self.session.id != id {
             self.load(id)?;
         }
@@ -354,6 +353,7 @@ impl<'a> Manager<'_> {
             csr::cri_con::selected_write(2);
         }
 
+        rtio::at_mu(timestamp as i64);
         self.control.tx.send(kernel::Message::StartRequest);
         Ok(())
     }
@@ -390,15 +390,19 @@ impl<'a> Manager<'_> {
         if self.session.id == id && self.session.kernel_state == KernelState::Loaded {
             return Ok(());
         }
-        if !self.kernels.get(&id)?.complete {
+        if !self.kernels.get(&id).ok_or_else(|| Error::KernelNotFound)?.complete {
             return Err(Error::KernelNotFound);
         }
         self.session = Session::new(id);
         self.control.restart();
 
-        self.control
-            .tx
-            .send(kernel::Message::LoadRequest(self.kernels.get(&id)?.library.clone()));
+        self.control.tx.send(kernel::Message::LoadRequest(
+            self.kernels
+                .get(&id)
+                .ok_or_else(|| Error::KernelNotFound)?
+                .library
+                .clone(),
+        ));
         let reply = self.control.rx.recv();
         match reply {
             kernel::Message::LoadCompleted => Ok(()),
@@ -410,9 +414,9 @@ impl<'a> Manager<'_> {
         }
     }
 
-    pub fn exception_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
+    pub fn exception_get_slice(&mut self, data_slice: &mut [u8; MASTER_PAYLOAD_MAX_SIZE]) -> SliceMeta {
         match self.session.last_exception.as_mut() {
-            Some(exception) => exception.get_slice_sat(data_slice),
+            Some(exception) => exception.get_slice_master(data_slice),
             None => SliceMeta {
                 destination: 0,
                 len: 0,
@@ -540,7 +544,7 @@ impl<'a> Manager<'_> {
             return;
         }
 
-        match self.process_external_messages(timer) {
+        match self.process_external_messages(router, routing_table, rank, destination, timer) {
             Ok(()) => (),
             Err(Error::AwaitingMessage) => return, // kernel still waiting, do not process kernel messages
             Err(Error::KernelException(exception)) => {
@@ -596,6 +600,41 @@ impl<'a> Manager<'_> {
         }
     }
 
+    fn check_finished_kernels(
+        &mut self,
+        id: u32,
+        router: &mut Router,
+        routing_table: &RoutingTable,
+        rank: u8,
+        self_destination: u8,
+    ) {
+        for (i, (status, exception_source)) in self.session.subkernels_finished.iter().enumerate() {
+            if *status == id {
+                if exception_source.is_none() {
+                    self.control.tx.send(kernel::Message::SubkernelAwaitFinishReply);
+                    self.session.kernel_state = KernelState::Running;
+                    self.session.subkernels_finished.swap_remove(i);
+                } else {
+                    let destination = exception_source.unwrap();
+                    self.session.external_exception = Some(Vec::new());
+                    self.session.kernel_state = KernelState::SubkernelRetrievingException {
+                        destination: destination,
+                    };
+                    router.route(
+                        drtioaux::Packet::SubkernelExceptionRequest {
+                            source: self_destination,
+                            destination: destination,
+                        },
+                        &routing_table,
+                        rank,
+                        self_destination,
+                    );
+                }
+                break;
+            }
+        }
+    }
+
     pub fn subkernel_load_run_reply(&mut self, succeeded: bool) {
         if self.session.kernel_state == KernelState::SubkernelAwaitLoad {
             self.control
@@ -608,16 +647,46 @@ impl<'a> Manager<'_> {
     }
 
     pub fn remote_subkernel_finished(&mut self, id: u32, with_exception: bool, exception_source: u8) {
-        if with_exception {
-            self.kernel_stop();
-            self.last_finished = Some(SubkernelFinished {
-                source: self.session.source,
-                id: self.session.id,
-                with_exception: true,
-                exception_source: exception_source,
-            })
+        let exception_src = if with_exception { Some(exception_source) } else { None };
+        self.session.subkernels_finished.push((id, exception_src));
+    }
+
+    pub fn received_exception(
+        &mut self,
+        exception_data: &[u8],
+        last: bool,
+        router: &mut Router,
+        routing_table: &RoutingTable,
+        rank: u8,
+        self_destination: u8,
+    ) {
+        if let KernelState::SubkernelRetrievingException { destination } = self.session.kernel_state {
+            self.session
+                .external_exception
+                .as_mut()
+                .unwrap()
+                .extend_from_slice(exception_data);
+            if last {
+                self.control
+                    .tx
+                    .send(kernel::Message::SubkernelError(kernel::SubkernelStatus::Exception(
+                        self.session.external_exception.take().unwrap(),
+                    )));
+                self.session.kernel_state = KernelState::Running;
+            } else {
+                /* fetch another slice */
+                router.route(
+                    drtioaux::Packet::SubkernelExceptionRequest {
+                        source: self_destination,
+                        destination: destination,
+                    },
+                    routing_table,
+                    rank,
+                    self_destination,
+                );
+            }
         } else {
-            self.session.subkernels_finished.push(id);
+            warn!("Received unsolicited exception data");
         }
     }
 
@@ -742,6 +811,7 @@ impl<'a> Manager<'_> {
                 id,
                 destination: sk_destination,
                 run,
+                timestamp,
             } => {
                 self.session.kernel_state = KernelState::SubkernelAwaitLoad;
                 router.route(
@@ -750,6 +820,7 @@ impl<'a> Manager<'_> {
                         destination: sk_destination,
                         id: id,
                         run: run,
+                        timestamp,
                     },
                     routing_table,
                     rank,
@@ -780,28 +851,35 @@ impl<'a> Manager<'_> {
         Ok(false)
     }
 
-    fn process_external_messages(&mut self, timer: &GlobalTimer) -> Result<(), Error> {
+    fn process_external_messages(
+        &mut self,
+        router: &mut Router,
+        routing_table: &RoutingTable,
+        rank: u8,
+        self_destination: u8,
+        timer: &GlobalTimer,
+    ) -> Result<(), Error> {
         match &self.session.kernel_state {
             KernelState::MsgAwait { max_time, id, tags } => {
                 if let Some(max_time) = *max_time {
                     if timer.get_time() > max_time {
-                        self.control.tx.send(kernel::Message::SubkernelMsgRecvReply {
-                            status: kernel::SubkernelStatus::Timeout,
-                            count: 0,
-                        });
+                        self.control
+                            .tx
+                            .send(kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout));
                         self.session.kernel_state = KernelState::Running;
                         return Ok(());
                     }
                 }
                 if let Some(message) = self.session.messages.get_incoming(*id) {
-                    self.control.tx.send(kernel::Message::SubkernelMsgRecvReply {
-                        status: kernel::SubkernelStatus::NoError,
-                        count: message.count,
-                    });
+                    self.control
+                        .tx
+                        .send(kernel::Message::SubkernelMsgRecvReply { count: message.count });
                     let tags = tags.clone();
                     self.session.kernel_state = KernelState::Running;
                     self.pass_message_to_kernel(&message, tags, timer)
                 } else {
+                    let id = *id;
+                    self.check_finished_kernels(id, router, routing_table, rank, self_destination);
                     Err(Error::AwaitingMessage)
                 }
             }
@@ -817,27 +895,18 @@ impl<'a> Manager<'_> {
             KernelState::SubkernelAwaitFinish { max_time, id } => {
                 if let Some(max_time) = *max_time {
                     if timer.get_time() > max_time {
-                        self.control.tx.send(kernel::Message::SubkernelAwaitFinishReply {
-                            status: kernel::SubkernelStatus::Timeout,
-                        });
+                        self.control
+                            .tx
+                            .send(kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout));
                         self.session.kernel_state = KernelState::Running;
                         return Ok(());
                     }
                 }
-                let mut i = 0;
-                for status in &self.session.subkernels_finished {
-                    if *status == *id {
-                        self.control.tx.send(kernel::Message::SubkernelAwaitFinishReply {
-                            status: kernel::SubkernelStatus::NoError,
-                        });
-                        self.session.kernel_state = KernelState::Running;
-                        self.session.subkernels_finished.swap_remove(i);
-                        break;
-                    }
-                    i += 1;
-                }
+                let id = *id;
+                self.check_finished_kernels(id, router, routing_table, rank, self_destination);
                 Ok(())
             }
+            KernelState::SubkernelRetrievingException { .. } => Err(Error::AwaitingMessage),
             KernelState::DmaAwait { max_time } | KernelState::DmaPendingAwait { max_time, .. } => {
                 if timer.get_time() > *max_time {
                     self.control.tx.send(kernel::Message::DmaAwaitRemoteReply {

test.py

@@ -0,0 +1,30 @@
+from math import lcm
+
+word_dw = 32
+size = 8
+
+
+sink_dw, source_dw = word_dw, size * 4
+shift_reg_size = lcm(sink_dw, source_dw)
+if (shift_reg_size // sink_dw) < 2:
+    shift_reg_size = shift_reg_size * 2
+if (shift_reg_size // source_dw) < 2:
+    shift_reg_size = shift_reg_size * 2
+
+
+sink_aligned_bound = sink_dw
+result = []
+for i in range(4, shift_reg_size // size):
+    if i % 4 == 0:
+        while not (sink_aligned_bound + sink_dw >= i * size >= sink_aligned_bound):
+            sink_aligned_bound += word_dw
+    else:
+        if sink_aligned_bound + sink_dw >= i * size >= sink_aligned_bound:
+            result.append(i)
+
+    # print(i * size, sink_aligned_bound, sink_aligned_bound + sink_dw)
+
+print(source_dw % sink_dw)
+
+
+