gateware: add fastino inject WIP

Currently the exact ordering is compared in the `pending_priorities`
test which is not necessary and is non-deterministic. This patch only
 asserts the middle priority experiment is ran before the high priority
 experiment scheduled in the future.

Signed-off-by: Kanyang Ying <lexugeyky@outlook.com>

README.rst

@@ -5,31 +5,27 @@
   :target: https://m-labs.hk/artiq
 
 ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) is a leading-edge control and data acquisition system for quantum information experiments.
-It is maintained and developed by `M-Labs <https://m-labs.hk>`_ and the initial development was for and in partnership with the `Ion Storage Group at NIST <https://www.nist.gov/pml/time-and-frequency-division/ion-storage>`_. ARTIQ is free software and offered to the entire research community as a solution equally applicable to other challenging control tasks, including outside the field of ion trapping. Many laboratories around the world have adopted ARTIQ as their control system, with over a hundred Sinara hardware crates deployed, and some have `contributed <https://m-labs.hk/experiment-control/funding/>`_ to it.
+It is maintained and developed by `M-Labs <https://m-labs.hk>`_ and the initial development was for and in partnership with the `Ion Storage Group at NIST <https://www.nist.gov/pml/time-and-frequency-division/ion-storage>`_. ARTIQ is free software and offered to the entire research community as a solution equally applicable to other challenging control tasks, including outside the field of ion trapping. Many laboratories around the world have adopted ARTIQ as their control system and some have `contributed <https://m-labs.hk/experiment-control/funding/>`_ to it.
 
-The system features a high-level programming language that helps describing complex experiments, which is compiled and executed on dedicated hardware with nanosecond timing resolution and sub-microsecond latency. It includes graphical user interfaces to parametrize and schedule experiments and to visualize and explore the results.
+The system features a high-level programming language, capable of describing complex experiments, which is compiled and executed on dedicated hardware with nanosecond timing resolution and sub-microsecond latency. It includes graphical user interfaces to parametrize and schedule experiments and to visualize and explore the results.
 
-ARTIQ uses FPGA hardware to perform its time-critical tasks. The `Sinara hardware <https://github.com/sinara-hw>`_, and in particular the Kasli FPGA carrier, is designed to work with ARTIQ.
-ARTIQ is designed to be portable to hardware platforms from different vendors and FPGA manufacturers.
-Several different configurations of a `FPGA evaluation kit <https://www.xilinx.com/products/boards-and-kits/ek-k7-kc705-g.html>`_ and of a `Zynq evaluation kit <https://www.xilinx.com/products/boards-and-kits/ek-z7-zc706-g.html>`_ are also used and supported. FPGA platforms can be combined with any number of additional peripherals, either already accessible from ARTIQ or made accessible with little effort.
+ARTIQ uses FPGA hardware to perform its time-critical tasks. The `Sinara hardware <https://github.com/sinara-hw>`_, and in particular the Kasli FPGA carrier, are designed to work with ARTIQ. ARTIQ is designed to be portable to hardware platforms from different vendors and FPGA manufacturers. Several different configurations of a `FPGA evaluation kit <https://www.xilinx.com/products/boards-and-kits/ek-k7-kc705-g.html>`_ and a `Zynq evaluation kit <https://www.xilinx.com/products/boards-and-kits/ek-z7-zc706-g.html>`_ are also used and supported. FPGA platforms can be combined with any number of additional peripherals, either already accessible from ARTIQ or made accessible with little effort.
 
-ARTIQ and its dependencies are available in the form of Nix packages (for Linux) and MSYS2 packages (for Windows). See `the manual <https://m-labs.hk/experiment-control/resources/>`_ for installation instructions.
-Packages containing pre-compiled binary images to be loaded onto the hardware platforms are supplied for each configuration.
-Like any open source software ARTIQ can equally be built and installed directly from `source <https://github.com/m-labs/artiq>`_.
+ARTIQ and its dependencies are available in the form of Nix packages (for Linux) and MSYS2 packages (for Windows). See `the manual <https://m-labs.hk/experiment-control/resources/>`_ for installation instructions. Packages containing pre-compiled binary images to be loaded onto the hardware platforms are supplied for each configuration. Like any open-source software ARTIQ can equally be built and installed directly from `source <https://github.com/m-labs/artiq>`_.
 
-ARTIQ is supported by M-Labs and developed openly.
-Components, features, fixes, improvements, and extensions are often `funded <https://m-labs.hk/experiment-control/funding/>`_ by and developed for the partnering research groups.
+ARTIQ is supported by M-Labs and developed openly. Components, features, fixes, improvements, and extensions are often `funded <https://m-labs.hk/experiment-control/funding/>`_ by and developed for the partnering research groups.
 
 Core technologies employed include `Python <https://www.python.org/>`_, `Migen <https://github.com/m-labs/migen>`_, `Migen-AXI <https://github.com/peteut/migen-axi>`_, `Rust <https://www.rust-lang.org/>`_, `MiSoC <https://github.com/m-labs/misoc>`_/`VexRiscv <https://github.com/SpinalHDL/VexRiscv>`_, `LLVM <https://llvm.org/>`_/`llvmlite <https://github.com/numba/llvmlite>`_, and `Qt5 <https://www.qt.io/>`_.
 
-Website: https://m-labs.hk/artiq
+| Website: https://m-labs.hk/experiment-control/artiq
+| (US-hosted mirror: https://m-labs-intl.com/experiment-control/artiq)
 
 `Cite ARTIQ <http://dx.doi.org/10.5281/zenodo.51303>`_ as ``Bourdeauducq, Sébastien et al. (2016). ARTIQ 1.0. Zenodo. 10.5281/zenodo.51303``.
 
 License
 =======
 
-Copyright (C) 2014-2023 M-Labs Limited.
+Copyright (C) 2014-2024 M-Labs Limited.
 
 ARTIQ is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by

RELEASE_NOTES.rst

@@ -3,9 +3,17 @@
 Release notes
 =============
 
-ARTIQ-8 (Unreleased)
+ARTIQ-9 (Unreleased)
 --------------------
 
+* Zotino monitoring in the dashboard now displays the values in volts.
+* afws_client now uses the "happy eyeballs" algorithm (RFC 6555) for a faster and more
+  reliable connection to the server.
+* The Zadig driver installer was added to the MSYS2 offline installer.
+
+ARTIQ-8
+-------
+
 Highlights:
 
 * New hardware support:
@@ -28,23 +36,38 @@ Highlights:
   clock, to facilitate implementation of local processing on DRTIO satellites, and to slightly
   reduce RTIO latency.
 * Support for DRTIO-over-EEM, used with Shuttler.
+* Support for WRPLL low-noise clock recovery.
+* Enabled event spreading on DRTIO satellites, using high watermark for lane switching.
 * Added channel names to RTIO error messages.
+* The RTIO analyzer is now proxied by ``aqctl_coreanalyzer_proxy`` typically running on the master
+  machine, similarly to ``aqctl_moninj_proxy``.
 * GUI:
+   - Integrated waveform analyzer, removing the need for external VCD viewers such as GtkWave.
    - Implemented Applet Request Interfaces which allow applets to modify datasets and set the
      current values of widgets in the dashboard's experiment windows.
-   - Implemented a new EntryArea widget which allows argument entry widgets to be used in applets.
+   - Implemented a new ``EntryArea`` widget which allows argument entry widgets to be used in applets.
    - The "Close all applets" command (shortcut: Ctrl-Alt-W) now ignores docked applets,
      making it a convenient way to clean up after exploratory work without destroying a
      carefully arranged default workspace.
    - Hotkeys now organize experiment windows in the order they were last interacted with:
       + CTRL+SHIFT+T tiles experiment windows
       + CTRL+SHIFT+C cascades experiment windows
+   - By enabling the ``quickstyle`` option, ``EnumerationValue`` entry widgets can now alternatively display 
+     its choices as buttons that submit the experiment on click.
+* Datasets can now be associated with units and scale factors, and displayed accordingly in the dashboard
+  including applets, like widgets such as ``NumberValue`` already did in earlier ARTIQ versions.
+* Experiments can now request arguments interactively from the user at any time.
 * Persistent datasets are now stored in a LMDB database for improved performance.
 * Python's built-in types (such as ``float``, or ``List[...]``) can now be used in type annotations on
   kernel functions.
-* Full Python 3.10 support.
 * MSYS2 packaging for Windows, which replaces Conda. Conda packages are still available to
   support legacy installations, but may be removed in a future release.
+* Experiments can now be submitted with revisions set to a branch / tag name instead of only git hashes.
+* Grabber image input now has an optional timeout.
+* On NAR3-supported devices (Kasli-SoC, ZC706), when a Rust panic occurs, a minimal environment is started
+  where the network and ``artiq_coremgmt`` can be used. This allows the user to inspect logs, change
+  configuration options, update the firmware, and reboot the device.
+* Full Python 3.11 support.
 
 Breaking changes:
 

artiq/applets/plot_xy.py

@@ -24,11 +24,11 @@ class XYPlot(pyqtgraph.PlotWidget):
             y = value[self.args.y]
         except KeyError:
             return
-        x = value.get(self.args.x, (False, None))
+        x = value.get(self.args.x)
         if x is None:
             x = np.arange(len(y))
-        error = value.get(self.args.error, (False, None))
-        fit = value.get(self.args.fit, (False, None))
+        error = value.get(self.args.error)
+        fit = value.get(self.args.fit)
 
         if not len(y) or len(y) != len(x):
             self.mismatch['X values'] = True

artiq/applets/simple.py

@@ -42,13 +42,13 @@ class _AppletRequestInterface:
         """
         raise NotImplementedError
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         """
         Temporarily set the value of an argument in a experiment in the dashboard.
         The value resets to default value when recomputing the argument.
 
         :param expurl: Experiment URL identifying the experiment in the dashboard. Example: 'repo:ArgumentsDemo'.
-        :param name: Name of the argument in the experiment.
+        :param key: Name of the argument in the experiment.
         :param value: Object representing the new temporary value of the argument. For ``Scannable`` arguments, this parameter
             should be a ``ScanObject``. The type of the ``ScanObject`` will be set as the selected type when this function is called. 
         """
@@ -77,10 +77,10 @@ class AppletRequestIPC(_AppletRequestInterface):
         mod = {"action": "append", "path": [key, 1], "x": value}
         self.ipc.update_dataset(mod)
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         if isinstance(value, ScanObject):
             value = value.describe()
-        self.ipc.set_argument_value(expurl, name, value)
+        self.ipc.set_argument_value(expurl, key, value)
 
 
 class AppletRequestRPC(_AppletRequestInterface):
@@ -182,10 +182,10 @@ class AppletIPCClient(AsyncioChildComm):
         self.write_pyon({"action": "update_dataset",
                          "mod": mod})
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         self.write_pyon({"action": "set_argument_value",
                          "expurl": expurl,
-                         "name": name,
+                         "key": key,
                          "value": value})
 
 
@@ -255,7 +255,7 @@ class SimpleApplet:
         if self.embed is None:
             dataset_ctl = RPCClient()
             self.loop.run_until_complete(dataset_ctl.connect_rpc(
-                self.args.server, self.args.port_control, "master_dataset_db"))
+                self.args.server, self.args.port_control, "dataset_db"))
             self.req = AppletRequestRPC(self.loop, dataset_ctl)
         else:
             self.req = AppletRequestIPC(self.ipc)

artiq/browser/datasets.py

@@ -33,7 +33,7 @@ class DatasetCtl:
         try:
             remote = RPCClient()
             await remote.connect_rpc(self.master_host, self.master_port,
-                                     "master_dataset_db")
+                                     "dataset_db")
             try:
                 if op_name == "set":
                     await remote.set(key_or_mod, value, persist, metadata)

artiq/browser/experiments.py

@@ -10,87 +10,26 @@ import h5py
 from sipyco import pyon
 
 from artiq import __artiq_dir__ as artiq_dir
-from artiq.gui.tools import (LayoutWidget, WheelFilter, 
-                             log_level_to_name, get_open_file_name)
-from artiq.gui.entries import procdesc_to_entry
+from artiq.gui.tools import (LayoutWidget, log_level_to_name, get_open_file_name)
+from artiq.gui.entries import procdesc_to_entry, EntryTreeWidget
 from artiq.master.worker import Worker, log_worker_exception
 
 logger = logging.getLogger(__name__)
 
 
-class _ArgumentEditor(QtWidgets.QTreeWidget):
+class _ArgumentEditor(EntryTreeWidget):
     def __init__(self, dock):
-        QtWidgets.QTreeWidget.__init__(self)
-        self.setColumnCount(3)
-        self.header().setStretchLastSection(False)
-        try:
-            set_resize_mode = self.header().setSectionResizeMode
-        except AttributeError:
-            set_resize_mode = self.header().setResizeMode
-        set_resize_mode(0, QtWidgets.QHeaderView.ResizeToContents)
-        set_resize_mode(1, QtWidgets.QHeaderView.Stretch)
-        set_resize_mode(2, QtWidgets.QHeaderView.ResizeToContents)
-        self.header().setVisible(False)
-        self.setSelectionMode(self.NoSelection)
-        self.setHorizontalScrollMode(self.ScrollPerPixel)
-        self.setVerticalScrollMode(self.ScrollPerPixel)
-
-        self.setStyleSheet("QTreeWidget {background: " +
-                           self.palette().midlight().color().name() + " ;}")
-
-        self.viewport().installEventFilter(WheelFilter(self.viewport(), True))
-
-        self._groups = dict()
-        self._arg_to_widgets = dict()
+        EntryTreeWidget.__init__(self)
         self._dock = dock
 
         if not self._dock.arguments:
-            self.addTopLevelItem(QtWidgets.QTreeWidgetItem(["No arguments"]))
-        gradient = QtGui.QLinearGradient(
-            0, 0, 0, QtGui.QFontMetrics(self.font()).lineSpacing()*2.5)
-        gradient.setColorAt(0, self.palette().base().color())
-        gradient.setColorAt(1, self.palette().midlight().color())
+            self.insertTopLevelItem(0, QtWidgets.QTreeWidgetItem(["No arguments"]))
 
         for name, argument in self._dock.arguments.items():
-            widgets = dict()
-            self._arg_to_widgets[name] = widgets
+            self.set_argument(name, argument)
 
-            entry = procdesc_to_entry(argument["desc"])(argument)
-            widget_item = QtWidgets.QTreeWidgetItem([name])
-            if argument["tooltip"]:
-                widget_item.setToolTip(0, argument["tooltip"])
-            widgets["entry"] = entry
-            widgets["widget_item"] = widget_item
+        self.quickStyleClicked.connect(self._dock._run_clicked)
 
-            for col in range(3):
-                widget_item.setBackground(col, gradient)
-            font = widget_item.font(0)
-            font.setBold(True)
-            widget_item.setFont(0, font)
-
-            if argument["group"] is None:
-                self.addTopLevelItem(widget_item)
-            else:
-                self._get_group(argument["group"]).addChild(widget_item)
-            fix_layout = LayoutWidget()
-            widgets["fix_layout"] = fix_layout
-            fix_layout.addWidget(entry)
-            self.setItemWidget(widget_item, 1, fix_layout)
-
-            recompute_argument = QtWidgets.QToolButton()
-            recompute_argument.setToolTip("Re-run the experiment's build "
-                                          "method and take the default value")
-            recompute_argument.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_BrowserReload))
-            recompute_argument.clicked.connect(
-                partial(self._recompute_argument_clicked, name))
-            fix_layout = LayoutWidget()
-            fix_layout.addWidget(recompute_argument)
-            self.setItemWidget(widget_item, 2, fix_layout)
-
-        widget_item = QtWidgets.QTreeWidgetItem()
-        self.addTopLevelItem(widget_item)
         recompute_arguments = QtWidgets.QPushButton("Recompute all arguments")
         recompute_arguments.setIcon(
             QtWidgets.QApplication.style().standardIcon(
@@ -108,21 +47,7 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         buttons.addWidget(load, 1, 2)
         for i, s in enumerate((1, 0, 0, 1)):
             buttons.layout.setColumnStretch(i, s)
-        self.setItemWidget(widget_item, 1, buttons)
-
-    def _get_group(self, name):
-        if name in self._groups:
-            return self._groups[name]
-        group = QtWidgets.QTreeWidgetItem([name])
-        for col in range(3):
-            group.setBackground(col, self.palette().mid())
-            group.setForeground(col, self.palette().brightText())
-            font = group.font(col)
-            font.setBold(True)
-            group.setFont(col, font)
-        self.addTopLevelItem(group)
-        self._groups[name] = group
-        return group
+        self.setItemWidget(self.bottom_item, 1, buttons)
 
     def _load_clicked(self):
         asyncio.ensure_future(self._dock.load_hdf5_task())
@@ -130,8 +55,8 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
     def _recompute_arguments_clicked(self):
         asyncio.ensure_future(self._dock._recompute_arguments())
 
-    def _recompute_argument_clicked(self, name):
-        asyncio.ensure_future(self._recompute_argument(name))
+    def reset_entry(self, key):
+        asyncio.ensure_future(self._recompute_argument(key))
 
     async def _recompute_argument(self, name):
         try:
@@ -146,29 +71,7 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         state = procdesc_to_entry(procdesc).default_state(procdesc)
         argument["desc"] = procdesc
         argument["state"] = state
-
-        widgets = self._arg_to_widgets[name]
-
-        widgets["entry"].deleteLater()
-        widgets["entry"] = procdesc_to_entry(procdesc)(argument)
-        widgets["fix_layout"] = LayoutWidget()
-        widgets["fix_layout"].addWidget(widgets["entry"])
-        self.setItemWidget(widgets["widget_item"], 1, widgets["fix_layout"])
-        self.updateGeometries()
-
-    def save_state(self):
-        expanded = []
-        for k, v in self._groups.items():
-            if v.isExpanded():
-                expanded.append(k)
-        return {"expanded": expanded}
-
-    def restore_state(self, state):
-        for e in state["expanded"]:
-            try:
-                self._groups[e].setExpanded(True)
-            except KeyError:
-                pass
+        self.update_argument(name, argument)
 
 
 log_levels = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
@@ -277,8 +180,8 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
         state = self.argeditor.save_state()
         self.argeditor.deleteLater()
         self.argeditor = _ArgumentEditor(self)
-        self.argeditor.restore_state(state)
         self.layout.addWidget(self.argeditor, 0, 0, 1, 5)
+        self.argeditor.restore_state(state)
 
     async def load_hdf5_task(self, filename=None):
         if filename is None:
@@ -466,6 +369,8 @@ class ExperimentsArea(QtWidgets.QMdiArea):
     def initialize_submission_arguments(self, arginfo):
         arguments = OrderedDict()
         for name, (procdesc, group, tooltip) in arginfo.items():
+            if procdesc["ty"] == "EnumerationValue" and procdesc["quickstyle"]:
+                procdesc["quickstyle"] = False
             state = procdesc_to_entry(procdesc).default_state(procdesc)
             arguments[name] = {
                 "desc": procdesc,

artiq/browser/files.py

@@ -83,7 +83,7 @@ class ZoomIconView(QtWidgets.QListView):
             w = self.iconSize().width()*self.zoom_step**(
                 ev.angleDelta().y()/120.)
             if a <= w <= b:
-                self.setIconSize(QtCore.QSize(w, w*self.aspect))
+                self.setIconSize(QtCore.QSize(int(w), int(w*self.aspect)))
         else:
             QtWidgets.QListView.wheelEvent(self, ev)
 

artiq/compiler/builtins.py

@@ -253,6 +253,12 @@ def fn_subkernel_await():
 def fn_subkernel_preload():
     return types.TBuiltinFunction("subkernel_preload")
 
+def fn_subkernel_send():
+    return types.TBuiltinFunction("subkernel_send")
+
+def fn_subkernel_recv():
+    return types.TBuiltinFunction("subkernel_recv")
+
 # Accessors
 
 def is_none(typ):

artiq/compiler/embedding.py

@@ -47,8 +47,15 @@ class SpecializedFunction:
         return hash((self.instance_type, self.host_function))
 
 
+class SubkernelMessageType:
+    def __init__(self, name, value_type):
+        self.name = name
+        self.value_type = value_type
+        self.send_loc = None
+        self.recv_loc = None
+
 class EmbeddingMap:
-    def __init__(self):
+    def __init__(self, old_embedding_map=None):
         self.object_current_key = 0
         self.object_forward_map = {}
         self.object_reverse_map = {}
@@ -65,6 +72,22 @@ class EmbeddingMap:
         self.str_forward_map = {}
         self.str_reverse_map = {}
 
+        # mapping `name` to object ID
+        self.subkernel_message_map = {}
+
+        # subkernels: dict of ID: function, just like object_forward_map
+        # allow the embedding map to be aware of subkernels from other kernels
+        if not old_embedding_map is None:
+            for key, obj_ref in old_embedding_map.subkernels().items():
+                self.object_forward_map[key] = obj_ref
+                obj_id = id(obj_ref)
+                self.object_reverse_map[obj_id] = key
+            for msg_id, msg_type in old_embedding_map.subkernel_messages().items():
+                self.object_forward_map[msg_id] = msg_type
+                obj_id = id(msg_type)
+                self.subkernel_message_map[msg_type.name] = msg_id
+                self.object_reverse_map[obj_id] = msg_id
+
         self.preallocate_runtime_exception_names(["RuntimeError",
                                                   "RTIOUnderflow",
                                                   "RTIOOverflow",
@@ -165,6 +188,11 @@ class EmbeddingMap:
             return self.object_reverse_map[obj_id]
 
         self.object_current_key += 1
+        while self.object_forward_map.get(self.object_current_key):
+            # make sure there's no collisions with previously inserted subkernels
+            # their identifiers must be consistent across all kernels/subkernels
+            self.object_current_key += 1
+        
         self.object_forward_map[self.object_current_key] = obj_ref
         self.object_reverse_map[obj_id] = self.object_current_key
         return self.object_current_key
@@ -177,7 +205,7 @@ class EmbeddingMap:
             obj_ref = self.object_forward_map[obj_id]
             if isinstance(obj_ref, (pytypes.FunctionType, pytypes.MethodType,
                                     pytypes.BuiltinFunctionType, pytypes.ModuleType,
-                                    SpecializedFunction)):
+                                    SpecializedFunction, SubkernelMessageType)):
                 continue
             elif isinstance(obj_ref, type):
                 _, obj_typ = self.type_map[obj_ref]
@@ -193,6 +221,35 @@ class EmbeddingMap:
                     subkernels[k] = v
         return subkernels
 
+    def store_subkernel_message(self, name, value_type, function_type, function_loc):
+        if name in self.subkernel_message_map:
+            msg_id = self.subkernel_message_map[name]
+        else:
+            msg_id = self.store_object(SubkernelMessageType(name, value_type))
+            self.subkernel_message_map[name] = msg_id
+        subkernel_msg = self.retrieve_object(msg_id)
+        if function_type == "send":
+            subkernel_msg.send_loc = function_loc
+        elif function_type == "recv":
+            subkernel_msg.recv_loc = function_loc
+        else:
+            assert False
+        return msg_id, subkernel_msg
+
+    def subkernel_messages(self):
+        messages = {}
+        for msg_id in self.subkernel_message_map.values():
+            messages[msg_id] = self.retrieve_object(msg_id)
+        return messages
+
+    def subkernel_messages_unpaired(self):
+        unpaired = []
+        for msg_id in self.subkernel_message_map.values():
+            msg_obj = self.retrieve_object(msg_id)
+            if msg_obj.send_loc is None or msg_obj.recv_loc is None:
+                unpaired.append(msg_obj)
+        return unpaired
+
     def has_rpc(self):
         return any(filter(
                 lambda x: (inspect.isfunction(x) or inspect.ismethod(x)) and \
@@ -200,10 +257,6 @@ class EmbeddingMap:
                 self.object_forward_map.values()
             ))
 
-    def has_rpc_or_subkernel(self):
-        return any(filter(lambda x: inspect.isfunction(x) or inspect.ismethod(x),
-                          self.object_forward_map.values()))
-
 
 class ASTSynthesizer:
     def __init__(self, embedding_map, value_map, quote_function=None, expanded_from=None):
@@ -694,9 +747,9 @@ class StitchingInferencer(Inferencer):
                 if elt.__class__ == float:
                     state |= IS_FLOAT
                 elif elt.__class__ == int:
-                    if -2**31 < elt < 2**31-1:
+                    if -2**31 <= elt <= 2**31-1:
                         state |= IS_INT32
-                    elif -2**63 < elt < 2**63-1:
+                    elif -2**63 <= elt <= 2**63-1:
                         state |= IS_INT64
                     else:
                         state = -1
@@ -794,7 +847,7 @@ class TypedtreeHasher(algorithm.Visitor):
         return hash(tuple(freeze(getattr(node, field_name)) for field_name in fields))
 
 class Stitcher:
-    def __init__(self, core, dmgr, engine=None, print_as_rpc=True, destination=0, subkernel_arg_types=[]):
+    def __init__(self, core, dmgr, engine=None, print_as_rpc=True, destination=0, subkernel_arg_types=[], old_embedding_map=None):
         self.core = core
         self.dmgr = dmgr
         if engine is None:
@@ -816,7 +869,7 @@ class Stitcher:
 
         self.functions = {}
 
-        self.embedding_map = EmbeddingMap()
+        self.embedding_map = EmbeddingMap(old_embedding_map)
         self.value_map = defaultdict(lambda: [])
         self.definitely_changed = False
 

artiq/compiler/math_fns.py

@@ -61,22 +61,6 @@ unary_fp_runtime_calls = [
     ("cbrt", "cbrt"),
 ]
 
-#: float -> float numpy.* math functions lowered to runtime calls.
-unary_fp_runtime_calls = [
-    ("tan", "tan"),
-    ("arcsin", "asin"),
-    ("arccos", "acos"),
-    ("arctan", "atan"),
-    ("sinh", "sinh"),
-    ("cosh", "cosh"),
-    ("tanh", "tanh"),
-    ("arcsinh", "asinh"),
-    ("arccosh", "acosh"),
-    ("arctanh", "atanh"),
-    ("expm1", "expm1"),
-    ("cbrt", "cbrt"),
-]
-
 scipy_special_unary_runtime_calls = [
     ("erf", "erf"),
     ("erfc", "erfc"),

artiq/compiler/prelude.py

@@ -59,4 +59,6 @@ def globals():
         # ARTIQ subkernel utility functions
         "subkernel_await":     builtins.fn_subkernel_await(),
         "subkernel_preload":   builtins.fn_subkernel_preload(),
+        "subkernel_send":      builtins.fn_subkernel_send(),
+        "subkernel_recv":      builtins.fn_subkernel_recv(), 
     }

artiq/compiler/transforms/artiq_ir_generator.py

@@ -2537,7 +2537,7 @@ class ARTIQIRGenerator(algorithm.Visitor):
                 timeout = self.visit(node.args[1])
             elif len(node.args) == 1 and len(node.keywords) == 0:
                 fn = node.args[0].type
-                timeout = ir.Constant(10_000, builtins.TInt64())
+                timeout = ir.Constant(-1, builtins.TInt64())
             else:
                 assert False
             if types.is_method(fn):
@@ -2557,7 +2557,44 @@ class ARTIQIRGenerator(algorithm.Visitor):
             if types.is_method(fn):
                 fn = types.get_method_function(fn)
             sid = ir.Constant(fn.sid, builtins.TInt32())
-            return self.append(ir.Builtin("subkernel_preload", [sid], builtins.TNone()))
+            dest = ir.Constant(fn.destination, builtins.TInt32())
+            return self.append(ir.Builtin("subkernel_preload", [sid, dest], builtins.TNone()))
+        elif types.is_builtin(typ, "subkernel_send"):
+            if len(node.args) == 3 and len(node.keywords) == 0:
+                dest = self.visit(node.args[0])
+                name = node.args[1].s
+                value = self.visit(node.args[2])
+            else:
+                assert False
+            msg_id, msg = self.embedding_map.store_subkernel_message(name, value.type, "send", node.loc)
+            msg_id = ir.Constant(msg_id, builtins.TInt32())
+            if value.type != msg.value_type:
+                diag = diagnostic.Diagnostic("error",
+                    "type mismatch for subkernel message '{name}', receiver expects {recv} while sending {send}",
+                    {"name": name, "recv": msg.value_type, "send": value.type},
+                    node.loc)
+                self.engine.process(diag)
+            return self.append(ir.Builtin("subkernel_send", [msg_id, dest, value], builtins.TNone()))
+        elif types.is_builtin(typ, "subkernel_recv"):
+            if len(node.args) == 2 and len(node.keywords) == 0:
+                name = node.args[0].s
+                vartype = node.args[1].value
+                timeout = ir.Constant(-1, builtins.TInt64())
+            elif len(node.args) == 3 and len(node.keywords) == 0:
+                name = node.args[0].s
+                vartype = node.args[1].value
+                timeout = self.visit(node.args[2])
+            else:
+                assert False
+            msg_id, msg = self.embedding_map.store_subkernel_message(name, vartype, "recv", node.loc)
+            msg_id = ir.Constant(msg_id, builtins.TInt32())
+            if vartype != msg.value_type:
+                diag = diagnostic.Diagnostic("error",
+                    "type mismatch for subkernel message '{name}', receiver expects {recv} while sending {send}",
+                    {"name": name, "recv": vartype, "send": msg.value_type},
+                    node.loc)
+                self.engine.process(diag)
+            return self.append(ir.Builtin("subkernel_recv", [msg_id, timeout], vartype))
         elif types.is_exn_constructor(typ):
             return self.alloc_exn(node.type, *[self.visit(arg_node) for arg_node in node.args])
         elif types.is_constructor(typ):

artiq/compiler/transforms/inferencer.py

@@ -1343,6 +1343,57 @@ class Inferencer(algorithm.Visitor):
                                 node.loc, None)
             else:
                 diagnose(valid_forms())
+        elif types.is_builtin(typ, "subkernel_send"):
+            valid_forms = lambda: [
+                valid_form("subkernel_send(dest: numpy.int?, name: str, value: V) -> None"),
+            ]
+            self._unify(node.type, builtins.TNone(),
+                        node.loc, None)
+            if len(node.args) == 3:
+                arg0 = node.args[0]
+                if types.is_var(arg0.type):
+                    pass  # undetermined yet
+                else:
+                    if builtins.is_int(arg0.type):
+                        self._unify(arg0.type, builtins.TInt8(),
+                                    arg0.loc, None)
+                    else:
+                        diagnose(valid_forms())
+                arg1 = node.args[1]
+                self._unify(arg1.type, builtins.TStr(),
+                        arg1.loc, None)
+            else:
+                diagnose(valid_forms())
+        elif types.is_builtin(typ, "subkernel_recv"):
+            valid_forms = lambda: [
+                valid_form("subkernel_recv(name: str, value_type: type) -> value_type"),
+                valid_form("subkernel_recv(name: str, value_type: type, timeout: numpy.int64) -> value_type"),
+            ]
+            if 2 <= len(node.args) <= 3:
+                arg0 = node.args[0]
+                if types.is_var(arg0.type):
+                    pass
+                else:
+                    self._unify(arg0.type, builtins.TStr(),
+                            arg0.loc, None)
+                arg1 = node.args[1]
+                if types.is_var(arg1.type):
+                    pass
+                else:
+                    self._unify(node.type, arg1.value,
+                                node.loc, None)
+                if len(node.args) == 3:
+                    arg2 = node.args[2]
+                    if types.is_var(arg2.type):
+                        pass
+                    elif builtins.is_int(arg2.type):
+                        # promote to TInt64
+                        self._unify(arg2.type, builtins.TInt64(),
+                                    arg2.loc, None)
+                    else:
+                        diagnose(valid_forms())
+            else:
+                diagnose(valid_forms())
         else:
             assert False
 

artiq/compiler/transforms/int_monomorphizer.py

@@ -14,9 +14,9 @@ class IntMonomorphizer(algorithm.Visitor):
     def visit_NumT(self, node):
         if builtins.is_int(node.type):
             if types.is_var(node.type["width"]):
-                if -2**31 < node.n < 2**31-1:
+                if -2**31 <= node.n <= 2**31-1:
                     width = 32
-                elif -2**63 < node.n < 2**63-1:
+                elif -2**63 <= node.n <= 2**63-1:
                     width = 64
                 else:
                     diag = diagnostic.Diagnostic("error",

artiq/compiler/transforms/llvm_ir_generator.py

@@ -399,9 +399,9 @@ class LLVMIRGenerator:
             llty = ll.FunctionType(lli32, [llptr])
 
         elif name == "subkernel_send_message":
-            llty = ll.FunctionType(llvoid, [lli32, lli8, llsliceptr, llptrptr])
+            llty = ll.FunctionType(llvoid, [lli32, lli1, lli8, lli8, llsliceptr, llptrptr])
         elif name == "subkernel_load_run":
-            llty = ll.FunctionType(llvoid, [lli32, lli1])
+            llty = ll.FunctionType(llvoid, [lli32, lli8, lli1])
         elif name == "subkernel_await_finish":
             llty = ll.FunctionType(llvoid, [lli32, lli64])
         elif name == "subkernel_await_message":
@@ -1417,8 +1417,23 @@ class LLVMIRGenerator:
             return self._build_rpc_recv(insn.type, llstackptr)
         elif insn.op == "subkernel_preload":
             llsid = self.map(insn.operands[0])
-            return self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, ll.Constant(lli1, 0)], 
+            lldest = ll.Constant(lli8, insn.operands[1].value)
+            return self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, lldest, ll.Constant(lli1, 0)], 
                                 name="subkernel.preload")
+        elif insn.op == "subkernel_send":
+            llmsgid = self.map(insn.operands[0])
+            lldest = self.map(insn.operands[1])
+            return self._build_subkernel_message(llmsgid, lldest, [insn.operands[2]])
+        elif insn.op == "subkernel_recv":
+            llmsgid = self.map(insn.operands[0])
+            lltimeout = self.map(insn.operands[1])
+            lltagptr = self._build_subkernel_tags([insn.type])
+            self.llbuilder.call(self.llbuiltin("subkernel_await_message"), 
+                                [llmsgid, lltimeout, lltagptr, ll.Constant(lli8, 1), ll.Constant(lli8, 1)],
+                                name="subkernel.await.message")
+            llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [],
+                                             name="subkernel.arg.stack")
+            return self._build_rpc_recv(insn.type, llstackptr)
         else:
             assert False
 
@@ -1427,7 +1442,7 @@ class LLVMIRGenerator:
         llmax = self.map(insn.operands[1])
         lltagptr = self._build_subkernel_tags(insn.arg_types)
         return self.llbuilder.call(self.llbuiltin("subkernel_await_message"), 
-                                    [ll.Constant(lli32, 0), ll.Constant(lli64, 10_000), lltagptr, llmin, llmax],
+                                    [ll.Constant(lli32, -1), ll.Constant(lli64, 10_000), lltagptr, llmin, llmax],
                                     name="subkernel.await.args")
 
     def process_Closure(self, insn):
@@ -1579,11 +1594,8 @@ class LLVMIRGenerator:
             self.llbuilder.branch(llnormalblock)
         return llret
 
-    def _build_rpc(self, fun_loc, fun_type, args, llnormalblock, llunwindblock):
-        llservice = ll.Constant(lli32, fun_type.service)
-
+    def _build_arg_tag(self, args, call_type):
         tag = b""
-
         for arg in args:
             def arg_error_handler(typ):
                 printer = types.TypePrinter()
@@ -1592,12 +1604,18 @@ class LLVMIRGenerator:
                     {"type": printer.name(typ)},
                     arg.loc)
                 diag = diagnostic.Diagnostic("error",
-                    "type {type} is not supported in remote procedure calls",
-                    {"type": printer.name(arg.type)},
+                    "type {type} is not supported in {call_type} calls",
+                    {"type": printer.name(arg.type), "call_type": call_type},
                     arg.loc, notes=[note])
                 self.engine.process(diag)
             tag += ir.rpc_tag(arg.type, arg_error_handler)
         tag += b":"
+        return tag
+
+    def _build_rpc(self, fun_loc, fun_type, args, llnormalblock, llunwindblock):
+        llservice = ll.Constant(lli32, fun_type.service)
+
+        tag = self._build_arg_tag(args, call_type="remote procedure")
 
         def ret_error_handler(typ):
             printer = types.TypePrinter()
@@ -1660,36 +1678,26 @@ class LLVMIRGenerator:
 
     def _build_subkernel_call(self, fun_loc, fun_type, args):
         llsid = ll.Constant(lli32, fun_type.sid)
-        tag = b""
-
-        for arg in args:
-            def arg_error_handler(typ):
-                printer = types.TypePrinter()
-                note = diagnostic.Diagnostic("note",
-                    "value of type {type}",
-                    {"type": printer.name(typ)},
-                    arg.loc)
-                diag = diagnostic.Diagnostic("error",
-                    "type {type} is not supported in subkernel calls",
-                    {"type": printer.name(arg.type)},
-                    arg.loc, notes=[note])
-                self.engine.process(diag)
-            tag += ir.rpc_tag(arg.type, arg_error_handler)
-        tag += b":"
-
+        lldest = ll.Constant(lli8, fun_type.destination)
         # run the kernel first
-        self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, ll.Constant(lli1, 1)])
+        self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, lldest, ll.Constant(lli1, 1)])
+
+        if args:
+            # only send args if there's anything to send, 'self' is excluded
+            self._build_subkernel_message(llsid, lldest, args)
+
+        return llsid
+
+    def _build_subkernel_message(self, llid, lldest, args):
+        # args (or messages) are sent in the same vein as RPC
+        tag = self._build_arg_tag(args, call_type="subkernel")
 
-        # arg sent in the same vein as RPC
         llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [],
                                             name="subkernel.stack")
-
         lltag = self.llconst_of_const(ir.Constant(tag, builtins.TStr()))
         lltagptr = self.llbuilder.alloca(lltag.type)
         self.llbuilder.store(lltag, lltagptr)
 
-        if args:
-            # only send args if there's anything to send, 'self' is excluded
         llargs = self.llbuilder.alloca(llptr, ll.Constant(lli32, len(args)),
                                     name="subkernel.args")
         for index, arg in enumerate(args):
@@ -1708,11 +1716,10 @@ class LLVMIRGenerator:
 
         llargcount = ll.Constant(lli8, len(args))
 
+        llisreturn = ll.Constant(lli1, False)
         self.llbuilder.call(self.llbuiltin("subkernel_send_message"),
-                                [llsid, llargcount, lltagptr, llargs])
-            self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
-
-        return llsid
+                            [llid, llisreturn, lldest, llargcount, lltagptr, llargs])
+        return self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
 
     def _build_subkernel_return(self, insn):
         # builds a remote return.
@@ -1746,10 +1753,12 @@ class LLVMIRGenerator:
         llretslot = self.llbuilder.bitcast(llretslot, llptr)
         self.llbuilder.store(llretslot, llrets)
 
-        llsid = ll.Constant(lli32, 0)  # return goes back to master, sid is ignored
+        llsid = ll.Constant(lli32, 0)  # return goes back to the caller, sid is ignored
         lltagcount = ll.Constant(lli8, 1)  # only one thing is returned
+        llisreturn = ll.Constant(lli1, True)  # it's a return, so destination is ignored
+        lldest = ll.Constant(lli8, 0)
         self.llbuilder.call(self.llbuiltin("subkernel_send_message"),
-                            [llsid, lltagcount, lltagptr, llrets])
+                            [llsid, llisreturn, lldest, lltagcount, lltagptr, llrets])
 
     def process_Call(self, insn):
         functiontyp = insn.target_function().type

artiq/coredevice/ad9910.py

@@ -999,7 +999,7 @@ class AD9910:
         """
         if not self.cpld.sync_div:
             raise ValueError("parent cpld does not drive SYNC")
-        search_span = 31
+        search_span = 13
         # FIXME https://github.com/sinara-hw/Urukul/issues/16
         # should both be 2-4 once kasli sync_in jitter is identified
         min_window = 0

artiq/coredevice/ad9912.py

@@ -44,7 +44,11 @@ class AD9912:
         self.pll_en = pll_en
         self.pll_n = pll_n
         if pll_en:
-            sysclk = self.cpld.refclk / [1, 1, 2, 4][self.cpld.clk_div] * pll_n
+            refclk = self.cpld.refclk
+            if refclk < 11e6:
+                # use SYSCLK PLL Doubler
+                refclk = refclk * 2
+            sysclk = refclk / [1, 1, 2, 4][self.cpld.clk_div] * pll_n
         else:
             sysclk = self.cpld.refclk
         assert sysclk <= 1e9
@@ -115,6 +119,10 @@ class AD9912:
             self.write(AD9912_N_DIV, self.pll_n // 2 - 2, length=1)
             self.cpld.io_update.pulse(2 * us)
             # I_cp = 375 µA, VCO high range
+            if self.cpld.refclk < 11e6:
+                # enable SYSCLK PLL Doubler
+                self.write(AD9912_PLLCFG, 0b00001101, length=1)
+            else:
                 self.write(AD9912_PLLCFG, 0b00000101, length=1)
             self.cpld.io_update.pulse(2 * us)
         delay(1 * ms)

artiq/coredevice/adf5356.py

@@ -84,10 +84,11 @@ class ADF5356:
 
         :param blind: Do not attempt to verify presence.
         """
+        self.sync()
         if not blind:
             # MUXOUT = VDD
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 1)
-            self.sync()
+            self.write(self.regs[4])
             delay(1000 * us)
             if not self.read_muxout():
                 raise ValueError("MUXOUT not high")
@@ -95,7 +96,7 @@ class ADF5356:
 
             # MUXOUT = DGND
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 2)
-            self.sync()
+            self.write(self.regs[4])
             delay(1000 * us)
             if self.read_muxout():
                 raise ValueError("MUXOUT not low")
@@ -103,8 +104,7 @@ class ADF5356:
 
             # MUXOUT = digital lock-detect
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 6)
-        else:
-            self.sync()
+            self.write(self.regs[4])
 
     @kernel
     def set_att(self, att):

artiq/coredevice/almazny.py

@@ -1,4 +1,5 @@
 from artiq.language.core import kernel, portable
+from artiq.language.units import us
 
 from numpy import int32
 
@@ -117,11 +118,6 @@ class AlmaznyLegacy:
         )
         delay(100 * us)
 
-    @kernel
-    def _update_all_registers(self):
-        for i in range(4):
-            self._update_register(i)
-
 
 class AlmaznyChannel:
     """

artiq/coredevice/comm_analyzer.py

@@ -2,15 +2,22 @@ from operator import itemgetter
 from collections import namedtuple
 from itertools import count
 from contextlib import contextmanager
+from sipyco import keepalive
+import asyncio
 from enum import Enum
 import struct
 import logging
 import socket
+import math
 
 
 logger = logging.getLogger(__name__)
 
 
+DEFAULT_REF_PERIOD = 1e-9
+ANALYZER_MAGIC = b"ARTIQ Analyzer Proxy\n"
+
+
 class MessageType(Enum):
     output = 0b00
     input = 0b01
@@ -34,6 +41,13 @@ class ExceptionType(Enum):
     i_overflow = 0b100001
 
 
+class WaveformType(Enum):
+    ANALOG = 0
+    BIT = 1
+    VECTOR = 2
+    LOG = 3
+
+
 def get_analyzer_dump(host, port=1382):
     sock = socket.create_connection((host, port))
     try:
@@ -104,6 +118,8 @@ def decode_dump(data):
     (sent_bytes, total_byte_count,
      error_occurred, log_channel, dds_onehot_sel) = parts
 
+    logger.debug("analyzer dump has length %d", sent_bytes)
+
     expected_len = sent_bytes + 15
     if expected_len != len(data):
         raise ValueError("analyzer dump has incorrect length "
@@ -115,15 +131,83 @@ def decode_dump(data):
     if total_byte_count > sent_bytes:
         logger.info("analyzer ring buffer has wrapped %d times",
                     total_byte_count//sent_bytes)
+    if sent_bytes == 0:
+        logger.warning("analyzer dump is empty")
 
     position = 15
     messages = []
     for _ in range(sent_bytes//32):
         messages.append(decode_message(data[position:position+32]))
         position += 32
+
+    if len(messages) == 1 and isinstance(messages[0], StoppedMessage):
+        logger.warning("analyzer dump is empty aside from stop message")
+
     return DecodedDump(log_channel, bool(dds_onehot_sel), messages)
 
 
+# simplified from sipyco broadcast Receiver
+class AnalyzerProxyReceiver:
+    def __init__(self, receive_cb, disconnect_cb=None):
+        self.receive_cb = receive_cb
+        self.disconnect_cb = disconnect_cb
+
+    async def connect(self, host, port):
+        self.reader, self.writer = \
+            await keepalive.async_open_connection(host, port)
+        try:
+            line = await self.reader.readline()
+            assert line == ANALYZER_MAGIC
+            self.receive_task = asyncio.create_task(self._receive_cr())
+        except:
+            self.writer.close()
+            del self.reader
+            del self.writer
+            raise
+
+    async def close(self):
+        self.disconnect_cb = None
+        try:
+            self.receive_task.cancel()
+            try:
+                await self.receive_task
+            except asyncio.CancelledError:
+                pass
+        finally:
+            self.writer.close()
+            del self.reader
+            del self.writer
+
+    async def _receive_cr(self):
+        try:
+            while True:
+                endian_byte = await self.reader.read(1)
+                if endian_byte == b"E":
+                    endian = '>'
+                elif endian_byte == b"e":
+                    endian = '<'
+                elif endian_byte == b"":
+                    # EOF reached, connection lost
+                    return
+                else:
+                    raise ValueError
+                payload_length_word = await self.reader.readexactly(4)
+                payload_length = struct.unpack(endian + "I", payload_length_word)[0]
+                if payload_length > 10 * 512 * 1024:
+                    # 10x buffer size of firmware
+                    raise ValueError
+
+                # The remaining header length is 11 bytes.
+                remaining_data = await self.reader.readexactly(payload_length + 11)
+                data = endian_byte + payload_length_word + remaining_data
+                self.receive_cb(data)
+        except Exception:
+            logger.error("analyzer receiver connection terminating with exception", exc_info=True)
+        finally:
+            if self.disconnect_cb is not None:
+                self.disconnect_cb()
+
+
 def vcd_codes():
     codechars = [chr(i) for i in range(33, 127)]
     for n in count():
@@ -150,38 +234,129 @@ class VCDChannel:
         integer_cast = struct.unpack(">Q", struct.pack(">d", x))[0]
         self.set_value("{:064b}".format(integer_cast))
 
+    def set_log(self, log_message):
+        value = ""
+        for c in log_message:
+            value += "{:08b}".format(ord(c))
+        self.set_value(value)
+
 
 class VCDManager:
     def __init__(self, fileobj):
         self.out = fileobj
         self.codes = vcd_codes()
         self.current_time = None
+        self.start_time = 0
 
     def set_timescale_ps(self, timescale):
         self.out.write("$timescale {}ps $end\n".format(round(timescale)))
 
-    def get_channel(self, name, width):
+    def get_channel(self, name, width, ty, precision=0, unit=""):
         code = next(self.codes)
         self.out.write("$var wire {width} {code} {name} $end\n"
                        .format(name=name, code=code, width=width))
         return VCDChannel(self.out, code)
 
     @contextmanager
-    def scope(self, name):
-        self.out.write("$scope module {} $end\n".format(name))
+    def scope(self, scope, name):
+        self.out.write("$scope module {}/{} $end\n".format(scope, name))
         yield
         self.out.write("$upscope $end\n")
 
     def set_time(self, time):
+        time -= self.start_time
         if time != self.current_time:
             self.out.write("#{}\n".format(time))
             self.current_time = time
 
+    def set_start_time(self, time):
+        self.start_time = time
+
+    def set_end_time(self, time):
+        pass
+
+
+class WaveformManager:
+    def __init__(self):
+        self.current_time = 0
+        self.start_time = 0
+        self.end_time = 0
+        self.channels = list()
+        self.current_scope = ""
+        self.trace = {"timescale": 1, "stopped_x": None, "logs": dict(), "data": dict()}
+
+    def set_timescale_ps(self, timescale):
+        self.trace["timescale"] = int(timescale)
+
+    def get_channel(self, name, width, ty, precision=0, unit=""):
+        if ty == WaveformType.LOG:
+            self.trace["logs"][self.current_scope + name] = (ty, width, precision, unit)
+        data = self.trace["data"][self.current_scope + name] = list()
+        channel = WaveformChannel(data, self.current_time)
+        self.channels.append(channel)
+        return channel
+
+    @contextmanager
+    def scope(self, scope, name):
+        old_scope = self.current_scope
+        self.current_scope = scope + "/"
+        yield
+        self.current_scope = old_scope
+
+    def set_time(self, time):
+        time -= self.start_time
+        for channel in self.channels:
+            channel.set_time(time)
+
+    def set_start_time(self, time):
+        self.start_time = time
+        if self.trace["stopped_x"] is not None:
+            self.trace["stopped_x"] = self.end_time - self.start_time
+
+    def set_end_time(self, time):
+        self.end_time = time
+        self.trace["stopped_x"] = self.end_time - self.start_time
+
+
+class WaveformChannel:
+    def __init__(self, data, current_time):
+        self.data = data
+        self.current_time = current_time
+
+    def set_value(self, value):
+        self.data.append((self.current_time, value))
+
+    def set_value_double(self, x):
+        self.data.append((self.current_time, x))
+
+    def set_time(self, time):
+        self.current_time = time
+
+    def set_log(self, log_message):
+        self.data.append((self.current_time, log_message))
+
+
+class ChannelSignatureManager:
+    def __init__(self):
+        self.current_scope = ""
+        self.channels = dict()
+
+    def get_channel(self, name, width, ty, precision=0, unit=""):
+        self.channels[self.current_scope + name] = (ty, width, precision, unit)
+        return None
+
+    @contextmanager
+    def scope(self, scope, name):
+        old_scope = self.current_scope
+        self.current_scope = scope + "/"
+        yield
+        self.current_scope = old_scope
+
 
 class TTLHandler:
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self.name = name
-        self.channel_value = vcd_manager.get_channel("ttl/" + name, 1)
+        self.channel_value = manager.get_channel("ttl/" + name, 1, ty=WaveformType.BIT)
         self.last_value = "X"
         self.oe = True
 
@@ -206,11 +381,12 @@ class TTLHandler:
 
 
 class TTLClockGenHandler:
-    def __init__(self, vcd_manager, name, ref_period):
+    def __init__(self, manager, name, ref_period):
         self.name = name
         self.ref_period = ref_period
-        self.channel_frequency = vcd_manager.get_channel(
-            "ttl_clkgen/" + name, 64)
+        precision = max(0, math.ceil(math.log10(2**24 * ref_period) + 6))
+        self.channel_frequency = manager.get_channel(
+            "ttl_clkgen/" + name, 64, ty=WaveformType.ANALOG, precision=precision, unit="MHz")
 
     def process_message(self, message):
         if isinstance(message, OutputMessage):
@@ -221,8 +397,8 @@ class TTLClockGenHandler:
 
 
 class DDSHandler:
-    def __init__(self, vcd_manager, onehot_sel, sysclk):
-        self.vcd_manager = vcd_manager
+    def __init__(self, manager, onehot_sel, sysclk):
+        self.manager = manager
         self.onehot_sel = onehot_sel
         self.sysclk = sysclk
 
@@ -231,11 +407,18 @@ class DDSHandler:
 
     def add_dds_channel(self, name, dds_channel_nr):
         dds_channel = dict()
-        with self.vcd_manager.scope("dds/{}".format(name)):
+        frequency_precision = max(0, math.ceil(math.log10(2**32 / self.sysclk) + 6))
+        phase_precision = max(0, math.ceil(math.log10(2**16)))
+        with self.manager.scope("dds", name):
             dds_channel["vcd_frequency"] = \
-                self.vcd_manager.get_channel(name + "/frequency", 64)
+                self.manager.get_channel(name + "/frequency", 64, 
+                                         ty=WaveformType.ANALOG, 
+                                         precision=frequency_precision,
+                                         unit="MHz")
             dds_channel["vcd_phase"] = \
-                self.vcd_manager.get_channel(name + "/phase", 64)
+                self.manager.get_channel(name + "/phase", 64, 
+                                         ty=WaveformType.ANALOG,
+                                         precision=phase_precision)
         dds_channel["ftw"] = [None, None]
         dds_channel["pow"] = None
         self.dds_channels[dds_channel_nr] = dds_channel
@@ -285,10 +468,10 @@ class DDSHandler:
 
 
 class WishboneHandler:
-    def __init__(self, vcd_manager, name, read_bit):
+    def __init__(self, manager, name, read_bit):
         self._reads = []
         self._read_bit = read_bit
-        self.stb = vcd_manager.get_channel("{}/{}".format(name, "stb"), 1)
+        self.stb = manager.get_channel(name + "/stb", 1, ty=WaveformType.BIT)
 
     def process_message(self, message):
         self.stb.set_value("1")
@@ -318,16 +501,17 @@ class WishboneHandler:
 
 
 class SPIMasterHandler(WishboneHandler):
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self.channels = {}
-        with vcd_manager.scope("spi/{}".format(name)):
-            super().__init__(vcd_manager, name, read_bit=0b100)
+        self.scope = "spi"
+        with manager.scope("spi", name):
+            super().__init__(manager, name, read_bit=0b100)
             for reg_name, reg_width in [
                     ("config", 32), ("chip_select", 16),
                     ("write_length", 8), ("read_length", 8),
                     ("write", 32), ("read", 32)]:
-                self.channels[reg_name] = vcd_manager.get_channel(
-                        "{}/{}".format(name, reg_name), reg_width)
+                self.channels[reg_name] = manager.get_channel(
+                    "{}/{}".format(name, reg_name), reg_width, ty=WaveformType.VECTOR)
 
     def process_write(self, address, data):
         if address == 0:
@@ -352,11 +536,12 @@ class SPIMasterHandler(WishboneHandler):
 
 
 class SPIMaster2Handler(WishboneHandler):
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self._reads = []
         self.channels = {}
-        with vcd_manager.scope("spi2/{}".format(name)):
-            self.stb = vcd_manager.get_channel("{}/{}".format(name, "stb"), 1)
+        self.scope = "spi2"
+        with manager.scope("spi2", name):
+            self.stb = manager.get_channel(name + "/stb", 1, ty=WaveformType.BIT)
             for reg_name, reg_width in [
                     ("flags", 8),
                     ("length", 5),
@@ -364,8 +549,8 @@ class SPIMaster2Handler(WishboneHandler):
                     ("chip_select", 8),
                     ("write", 32),
                     ("read", 32)]:
-                self.channels[reg_name] = vcd_manager.get_channel(
-                        "{}/{}".format(name, reg_name), reg_width)
+                self.channels[reg_name] = manager.get_channel(
+                    "{}/{}".format(name, reg_name), reg_width, ty=WaveformType.VECTOR)
 
     def process_message(self, message):
         self.stb.set_value("1")
@@ -413,11 +598,12 @@ def _extract_log_chars(data):
 
 
 class LogHandler:
-    def __init__(self, vcd_manager, vcd_log_channels):
-        self.vcd_channels = dict()
-        for name, maxlength in vcd_log_channels.items():
-            self.vcd_channels[name] = vcd_manager.get_channel("log/" + name,
-                                                              maxlength*8)
+    def __init__(self, manager, log_channels):
+        self.channels = dict()
+        for name, maxlength in log_channels.items():
+            self.channels[name] = manager.get_channel("logs/" + name,
+                                                      maxlength * 8,
+                                                      ty=WaveformType.LOG)
         self.current_entry = ""
 
     def process_message(self, message):
@@ -425,15 +611,12 @@ class LogHandler:
             self.current_entry += _extract_log_chars(message.data)
             if len(self.current_entry) > 1 and self.current_entry[-1] == "\x1D":
                 channel_name, log_message = self.current_entry[:-1].split("\x1E", maxsplit=1)
-                vcd_value = ""
-                for c in log_message:
-                    vcd_value += "{:08b}".format(ord(c))
-                self.vcd_channels[channel_name].set_value(vcd_value)
+                self.channels[channel_name].set_log(log_message)
                 self.current_entry = ""
 
 
-def get_vcd_log_channels(log_channel, messages):
-    vcd_log_channels = dict()
+def get_log_channels(log_channel, messages):
+    log_channels = dict()
     log_entry = ""
     for message in messages:
         if (isinstance(message, OutputMessage)
@@ -442,13 +625,13 @@ def get_vcd_log_channels(log_channel, messages):
             if len(log_entry) > 1 and log_entry[-1] == "\x1D":
                 channel_name, log_message = log_entry[:-1].split("\x1E", maxsplit=1)
                 l = len(log_message)
-                if channel_name in vcd_log_channels:
-                    if vcd_log_channels[channel_name] < l:
-                        vcd_log_channels[channel_name] = l
+                if channel_name in log_channels:
+                    if log_channels[channel_name] < l:
+                        log_channels[channel_name] = l
                 else:
-                    vcd_log_channels[channel_name] = l
+                    log_channels[channel_name] = l
                 log_entry = ""
-    return vcd_log_channels
+    return log_channels
 
 
 def get_single_device_argument(devices, module, cls, argument):
@@ -475,7 +658,7 @@ def get_dds_sysclk(devices):
                                       ("AD9914",), "sysclk")
 
 
-def create_channel_handlers(vcd_manager, devices, ref_period,
+def create_channel_handlers(manager, devices, ref_period,
                             dds_sysclk, dds_onehot_sel):
     channel_handlers = dict()
     for name, desc in sorted(devices.items(), key=itemgetter(0)):
@@ -483,11 +666,11 @@ def create_channel_handlers(vcd_manager, devices, ref_period,
             if (desc["module"] == "artiq.coredevice.ttl"
                     and desc["class"] in {"TTLOut", "TTLInOut"}):
                 channel = desc["arguments"]["channel"]
-                channel_handlers[channel] = TTLHandler(vcd_manager, name)
+                channel_handlers[channel] = TTLHandler(manager, name)
             if (desc["module"] == "artiq.coredevice.ttl"
                     and desc["class"] == "TTLClockGen"):
                 channel = desc["arguments"]["channel"]
-                channel_handlers[channel] = TTLClockGenHandler(vcd_manager, name, ref_period)
+                channel_handlers[channel] = TTLClockGenHandler(manager, name, ref_period)
             if (desc["module"] == "artiq.coredevice.ad9914"
                     and desc["class"] == "AD9914"):
                 dds_bus_channel = desc["arguments"]["bus_channel"]
@@ -495,37 +678,60 @@ def create_channel_handlers(vcd_manager, devices, ref_period,
                 if dds_bus_channel in channel_handlers:
                     dds_handler = channel_handlers[dds_bus_channel]
                 else:
-                    dds_handler = DDSHandler(vcd_manager, dds_onehot_sel, dds_sysclk)
+                    dds_handler = DDSHandler(manager, dds_onehot_sel, dds_sysclk)
                     channel_handlers[dds_bus_channel] = dds_handler
                 dds_handler.add_dds_channel(name, dds_channel)
             if (desc["module"] == "artiq.coredevice.spi2" and
                     desc["class"] == "SPIMaster"):
                 channel = desc["arguments"]["channel"]
                 channel_handlers[channel] = SPIMaster2Handler(
-                        vcd_manager, name)
+                        manager, name)
     return channel_handlers
 
 
+def get_channel_list(devices):
+    manager = ChannelSignatureManager()
+    create_channel_handlers(manager, devices, 1e-9, 3e9, False)
+    ref_period = get_ref_period(devices)
+    if ref_period is None:
+        ref_period = DEFAULT_REF_PERIOD
+    precision = max(0, math.ceil(math.log10(1 / ref_period) - 6))
+    manager.get_channel("rtio_slack", 64, ty=WaveformType.ANALOG, precision=precision, unit="us")
+    return manager.channels
+
+
 def get_message_time(message):
     return getattr(message, "timestamp", message.rtio_counter)
 
 
 def decoded_dump_to_vcd(fileobj, devices, dump, uniform_interval=False):
     vcd_manager = VCDManager(fileobj)
+    decoded_dump_to_target(vcd_manager, devices, dump, uniform_interval)
+
+
+def decoded_dump_to_waveform_data(devices, dump, uniform_interval=False):
+    manager = WaveformManager()
+    decoded_dump_to_target(manager, devices, dump, uniform_interval)
+    return manager.trace
+
+
+def decoded_dump_to_target(manager, devices, dump, uniform_interval):
     ref_period = get_ref_period(devices)
 
-    if ref_period is not None:
-        if not uniform_interval:
-            vcd_manager.set_timescale_ps(ref_period*1e12)
-    else:
+    if ref_period is None:
         logger.warning("unable to determine core device ref_period")
-        ref_period = 1e-9  # guess
+        ref_period = DEFAULT_REF_PERIOD
+    if not uniform_interval:
+        manager.set_timescale_ps(ref_period*1e12)
     dds_sysclk = get_dds_sysclk(devices)
     if dds_sysclk is None:
         logger.warning("unable to determine DDS sysclk")
         dds_sysclk = 3e9  # guess
 
     if isinstance(dump.messages[-1], StoppedMessage):
+        m = dump.messages[-1]
+        end_time = get_message_time(m)
+        manager.set_end_time(end_time)
         messages = dump.messages[:-1]
     else:
         logger.warning("StoppedMessage missing")
@@ -533,38 +739,39 @@ def decoded_dump_to_vcd(fileobj, devices, dump, uniform_interval=False):
     messages = sorted(messages, key=get_message_time)
 
     channel_handlers = create_channel_handlers(
-        vcd_manager, devices, ref_period,
+        manager, devices, ref_period,
         dds_sysclk, dump.dds_onehot_sel)
-    vcd_log_channels = get_vcd_log_channels(dump.log_channel, messages)
+    log_channels = get_log_channels(dump.log_channel, messages)
     channel_handlers[dump.log_channel] = LogHandler(
-        vcd_manager, vcd_log_channels)
+        manager, log_channels)
     if uniform_interval:
         # RTIO event timestamp in machine units
-        timestamp = vcd_manager.get_channel("timestamp", 64)
+        timestamp = manager.get_channel("timestamp", 64, ty=WaveformType.VECTOR)
         # RTIO time interval between this and the next timed event
         # in SI seconds
-        interval = vcd_manager.get_channel("interval", 64)
-    slack = vcd_manager.get_channel("rtio_slack", 64)
+        interval = manager.get_channel("interval", 64, ty=WaveformType.ANALOG)
+    slack = manager.get_channel("rtio_slack", 64, ty=WaveformType.ANALOG)
 
-    vcd_manager.set_time(0)
+    manager.set_time(0)
     start_time = 0
     for m in messages:
         start_time = get_message_time(m)
         if start_time:
             break
-
-    t0 = 0
+    if not uniform_interval:
+        manager.set_start_time(start_time)
+    t0 = start_time
     for i, message in enumerate(messages):
         if message.channel in channel_handlers:
-            t = get_message_time(message) - start_time
+            t = get_message_time(message)
             if t >= 0:
                 if uniform_interval:
                     interval.set_value_double((t - t0)*ref_period)
-                    vcd_manager.set_time(i)
+                    manager.set_time(i)
                     timestamp.set_value("{:064b}".format(t))
                     t0 = t
                 else:
-                    vcd_manager.set_time(t)
+                    manager.set_time(t)
             channel_handlers[message.channel].process_message(message)
             if isinstance(message, OutputMessage):
                 slack.set_value_double(

artiq/coredevice/comm_kernel.py

@@ -465,12 +465,12 @@ class CommKernel:
             self._write_bool(value)
         elif tag == "i":
             check(isinstance(value, (int, numpy.int32)) and
-                  (-2**31 <= value < 2**31),
+                  (-2**31 <= value <= 2**31-1),
                   lambda: "32-bit int")
             self._write_int32(value)
         elif tag == "I":
             check(isinstance(value, (int, numpy.int32, numpy.int64)) and
-                  (-2**63 <= value < 2**63),
+                  (-2**63 <= value <= 2**63-1),
                   lambda: "64-bit int")
             self._write_int64(value)
         elif tag == "f":
@@ -479,8 +479,8 @@ class CommKernel:
             self._write_float64(value)
         elif tag == "F":
             check(isinstance(value, Fraction) and
-                  (-2**63 <= value.numerator < 2**63) and
-                  (-2**63 <= value.denominator < 2**63),
+                  (-2**63 <= value.numerator <= 2**63-1) and
+                  (-2**63 <= value.denominator <= 2**63-1),
                   lambda: "64-bit Fraction")
             self._write_int64(value.numerator)
             self._write_int64(value.denominator)

artiq/coredevice/comm_moninj.py

@@ -94,9 +94,7 @@ class CommMonInj:
                     self.injection_status_cb(channel, override, value)
                 else:
                     raise ValueError("Unknown packet type", ty)
-        except asyncio.CancelledError:
-            raise
-        except:
+        except Exception:
             logger.error("Moninj connection terminating with exception", exc_info=True)
         finally:
             if self.disconnect_cb is not None:

artiq/coredevice/core.py

@@ -120,13 +120,15 @@ class Core:
 
     def compile(self, function, args, kwargs, set_result=None,
                 attribute_writeback=True, print_as_rpc=True,
-                target=None, destination=0, subkernel_arg_types=[]):
+                target=None, destination=0, subkernel_arg_types=[],
+                old_embedding_map=None):
         try:
             engine = _DiagnosticEngine(all_errors_are_fatal=True)
 
             stitcher = Stitcher(engine=engine, core=self, dmgr=self.dmgr,
                                 print_as_rpc=print_as_rpc,
-                                destination=destination, subkernel_arg_types=subkernel_arg_types)
+                                destination=destination, subkernel_arg_types=subkernel_arg_types,
+                                old_embedding_map=old_embedding_map)
             stitcher.stitch_call(function, args, kwargs, set_result)
             stitcher.finalize()
 
@@ -165,7 +167,7 @@ class Core:
         self._run_compiled(kernel_library, embedding_map, symbolizer, demangler)
         return result
 
-    def compile_subkernel(self, sid, subkernel_fn, embedding_map, args, subkernel_arg_types):
+    def compile_subkernel(self, sid, subkernel_fn, embedding_map, args, subkernel_arg_types, subkernels):
         # pass self to subkernels (if applicable)
         # assuming the first argument is self
         subkernel_args = getfullargspec(subkernel_fn.artiq_embedded.function)
@@ -179,17 +181,49 @@ class Core:
         object_map, kernel_library, _, _, _ = \
             self.compile(subkernel_fn, self_arg, {}, attribute_writeback=False,
                         print_as_rpc=False, target=target, destination=destination, 
-                        subkernel_arg_types=subkernel_arg_types.get(sid, []))
-        if object_map.has_rpc_or_subkernel():
-            raise ValueError("Subkernel must not use RPC or subkernels in other destinations")
-        return destination, kernel_library
+                        subkernel_arg_types=subkernel_arg_types.get(sid, []),
+                        old_embedding_map=embedding_map)
+        if object_map.has_rpc():
+            raise ValueError("Subkernel must not use RPC")
+        return destination, kernel_library, object_map
 
     def compile_and_upload_subkernels(self, embedding_map, args, subkernel_arg_types):
-        for sid, subkernel_fn in embedding_map.subkernels().items():
-            destination, kernel_library = \
+        subkernels = embedding_map.subkernels()
+        subkernels_compiled = []
+        while True:
+            new_subkernels = {}
+            for sid, subkernel_fn in subkernels.items():
+                if sid in subkernels_compiled:
+                    continue
+                destination, kernel_library, embedding_map = \
                     self.compile_subkernel(sid, subkernel_fn, embedding_map,
-                                       args, subkernel_arg_types)
+                                        args, subkernel_arg_types, subkernels)
                 self.comm.upload_subkernel(kernel_library, sid, destination)
+                new_subkernels.update(embedding_map.subkernels())
+                subkernels_compiled.append(sid)
+            if new_subkernels == subkernels:
+                break
+            subkernels.update(new_subkernels)
+        # check for messages without a send/recv pair
+        unpaired_messages = embedding_map.subkernel_messages_unpaired()
+        if unpaired_messages:
+            for unpaired_message in unpaired_messages:
+                engine = _DiagnosticEngine(all_errors_are_fatal=False)
+                # errors are non-fatal in order to display
+                # all unpaired message errors before raising an excption
+                if unpaired_message.send_loc is None:
+                    diag = diagnostic.Diagnostic("error",
+                        "subkernel message '{name}' only has a receiver but no sender",
+                        {"name": unpaired_message.name},
+                        unpaired_message.recv_loc)
+                else:
+                    diag = diagnostic.Diagnostic("error",
+                        "subkernel message '{name}' only has a sender but no receiver",
+                        {"name": unpaired_message.name},
+                        unpaired_message.send_loc)
+                engine.process(diag)
+            raise ValueError("Found subkernel message(s) without a full send/recv pair")
+
 
     def precompile(self, function, *args, **kwargs):
         """Precompile a kernel and return a callable that executes it on the core device
@@ -319,6 +353,4 @@ class Core:
         if self.analyzer_proxy is None:
             raise IOError("No analyzer proxy configured")
         else:
-            success = self.analyzer_proxy.trigger()
-            if not success:
-                raise IOError("Analyzer proxy reported failure")
+            self.analyzer_proxy.trigger()

artiq/coredevice/coredevice_generic.schema.json

@@ -49,6 +49,10 @@
             "default": 125e6,
             "description": "RTIO frequency"
         },
+        "enable_wrpll": {
+            "type": "boolean",
+            "default": false
+        },
         "core_addr": {
             "type": "string",
             "format": "ipv4",
@@ -308,8 +312,7 @@
                         "clk_div": {
                             "type": "integer",
                             "minimum": 0,
-                            "maximum": 3,
-                            "default": 0
+                            "maximum": 3
                         },
                         "pll_n": {
                             "type": "integer"
@@ -631,6 +634,10 @@
                         },
                         "drtio_destination": {
                             "type": "integer"
+                        },
+                        "hw_rev": {
+                            "type": "string",
+                            "enum": ["v1.0", "v1.1"]
                         }
                     },
                     "required": ["ports"]

artiq/coredevice/grabber.py

@@ -2,7 +2,7 @@ from numpy import int32, int64
 
 from artiq.language.core import *
 from artiq.language.types import *
-from artiq.coredevice.rtio import rtio_output, rtio_input_data
+from artiq.coredevice.rtio import rtio_output, rtio_input_timestamped_data
 
 
 class OutOfSyncException(Exception):
@@ -11,6 +11,11 @@ class OutOfSyncException(Exception):
     pass
 
 
+class GrabberTimeoutException(Exception):
+    """Raised when a timeout occurs while attempting to read Grabber RTIO input events."""
+    pass
+
+
 class Grabber:
     """Driver for the Grabber camera interface."""
     kernel_invariants = {"core", "channel_base", "sentinel"}
@@ -82,10 +87,10 @@ class Grabber:
         self.gate_roi(0)
 
     @kernel
-    def input_mu(self, data):
+    def input_mu(self, data, timeout_mu=-1):
         """
         Retrieves the accumulated values for one frame from the ROI engines.
-        Blocks until values are available.
+        Blocks until values are available or timeout is reached.
 
         The input list must be a list of integers of the same length as there
         are enabled ROI engines. This method replaces the elements of the
@@ -95,15 +100,26 @@ class Grabber:
         If the number of elements in the list does not match the number of
         ROI engines that produced output, an exception will be raised during
         this call or the next.
+        
+        If the timeout is reached before data is available, the exception
+        GrabberTimeoutException is raised.
+
+        :param timeout_mu: Timestamp at which a timeout will occur. Set to -1
+                           (default) to disable timeout. 
         """
         channel = self.channel_base + 1
 
-        sentinel = rtio_input_data(channel)
+        timestamp, sentinel = rtio_input_timestamped_data(timeout_mu, channel)
+        if timestamp == -1:
+            raise GrabberTimeoutException("Timeout before Grabber frame available")
         if sentinel != self.sentinel:
             raise OutOfSyncException
 
         for i in range(len(data)):
-            roi_output = rtio_input_data(channel)
+            timestamp, roi_output = rtio_input_timestamped_data(timeout_mu, channel)
             if roi_output == self.sentinel:
                 raise OutOfSyncException
+            if timestamp == -1:
+                raise GrabberTimeoutException(
+                    "Timeout retrieving ROIs (attempting to read more ROIs than enabled?)")
             data[i] = roi_output

artiq/coredevice/kasli_i2c.py

@@ -25,14 +25,14 @@ port_mapping = {
 
 
 class KasliEEPROM:
-    def __init__(self, dmgr, port, busno=0,
+    def __init__(self, dmgr, port, address=0xa0, busno=0,
             core_device="core", sw0_device="i2c_switch0", sw1_device="i2c_switch1"):
         self.core = dmgr.get(core_device)
         self.sw0 = dmgr.get(sw0_device)
         self.sw1 = dmgr.get(sw1_device)
         self.busno = busno
         self.port = port_mapping[port]
-        self.address = 0xa0  # i2c 8 bit
+        self.address = address  # i2c 8 bit
 
     @kernel
     def select(self):

artiq/dashboard/datasets.py

@@ -8,7 +8,6 @@ from sipyco import pyon
 from artiq.tools import scale_from_metadata, short_format, exc_to_warning
 from artiq.gui.tools import LayoutWidget, QRecursiveFilterProxyModel
 from artiq.gui.models import DictSyncTreeSepModel
-from artiq.gui.scientific_spinbox import ScientificSpinBox
 
 
 logger = logging.getLogger(__name__)
@@ -81,8 +80,7 @@ class CreateEditDialog(QtWidgets.QDialog):
             t = value.dtype if value is np.ndarray else type(value)
             if scale != 1 and np.issubdtype(t, np.number):
                 # degenerates to float type
-                value_edit_string = self.value_to_edit_string(
-                        np.float64(value / scale))
+                value_edit_string = self.value_to_edit_string(value / scale)
             self.unit_widget.setText(metadata.get('unit', ''))
             self.scale_widget.setText(str(metadata.get('scale', '')))
             self.precision_widget.setText(str(metadata.get('precision', '')))
@@ -109,11 +107,13 @@ class CreateEditDialog(QtWidgets.QDialog):
         t = value.dtype if value is np.ndarray else type(value)
         if scale != 1 and np.issubdtype(t, np.number):
             # degenerates to float type
-            value = np.float64(value * scale)
+            value = float(value * scale)
         if self.key and self.key != key:
-            asyncio.ensure_future(exc_to_warning(rename(self.key, key, value, metadata, persist, self.dataset_ctl)))
+            asyncio.ensure_future(exc_to_warning(rename(self.key, key, value, metadata, persist,
+                                                        self.dataset_ctl)))
         else:
-            asyncio.ensure_future(exc_to_warning(self.dataset_ctl.set(key, value, metadata=metadata, persist=persist)))
+            asyncio.ensure_future(exc_to_warning(self.dataset_ctl.set(key, value, metadata=metadata,
+                                                                      persist=persist)))
         self.key = key
         QtWidgets.QDialog.accept(self)
 

artiq/dashboard/experiments.py

@@ -9,10 +9,9 @@ import h5py
 
 from sipyco import pyon
 
-from artiq.gui.entries import procdesc_to_entry, ScanEntry
+from artiq.gui.entries import procdesc_to_entry, EntryTreeWidget
 from artiq.gui.fuzzy_select import FuzzySelectWidget
-from artiq.gui.tools import (LayoutWidget, WheelFilter, 
-                             log_level_to_name, get_open_file_name)
+from artiq.gui.tools import (LayoutWidget, log_level_to_name, get_open_file_name)
 from artiq.tools import parse_devarg_override, unparse_devarg_override
 
 
@@ -25,99 +24,23 @@ logger = logging.getLogger(__name__)
 # 2. file:<class name>@<file name>
 
 
-class _ArgumentEditor(QtWidgets.QTreeWidget):
+class _ArgumentEditor(EntryTreeWidget):
     def __init__(self, manager, dock, expurl):
         self.manager = manager
         self.expurl = expurl
 
-        QtWidgets.QTreeWidget.__init__(self)
-        self.setColumnCount(3)
-        self.header().setStretchLastSection(False)
-        if hasattr(self.header(), "setSectionResizeMode"):
-            set_resize_mode = self.header().setSectionResizeMode
-        else:
-            set_resize_mode = self.header().setResizeMode
-        set_resize_mode(0, QtWidgets.QHeaderView.ResizeToContents)
-        set_resize_mode(1, QtWidgets.QHeaderView.Stretch)
-        set_resize_mode(2, QtWidgets.QHeaderView.ResizeToContents)
-        self.header().setVisible(False)
-        self.setSelectionMode(self.NoSelection)
-        self.setHorizontalScrollMode(self.ScrollPerPixel)
-        self.setVerticalScrollMode(self.ScrollPerPixel)
-
-        self.setStyleSheet("QTreeWidget {background: " +
-                           self.palette().midlight().color().name() + " ;}")
-
-        self.viewport().installEventFilter(WheelFilter(self.viewport(), True))
-
-        self._groups = dict()
-        self._arg_to_widgets = dict()
+        EntryTreeWidget.__init__(self)
 
         arguments = self.manager.get_submission_arguments(self.expurl)
 
         if not arguments:
-            self.addTopLevelItem(QtWidgets.QTreeWidgetItem(["No arguments"]))
+            self.insertTopLevelItem(0, QtWidgets.QTreeWidgetItem(["No arguments"]))
 
-        gradient = QtGui.QLinearGradient(
-            0, 0, 0, QtGui.QFontMetrics(self.font()).lineSpacing()*2.5)
-        gradient.setColorAt(0, self.palette().base().color())
-        gradient.setColorAt(1, self.palette().midlight().color())
         for name, argument in arguments.items():
-            widgets = dict()
-            self._arg_to_widgets[name] = widgets
+            self.set_argument(name, argument)
 
-            entry = procdesc_to_entry(argument["desc"])(argument)
-            widget_item = QtWidgets.QTreeWidgetItem([name])
-            if argument["tooltip"]:
-                widget_item.setToolTip(0, argument["tooltip"])
-            widgets["entry"] = entry
-            widgets["widget_item"] = widget_item
+        self.quickStyleClicked.connect(dock.submit_clicked)
 
-            for col in range(3):
-                widget_item.setBackground(col, gradient)
-            font = widget_item.font(0)
-            font.setBold(True)
-            widget_item.setFont(0, font)
-
-            if argument["group"] is None:
-                self.addTopLevelItem(widget_item)
-            else:
-                self._get_group(argument["group"]).addChild(widget_item)
-            fix_layout = LayoutWidget()
-            widgets["fix_layout"] = fix_layout
-            fix_layout.addWidget(entry)
-            self.setItemWidget(widget_item, 1, fix_layout)
-            recompute_argument = QtWidgets.QToolButton()
-            recompute_argument.setToolTip("Re-run the experiment's build "
-                                          "method and take the default value")
-            recompute_argument.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_BrowserReload))
-            recompute_argument.clicked.connect(
-                partial(self._recompute_argument_clicked, name))
-
-            tool_buttons = LayoutWidget()
-            tool_buttons.addWidget(recompute_argument, 1)
-
-            disable_other_scans = QtWidgets.QToolButton()
-            widgets["disable_other_scans"] = disable_other_scans
-            disable_other_scans.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_DialogResetButton))
-            disable_other_scans.setToolTip("Disable all other scans in "
-                                           "this experiment")
-            disable_other_scans.clicked.connect(
-                partial(self._disable_other_scans, name))
-            tool_buttons.layout.setRowStretch(0, 1)
-            tool_buttons.layout.setRowStretch(3, 1)
-            tool_buttons.addWidget(disable_other_scans, 2)
-            if not isinstance(entry, ScanEntry):
-                disable_other_scans.setVisible(False)
-
-            self.setItemWidget(widget_item, 2, tool_buttons)
-
-        widget_item = QtWidgets.QTreeWidgetItem()
-        self.addTopLevelItem(widget_item)
         recompute_arguments = QtWidgets.QPushButton("Recompute all arguments")
         recompute_arguments.setIcon(
             QtWidgets.QApplication.style().standardIcon(
@@ -136,41 +59,10 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         buttons.layout.setColumnStretch(1, 0)
         buttons.layout.setColumnStretch(2, 0)
         buttons.layout.setColumnStretch(3, 1)
-        self.setItemWidget(widget_item, 1, buttons)
+        self.setItemWidget(self.bottom_item, 1, buttons)
 
-    def _get_group(self, name):
-        if name in self._groups:
-            return self._groups[name]
-        group = QtWidgets.QTreeWidgetItem([name])
-        for col in range(3):
-            group.setBackground(col, self.palette().mid())
-            group.setForeground(col, self.palette().brightText())
-            font = group.font(col)
-            font.setBold(True)
-            group.setFont(col, font)
-        self.addTopLevelItem(group)
-        self._groups[name] = group
-        return group
-
-    def update_argument(self, name, argument):
-        widgets = self._arg_to_widgets[name]
-
-        # Qt needs a setItemWidget() to handle layout correctly,
-        # simply replacing the entry inside the LayoutWidget
-        # results in a bug.
-
-        widgets["entry"].deleteLater()
-        widgets["entry"] = procdesc_to_entry(argument["desc"])(argument)
-        widgets["disable_other_scans"].setVisible(
-            isinstance(widgets["entry"], ScanEntry))
-        widgets["fix_layout"].deleteLater()
-        widgets["fix_layout"] = LayoutWidget()
-        widgets["fix_layout"].addWidget(widgets["entry"])
-        self.setItemWidget(widgets["widget_item"], 1, widgets["fix_layout"])
-        self.updateGeometries()
-
-    def _recompute_argument_clicked(self, name):
-        asyncio.ensure_future(self._recompute_argument(name))
+    def reset_entry(self, key):
+        asyncio.ensure_future(self._recompute_argument(key))
 
     async def _recompute_argument(self, name):
         try:
@@ -187,30 +79,6 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         argument["state"] = state
         self.update_argument(name, argument)
 
-    def _disable_other_scans(self, current_name):
-        for name, widgets in self._arg_to_widgets.items():
-            if (name != current_name
-                    and isinstance(widgets["entry"], ScanEntry)):
-                widgets["entry"].disable()
-
-    def save_state(self):
-        expanded = []
-        for k, v in self._groups.items():
-            if v.isExpanded():
-                expanded.append(k)
-        return {
-            "expanded": expanded,
-            "scroll": self.verticalScrollBar().value()
-        }
-
-    def restore_state(self, state):
-        for e in state["expanded"]:
-            try:
-                self._groups[e].setExpanded(True)
-            except KeyError:
-                pass
-        self.verticalScrollBar().setValue(state["scroll"])
-
     # Hooks that allow user-supplied argument editors to react to imminent user
     # actions. Here, we always keep the manager-stored submission arguments
     # up-to-date, so no further action is required.
@@ -349,9 +217,10 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
             repo_rev = QtWidgets.QLineEdit()
             repo_rev.setPlaceholderText("current")
             repo_rev.setClearButtonEnabled(True)
-            repo_rev_label = QtWidgets.QLabel("Revision:")
+            repo_rev_label = QtWidgets.QLabel("Rev / ref:")
             repo_rev_label.setToolTip("Experiment repository revision "
-                                      "(commit ID) to use")
+                                      "(commit ID) or reference (branch "
+                                      "or tag) to use")
             self.layout.addWidget(repo_rev_label, 3, 2)
             self.layout.addWidget(repo_rev, 3, 3)
 
@@ -420,8 +289,7 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
         arginfo = expdesc["arginfo"]
         for k, v in overrides.items():
             # Some values (e.g. scans) may have multiple defaults in a list
-            if ("default" in arginfo[k][0]
-                    and isinstance(arginfo[k][0]["default"], list)):
+            if ("default" in arginfo[k][0] and isinstance(arginfo[k][0]["default"], list)):
                 arginfo[k][0]["default"].insert(0, v)
             else:
                 arginfo[k][0]["default"] = v
@@ -432,8 +300,8 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
 
         editor_class = self.manager.get_argument_editor_class(self.expurl)
         self.argeditor = editor_class(self.manager, self, self.expurl)
-        self.argeditor.restore_state(argeditor_state)
         self.layout.addWidget(self.argeditor, 0, 0, 1, 5)
+        self.argeditor.restore_state(argeditor_state)
 
     def contextMenuEvent(self, event):
         menu = QtWidgets.QMenu(self)
@@ -486,9 +354,9 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
                     unparse_devarg_override(expid["devarg_override"]))
             self.log_level.setCurrentIndex(log_levels.index(
                 log_level_to_name(expid["log_level"])))
-            if ("repo_rev" in expid and
-                    expid["repo_rev"] != "N/A" and
-                    hasattr(self, "repo_rev")):
+            if "repo_rev" in expid and \
+               expid["repo_rev"] != "N/A" and \
+               hasattr(self, "repo_rev"):
                 self.repo_rev.setText(expid["repo_rev"])
         except:
             logger.error("Could not set submission options from HDF5 expid",
@@ -691,7 +559,8 @@ class ExperimentManager:
             if expurl in self.open_experiments.keys():
                 self.open_experiments[expurl].argeditor.update_argument(name, argument)
         except:
-            logger.warn("Failed to set value for argument \"{}\" in experiment: {}.".format(name, expurl), exc_info=1)
+            logger.warn("Failed to set value for argument \"{}\" in experiment: {}."
+                        .format(name, expurl), exc_info=1)
 
     def get_submission_arguments(self, expurl):
         if expurl in self.submission_arguments:
@@ -701,8 +570,8 @@ class ExperimentManager:
                 raise ValueError("Submission arguments must be preinitialized "
                                  "when not using repository")
             class_desc = self.explist[expurl[5:]]
-            return self.initialize_submission_arguments(expurl,
-                class_desc["arginfo"], class_desc.get("argument_ui", None))
+            return self.initialize_submission_arguments(expurl, class_desc["arginfo"],
+                                                        class_desc.get("argument_ui", None))
 
     def open_experiment(self, expurl):
         if expurl in self.open_experiments:
@@ -739,7 +608,12 @@ class ExperimentManager:
         del self.open_experiments[expurl]
 
     async def _submit_task(self, expurl, *args):
+        try:
             rid = await self.schedule_ctl.submit(*args)
+        except KeyError:
+            expid = args[1]
+            logger.error("Submission failed - revision \"%s\" was not found", expid["repo_rev"])
+        else:
             logger.info("Submitted '%s', RID is %d", expurl, rid)
 
     def submit(self, expurl):
@@ -797,9 +671,9 @@ class ExperimentManager:
                 repo_match = "repo_rev" in expid
             else:
                 repo_match = "repo_rev" not in expid
-            if (repo_match and
-                    ("file" in expid and expid["file"] == file) and
-                    expid["class_name"] == class_name):
+            if repo_match and \
+               ("file" in expid and expid["file"] == file) and \
+               expid["class_name"] == class_name:
                 rids.append(rid)
         asyncio.ensure_future(self._request_term_multiple(rids))
 
@@ -853,6 +727,7 @@ class ExperimentManager:
 
         self.is_quick_open_shown = True
         dialog = _QuickOpenDialog(self)
+
         def closed():
             self.is_quick_open_shown = False
         dialog.closed.connect(closed)

artiq/dashboard/explorer.py

@@ -103,6 +103,7 @@ class _OpenFileDialog(QtWidgets.QDialog):
                 asyncio.ensure_future(self.refresh_view())
             else:
                 file = self.explorer.current_directory + selected
+
                 async def open_task():
                     try:
                         await self.exp_manager.open_file(file)
@@ -246,12 +247,14 @@ class ExplorerDock(QtWidgets.QDockWidget):
 
         scan_repository_action = QtWidgets.QAction("Scan repository HEAD",
                                                    self.el)
+
         def scan_repository():
             asyncio.ensure_future(experiment_db_ctl.scan_repository_async())
         scan_repository_action.triggered.connect(scan_repository)
         self.el.addAction(scan_repository_action)
 
         scan_ddb_action = QtWidgets.QAction("Scan device database", self.el)
+
         def scan_ddb():
             asyncio.ensure_future(device_db_ctl.scan())
         scan_ddb_action.triggered.connect(scan_ddb)

artiq/dashboard/interactive_args.py

@@ -0,0 +1,155 @@
+import logging
+import asyncio
+
+from PyQt5 import QtCore, QtWidgets, QtGui
+
+from artiq.gui.models import DictSyncModel
+from artiq.gui.entries import EntryTreeWidget, procdesc_to_entry
+from artiq.gui.tools import LayoutWidget
+
+
+logger = logging.getLogger(__name__)
+
+
+class Model(DictSyncModel):
+    def __init__(self, init):
+        DictSyncModel.__init__(self, ["RID", "Title", "Args"], init)
+
+    def convert(self, k, v, column):
+        if column == 0:
+            return k
+        elif column == 1:
+            txt = ": " + v["title"] if v["title"] != "" else ""
+            return str(k) + txt
+        elif column == 2:
+            return v["arglist_desc"]
+        else:
+            raise ValueError
+
+    def sort_key(self, k, v):
+        return k
+
+
+class _InteractiveArgsRequest(EntryTreeWidget):
+    supplied = QtCore.pyqtSignal(int, dict)
+    cancelled = QtCore.pyqtSignal(int)
+
+    def __init__(self, rid, arglist_desc):
+        EntryTreeWidget.__init__(self)
+        self.rid = rid
+        self.arguments = dict()
+        for key, procdesc, group, tooltip in arglist_desc:
+            self.arguments[key] = {"desc": procdesc, "group": group, "tooltip": tooltip}
+            self.set_argument(key, self.arguments[key])
+        self.quickStyleClicked.connect(self.supply)
+        cancel_btn = QtWidgets.QPushButton("Cancel")
+        cancel_btn.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_DialogCancelButton))
+        cancel_btn.clicked.connect(self.cancel)
+        supply_btn = QtWidgets.QPushButton("Supply")
+        supply_btn.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_DialogOkButton))
+        supply_btn.clicked.connect(self.supply)
+        buttons = LayoutWidget()
+        buttons.addWidget(cancel_btn, 1, 1)
+        buttons.addWidget(supply_btn, 1, 2)
+        buttons.layout.setColumnStretch(0, 1)
+        buttons.layout.setColumnStretch(1, 0)
+        buttons.layout.setColumnStretch(2, 0)
+        buttons.layout.setColumnStretch(3, 1)
+        self.setItemWidget(self.bottom_item, 1, buttons)
+
+    def supply(self):
+        argument_values = dict()
+        for key, argument in self.arguments.items():
+            entry_cls = procdesc_to_entry(argument["desc"])
+            argument_values[key] = entry_cls.state_to_value(argument["state"])
+        self.supplied.emit(self.rid, argument_values)
+
+    def cancel(self):
+        self.cancelled.emit(self.rid)
+
+
+class _InteractiveArgsView(QtWidgets.QStackedWidget):
+    supplied = QtCore.pyqtSignal(int, dict)
+    cancelled = QtCore.pyqtSignal(int)
+
+    def __init__(self):
+        QtWidgets.QStackedWidget.__init__(self)
+        self.tabs = QtWidgets.QTabWidget()
+        self.default_label = QtWidgets.QLabel("No pending interactive arguments requests.")
+        self.default_label.setAlignment(QtCore.Qt.AlignCenter)
+        font = QtGui.QFont(self.default_label.font())
+        font.setItalic(True)
+        self.default_label.setFont(font)
+        self.addWidget(self.tabs)
+        self.addWidget(self.default_label)
+        self.model = Model({})
+
+    def setModel(self, model):
+        self.setCurrentIndex(1)
+        for i in range(self.tabs.count()):
+            widget = self.tabs.widget(i)
+            self.tabs.removeTab(i)
+            widget.deleteLater()
+        self.model = model
+        self.model.rowsInserted.connect(self.rowsInserted)
+        self.model.rowsRemoved.connect(self.rowsRemoved)
+        for i in range(self.model.rowCount(QtCore.QModelIndex())):
+            self._insert_widget(i)
+
+    def _insert_widget(self, row):
+        rid = self.model.data(self.model.index(row, 0), QtCore.Qt.DisplayRole)
+        title = self.model.data(self.model.index(row, 1), QtCore.Qt.DisplayRole)
+        arglist_desc = self.model.data(self.model.index(row, 2), QtCore.Qt.DisplayRole)
+        inter_args_request = _InteractiveArgsRequest(rid, arglist_desc)
+        inter_args_request.supplied.connect(self.supplied)
+        inter_args_request.cancelled.connect(self.cancelled)
+        self.tabs.insertTab(row, inter_args_request, title)
+
+    def rowsInserted(self, parent, first, last):
+        assert first == last
+        self.setCurrentIndex(0)
+        self._insert_widget(first)
+
+    def rowsRemoved(self, parent, first, last):
+        assert first == last
+        widget = self.tabs.widget(first)
+        self.tabs.removeTab(first)
+        widget.deleteLater()
+        if self.tabs.count() == 0:
+            self.setCurrentIndex(1)
+
+
+class InteractiveArgsDock(QtWidgets.QDockWidget):
+    def __init__(self, interactive_args_sub, interactive_args_rpc):
+        QtWidgets.QDockWidget.__init__(self, "Interactive Args")
+        self.setObjectName("Interactive Args")
+        self.setFeatures(
+            QtWidgets.QDockWidget.DockWidgetMovable | QtWidgets.QDockWidget.DockWidgetFloatable)
+        self.interactive_args_rpc = interactive_args_rpc
+        self.request_view = _InteractiveArgsView()
+        self.request_view.supplied.connect(self.supply)
+        self.request_view.cancelled.connect(self.cancel)
+        self.setWidget(self.request_view)
+        interactive_args_sub.add_setmodel_callback(self.request_view.setModel)
+
+    def supply(self, rid, values):
+        asyncio.ensure_future(self._supply_task(rid, values))
+
+    async def _supply_task(self, rid, values):
+        try:
+            await self.interactive_args_rpc.supply(rid, values)
+        except Exception:
+            logger.error("failed to supply interactive arguments for experiment: %d",
+                         rid, exc_info=True)
+
+    def cancel(self, rid):
+        asyncio.ensure_future(self._cancel_task(rid))
+
+    async def _cancel_task(self, rid):
+        try:
+            await self.interactive_args_rpc.cancel(rid)
+        except Exception:
+            logger.error("failed to cancel interactive args request for experiment: %d",
+                         rid, exc_info=True)

artiq/dashboard/moninj.py

@@ -2,23 +2,21 @@ import asyncio
 import logging
 import textwrap
 from collections import namedtuple
+from functools import partial
 
-from PyQt5 import QtCore, QtWidgets, QtGui
+from PyQt5 import QtCore, QtWidgets
 
-from sipyco.sync_struct import Subscriber
-
-from artiq.coredevice.comm_moninj import *
-from artiq.coredevice.ad9910 import (
-    _AD9910_REG_PROFILE0, _AD9910_REG_PROFILE7, 
-    _AD9910_REG_FTW, _AD9910_REG_CFR1
-)
-from artiq.coredevice.ad9912_reg import AD9912_POW1, AD9912_SER_CONF
-from artiq.gui.tools import LayoutWidget
-from artiq.gui.flowlayout import FlowLayout
+from artiq.coredevice.comm_moninj import CommMonInj, TTLOverride, TTLProbe
+from artiq.coredevice.ad9912_reg import AD9912_SER_CONF
+from artiq.gui.tools import LayoutWidget, QDockWidgetCloseDetect, DoubleClickLineEdit
+from artiq.gui.dndwidgets import VDragScrollArea, DragDropFlowLayoutWidget
+from artiq.gui.models import DictSyncTreeSepModel
+from artiq.tools import elide
 
 
 logger = logging.getLogger(__name__)
 
+
 class _CancellableLineEdit(QtWidgets.QLineEdit):
     def escapePressedConnect(self, cb):
         self.esc_cb = cb
@@ -30,30 +28,37 @@ class _CancellableLineEdit(QtWidgets.QLineEdit):
         QtWidgets.QLineEdit.keyPressEvent(self, event)
 
 
-class _TTLWidget(QtWidgets.QFrame):
-    def __init__(self, dm, channel, force_out, title):
+class _MoninjWidget(QtWidgets.QFrame):
+    def __init__(self, title, channel=None):
         QtWidgets.QFrame.__init__(self)
+        self.setFrameShape(QtWidgets.QFrame.Box)
+        self.setFrameShape(QtWidgets.QFrame.Box)
+        self.setFrameShadow(QtWidgets.QFrame.Raised)
+        self.setFixedHeight(100)
+        self.setFixedWidth(150)
+        self.grid = QtWidgets.QGridLayout()
+        self.grid.setContentsMargins(0, 0, 0, 0)
+        self.grid.setHorizontalSpacing(0)
+        self.grid.setVerticalSpacing(0)
+        self.setLayout(self.grid)
+        title = elide(title, 17)
+        title += "" if channel is None else "[{}]".format(channel) 
+        label = QtWidgets.QLabel(title)
+        label.setAlignment(QtCore.Qt.AlignHCenter | QtCore.Qt.AlignTop)
+        self.grid.addWidget(label, 1, 1)
+
+
+class _TTLWidget(_MoninjWidget):
+    def __init__(self, dm, channel, force_out, title):
+        _MoninjWidget.__init__(self, title)
 
         self.channel = channel
         self.set_mode = dm.ttl_set_mode
         self.force_out = force_out
-
-        self.setFrameShape(QtWidgets.QFrame.Box)
-        self.setFrameShadow(QtWidgets.QFrame.Raised)
-
-        grid = QtWidgets.QGridLayout()
-        grid.setContentsMargins(0, 0, 0, 0)
-        grid.setHorizontalSpacing(0)
-        grid.setVerticalSpacing(0)
-        self.setLayout(grid)
-        label = QtWidgets.QLabel(title)
-        label.setAlignment(QtCore.Qt.AlignCenter)
-        label.setSizePolicy(QtWidgets.QSizePolicy.Ignored,
-                            QtWidgets.QSizePolicy.Preferred)
-        grid.addWidget(label, 1, 1)
+        self.title = title
 
         self.stack = QtWidgets.QStackedWidget()
-        grid.addWidget(self.stack, 2, 1)
+        self.grid.addWidget(self.stack, 2, 1)
 
         self.direction = QtWidgets.QLabel()
         self.direction.setAlignment(QtCore.Qt.AlignCenter)
@@ -77,12 +82,12 @@ class _TTLWidget(QtWidgets.QFrame):
 
         self.value = QtWidgets.QLabel()
         self.value.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(self.value, 3, 1)
+        self.grid.addWidget(self.value, 3, 1)
 
-        grid.setRowStretch(1, 1)
-        grid.setRowStretch(2, 0)
-        grid.setRowStretch(3, 0)
-        grid.setRowStretch(4, 1)
+        self.grid.setRowStretch(1, 1)
+        self.grid.setRowStretch(2, 0)
+        self.grid.setRowStretch(3, 0)
+        self.grid.setRowStretch(4, 1)
 
         self.programmatic_change = False
         self.override.clicked.connect(self.override_toggled)
@@ -148,40 +153,13 @@ class _TTLWidget(QtWidgets.QFrame):
             self.programmatic_change = False
 
     def sort_key(self):
-        return self.channel
+        return (0, self.channel, 0)
 
+    def uid(self):
+        return self.title
 
-class _SimpleDisplayWidget(QtWidgets.QFrame):
-    def __init__(self, title):
-        QtWidgets.QFrame.__init__(self)
-
-        self.setFrameShape(QtWidgets.QFrame.Box)
-        self.setFrameShadow(QtWidgets.QFrame.Raised)
-
-        grid = QtWidgets.QGridLayout()
-        grid.setContentsMargins(0, 0, 0, 0)
-        grid.setHorizontalSpacing(0)
-        grid.setVerticalSpacing(0)
-        self.setLayout(grid)
-        label = QtWidgets.QLabel(title)
-        label.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(label, 1, 1)
-
-        self.value = QtWidgets.QLabel()
-        self.value.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(self.value, 2, 1, 6, 1)
-
-        grid.setRowStretch(1, 1)
-        grid.setRowStretch(2, 0)
-        grid.setRowStretch(3, 1)
-
-        self.refresh_display()
-
-    def refresh_display(self):
-        raise NotImplementedError
-
-    def sort_key(self):
-        raise NotImplementedError
+    def to_model_path(self):
+        return "ttl/{}".format(self.title)
 
 
 class _DDSModel:
@@ -215,28 +193,18 @@ class _DDSModel:
         return ftw / self.ftw_per_hz
 
 
-class _DDSWidget(QtWidgets.QFrame):
-    def __init__(self, dm, title, bus_channel=0, channel=0, dds_model=None):
+class _DDSWidget(_MoninjWidget):
+    def __init__(self, dm, title, bus_channel, channel,
+                 dds_type, ref_clk, cpld=None, pll=1, clk_div=0):
         self.dm = dm
         self.bus_channel = bus_channel
         self.channel = channel
         self.dds_name = title
         self.cur_frequency = 0
-        self.dds_model = dds_model
+        self.dds_model = _DDSModel(dds_type, ref_clk, cpld, pll, clk_div)
+        self.title = title
 
-        QtWidgets.QFrame.__init__(self)
-
-        self.setFrameShape(QtWidgets.QFrame.Box)
-        self.setFrameShadow(QtWidgets.QFrame.Raised)
-
-        grid = QtWidgets.QGridLayout()
-        grid.setContentsMargins(0, 0, 0, 0)
-        grid.setHorizontalSpacing(0)
-        grid.setVerticalSpacing(0)
-        self.setLayout(grid)
-        label = QtWidgets.QLabel(title)
-        label.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(label, 1, 1)
+        _MoninjWidget.__init__(self, title)
 
         # FREQ DATA/EDIT FIELD
         self.data_stack = QtWidgets.QStackedWidget()
@@ -271,7 +239,7 @@ class _DDSWidget(QtWidgets.QFrame):
         grid_edit.addWidget(unit, 0, 3, 1, 1)
         self.data_stack.addWidget(grid_edit)
 
-        grid.addWidget(self.data_stack, 2, 1)
+        self.grid.addWidget(self.data_stack, 2, 1)
 
         # BUTTONS
         self.button_stack = QtWidgets.QStackedWidget()
@@ -304,11 +272,11 @@ class _DDSWidget(QtWidgets.QFrame):
         cancel.setToolTip("Cancel changes")
         apply_grid.addWidget(cancel, 0, 2, 1, 1)
         self.button_stack.addWidget(apply_grid)
-        grid.addWidget(self.button_stack, 3, 1)
+        self.grid.addWidget(self.button_stack, 3, 1)
 
-        grid.setRowStretch(1, 1)
-        grid.setRowStretch(2, 1)
-        grid.setRowStretch(3, 1)
+        self.grid.setRowStretch(1, 1)
+        self.grid.setRowStretch(2, 1)
+        self.grid.setRowStretch(3, 1)
 
         set_btn.clicked.connect(self.set_clicked)
         apply.clicked.connect(self.apply_changes)
@@ -327,8 +295,7 @@ class _DDSWidget(QtWidgets.QFrame):
     def set_clicked(self, set):
         self.data_stack.setCurrentIndex(1)
         self.button_stack.setCurrentIndex(1)
-        self.value_edit.setText("{:.7f}"
-                .format(self.cur_frequency/1e6))
+        self.value_edit.setText("{:.7f}".format(self.cur_frequency / 1e6))
         self.value_edit.setFocus()
         self.value_edit.selectAll()
 
@@ -347,28 +314,53 @@ class _DDSWidget(QtWidgets.QFrame):
 
     def refresh_display(self):
         self.cur_frequency = self.dds_model.cur_frequency
-        self.value_label.setText("<font size=\"4\">{:.7f}</font>"
-                           .format(self.cur_frequency/1e6))
-        self.value_edit.setText("{:.7f}"
-                           .format(self.cur_frequency/1e6))
+        self.value_label.setText("<font size=\"4\">{:.7f}</font>".format(self.cur_frequency / 1e6))
+        self.value_edit.setText("{:.7f}".format(self.cur_frequency / 1e6))
 
     def sort_key(self):
-        return (self.bus_channel, self.channel)
+        return (1, self.bus_channel, self.channel)
+
+    def uid(self):
+        return self.title
+
+    def to_model_path(self):
+        return "dds/{}".format(self.title)
 
 
-class _DACWidget(_SimpleDisplayWidget):
-    def __init__(self, dm, spi_channel, channel, title):
+class _DACWidget(_MoninjWidget):
+    def __init__(self, dm, spi_channel, channel, title, vref, offset_dacs):
+        _MoninjWidget.__init__(self, title, channel)
         self.spi_channel = spi_channel
         self.channel = channel
-        self.cur_value = 0
-        _SimpleDisplayWidget.__init__(self, "{} ch{}".format(title, channel))
+        self.cur_value = 0x8000
+        self.title = title
+        self.vref = vref
+        self.offset_dacs = offset_dacs
+
+        self.value = QtWidgets.QLabel()
+        self.value.setAlignment(QtCore.Qt.AlignHCenter | QtCore.Qt.AlignTop)
+        self.grid.addWidget(self.value, 2, 1, 6, 1)
+
+        self.grid.setRowStretch(1, 1)
+        self.grid.setRowStretch(2, 1)
+
+        self.refresh_display()
+
+    def mu_to_voltage(self, code):
+        return ((code - self.offset_dacs * 0x4) / (1 << 16)) * (4. * self.vref)
 
     def refresh_display(self):
-        self.value.setText("<font size=\"4\">{:.3f}</font><font size=\"2\"> %</font>"
-                           .format(self.cur_value*100/2**16))
+        self.value.setText("<font size=\"4\">{:+.3f} V</font>"
+                           .format(self.mu_to_voltage(self.cur_value)))
 
     def sort_key(self):
-        return (self.spi_channel, self.channel)
+        return (2, self.spi_channel, self.channel)
+
+    def uid(self):
+        return (self.title, self.channel)
+
+    def to_model_path(self):
+        return "dac/{}[{}]".format(self.title, self.channel)
 
 
 _WidgetDesc = namedtuple("_WidgetDesc", "uid comment cls arguments")
@@ -386,24 +378,19 @@ def setup_from_ddb(ddb):
                 comment = v.get("comment")
                 if v["type"] == "local":
                     if v["module"] == "artiq.coredevice.ttl":
-                        if "ttl_urukul" in k:
-                            continue
                         channel = v["arguments"]["channel"]
                         force_out = v["class"] == "TTLOut"
                         widget = _WidgetDesc(k, comment, _TTLWidget, (channel, force_out, k))
                         description.add(widget)
-                    elif (v["module"] == "artiq.coredevice.ad9914"
-                            and v["class"] == "AD9914"):
+                    elif (v["module"] == "artiq.coredevice.ad9914" and v["class"] == "AD9914"):
                         bus_channel = v["arguments"]["bus_channel"]
                         channel = v["arguments"]["channel"]
                         dds_sysclk = v["arguments"]["sysclk"]
-                        model = _DDSModel(v["class"], dds_sysclk)
-                        widget = _WidgetDesc(k, comment, _DDSWidget, (k, bus_channel, channel, model))
+                        widget = _WidgetDesc(k, comment, _DDSWidget,
+                                             (k, bus_channel, channel, v["class"], dds_sysclk))
                         description.add(widget)
-                    elif (v["module"] == "artiq.coredevice.ad9910"
-                                and v["class"] == "AD9910") or \
-                            (v["module"] == "artiq.coredevice.ad9912"
-                                and v["class"] == "AD9912"):
+                    elif (v["module"] == "artiq.coredevice.ad9910" and v["class"] == "AD9910") or \
+                         (v["module"] == "artiq.coredevice.ad9912" and v["class"] == "AD9912"):
                         channel = v["arguments"]["chip_select"] - 4
                         if channel < 0:
                             continue
@@ -413,18 +400,28 @@ def setup_from_ddb(ddb):
                         pll = v["arguments"]["pll_n"]
                         refclk = ddb[dds_cpld]["arguments"]["refclk"]
                         clk_div = v["arguments"].get("clk_div", 0)
-                        model = _DDSModel( v["class"], refclk, dds_cpld, pll, clk_div)
-                        widget = _WidgetDesc(k, comment, _DDSWidget, (k, bus_channel, channel, model))
+                        widget = _WidgetDesc(k, comment, _DDSWidget,
+                                             (k, bus_channel, channel, v["class"],
+                                              refclk, dds_cpld, pll, clk_div))
                         description.add(widget)
-                    elif (   (v["module"] == "artiq.coredevice.ad53xx" and v["class"] == "AD53xx")
-                          or (v["module"] == "artiq.coredevice.zotino" and v["class"] == "Zotino")):
+                    elif (v["module"] == "artiq.coredevice.ad53xx" and v["class"] == "AD53xx") or \
+                         (v["module"] == "artiq.coredevice.zotino" and v["class"] == "Zotino"):
                         spi_device = v["arguments"]["spi_device"]
                         spi_device = ddb[spi_device]
                         while isinstance(spi_device, str):
                             spi_device = ddb[spi_device]
                         spi_channel = spi_device["arguments"]["channel"]
+                        vref = v["arguments"].get("vref", 5.)
+                        offset_dacs = v["arguments"].get("offset_dacs", 8192)
                         for channel in range(32):
-                            widget = _WidgetDesc((k, channel), comment, _DACWidget, (spi_channel, channel, k))
+                            widget = _WidgetDesc((k, channel), comment, _DACWidget,
+                                                 (spi_channel, channel, k, vref, offset_dacs))
+                            description.add(widget)
+                    elif (v["module"] == "artiq.coredevice.fastino" and v["class"] == "Fastino"):
+                        bus_channel = v["arguments"]["channel"]
+                        for channel in range(0, 32):
+                            widget = _WidgetDesc((k, channel), comment, _DACWidget,
+                                                 (bus_channel, channel, k, 5, 8192))
                             description.add(widget)
                 elif v["type"] == "controller" and k == "core_moninj":
                     mi_addr = v["host"]
@@ -449,18 +446,15 @@ class _DeviceManager:
         self.widgets_by_uid = dict()
 
         self.dds_sysclk = 0
-        self.ttl_cb = lambda: None
         self.ttl_widgets = dict()
-        self.dds_cb = lambda: None
         self.dds_widgets = dict()
-        self.dac_cb = lambda: None
         self.dac_widgets = dict()
+        self.channels_cb = lambda: None
 
     def init_ddb(self, ddb):
         self.ddb = ddb
-        return ddb
 
-    def notify(self, mod):
+    def notify_ddb(self, mod):
         mi_addr, mi_port, description = setup_from_ddb(self.ddb)
 
         if (mi_addr, mi_port) != (self.mi_addr, self.mi_port):
@@ -471,24 +465,8 @@ class _DeviceManager:
         for to_remove in self.description - description:
             widget = self.widgets_by_uid[to_remove.uid]
             del self.widgets_by_uid[to_remove.uid]
-
-            if isinstance(widget, _TTLWidget):
-                self.setup_ttl_monitoring(False, widget.channel)
+            self.setup_monitoring(False, widget)
             widget.deleteLater()
-                del self.ttl_widgets[widget.channel]
-                self.ttl_cb()
-            elif isinstance(widget, _DDSWidget):
-                self.setup_dds_monitoring(False, widget.bus_channel, widget.channel)
-                widget.deleteLater()
-                del self.dds_widgets[(widget.bus_channel, widget.channel)]
-                self.dds_cb()
-            elif isinstance(widget, _DACWidget):
-                self.setup_dac_monitoring(False, widget.spi_channel, widget.channel)
-                widget.deleteLater()
-                del self.dac_widgets[(widget.spi_channel, widget.channel)]
-                self.dac_cb()     
-            else:
-                raise ValueError
 
         for to_add in description - self.description:
             widget = to_add.cls(self, *to_add.arguments)
@@ -496,26 +474,15 @@ class _DeviceManager:
                 widget.setToolTip(to_add.comment)
             self.widgets_by_uid[to_add.uid] = widget
 
-            if isinstance(widget, _TTLWidget):
-                self.ttl_widgets[widget.channel] = widget
-                self.ttl_cb()
-                self.setup_ttl_monitoring(True, widget.channel)
-            elif isinstance(widget, _DDSWidget):
-                self.dds_widgets[(widget.bus_channel, widget.channel)] = widget
-                self.dds_cb()
-                self.setup_dds_monitoring(True, widget.bus_channel, widget.channel)
-            elif isinstance(widget, _DACWidget):
-                self.dac_widgets[(widget.spi_channel, widget.channel)] = widget
-                self.dac_cb()
-                self.setup_dac_monitoring(True, widget.spi_channel, widget.channel)
-            else:
-                raise ValueError
+        if description != self.description:
+            self.channels_cb()
 
         self.description = description
 
     def ttl_set_mode(self, channel, mode):
         if self.mi_connection is not None:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if mode == "0":
                 widget.cur_override = True
                 widget.cur_level = False
@@ -664,6 +631,31 @@ class _DeviceManager:
             "ToggleDDS",
             "Toggle DDS {} {}".format(dds_channel, "on" if sw else "off"))
 
+    def setup_monitoring(self, enable, widget):
+        if isinstance(widget, _TTLWidget):
+            key = widget.channel
+            args = (key,)
+            subscribers = self.ttl_widgets
+            subscribe_func = self.setup_ttl_monitoring
+        elif isinstance(widget, _DDSWidget):
+            key = (widget.bus_channel, widget.channel)
+            args = key
+            subscribers = self.dds_widgets
+            subscribe_func = self.setup_dds_monitoring
+        elif isinstance(widget, _DACWidget):
+            key = (widget.spi_channel, widget.channel)
+            args = key
+            subscribers = self.dac_widgets
+            subscribe_func = self.setup_dac_monitoring
+        else:
+            raise ValueError
+        if enable and key not in subscribers:
+            subscribers[key] = widget.uid()
+            subscribe_func(enable, *args)
+        elif not enable and key in subscribers:
+            subscribe_func(enable, *args)
+            del subscribers[key]
+
     def setup_ttl_monitoring(self, enable, channel):
         if self.mi_connection is not None:
             self.mi_connection.monitor_probe(enable, channel, TTLProbe.level.value)
@@ -683,24 +675,28 @@ class _DeviceManager:
 
     def monitor_cb(self, channel, probe, value):
         if channel in self.ttl_widgets:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if probe == TTLProbe.level.value:
                 widget.cur_level = bool(value)
             elif probe == TTLProbe.oe.value:
                 widget.cur_oe = bool(value)
             widget.refresh_display()
         elif (channel, probe) in self.dds_widgets:
-            widget = self.dds_widgets[(channel, probe)]
+            widget_uid = self.dds_widgets[(channel, probe)]
+            widget = self.widgets_by_uid[widget_uid]
             widget.dds_model.monitor_update(probe, value)
             widget.refresh_display()
         elif (channel, probe) in self.dac_widgets:
-            widget = self.dac_widgets[(channel, probe)]
+            widget_uid = self.dac_widgets[(channel, probe)]
+            widget = self.widgets_by_uid[widget_uid]
             widget.cur_value = value
             widget.refresh_display()
 
     def injection_status_cb(self, channel, override, value):
         if channel in self.ttl_widgets:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if override == TTLOverride.en.value:
                 widget.cur_override = bool(value)
             if override == TTLOverride.level.value:
@@ -722,11 +718,9 @@ class _DeviceManager:
                                            self.disconnect_cb)
             try:
                 await new_mi_connection.connect(self.mi_addr, self.mi_port)
-            except asyncio.CancelledError:
-                logger.info("cancelled connection to moninj")
-                break
-            except:
-                logger.error("failed to connect to moninj. Is aqctl_moninj_proxy running?", exc_info=True)
+            except Exception:
+                logger.error("failed to connect to moninj. Is aqctl_moninj_proxy running?",
+                             exc_info=True)
                 await asyncio.sleep(10.)
                 self.reconnect_mi.set()
             else:
@@ -750,48 +744,234 @@ class _DeviceManager:
             await self.mi_connection.close()
 
 
-class _MonInjDock(QtWidgets.QDockWidget):
-    def __init__(self, name):
-        QtWidgets.QDockWidget.__init__(self, name)
+class Model(DictSyncTreeSepModel):
+    def __init__(self, init):
+        DictSyncTreeSepModel.__init__(self, "/", ["Channels"], init)
+
+    def clear(self):
+        for k in self.backing_store:
+            self._del_item(self, k.split(self.separator))
+        self.backing_store.clear()
+
+    def update(self, d):
+        for k, v in d.items():
+            self[v.to_model_path()] = v
+
+
+class _AddChannelDialog(QtWidgets.QDialog):
+    def __init__(self, parent, model):
+        QtWidgets.QDialog.__init__(self, parent=parent)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+        self.setWindowTitle("Add channels")
+
+        layout = QtWidgets.QVBoxLayout()
+        self.setLayout(layout)
+
+        self._model = model
+        self._tree_view = QtWidgets.QTreeView()
+        self._tree_view.setHeaderHidden(True)
+        self._tree_view.setSelectionBehavior(
+            QtWidgets.QAbstractItemView.SelectItems)
+        self._tree_view.setSelectionMode(
+            QtWidgets.QAbstractItemView.ExtendedSelection)
+        self._tree_view.setModel(self._model)
+        layout.addWidget(self._tree_view)
+
+        self._button_box = QtWidgets.QDialogButtonBox(
+            QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel
+        )
+        self._button_box.setCenterButtons(True)
+        self._button_box.accepted.connect(self.add_channels)
+        self._button_box.rejected.connect(self.reject)
+        layout.addWidget(self._button_box)
+
+    def add_channels(self):
+        selection = self._tree_view.selectedIndexes()
+        channels = []
+        for select in selection:
+            key = self._model.index_to_key(select)
+            if key is not None:
+                channels.append(self._model[key].ref)
+        self.channels = channels
+        self.accept()
+
+
+class _MonInjDock(QDockWidgetCloseDetect):
+    def __init__(self, name, manager):
+        QtWidgets.QDockWidget.__init__(self, "MonInj")
         self.setObjectName(name)
         self.setFeatures(QtWidgets.QDockWidget.DockWidgetMovable |
                          QtWidgets.QDockWidget.DockWidgetFloatable)
+        grid = LayoutWidget()
+        self.setWidget(grid)
+        self.manager = manager
+        self.widget_uids = None
+
+        newdock = QtWidgets.QToolButton()
+        newdock.setToolTip("Create new moninj dock")
+        newdock.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_FileDialogNewFolder))
+        newdock.clicked.connect(lambda: self.manager.create_new_dock())
+        grid.addWidget(newdock, 0, 0)
+
+        self.channel_dialog = _AddChannelDialog(self, self.manager.channel_model)
+        self.channel_dialog.accepted.connect(self.add_channels)
+
+        dialog_btn = QtWidgets.QToolButton()
+        dialog_btn.setToolTip("Add channels")
+        dialog_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogListView))
+        dialog_btn.clicked.connect(self.channel_dialog.open)
+        grid.addWidget(dialog_btn, 0, 1)
+
+        self.label = DoubleClickLineEdit(name)
+        self.label.setStyleSheet("background:transparent;")
+        grid.addWidget(self.label, 0, 2)
+
+        scroll_area = VDragScrollArea(self)
+        grid.addWidget(scroll_area, 1, 0, 1, 10)
+        self.flow = DragDropFlowLayoutWidget()
+        scroll_area.setWidgetResizable(True)
+        scroll_area.setWidget(self.flow)
+        self.flow.setContextMenuPolicy(QtCore.Qt.CustomContextMenu)
+        self.flow.customContextMenuRequested.connect(self.custom_context_menu)
+
+    def custom_context_menu(self, pos):
+        index = self.flow._get_index(pos)
+        if index == -1:
+            return
+        menu = QtWidgets.QMenu()
+        delete_action = QtWidgets.QAction("Delete widget", menu)
+        delete_action.triggered.connect(partial(self.delete_widget, index))
+        menu.addAction(delete_action)
+        menu.exec_(self.flow.mapToGlobal(pos))
+
+    def delete_all_widgets(self):
+        for index in reversed(range(self.flow.count())):
+            self.delete_widget(index, True)
+
+
+    def delete_widget(self, index, checked):
+        widget = self.flow.itemAt(index).widget()
+        self.manager.dm.setup_monitoring(False, widget)
+        self.flow.layout.takeAt(index)
+        widget.setParent(self.manager.main_window)
+        widget.hide()
+
+    def add_channels(self):
+        channels = self.channel_dialog.channels
+        self.layout_widgets(channels)
 
     def layout_widgets(self, widgets):
-        scroll_area = QtWidgets.QScrollArea()
-        self.setWidget(scroll_area)
-
-        grid = FlowLayout()
-        grid_widget = QtWidgets.QWidget()
-        grid_widget.setLayout(grid)
-
         for widget in sorted(widgets, key=lambda w: w.sort_key()):
-            grid.addWidget(widget)
+            self.manager.dm.setup_monitoring(True, widget)
+            self.flow.addWidget(widget)
+            widget.show()
 
-        scroll_area.setWidgetResizable(True)
-        scroll_area.setWidget(grid_widget)
+    def restore_widgets(self):
+        if self.widget_uids is not None:
+            widgets_by_uid = self.manager.dm.widgets_by_uid
+            widgets = list()
+            for uid in self.widget_uids:
+                if uid in widgets_by_uid:
+                    widgets.append(widgets_by_uid[uid])
+                else:
+                    logger.warning("removing moninj widget {}".format(uid))
+            self.layout_widgets(widgets)
+            self.widget_uids = None
+
+    def _save_widget_uids(self):
+        uids = []
+        for i in range(self.flow.count()):
+            uids.append(self.flow.itemAt(i).widget().uid())
+        return uids
+
+    def save_state(self):
+        return {
+            "dock_label": self.label.text(),
+            "widget_uids": self._save_widget_uids()
+        }
+
+    def restore_state(self, state):
+        try:
+            label = state["dock_label"]
+        except KeyError:
+            pass
+        else:
+            self.label._text = label
+            self.label.setText(label)
+        try:
+            self.widget_uids = state["widget_uids"]
+        except KeyError:
+            pass
 
 
 class MonInj:
-    def __init__(self, schedule_ctl):
-        self.ttl_dock = _MonInjDock("TTL")
-        self.dds_dock = _MonInjDock("DDS")
-        self.dac_dock = _MonInjDock("DAC")
-
+    def __init__(self, schedule_ctl, main_window):
+        self.docks = dict()
+        self.main_window = main_window
         self.dm = _DeviceManager(schedule_ctl)
-        self.dm.ttl_cb = lambda: self.ttl_dock.layout_widgets(
-                            self.dm.ttl_widgets.values())
-        self.dm.dds_cb = lambda: self.dds_dock.layout_widgets(
-                            self.dm.dds_widgets.values())
-        self.dm.dac_cb = lambda: self.dac_dock.layout_widgets(
-                            self.dm.dac_widgets.values())
+        self.dm.channels_cb = self.add_channels
+        self.channel_model = Model({})
 
-        self.subscriber = Subscriber("devices", self.dm.init_ddb, self.dm.notify)
+    def add_channels(self):
+        self.channel_model.clear()
+        self.channel_model.update(self.dm.widgets_by_uid)
+        for dock in self.docks.values():
+            dock.restore_widgets()
 
-    async def start(self, server, port):
-        await self.subscriber.connect(server, port)
+    def create_new_dock(self, add_to_area=True):
+        n = 0
+        name = "moninj0"
+        while name in self.docks:
+            n += 1
+            name = "moninj" + str(n)
+
+        dock = _MonInjDock(name, self)
+        self.docks[name] = dock
+        if add_to_area:
+            self.main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
+            dock.setFloating(True)
+        dock.sigClosed.connect(partial(self.on_dock_closed, name))
+        self.update_closable()
+        return dock
+
+    def on_dock_closed(self, name):
+        dock = self.docks[name]
+        del self.docks[name]
+        self.update_closable()
+        dock.delete_all_widgets()
+        dock.deleteLater()
+
+    def update_closable(self):
+        flags = (QtWidgets.QDockWidget.DockWidgetMovable |
+                 QtWidgets.QDockWidget.DockWidgetFloatable)
+        if len(self.docks) > 1:
+            flags |= QtWidgets.QDockWidget.DockWidgetClosable
+        for dock in self.docks.values():
+            dock.setFeatures(flags)
+
+    def first_moninj_dock(self):
+        if self.docks:
+            return None
+        dock = self.create_new_dock(False)
+        return dock
+
+    def save_state(self):
+        return {name: dock.save_state() for name, dock in self.docks.items()}
+
+    def restore_state(self, state):
+        if self.docks:
+            raise NotImplementedError
+        for name, dock_state in state.items():
+            dock = _MonInjDock(name, self)
+            self.docks[name] = dock
+            dock.restore_state(dock_state)
+            self.main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
+            dock.sigClosed.connect(partial(self.on_dock_closed, name))
+        self.update_closable()
 
     async def stop(self):
-        await self.subscriber.close()
         if self.dm is not None:
             await self.dm.close()

artiq/dashboard/schedule.py

@@ -16,8 +16,7 @@ class Model(DictSyncModel):
     def __init__(self, init):
         DictSyncModel.__init__(self,
                                ["RID", "Pipeline", "Status", "Prio", "Due date",
-             "Revision", "File", "Class name"],
-            init)
+                                "Revision", "File", "Class name"], init)
 
     def sort_key(self, k, v):
         # order by priority, and then by due date and RID
@@ -151,7 +150,6 @@ class ScheduleDock(QtWidgets.QDockWidget):
                     rids.add(rid)
             asyncio.ensure_future(self.request_term_multiple(rids))
 
-
     def save_state(self):
         return bytes(self.table.horizontalHeader().saveState())
 

artiq/dashboard/shortcuts.py

@@ -3,8 +3,6 @@ from functools import partial
 
 from PyQt5 import QtCore, QtWidgets
 
-from artiq.gui.tools import LayoutWidget
-
 
 logger = logging.getLogger(__name__)
 

artiq/dashboard/waveform.py

@@ -0,0 +1,914 @@
+import os
+import asyncio
+import logging
+import bisect
+import itertools
+import math
+
+from PyQt5 import QtCore, QtWidgets, QtGui
+
+import pyqtgraph as pg
+import numpy as np
+
+from sipyco.pc_rpc import AsyncioClient
+from sipyco import pyon
+
+from artiq.tools import exc_to_warning, short_format
+from artiq.coredevice import comm_analyzer
+from artiq.coredevice.comm_analyzer import WaveformType
+from artiq.gui.tools import LayoutWidget, get_open_file_name, get_save_file_name
+from artiq.gui.models import DictSyncTreeSepModel
+from artiq.gui.dndwidgets import VDragScrollArea, VDragDropSplitter
+
+
+logger = logging.getLogger(__name__)
+
+WAVEFORM_MIN_HEIGHT = 50
+WAVEFORM_MAX_HEIGHT = 200
+
+
+class ProxyClient():
+    def __init__(self, receive_cb, timeout=5, timer=5, timer_backoff=1.1):
+        self.receive_cb = receive_cb
+        self.receiver = None
+        self.addr = None
+        self.port_proxy = None
+        self.port = None
+        self._reconnect_event = asyncio.Event()
+        self.timeout = timeout
+        self.timer = timer
+        self.timer_cur = timer
+        self.timer_backoff = timer_backoff
+        self._reconnect_task = asyncio.ensure_future(self._reconnect())
+
+    def update_address(self, addr, port, port_proxy):
+        self.addr = addr
+        self.port = port
+        self.port_proxy = port_proxy
+        self._reconnect_event.set()
+
+    async def trigger_proxy_task(self):
+        remote = AsyncioClient()
+        try:
+            try:
+                if self.addr is None:
+                    logger.error("missing core_analyzer host in device db")
+                    return
+                await remote.connect_rpc(self.addr, self.port, "coreanalyzer_proxy_control")
+            except:
+                logger.error("error connecting to analyzer proxy control", exc_info=True)
+                return
+            await remote.trigger()
+        except:
+            logger.error("analyzer proxy reported failure", exc_info=True)
+        finally:
+            remote.close_rpc()
+
+    async def _reconnect(self):
+        while True:
+            await self._reconnect_event.wait()
+            self._reconnect_event.clear()
+            if self.receiver is not None:
+                await self.receiver.close()
+                self.receiver = None
+            new_receiver = comm_analyzer.AnalyzerProxyReceiver(
+                self.receive_cb, self.disconnect_cb)
+            try:
+                if self.addr is not None:
+                    await asyncio.wait_for(new_receiver.connect(self.addr, self.port_proxy),
+                                           self.timeout)
+                    logger.info("ARTIQ dashboard connected to analyzer proxy (%s)", self.addr)
+                    self.timer_cur = self.timer
+                    self.receiver = new_receiver
+                continue
+            except Exception:
+                logger.error("error connecting to analyzer proxy", exc_info=True)
+            try:
+                await asyncio.wait_for(self._reconnect_event.wait(), self.timer_cur)
+            except asyncio.TimeoutError:
+                self.timer_cur *= self.timer_backoff
+                self._reconnect_event.set()
+            else:
+                self.timer_cur = self.timer
+
+    async def close(self):
+        self._reconnect_task.cancel()
+        try:
+            await asyncio.wait_for(self._reconnect_task, None)
+        except asyncio.CancelledError:
+            pass
+        if self.receiver is not None:
+            await self.receiver.close()
+
+    def disconnect_cb(self):
+        logger.error("lost connection to analyzer proxy")
+        self._reconnect_event.set()
+
+
+class _BackgroundItem(pg.GraphicsWidgetAnchor, pg.GraphicsWidget):
+    def __init__(self, parent, rect):
+        pg.GraphicsWidget.__init__(self, parent)
+        pg.GraphicsWidgetAnchor.__init__(self)
+        self.item = QtWidgets.QGraphicsRectItem(rect, self)
+        brush = QtGui.QBrush(QtGui.QColor(10, 10, 10, 140))
+        self.item.setBrush(brush)
+
+
+class _BaseWaveform(pg.PlotWidget):
+    cursorMove = QtCore.pyqtSignal(float)
+
+    def __init__(self, name, width, precision, unit,
+                 parent=None, pen="r", stepMode="right", connect="finite"):
+        pg.PlotWidget.__init__(self,
+                               parent=parent,
+                               x=None,
+                               y=None,
+                               pen=pen,
+                               stepMode=stepMode,
+                               connect=connect)
+
+        self.setMinimumHeight(WAVEFORM_MIN_HEIGHT)
+        self.setMaximumHeight(WAVEFORM_MAX_HEIGHT)
+        self.setMenuEnabled(False)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+
+        self.name = name
+        self.width = width
+        self.precision = precision
+        self.unit = unit
+
+        self.x_data = []
+        self.y_data = []
+
+        self.plot_item = self.getPlotItem()
+        self.plot_item.hideButtons()
+        self.plot_item.hideAxis("top")
+        self.plot_item.getAxis("bottom").setStyle(showValues=False, tickLength=0)
+        self.plot_item.getAxis("left").setStyle(showValues=False, tickLength=0)
+        self.plot_item.setRange(yRange=(0, 1), padding=0.1)
+        self.plot_item.showGrid(x=True, y=True)
+
+        self.plot_data_item = self.plot_item.listDataItems()[0]
+        self.plot_data_item.setClipToView(True)
+
+        self.view_box = self.plot_item.getViewBox()
+        self.view_box.setMouseEnabled(x=True, y=False)
+        self.view_box.disableAutoRange(axis=pg.ViewBox.YAxis)
+        self.view_box.setLimits(xMin=0, minXRange=20)
+
+        self.title_label = pg.LabelItem(self.name, parent=self.plot_item)
+        self.title_label.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 0))
+        self.title_label.setAttr('justify', 'left')
+        self.title_label.setZValue(10)
+
+        rect = self.title_label.boundingRect()
+        rect.setHeight(rect.height() * 2)
+        rect.setWidth(225)
+        self.label_bg = _BackgroundItem(parent=self.plot_item, rect=rect)
+        self.label_bg.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 0))
+
+        self.cursor = pg.InfiniteLine()
+        self.cursor_y = None
+        self.addItem(self.cursor)
+
+        self.cursor_label = pg.LabelItem('', parent=self.plot_item)
+        self.cursor_label.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 20))
+        self.cursor_label.setAttr('justify', 'left')
+        self.cursor_label.setZValue(10)
+
+    def setStoppedX(self, stopped_x):
+        self.stopped_x = stopped_x
+        self.view_box.setLimits(xMax=stopped_x)
+
+    def setData(self, data):
+        if len(data) == 0:
+            self.x_data, self.y_data = [], []
+        else:
+            self.x_data, self.y_data = zip(*data)
+
+    def onDataChange(self, data):
+        raise NotImplementedError
+
+    def onCursorMove(self, x):
+        self.cursor.setValue(x)
+        if len(self.x_data) < 1:
+            return
+        ind = bisect.bisect_left(self.x_data, x) - 1
+        dr = self.plot_data_item.dataRect()
+        self.cursor_y = None
+        if dr is not None and 0 <= ind < len(self.y_data):
+            self.cursor_y = self.y_data[ind]
+
+    def mouseMoveEvent(self, e):
+        if e.buttons() == QtCore.Qt.LeftButton \
+           and e.modifiers() == QtCore.Qt.ShiftModifier:
+            drag = QtGui.QDrag(self)
+            mime = QtCore.QMimeData()
+            drag.setMimeData(mime)
+            pixmapi = QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileIcon)
+            drag.setPixmap(pixmapi.pixmap(32))
+            drag.exec_(QtCore.Qt.MoveAction)
+        else:
+            super().mouseMoveEvent(e)
+
+    def wheelEvent(self, e):
+        if e.modifiers() & QtCore.Qt.ControlModifier:
+            super().wheelEvent(e)
+
+    def mouseDoubleClickEvent(self, e):
+        pos = self.view_box.mapSceneToView(e.pos())
+        self.cursorMove.emit(pos.x())
+
+
+class BitWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, unit, parent)
+        self.plot_item.showGrid(x=True, y=False)
+        self._arrows = []
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for arw in self._arrows:
+                self.removeItem(arw)
+            self._arrows = []
+            l = len(data)
+            display_y = np.empty(l)
+            display_x = np.empty(l)
+            display_map = {
+                "X": 0.5,
+                "1": 1,
+                "0": 0
+            }
+            previous_y = None
+            for i, coord in enumerate(data):
+                x, y = coord
+                dis_y = display_map[y]
+                if previous_y == y:
+                    arw = pg.ArrowItem(pxMode=True, angle=90)
+                    self.addItem(arw)
+                    self._arrows.append(arw)
+                    arw.setPos(x, dis_y)
+                display_y[i] = dis_y
+                display_x[i] = x
+                previous_y = y
+            self.plot_data_item.setData(x=display_x, y=display_y)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for arw in self._arrows:
+                self.removeItem(arw)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            self.cursor_label.setText(self.cursor_y)
+        else:
+            self.cursor_label.setText("")
+
+
+class AnalogWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, unit, parent)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            self.plot_data_item.setData(x=self.x_data, y=self.y_data)
+            if len(data) > 0:
+                max_y = max(self.y_data)
+                min_y = min(self.y_data)
+                self.plot_item.setRange(yRange=(min_y, max_y), padding=0.1)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            t = short_format(self.cursor_y, {"precision": self.precision, "unit": self.unit})
+        else:
+            t = ""
+        self.cursor_label.setText(t)
+
+
+class BitVectorWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, parent)
+        self._labels = []
+        self._format_string = "{:0=" + str(math.ceil(width / 4)) + "X}"
+        self.view_box.sigTransformChanged.connect(self._update_labels)
+        self.plot_item.showGrid(x=True, y=False)
+
+    def _update_labels(self):
+        for label in self._labels:
+            self.removeItem(label)
+        xmin, xmax = self.view_box.viewRange()[0]
+        left_label_i = bisect.bisect_left(self.x_data, xmin)
+        right_label_i = bisect.bisect_right(self.x_data, xmax) + 1
+        for i, j in itertools.pairwise(range(left_label_i, right_label_i)):
+            x1 = self.x_data[i]
+            x2 = self.x_data[j] if j < len(self.x_data) else self.stopped_x
+            lbl = self._labels[i]
+            bounds = lbl.boundingRect()
+            bounds_view = self.view_box.mapSceneToView(bounds)
+            if bounds_view.boundingRect().width() < x2 - x1:
+                self.addItem(lbl)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self._labels = []
+            l = len(data)
+            display_x = np.empty(l * 2)
+            display_y = np.empty(l * 2)
+            for i, coord in enumerate(data):
+                x, y = coord
+                display_x[i * 2] = x
+                display_x[i * 2 + 1] = x
+                display_y[i * 2] = 0
+                display_y[i * 2 + 1] = int(int(y) != 0)
+                lbl = pg.TextItem(
+                    self._format_string.format(int(y, 2)), anchor=(0, 0.5))
+                lbl.setPos(x, 0.5)
+                lbl.setTextWidth(100)
+                self._labels.append(lbl)
+            self.plot_data_item.setData(x=display_x, y=display_y)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            t = self._format_string.format(int(self.cursor_y, 2))
+        else:
+            t = ""
+        self.cursor_label.setText(t)
+
+
+class LogWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, parent)
+        self.plot_data_item.opts['pen'] = None
+        self.plot_data_item.opts['symbol'] = 'x'
+        self._labels = []
+        self.plot_item.showGrid(x=True, y=False)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self._labels = []
+            self.plot_data_item.setData(
+                x=self.x_data, y=np.ones(len(self.x_data)))
+            if len(data) == 0:
+                return
+            old_x = data[0][0]
+            old_msg = data[0][1]
+            for x, msg in data[1:]:
+                if x == old_x:
+                    old_msg += "\n" + msg
+                else:
+                    lbl = pg.TextItem(old_msg)
+                    self.addItem(lbl)
+                    self._labels.append(lbl)
+                    lbl.setPos(old_x, 1)
+                    old_msg = msg
+                    old_x = x
+            lbl = pg.TextItem(old_msg)
+            self.addItem(lbl)
+            self._labels.append(lbl)
+            lbl.setPos(old_x, 1)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self.plot_data_item.setData(x=[], y=[])
+
+
+class _WaveformView(QtWidgets.QWidget):
+    cursorMove = QtCore.pyqtSignal(float)
+
+    def __init__(self, parent):
+        QtWidgets.QWidget.__init__(self, parent=parent)
+
+        self._stopped_x = None
+        self._timescale = 1
+        self._cursor_x = 0
+
+        layout = QtWidgets.QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+        layout.setSpacing(0)
+        self.setLayout(layout)
+
+        self._ref_axis = pg.PlotWidget()
+        self._ref_axis.hideAxis("bottom")
+        self._ref_axis.hideAxis("left")
+        self._ref_axis.hideButtons()
+        self._ref_axis.setFixedHeight(45)
+        self._ref_axis.setMenuEnabled(False)
+        self._top = pg.AxisItem("top")
+        self._top.setScale(1e-12)
+        self._top.setLabel(units="s")
+        self._ref_axis.setAxisItems({"top": self._top})
+        layout.addWidget(self._ref_axis)
+
+        self._ref_vb = self._ref_axis.getPlotItem().getViewBox()
+        self._ref_vb.setFixedHeight(0)
+        self._ref_vb.setMouseEnabled(x=True, y=False)
+        self._ref_vb.setLimits(xMin=0)
+
+        scroll_area = VDragScrollArea(self)
+        scroll_area.setWidgetResizable(True)
+        scroll_area.setContentsMargins(0, 0, 0, 0)
+        scroll_area.setFrameShape(QtWidgets.QFrame.NoFrame)
+        scroll_area.setVerticalScrollBarPolicy(QtCore.Qt.ScrollBarAlwaysOff)
+        layout.addWidget(scroll_area)
+
+        self._splitter = VDragDropSplitter(parent=scroll_area)
+        self._splitter.setHandleWidth(1)
+        scroll_area.setWidget(self._splitter)
+
+        self.cursorMove.connect(self.onCursorMove)
+
+        self.confirm_delete_dialog = QtWidgets.QMessageBox(self)
+        self.confirm_delete_dialog.setIcon(
+            QtWidgets.QMessageBox.Icon.Warning
+        )
+        self.confirm_delete_dialog.setText("Delete all waveforms?")
+        self.confirm_delete_dialog.setStandardButtons(
+            QtWidgets.QMessageBox.Ok | QtWidgets.QMessageBox.Cancel
+        )
+        self.confirm_delete_dialog.setDefaultButton(
+            QtWidgets.QMessageBox.Ok
+        )
+
+    def setModel(self, model):
+        self._model = model
+        self._model.dataChanged.connect(self.onDataChange)
+        self._model.rowsInserted.connect(self.onInsert)
+        self._model.rowsRemoved.connect(self.onRemove)
+        self._model.rowsMoved.connect(self.onMove)
+        self._splitter.dropped.connect(self._model.move)
+        self.confirm_delete_dialog.accepted.connect(self._model.clear)
+
+    def setTimescale(self, timescale):
+        self._timescale = timescale
+        self._top.setScale(1e-12 * timescale)
+
+    def setStoppedX(self, stopped_x):
+        self._stopped_x = stopped_x
+        self._ref_vb.setLimits(xMax=stopped_x)
+        self._ref_vb.setRange(xRange=(0, stopped_x))
+        for i in range(self._model.rowCount()):
+            self._splitter.widget(i).setStoppedX(stopped_x)
+
+    def resetZoom(self):
+        if self._stopped_x is not None:
+            self._ref_vb.setRange(xRange=(0, self._stopped_x))
+
+    def onDataChange(self, top, bottom, roles):
+        self.cursorMove.emit(0)
+        first = top.row()
+        last = bottom.row()
+        data_row = self._model.headers.index("data")
+        for i in range(first, last + 1):
+            data = self._model.data(self._model.index(i, data_row))
+            self._splitter.widget(i).onDataChange(data)
+
+    def onInsert(self, parent, first, last):
+        for i in range(first, last + 1):
+            w = self._create_waveform(i)
+            self._splitter.insertWidget(i, w)
+        self._resize()
+
+    def onRemove(self, parent, first, last):
+        for i in reversed(range(first, last + 1)):
+            w = self._splitter.widget(i)
+            w.deleteLater()
+        self._splitter.refresh()
+        self._resize()
+
+    def onMove(self, src_parent, src_start, src_end, dest_parent, dest_row):
+        w = self._splitter.widget(src_start)
+        self._splitter.insertWidget(dest_row, w)
+
+    def onCursorMove(self, x):
+        self._cursor_x = x
+        for i in range(self._model.rowCount()):
+            self._splitter.widget(i).onCursorMove(x)
+
+    def _create_waveform(self, row):
+        name, ty, width, precision, unit = (
+            self._model.data(self._model.index(row, i)) for i in range(5))
+        waveform_cls = {
+            WaveformType.BIT: BitWaveform,
+            WaveformType.VECTOR: BitVectorWaveform,
+            WaveformType.ANALOG: AnalogWaveform,
+            WaveformType.LOG: LogWaveform
+        }[ty]
+        w = waveform_cls(name, width, precision, unit, parent=self._splitter)
+        w.setXLink(self._ref_vb)
+        w.setStoppedX(self._stopped_x)
+        w.cursorMove.connect(self.cursorMove)
+        w.onCursorMove(self._cursor_x)
+        action = QtWidgets.QAction("Delete waveform", w)
+        action.triggered.connect(lambda: self._delete_waveform(w))
+        w.addAction(action)
+        action = QtWidgets.QAction("Delete all waveforms", w)
+        action.triggered.connect(self.confirm_delete_dialog.open)
+        w.addAction(action)
+        return w
+
+    def _delete_waveform(self, waveform):
+        row = self._splitter.indexOf(waveform)
+        self._model.pop(row)
+
+    def _resize(self):
+        self._splitter.setFixedHeight(
+            int((WAVEFORM_MIN_HEIGHT + WAVEFORM_MAX_HEIGHT) * self._model.rowCount() / 2))
+
+
+class _WaveformModel(QtCore.QAbstractTableModel):
+    def __init__(self):
+        self.backing_struct = []
+        self.headers = ["name", "type", "width", "precision", "unit", "data"]
+        QtCore.QAbstractTableModel.__init__(self)
+
+    def rowCount(self, parent=QtCore.QModelIndex()):
+        return len(self.backing_struct)
+
+    def columnCount(self, parent=QtCore.QModelIndex()):
+        return len(self.headers)
+
+    def data(self, index, role=QtCore.Qt.DisplayRole):
+        if index.isValid():
+            return self.backing_struct[index.row()][index.column()]
+        return None
+
+    def extend(self, data):
+        length = len(self.backing_struct)
+        len_data = len(data)
+        self.beginInsertRows(QtCore.QModelIndex(), length, length + len_data - 1)
+        self.backing_struct.extend(data)
+        self.endInsertRows()
+
+    def pop(self, row):
+        self.beginRemoveRows(QtCore.QModelIndex(), row, row)
+        self.backing_struct.pop(row)
+        self.endRemoveRows()
+
+    def move(self, src, dest):
+        if src == dest:
+            return
+        if src < dest:
+            dest, src = src, dest
+        self.beginMoveRows(QtCore.QModelIndex(), src, src, QtCore.QModelIndex(), dest)
+        self.backing_struct.insert(dest, self.backing_struct.pop(src))
+        self.endMoveRows()
+
+    def clear(self):
+        self.beginRemoveRows(QtCore.QModelIndex(), 0, len(self.backing_struct) - 1)
+        self.backing_struct.clear()
+        self.endRemoveRows()
+
+    def export_list(self):
+        return [[row[0], row[1].value, *row[2:5]] for row in self.backing_struct]
+
+    def import_list(self, channel_list):
+        self.clear()
+        data = [[row[0], WaveformType(row[1]), *row[2:5], []] for row in channel_list]
+        self.extend(data)
+
+    def update_data(self, waveform_data, top, bottom):
+        name_col = self.headers.index("name")
+        data_col = self.headers.index("data")
+        for i in range(top, bottom):
+            name = self.data(self.index(i, name_col))
+            self.backing_struct[i][data_col] = waveform_data.get(name, [])
+            self.dataChanged.emit(self.index(i, data_col),
+                                  self.index(i, data_col))
+
+    def update_all(self, waveform_data):
+        self.update_data(waveform_data, 0, self.rowCount())
+
+
+class _CursorTimeControl(QtWidgets.QLineEdit):
+    submit = QtCore.pyqtSignal(float)
+
+    def __init__(self, parent):
+        QtWidgets.QLineEdit.__init__(self, parent=parent)
+        self._text = ""
+        self._value = 0
+        self._timescale = 1
+        self.setDisplayValue(0)
+        self.textChanged.connect(self._onTextChange)
+        self.returnPressed.connect(self._onReturnPress)
+
+    def setTimescale(self, timescale):
+        self._timescale = timescale
+
+    def _onTextChange(self, text):
+        self._text = text
+
+    def setDisplayValue(self, value):
+        self._value = value
+        self._text = pg.siFormat(value * 1e-12 * self._timescale,
+                                 suffix="s",
+                                 allowUnicode=False,
+                                 precision=15)
+        self.setText(self._text)
+
+    def _setValueFromText(self, text):
+        try:
+            self._value = pg.siEval(text) * (1e12 / self._timescale)
+        except:
+            logger.error("Error when parsing cursor time input", exc_info=True)
+
+    def _onReturnPress(self):
+        self._setValueFromText(self._text)
+        self.setDisplayValue(self._value)
+        self.submit.emit(self._value)
+        self.clearFocus()
+
+
+class Model(DictSyncTreeSepModel):
+    def __init__(self, init):
+        DictSyncTreeSepModel.__init__(self, "/", ["Channels"], init)
+
+    def clear(self):
+        for k in self.backing_store:
+            self._del_item(self, k.split(self.separator))
+        self.backing_store.clear()
+
+    def update(self, d):
+        for k, v in d.items():
+            self[k] = v
+
+
+class _AddChannelDialog(QtWidgets.QDialog):
+    def __init__(self, parent, model):
+        QtWidgets.QDialog.__init__(self, parent=parent)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+        self.setWindowTitle("Add channels")
+
+        layout = QtWidgets.QVBoxLayout()
+        self.setLayout(layout)
+
+        self._model = model
+        self._tree_view = QtWidgets.QTreeView()
+        self._tree_view.setHeaderHidden(True)
+        self._tree_view.setSelectionBehavior(
+            QtWidgets.QAbstractItemView.SelectItems)
+        self._tree_view.setSelectionMode(
+            QtWidgets.QAbstractItemView.ExtendedSelection)
+        self._tree_view.setModel(self._model)
+        layout.addWidget(self._tree_view)
+
+        self._button_box = QtWidgets.QDialogButtonBox(
+            QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel
+        )
+        self._button_box.setCenterButtons(True)
+        self._button_box.accepted.connect(self.add_channels)
+        self._button_box.rejected.connect(self.reject)
+        layout.addWidget(self._button_box)
+
+    def add_channels(self):
+        selection = self._tree_view.selectedIndexes()
+        channels = []
+        for select in selection:
+            key = self._model.index_to_key(select)
+            if key is not None:
+                channels.append([key, *self._model[key].ref, []])
+        self.channels = channels
+        self.accept()
+
+
+class WaveformDock(QtWidgets.QDockWidget):
+    def __init__(self, timeout, timer, timer_backoff):
+        QtWidgets.QDockWidget.__init__(self, "Waveform")
+        self.setObjectName("Waveform")
+        self.setFeatures(
+            QtWidgets.QDockWidget.DockWidgetMovable | QtWidgets.QDockWidget.DockWidgetFloatable)
+
+        self._channel_model = Model({})
+        self._waveform_model = _WaveformModel()
+
+        self._ddb = None
+        self._dump = None
+
+        self._waveform_data = {
+            "timescale": 1,
+            "stopped_x": None,
+            "logs": dict(),
+            "data": dict(),
+        }
+
+        self._current_dir = os.getcwd()
+
+        self.proxy_client = ProxyClient(self.on_dump_receive,
+                                        timeout,
+                                        timer,
+                                        timer_backoff)
+
+        grid = LayoutWidget()
+        self.setWidget(grid)
+
+        self._menu_btn = QtWidgets.QPushButton()
+        self._menu_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogStart))
+        grid.addWidget(self._menu_btn, 0, 0)
+
+        self._request_dump_btn = QtWidgets.QToolButton()
+        self._request_dump_btn.setToolTip("Fetch analyzer data from device")
+        self._request_dump_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_BrowserReload))
+        self._request_dump_btn.clicked.connect(
+            lambda: asyncio.ensure_future(exc_to_warning(self.proxy_client.trigger_proxy_task())))
+        grid.addWidget(self._request_dump_btn, 0, 1)
+
+        self._add_channel_dialog = _AddChannelDialog(self, self._channel_model)
+        self._add_channel_dialog.accepted.connect(self._add_channels)
+
+        self._add_btn = QtWidgets.QToolButton()
+        self._add_btn.setToolTip("Add channels...")
+        self._add_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogListView))
+        self._add_btn.clicked.connect(self._add_channel_dialog.open)
+        grid.addWidget(self._add_btn, 0, 2)
+
+        self._file_menu = QtWidgets.QMenu()
+        self._add_async_action("Open trace...", self.load_trace)
+        self._add_async_action("Save trace...", self.save_trace)
+        self._add_async_action("Save trace as VCD...", self.save_vcd)
+        self._add_async_action("Open channel list...", self.load_channels)
+        self._add_async_action("Save channel list...", self.save_channels)
+        self._menu_btn.setMenu(self._file_menu)
+
+        self._waveform_view = _WaveformView(self)
+        self._waveform_view.setModel(self._waveform_model)
+        grid.addWidget(self._waveform_view, 1, 0, colspan=12)
+
+        self._reset_zoom_btn = QtWidgets.QToolButton()
+        self._reset_zoom_btn.setToolTip("Reset zoom")
+        self._reset_zoom_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_TitleBarMaxButton))
+        self._reset_zoom_btn.clicked.connect(self._waveform_view.resetZoom)
+        grid.addWidget(self._reset_zoom_btn, 0, 3)
+
+        self._cursor_control = _CursorTimeControl(self)
+        self._waveform_view.cursorMove.connect(self._cursor_control.setDisplayValue)
+        self._cursor_control.submit.connect(self._waveform_view.onCursorMove)
+        grid.addWidget(self._cursor_control, 0, 4, colspan=6)
+
+    def _add_async_action(self, label, coro):
+        action = QtWidgets.QAction(label, self)
+        action.triggered.connect(
+            lambda: asyncio.ensure_future(exc_to_warning(coro())))
+        self._file_menu.addAction(action)
+
+    def _add_channels(self):
+        channels = self._add_channel_dialog.channels
+        count = self._waveform_model.rowCount()
+        self._waveform_model.extend(channels)
+        self._waveform_model.update_data(self._waveform_data['data'],
+                                         count,
+                                         count + len(channels))
+
+    def on_dump_receive(self, dump):
+        self._dump = dump
+        decoded_dump = comm_analyzer.decode_dump(dump)
+        waveform_data = comm_analyzer.decoded_dump_to_waveform_data(self._ddb, decoded_dump)
+        self._waveform_data.update(waveform_data)
+        self._channel_model.update(self._waveform_data['logs'])
+        self._waveform_model.update_all(self._waveform_data['data'])
+        self._waveform_view.setStoppedX(self._waveform_data['stopped_x'])
+        self._waveform_view.setTimescale(self._waveform_data['timescale'])
+        self._cursor_control.setTimescale(self._waveform_data['timescale'])
+
+    async def load_trace(self):
+        try:
+            filename = await get_open_file_name(
+                self,
+                "Load Analyzer Trace",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            with open(filename, 'rb') as f:
+                dump = f.read()
+            self.on_dump_receive(dump)
+        except:
+            logger.error("Failed to open analyzer trace", exc_info=True)
+
+    async def save_trace(self):
+        if self._dump is None:
+            logger.error("No analyzer trace stored in dashboard, "
+                         "try loading from file or fetching from device")
+            return
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save Analyzer Trace",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            with open(filename, 'wb') as f:
+                f.write(self._dump)
+        except:
+            logger.error("Failed to save analyzer trace", exc_info=True)
+
+    async def save_vcd(self):
+        if self._dump is None:
+            logger.error("No analyzer trace stored in dashboard, "
+                         "try loading from file or fetching from device")
+            return
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save VCD",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            decoded_dump = comm_analyzer.decode_dump(self._dump)
+            with open(filename, 'w') as f:
+                comm_analyzer.decoded_dump_to_vcd(f, self._ddb, decoded_dump)
+        except:
+            logger.error("Failed to save trace as VCD", exc_info=True)
+
+    async def load_channels(self):
+        try:
+            filename = await get_open_file_name(
+                self,
+                "Open channel list",
+                self._current_dir,
+                "PYON files (*.pyon);;All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            channel_list = pyon.load_file(filename)
+            self._waveform_model.import_list(channel_list)
+            self._waveform_model.update_all(self._waveform_data['data'])
+        except:
+            logger.error("Failed to open channel list", exc_info=True)
+
+    async def save_channels(self):
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save channel list",
+                self._current_dir,
+                "PYON files (*.pyon);;All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            channel_list = self._waveform_model.export_list()
+            pyon.store_file(filename, channel_list)
+        except:
+            logger.error("Failed to save channel list", exc_info=True)
+
+    def _process_ddb(self):
+        channel_list = comm_analyzer.get_channel_list(self._ddb)
+        self._channel_model.clear()
+        self._channel_model.update(channel_list)
+        desc = self._ddb.get("core_analyzer")
+        if desc is not None:
+            addr = desc["host"]
+            port_proxy = desc.get("port_proxy", 1385)
+            port = desc.get("port", 1386)
+            self.proxy_client.update_address(addr, port, port_proxy)
+        else:
+            self.proxy_client.update_address(None, None, None)
+
+    def init_ddb(self, ddb):
+        self._ddb = ddb
+        self._process_ddb()
+        return ddb
+
+    def notify_ddb(self, mod):
+        self._process_ddb()
+
+    async def stop(self):
+        if self.proxy_client is not None:
+            await self.proxy_client.close()

artiq/examples/artiq_ipython_notebook.ipynb

@@ -127,7 +127,7 @@
     "# let's connect to the master\n",
     "\n",
     "schedule, exps, datasets = [\n",
-    "    Client(\"::1\", 3251, \"master_\" + i) for i in\n",
+    "    Client(\"::1\", 3251, i) for i in\n",
     "    \"schedule experiment_db dataset_db\".split()]\n",
     "\n",
     "print(\"current schedule\")\n",

artiq/examples/kasli_shuttler/kasli_shuttler.json

@@ -6,6 +6,7 @@
     "peripherals": [
         {
             "type": "shuttler",
+            "hw_rev": "v1.1",
             "ports": [0]
         },
         {

artiq/examples/no_hardware/repository/interactive.py

@@ -0,0 +1,34 @@
+from artiq.experiment import *
+
+
+class InteractiveDemo(EnvExperiment):
+    def build(self):
+        pass
+
+    def run(self):
+        print("Waiting for user input...")
+        with self.interactive(title="Interactive Demo") as interactive:
+            interactive.setattr_argument("pyon_value",
+                                         PYONValue(self.get_dataset("foo", default=42)))
+            interactive.setattr_argument("number", NumberValue(42e-6,
+                                         unit="us",
+                                         precision=4))
+            interactive.setattr_argument("integer", NumberValue(42,
+                                         step=1, precision=0))
+            interactive.setattr_argument("string", StringValue("Hello World"))
+            interactive.setattr_argument("scan", Scannable(global_max=400,
+                                         default=NoScan(325),
+                                         precision=6))
+            interactive.setattr_argument("boolean", BooleanValue(True), "Group")
+            interactive.setattr_argument("enum",
+                                         EnumerationValue(["foo", "bar", "quux"], "foo"),
+                                         "Group")
+        print("Done! Values:")
+        print(interactive.pyon_value)
+        print(interactive.boolean)
+        print(interactive.enum)
+        print(interactive.number, type(interactive.number))
+        print(interactive.integer, type(interactive.integer))
+        print(interactive.string)
+        for i in interactive.scan:
+            print(i)

artiq/firmware/Cargo.lock

@@ -1,10 +1,50 @@
 # This file is automatically @generated by Cargo.
 # It is not intended for manual editing.
+# And yet, manual edits have been made. Crate yanking should be illegal.
+version = 3
+
+[[package]]
+name = "addr2line"
+version = "0.16.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3e61f2b7f93d2c7d2b08263acaa4a363b3e276806c68af6134c44f523bf1aacd"
+dependencies = [
+ "cpp_demangle",
+ "fallible-iterator",
+ "gimli",
+ "object",
+ "rustc-demangle",
+ "smallvec",
+]
+
+[[package]]
+name = "adler"
+version = "0.2.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ee2a4ec343196209d6594e19543ae87a39f96d5534d7174822a3ad825dd6ed7e"
+
+[[package]]
+name = "adler"
+version = "1.0.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f26201604c87b1e01bd3d98f8d5d9a8fcbb815e8cedb41ffccbeb4bf593a35fe"
+
+[[package]]
+name = "ahash"
+version = "0.7.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "efa60d2eadd8b12a996add391db32bd1153eac697ba4869660c0016353611426"
+dependencies = [
+ "getrandom",
+ "once_cell",
+ "version_check",
+]
+
 [[package]]
 name = "aho-corasick"
-version = "0.7.18"
+version = "0.7.20"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1e37cfd5e7657ada45f742d6e99ca5788580b5c529dc78faf11ece6dc702656f"
+checksum = "cc936419f96fa211c1b9166887b38e5e40b19958e5b895be7c1f93adec7071ac"
 dependencies = [
  "memchr",
 ]
@@ -30,9 +70,9 @@ dependencies = [
 
 [[package]]
 name = "bit_field"
-version = "0.10.1"
+version = "0.10.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dcb6dd1c2376d2e096796e234a70e17e94cc2d5d54ff8ce42b28cef1d0d359a4"
+checksum = "dc827186963e592360843fb5ba4b973e145841266c1357f7180c43526f2e5b61"
 
 [[package]]
 name = "bitflags"
@@ -72,12 +112,30 @@ dependencies = [
 name = "bootloader"
 version = "0.0.0"
 dependencies = [
+ "addr2line",
+ "ahash",
  "board_misoc",
  "build_misoc",
  "byteorder",
+ "cc",
+ "compiler_builtins",
  "crc",
+ "dlmalloc",
+ "fortanix-sgx-abi",
+ "getopts",
+ "getrandom",
+ "hashbrown",
+ "hermit-abi",
+ "libc 0.2.99",
+ "miniz_oxide 0.4.0",
+ "object",
+ "once_cell",
  "riscv",
+ "rustc-demangle",
  "smoltcp",
+ "unicode-width",
+ "version_check",
+ "wasi",
 ]
 
 [[package]]
@@ -98,9 +156,9 @@ checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
 
 [[package]]
 name = "cc"
-version = "1.0.70"
+version = "1.0.69"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d26a6ce4b6a484fa3edb70f7efa6fc430fd2b87285fe8b84304fd0936faa0dc0"
+checksum = "e70cc2f62c6ce1868963827bd677764c62d07c3d9a3e1fb1177ee1a9ab199eb2"
 
 [[package]]
 name = "cfg-if"
@@ -116,9 +174,18 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
 
 [[package]]
 name = "compiler_builtins"
-version = "0.1.39"
+version = "0.1.49"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3748f82c7d366a0b4950257d19db685d4958d2fa27c6d164a3f069fec42b748b"
+checksum = "20b1438ef42c655665a8ab2c1c6d605a305f031d38d9be689ddfef41a20f3aa2"
+
+[[package]]
+name = "cpp_demangle"
+version = "0.3.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "eeaa953eaad386a53111e47172c2fedba671e5684c8dd601a5f474f4f118710f"
+dependencies = [
+ "cfg-if 1.0.0",
+]
 
 [[package]]
 name = "crc"
@@ -129,12 +196,30 @@ dependencies = [
  "build_const",
 ]
 
+[[package]]
+name = "crc32fast"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b3855a8a784b474f333699ef2bbca9db2c4a1f6d9088a90a2d25b1eb53111eaa"
+dependencies = [
+ "cfg-if 1.0.0",
+]
+
 [[package]]
 name = "cslice"
 version = "0.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "0f8cb7306107e4b10e64994de6d3274bd08996a7c1322a27b86482392f96be0a"
 
+[[package]]
+name = "dlmalloc"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "332570860c2edf2d57914987bf9e24835425f75825086b6ba7d1e6a3e4f1f254"
+dependencies = [
+ "libc 0.2.99",
+]
+
 [[package]]
 name = "dyld"
 version = "0.0.0"
@@ -143,7 +228,6 @@ version = "0.0.0"
 name = "eh"
 version = "0.0.0"
 dependencies = [
- "compiler_builtins",
  "cslice",
  "libc 0.1.0",
  "unwind",
@@ -166,12 +250,82 @@ dependencies = [
  "synstructure",
 ]
 
+[[package]]
+name = "fallible-iterator"
+version = "0.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
+
+[[package]]
+name = "flate2"
+version = "1.0.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5f54427cfd1c7829e2a139fcefea601bf088ebca651d2bf53ebc600eac295dae"
+dependencies = [
+ "crc32fast",
+ "miniz_oxide 0.7.2",
+]
+
+[[package]]
+name = "fortanix-sgx-abi"
+version = "0.3.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c56c422ef86062869b2d57ae87270608dc5929969dd130a6e248979cf4fb6ca6"
+
 [[package]]
 name = "fringe"
 version = "1.2.1"
-source = "git+https://git.m-labs.hk/M-Labs/libfringe.git?rev=3ecbe5#3ecbe53f7644b18ee46ebd5b2ca12c9cbceec43a"
+source = "git+https://git.m-labs.hk/M-Labs/libfringe.git?rev=53a964#53a964a63d2d384b22ae1949a471a732003a30b9"
 dependencies = [
- "libc 0.2.101",
+ "libc 0.2.99",
+]
+
+[[package]]
+name = "getopts"
+version = "0.2.21"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "14dbbfd5c71d70241ecf9e6f13737f7b5ce823821063188d7e46c41d371eebd5"
+dependencies = [
+ "unicode-width",
+]
+
+[[package]]
+name = "getrandom"
+version = "0.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ee8025cf36f917e6a52cce185b7c7177689b838b7ec138364e50cc2277a56cf4"
+dependencies = [
+ "cfg-if 0.1.10",
+ "libc 0.2.99",
+ "wasi",
+]
+
+[[package]]
+name = "gimli"
+version = "0.25.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0a01e0497841a3b2db4f8afa483cce65f7e96a3498bd6c541734792aeac8fe7"
+dependencies = [
+ "fallible-iterator",
+ "stable_deref_trait",
+]
+
+[[package]]
+name = "hashbrown"
+version = "0.11.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "362385356d610bd1e5a408ddf8d022041774b683f345a1d2cfcb4f60f8ae2db5"
+dependencies = [
+ "ahash",
+]
+
+[[package]]
+name = "hermit-abi"
+version = "0.1.19"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "62b467343b94ba476dcb2500d242dadbb39557df889310ac77c5d99100aaac33"
+dependencies = [
+ "libc 0.2.99",
 ]
 
 [[package]]
@@ -212,9 +366,9 @@ version = "0.1.0"
 
 [[package]]
 name = "libc"
-version = "0.2.101"
+version = "0.2.99"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3cb00336871be5ed2c8ed44b60ae9959dc5b9f08539422ed43f09e34ecaeba21"
+checksum = "a7f823d141fe0a24df1e23b4af4e3c7ba9e5966ec514ea068c93024aa7deb765"
 
 [[package]]
 name = "log"
@@ -258,6 +412,40 @@ version = "2.4.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "308cc39be01b73d0d18f82a0e7b2a3df85245f84af96fdddc5d202d27e47b86a"
 
+[[package]]
+name = "miniz_oxide"
+version = "0.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "be0f75932c1f6cfae3c04000e40114adf955636e19040f9c0a2c380702aa1c7f"
+dependencies = [
+ "adler 0.2.3",
+]
+
+[[package]]
+name = "miniz_oxide"
+version = "0.7.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9d811f3e15f28568be3407c8e7fdb6514c1cda3cb30683f15b6a1a1dc4ea14a7"
+dependencies = [
+ "adler 1.0.2",
+]
+
+[[package]]
+name = "object"
+version = "0.26.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39f37e50073ccad23b6d09bcb5b263f4e76d3bb6038e4a3c08e52162ffa8abc2"
+dependencies = [
+ "flate2",
+ "memchr",
+]
+
+[[package]]
+name = "once_cell"
+version = "1.8.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "692fcb63b64b1758029e0a96ee63e049ce8c5948587f2f7208df04625e5f6b56"
+
 [[package]]
 name = "proto_artiq"
 version = "0.0.0"
@@ -280,9 +468,9 @@ checksum = "7a6e920b65c65f10b2ae65c831a81a073a89edd28c7cce89475bff467ab4167a"
 
 [[package]]
 name = "regex"
-version = "1.5.4"
+version = "1.7.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d07a8629359eb56f1e2fb1652bb04212c072a87ba68546a04065d525673ac461"
+checksum = "8b1f693b24f6ac912f4893ef08244d70b6067480d2f1a46e950c9691e6749d1d"
 dependencies = [
  "aho-corasick",
  "memchr",
@@ -291,9 +479,9 @@ dependencies = [
 
 [[package]]
 name = "regex-syntax"
-version = "0.6.25"
+version = "0.6.29"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f497285884f3fcff424ffc933e56d7cbca511def0c9831a7f9b5f6153e3cc89b"
+checksum = "f162c6dd7b008981e4d40210aca20b4bd0f9b60ca9271061b07f78537722f2e1"
 
 [[package]]
 name = "riscv"
@@ -342,6 +530,12 @@ dependencies = [
  "unwind_backtrace",
 ]
 
+[[package]]
+name = "rustc-demangle"
+version = "0.1.21"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7ef03e0a2b150c7a90d01faf6254c9c48a41e95fb2a8c2ac1c6f0d2b9aefc342"
+
 [[package]]
 name = "rustc_version"
 version = "0.2.3"
@@ -382,6 +576,12 @@ version = "0.7.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3"
 
+[[package]]
+name = "smallvec"
+version = "1.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3c5e1a9a646d36c3599cd173a41282daf47c44583ad367b8e6837255952e5c67"
+
 [[package]]
 name = "smoltcp"
 version = "0.8.2"
@@ -393,6 +593,12 @@ dependencies = [
  "managed 0.8.0",
 ]
 
+[[package]]
+name = "stable_deref_trait"
+version = "1.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3"
+
 [[package]]
 name = "syn"
 version = "0.11.11"
@@ -433,6 +639,12 @@ dependencies = [
  "log",
 ]
 
+[[package]]
+name = "unicode-width"
+version = "0.1.8"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9337591893a19b88d8d87f2cec1e73fad5cdfd10e5a6f349f498ad6ea2ffb1e3"
+
 [[package]]
 name = "unicode-xid"
 version = "0.0.4"
@@ -454,3 +666,15 @@ dependencies = [
  "libc 0.1.0",
  "unwind",
 ]
+
+[[package]]
+name = "version_check"
+version = "0.9.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5fecdca9a5291cc2b8dcf7dc02453fee791a280f3743cb0905f8822ae463b3fe"
+
+[[package]]
+name = "wasi"
+version = "0.9.0+wasi-snapshot-preview1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

artiq/firmware/bootloader/Cargo.toml

@@ -13,8 +13,30 @@ path = "main.rs"
 build_misoc = { path = "../libbuild_misoc" }
 
 [dependencies]
-byteorder = { version = "1.0", default-features = false }
+byteorder = { version = "=1.4.3", default-features = false }
 crc = { version = "1.7", default-features = false }
 board_misoc = { path = "../libboard_misoc", features = ["uart_console", "smoltcp"] }
 smoltcp = { version = "0.8.2", default-features = false, features = ["medium-ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"] }
 riscv = { version = "0.6.0", features = ["inline-asm"] }
+
+# insanity required by using cargo build-std over xbuild with nix
+# cargo update does not work thanks to ahash 0.7 problems
+[dev-dependencies]
+getopts = "=0.2.21"
+libc = "=0.2.99"
+unicode-width = "=0.1.8"
+addr2line = "=0.16.0"
+hashbrown = "=0.11.0" # must be injected into lockfile manually
+ahash = "=0.7.0" # must be injected into lockfile manually
+miniz_oxide = "=0.4.0"
+rustc-demangle = "=0.1.21"
+hermit-abi = "=0.1.19"
+dlmalloc = "=0.2.1"
+fortanix-sgx-abi = "=0.3.3"
+cc = "=1.0.69"
+compiler_builtins = "=0.1.49"
+version_check = "=0.9.3"
+once_cell = "=1.8.0"
+wasi = "=0.9.0"
+getrandom = "=0.2.0"
+object = "=0.26.2"

artiq/firmware/bootloader/Makefile

@@ -3,8 +3,6 @@ include $(MISOC_DIRECTORY)/software/common.mak
 
 RUSTFLAGS += -Cpanic=abort
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: bootloader.bin
 
 .PHONY: $(RUSTOUT)/libbootloader.a

artiq/firmware/bootloader/main.rs

@@ -500,7 +500,7 @@ pub extern fn main() -> i32 {
     println!(r"|_|  |_|_|____/ \___/ \____|");
     println!("");
     println!("MiSoC Bootloader");
-    println!("Copyright (c) 2017-2023 M-Labs Limited");
+    println!("Copyright (c) 2017-2024 M-Labs Limited");
     println!("");
 
     #[cfg(has_ethmac)]

artiq/firmware/ksupport/Makefile

@@ -14,8 +14,6 @@ LDFLAGS += --eh-frame-hdr \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: ksupport.elf
 
 .PHONY: $(RUSTOUT)/libksupport.a

artiq/firmware/ksupport/api.rs

@@ -105,6 +105,7 @@ static mut API: &'static [(&'static str, *const ())] = &[
     api!(nextafter),
     api!(pow),
     api!(round),
+    api!(rint),
     api!(sin),
     api!(sinh),
     api!(sqrt),

artiq/firmware/ksupport/eh_artiq.rs

@@ -55,12 +55,14 @@ struct ExceptionBuffer {
     exception_count: usize,
 }
 
-static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
-    uw_exceptions: [uw::_Unwind_Exception {
+const EXCEPTION: uw::_Unwind_Exception = uw::_Unwind_Exception {
     exception_class:   EXCEPTION_CLASS,
     exception_cleanup: cleanup,
     private:           [0; uw::unwinder_private_data_size],
-    }; MAX_INFLIGHT_EXCEPTIONS],
+};
+
+static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
+    uw_exceptions: [EXCEPTION; MAX_INFLIGHT_EXCEPTIONS],
     exceptions: [None; MAX_INFLIGHT_EXCEPTIONS + 1],
     exception_stack: [-1; MAX_INFLIGHT_EXCEPTIONS + 1],
     backtrace: [(0, 0); MAX_BACKTRACE_SIZE],
@@ -74,11 +76,7 @@ static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
 };
 
 pub unsafe extern fn reset_exception_buffer(payload_addr: usize) {
-    EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception {
-        exception_class:   EXCEPTION_CLASS,
-        exception_cleanup: cleanup,
-        private:           [0; uw::unwinder_private_data_size],
-    }; MAX_INFLIGHT_EXCEPTIONS];
+    EXCEPTION_BUFFER.uw_exceptions = [EXCEPTION; MAX_INFLIGHT_EXCEPTIONS];
     EXCEPTION_BUFFER.exceptions = [None; MAX_INFLIGHT_EXCEPTIONS + 1];
     EXCEPTION_BUFFER.exception_stack = [-1; MAX_INFLIGHT_EXCEPTIONS + 1];
     EXCEPTION_BUFFER.backtrace_size = 0;
@@ -151,8 +149,7 @@ pub extern fn personality(version: c_int,
 }
 
 #[export_name="__artiq_raise"]
-#[unwind(allowed)]
-pub unsafe extern fn raise(exception: *const Exception) -> ! {
+pub unsafe extern "C-unwind" fn raise(exception: *const Exception) -> ! {
     let count = EXCEPTION_BUFFER.exception_count;
     let stack = &mut EXCEPTION_BUFFER.exception_stack;
     let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize;
@@ -222,8 +219,7 @@ pub unsafe extern fn raise(exception: *const Exception) -> ! {
 
 
 #[export_name="__artiq_resume"]
-#[unwind(allowed)]
-pub unsafe extern fn resume() -> ! {
+pub unsafe extern "C-unwind" fn resume() -> ! {
     assert!(EXCEPTION_BUFFER.exception_count != 0);
     let i = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
     assert!(i != -1);
@@ -233,8 +229,7 @@ pub unsafe extern fn resume() -> ! {
 }
 
 #[export_name="__artiq_end_catch"]
-#[unwind(allowed)]
-pub unsafe extern fn end_catch() {
+pub unsafe extern "C-unwind" fn end_catch() {
     let mut count = EXCEPTION_BUFFER.exception_count;
     assert!(count != 0);
     // we remove all exceptions with SP <= current exception SP

artiq/firmware/ksupport/lib.rs

@@ -1,5 +1,5 @@
-#![feature(lang_items, llvm_asm, panic_unwind, libc, unwind_attributes,
-           panic_info_message, nll, const_in_array_repeat_expressions)]
+#![feature(lang_items, asm, panic_unwind, libc,
+           panic_info_message, nll, c_unwind)]
 #![no_std]
 
 extern crate libc;
@@ -30,8 +30,9 @@ fn send(request: &Message) {
 }
 
 fn recv<R, F: FnOnce(&Message) -> R>(f: F) -> R {
-    while mailbox::receive() == 0 {}
-    let result = f(unsafe { &*(mailbox::receive() as *const Message) });
+    let mut msg_ptr = 0;
+    while msg_ptr == 0 { msg_ptr = mailbox::receive(); }
+    let result = f(unsafe { &*(msg_ptr as *const Message) });
     mailbox::acknowledge();
     result
 }
@@ -121,7 +122,6 @@ pub extern fn send_to_rtio_log(text: CSlice<u8>) {
     rtio::log(text.as_ref())
 }
 
-#[unwind(aborts)]
 extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while !rpc_queue::empty() {}
     send(&RpcSend {
@@ -132,7 +132,6 @@ extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     })
 }
 
-#[unwind(aborts)]
 extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while rpc_queue::full() {}
     rpc_queue::enqueue(|mut slice| {
@@ -170,8 +169,7 @@ extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ())
 /// to the maximum required for any of the possible types according to the target ABI).
 ///
 /// If the RPC call resulted in an exception, it is reconstructed and raised.
-#[unwind(allowed)]
-extern fn rpc_recv(slot: *mut ()) -> usize {
+extern "C-unwind" fn rpc_recv(slot: *mut ()) -> usize {
     send(&RpcRecvRequest(slot));
     recv!(&RpcRecvReply(ref result) => {
         match result {
@@ -203,7 +201,6 @@ fn terminate(exceptions: &'static [Option<eh_artiq::Exception<'static>>],
     loop {}
 }
 
-#[unwind(aborts)]
 extern fn cache_get<'a>(key: &CSlice<u8>) -> *const CSlice<'a, i32> {
     send(&CacheGetRequest {
         key:   str::from_utf8(key.as_ref()).unwrap()
@@ -213,8 +210,7 @@ extern fn cache_get<'a>(key: &CSlice<u8>) -> *const CSlice<'a, i32> {
     })
 }
 
-#[unwind(allowed)]
-extern fn cache_put(key: &CSlice<u8>, list: &CSlice<i32>) {
+extern "C-unwind" fn cache_put(key: &CSlice<u8>, list: &CSlice<i32>) {
     send(&CachePutRequest {
         key:   str::from_utf8(key.as_ref()).unwrap(),
         value: list.as_ref()
@@ -247,8 +243,7 @@ fn dma_record_flush() {
     }
 }
 
-#[unwind(allowed)]
-extern fn dma_record_start(name: &CSlice<u8>) {
+extern "C-unwind" fn dma_record_start(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     unsafe {
@@ -267,8 +262,7 @@ extern fn dma_record_start(name: &CSlice<u8>) {
     }
 }
 
-#[unwind(allowed)]
-extern fn dma_record_stop(duration: i64, enable_ddma: bool) {
+extern "C-unwind" fn dma_record_stop(duration: i64, enable_ddma: bool) {
     unsafe {
         dma_record_flush();
 
@@ -290,7 +284,6 @@ extern fn dma_record_stop(duration: i64, enable_ddma: bool) {
     }
 }
 
-#[unwind(aborts)]
 #[inline(always)]
 unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
                                     words: usize) -> &'static mut [u8] {
@@ -327,7 +320,6 @@ unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
     data
 }
 
-#[unwind(aborts)]
 extern fn dma_record_output(target: i32, word: i32) {
     unsafe {
         let timestamp = ((csr::rtio::now_hi_read() as i64) << 32) | (csr::rtio::now_lo_read() as i64);
@@ -341,7 +333,6 @@ extern fn dma_record_output(target: i32, word: i32) {
     }
 }
 
-#[unwind(aborts)]
 extern fn dma_record_output_wide(target: i32, words: &CSlice<i32>) {
     assert!(words.len() <= 16); // enforce the hardware limit
 
@@ -360,7 +351,6 @@ extern fn dma_record_output_wide(target: i32, words: &CSlice<i32>) {
     }
 }
 
-#[unwind(aborts)]
 extern fn dma_erase(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
@@ -374,8 +364,7 @@ struct DmaTrace {
     uses_ddma: bool,
 }
 
-#[unwind(allowed)]
-extern fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
+extern "C-unwind" fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     send(&DmaRetrieveRequest { name: name });
@@ -396,8 +385,7 @@ extern fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
 }
 
 #[cfg(kernel_has_rtio_dma)]
-#[unwind(allowed)]
-extern fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
+extern "C-unwind" fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
     assert!(ptr % 64 == 0);
 
     unsafe {
@@ -453,15 +441,39 @@ extern fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
     }
 }
 
-#[cfg(not(kernel_has_rtio_dma))]
-#[unwind(allowed)]
-extern fn dma_playback(_timestamp: i64, _ptr: i32, _uses_ddma: bool) {
+#[cfg(all(not(kernel_has_rtio_dma), not(has_rtio_dma)))]
+extern "C-unwind" fn dma_playback(_timestamp: i64, _ptr: i32, _uses_ddma: bool) {
     unimplemented!("not(kernel_has_rtio_dma)")
 }
 
-#[unwind(allowed)]
-extern fn subkernel_load_run(id: u32, run: bool) {
-    send(&SubkernelLoadRunRequest { id: id, run: run });
+// for satellite (has_rtio_dma but not in kernel)
+#[cfg(all(not(kernel_has_rtio_dma), has_rtio_dma))]
+extern "C-unwind" fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
+    // DDMA is always used on satellites, so the `uses_ddma` setting is ignored
+    // StartRemoteRequest reused as "normal" start request
+    send(&DmaStartRemoteRequest { id: ptr as i32, timestamp: timestamp });
+    // skip awaitremoterequest - it's a given
+    recv!(&DmaAwaitRemoteReply { timeout, error, channel, timestamp } => {
+        if timeout {
+            raise!("DMAError",
+                "Error running DMA on satellite device, timed out waiting for results");
+        }
+        if error & 1 != 0 {
+            raise!("RTIOUnderflow",
+                "RTIO underflow at channel {rtio_channel_info:0}, {1} mu",
+                channel as i64, timestamp as i64, 0);
+        }
+        if error & 2 != 0 {
+            raise!("RTIODestinationUnreachable",
+                "RTIO destination unreachable, output, at channel {rtio_channel_info:0}, {1} mu",
+                channel as i64, timestamp as i64, 0);
+        }
+    });
+}
+
+
+extern "C-unwind" fn subkernel_load_run(id: u32, destination: u8, run: bool) {
+    send(&SubkernelLoadRunRequest { id: id, destination: destination, run: run });
     recv!(&SubkernelLoadRunReply { succeeded } => {
         if !succeeded {
             raise!("SubkernelError",
@@ -470,8 +482,7 @@ extern fn subkernel_load_run(id: u32, run: bool) {
     });
 }
 
-#[unwind(allowed)]
-extern fn subkernel_await_finish(id: u32, timeout: u64) {
+extern "C-unwind" fn subkernel_await_finish(id: u32, timeout: i64) {
     send(&SubkernelAwaitFinishRequest { id: id, timeout: timeout });
     recv!(SubkernelAwaitFinishReply { status } => {
         match status {
@@ -488,18 +499,18 @@ extern fn subkernel_await_finish(id: u32, timeout: u64) {
     })
 }
 
-#[unwind(aborts)]
-extern fn subkernel_send_message(id: u32, count: u8, tag: &CSlice<u8>, data: *const *const ()) {
+extern fn subkernel_send_message(id: u32, is_return: bool, destination: u8, 
+    count: u8, tag: &CSlice<u8>, data: *const *const ()) {
     send(&SubkernelMsgSend { 
         id: id,
+        destination: if is_return { None } else { Some(destination) },
         count: count,
         tag: tag.as_ref(),
         data: data 
     });
 }
 
-#[unwind(allowed)]
-extern fn subkernel_await_message(id: u32, timeout: u64, tags: &CSlice<u8>, min: u8, max: u8) -> u8 {
+extern "C-unwind" fn subkernel_await_message(id: i32, timeout: i64, tags: &CSlice<u8>, min: u8, max: u8) -> u8 {
     send(&SubkernelMsgRecvRequest { id: id, timeout: timeout, tags: tags.as_ref() });
     recv!(SubkernelMsgRecvReply { status, count } => {
         match status {
@@ -624,8 +635,7 @@ pub unsafe fn main() {
 }
 
 #[no_mangle]
-#[unwind(allowed)]
-pub unsafe extern fn exception(_regs: *const u32) {
+pub unsafe extern "C-unwind" fn exception(_regs: *const u32) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
     let mtval = mtval::read();
@@ -643,7 +653,6 @@ pub unsafe extern fn exception(_regs: *const u32) {
 }
 
 #[no_mangle]
-#[unwind(allowed)]
-pub extern fn abort() {
+pub extern "C-unwind" fn abort() {
     panic!("aborted")
 }

artiq/firmware/libboard_artiq/Cargo.toml

@@ -25,3 +25,4 @@ proto_artiq = { path = "../libproto_artiq" }
 [features]
 uart_console = []
 alloc = []
+calibrate_wrpll_skew = []

artiq/firmware/libboard_artiq/drtioaux.rs

@@ -69,9 +69,9 @@ fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error<!>>
     let linkidx = linkno as usize;
     unsafe {
         if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
-            let ptr = DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2;
-            let len = (DRTIOAUX[linkidx].aux_rx_length_read)();
-            let result = f(slice::from_raw_parts(ptr as *mut u8, len as usize));
+            let read_ptr = (DRTIOAUX[linkidx].aux_read_pointer_read)() as usize;
+            let ptr = DRTIOAUX_MEM[linkidx].base + (DRTIOAUX_MEM[linkidx].size / 2) + (read_ptr * 0x400);
+            let result = f(slice::from_raw_parts(ptr as *mut u8, 0x400 as usize));
             (DRTIOAUX[linkidx].aux_rx_present_write)(1);
             Ok(Some(result?))
         } else {
@@ -86,21 +86,17 @@ pub fn recv(linkno: u8) -> Result<Option<Packet>, Error<!>> {
     }
 
     receive(linkno, |buffer| {
-        if buffer.len() < 8 {
-            return Err(IoError::UnexpectedEnd.into())
-        }
-
         let mut reader = Cursor::new(buffer);
 
-        let checksum_at = buffer.len() - 4;
+        let packet = Packet::read_from(&mut reader)?;
+        let padding = (12 - (reader.position() % 8)) % 8;
+        let checksum_at = reader.position() + padding;
         let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
         reader.set_position(checksum_at);
         if reader.read_u32()? != checksum {
             return Err(Error::CorruptedPacket)
         }
-        reader.set_position(0);
-
-        Ok(Packet::read_from(&mut reader)?)
+        Ok(packet)
     })
 }
 

artiq/firmware/libboard_artiq/lib.rs

@@ -1,4 +1,4 @@
-#![feature(lang_items, never_type)]
+#![feature(asm, lang_items, never_type)]
 #![no_std]
 
 extern crate failure;
@@ -24,6 +24,8 @@ pub mod rpc_queue;
 
 #[cfg(has_si5324)]
 pub mod si5324;
+#[cfg(has_si549)]
+pub mod si549;
 
 #[cfg(has_grabber)]
 pub mod grabber;

artiq/firmware/libboard_artiq/mailbox.rs

@@ -6,7 +6,11 @@ static mut LAST: usize = 0;
 
 pub unsafe fn send(data: usize) {
     LAST = data;
-    write_volatile(MAILBOX, data)
+    // after Rust toolchain update to LLVM12, this empty asm! block is required
+    // to ensure that the compiler doesn't take any shortcuts
+    // otherwise, the comm CPU will read garbage data and crash
+    asm!("", options(preserves_flags, readonly, nostack));
+    write_volatile(MAILBOX, data);
 }
 
 pub fn acknowledged() -> bool {

artiq/firmware/libboard_artiq/si549.rs

@@ -0,0 +1,862 @@
+use board_misoc::{clock, csr};
+use log::info;
+
+const ADDRESS: u8 = 0x67;
+
+const ADPLL_MAX: i32 = (950.0 / 0.0001164) as i32;
+
+pub struct DividerConfig {
+    pub hsdiv: u16,
+    pub lsdiv: u8,
+    pub fbdiv: u64,
+}
+
+pub struct FrequencySetting {
+    pub main: DividerConfig,
+    pub helper: DividerConfig,
+}
+
+mod i2c {
+    use super::*;
+
+    #[derive(Clone, Copy)]
+    pub enum DCXO {
+        Main,
+        Helper,
+    }
+
+    fn half_period() {
+        clock::spin_us(1)
+    }
+
+    fn sda_i(dcxo: DCXO) -> bool {
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_in_read() == 1 },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_in_read() == 1 },
+        }
+    }
+
+    fn sda_oe(dcxo: DCXO, oe: bool) {
+        let val = if oe { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_oe_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_oe_write(val) },
+        };
+    }
+
+    fn sda_o(dcxo: DCXO, o: bool) {
+        let val = if o { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_out_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_out_write(val) },
+        };
+    }
+
+    fn scl_oe(dcxo: DCXO, oe: bool) {
+        let val = if oe { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_oe_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_oe_write(val) },
+        };
+    }
+
+    fn scl_o(dcxo: DCXO, o: bool) {
+        let val = if o { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_out_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_out_write(val) },
+        };
+    }
+
+    pub fn init(dcxo: DCXO) -> Result<(), &'static str> {
+        // Set SCL as output, and high level
+        scl_o(dcxo, true);
+        scl_oe(dcxo, true);
+        // Prepare a zero level on SDA so that sda_oe pulls it down
+        sda_o(dcxo, false);
+        // Release SDA
+        sda_oe(dcxo, false);
+
+        // Check the I2C bus is ready
+        half_period();
+        half_period();
+        if !sda_i(dcxo) {
+            // Try toggling SCL a few times
+            for _bit in 0..8 {
+                scl_o(dcxo, false);
+                half_period();
+                scl_o(dcxo, true);
+                half_period();
+            }
+        }
+
+        if !sda_i(dcxo) {
+            return Err("SDA is stuck low and doesn't get unstuck");
+        }
+        Ok(())
+    }
+
+    pub fn start(dcxo: DCXO) {
+        // Set SCL high then SDA low
+        scl_o(dcxo, true);
+        half_period();
+        sda_oe(dcxo, true);
+        half_period();
+    }
+
+    pub fn stop(dcxo: DCXO) {
+        // First, make sure SCL is low, so that the target releases the SDA line
+        scl_o(dcxo, false);
+        half_period();
+        // Set SCL high then SDA high
+        sda_oe(dcxo, true);
+        scl_o(dcxo, true);
+        half_period();
+        sda_oe(dcxo, false);
+        half_period();
+    }
+
+    pub fn write(dcxo: DCXO, data: u8) -> bool {
+        // MSB first
+        for bit in (0..8).rev() {
+            // Set SCL low and set our bit on SDA
+            scl_o(dcxo, false);
+            sda_oe(dcxo, data & (1 << bit) == 0);
+            half_period();
+            // Set SCL high ; data is shifted on the rising edge of SCL
+            scl_o(dcxo, true);
+            half_period();
+        }
+        // Check ack
+        // Set SCL low, then release SDA so that the I2C target can respond
+        scl_o(dcxo, false);
+        half_period();
+        sda_oe(dcxo, false);
+        // Set SCL high and check for ack
+        scl_o(dcxo, true);
+        half_period();
+        // returns true if acked (I2C target pulled SDA low)
+        !sda_i(dcxo)
+    }
+
+    pub fn read(dcxo: DCXO, ack: bool) -> u8 {
+        // Set SCL low first, otherwise setting SDA as input may cause a transition
+        // on SDA with SCL high which will be interpreted as START/STOP condition.
+        scl_o(dcxo, false);
+        half_period(); // make sure SCL has settled low
+        sda_oe(dcxo, false);
+
+        let mut data: u8 = 0;
+
+        // MSB first
+        for bit in (0..8).rev() {
+            scl_o(dcxo, false);
+            half_period();
+            // Set SCL high and shift data
+            scl_o(dcxo, true);
+            half_period();
+            if sda_i(dcxo) {
+                data |= 1 << bit
+            }
+        }
+        // Send ack
+        // Set SCL low and pull SDA low when acking
+        scl_o(dcxo, false);
+        if ack {
+            sda_oe(dcxo, true)
+        }
+        half_period();
+        // then set SCL high
+        scl_o(dcxo, true);
+        half_period();
+
+        data
+    }
+}
+
+fn write(dcxo: i2c::DCXO, reg: u8, val: u8) -> Result<(), &'static str> {
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, ADDRESS << 1) {
+        return Err("Si549 failed to ack write address");
+    }
+    if !i2c::write(dcxo, reg) {
+        return Err("Si549 failed to ack register");
+    }
+    if !i2c::write(dcxo, val) {
+        return Err("Si549 failed to ack value");
+    }
+    i2c::stop(dcxo);
+    Ok(())
+}
+
+fn read(dcxo: i2c::DCXO, reg: u8) -> Result<u8, &'static str> {
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, ADDRESS << 1) {
+        return Err("Si549 failed to ack write address");
+    }
+    if !i2c::write(dcxo, reg) {
+        return Err("Si549 failed to ack register");
+    }
+    i2c::stop(dcxo);
+
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, (ADDRESS << 1) | 1) {
+        return Err("Si549 failed to ack read address");
+    }
+    let val = i2c::read(dcxo, false);
+    i2c::stop(dcxo);
+    Ok(val)
+}
+
+fn setup(dcxo: i2c::DCXO, config: &DividerConfig) -> Result<(), &'static str> {
+    i2c::init(dcxo)?;
+
+    write(dcxo, 255, 0x00)?; // PAGE
+    write(dcxo, 69, 0x00)?; // Disable FCAL override.
+    write(dcxo, 17, 0x00)?; // Synchronously disable output
+
+    // The Si549 has no ID register, so we check that it responds correctly
+    // by writing values to a RAM-like register and reading them back.
+    for test_value in 0..255 {
+        write(dcxo, 23, test_value)?;
+        let readback = read(dcxo, 23)?;
+        if readback != test_value {
+            return Err("Si549 detection failed");
+        }
+    }
+
+    write(dcxo, 23, config.hsdiv as u8)?;
+    write(dcxo, 24, (config.hsdiv >> 8) as u8 | (config.lsdiv << 4))?;
+    write(dcxo, 26, config.fbdiv as u8)?;
+    write(dcxo, 27, (config.fbdiv >> 8) as u8)?;
+    write(dcxo, 28, (config.fbdiv >> 16) as u8)?;
+    write(dcxo, 29, (config.fbdiv >> 24) as u8)?;
+    write(dcxo, 30, (config.fbdiv >> 32) as u8)?;
+    write(dcxo, 31, (config.fbdiv >> 40) as u8)?;
+
+    write(dcxo, 7, 0x08)?; // Start FCAL
+    clock::spin_us(30_000); // Internal FCAL VCO calibration
+    write(dcxo, 17, 0x01)?; // Synchronously enable output
+
+    Ok(())
+}
+
+pub fn main_setup(settings: &FrequencySetting) -> Result<(), &'static str> {
+    unsafe {
+        csr::wrpll::main_dcxo_bitbang_enable_write(1);
+        csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
+    }
+
+    setup(i2c::DCXO::Main, &settings.main)?;
+
+    // Si549 maximum settling time for large frequency change.
+    clock::spin_us(40_000);
+
+    unsafe {
+        csr::wrpll::main_dcxo_bitbang_enable_write(0);
+    }
+
+    info!("Main Si549 started");
+    Ok(())
+}
+
+pub fn helper_setup(settings: &FrequencySetting) -> Result<(), &'static str> {
+    unsafe {
+        csr::wrpll::helper_reset_write(1);
+        csr::wrpll::helper_dcxo_bitbang_enable_write(1);
+        csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
+    }
+
+    setup(i2c::DCXO::Helper, &settings.helper)?;
+
+    // Si549 maximum settling time for large frequency change.
+    clock::spin_us(40_000);
+
+    unsafe {
+        csr::wrpll::helper_reset_write(0);
+        csr::wrpll::helper_dcxo_bitbang_enable_write(0);
+    }
+    info!("Helper Si549 started");
+    Ok(())
+}
+
+fn set_adpll(dcxo: i2c::DCXO, adpll: i32) -> Result<(), &'static str> {
+    if adpll.abs() > ADPLL_MAX {
+        return Err("adpll is too large");
+    }
+
+    match dcxo {
+        i2c::DCXO::Main => unsafe {
+            if csr::wrpll::main_dcxo_bitbang_enable_read() == 1 {
+                return Err("Main si549 bitbang mode is active when using gateware i2c");
+            }
+
+            while csr::wrpll::main_dcxo_adpll_busy_read() == 1 {}
+            if csr::wrpll::main_dcxo_nack_read() == 1 {
+                return Err("Main si549 failed to ack adpll write");
+            }
+
+            csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
+            csr::wrpll::main_dcxo_adpll_write(adpll as u32);
+
+            csr::wrpll::main_dcxo_adpll_stb_write(1);
+        },
+        i2c::DCXO::Helper => unsafe {
+            if csr::wrpll::helper_dcxo_bitbang_enable_read() == 1 {
+                return Err("Helper si549 bitbang mode is active when using gateware i2c");
+            }
+
+            while csr::wrpll::helper_dcxo_adpll_busy_read() == 1 {}
+            if csr::wrpll::helper_dcxo_nack_read() == 1 {
+                return Err("Helper si549 failed to ack adpll write");
+            }
+
+            csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
+            csr::wrpll::helper_dcxo_adpll_write(adpll as u32);
+
+            csr::wrpll::helper_dcxo_adpll_stb_write(1);
+        },
+    };
+
+    Ok(())
+}
+
+#[cfg(has_wrpll)]
+pub mod wrpll {
+
+    use super::*;
+
+    const BEATING_PERIOD: i32 = 0x8000;
+    const BEATING_HALFPERIOD: i32 = 0x4000;
+    const COUNTER_WIDTH: u32 = 24;
+    const DIV_WIDTH: u32 = 2;
+
+    // y[n] = b0*x[n] + b1*x[n-1] + b2*x[n-2] - a1*y[n-1] - a2*y[n-2]
+    struct FilterParameters {
+        pub b0: i64,
+        pub b1: i64,
+        pub b2: i64,
+        pub a1: i64,
+        pub a2: i64,
+    }
+
+    #[cfg(rtio_frequency = "100.0")]
+    const LPF: FilterParameters = FilterParameters {
+        b0: 10905723400,   // 0.03967479060647884 * 1 << 38
+        b1: 21811446800,   // 0.07934958121295768 * 1 << 38
+        b2: 10905723400,   // 0.03967479060647884 * 1 << 38
+        a1: -381134538612, // -1.3865593741228928 * 1 << 38
+        a2: 149879525269,  // 0.5452585365488082  * 1 << 38
+    };
+
+    #[cfg(rtio_frequency = "125.0")]
+    const LPF: FilterParameters = FilterParameters {
+        b0: 19816511911,   // 0.07209205036273991  * 1 << 38
+        b1: 39633023822,   // 0.14418410072547982  * 1 << 38
+        b2: 19816511911,   // 0.07209205036273991  * 1 << 38
+        a1: -168062510414, // -0.6114078511562919  * 1 << 38
+        a2: -27549348884,  // -0.10022394739274834 * 1 << 38
+    };
+
+    static mut H_ADPLL1: i32 = 0;
+    static mut H_ADPLL2: i32 = 0;
+    static mut PERIOD_ERR1: i32 = 0;
+    static mut PERIOD_ERR2: i32 = 0;
+
+    static mut M_ADPLL1: i32 = 0;
+    static mut M_ADPLL2: i32 = 0;
+    static mut PHASE_ERR1: i32 = 0;
+    static mut PHASE_ERR2: i32 = 0;
+
+    static mut BASE_ADPLL: i32 = 0;
+
+    #[derive(Clone, Copy)]
+    pub enum ISR {
+        RefTag,
+        MainTag,
+    }
+
+    mod tag_collector {
+        use super::*;
+
+        #[cfg(wrpll_ref_clk = "GT_CDR")]
+        static mut TAG_OFFSET: u32 = 23890;
+        #[cfg(wrpll_ref_clk = "SMA_CLKIN")]
+        static mut TAG_OFFSET: u32 = 0;
+        static mut REF_TAG: u32 = 0;
+        static mut REF_TAG_READY: bool = false;
+        static mut MAIN_TAG: u32 = 0;
+        static mut MAIN_TAG_READY: bool = false;
+
+        pub fn reset() {
+            clear_phase_diff_ready();
+            unsafe {
+                REF_TAG = 0;
+                MAIN_TAG = 0;
+            }
+        }
+
+        pub fn clear_phase_diff_ready() {
+            unsafe {
+                REF_TAG_READY = false;
+                MAIN_TAG_READY = false;
+            }
+        }
+
+        pub fn collect_tags(interrupt: ISR) {
+            match interrupt {
+                ISR::RefTag => unsafe {
+                    REF_TAG = csr::wrpll::ref_tag_read();
+                    REF_TAG_READY = true;
+                },
+                ISR::MainTag => unsafe {
+                    MAIN_TAG = csr::wrpll::main_tag_read();
+                    MAIN_TAG_READY = true;
+                },
+            }
+        }
+
+        pub fn phase_diff_ready() -> bool {
+            unsafe { REF_TAG_READY && MAIN_TAG_READY }
+        }
+
+        #[cfg(feature = "calibrate_wrpll_skew")]
+        pub fn set_tag_offset(offset: u32) {
+            unsafe {
+                TAG_OFFSET = offset;
+            }
+        }
+
+        #[cfg(feature = "calibrate_wrpll_skew")]
+        pub fn get_tag_offset() -> u32 {
+            unsafe { TAG_OFFSET }
+        }
+
+        pub fn get_period_error() -> i32 {
+            // n * BEATING_PERIOD - REF_TAG(n) mod BEATING_PERIOD
+            let mut period_error = unsafe {
+                REF_TAG
+                    .overflowing_neg()
+                    .0
+                    .rem_euclid(BEATING_PERIOD as u32) as i32
+            };
+            // mapping tags from [0, 2π] -> [-π, π]
+            if period_error > BEATING_HALFPERIOD {
+                period_error -= BEATING_PERIOD
+            }
+            period_error
+        }
+
+        pub fn get_phase_error() -> i32 {
+            // MAIN_TAG(n) - REF_TAG(n) - TAG_OFFSET mod BEATING_PERIOD
+            let mut phase_error = unsafe {
+                MAIN_TAG
+                    .overflowing_sub(REF_TAG + TAG_OFFSET)
+                    .0
+                    .rem_euclid(BEATING_PERIOD as u32) as i32
+            };
+
+            // mapping tags from [0, 2π] -> [-π, π]
+            if phase_error > BEATING_HALFPERIOD {
+                phase_error -= BEATING_PERIOD
+            }
+            phase_error
+        }
+    }
+
+    fn set_isr(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll::ref_tag_ev_enable_write(val);
+            csr::wrpll::main_tag_ev_enable_write(val);
+        }
+    }
+
+    fn set_base_adpll() -> Result<(), &'static str> {
+        let count2adpll = |error: i32| {
+            ((error as f64 * 1e6) / (0.0001164 * (1 << (COUNTER_WIDTH - DIV_WIDTH)) as f64)) as i32
+        };
+
+        let (ref_count, main_count) = get_freq_counts();
+        unsafe {
+            BASE_ADPLL = count2adpll(ref_count as i32 - main_count as i32);
+            set_adpll(i2c::DCXO::Main, BASE_ADPLL)?;
+            set_adpll(i2c::DCXO::Helper, BASE_ADPLL)?;
+        }
+        Ok(())
+    }
+
+    fn get_freq_counts() -> (u32, u32) {
+        unsafe {
+            csr::wrpll::frequency_counter_update_write(1);
+            while csr::wrpll::frequency_counter_busy_read() == 1 {}
+            #[cfg(wrpll_ref_clk = "GT_CDR")]
+            let ref_count = csr::wrpll::frequency_counter_counter_rtio_rx0_read();
+            #[cfg(wrpll_ref_clk = "SMA_CLKIN")]
+            let ref_count = csr::wrpll::frequency_counter_counter_ref_read();
+            let main_count = csr::wrpll::frequency_counter_counter_sys_read();
+
+            (ref_count, main_count)
+        }
+    }
+
+    fn reset_plls() -> Result<(), &'static str> {
+        unsafe {
+            H_ADPLL1 = 0;
+            H_ADPLL2 = 0;
+            PERIOD_ERR1 = 0;
+            PERIOD_ERR2 = 0;
+            M_ADPLL1 = 0;
+            M_ADPLL2 = 0;
+            PHASE_ERR1 = 0;
+            PHASE_ERR2 = 0;
+        }
+        set_adpll(i2c::DCXO::Main, 0)?;
+        set_adpll(i2c::DCXO::Helper, 0)?;
+        // wait for adpll to transfer and DCXO to settle
+        clock::spin_us(200);
+        Ok(())
+    }
+
+    fn clear_pending(interrupt: ISR) {
+        match interrupt {
+            ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_write(1) },
+            ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_write(1) },
+        };
+    }
+
+    fn is_pending(interrupt: ISR) -> bool {
+        match interrupt {
+            ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_read() == 1 },
+            ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_read() == 1 },
+        }
+    }
+
+    pub fn interrupt_handler() {
+        if is_pending(ISR::RefTag) {
+            tag_collector::collect_tags(ISR::RefTag);
+            clear_pending(ISR::RefTag);
+            helper_pll().expect("failed to run helper DCXO PLL");
+        }
+
+        if is_pending(ISR::MainTag) {
+            tag_collector::collect_tags(ISR::MainTag);
+            clear_pending(ISR::MainTag);
+        }
+
+        if tag_collector::phase_diff_ready() {
+            main_pll().expect("failed to run main DCXO PLL");
+            tag_collector::clear_phase_diff_ready();
+        }
+    }
+
+    fn helper_pll() -> Result<(), &'static str> {
+        let period_err = tag_collector::get_period_error();
+        unsafe {
+            let adpll = (((LPF.b0 * period_err as i64)
+                + (LPF.b1 * PERIOD_ERR1 as i64)
+                + (LPF.b2 * PERIOD_ERR2 as i64)
+                - (LPF.a1 * H_ADPLL1 as i64)
+                - (LPF.a2 * H_ADPLL2 as i64))
+                >> 38) as i32;
+            set_adpll(i2c::DCXO::Helper, BASE_ADPLL + adpll)?;
+            H_ADPLL2 = H_ADPLL1;
+            PERIOD_ERR2 = PERIOD_ERR1;
+            H_ADPLL1 = adpll;
+            PERIOD_ERR1 = period_err;
+        };
+        Ok(())
+    }
+
+    fn main_pll() -> Result<(), &'static str> {
+        let phase_err = tag_collector::get_phase_error();
+        unsafe {
+            let adpll = (((LPF.b0 * phase_err as i64)
+                + (LPF.b1 * PHASE_ERR1 as i64)
+                + (LPF.b2 * PHASE_ERR2 as i64)
+                - (LPF.a1 * M_ADPLL1 as i64)
+                - (LPF.a2 * M_ADPLL2 as i64))
+                >> 38) as i32;
+            set_adpll(i2c::DCXO::Main, BASE_ADPLL + adpll)?;
+            M_ADPLL2 = M_ADPLL1;
+            PHASE_ERR2 = PHASE_ERR1;
+            M_ADPLL1 = adpll;
+            PHASE_ERR1 = phase_err;
+        };
+        Ok(())
+    }
+
+    #[cfg(wrpll_ref_clk = "GT_CDR")]
+    fn test_skew() -> Result<(), &'static str> {
+        // wait for PLL to stabilize
+        clock::spin_us(20_000);
+
+        info!("testing the skew of SYS CLK...");
+        if has_timing_error() {
+            return Err("the skew cannot satisfy setup/hold time constraint of RX synchronizer");
+        }
+        info!("the skew of SYS CLK met the timing constraint");
+        Ok(())
+    }
+
+    #[cfg(wrpll_ref_clk = "GT_CDR")]
+    fn has_timing_error() -> bool {
+        unsafe {
+            csr::wrpll_skewtester::error_write(1);
+        }
+        clock::spin_us(5_000);
+        unsafe { csr::wrpll_skewtester::error_read() == 1 }
+    }
+
+    #[cfg(feature = "calibrate_wrpll_skew")]
+    fn find_edge(target: bool) -> Result<u32, &'static str> {
+        const STEP: u32 = 8;
+        const STABLE_THRESHOLD: u32 = 10;
+
+        enum FSM {
+            Init,
+            WaitEdge,
+            GotEdge,
+        }
+
+        let mut state: FSM = FSM::Init;
+        let mut offset: u32 = tag_collector::get_tag_offset();
+        let mut median_edge: u32 = 0;
+        let mut stable_counter: u32 = 0;
+
+        for _ in 0..(BEATING_PERIOD as u32 / STEP) as usize {
+            tag_collector::set_tag_offset(offset);
+            offset += STEP;
+            // wait for PLL to stabilize
+            clock::spin_us(20_000);
+
+            let error = has_timing_error();
+            // A median edge deglitcher
+            match state {
+                FSM::Init => {
+                    if error != target {
+                        stable_counter += 1;
+                    } else {
+                        stable_counter = 0;
+                    }
+
+                    if stable_counter >= STABLE_THRESHOLD {
+                        state = FSM::WaitEdge;
+                        stable_counter = 0;
+                    }
+                }
+                FSM::WaitEdge => {
+                    if error == target {
+                        state = FSM::GotEdge;
+                        median_edge = offset;
+                    }
+                }
+                FSM::GotEdge => {
+                    if error != target {
+                        median_edge += STEP;
+                        stable_counter = 0;
+                    } else {
+                        stable_counter += 1;
+                    }
+
+                    if stable_counter >= STABLE_THRESHOLD {
+                        return Ok(median_edge);
+                    }
+                }
+            }
+        }
+        return Err("failed to find timing error edge");
+    }
+
+    #[cfg(feature = "calibrate_wrpll_skew")]
+    fn calibrate_skew() -> Result<(), &'static str> {
+        info!("calibrating skew to meet timing constraint...");
+
+        // clear calibrated value
+        tag_collector::set_tag_offset(0);
+        let rising = find_edge(true)? as i32;
+        let falling = find_edge(false)? as i32;
+
+        let width = BEATING_PERIOD - (falling - rising);
+        let result = falling + width / 2;
+        tag_collector::set_tag_offset(result as u32);
+
+        info!(
+            "calibration successful, error zone: {} -> {}, width: {} ({}deg), middle of working region: {}",
+            rising,
+            falling,
+            width,
+            360 * width / BEATING_PERIOD,
+            result,
+        );
+
+        Ok(())
+    }
+
+    pub fn select_recovered_clock(rc: bool) {
+        set_isr(false);
+
+        if rc {
+            tag_collector::reset();
+            reset_plls().expect("failed to reset main and helper PLL");
+
+            // get within capture range
+            set_base_adpll().expect("failed to set base adpll");
+
+            // clear gateware pending flag
+            clear_pending(ISR::RefTag);
+            clear_pending(ISR::MainTag);
+
+            // use nFIQ to avoid IRQ being disabled by mutex lock and mess up PLL
+            set_isr(true);
+            info!("WRPLL interrupt enabled");
+
+            #[cfg(feature = "calibrate_wrpll_skew")]
+            calibrate_skew().expect("failed to set the correct skew");
+
+            #[cfg(wrpll_ref_clk = "GT_CDR")]
+            test_skew().expect("skew test failed");
+        }
+    }
+}
+
+#[cfg(has_wrpll_refclk)]
+pub mod wrpll_refclk {
+    use super::*;
+
+    pub struct MmcmSetting {
+        pub clkout0_reg1: u16,  //0x08
+        pub clkout0_reg2: u16,  //0x09
+        pub clkfbout_reg1: u16, //0x14
+        pub clkfbout_reg2: u16, //0x15
+        pub div_reg: u16,       //0x16
+        pub lock_reg1: u16,     //0x18
+        pub lock_reg2: u16,     //0x19
+        pub lock_reg3: u16,     //0x1A
+        pub power_reg: u16,     //0x28
+        pub filt_reg1: u16,     //0x4E
+        pub filt_reg2: u16,     //0x4F
+    }
+
+    fn one_clock_cycle() {
+        unsafe {
+            csr::wrpll_refclk::mmcm_dclk_write(1);
+            csr::wrpll_refclk::mmcm_dclk_write(0);
+        }
+    }
+
+    fn set_addr(address: u8) {
+        unsafe {
+            csr::wrpll_refclk::mmcm_daddr_write(address);
+        }
+    }
+
+    fn set_data(value: u16) {
+        unsafe {
+            csr::wrpll_refclk::mmcm_din_write(value);
+        }
+    }
+
+    fn set_enable(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_den_write(val);
+        }
+    }
+
+    fn set_write_enable(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_dwen_write(val);
+        }
+    }
+
+    fn get_data() -> u16 {
+        unsafe { csr::wrpll_refclk::mmcm_dout_read() }
+    }
+
+    fn drp_ready() -> bool {
+        unsafe { csr::wrpll_refclk::mmcm_dready_read() == 1 }
+    }
+
+    #[allow(dead_code)]
+    fn read(address: u8) -> u16 {
+        set_addr(address);
+        set_enable(true);
+        // Set DADDR on the mmcm and assert DEN for one clock cycle
+        one_clock_cycle();
+
+        set_enable(false);
+        while !drp_ready() {
+            // keep the clock signal until data is ready
+            one_clock_cycle();
+        }
+        get_data()
+    }
+
+    fn write(address: u8, value: u16) {
+        set_addr(address);
+        set_data(value);
+        set_write_enable(true);
+        set_enable(true);
+        // Set DADDR, DI on the mmcm and assert DWE, DEN for one clock cycle
+        one_clock_cycle();
+
+        set_write_enable(false);
+        set_enable(false);
+        while !drp_ready() {
+            // keep the clock signal until write is finished
+            one_clock_cycle();
+        }
+    }
+
+    fn reset(rst: bool) {
+        let val = if rst { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_reset_write(val)
+        }
+    }
+
+    pub fn setup(settings: MmcmSetting, mmcm_bypass: bool) -> Result<(), &'static str> {
+        unsafe {
+            csr::wrpll_refclk::refclk_reset_write(1);
+        }
+
+        if mmcm_bypass {
+            info!("Bypassing mmcm");
+            unsafe {
+                csr::wrpll_refclk::mmcm_bypass_write(1);
+            }
+        } else {
+            // Based on "DRP State Machine" from XAPP888
+            // hold reset HIGH during mmcm config
+            reset(true);
+            write(0x08, settings.clkout0_reg1);
+            write(0x09, settings.clkout0_reg2);
+            write(0x14, settings.clkfbout_reg1);
+            write(0x15, settings.clkfbout_reg2);
+            write(0x16, settings.div_reg);
+            write(0x18, settings.lock_reg1);
+            write(0x19, settings.lock_reg2);
+            write(0x1A, settings.lock_reg3);
+            write(0x28, settings.power_reg);
+            write(0x4E, settings.filt_reg1);
+            write(0x4F, settings.filt_reg2);
+            reset(false);
+
+            // wait for the mmcm to lock
+            clock::spin_us(100);
+
+            let locked = unsafe { csr::wrpll_refclk::mmcm_locked_read() == 1 };
+            if !locked {
+                return Err("mmcm failed to generate 125MHz ref clock from SMA CLKIN");
+            }
+        }
+
+        unsafe {
+            csr::wrpll_refclk::refclk_reset_write(0);
+        }
+
+        Ok(())
+    }
+}

artiq/firmware/libboard_misoc/io_expander.rs

@@ -27,6 +27,26 @@ impl IoExpander {
         const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 6), (1, 1, 6)];
         const VIRTUAL_LED_MAPPING1: [(u8, u8, u8); 2] = [(2, 0, 6), (3, 1, 6)];
 
+        #[cfg(has_si549)]
+        const IODIR_CLK_SEL: u8 = 0x80; // out
+        #[cfg(has_si5324)]
+        const IODIR_CLK_SEL: u8 = 0x00; // in
+        
+        #[cfg(has_si549)]
+        const CLK_SEL_OUT: u8 = 1 << 7;
+        #[cfg(has_si5324)]
+        const CLK_SEL_OUT: u8 = 0;
+        
+        const IODIR0 : [u8; 2] = [
+            0xFF,
+            0xFF & !IODIR_CLK_SEL
+        ];
+        
+        const OUT_TAR0 : [u8; 2] = [
+            0,
+            CLK_SEL_OUT
+        ];        
+
         // Both expanders on SHARED I2C bus
         let mut io_expander = match index {
             0 => IoExpander {
@@ -34,9 +54,9 @@ impl IoExpander {
                 port: 11,
                 address: 0x40,
                 virtual_led_mapping: &VIRTUAL_LED_MAPPING0,
-                iodir: [0xff; 2],
+                iodir: IODIR0,
                 out_current: [0; 2],
-                out_target: [0; 2],
+                out_target: OUT_TAR0,
                 registers: Registers {
                     iodira: 0x00,
                     iodirb: 0x01,
@@ -153,10 +173,10 @@ impl IoExpander {
         }
         self.update_iodir()?;
 
-        self.out_current[0] = 0x00;
-        self.write(self.registers.gpioa, 0x00)?;
-        self.out_current[1] = 0x00;
-        self.write(self.registers.gpiob, 0x00)?;
+        self.write(self.registers.gpioa, self.out_target[0])?;
+        self.out_current[0] = self.out_target[0];
+        self.write(self.registers.gpiob, self.out_target[1])?;
+        self.out_current[1] = self.out_target[1];
         Ok(())
     }
 

artiq/firmware/libboard_misoc/lib.rs

@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(llvm_asm)]
+#![feature(asm)]
 
 extern crate byteorder;
 #[cfg(feature = "log")]

artiq/firmware/libboard_misoc/riscv32/boot.rs

@@ -2,29 +2,26 @@ use super::{cache, pmp};
 use riscv::register::*;
 
 pub unsafe fn reset() -> ! {
-    llvm_asm!(r#"
-        j          _reset_handler
-         nop
-    "# : : : : "volatile");
-    loop {}
+    asm!("j          _reset_handler",
+         "nop",
+         options(nomem, nostack, noreturn)
+    );
 }
 
 pub unsafe fn jump(addr: usize) -> ! {
     cache::flush_cpu_icache();
-    llvm_asm!(r#"
-        jalr       x0, 0($0)
-         nop
-    "# : : "r"(addr) : : "volatile");
-    loop {}
+    asm!("jalr       x0, 0({0})",
+         "nop",
+         in(reg) addr,
+         options(nomem, nostack, noreturn)
+    );
 }
 
 pub unsafe fn start_user(addr: usize) -> ! {
     pmp::enable_user_memory();
     mstatus::set_mpp(mstatus::MPP::User);
     mepc::write(addr);
-    llvm_asm!(
-        "mret"
-        : : : : "volatile"
+    asm!("mret",
+        options(nomem, nostack, noreturn)
     );
-    unreachable!()
 }

artiq/firmware/libboard_misoc/riscv32/cache.rs

@@ -7,20 +7,24 @@ use mem;
 
 pub fn flush_cpu_icache() {
     unsafe {
-        llvm_asm!(r#"
-            fence.i
-            nop
-            nop
-            nop
-            nop
-            nop
-        "# : : : : "volatile");
+        asm!(
+            "fence.i",
+            "nop",
+            "nop",
+            "nop",
+            "nop",
+            "nop",
+            options(preserves_flags, nostack)
+        );
     }
 }
 
 pub fn flush_cpu_dcache() {
     unsafe {
-        llvm_asm!(".word(0x500F)" : : : : "volatile");
+        asm!(
+            ".word(0x500F)",
+            options(preserves_flags)
+        );
     }
 }
 

artiq/firmware/libboard_misoc/riscv32/irq.rs

@@ -0,0 +1,49 @@
+use riscv::register::{mie, mstatus};
+
+fn vmim_write(val: usize) {
+    unsafe {
+        asm!("csrw {csr}, {rs}", rs = in(reg) val, csr = const 0xBC0);
+    }
+}
+
+fn vmim_read() -> usize {
+    let r: usize;
+    unsafe {
+        asm!("csrr {rd}, {csr}", rd = out(reg) r, csr = const 0xBC0);
+    }
+    r
+}
+
+fn vmip_read() -> usize {
+    let r: usize;
+    unsafe {
+        asm!("csrr {rd}, {csr}", rd = out(reg) r, csr = const 0xFC0);
+    }
+    r
+}
+
+pub fn enable_interrupts() {
+    unsafe {
+        mstatus::set_mie();
+        mie::set_mext();
+    }
+}
+
+pub fn disable_interrupts() {
+    unsafe {
+        mstatus::clear_mie();
+        mie::clear_mext();
+    }
+}
+
+pub fn enable(id: u32) {
+    vmim_write(vmim_read() | (1 << id));
+}
+
+pub fn disable(id: u32) {
+    vmim_write(vmim_read() & !(1 << id));
+}
+
+pub fn is_pending(id: u32) -> bool {
+    (vmip_read() >> id) & 1 == 1
+}

artiq/firmware/libboard_misoc/riscv32/mod.rs

@@ -1,3 +1,4 @@
 pub mod cache;
 pub mod boot;
+pub mod irq;
 pub mod pmp;

artiq/firmware/libeh/Cargo.toml

@@ -11,4 +11,3 @@ path = "lib.rs"
 cslice = { version = "0.3" }
 libc = { path = "../libc" }
 unwind = { path = "../libunwind" }
-compiler_builtins = "=0.1.39"    # A dependency of alloc, libeh doesn't need it

artiq/firmware/libeh/lib.rs

@@ -1,4 +1,4 @@
-#![feature(lang_items, panic_unwind, libc, unwind_attributes, int_bits_const)]
+#![feature(lang_items, panic_unwind, libc)]
 #![no_std]
 
 extern crate cslice;

artiq/firmware/libproto_artiq/drtioaux_proto.rs

@@ -16,9 +16,9 @@ impl<T> From<IoError<T>> for Error<T> {
 
 // maximum size of arbitrary payloads
 // used by satellite -> master analyzer, subkernel exceptions
-pub const SAT_PAYLOAD_MAX_SIZE: usize  = /*max size*/512 - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
+pub const SAT_PAYLOAD_MAX_SIZE: usize  = /*max size*/1024 - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
 // used by DDMA, subkernel program data (need to provide extra ID and destination)
-pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*destination*/1 - /*ID*/4;
+pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*source*/1 - /*destination*/1 - /*ID*/4;
 
 #[derive(PartialEq, Clone, Copy, Debug)]
 #[repr(u8)]
@@ -106,22 +106,26 @@ pub enum Packet {
     AnalyzerDataRequest { destination: u8 },
     AnalyzerData { last: bool, length: u16, data: [u8; SAT_PAYLOAD_MAX_SIZE]},
 
-    DmaAddTraceRequest { destination: u8, id: u32, status: PayloadStatus, length: u16, trace: [u8; MASTER_PAYLOAD_MAX_SIZE] },
-    DmaAddTraceReply { succeeded: bool },
-    DmaRemoveTraceRequest { destination: u8, id: u32 },
-    DmaRemoveTraceReply { succeeded: bool },
-    DmaPlaybackRequest { destination: u8, id: u32, timestamp: u64 },
-    DmaPlaybackReply { succeeded: bool },
-    DmaPlaybackStatus { destination: u8, id: u32, error: u8, channel: u32, timestamp: u64 },
+    DmaAddTraceRequest { 
+        source: u8, destination: u8, 
+        id: u32, status: PayloadStatus, 
+        length: u16, trace: [u8; MASTER_PAYLOAD_MAX_SIZE] 
+    },
+    DmaAddTraceReply { source: u8, destination: u8, id: u32, succeeded: bool },
+    DmaRemoveTraceRequest { source: u8, destination: u8, id: u32 },
+    DmaRemoveTraceReply { destination: u8, succeeded: bool },
+    DmaPlaybackRequest { source: u8, destination: u8, id: u32, timestamp: u64 },
+    DmaPlaybackReply { destination: u8, succeeded: bool },
+    DmaPlaybackStatus { source: u8, destination: u8, id: u32, error: u8, channel: u32, timestamp: u64 },
 
     SubkernelAddDataRequest { destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
     SubkernelAddDataReply { succeeded: bool },
-    SubkernelLoadRunRequest { destination: u8, id: u32, run: bool },
-    SubkernelLoadRunReply { succeeded: bool },
-    SubkernelFinished { id: u32, with_exception: bool },
+    SubkernelLoadRunRequest { source: u8, destination: u8, id: u32, run: bool },
+    SubkernelLoadRunReply { destination: u8, succeeded: bool },
+    SubkernelFinished { destination: u8, id: u32, with_exception: bool, exception_src: u8 },
     SubkernelExceptionRequest { destination: u8 },
     SubkernelException { last: bool, length: u16, data: [u8; SAT_PAYLOAD_MAX_SIZE] },
-    SubkernelMessage { destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
+    SubkernelMessage { source: u8, destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
     SubkernelMessageAck { destination: u8 },
 }
 
@@ -278,6 +282,7 @@ impl Packet {
             },
 
             0xb0 => {
+                let source = reader.read_u8()?;
                 let destination = reader.read_u8()?;
                 let id = reader.read_u32()?;
                 let status = reader.read_u8()?;
@@ -285,6 +290,7 @@ impl Packet {
                 let mut trace: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 reader.read_exact(&mut trace[0..length as usize])?;
                 Packet::DmaAddTraceRequest {
+                    source: source,
                     destination: destination,
                     id: id,
                     status: PayloadStatus::from(status),
@@ -293,24 +299,32 @@ impl Packet {
                 }
             },
             0xb1 => Packet::DmaAddTraceReply {
+                source: reader.read_u8()?,
+                destination: reader.read_u8()?,
+                id: reader.read_u32()?,
                 succeeded: reader.read_bool()?
             },
             0xb2 => Packet::DmaRemoveTraceRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?
             },
             0xb3 => Packet::DmaRemoveTraceReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xb4 => Packet::DmaPlaybackRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 timestamp: reader.read_u64()?
             },
             0xb5 => Packet::DmaPlaybackReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xb6 => Packet::DmaPlaybackStatus {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 error: reader.read_u8()?,
@@ -337,16 +351,20 @@ impl Packet {
                 succeeded: reader.read_bool()?
             },
             0xc4 => Packet::SubkernelLoadRunRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 run: reader.read_bool()?
             },
             0xc5 => Packet::SubkernelLoadRunReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xc8 => Packet::SubkernelFinished {
+                destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 with_exception: reader.read_bool()?,
+                exception_src: reader.read_u8()?
             },
             0xc9 => Packet::SubkernelExceptionRequest {
                 destination: reader.read_u8()?
@@ -363,6 +381,7 @@ impl Packet {
                 }
             },
             0xcb => {
+                let source = reader.read_u8()?;
                 let destination = reader.read_u8()?;
                 let id = reader.read_u32()?;
                 let status = reader.read_u8()?;
@@ -370,6 +389,7 @@ impl Packet {
                 let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 reader.read_exact(&mut data[0..length as usize])?;
                 Packet::SubkernelMessage {
+                    source: source,
                     destination: destination,
                     id: id,
                     status: PayloadStatus::from(status),
@@ -561,8 +581,9 @@ impl Packet {
                 writer.write_all(&data[0..length as usize])?;
             },
 
-            Packet::DmaAddTraceRequest { destination, id, status, trace, length } => {
+            Packet::DmaAddTraceRequest { source, destination, id, status, trace, length } => {
                 writer.write_u8(0xb0)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(status as u8)?;
@@ -571,31 +592,39 @@ impl Packet {
                 writer.write_u16(length)?;
                 writer.write_all(&trace[0..length as usize])?;
             },
-            Packet::DmaAddTraceReply { succeeded } => {
+            Packet::DmaAddTraceReply { source, destination, id, succeeded } => {
                 writer.write_u8(0xb1)?;
+                writer.write_u8(source)?;
+                writer.write_u8(destination)?;
+                writer.write_u32(id)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaRemoveTraceRequest { destination, id } => {
+            Packet::DmaRemoveTraceRequest { source, destination, id } => {
                 writer.write_u8(0xb2)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
             },
-            Packet::DmaRemoveTraceReply { succeeded } => {
+            Packet::DmaRemoveTraceReply { destination, succeeded } => {
                 writer.write_u8(0xb3)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaPlaybackRequest { destination, id, timestamp } => {
+            Packet::DmaPlaybackRequest { source, destination, id, timestamp } => {
                 writer.write_u8(0xb4)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u64(timestamp)?;
             },
-            Packet::DmaPlaybackReply { succeeded } => {
+            Packet::DmaPlaybackReply { destination, succeeded } => {
                 writer.write_u8(0xb5)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaPlaybackStatus { destination, id, error, channel, timestamp } => {
+            Packet::DmaPlaybackStatus { source, destination, id, error, channel, timestamp } => {
                 writer.write_u8(0xb6)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(error)?;
@@ -615,20 +644,24 @@ impl Packet {
                 writer.write_u8(0xc1)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::SubkernelLoadRunRequest { destination, id, run } => {
+            Packet::SubkernelLoadRunRequest { source, destination, id, run } => {
                 writer.write_u8(0xc4)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_bool(run)?;
             },
-            Packet::SubkernelLoadRunReply { succeeded } => {
+            Packet::SubkernelLoadRunReply { destination, succeeded } => {
                 writer.write_u8(0xc5)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::SubkernelFinished { id, with_exception } => {
+            Packet::SubkernelFinished { destination, id, with_exception, exception_src } => {
                 writer.write_u8(0xc8)?;
+                writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_bool(with_exception)?;
+                writer.write_u8(exception_src)?;
             },
             Packet::SubkernelExceptionRequest { destination } => {
                 writer.write_u8(0xc9)?;
@@ -640,8 +673,9 @@ impl Packet {
                 writer.write_u16(length)?;
                 writer.write_all(&data[0..length as usize])?;
             },
-            Packet::SubkernelMessage { destination, id, status, data, length } => {
+            Packet::SubkernelMessage { source, destination, id, status, data, length } => {
                 writer.write_u8(0xcb)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(status as u8)?;
@@ -655,4 +689,36 @@ impl Packet {
         }
         Ok(())
     }
+
+    pub fn routable_destination(&self) -> Option<u8> {
+        // only for packets that could be re-routed, not only forwarded
+        match self {
+            Packet::DmaAddTraceRequest      { destination, .. } => Some(*destination),
+            Packet::DmaAddTraceReply        { destination, .. } => Some(*destination),
+            Packet::DmaRemoveTraceRequest   { destination, .. } => Some(*destination),
+            Packet::DmaRemoveTraceReply     { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackRequest      { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackReply        { destination, .. } => Some(*destination),
+            Packet::SubkernelLoadRunRequest { destination, .. } => Some(*destination),
+            Packet::SubkernelLoadRunReply   { destination, .. } => Some(*destination),
+            Packet::SubkernelMessage        { destination, .. } => Some(*destination),
+            Packet::SubkernelMessageAck     { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackStatus       { destination, .. } => Some(*destination),
+            Packet::SubkernelFinished       { destination, .. } => Some(*destination),
+            _ => None
+        }
+    }
+
+    pub fn expects_response(&self) -> bool {
+        // returns true if the routable packet should elicit a response
+        // e.g. reply, ACK packets end a conversation,
+        // and firmware should not wait for response
+        match self {
+            Packet::DmaAddTraceReply { .. } | Packet::DmaRemoveTraceReply { .. } |
+                Packet::DmaPlaybackReply { .. } | Packet::SubkernelLoadRunReply { .. } |
+                Packet::SubkernelMessageAck { .. } | Packet::DmaPlaybackStatus { .. } |
+                Packet::SubkernelFinished { .. } => false,
+            _ => true
+        }
+    }
 }

artiq/firmware/libproto_artiq/kernel_proto.rs

@@ -103,12 +103,12 @@ pub enum Message<'a> {
     SpiReadReply { succeeded: bool, data: u32 },
     SpiBasicReply { succeeded: bool },
 
-    SubkernelLoadRunRequest { id: u32, run: bool },
+    SubkernelLoadRunRequest { id: u32, destination: u8, run: bool },
     SubkernelLoadRunReply { succeeded: bool },
-    SubkernelAwaitFinishRequest { id: u32, timeout: u64 },
+    SubkernelAwaitFinishRequest { id: u32, timeout: i64 },
     SubkernelAwaitFinishReply { status: SubkernelStatus },
-    SubkernelMsgSend { id: u32, count: u8, tag: &'a [u8], data: *const *const () },
-    SubkernelMsgRecvRequest { id: u32, timeout: u64, tags: &'a [u8] },
+    SubkernelMsgSend { id: u32, destination: Option<u8>, count: u8, tag: &'a [u8], data: *const *const () },
+    SubkernelMsgRecvRequest { id: i32, timeout: i64, tags: &'a [u8] },
     SubkernelMsgRecvReply { status: SubkernelStatus, count: u8 },
 
     Log(fmt::Arguments<'a>),

artiq/firmware/libunwind/src/lib.rs

@@ -3,8 +3,8 @@
 #![feature(link_cfg)]
 #![feature(nll)]
 #![feature(staged_api)]
-#![feature(unwind_attributes)]
 #![feature(static_nobundle)]
+#![feature(c_unwind)]
 #![cfg_attr(not(target_env = "msvc"), feature(libc))]
 
 mod libunwind;

artiq/firmware/libunwind/src/libunwind.rs

@@ -81,9 +81,14 @@ pub type _Unwind_Exception_Cleanup_Fn =
     all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
     link(name = "unwind", kind = "static")
 )]
-extern "C" {
-    #[unwind(allowed)]
+extern "C-unwind" {
     pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !;
+}
+#[cfg_attr(
+    all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
+    link(name = "unwind", kind = "static")
+)]
+extern "C" {
     pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception);
     pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void;
     pub fn _Unwind_GetRegionStart(ctx: *mut _Unwind_Context) -> _Unwind_Ptr;
@@ -230,9 +235,13 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
+    }
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
         pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn,
                                  trace_argument: *mut c_void)
                                  -> _Unwind_Reason_Code;
@@ -242,8 +251,7 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
     }
 

artiq/firmware/riscv32g-unknown-none-elf.json

@@ -21,20 +21,6 @@
   },
   "relro-level": "full",
   "target-family": "unix",
-  "target-pointer-width": "32",
-  "unsupported-abis": [
-    "cdecl",
-    "stdcall",
-    "fastcall",
-    "vectorcall",
-    "thiscall",
-    "aapcs",
-    "win64",
-    "sysv64",
-    "ptx-kernel",
-    "msp430-interrupt",
-    "x86-interrupt",
-    "amdgpu-kernel"
-  ]
+  "target-pointer-width": "32"
 }
   

artiq/firmware/riscv32ima-unknown-none-elf.json

@@ -13,19 +13,5 @@
   "max-atomic-width": 32,
   "panic-strategy": "unwind",
   "relocation-model": "static",
-  "target-pointer-width": "32",
-  "unsupported-abis": [
-    "cdecl",
-    "stdcall",
-    "fastcall",
-    "vectorcall",
-    "thiscall",
-    "aapcs",
-    "win64",
-    "sysv64",
-    "ptx-kernel",
-    "msp430-interrupt",
-    "x86-interrupt",
-    "amdgpu-kernel"
-  ]
+  "target-pointer-width": "32"
 }

artiq/firmware/runtime/Cargo.toml

@@ -17,7 +17,7 @@ failure = { version = "0.1", default-features = false }
 failure_derive = { version = "0.1", default-features = false }
 byteorder = { version = "1.0", default-features = false }
 cslice = { version = "0.3" }
-log = { version = "0.4", default-features = false }
+log = { version = "=0.4.14", default-features = false }
 managed = { version = "^0.7.1", default-features = false, features = ["alloc", "map"] }
 dyld = { path = "../libdyld" }
 eh = { path = "../libeh" }
@@ -37,7 +37,7 @@ features = ["alloc", "medium-ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"
 
 [dependencies.fringe]
 git = "https://git.m-labs.hk/M-Labs/libfringe.git"
-rev = "3ecbe5"
+rev = "53a964"
 default-features = false
 features = ["alloc"]
 

artiq/firmware/runtime/Makefile

@@ -10,8 +10,6 @@ LDFLAGS += \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: runtime.bin runtime.fbi
 
 .PHONY: $(RUSTOUT)/libruntime.a

artiq/firmware/runtime/analyzer.rs

@@ -52,7 +52,7 @@ pub mod remote_analyzer {
         pub data: Vec<u8>
     }
 
-    pub fn get_data(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
+    pub fn get_data(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
         up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
     ) -> Result<RemoteBuffer, drtio::Error> {
             // gets data from satellites and returns consolidated data
@@ -62,7 +62,7 @@ pub mod remote_analyzer {
             let mut remote_total_bytes = 0;
 
             let data_vec = drtio::analyzer_query(
-                io, aux_mutex, routing_table, up_destinations
+                io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, up_destinations
             )?;
             for data in data_vec {
                 remote_total_bytes += data.total_byte_count;
@@ -83,6 +83,7 @@ pub mod remote_analyzer {
 
 
 fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
+    _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex,
     _routing_table: &drtio_routing::RoutingTable,
     _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
 ) -> Result<(), IoError<SchedError>> {
@@ -96,7 +97,7 @@ fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
 
     #[cfg(has_drtio)]
     let remote = remote_analyzer::get_data(
-        _io, _aux_mutex, _routing_table, _up_destinations);
+        _io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, _up_destinations);
     #[cfg(has_drtio)]
     let (header, remote_data) = match remote {
         Ok(remote) => (Header {
@@ -143,7 +144,7 @@ fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
     Ok(())
 }
 
-pub fn thread(io: Io, aux_mutex: &Mutex,
+pub fn thread(io: Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
     routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>,
     up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>) {
     let listener = TcpListener::new(&io, 65535);
@@ -158,7 +159,7 @@ pub fn thread(io: Io, aux_mutex: &Mutex,
         disarm();
 
         let routing_table = routing_table.borrow();
-        match worker(&mut stream, &io, aux_mutex, &routing_table, up_destinations) {
+        match worker(&mut stream, &io, aux_mutex, ddma_mutex, subkernel_mutex, &routing_table, up_destinations) {
             Ok(())   => (),
             Err(err) => error!("analyzer aborted: {}", err)
         }

artiq/firmware/runtime/kern_hwreq.rs

@@ -11,13 +11,15 @@ use board_artiq::spi as local_spi;
 #[cfg(has_drtio)]
 mod remote_i2c {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn start(io: &Io, aux_mutex: &Mutex,
+    pub fn start(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cStartRequest {
                 destination: destination,
                 busno: busno
@@ -37,10 +39,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn restart(io: &Io, aux_mutex: &Mutex, 
+    pub fn restart(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cRestartRequest {
                 destination: destination,
                 busno: busno
@@ -60,10 +63,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn stop(io: &Io, aux_mutex: &Mutex,
+    pub fn stop(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno,
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
             &drtioaux::Packet::I2cStopRequest  {
                 destination: destination,
                 busno: busno
@@ -83,10 +87,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn write(io: &Io, aux_mutex: &Mutex,
+    pub fn write(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, data: u8
     ) -> Result<bool, &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cWriteRequest {
                 destination: destination,
                 busno: busno,
@@ -107,10 +112,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn read(io: &Io, aux_mutex: &Mutex,
+    pub fn read(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, ack: bool
     ) -> Result<u8, &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cReadRequest {
                 destination: destination,
                 busno: busno,
@@ -131,10 +137,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn switch_select(io: &Io, aux_mutex: &Mutex,
+    pub fn switch_select(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, address: u8, mask: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cSwitchSelectRequest {
                 destination: destination,
                 busno: busno,
@@ -160,13 +167,15 @@ mod remote_i2c {
 #[cfg(has_drtio)]
 mod remote_spi {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn set_config(io: &Io, aux_mutex: &Mutex,
+    pub fn set_config(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, flags: u8, length: u8, div: u8, cs: u8
     ) -> Result<(), ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::SpiSetConfigRequest {
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, &drtioaux::Packet::SpiSetConfigRequest {
             destination: destination,
             busno: busno,
             flags: flags,
@@ -189,10 +198,11 @@ mod remote_spi {
         }
     }
 
-    pub fn write(io: &Io, aux_mutex: &Mutex,
+    pub fn write(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, data: u32
     ) -> Result<(), ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::SpiWriteRequest {
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, &drtioaux::Packet::SpiWriteRequest {
             destination: destination,
             busno: busno,
             data: data
@@ -212,9 +222,10 @@ mod remote_spi {
         }
     }
 
-    pub fn read(io: &Io, aux_mutex: &Mutex, linkno: u8, destination: u8, busno: u8
+    pub fn read(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, linkno: u8, destination: u8, busno: u8
     ) -> Result<u32, ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::SpiReadRequest {
                 destination: destination,
                 busno: busno
@@ -238,7 +249,7 @@ mod remote_spi {
 
 #[cfg(has_drtio)]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
         let destination = ($busno >> 16) as u8;
         let busno = $busno as u8;
         let hop = $routing_table.0[destination as usize][0];
@@ -246,27 +257,27 @@ macro_rules! dispatch {
             $mod_local::$func(busno, $($param, )*)
         } else {
             let linkno = hop - 1;
-            $mod_remote::$func($io, $aux_mutex, linkno, destination, busno, $($param, )*)
+            $mod_remote::$func($io, $aux_mutex, $ddma_mutex, $subkernel_mutex, $routing_table, linkno, destination, busno, $($param, )*)
         }
     }}
 }
 
 #[cfg(not(has_drtio))]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
         let busno = $busno as u8;
         $mod_local::$func(busno, $($param, )*)
     }}
 }
 
-pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex,
-        _routing_table: &drtio_routing::RoutingTable,
+pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
         request: &kern::Message) -> Result<bool, Error<SchedError>> {
     match request {
         &kern::RtioInitRequest => {
             info!("resetting RTIO");
-            rtio_mgt::reset(io, aux_mutex);
+            rtio_mgt::reset(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table);
             kern_acknowledge()
         }
 
@@ -282,47 +293,47 @@ pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex,
         }
 
         &kern::I2cStartRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, start).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, start).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cRestartRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, restart).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, restart).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cStopRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, stop).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, stop).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cWriteRequest { busno, data } => {
-            match dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, write, data) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, write, data) {
                 Ok(ack) => kern_send(io, &kern::I2cWriteReply { succeeded: true, ack: ack }),
                 Err(_) => kern_send(io, &kern::I2cWriteReply { succeeded: false, ack: false })
             }
         }
         &kern::I2cReadRequest { busno, ack } => {
-            match dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, read, ack) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, read, ack) {
                 Ok(data) => kern_send(io, &kern::I2cReadReply { succeeded: true, data: data }),
                 Err(_) => kern_send(io, &kern::I2cReadReply { succeeded: false, data: 0xff })
             }
         }
         &kern::I2cSwitchSelectRequest { busno, address, mask } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno,
                 switch_select, address, mask).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
 
         &kern::SpiSetConfigRequest { busno, flags, length, div, cs } => {
-            let succeeded = dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno,
                 set_config, flags, length, div, cs).is_ok();
             kern_send(io, &kern::SpiBasicReply { succeeded: succeeded })
         },
         &kern::SpiWriteRequest { busno, data } => {
-            let succeeded = dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno,
                 write, data).is_ok();
             kern_send(io, &kern::SpiBasicReply { succeeded: succeeded })
         }
         &kern::SpiReadRequest { busno } => {
-            match dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno, read) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno, read) {
                 Ok(data) => kern_send(io, &kern::SpiReadReply { succeeded: true, data: data }),
                 Err(_) => kern_send(io, &kern::SpiReadReply { succeeded: false, data: 0 })
             }

artiq/firmware/runtime/kernel.rs

@@ -103,7 +103,7 @@ pub mod subkernel {
     pub enum FinishStatus {
         Ok,
         CommLost,
-        Exception
+        Exception(u8) // exception source
     }
 
     #[derive(Debug, PartialEq, Clone, Copy)]
@@ -181,17 +181,17 @@ pub mod subkernel {
         Ok(())
     }
 
-    pub fn upload(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex, 
+    pub fn upload(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
              routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
-        drtio::subkernel_upload(io, aux_mutex, routing_table, id, 
+        drtio::subkernel_upload(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, 
             subkernel.destination, &subkernel.data)?;
         subkernel.state = SubkernelState::Uploaded; 
         Ok(()) 
     }
 
-    pub fn load(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &RoutingTable,
+    pub fn load(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &RoutingTable,
             id: u32, run: bool) -> Result<(), Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
@@ -199,7 +199,7 @@ pub mod subkernel {
             error!("for id: {} expected Uploaded, got: {:?}", id, subkernel.state);
             return Err(Error::IncorrectState);
         }
-        drtio::subkernel_load(io, aux_mutex, routing_table, id, subkernel.destination, run)?;
+        drtio::subkernel_load(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, subkernel.destination, run)?;
         if run {
             subkernel.state = SubkernelState::Running;
         }
@@ -216,7 +216,7 @@ pub mod subkernel {
         Ok(())
     }
 
-    pub fn subkernel_finished(io: &Io, subkernel_mutex: &Mutex, id: u32, with_exception: bool) {
+    pub fn subkernel_finished(io: &Io, subkernel_mutex: &Mutex, id: u32, with_exception: bool, exception_src: u8) {
         // called upon receiving DRTIO SubkernelRunDone
         let _lock = subkernel_mutex.lock(io).unwrap();
         let subkernel = unsafe { SUBKERNELS.get_mut(&id) };
@@ -226,7 +226,7 @@ pub mod subkernel {
             if subkernel.state == SubkernelState::Running {
                 subkernel.state = SubkernelState::Finished {
                     status: match with_exception {
-                    true => FinishStatus::Exception,
+                    true => FinishStatus::Exception(exception_src),
                     false => FinishStatus::Ok,
                     }
                 }
@@ -234,14 +234,14 @@ pub mod subkernel {
         }
     }
 
-    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
+    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
              routing_table: &RoutingTable, destination: u8, up: bool) {
         let _lock = subkernel_mutex.lock(io).unwrap();
         let subkernels_iter = unsafe { SUBKERNELS.iter_mut() };
         for (id, subkernel) in subkernels_iter {
             if subkernel.destination == destination {
                 if up {
-                    match drtio::subkernel_upload(io, aux_mutex, routing_table, *id, destination, &subkernel.data)
+                    match drtio::subkernel_upload(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id, destination, &subkernel.data)
                     {
                         Ok(_) => subkernel.state = SubkernelState::Uploaded,
                         Err(e) => error!("Error adding subkernel on destination {}: {}", destination, e)
@@ -256,7 +256,7 @@ pub mod subkernel {
         }
     }
 
-    pub fn retrieve_finish_status(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
+    pub fn retrieve_finish_status(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &RoutingTable, id: u32) -> Result<SubkernelFinished, Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let mut subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
@@ -266,9 +266,9 @@ pub mod subkernel {
                 Ok(SubkernelFinished {
                     id: id,
                     comm_lost: status == FinishStatus::CommLost,
-                    exception: if status == FinishStatus::Exception { 
-                        Some(drtio::subkernel_retrieve_exception(io, aux_mutex,
-                            routing_table, subkernel.destination)?) 
+                    exception: if let FinishStatus::Exception(dest) = status { 
+                        Some(drtio::subkernel_retrieve_exception(io, aux_mutex, ddma_mutex, subkernel_mutex,
+                            routing_table, dest)?) 
                     } else { None }
                 })
             },
@@ -278,8 +278,8 @@ pub mod subkernel {
         }
     }
 
-    pub fn await_finish(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
-        routing_table: &RoutingTable, id: u32, timeout: u64) -> Result<SubkernelFinished, Error> {
+    pub fn await_finish(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &RoutingTable, id: u32, timeout: i64) -> Result<SubkernelFinished, Error> {
         {
             let _lock = subkernel_mutex.lock(io)?;
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
@@ -291,7 +291,7 @@ pub mod subkernel {
         }
         let max_time = clock::get_ms() + timeout as u64;
         let _res = io.until(|| {
-            if clock::get_ms() > max_time {
+            if timeout > 0 && clock::get_ms() > max_time {
                 return true;
             }
             if subkernel_mutex.test_lock() {
@@ -305,11 +305,11 @@ pub mod subkernel {
                 _ => false
             }
         })?;
-        if clock::get_ms() > max_time {
+        if timeout > 0 && clock::get_ms() > max_time {
             error!("Remote subkernel finish await timed out");
             return Err(Error::Timeout);
         }
-        retrieve_finish_status(io, aux_mutex, subkernel_mutex, routing_table, id)
+        retrieve_finish_status(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id)
     }
 
     pub struct Message {
@@ -332,8 +332,9 @@ pub mod subkernel {
             Err(_) => return,
         };
         let subkernel = unsafe { SUBKERNELS.get(&id) };
-        if subkernel.is_none() || subkernel.unwrap().state != SubkernelState::Running {
-            // do not add messages for non-existing, non-running or deleted subkernels
+        if subkernel.is_some() && subkernel.unwrap().state != SubkernelState::Running {
+            warn!("received a message for a non-running subkernel #{}", id);
+            // do not add messages for non-running or deleted subkernels
             return
         }
         if status.is_first() {
@@ -359,19 +360,26 @@ pub mod subkernel {
         }
     }
 
-    pub fn message_await(io: &Io, subkernel_mutex: &Mutex, id: u32, timeout: u64
+    pub fn message_await(io: &Io, subkernel_mutex: &Mutex, id: u32, timeout: i64
     ) -> Result<Message, Error> {
-        {
+        let is_subkernel = {
             let _lock = subkernel_mutex.lock(io)?;
+            let is_subkernel = unsafe { SUBKERNELS.get(&id).is_some() };
+            if is_subkernel {
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
-                SubkernelState::Finished { .. } => return Err(Error::SubkernelFinished),
+                SubkernelState::Finished { status: FinishStatus::Ok } |
                 SubkernelState::Running => (),
+                SubkernelState::Finished {
+                    status: FinishStatus::CommLost,
+                    } => return Err(Error::SubkernelFinished),
                 _ => return Err(Error::IncorrectState)
             }
         }
+            is_subkernel
+        };
         let max_time = clock::get_ms() + timeout as u64;
         let message = io.until_ok(|| {
-            if clock::get_ms() > max_time {
+            if timeout > 0 && clock::get_ms() > max_time {
                 return Ok(None);
             }
             if subkernel_mutex.test_lock() {
@@ -384,10 +392,13 @@ pub mod subkernel {
                     return Ok(Some(unsafe { MESSAGE_QUEUE.remove(i) }));
                 }
             }
+            if is_subkernel {
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
-                SubkernelState::Finished { .. } => return Ok(None),
+                SubkernelState::Finished { status: FinishStatus::CommLost } | 
+                    SubkernelState::Finished { status: FinishStatus::Exception(_) }  => return Ok(None),
                 _ => ()
                 }
+            }
             Err(())
         });
         match message {
@@ -407,20 +418,22 @@ pub mod subkernel {
         }
     }
 
-    pub fn message_send<'a>(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
-        routing_table: &RoutingTable, id: u32, count: u8, tag: &'a [u8], message: *const *const ()
+    pub fn message_send<'a>(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &RoutingTable, id: u32, destination: Option<u8>, count: u8, tag: &'a [u8], message: *const *const ()
     ) -> Result<(), Error> {
         let mut writer = Cursor::new(Vec::new());
-        let _lock = subkernel_mutex.lock(io)?;
-        let destination = unsafe { SUBKERNELS.get(&id).unwrap().destination };
-
         // reuse rpc code for sending arbitrary data
         rpc::send_args(&mut writer, 0, tag, message, false)?;
         // skip service tag, but overwrite first byte with tag count
+        let destination = destination.unwrap_or_else(|| {
+                let _lock = subkernel_mutex.lock(io).unwrap();
+                unsafe { SUBKERNELS.get(&id).unwrap().destination }
+            }
+        );
         let data = &mut writer.into_inner()[3..];
         data[0] = count;
         Ok(drtio::subkernel_send_message(
-            io, aux_mutex, routing_table, id, destination, data
+            io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, destination, data
         )?)
     }
 }

artiq/firmware/runtime/main.rs

@@ -36,12 +36,16 @@ use smoltcp::wire::HardwareAddress;
 use board_misoc::{csr, ident, clock, spiflash, config, net_settings, pmp, boot};
 #[cfg(has_ethmac)]
 use board_misoc::ethmac;
+#[cfg(soc_platform = "kasli")]
+use board_misoc::irq;
 use board_misoc::net_settings::{Ipv4AddrConfig};
 #[cfg(has_drtio)]
 use board_artiq::drtioaux;
 use board_artiq::drtio_routing;
 use board_artiq::{mailbox, rpc_queue};
 use proto_artiq::{mgmt_proto, moninj_proto, rpc_proto, session_proto, kernel_proto};
+#[cfg(has_wrpll)]
+use board_artiq::si549;
 #[cfg(has_drtio_eem)]
 use board_artiq::drtio_eem;
 #[cfg(has_rtio_analyzer)]
@@ -214,15 +218,19 @@ fn startup() {
     #[cfg(any(has_rtio_moninj, has_drtio))]
     {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let drtio_routing_table = drtio_routing_table.clone();
-        io.spawn(4096, move |io| { moninj::thread(io, &aux_mutex, &drtio_routing_table) });
+        io.spawn(4096, move |io| { moninj::thread(io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &drtio_routing_table) });
     }
     #[cfg(has_rtio_analyzer)]
     {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let drtio_routing_table = drtio_routing_table.clone();
         let up_destinations = up_destinations.clone();
-        io.spawn(8192, move |io| { analyzer::thread(io, &aux_mutex, &drtio_routing_table, &up_destinations) });
+        io.spawn(8192, move |io| { analyzer::thread(io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &drtio_routing_table, &up_destinations) });
     }
 
     #[cfg(has_grabber)]
@@ -261,6 +269,11 @@ pub extern fn main() -> i32 {
 
         pmp::init_stack_guard(&_sstack_guard as *const u8 as usize);
 
+        #[cfg(soc_platform = "kasli")]
+        irq::enable_interrupts();
+        #[cfg(has_wrpll)]
+        irq::enable(csr::WRPLL_INTERRUPT);
+
         logger_artiq::BufferLogger::new(&mut LOG_BUFFER[..]).register(||
             boot::start_user(startup as usize)
         );
@@ -295,8 +308,11 @@ pub extern fn exception(regs: *const TrapFrame) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
     match cause {
-        mcause::Trap::Interrupt(source) => {
-            info!("Called interrupt with {:?}", source);
+        mcause::Trap::Interrupt(_source) => {
+            #[cfg(has_wrpll)]
+            if irq::is_pending(csr::WRPLL_INTERRUPT) {
+                si549::wrpll::interrupt_handler();
+            }
         },
 
         mcause::Trap::Exception(mcause::Exception::UserEnvCall) => {

artiq/firmware/runtime/moninj.rs

@@ -50,12 +50,15 @@ mod local_moninj {
 #[cfg(has_drtio)]
 mod remote_moninj {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn read_probe(io: &Io, aux_mutex: &Mutex, linkno: u8,
+    pub fn read_probe(io: &Io, aux_mutex: &Mutex, 
+        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, probe: u8) -> u64 {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::MonitorRequest { 
                 destination: destination,
                 channel: channel,
@@ -69,7 +72,9 @@ mod remote_moninj {
         0
     }
 
-    pub fn inject(io: &Io, aux_mutex: &Mutex, linkno: u8, 
+    pub fn inject(io: &Io, aux_mutex: &Mutex,
+        _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, 
+        _routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, overrd: u8, value: u8) {
         let _lock = aux_mutex.lock(io).unwrap();
         drtioaux::send(linkno, &drtioaux::Packet::InjectionRequest {
@@ -80,9 +85,11 @@ mod remote_moninj {
         }).unwrap();
     }
 
-    pub fn read_injection_status(io: &Io, aux_mutex: &Mutex, linkno: u8,
+    pub fn read_injection_status(io: &Io, aux_mutex: &Mutex,
+        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, overrd: u8) -> u8 {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::InjectionStatusRequest {
                 destination: destination,
                 channel: channel,
@@ -99,7 +106,7 @@ mod remote_moninj {
 
 #[cfg(has_drtio)]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
         let destination = ($channel >> 16) as u8;
         let channel = $channel as u16;
         let hop = $routing_table.0[destination as usize][0];
@@ -107,21 +114,21 @@ macro_rules! dispatch {
             local_moninj::$func(channel, $($param, )*)
         } else {
             let linkno = hop - 1;
-            remote_moninj::$func($io, $aux_mutex, linkno, destination, channel, $($param, )*)
+            remote_moninj::$func($io, $aux_mutex, $ddma_mutex, $subkernel_mutex, $routing_table, linkno, destination, channel, $($param, )*)
         }
     }}
 }
 
 #[cfg(not(has_drtio))]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
         let channel = $channel as u16;
         local_moninj::$func(channel, $($param, )*)
     }}
 }
 
-fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing::RoutingTable,
-    mut stream: &mut TcpStream) -> Result<(), Error<SchedError>> {
+fn connection_worker(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex,
+    _routing_table: &drtio_routing::RoutingTable, mut stream: &mut TcpStream) -> Result<(), Error<SchedError>> {
     let mut probe_watch_list = BTreeMap::new();
     let mut inject_watch_list = BTreeMap::new();
     let mut next_check = 0;
@@ -150,9 +157,9 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
                     }
                 },
                 HostMessage::Inject { channel, overrd, value } => dispatch!(
-                    io, _aux_mutex, _routing_table, channel, inject, overrd, value),
+                    io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, inject, overrd, value),
                 HostMessage::GetInjectionStatus { channel, overrd } => {
-                    let value = dispatch!(io, _aux_mutex, _routing_table, channel, read_injection_status, overrd);
+                    let value = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_injection_status, overrd);
                     let reply = DeviceMessage::InjectionStatus {
                         channel: channel,
                         overrd: overrd,
@@ -169,7 +176,7 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
 
         if clock::get_ms() > next_check {
             for (&(channel, probe), previous) in probe_watch_list.iter_mut() {
-                let current = dispatch!(io, _aux_mutex, _routing_table, channel, read_probe, probe);
+                let current = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_probe, probe);
                 if previous.is_none() || previous.unwrap() != current {
                     let message = DeviceMessage::MonitorStatus {
                         channel: channel,
@@ -184,7 +191,7 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
                 }
             }
             for (&(channel, overrd), previous) in inject_watch_list.iter_mut() {
-                let current = dispatch!(io, _aux_mutex, _routing_table, channel, read_injection_status, overrd);
+                let current = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_injection_status, overrd);
                 if previous.is_none() || previous.unwrap() != current {
                     let message = DeviceMessage::InjectionStatus {
                         channel: channel,
@@ -205,18 +212,20 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
     }
 }
 
-pub fn thread(io: Io, aux_mutex: &Mutex, routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>) {
+pub fn thread(io: Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>) {
     let listener = TcpListener::new(&io, 2047);
     listener.listen(1383).expect("moninj: cannot listen");
 
     loop {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let routing_table = routing_table.clone();
         let stream = listener.accept().expect("moninj: cannot accept").into_handle();
         io.spawn(16384, move |io| {
             let routing_table = routing_table.borrow();
             let mut stream = TcpStream::from_handle(&io, stream);
-            match connection_worker(&io, &aux_mutex, &routing_table, &mut stream) {
+            match connection_worker(&io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &routing_table, &mut stream) {
                 Ok(()) => {},
                 Err(err) => error!("moninj aborted: {}", err)
             }

artiq/firmware/runtime/rtio_clocking.rs

@@ -1,8 +1,11 @@
 use board_misoc::config;
+#[cfg(has_si5324)]
 use board_artiq::si5324;
+#[cfg(has_si549)]
+use board_artiq::si549;
 use board_misoc::{csr, clock};
 
-#[derive(Debug, PartialEq)]
+#[derive(Debug, PartialEq, Copy, Clone)]
 #[allow(non_camel_case_types)]
 pub enum RtioClock {
     Default,
@@ -89,13 +92,14 @@ pub mod crg {
 
 // Si5324 input to select for locking to an external clock (as opposed to
 // a recovered link clock in DRTIO satellites, which is handled elsewhere).
-#[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
+#[cfg(all(has_si5324, soc_platform = "kasli", hw_rev = "v2.0"))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin1;
-#[cfg(all(soc_platform = "kasli", not(hw_rev = "v2.0")))]
+#[cfg(all(has_si5324, soc_platform = "kasli", not(hw_rev = "v2.0")))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin2;
-#[cfg(all(soc_platform = "kc705"))]
+#[cfg(all(has_si5324, soc_platform = "kc705"))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin2;
 
+#[cfg(has_si5324)]
 fn setup_si5324_pll(cfg: RtioClock) {
     let (si5324_settings, si5324_ref_input) = match cfg {
         RtioClock::Ext0_Synth0_10to125 => { // 125 MHz output from 10 MHz CLKINx reference, 504 Hz BW
@@ -214,7 +218,7 @@ fn setup_si5324_pll(cfg: RtioClock) {
     si5324::setup(&si5324_settings, si5324_ref_input).expect("cannot initialize Si5324");
 }
 
-fn setup_si5324(clock_cfg: RtioClock) {
+fn sysclk_setup(clock_cfg: RtioClock) {
     let switched = unsafe {
         csr::crg::switch_done_read()
     };
@@ -222,6 +226,8 @@ fn setup_si5324(clock_cfg: RtioClock) {
         info!("Clocking has already been set up.");
         return;
     }
+
+    #[cfg(has_si5324)]
     match clock_cfg {
         RtioClock::Ext0_Bypass => {
             info!("using external RTIO clock with PLL bypass");
@@ -230,7 +236,10 @@ fn setup_si5324(clock_cfg: RtioClock) {
         _ => setup_si5324_pll(clock_cfg),
     }
 
-    // switch sysclk source to si5324
+    #[cfg(has_si549)]
+    si549::main_setup(&get_si549_setting(clock_cfg)).expect("cannot initialize main Si549");
+
+    // switch sysclk source
     #[cfg(not(has_drtio))]
     {
         info!("Switching sys clock, rebooting...");
@@ -244,9 +253,153 @@ fn setup_si5324(clock_cfg: RtioClock) {
 }
 
 
+#[cfg(all(has_si549, has_wrpll))]
+fn wrpll_setup(clk: RtioClock, si549_settings: &si549::FrequencySetting) {
+    // register values are directly copied from preconfigured mmcm
+    let (mmcm_setting, mmcm_bypass) = match clk {
+        RtioClock::Ext0_Synth0_10to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 62.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 5
+                clkout0_reg1: 0x1083,
+                clkout0_reg2: 0x0080,
+                clkfbout_reg1: 0x179e,
+                clkfbout_reg2: 0x4c00,
+                div_reg: 0x1041,
+                lock_reg1: 0x00fa,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x1008,
+                filt_reg2: 0x8800,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_80to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 15.625, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x11c7,
+                clkfbout_reg2: 0x5880,
+                div_reg: 0x1041,
+                lock_reg1: 0x028a,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x9908,
+                filt_reg2: 0x8100,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_100to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 12.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x1145,
+                clkfbout_reg2: 0x4c00,
+                div_reg: 0x1041,
+                lock_reg1: 0x0339,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x9108,
+                filt_reg2: 0x0100,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_125to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 10, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x1145,
+                clkfbout_reg2: 0x0000,
+                div_reg: 0x1041,
+                lock_reg1: 0x03e8,
+                lock_reg2: 0x7001,
+                lock_reg3: 0xf3e9,
+                power_reg: 0x0100,
+                filt_reg1: 0x9908,
+                filt_reg2: 0x1100,
+            },
+            true,
+        ),
+        _ => unreachable!(),
+    };
+
+    si549::helper_setup(&si549_settings).expect("cannot initialize helper Si549");
+    si549::wrpll_refclk::setup(mmcm_setting, mmcm_bypass).expect("cannot initialize ref clk for wrpll");
+    si549::wrpll::select_recovered_clock(true);
+}
+
+#[cfg(has_si549)]
+fn get_si549_setting(clk: RtioClock) -> si549::FrequencySetting {
+    match clk {
+        RtioClock::Ext0_Synth0_10to125 => {
+            info!("using 10MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_80to125 => {
+            info!("using 80MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_100to125 => {
+            info!("using 100MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_125to125 => {
+            info!("using 125MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Int_100 => {
+            info!("using internal 100MHz RTIO clock");
+        }
+        RtioClock::Int_125 => {
+            info!("using internal 125MHz RTIO clock");
+        }
+        _ => {
+            warn!(
+                "rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.",
+                clk
+            );
+        }
+    };
+
+    match clk {
+        RtioClock::Int_100 => {
+            si549::FrequencySetting {
+                main: si549::DividerConfig {
+                    hsdiv: 0x06C,
+                    lsdiv: 0,
+                    fbdiv: 0x046C5F49797,
+                },
+                helper: si549::DividerConfig {
+                    // 100MHz*32767/32768
+                    hsdiv: 0x06C,
+                    lsdiv: 0,
+                    fbdiv: 0x046C5670BBD,
+                },
+            }
+        }
+        _ => {
+            // Everything else use 125MHz
+            si549::FrequencySetting {
+                main: si549::DividerConfig {
+                    hsdiv: 0x058,
+                    lsdiv: 0,
+                    fbdiv: 0x04815791F25,
+                },
+                helper: si549::DividerConfig {
+                    // 125MHz*32767/32768
+                    hsdiv: 0x058,
+                    lsdiv: 0,
+                    fbdiv: 0x04814E8F442,
+                },
+            }
+        }
+    }
+}
+
 pub fn init() {
     let clock_cfg = get_rtio_clock_cfg();
-    setup_si5324(clock_cfg);
+    sysclk_setup(clock_cfg);
 
     #[cfg(has_drtio)]
     {
@@ -282,4 +435,18 @@ pub fn init() {
             error!("RTIO clock failed");
         }
     }
+
+    #[cfg(all(has_si549, has_wrpll))]
+    {
+        // SYS CLK switch will reset CSRs that are used by WRPLL
+        match clock_cfg {
+            RtioClock::Ext0_Synth0_10to125
+            | RtioClock::Ext0_Synth0_80to125
+            | RtioClock::Ext0_Synth0_100to125
+            | RtioClock::Ext0_Synth0_125to125 => {
+                wrpll_setup(clock_cfg, &get_si549_setting(clock_cfg));
+            }
+            _ => {}
+        }
+    }
 }

artiq/firmware/runtime/rtio_dma.rs

@@ -120,12 +120,12 @@ pub mod remote_dma {
         Ok(playback_state)
     }
 
-    pub fn erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = ddma_mutex.lock(io)?;
         let destinations = unsafe { TRACES.get(&id).unwrap() };
         for destination in destinations.keys() {
-            match drtio::ddma_send_erase(io, aux_mutex, routing_table, id, *destination) {
+            match drtio::ddma_send_erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination) {
                 Ok(_) => (),
                 Err(e) => error!("Error erasing trace on DMA: {}", e)
             } 
@@ -134,18 +134,18 @@ pub mod remote_dma {
         Ok(())
     }
 
-    pub fn upload_traces(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn upload_traces(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = ddma_mutex.lock(io)?;
         let traces = unsafe { TRACES.get_mut(&id).unwrap() };
         for (destination, mut trace) in traces {
-            drtio::ddma_upload_trace(io, aux_mutex, routing_table, id, *destination, trace.get_trace())?;
+            drtio::ddma_upload_trace(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination, trace.get_trace())?;
             trace.state = RemoteState::Loaded;
         }
         Ok(())
     }
 
-    pub fn playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32, timestamp: u64) -> Result<(), Error>{
         // triggers playback on satellites
         let destinations = unsafe { 
@@ -161,16 +161,16 @@ pub mod remote_dma {
                     return Err(Error::IncorrectState);
                 }
             }
-            drtio::ddma_send_playback(io, aux_mutex, routing_table, id, *destination, timestamp)?;
+            drtio::ddma_send_playback(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination, timestamp)?;
         }
         Ok(())
     }
 
     pub fn playback_done(io: &Io, ddma_mutex: &Mutex, 
-            id: u32, destination: u8, error: u8, channel: u32, timestamp: u64) {
+            id: u32, source: u8, error: u8, channel: u32, timestamp: u64) {
         // called upon receiving PlaybackDone aux packet
         let _lock = ddma_mutex.lock(io).unwrap();
-        let mut trace = unsafe { TRACES.get_mut(&id).unwrap().get_mut(&destination).unwrap() };
+        let mut trace = unsafe { TRACES.get_mut(&id).unwrap().get_mut(&source).unwrap() };
         trace.state = RemoteState::PlaybackEnded {
             error: error, 
             channel: channel, 
@@ -178,7 +178,7 @@ pub mod remote_dma {
         };
     }
 
-    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
         routing_table: &RoutingTable, destination: u8, up: bool) {
         // update state of the destination, resend traces if it's up
         let _lock = ddma_mutex.lock(io).unwrap();
@@ -186,7 +186,7 @@ pub mod remote_dma {
         for (id, dest_traces) in traces_iter {
             if let Some(trace) = dest_traces.get_mut(&destination) {
                 if up {
-                    match drtio::ddma_upload_trace(io, aux_mutex, routing_table, *id, destination, trace.get_trace())
+                    match drtio::ddma_upload_trace(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id, destination, trace.get_trace())
                     {
                         Ok(_) => trace.state = RemoteState::Loaded,
                         Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e)

artiq/firmware/runtime/rtio_mgt.rs

@@ -65,7 +65,7 @@ pub mod drtio {
         let up_destinations = up_destinations.clone();
         let ddma_mutex = ddma_mutex.clone();
         let subkernel_mutex = subkernel_mutex.clone();
-        io.spawn(8192, move |io| {
+        io.spawn(16384, move |io| {
             let routing_table = routing_table.borrow();
             link_thread(io, &aux_mutex, &routing_table, &up_destinations, &ddma_mutex, &subkernel_mutex);
         });
@@ -96,36 +96,76 @@ pub mod drtio {
         }
     }
 
-    fn process_async_packets(io: &Io, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, linkno: u8,
-            packet: drtioaux::Packet) -> Option<drtioaux::Packet> {
-        // returns None if an async packet has been consumed
+    fn process_async_packets(io: &Io, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+            routing_table: &drtio_routing::RoutingTable, linkno: u8, packet: &drtioaux::Packet
+    ) -> bool {
         match packet {
-            drtioaux::Packet::DmaPlaybackStatus { id, destination, error, channel, timestamp } => {
-                remote_dma::playback_done(io, ddma_mutex, id, destination, error, channel, timestamp);
-                None
+            // packets to be consumed locally
+            drtioaux::Packet::DmaPlaybackStatus { id, source, destination: 0, error, channel, timestamp } => {
+                remote_dma::playback_done(io, ddma_mutex, *id, *source, *error, *channel, *timestamp);
+                true
             },
-            drtioaux::Packet::SubkernelFinished { id, with_exception } => {
-                subkernel::subkernel_finished(io, subkernel_mutex, id, with_exception);
-                None
+            drtioaux::Packet::SubkernelFinished { id, destination: 0, with_exception, exception_src } => {
+                subkernel::subkernel_finished(io, subkernel_mutex, *id, *with_exception, *exception_src);
+                true
             },
-            drtioaux::Packet::SubkernelMessage { id, destination: from, status, length, data } => {
-                subkernel::message_handle_incoming(io, subkernel_mutex, id, status, length as usize, &data);
+            drtioaux::Packet::SubkernelMessage { id, source: from, destination: 0, status, length, data } => {
+                subkernel::message_handle_incoming(io, subkernel_mutex, *id, *status, *length as usize, data);
                 // acknowledge receiving part of the message
                 drtioaux::send(linkno, 
-                    &drtioaux::Packet::SubkernelMessageAck { destination: from }
+                    &drtioaux::Packet::SubkernelMessageAck { destination: *from }
                 ).unwrap();
-                None
+                true
+            },
+            // (potentially) routable packets
+            drtioaux::Packet::DmaAddTraceRequest      { destination, .. } |
+                drtioaux::Packet::DmaAddTraceReply        { destination, .. } |
+                drtioaux::Packet::DmaRemoveTraceRequest   { destination, .. } |
+                drtioaux::Packet::DmaRemoveTraceReply     { destination, .. } |
+                drtioaux::Packet::DmaPlaybackRequest      { destination, .. } |
+                drtioaux::Packet::DmaPlaybackReply        { destination, .. } |
+                drtioaux::Packet::SubkernelLoadRunRequest { destination, .. } |
+                drtioaux::Packet::SubkernelLoadRunReply   { destination, .. } |
+                drtioaux::Packet::SubkernelMessage        { destination, .. } |
+                drtioaux::Packet::SubkernelMessageAck     { destination, .. } |
+                drtioaux::Packet::DmaPlaybackStatus       { destination, .. } |
+                drtioaux::Packet::SubkernelFinished       { destination, .. } => {
+                if *destination == 0 {
+                    false
+                } else {
+                    let dest_link = routing_table.0[*destination as usize][0] - 1;
+                    if dest_link == linkno {
+                        warn!("[LINK#{}] Re-routed packet would return to the same link, dropping: {:?}", linkno, packet);
+                    } else {
+                        drtioaux::send(dest_link, &packet).unwrap();
                     }
-            other => Some(other)
+                    true
+                }
+            }
+            _ => false
         }
     }
 
-    pub fn aux_transact(io: &Io, aux_mutex: &Mutex, linkno: u8, request: &drtioaux::Packet
+    pub fn aux_transact(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, linkno: u8, request: &drtioaux::Packet
     ) -> Result<drtioaux::Packet, Error> {
         let _lock = aux_mutex.lock(io)?;
         drtioaux::send(linkno, request).unwrap();
+        loop {
             let reply = recv_aux_timeout(io, linkno, 200)?;
-        Ok(reply)
+            if !process_async_packets(io, ddma_mutex, subkernel_mutex, routing_table, linkno, &reply) {
+                // returns none if it was an async packet
+                return Ok(reply);
+            }
+        }
+    }
+
+    fn setup_transact(io: &Io, aux_mutex: &Mutex, linkno: u8, request: &drtioaux::Packet) -> Result<drtioaux::Packet, Error> {
+        /* shorter aux_transact for setup purposes, no checking for async packets,
+           as they should not be generated yet */
+        let _lock = aux_mutex.lock(io)?;
+        drtioaux::send(linkno, request).unwrap();
+        recv_aux_timeout(io, linkno, 200)
     }
 
     pub fn clear_buffers(io: &Io, aux_mutex: &Mutex) {
@@ -148,7 +188,7 @@ pub mod drtio {
             if count > 100 {
                 return 0;
             }
-            let reply = aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::EchoRequest);
+            let reply = setup_transact(io, aux_mutex, linkno, &drtioaux::Packet::EchoRequest);
             match reply {
                 Ok(drtioaux::Packet::EchoReply) => {
                     // make sure receive buffer is drained
@@ -187,7 +227,7 @@ pub mod drtio {
     fn load_routing_table(io: &Io, aux_mutex: &Mutex,
         linkno: u8, routing_table: &drtio_routing::RoutingTable) -> Result<(), Error> {
         for i in 0..drtio_routing::DEST_COUNT {
-            let reply = aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::RoutingSetPath {
+            let reply = setup_transact(io, aux_mutex, linkno, &drtioaux::Packet::RoutingSetPath {
                 destination: i as u8,
                 hops: routing_table.0[i]
             })?;
@@ -200,7 +240,7 @@ pub mod drtio {
 
     fn set_rank(io: &Io, aux_mutex: &Mutex,
         linkno: u8, rank: u8) -> Result<(), Error> {
-        let reply = aux_transact(io, aux_mutex, linkno,
+        let reply = setup_transact(io, aux_mutex, linkno,
             &drtioaux::Packet::RoutingSetRank {
                 rank: rank
             })?;
@@ -223,16 +263,19 @@ pub mod drtio {
         }
     }
 
-    fn process_unsolicited_aux(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, linkno: u8) {
+    fn process_unsolicited_aux(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8) {
         let _lock = aux_mutex.lock(io).unwrap();
+        loop {
             match drtioaux::recv(linkno) {
                 Ok(Some(packet)) => {
-                if let Some(packet) = process_async_packets(io, ddma_mutex, subkernel_mutex, linkno, packet) {
+                    if !process_async_packets(&io, ddma_mutex, subkernel_mutex, routing_table, linkno, &packet) {
                         warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet);
                     }
+                },
+                Ok(None) => return,
+                Err(_) => { warn!("[LINK#{}] aux packet error", linkno); return }
             }
-            Ok(None) => (),
-            Err(_) => warn!("[LINK#{}] aux packet error", linkno)
         }
     }
 
@@ -293,53 +336,46 @@ pub mod drtio {
                 let linkno = hop - 1;
                 if destination_up(up_destinations, destination) {
                     if up_links[linkno as usize] {
-                        loop {
-                            let reply = aux_transact(io, aux_mutex, linkno, 
+                        let reply = aux_transact(io, aux_mutex,
+                             ddma_mutex, subkernel_mutex, routing_table, linkno,
                             &drtioaux::Packet::DestinationStatusRequest {
                                 destination: destination
                             });
                         if let Ok(reply) = reply {
-                                let reply = process_async_packets(io, ddma_mutex, subkernel_mutex, linkno, reply);
                             match reply {
-                                    Some(drtioaux::Packet::DestinationDownReply) => {
+                                drtioaux::Packet::DestinationDownReply => {
                                     destination_set_up(routing_table, up_destinations, destination, false);
-                                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, false);
-                                        subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, false);
+                                    remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
+                                    subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
                                 }
-                                    Some(drtioaux::Packet::DestinationOkReply) => (),
-                                    Some(drtioaux::Packet::DestinationSequenceErrorReply { channel }) => {
+                                drtioaux::Packet::DestinationOkReply => (),
+                                drtioaux::Packet::DestinationSequenceErrorReply { channel } => {
                                     error!("[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
                                     unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
                                 }
-                                    Some(drtioaux::Packet::DestinationCollisionReply { channel }) => {
+                                drtioaux::Packet::DestinationCollisionReply { channel } => {
                                     error!("[DEST#{}] RTIO collision involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
                                     unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
                                 }
-                                    Some(drtioaux::Packet::DestinationBusyReply { channel }) => {
+                                drtioaux::Packet::DestinationBusyReply { channel } => {
                                     error!("[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
                                     unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
                                 }
-                                    Some(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
-                                    None => { 
-                                        // continue asking until we get Destination...Reply or error out
-                                        // wait a bit not to overwhelm the receiver causing gateway errors
-                                        io.sleep(10).unwrap(); 
-                                        continue;
-                                    }
+                                packet => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
+                                
                             }
                         } else { 
                             error!("[DEST#{}] communication failed ({:?})", destination, reply.unwrap_err()); 
                         }
-                            break;
-                        }
                     } else {
                         destination_set_up(routing_table, up_destinations, destination, false);
-                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, false);
-                        subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, false);
+                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
+                        subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
                     }
                 } else {
                     if up_links[linkno as usize] {
-                        let reply = aux_transact(io, aux_mutex, linkno, 
+                        let reply = aux_transact(io, aux_mutex, ddma_mutex,
+                            subkernel_mutex, routing_table, linkno,
                             &drtioaux::Packet::DestinationStatusRequest {
                                 destination: destination
                             });
@@ -348,8 +384,8 @@ pub mod drtio {
                             Ok(drtioaux::Packet::DestinationOkReply) => {
                                 destination_set_up(routing_table, up_destinations, destination, true);
                                 init_buffer_space(destination as u8, linkno);
-                                remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, true);
-                                subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, true);
+                                remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, true);
+                                subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, true);
                             },
                             Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
                             Err(e) => error!("[DEST#{}] communication failed ({:?})", destination, e)
@@ -371,7 +407,7 @@ pub mod drtio {
                 if up_links[linkno as usize] {
                     /* link was previously up */
                     if link_rx_up(linkno) {
-                        process_unsolicited_aux(&io, aux_mutex, ddma_mutex, subkernel_mutex, linkno);
+                        process_unsolicited_aux(&io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno);
                         process_local_errors(linkno);
                     } else {
                         info!("[LINK#{}] link is down", linkno);
@@ -406,7 +442,7 @@ pub mod drtio {
         }
     }
 
-    pub fn reset(io: &Io, aux_mutex: &Mutex) {
+    pub fn reset(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable) {
         for linkno in 0..csr::DRTIO.len() {
             unsafe {
                 (csr::DRTIO[linkno].reset_write)(1);
@@ -422,7 +458,7 @@ pub mod drtio {
         for linkno in 0..csr::DRTIO.len() {
             let linkno = linkno as u8;
             if link_rx_up(linkno) {
-                let reply = aux_transact(io, aux_mutex, linkno,
+                let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
                     &drtioaux::Packet::ResetRequest);
                 match reply {
                     Ok(drtioaux::Packet::ResetAck) => (),
@@ -449,52 +485,52 @@ pub mod drtio {
             Ok(())
         }
 
-    pub fn ddma_upload_trace(io: &Io, aux_mutex: &Mutex,
-            routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, trace: &[u8]) -> Result<(), Error> {
+    pub fn ddma_upload_trace(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, trace: &[u8]
+    ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(trace, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
                 &drtioaux::Packet::DmaAddTraceRequest {
-                    id: id, destination: destination, status: status, length: len as u16, trace: *slice})?;
+                    id: id, source: 0, destination: destination, status: status, length: len as u16, trace: *slice})?;
             match reply {
-                drtioaux::Packet::DmaAddTraceReply { succeeded: true } => Ok(()),
-                drtioaux::Packet::DmaAddTraceReply { succeeded: false } => Err(Error::DmaAddTraceFail(destination)),
+                drtioaux::Packet::DmaAddTraceReply { destination: 0, succeeded: true, .. } => Ok(()),
+                drtioaux::Packet::DmaAddTraceReply { destination: 0, succeeded: false, .. } => Err(Error::DmaAddTraceFail(destination)),
                 packet => Err(Error::UnexpectedPacket(packet)),
             }
         })
     }
 
-    pub fn ddma_send_erase(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable, 
-            id: u32, destination: u8) -> Result<(), Error> {
+    pub fn ddma_send_erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::DmaRemoveTraceRequest { id: id, destination: destination })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::DmaRemoveTraceRequest { id: id, source: 0, destination: destination })?;
         match reply {
-            drtioaux::Packet::DmaRemoveTraceReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::DmaRemoveTraceReply { succeeded: false } => Err(Error::DmaEraseFail(destination)),
+            drtioaux::Packet::DmaRemoveTraceReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::DmaRemoveTraceReply { destination: 0, succeeded: false } => Err(Error::DmaEraseFail(destination)),
             packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
-    pub fn ddma_send_playback(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, timestamp: u64) -> Result<(), Error> {
+    pub fn ddma_send_playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, timestamp: u64) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::DmaPlaybackRequest{ id: id, destination: destination, timestamp: timestamp })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::DmaPlaybackRequest{ id: id, source: 0, destination: destination, timestamp: timestamp })?;
         match reply {
-            drtioaux::Packet::DmaPlaybackReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::DmaPlaybackReply { succeeded: false } =>
+            drtioaux::Packet::DmaPlaybackReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::DmaPlaybackReply { destination: 0, succeeded: false } =>
                     Err(Error::DmaPlaybackFail(destination)),
             packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
     #[cfg(has_rtio_analyzer)]
-    fn analyzer_get_data(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-        destination: u8) -> Result<RemoteBuffer, Error> {
+    fn analyzer_get_data(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, destination: u8) -> Result<RemoteBuffer, Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::AnalyzerHeaderRequest { destination: destination })?;
         let (sent, total, overflow) = match reply {
             drtioaux::Packet::AnalyzerHeader { sent_bytes, total_byte_count, overflow_occurred } => 
@@ -506,7 +542,7 @@ pub mod drtio {
         if sent > 0 {
             let mut last_packet = false;
             while !last_packet {
-                let reply = aux_transact(io, aux_mutex, linkno, 
+                let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                     &drtioaux::Packet::AnalyzerDataRequest { destination: destination })?;
                 match reply {
                     drtioaux::Packet::AnalyzerData { last, length, data } => { 
@@ -527,23 +563,23 @@ pub mod drtio {
     }
 
     #[cfg(has_rtio_analyzer)]
-    pub fn analyzer_query(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-        up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
+    pub fn analyzer_query(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
     ) -> Result<Vec<RemoteBuffer>, Error> {
         let mut remote_buffers: Vec<RemoteBuffer> = Vec::new();
         for i in 1..drtio_routing::DEST_COUNT {
             if destination_up(up_destinations, i as u8) {
-                remote_buffers.push(analyzer_get_data(io, aux_mutex, routing_table, i as u8)?);
+                remote_buffers.push(analyzer_get_data(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, i as u8)?);
             }
         }
         Ok(remote_buffers)
     }
 
-    pub fn subkernel_upload(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, data: &Vec<u8>) -> Result<(), Error> {
+    pub fn subkernel_upload(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, data: &Vec<u8>) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(data, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelAddDataRequest {
                     id: id, destination: destination, status: status, length: len as u16, data: *slice})?;
             match reply {
@@ -555,26 +591,27 @@ pub mod drtio {
         })
     }
 
-    pub fn subkernel_load(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, run: bool) -> Result<(), Error> {
+    pub fn subkernel_load(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, run: bool
+        ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::SubkernelLoadRunRequest{ id: id, destination: destination, run: run })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::SubkernelLoadRunRequest{ id: id, source: 0, destination: destination, run: run })?;
         match reply {
-            drtioaux::Packet::SubkernelLoadRunReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::SubkernelLoadRunReply { succeeded: false } =>
+            drtioaux::Packet::SubkernelLoadRunReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::SubkernelLoadRunReply { destination: 0, succeeded: false } =>
                     Err(Error::SubkernelRunFail(destination)),
                 packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
-    pub fn subkernel_retrieve_exception(io: &Io, aux_mutex: &Mutex,
+    pub fn subkernel_retrieve_exception(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &drtio_routing::RoutingTable, destination: u8
     ) -> Result<Vec<u8>, Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         let mut remote_data: Vec<u8> = Vec::new();
         loop {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelExceptionRequest { destination: destination })?;
             match reply {
                 drtioaux::Packet::SubkernelException { last, length, data } => { 
@@ -588,21 +625,21 @@ pub mod drtio {
         }
     }
 
-    pub fn subkernel_send_message(io: &Io, aux_mutex: &Mutex,
+    pub fn subkernel_send_message(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, message: &[u8]
     ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(message, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelMessage {
-                    destination: destination, id: id, status: status, length: len as u16, data: *slice})?;
+                    source: 0, destination: destination,
+                    id: id, status: status, length: len as u16, data: *slice})?;
             match reply {
                 drtioaux::Packet::SubkernelMessageAck { .. } => Ok(()),
                 packet => Err(Error::UnexpectedPacket(packet)),
             }
         })
     }
-
 }
 
 #[cfg(not(has_drtio))]
@@ -613,7 +650,7 @@ pub mod drtio {
         _routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>,
         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
         _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex) {}
-    pub fn reset(_io: &Io, _aux_mutex: &Mutex) {}
+    pub fn reset(_io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, _routing_table: &drtio_routing::RoutingTable) {}
 }
 
 static mut SEEN_ASYNC_ERRORS: u8 = 0;
@@ -673,11 +710,30 @@ fn read_device_map() -> DeviceMap {
     device_map
 }
 
+fn toggle_sed_spread(val: u8) {
+    unsafe { csr::rtio_core::sed_spread_enable_write(val); }
+}
+
+fn setup_sed_spread() {
+    config::read_str("sed_spread_enable", |r| {
+        match r {
+            Ok("1") => { info!("SED spreading enabled"); toggle_sed_spread(1); },
+            Ok("0") => { info!("SED spreading disabled"); toggle_sed_spread(0); },
+            Ok(_) => { 
+                warn!("sed_spread_enable value not supported (only 1, 0 allowed), disabling by default");
+                toggle_sed_spread(0);
+            },
+            Err(_) => { info!("SED spreading disabled by default"); toggle_sed_spread(0) },
+        }
+    });
+}
+
 pub fn startup(io: &Io, aux_mutex: &Mutex,
         routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>,
         up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
         ddma_mutex: &Mutex, subkernel_mutex: &Mutex) {
     set_device_map(read_device_map());
+    setup_sed_spread();
     drtio::startup(io, aux_mutex, routing_table, up_destinations, ddma_mutex, subkernel_mutex);
     unsafe {
         csr::rtio_core::reset_phy_write(1);
@@ -685,9 +741,10 @@ pub fn startup(io: &Io, aux_mutex: &Mutex,
     io.spawn(4096, async_error_thread);
 }
 
-pub fn reset(io: &Io, aux_mutex: &Mutex) {
+pub fn reset(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+    routing_table: &drtio_routing::RoutingTable) {
     unsafe {
         csr::rtio_core::reset_write(1);
     }
-    drtio::reset(io, aux_mutex)
+    drtio::reset(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table)
 }

artiq/firmware/runtime/session.rs

@@ -243,12 +243,13 @@ pub fn kern_send(io: &Io, request: &kern::Message) -> Result<(), Error<SchedErro
 
 fn kern_recv_notrace<R, F>(io: &Io, f: F) -> Result<R, Error<SchedError>>
         where F: FnOnce(&kern::Message) -> Result<R, Error<SchedError>> {
-    io.until(|| mailbox::receive() != 0)?;
-    if !kernel::validate(mailbox::receive()) {
-        return Err(Error::InvalidPointer(mailbox::receive()))
+    let mut msg_ptr = 0;
+    io.until(|| { msg_ptr = mailbox::receive(); msg_ptr != 0 })?;
+    if !kernel::validate(msg_ptr) {
+        return Err(Error::InvalidPointer(msg_ptr))
     }
 
-    f(unsafe { &*(mailbox::receive() as *const kern::Message) })
+    f(unsafe { &*(msg_ptr as *const kern::Message) })
 }
 
 fn kern_recv_dotrace(reply: &kern::Message) {
@@ -316,7 +317,7 @@ fn kern_run(session: &mut Session) -> Result<(), Error<SchedError>> {
 }
 
 
-fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex, 
+fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex, _ddma_mutex: &Mutex,
                         _routing_table: &drtio_routing::RoutingTable,
                         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
                         session: &mut Session, kernel: &[u8]
@@ -355,7 +356,7 @@ fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex,
                     if up {
                         let subkernel_lib = entry.data().to_vec();
                         subkernel::add_subkernel(io, _subkernel_mutex, sid, dest, subkernel_lib)?;
-                        subkernel::upload(io, _aux_mutex, _subkernel_mutex, _routing_table, sid)?;
+                        subkernel::upload(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, sid)?;
                     } else {
                         return Err(Error::DestinationDown);
                     }
@@ -468,9 +469,10 @@ fn process_host_message(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subke
             #[cfg(has_drtio)]
             {
                 subkernel::add_subkernel(io, _subkernel_mutex, _id, _dest, _kernel)?;
-                match subkernel::upload(io, _aux_mutex, _subkernel_mutex, _routing_table, _id) {
+                match subkernel::upload(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, _id) {
                     Ok(_) => host_write(stream, host::Reply::LoadCompleted)?,
                     Err(error) => {
+                        subkernel::clear_subkernels(io, _subkernel_mutex)?;
                         let mut description = String::new();
                         write!(&mut description, "{}", error).unwrap();
                         host_write(stream, host::Reply::LoadFailed(&description))?
@@ -488,7 +490,7 @@ fn process_host_message(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subke
 fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                         routing_table: &drtio_routing::RoutingTable,
                         up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
-                        ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, mut stream: Option<&mut TcpStream>,
+                        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, mut stream: Option<&mut TcpStream>,
                         session: &mut Session) -> Result<bool, Error<SchedError>> {
     kern_recv_notrace(io, |request| {
         match (request, session.kernel_state) {
@@ -506,7 +508,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
 
         kern_recv_dotrace(request);
 
-        if kern_hwreq::process_kern_hwreq(io, aux_mutex, routing_table, up_destinations, request)? {
+        if kern_hwreq::process_kern_hwreq(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, up_destinations, request)? {
             return Ok(false)
         }
 
@@ -530,7 +532,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 if let Some(_id) = session.congress.dma_manager.record_start(name) {
                     // replace the record
                     #[cfg(has_drtio)]
-                    remote_dma::erase(io, aux_mutex, ddma_mutex, routing_table, _id)?;
+                    remote_dma::erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id)?;
                 }
                 kern_acknowledge()
             }
@@ -542,7 +544,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 let _id = session.congress.dma_manager.record_stop(duration, enable_ddma, io, ddma_mutex)?;
                 #[cfg(has_drtio)]
                 if enable_ddma {
-                    remote_dma::upload_traces(io, aux_mutex, ddma_mutex, routing_table, _id)?;
+                    remote_dma::upload_traces(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id)?;
                 }
                 cache::flush_l2_cache();
                 kern_acknowledge()
@@ -550,7 +552,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             &kern::DmaEraseRequest { name } => {
                 #[cfg(has_drtio)]
                 if let Some(id) = session.congress.dma_manager.get_id(name) {
-                    remote_dma::erase(io, aux_mutex, ddma_mutex, routing_table, *id)?;
+                    remote_dma::erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id)?;
                 }
                 session.congress.dma_manager.erase(name);
                 kern_acknowledge()
@@ -573,7 +575,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             }
             &kern::DmaStartRemoteRequest { id: _id, timestamp: _timestamp } => {
                 #[cfg(has_drtio)]
-                remote_dma::playback(io, aux_mutex, ddma_mutex, routing_table, _id as u32, _timestamp as u64)?;
+                remote_dma::playback(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id as u32, _timestamp as u64)?;
                 kern_acknowledge()
             }
             &kern::DmaAwaitRemoteRequest { id: _id } => {
@@ -631,6 +633,8 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 unsafe { kernel::stop() }
                 session.kernel_state = KernelState::Absent;
                 unsafe { session.congress.cache.unborrow() }
+                #[cfg(has_drtio)]
+                subkernel::clear_subkernels(io, subkernel_mutex)?;
 
                 match stream {
                     None => return Ok(true),
@@ -648,6 +652,8 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 unsafe { kernel::stop() }
                 session.kernel_state = KernelState::Absent;
                 unsafe { session.congress.cache.unborrow() }
+                #[cfg(has_drtio)]
+                subkernel::clear_subkernels(io, subkernel_mutex)?;
 
                 match stream {
                     None => {
@@ -668,9 +674,9 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 }
             }
             #[cfg(has_drtio)]
-            &kern::SubkernelLoadRunRequest { id, run } => {
+            &kern::SubkernelLoadRunRequest { id, destination: _, run } => {
                 let succeeded = match subkernel::load(
-                    io, aux_mutex, _subkernel_mutex, routing_table, id, run) {
+                    io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, run) {
                         Ok(()) => true,
                         Err(e) => { error!("Error loading subkernel: {}", e); false }
                     };
@@ -678,7 +684,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             }
             #[cfg(has_drtio)]
             &kern::SubkernelAwaitFinishRequest{ id, timeout } => {
-                let res = subkernel::await_finish(io, aux_mutex, _subkernel_mutex, routing_table,
+                let res = subkernel::await_finish(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table,
                     id, timeout);
                 let status = match res {
                     Ok(ref res) => {
@@ -699,20 +705,20 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 kern_send(io, &kern::SubkernelAwaitFinishReply { status: status })
             }
             #[cfg(has_drtio)]
-            &kern::SubkernelMsgSend { id, count, tag, data } => {
-                subkernel::message_send(io, aux_mutex, _subkernel_mutex, routing_table, id, count, tag, data)?;
+            &kern::SubkernelMsgSend { id, destination, count, tag, data } => {
+                subkernel::message_send(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, destination, count, tag, data)?;
                 kern_acknowledge()
             }
             #[cfg(has_drtio)]
             &kern::SubkernelMsgRecvRequest { id, timeout, tags } => {
-                let message_received = subkernel::message_await(io, _subkernel_mutex, id, timeout);
+                let message_received = subkernel::message_await(io, subkernel_mutex, id as u32, timeout);
                 let (status, count) = match message_received {
                     Ok(ref message) => (kern::SubkernelStatus::NoError, message.count),
                     Err(SubkernelError::Timeout) => (kern::SubkernelStatus::Timeout, 0),
                     Err(SubkernelError::IncorrectState) => (kern::SubkernelStatus::IncorrectState, 0),
                     Err(SubkernelError::SubkernelFinished) => {
-                        let res = subkernel::retrieve_finish_status(io, aux_mutex, _subkernel_mutex,
-                            routing_table, id)?;
+                        let res = subkernel::retrieve_finish_status(io, aux_mutex, ddma_mutex, subkernel_mutex,
+                            routing_table, id as u32)?;
                         if res.comm_lost {
                             (kern::SubkernelStatus::CommLost, 0)
                         } else if let Some(exception) = &res.exception {
@@ -842,7 +848,7 @@ fn flash_kernel_worker(io: &Io, aux_mutex: &Mutex,
         match result {
             Ok(kernel) => {
                 // process .ELF or .TAR kernels
-                let res = process_flash_kernel(io, aux_mutex, subkernel_mutex, routing_table, up_destinations, &mut session, kernel);
+                let res = process_flash_kernel(io, aux_mutex, subkernel_mutex, ddma_mutex, routing_table, up_destinations, &mut session, kernel);
                 #[cfg(has_drtio)]
                 match res {
                     // wait to establish the DRTIO connection
@@ -887,7 +893,7 @@ fn respawn<F>(io: &Io, handle: &mut Option<ThreadHandle>, f: F)
         }
     }
 
-    *handle = Some(io.spawn(24576, f))
+    *handle = Some(io.spawn(32768, f))
 }
 
 pub fn thread(io: Io, aux_mutex: &Mutex,
@@ -971,7 +977,13 @@ pub fn thread(io: Io, aux_mutex: &Mutex,
                         drtio::clear_buffers(&io, &aux_mutex);
                     }
                 }
-                stream.close().expect("session: close socket");
+                loop {
+                    match stream.close() {
+                        Ok(_) => break,
+                        Err(SchedError::Interrupted) => (),
+                        Err(e) => panic!("session: close socket: {:?}", e)
+                    };
+                }
             });
         }
 

artiq/firmware/satman/Makefile

@@ -10,8 +10,6 @@ LDFLAGS += \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: satman.bin satman.fbi
 
 .PHONY: $(RUSTOUT)/libsatman.a

artiq/firmware/satman/dma.rs

@@ -1,41 +1,190 @@
+use alloc::{vec::Vec, collections::btree_map::BTreeMap, string::String};
+use core::mem;
+use board_artiq::{drtioaux, drtio_routing::RoutingTable};
 use board_misoc::{csr, cache::flush_l2_cache};
 use proto_artiq::drtioaux_proto::PayloadStatus;
-use alloc::{vec::Vec, collections::btree_map::BTreeMap};
-use ::{cricon_select, RtioMaster};
+use routing::{Router, Sliceable};
+use kernel::Manager as KernelManager;
+use ::{cricon_select, cricon_read, RtioMaster, MASTER_PAYLOAD_MAX_SIZE};
 
 const ALIGNMENT: usize = 64;
 
-#[derive(Debug, PartialEq)]
+#[derive(PartialEq)]
 enum ManagerState {
     Idle,
     Playback
 }
 
 pub struct RtioStatus {
+    pub source: u8,
     pub id: u32, 
     pub error: u8,
     pub channel: u32, 
     pub timestamp: u64
 }
 
+#[derive(Debug)]
 pub enum Error {
     IdNotFound,
     PlaybackInProgress,
-    EntryNotComplete
+    EntryNotComplete,
+    MasterDmaFound,
+    UploadFail,
 }
 
-#[derive(Debug)]
 struct Entry {
     trace: Vec<u8>,
     padding_len: usize,
-    complete: bool
+    complete: bool,
+    duration: u64, // relevant for locally ran DMA
+}
+
+impl Entry {
+    pub fn from_vec(data: Vec<u8>, duration: u64) -> Entry {
+        let mut entry = Entry {
+            trace: data,
+            padding_len: 0,
+            complete: true,
+            duration: duration,
+        };
+        entry.realign();
+        entry
+    }
+
+    pub fn id(&self) -> u32 {
+        self.trace[self.padding_len..].as_ptr() as u32
+    }
+
+    pub fn realign(&mut self) {
+        self.trace.push(0);
+        let data_len = self.trace.len();
+
+        self.trace.reserve(ALIGNMENT - 1);
+        let padding = ALIGNMENT - self.trace.as_ptr() as usize % ALIGNMENT;
+        let padding = if padding == ALIGNMENT { 0 } else { padding };
+        for _ in 0..padding {
+            // Vec guarantees that this will not reallocate
+            self.trace.push(0)
+        }
+        for i in 1..data_len + 1 {
+            self.trace[data_len + padding - i] = self.trace[data_len - i]
+        }
+        self.complete = true;
+        self.padding_len = padding;
+    }
+}
+
+enum RemoteTraceState {
+    Unsent,
+    Sending(usize),
+    Ready,
+    Running(usize),
+}
+
+struct RemoteTraces {
+    remote_traces: BTreeMap<u8, Sliceable>,
+    state: RemoteTraceState,
+}
+
+impl RemoteTraces {
+    pub fn new(traces: BTreeMap<u8, Sliceable>) -> RemoteTraces {
+        RemoteTraces {
+            remote_traces: traces,
+            state: RemoteTraceState::Unsent
+        }
+    }
+
+    // on subkernel request
+    pub fn upload_traces(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) -> usize {
+        let len = self.remote_traces.len();
+        if len > 0 {
+            self.state = RemoteTraceState::Sending(self.remote_traces.len());
+            for (dest, trace) in self.remote_traces.iter_mut() {
+                // queue up the first packet for all destinations, rest will be sent after first ACK
+                let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                let meta = trace.get_slice_master(&mut data_slice);
+                router.route(drtioaux::Packet::DmaAddTraceRequest {
+                    source: self_destination, destination: *dest, id: id,
+                    status: meta.status, length: meta.len, trace: data_slice
+                }, routing_table, rank, self_destination);
+            }
+        }
+        len
+    }
+
+    // on incoming Packet::DmaAddTraceReply
+    pub fn ack_upload(&mut self, kernel_manager: &mut KernelManager, source: u8, id: u32, succeeded: bool, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        if let RemoteTraceState::Sending(count) = self.state {
+            if let Some(trace) = self.remote_traces.get_mut(&source) {
+                if trace.at_end() {
+                        if count - 1 == 0 {
+                            self.state = RemoteTraceState::Ready;
+                            kernel_manager.ddma_remote_uploaded(succeeded);
+                        } else {
+                            self.state = RemoteTraceState::Sending(count - 1);
+                        }
+                } else {
+                    // send next slice
+                    let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                    let meta = trace.get_slice_master(&mut data_slice);
+                    router.route(drtioaux::Packet::DmaAddTraceRequest {
+                        source: self_destination, destination: meta.destination, id: id,
+                        status: meta.status, length: meta.len, trace: data_slice
+                    }, routing_table, rank, self_destination);
+                }
+            }
+        }
+        
+    }
+
+    // on subkernel request
+    pub fn playback(&mut self, id: u32, timestamp: u64, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        // route all the playback requests
+        // remote traces + local trace
+        self.state = RemoteTraceState::Running(self.remote_traces.len() + 1);
+        for (dest, _) in self.remote_traces.iter() {
+            router.route(drtioaux::Packet::DmaPlaybackRequest {
+                source: self_destination, destination: *dest, id: id, timestamp: timestamp
+            }, routing_table, rank, self_destination);
+            // response will be ignored (succeeded = false handled by the main thread)
+        }
+    }
+
+    // on incoming Packet::DmaPlaybackDone
+    pub fn remote_finished(&mut self, kernel_manager: &mut KernelManager, error: u8, channel: u32, timestamp: u64) {
+        if let RemoteTraceState::Running(count) = self.state {
+            if error != 0 || count - 1 == 0 {
+                // notify the kernel about a DDMA error or finish
+                kernel_manager.ddma_finished(error, channel, timestamp);
+                self.state = RemoteTraceState::Ready;
+                // further messages will be ignored (if there was an error)
+            } else { // no error and not the last one awaited
+                self.state = RemoteTraceState::Running(count - 1);
+            }
+        }
+    }
+
+    pub fn erase(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        for (dest, _) in self.remote_traces.iter() {
+            router.route(drtioaux::Packet::DmaRemoveTraceRequest { 
+                source: self_destination, destination: *dest, id: id
+            }, routing_table, rank, self_destination);
+            // response will be ignored as this object will stop existing too
+        }
+    }
 }
 
-#[derive(Debug)]
 pub struct Manager {
-    entries: BTreeMap<u32, Entry>,
+    entries: BTreeMap<(u8, u32), Entry>,
     state: ManagerState,
-    currentid: u32
+    current_id: u32,
+    current_source: u8,
+    previous_cri_master: RtioMaster,
+
+    remote_entries: BTreeMap<u32, RemoteTraces>,
+    name_map: BTreeMap<String, u32>,
+    recording_trace: Vec<u8>,
+    recording_name: String
 }
 
 impl Manager {
@@ -47,74 +196,197 @@ impl Manager {
         }
         Manager {
             entries: BTreeMap::new(),
-            currentid: 0,
+            current_id: 0,
+            current_source: 0,
+            previous_cri_master: RtioMaster::Drtio,
             state: ManagerState::Idle,
+            remote_entries: BTreeMap::new(),
+            name_map: BTreeMap::new(),
+            recording_trace: Vec::new(),
+            recording_name: String::new(),
         }
     }
 
-    pub fn add(&mut self, id: u32, status: PayloadStatus, trace: &[u8], trace_len: usize) -> Result<(), Error> {
+    pub fn add(&mut self, source: u8, id: u32, status: PayloadStatus, trace: &[u8], trace_len: usize) -> Result<(), Error> {
         if status.is_first() {
-            self.entries.remove(&id);
+            self.entries.remove(&(source, id));
         }
-        let entry = match self.entries.get_mut(&id) {
+        let entry = match self.entries.get_mut(&(source, id)) {
             Some(entry) => {
                 if entry.complete {
                     // replace entry
-                    self.entries.remove(&id);
-                    self.entries.insert(id, Entry {
+                    self.entries.remove(&(source, id));
+                    self.entries.insert((source, id), Entry {
                         trace: Vec::new(),
                         padding_len: 0,
-                        complete: false });
-                    self.entries.get_mut(&id).unwrap()
+                        complete: false,
+                        duration: 0
+                    });
+                    self.entries.get_mut(&(source, id)).unwrap()
                 } else {
                     entry
                 }
             },
             None => {
-                self.entries.insert(id, Entry {
+                self.entries.insert((source, id), Entry {
                     trace: Vec::new(),
                     padding_len: 0,
-                    complete: false });
-                self.entries.get_mut(&id).unwrap()
+                    complete: false,
+                    duration: 0,
+                });
+                self.entries.get_mut(&(source, id)).unwrap()
             },
         };
         entry.trace.extend(&trace[0..trace_len]);
 
         if status.is_last() {
-            entry.trace.push(0);
-            let data_len = entry.trace.len();
-    
-            // Realign.
-            entry.trace.reserve(ALIGNMENT - 1);
-            let padding = ALIGNMENT - entry.trace.as_ptr() as usize % ALIGNMENT;
-            let padding = if padding == ALIGNMENT { 0 } else { padding };
-            for _ in 0..padding {
-                // Vec guarantees that this will not reallocate
-                entry.trace.push(0)
-            }
-            for i in 1..data_len + 1 {
-                entry.trace[data_len + padding - i] = entry.trace[data_len - i]
-            }
-            entry.complete = true;
-            entry.padding_len = padding;
+            entry.realign();
             flush_l2_cache();
         }
         Ok(())
     }
+    // API for subkernel
+    pub fn record_start(&mut self, name: &str) {
+        self.recording_name = String::from(name);
+        self.recording_trace = Vec::new();
+    }
 
-    pub fn erase(&mut self, id: u32) -> Result<(), Error> {
-        match self.entries.remove(&id) {
+    // API for subkernel
+    pub fn record_append(&mut self, data: &[u8]) {
+        self.recording_trace.extend_from_slice(data);
+    }
+
+    // API for subkernel
+    pub fn record_stop(&mut self, duration: u64, self_destination: u8) -> Result<u32, Error> {
+        let mut trace = Vec::new();
+        mem::swap(&mut self.recording_trace, &mut trace);
+        trace.push(0);
+        let mut local_trace = Vec::new();
+        let mut remote_traces: BTreeMap<u8, Sliceable> = BTreeMap::new();
+        // analyze each entry and put in proper buckets, as the kernel core
+        // sends whole chunks, to limit comms/kernel CPU communication,
+        // and as only comms core has access to varios DMA buffers.
+        let mut ptr = 0;
+        while trace[ptr] != 0 {
+            // ptr + 3 = tgt >> 24 (destination)
+            let len = trace[ptr] as usize;
+            let destination = trace[ptr+3];
+            if destination == 0 {
+                return Err(Error::MasterDmaFound);
+            } else if destination == self_destination {
+                local_trace.extend(&trace[ptr..ptr+len]);
+            }
+            else {
+                if let Some(remote_trace) = remote_traces.get_mut(&destination) {
+                    remote_trace.extend(&trace[ptr..ptr+len]);
+                } else {
+                    remote_traces.insert(destination, Sliceable::new(destination, trace[ptr..ptr+len].to_vec()));
+                }
+            }
+            // and jump to the next event
+            ptr += len;
+        }
+        let local_entry = Entry::from_vec(local_trace, duration);
+        let id = local_entry.id();
+
+        self.entries.insert((self_destination, id), local_entry);
+        self.remote_entries.insert(id, RemoteTraces::new(remote_traces));
+        let mut name = String::new();
+        mem::swap(&mut self.recording_name, &mut name);
+        self.name_map.insert(name, id);
+
+        flush_l2_cache();
+
+        Ok(id)
+    }
+
+    pub fn upload_traces(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8,
+            routing_table: &RoutingTable) -> Result<usize, Error> {
+        let remote_traces = self.remote_entries.get_mut(&id);
+        let mut len = 0;
+        if let Some(traces) = remote_traces {
+            len = traces.upload_traces(id, router, rank, self_destination, routing_table);
+        }
+        Ok(len)
+    }
+
+    pub fn with_trace<F, R>(&self, self_destination: u8, name: &str, f: F) -> R
+        where F: FnOnce(Option<&[u8]>, u64) -> R {
+        if let Some(ptr) = self.name_map.get(name) {
+            match self.entries.get(&(self_destination, *ptr)) {
+                Some(entry) => f(Some(&entry.trace[entry.padding_len..]), entry.duration),
+                None => f(None, 0)
+            }
+        } else {
+            f(None, 0)
+        }
+    }
+
+    // API for subkernel
+    pub fn playback_remote(&mut self, id: u32, timestamp: u64, 
+        router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable
+    ) -> Result<(), Error> {
+        if let Some(traces) = self.remote_entries.get_mut(&id) {
+            traces.playback(id, timestamp, router, rank, self_destination, routing_table);
+            Ok(())
+        } else {
+            Err(Error::IdNotFound)
+        }
+    }
+
+    // API for subkernel
+    pub fn erase_name(&mut self, name: &str, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        if let Some(id) = self.name_map.get(name) {
+            if let Some(traces) = self.remote_entries.get_mut(&id) {
+                traces.erase(*id, router, rank, self_destination, routing_table);
+                self.remote_entries.remove(&id);
+            }
+            self.entries.remove(&(self_destination, *id));
+            self.name_map.remove(name);
+        }
+    }
+
+    // API for incoming DDMA (drtio)
+    pub fn erase(&mut self, source: u8, id: u32) -> Result<(), Error> {
+        match self.entries.remove(&(source, id)) {
             Some(_) => Ok(()),
             None => Err(Error::IdNotFound)
         }
     }
 
-    pub fn playback(&mut self, id: u32, timestamp: u64) -> Result<(), Error> {
+    pub fn remote_finished(&mut self, kernel_manager: &mut KernelManager, 
+        id: u32, error: u8, channel: u32, timestamp: u64) {
+        if let Some(entry) = self.remote_entries.get_mut(&id) {
+            entry.remote_finished(kernel_manager, error, channel, timestamp);
+        }
+    }
+
+    pub fn ack_upload(&mut self, kernel_manager: &mut KernelManager, source: u8, id: u32, succeeded: bool, 
+        router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+            if let Some(entry) = self.remote_entries.get_mut(&id) {
+                entry.ack_upload(kernel_manager, source, id, succeeded, router, rank, self_destination, routing_table);
+            }
+    }
+
+    pub fn cleanup(&mut self, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        // after subkernel ends, remove all self-generated traces
+        for (_, id) in self.name_map.iter_mut() {
+            if let Some(traces) = self.remote_entries.get_mut(&id) {
+                traces.erase(*id, router, rank, self_destination, routing_table);
+                self.remote_entries.remove(&id);
+            }
+            self.entries.remove(&(self_destination, *id));
+        }
+        self.name_map.clear();
+    }
+
+    // API for both incoming DDMA (drtio) and subkernel
+    pub fn playback(&mut self, source: u8, id: u32, timestamp: u64) -> Result<(), Error> {
         if self.state != ManagerState::Idle {
             return Err(Error::PlaybackInProgress);
         }
 
-        let entry = match self.entries.get(&id){
+        let entry = match self.entries.get(&(source, id)){
             Some(entry) => entry,
             None => { return Err(Error::IdNotFound); }
         };
@@ -125,7 +397,9 @@ impl Manager {
         assert!(ptr as u32 % 64 == 0);
 
         self.state = ManagerState::Playback;
-        self.currentid = id;
+        self.current_id = id;
+        self.current_source = source;
+        self.previous_cri_master = cricon_read();
 
         unsafe {
             csr::rtio_dma::base_address_write(ptr as u64);
@@ -149,7 +423,7 @@ impl Manager {
         } else {
             self.state = ManagerState::Idle;
             unsafe { 
-                cricon_select(RtioMaster::Drtio);
+                cricon_select(self.previous_cri_master);
                 let error = csr::rtio_dma::error_read();
                 let channel = csr::rtio_dma::error_channel_read();
                 let timestamp = csr::rtio_dma::error_timestamp_read();
@@ -157,7 +431,8 @@ impl Manager {
                     csr::rtio_dma::error_write(1);
                 }
                 return Some(RtioStatus {
-                    id: self.currentid, 
+                    source: self.current_source,
+                    id: self.current_id,
                     error: error,
                     channel: channel, 
                     timestamp: timestamp });

artiq/firmware/satman/kernel.rs

@@ -1,8 +1,8 @@
-use core::{mem, option::NoneError, cmp::min};
-use alloc::{string::String, format, vec::Vec, collections::{btree_map::BTreeMap, vec_deque::VecDeque}};
+use core::mem;
+use alloc::{string::String, format, vec::Vec, collections::btree_map::BTreeMap};
 use cslice::AsCSlice;
 
-use board_artiq::{mailbox, spi};
+use board_artiq::{drtioaux, drtio_routing::RoutingTable, mailbox, spi};
 use board_misoc::{csr, clock, i2c};
 use proto_artiq::{
     drtioaux_proto::PayloadStatus,
@@ -15,6 +15,8 @@ use kernel::eh_artiq::StackPointerBacktrace;
 
 use ::{cricon_select, RtioMaster};
 use cache::Cache;
+use dma::{Manager as DmaManager, Error as DmaError};
+use routing::{Router, Sliceable, SliceMeta};
 use SAT_PAYLOAD_MAX_SIZE;
 use MASTER_PAYLOAD_MAX_SIZE;
 
@@ -61,8 +63,12 @@ enum KernelState {
     Absent,
     Loaded,
     Running,
-    MsgAwait { max_time: u64, tags: Vec<u8> },
-    MsgSending
+    MsgAwait { id: u32, max_time: i64, tags: Vec<u8> },
+    MsgSending,
+    SubkernelAwaitLoad,
+    SubkernelAwaitFinish { max_time: i64, id: u32 },
+    DmaUploading { max_time: u64 },
+    DmaAwait { max_time: u64 },
 }
 
 #[derive(Debug)]
@@ -74,13 +80,9 @@ pub enum Error {
     NoMessage,
     AwaitingMessage,
     SubkernelIoError,
-    KernelException(Sliceable)
-}
-
-impl From<NoneError> for Error {
-    fn from(_: NoneError) -> Error {
-        Error::KernelNotFound
-    }
+    DrtioError,
+    KernelException(Sliceable),
+    DmaError(DmaError),
 }
 
 impl From<io::Error<!>> for Error {
@@ -89,19 +91,25 @@ impl From<io::Error<!>> for Error {
     }
 }
 
+impl From<drtioaux::Error<!>> for Error {
+    fn from(_value: drtioaux::Error<!>) -> Error {
+        Error::DrtioError
+    }
+}
+
+impl From<DmaError> for Error {
+    fn from(value: DmaError) -> Error {
+        Error::DmaError(value)
+    }
+}
+
 macro_rules! unexpected {
     ($($arg:tt)*) => (return Err(Error::Unexpected(format!($($arg)*))));
 }
 
-/* represents data that has to be sent to Master */
-#[derive(Debug)]
-pub struct Sliceable {
-    it: usize,
-    data: Vec<u8>
-}
-
 /* represents interkernel messages */
 struct Message {
+    id: u32,
     count: u8,
     data: Vec<u8>
 }
@@ -109,7 +117,6 @@ struct Message {
 #[derive(PartialEq)]
 enum OutMessageState {
     NoMessage,
-    MessageReady,
     MessageBeingSent,
     MessageSent,
     MessageAcknowledged
@@ -119,7 +126,7 @@ enum OutMessageState {
 struct MessageManager {
     out_message: Option<Sliceable>,
     out_state: OutMessageState,
-    in_queue: VecDeque<Message>,
+    in_queue: Vec<Message>,
     in_buffer: Option<Message>,
 }
 
@@ -128,7 +135,9 @@ struct Session {
     kernel_state: KernelState,
     log_buffer: String,
     last_exception: Option<Sliceable>,
-    messages: MessageManager
+    source: u8, // which destination requested running the kernel
+    messages: MessageManager,
+    subkernels_finished: Vec<u32> // ids of subkernels finished
 }
 
 #[derive(Debug)]
@@ -147,42 +156,9 @@ pub struct Manager {
 
 pub struct SubkernelFinished {
     pub id: u32,
-    pub with_exception: bool
-}
-
-pub struct SliceMeta {
-    pub len: u16,
-    pub status: PayloadStatus
-}
-
-macro_rules! get_slice_fn {
-    ( $name:tt, $size:expr ) => {
-        pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
-            let first = self.it == 0;
-            let len = min($size, self.data.len() - self.it);
-            let last = self.it + len == self.data.len();
-            let status = PayloadStatus::from_status(first, last);
-            data_slice[..len].clone_from_slice(&self.data[self.it..self.it+len]);
-            self.it += len;
-    
-            SliceMeta {
-                len: len as u16,
-                status: status
-            }
-        }
-    };
-}
-
-impl Sliceable {
-    pub fn new(data: Vec<u8>) -> Sliceable {
-        Sliceable {
-            it: 0,
-            data: data
-        }
-    }
-
-    get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
-    get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
+    pub with_exception: bool,
+    pub exception_source: u8,
+    pub source: u8
 }
 
 impl MessageManager {
@@ -190,12 +166,12 @@ impl MessageManager {
         MessageManager {
             out_message: None,
             out_state: OutMessageState::NoMessage,
-            in_queue: VecDeque::new(),
+            in_queue: Vec::new(),
             in_buffer: None
         }
     }
 
-    pub fn handle_incoming(&mut self, status: PayloadStatus, length: usize, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
+    pub fn handle_incoming(&mut self, status: PayloadStatus, length: usize, id: u32, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
         // called when receiving a message from master
         if status.is_first() {
             // clear the buffer for first message
@@ -205,6 +181,7 @@ impl MessageManager {
             Some(message) => message.data.extend(&data[..length]),
             None => {
                 self.in_buffer = Some(Message {
+                    id: id,
                     count: data[0],
                     data: data[1..length].to_vec()
                 });
@@ -212,18 +189,7 @@ impl MessageManager {
         };
         if status.is_last() {
             // when done, remove from working queue
-            self.in_queue.push_back(self.in_buffer.take().unwrap());
-        }
-    }
-
-    pub fn is_outgoing_ready(&mut self) -> bool {
-        // called by main loop, to see if there's anything to send, will send it afterwards
-        match self.out_state {
-            OutMessageState::MessageReady => {
-                self.out_state = OutMessageState::MessageBeingSent;
-                true
-            },
-            _ => false
+            self.in_queue.push(self.in_buffer.take().unwrap());
         }
     }
 
@@ -266,19 +232,42 @@ impl MessageManager {
         }
     }
 
-    pub fn accept_outgoing(&mut self, count: u8, tag: &[u8], data: *const *const ()) -> Result<(), Error>  {
+    pub fn accept_outgoing(&mut self, id: u32, self_destination: u8, destination: u8, 
+        count: u8, tag: &[u8], data: *const *const (), 
+        routing_table: &RoutingTable, rank: u8, router: &mut Router
+    ) -> Result<(), Error>  {
         let mut writer = Cursor::new(Vec::new());
         rpc::send_args(&mut writer, 0, tag, data, false)?;
         // skip service tag, but write the count
         let mut data = writer.into_inner().split_off(3);
         data[0] = count;
-        self.out_message = Some(Sliceable::new(data));
-        self.out_state = OutMessageState::MessageReady;
+        self.out_message = Some(Sliceable::new(destination, data));
+
+        let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+        self.out_state = OutMessageState::MessageBeingSent;
+        let meta = self.get_outgoing_slice(&mut data_slice).unwrap();
+        router.route(drtioaux::Packet::SubkernelMessage {
+                source: self_destination, destination: destination, id: id,
+                status: meta.status, length: meta.len as u16, data: data_slice
+        }, routing_table, rank, self_destination);
         Ok(())
     }
 
-    pub fn get_incoming(&mut self) -> Option<Message> {
-        self.in_queue.pop_front()
+    pub fn get_incoming(&mut self, id: u32) -> Option<Message> {
+        for i in 0..self.in_queue.len() {
+            if self.in_queue[i].id == id {
+                return Some(self.in_queue.remove(i));
+            }
+        }
+        None
+    }
+
+    pub fn pending_ids(&self) -> Vec<u32> {
+        let mut pending_ids: Vec<u32> = Vec::new();
+        for msg in self.in_queue.iter() {
+            pending_ids.push(msg.id);
+        }
+        pending_ids
     }
 }
 
@@ -288,15 +277,16 @@ impl Session {
             kernel_state: KernelState::Absent,
             log_buffer: String::new(),
             last_exception: None,
-            messages: MessageManager::new()
+            source: 0,
+            messages: MessageManager::new(),
+            subkernels_finished: Vec::new()
         }
     }
 
     fn running(&self) -> bool {
         match self.kernel_state {
             KernelState::Absent | KernelState::Loaded  => false,
-            KernelState::Running | KernelState::MsgAwait { .. } |
-                KernelState::MsgSending => true
+            _ => true
         }
     }
 
@@ -334,7 +324,7 @@ impl Manager {
                     self.kernels.insert(id, KernelLibrary {
                         library: Vec::new(),
                         complete: false });
-                    self.kernels.get_mut(&id)?
+                    self.kernels.get_mut(&id).unwrap()
                 } else {
                     kernel
                 }
@@ -343,7 +333,7 @@ impl Manager {
                 self.kernels.insert(id, KernelLibrary {
                     library: Vec::new(),
                     complete: false });
-                self.kernels.get_mut(&id)?
+                self.kernels.get_mut(&id).unwrap()
             },
         };
         kernel.library.extend(&data[0..data_len]);
@@ -369,23 +359,24 @@ impl Manager {
         unsafe { self.cache.unborrow() }
     }
 
-    pub fn run(&mut self, id: u32) -> Result<(), Error> {
+    pub fn run(&mut self, source: u8, id: u32) -> Result<(), Error> {
         info!("starting subkernel #{}", id);
         if self.session.kernel_state != KernelState::Loaded
             || self.current_id != id {
             self.load(id)?;
         }
+        self.session.source = source;
         self.session.kernel_state = KernelState::Running;
         cricon_select(RtioMaster::Kernel);
     
         kern_acknowledge()
     }
 
-    pub fn message_handle_incoming(&mut self, status: PayloadStatus, length: usize, slice: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
+    pub fn message_handle_incoming(&mut self, status: PayloadStatus, length: usize, id: u32, slice: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
         if !self.is_running() {
             return;
         }
-        self.session.messages.handle_incoming(status, length, slice);
+        self.session.messages.handle_incoming(status, length, id, slice);
     }
     
     pub fn message_get_slice(&mut self, slice: &mut [u8; MASTER_PAYLOAD_MAX_SIZE]) -> Option<SliceMeta> {
@@ -403,19 +394,11 @@ impl Manager {
         self.session.messages.ack_slice()
     }
 
-    pub fn message_is_ready(&mut self) -> bool {
-        self.session.messages.is_outgoing_ready()
-    }
-
-    pub fn get_last_finished(&mut self) -> Option<SubkernelFinished> {
-        self.last_finished.take()
-    }
-
     pub fn load(&mut self, id: u32) -> Result<(), Error> {
         if self.current_id == id && self.session.kernel_state == KernelState::Loaded {
             return Ok(())
         }
-        if !self.kernels.get(&id)?.complete {
+        if !self.kernels.get(&id).ok_or(Error::KernelNotFound)?.complete {
             return Err(Error::KernelNotFound)
         }
         self.current_id = id;
@@ -425,7 +408,7 @@ impl Manager {
         unsafe { 
             kernel_cpu::start();
 
-            kern_send(&kern::LoadRequest(&self.kernels.get(&id)?.library)).unwrap();
+            kern_send(&kern::LoadRequest(&self.kernels.get(&id).unwrap().library)).unwrap();
             kern_recv(|reply| {
                 match reply {
                     kern::LoadReply(Ok(())) => {
@@ -434,6 +417,7 @@ impl Manager {
                     }
                     kern::LoadReply(Err(error)) => {
                         kernel_cpu::stop();
+                        error!("load error: {:?}", error);
                         Err(Error::Load(format!("{}", error)))
                     }
                     other => {
@@ -447,7 +431,7 @@ impl Manager {
     pub fn exception_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
         match self.session.last_exception.as_mut() {
             Some(exception) => exception.get_slice_sat(data_slice),
-            None => SliceMeta { len: 0, status: PayloadStatus::FirstAndLast }
+            None => SliceMeta { destination: 0, len: 0, status: PayloadStatus::FirstAndLast }
         }
     }
 
@@ -472,12 +456,63 @@ impl Manager {
             backtrace: &[],
             async_errors: 0
         }).write_to(&mut writer) {
-            Ok(_) => self.session.last_exception = Some(Sliceable::new(writer.into_inner())),
+            Ok(_) => self.session.last_exception = Some(Sliceable::new(0, writer.into_inner())),
             Err(_) => error!("Error writing exception data")
         }
     }
 
-    pub fn process_kern_requests(&mut self, rank: u8) {
+    pub fn ddma_finished(&mut self, error: u8, channel: u32, timestamp: u64) {
+        if let KernelState::DmaAwait { .. } = self.session.kernel_state {
+            kern_send(&kern::DmaAwaitRemoteReply { 
+                timeout: false, error: error, channel: channel, timestamp: timestamp 
+            }).unwrap();
+            self.session.kernel_state = KernelState::Running;
+        }
+    }
+
+    pub fn ddma_nack(&mut self) {
+        // for simplicity treat it as a timeout for now...
+        if let KernelState::DmaAwait { .. } = self.session.kernel_state {
+            kern_send(&kern::DmaAwaitRemoteReply { 
+                timeout: true, error: 0, channel: 0, timestamp: 0
+            }).unwrap();
+            self.session.kernel_state = KernelState::Running;
+        }
+    }
+
+    pub fn ddma_remote_uploaded(&mut self, succeeded: bool) {
+        if let KernelState::DmaUploading { .. } = self.session.kernel_state {
+            if succeeded {
+                self.session.kernel_state = KernelState::Running;
+                kern_acknowledge().unwrap();
+            } else {
+                self.stop();
+                self.runtime_exception(Error::DmaError(DmaError::UploadFail));
+            }
+        }
+    }
+
+    pub fn process_kern_requests(&mut self, router: &mut Router, routing_table: &RoutingTable, rank: u8, destination: u8, dma_manager: &mut DmaManager) {
+        macro_rules! finished {
+            ($with_exception:expr) => {{ Some(SubkernelFinished { 
+                source: self.session.source, id: self.current_id, 
+                with_exception: $with_exception, exception_source: destination 
+            }) }}
+        }
+
+        if let Some(subkernel_finished) = self.last_finished.take() {
+            info!("subkernel {} finished, with exception: {}", subkernel_finished.id, subkernel_finished.with_exception);
+            let pending = self.session.messages.pending_ids();
+            if pending.len() > 0 {
+                warn!("subkernel terminated with messages still pending: {:?}", pending);
+            }
+            router.route(drtioaux::Packet::SubkernelFinished {
+                destination: subkernel_finished.source, id: subkernel_finished.id, 
+                with_exception: subkernel_finished.with_exception, exception_src: subkernel_finished.exception_source
+            }, &routing_table, rank, destination);
+            dma_manager.cleanup(router, rank, destination, routing_table);
+        }
+
         if !self.is_running() {
             return;
         }
@@ -490,39 +525,39 @@ impl Manager {
                 self.session.kernel_state = KernelState::Absent;
                 unsafe { self.cache.unborrow() }
                 self.session.last_exception = Some(exception);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
             },
             Err(e) => { 
                 error!("Error while running processing external messages: {:?}", e);
                 self.stop();
                 self.runtime_exception(e);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
              }
         }
 
-        match self.process_kern_message(rank) {
+        match self.process_kern_message(router, routing_table, rank, destination, dma_manager) {
             Ok(Some(with_exception)) => {
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: with_exception })
+                self.last_finished = finished!(with_exception)
             },
             Ok(None) | Err(Error::NoMessage) => (),
             Err(e) => { 
                 error!("Error while running kernel: {:?}", e); 
                 self.stop(); 
                 self.runtime_exception(e);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
             }
         }
     }
 
     fn process_external_messages(&mut self) -> Result<(), Error> {
         match &self.session.kernel_state {
-            KernelState::MsgAwait { max_time, tags } => {
-                if clock::get_ms() > *max_time {
+            KernelState::MsgAwait { id, max_time, tags } => {
+                if *max_time > 0 && clock::get_ms() > *max_time as u64 {
                     kern_send(&kern::SubkernelMsgRecvReply { status: kern::SubkernelStatus::Timeout, count: 0 })?;
                     self.session.kernel_state = KernelState::Running;
                     return Ok(())
                 }
-                if let Some(message) = self.session.messages.get_incoming() {
+                if let Some(message) = self.session.messages.get_incoming(*id) {
                     kern_send(&kern::SubkernelMsgRecvReply { status: kern::SubkernelStatus::NoError, count: message.count })?;
                     let tags = tags.clone();
                     self.session.kernel_state = KernelState::Running;
@@ -539,16 +574,88 @@ impl Manager {
                     Err(Error::AwaitingMessage)
                 }
             },
+            KernelState::SubkernelAwaitFinish { max_time, id } => {
+                if *max_time > 0 && clock::get_ms() > *max_time as u64 {
+                    kern_send(&kern::SubkernelAwaitFinishReply { status: kern::SubkernelStatus::Timeout })?;
+                    self.session.kernel_state = KernelState::Running;
+                } else {
+                    let mut i = 0;
+                    for status in &self.session.subkernels_finished {
+                        if *status == *id {
+                            kern_send(&kern::SubkernelAwaitFinishReply { status: kern::SubkernelStatus::NoError })?;
+                            self.session.kernel_state = KernelState::Running;
+                            self.session.subkernels_finished.swap_remove(i);
+                            break;
+                        }
+                        i += 1;
+                    }
+                }
+                Ok(())
+            }
+            KernelState::DmaAwait { max_time } => {
+                if clock::get_ms() > *max_time {
+                    kern_send(&kern::DmaAwaitRemoteReply { timeout: true, error: 0, channel: 0, timestamp: 0 })?;
+                    self.session.kernel_state = KernelState::Running;
+                }
+                // ddma_finished() and nack() covers the other case
+                Ok(())
+            }
+            KernelState::DmaUploading { max_time } => {
+                if clock::get_ms() > *max_time {
+                    unexpected!("DMAError: Timed out sending traces to remote");
+                }
+                Ok(())
+            }
             _ => Ok(())
         }
     }
 
-    fn process_kern_message(&mut self, rank: u8) -> Result<Option<bool>, Error> {
+    pub fn subkernel_load_run_reply(&mut self, succeeded: bool, self_destination: u8) {
+        if self.session.kernel_state == KernelState::SubkernelAwaitLoad {
+            if let Err(e) = kern_send(&kern::SubkernelLoadRunReply { succeeded: succeeded }) {
+                self.stop(); 
+                self.runtime_exception(e);
+                self.last_finished = Some(SubkernelFinished { 
+                    source: self.session.source, id: self.current_id, 
+                    with_exception: true, exception_source: self_destination 
+                })
+            } else {
+                self.session.kernel_state = KernelState::Running;
+            }
+        } else {
+            warn!("received unsolicited SubkernelLoadRunReply");
+        }
+    }
+
+    pub fn remote_subkernel_finished(&mut self, id: u32, with_exception: bool, exception_source: u8) {
+        if with_exception {
+            unsafe { kernel_cpu::stop() }
+            self.session.kernel_state = KernelState::Absent;
+            unsafe { self.cache.unborrow() }
+            self.last_finished = Some(SubkernelFinished {
+                source: self.session.source, id: self.current_id,
+                with_exception: true, exception_source: exception_source
+            })
+        } else {
+            self.session.subkernels_finished.push(id);
+        }
+    }
+
+    fn process_kern_message(&mut self, router: &mut Router, 
+        routing_table: &RoutingTable,
+        rank: u8, destination: u8,
+        dma_manager: &mut DmaManager
+    ) -> Result<Option<bool>, Error> {
         // returns Ok(with_exception) on finish
         // None if the kernel is still running
         kern_recv(|request| {
             match (request, &self.session.kernel_state) {
-                (&kern::LoadReply(_), KernelState::Loaded) => {
+                (&kern::LoadReply(_), KernelState::Loaded) |
+                    (_, KernelState::DmaUploading { .. }) |
+                    (_, KernelState::DmaAwait { .. }) |
+                    (_, KernelState::MsgSending) |
+                    (_, KernelState::SubkernelAwaitLoad) | 
+                    (_, KernelState::SubkernelAwaitFinish { .. }) => {
                     // We're standing by; ignore the message.
                     return Ok(None)
                 }
@@ -559,7 +666,7 @@ impl Manager {
                 },
             }
 
-            if process_kern_hwreq(request, rank)? {
+            if process_kern_hwreq(request, destination)? {
                 return Ok(None)
             }
 
@@ -613,19 +720,93 @@ impl Manager {
                     return Ok(Some(true))
                 }
 
-                &kern::SubkernelMsgSend { id: _, count, tag, data } => {
-                    self.session.messages.accept_outgoing(count, tag, data)?;
+                &kern::DmaRecordStart(name) => {
+                    dma_manager.record_start(name);
+                    kern_acknowledge()
+                }
+                &kern::DmaRecordAppend(data) => {
+                    dma_manager.record_append(data);
+                    kern_acknowledge()
+                }
+                &kern::DmaRecordStop { duration, enable_ddma: _ } => {
+                    // ddma is always used on satellites
+                    if let Ok(id) = dma_manager.record_stop(duration, destination) {
+                        let remote_count = dma_manager.upload_traces(id, router, rank, destination, routing_table)?;
+                        if remote_count > 0 {
+                            let max_time = clock::get_ms() + 10_000 as u64;
+                            self.session.kernel_state = KernelState::DmaUploading { max_time: max_time };
+                            Ok(())
+                        } else {
+                            kern_acknowledge()
+                        }
+                    } else {
+                        unexpected!("DMAError: found an unsupported call to RTIO devices on master") 
+                    }
+                }
+                &kern::DmaEraseRequest { name } => {
+                    dma_manager.erase_name(name, router, rank, destination, routing_table);
+                    kern_acknowledge()
+                }
+                &kern::DmaRetrieveRequest { name } => {
+                    dma_manager.with_trace(destination, name, |trace, duration| {
+                        kern_send(&kern::DmaRetrieveReply {
+                            trace:    trace,
+                            duration: duration,
+                            uses_ddma: true,
+                        })
+                    })
+                }
+                &kern::DmaStartRemoteRequest { id, timestamp } => {
+                    let max_time = clock::get_ms() + 10_000 as u64;
+                    self.session.kernel_state = KernelState::DmaAwait { max_time: max_time };
+                    dma_manager.playback_remote(id as u32, timestamp as u64, router, rank, destination, routing_table)?;
+                    dma_manager.playback(destination, id as u32, timestamp as u64)?;
+                    Ok(())
+                }
+
+                &kern::SubkernelMsgSend { id, destination: msg_dest, count, tag, data } => {
+                    let message_destination;
+                    let message_id;
+                    if let Some(dest) = msg_dest {
+                        message_destination = dest;
+                        message_id = id;
+                    } else {
+                        // return message, return to source
+                        message_destination = self.session.source;
+                        message_id = self.current_id;
+                    }
+                    self.session.messages.accept_outgoing(message_id, destination,
+                        message_destination, count, tag, data, 
+                        routing_table, rank, router)?;
                     // acknowledge after the message is sent
                     self.session.kernel_state = KernelState::MsgSending;
                     Ok(())
                 }
 
-                &kern::SubkernelMsgRecvRequest { id: _, timeout, tags } => {
-                    let max_time = clock::get_ms() + timeout as u64;
-                    self.session.kernel_state = KernelState::MsgAwait { max_time: max_time, tags: tags.to_vec() };
+                &kern::SubkernelMsgRecvRequest { id, timeout, tags } => {
+                    // negative timeout value means no timeout
+                    let max_time = if timeout > 0 { clock::get_ms() as i64 + timeout } else { timeout };
+                    // ID equal to -1 indicates wildcard for receiving arguments
+                    let id = if id == -1 { self.current_id } else { id as u32 };
+                    self.session.kernel_state = KernelState::MsgAwait { 
+                        id: id, max_time: max_time, tags: tags.to_vec() };
                     Ok(())
                 },
 
+                &kern::SubkernelLoadRunRequest { id, destination: sk_destination, run } => {
+                    self.session.kernel_state = KernelState::SubkernelAwaitLoad;
+                    router.route(drtioaux::Packet::SubkernelLoadRunRequest { 
+                        source: destination, destination: sk_destination, id: id, run: run 
+                    }, routing_table, rank, destination);
+                    Ok(())
+                }
+
+                &kern::SubkernelAwaitFinishRequest{ id, timeout } => {
+                    let max_time = if timeout > 0 { clock::get_ms() as i64 + timeout } else { timeout };
+                    self.session.kernel_state = KernelState::SubkernelAwaitFinish { max_time: max_time, id: id };
+                    Ok(())
+                }
+
                 request => unexpected!("unexpected request {:?} from kernel CPU", request)
             }.and(Ok(None))
         })
@@ -699,7 +880,7 @@ fn slice_kernel_exception(exceptions: &[Option<eh_artiq::Exception>],
         async_errors: 0
     }).write_to(&mut writer) {
         // save last exception data to be received by master
-        Ok(_) => Ok(Sliceable::new(writer.into_inner())),
+        Ok(_) => Ok(Sliceable::new(0, writer.into_inner())),
         Err(_) => Err(Error::SubkernelIoError)
     }
 }
@@ -759,7 +940,7 @@ fn pass_message_to_kernel(message: &Message, tags: &[u8]) -> Result<(), Error> {
     Ok(())
 }
 
-fn process_kern_hwreq(request: &kern::Message, rank: u8) -> Result<bool, Error> {
+fn process_kern_hwreq(request: &kern::Message, self_destination: u8) -> Result<bool, Error> {
     match request {
         &kern::RtioInitRequest => {
             unsafe {
@@ -774,7 +955,7 @@ fn process_kern_hwreq(request: &kern::Message, rank: u8) -> Result<bool, Error>
             // only local destination is considered "up"
             // no access to other DRTIO destinations
             kern_send(&kern::RtioDestinationStatusReply { 
-                up: destination == rank })
+                up: destination == self_destination })
         }
 
         &kern::I2cStartRequest { busno } => {

artiq/firmware/satman/main.rs

@@ -1,4 +1,4 @@
-#![feature(never_type, panic_info_message, llvm_asm, default_alloc_error_handler, try_trait)]
+#![feature(never_type, panic_info_message, asm, default_alloc_error_handler)]
 #![no_std]
 
 #[macro_use]
@@ -14,9 +14,13 @@ extern crate io;
 extern crate eh;
 
 use core::convert::TryFrom;
-use board_misoc::{csr, ident, clock, uart_logger, i2c, pmp};
+use board_misoc::{csr, ident, clock, config, uart_logger, i2c, pmp};
 #[cfg(has_si5324)]
 use board_artiq::si5324;
+#[cfg(has_si549)]
+use board_artiq::si549;
+#[cfg(soc_platform = "kasli")]
+use board_misoc::irq;
 use board_artiq::{spi, drtioaux, drtio_routing};
 #[cfg(soc_platform = "efc")]
 use board_artiq::ad9117;
@@ -32,6 +36,7 @@ use analyzer::Analyzer;
 static mut ALLOC: alloc_list::ListAlloc = alloc_list::EMPTY;
 
 mod repeater;
+mod routing;
 mod dma;
 mod analyzer;
 mod kernel;
@@ -65,6 +70,11 @@ fn drtiosat_tsc_loaded() -> bool {
     }
 }
 
+fn toggle_sed_spread(val: u8) {
+    unsafe { csr::drtiosat::sed_spread_enable_write(val); }
+}
+
+#[derive(Clone, Copy)]
 pub enum RtioMaster {
     Drtio,
     Dma,
@@ -82,14 +92,31 @@ pub fn cricon_select(master: RtioMaster) {
     }
 }
 
+pub fn cricon_read() -> RtioMaster {
+    let val = unsafe { csr::cri_con::selected_read() };
+    match val {
+        0 => RtioMaster::Drtio,
+        1 => RtioMaster::Dma,
+        2 => RtioMaster::Kernel,
+        _ => unreachable!()
+    }
+}
+
 #[cfg(has_drtio_routing)]
 macro_rules! forward {
-    ($routing_table:expr, $destination:expr, $rank:expr, $repeaters:expr, $packet:expr) => {{
+    ($router:expr, $routing_table:expr, $destination:expr, $rank:expr, $self_destination:expr, $repeaters:expr, $packet:expr) => {{
         let hop = $routing_table.0[$destination as usize][$rank as usize];
         if hop != 0 {
             let repno = (hop - 1) as usize;
             if repno < $repeaters.len() {
-                return $repeaters[repno].aux_forward($packet);
+                if $packet.expects_response() {
+                    return $repeaters[repno].aux_forward($packet, $router, $routing_table, $rank, $self_destination);
+                } else {
+                    let res = $repeaters[repno].aux_send($packet);
+                    // allow the satellite to parse the packet before next
+                    clock::spin_us(10_000);
+                    return res;
+                }
             } else {
                 return Err(drtioaux::Error::RoutingError);
             }
@@ -99,12 +126,13 @@ macro_rules! forward {
 
 #[cfg(not(has_drtio_routing))]
 macro_rules! forward {
-    ($routing_table:expr, $destination:expr, $rank:expr, $repeaters:expr, $packet:expr) => {}
+    ($router:expr, $routing_table:expr, $destination:expr, $rank:expr, $self_destination:expr, $repeaters:expr, $packet:expr) => {}
 }
 
 fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmgr: &mut KernelManager,
-        _repeaters: &mut [repeater::Repeater], _routing_table: &mut drtio_routing::RoutingTable, _rank: &mut u8,
-        packet: drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
+        _repeaters: &mut [repeater::Repeater], _routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8,
+        router: &mut routing::Router, self_destination: &mut u8, packet: drtioaux::Packet
+) -> Result<(), drtioaux::Error<!>> {
     // In the code below, *_chan_sel_write takes an u8 if there are fewer than 256 channels,
     // and u16 otherwise; hence the `as _` conversion.
     match packet {
@@ -125,29 +153,12 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
 
         drtioaux::Packet::DestinationStatusRequest { destination } => {
             #[cfg(has_drtio_routing)]
-            let hop = _routing_table.0[destination as usize][*_rank as usize];
+            let hop = _routing_table.0[destination as usize][*rank as usize];
             #[cfg(not(has_drtio_routing))]
             let hop = 0;
 
             if hop == 0 {
-                // async messages
-                if let Some(status) = dmamgr.get_status() {
-                    info!("playback done, error: {}, channel: {}, timestamp: {}", status.error, status.channel, status.timestamp);
-                    drtioaux::send(0, &drtioaux::Packet::DmaPlaybackStatus { 
-                        destination: destination, id: status.id, error: status.error, channel: status.channel, timestamp: status.timestamp })?;
-                } else if let Some(subkernel_finished) = kernelmgr.get_last_finished() {
-                    info!("subkernel {} finished, with exception: {}", subkernel_finished.id, subkernel_finished.with_exception);
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelFinished {
-                        id: subkernel_finished.id, with_exception: subkernel_finished.with_exception
-                    })?;
-                } else if kernelmgr.message_is_ready() {
-                    let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
-                    let meta = kernelmgr.message_get_slice(&mut data_slice).unwrap();
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelMessage {
-                        destination: destination, id: kernelmgr.get_current_id().unwrap(),
-                        status: meta.status, length: meta.len as u16, data: data_slice
-                    })?;
-                } else {
+                *self_destination = destination;
                 let errors;
                 unsafe {
                     errors = csr::drtiosat::rtio_error_read();
@@ -181,7 +192,6 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                     drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
                 }
             }
-            }
 
             #[cfg(has_drtio_routing)]
             {
@@ -191,7 +201,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                         let repno = hop - 1;
                         match _repeaters[repno].aux_forward(&drtioaux::Packet::DestinationStatusRequest {
                             destination: destination
-                        }) {
+                        }, router, _routing_table, *rank, *self_destination) {
                             Ok(()) => (),
                             Err(drtioaux::Error::LinkDown) => drtioaux::send(0, &drtioaux::Packet::DestinationDownReply)?,
                             Err(e) => {
@@ -204,7 +214,6 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                     }
                 }
             }
-
             Ok(())
         }
 
@@ -219,18 +228,18 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             drtioaux::send(0, &drtioaux::Packet::RoutingAck)
         }
         #[cfg(has_drtio_routing)]
-        drtioaux::Packet::RoutingSetRank { rank } => {
-            *_rank = rank;
-            drtio_routing::interconnect_enable_all(_routing_table, rank);
+        drtioaux::Packet::RoutingSetRank { rank: new_rank } => {
+            *rank = new_rank;
+            drtio_routing::interconnect_enable_all(_routing_table, new_rank);
 
-            let rep_rank = rank + 1;
+            let rep_rank = new_rank + 1;
             for rep in _repeaters.iter() {
                 if let Err(e) = rep.set_rank(rep_rank) {
                     error!("failed to set rank ({})", e);
                 }
             }
 
-            info!("rank: {}", rank);
+            info!("rank: {}", new_rank);
             info!("routing table: {}", _routing_table);
 
             drtioaux::send(0, &drtioaux::Packet::RoutingAck)
@@ -246,7 +255,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::MonitorRequest { destination: _destination, channel, probe } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let value;
             #[cfg(has_rtio_moninj)]
             unsafe {
@@ -263,7 +272,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             drtioaux::send(0, &reply)
         },
         drtioaux::Packet::InjectionRequest { destination: _destination, channel, overrd, value } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             #[cfg(has_rtio_moninj)]
             unsafe {
                 csr::rtio_moninj::inj_chan_sel_write(channel as _);
@@ -273,7 +282,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             Ok(())
         },
         drtioaux::Packet::InjectionStatusRequest { destination: _destination, channel, overrd } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let value;
             #[cfg(has_rtio_moninj)]
             unsafe {
@@ -289,22 +298,22 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         },
 
         drtioaux::Packet::I2cStartRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = i2c::start(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cRestartRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = i2c::restart(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cStopRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = i2c::stop(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cWriteRequest { destination: _destination, busno, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             match i2c::write(busno, data) {
                 Ok(ack) => drtioaux::send(0,
                     &drtioaux::Packet::I2cWriteReply { succeeded: true, ack: ack }),
@@ -313,7 +322,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             }
         }
         drtioaux::Packet::I2cReadRequest { destination: _destination, busno, ack } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             match i2c::read(busno, ack) {
                 Ok(data) => drtioaux::send(0,
                     &drtioaux::Packet::I2cReadReply { succeeded: true, data: data }),
@@ -322,25 +331,25 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             }
         }
         drtioaux::Packet::I2cSwitchSelectRequest { destination: _destination, busno, address, mask } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = i2c::switch_select(busno, address, mask).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
 
         drtioaux::Packet::SpiSetConfigRequest { destination: _destination, busno, flags, length, div, cs } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = spi::set_config(busno, flags, length, div, cs).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SpiBasicReply { succeeded: succeeded })
         },
         drtioaux::Packet::SpiWriteRequest { destination: _destination, busno, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let succeeded = spi::write(busno, data).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SpiBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::SpiReadRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             match spi::read(busno) {
                 Ok(data) => drtioaux::send(0,
                     &drtioaux::Packet::SpiReadReply { succeeded: true, data: data }),
@@ -350,7 +359,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::AnalyzerHeaderRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let header = analyzer.get_header();
             drtioaux::send(0, &drtioaux::Packet::AnalyzerHeader {
                 total_byte_count: header.total_byte_count,
@@ -360,7 +369,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::AnalyzerDataRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
             let meta = analyzer.get_data(&mut data_slice);
             drtioaux::send(0, &drtioaux::Packet::AnalyzerData {
@@ -370,34 +379,56 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             })
         }
 
-        drtioaux::Packet::DmaAddTraceRequest { destination: _destination, id, status, length, trace } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
-            let succeeded = dmamgr.add(id, status, &trace, length as usize).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaAddTraceReply { succeeded: succeeded })
+        drtioaux::Packet::DmaAddTraceRequest { source, destination, id, status, length, trace } => {
+            forward!(router, _routing_table, destination, *rank, *self_destination, _repeaters, &packet);
+            *self_destination = destination;
+            let succeeded = dmamgr.add(source, id, status, &trace, length as usize).is_ok();
+            router.send(drtioaux::Packet::DmaAddTraceReply { 
+                source: *self_destination, destination: source, id: id, succeeded: succeeded 
+            }, _routing_table, *rank, *self_destination)
         }
-        drtioaux::Packet::DmaRemoveTraceRequest { destination: _destination, id } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
-            let succeeded = dmamgr.erase(id).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaRemoveTraceReply { succeeded: succeeded })
+        drtioaux::Packet::DmaAddTraceReply { source, destination: _destination, id, succeeded } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            dmamgr.ack_upload(kernelmgr, source, id, succeeded, router, *rank, *self_destination, _routing_table);
+            Ok(())
         }
-        drtioaux::Packet::DmaPlaybackRequest { destination: _destination, id, timestamp } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::DmaRemoveTraceRequest { source, destination: _destination, id } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            let succeeded = dmamgr.erase(source, id).is_ok();
+            router.send(drtioaux::Packet::DmaRemoveTraceReply { 
+                destination: source, succeeded: succeeded 
+            }, _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::DmaPlaybackRequest { source, destination: _destination, id, timestamp } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             // no DMA with a running kernel
-            let succeeded = !kernelmgr.is_running() && dmamgr.playback(id, timestamp).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaPlaybackReply { succeeded: succeeded })
+            let succeeded = !kernelmgr.is_running() && dmamgr.playback(source, id, timestamp).is_ok();
+            router.send(drtioaux::Packet::DmaPlaybackReply { 
+                destination: source, succeeded: succeeded
+            }, _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::DmaPlaybackReply { destination: _destination, succeeded } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            if !succeeded {
+                kernelmgr.ddma_nack();
+            }
+            Ok(())
+        }
+        drtioaux::Packet::DmaPlaybackStatus { source: _, destination: _destination, id, error, channel, timestamp } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            dmamgr.remote_finished(kernelmgr, id, error, channel, timestamp);
+            Ok(())
         }
 
-        drtioaux::Packet::SubkernelAddDataRequest { destination: _destination, id, status, length, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::SubkernelAddDataRequest { destination, id, status, length, data } => {
+            forward!(router, _routing_table, destination, *rank, *self_destination, _repeaters, &packet);
+            *self_destination = destination;
             let succeeded = kernelmgr.add(id, status, &data, length as usize).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SubkernelAddDataReply { succeeded: succeeded })
         }
-        drtioaux::Packet::SubkernelLoadRunRequest { destination: _destination, id, run } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::SubkernelLoadRunRequest { source, destination: _destination, id, run } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let mut succeeded = kernelmgr.load(id).is_ok();
             // allow preloading a kernel with delayed run
             if run {
@@ -405,14 +436,27 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                     // cannot run kernel while DDMA is running
                     succeeded = false;
                 } else {
-                    succeeded |= kernelmgr.run(id).is_ok();
+                    succeeded |= kernelmgr.run(source, id).is_ok();
                 }
             }
-            drtioaux::send(0,
-                &drtioaux::Packet::SubkernelLoadRunReply { succeeded: succeeded })
+            router.send(drtioaux::Packet::SubkernelLoadRunReply { 
+                    destination: source, succeeded: succeeded 
+                }, 
+            _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::SubkernelLoadRunReply { destination: _destination, succeeded } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            // received if local subkernel started another, remote subkernel
+            kernelmgr.subkernel_load_run_reply(succeeded, *self_destination);
+            Ok(())
+        }
+        drtioaux::Packet::SubkernelFinished { destination: _destination, id, with_exception, exception_src } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            kernelmgr.remote_subkernel_finished(id, with_exception, exception_src);
+            Ok(())
         }
         drtioaux::Packet::SubkernelExceptionRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
             let meta = kernelmgr.exception_get_slice(&mut data_slice);
             drtioaux::send(0, &drtioaux::Packet::SubkernelException {
@@ -421,22 +465,23 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                 data: data_slice,
             })
         }
-        drtioaux::Packet::SubkernelMessage { destination, id: _id, status, length, data } => {
-            forward!(_routing_table, destination, *_rank, _repeaters, &packet);
-            kernelmgr.message_handle_incoming(status, length as usize, &data);
-            drtioaux::send(0, &drtioaux::Packet::SubkernelMessageAck {
-                destination: destination
-            })
+        drtioaux::Packet::SubkernelMessage { source, destination: _destination, id, status, length, data } => {
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
+            kernelmgr.message_handle_incoming(status, length as usize, id, &data);
+            router.send(drtioaux::Packet::SubkernelMessageAck {
+                    destination: source
+                }, _routing_table, *rank, *self_destination)
         }
         drtioaux::Packet::SubkernelMessageAck { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(router, _routing_table, _destination, *rank, *self_destination, _repeaters, &packet);
             if kernelmgr.message_ack_slice() {
                 let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 if let Some(meta) = kernelmgr.message_get_slice(&mut data_slice) {
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelMessage {
-                        destination: *_rank, id: kernelmgr.get_current_id().unwrap(),
+                    // route and not send immediately as ACKs are not a beginning of a transaction
+                    router.route(drtioaux::Packet::SubkernelMessage {
+                        source: *self_destination, destination: meta.destination, id: kernelmgr.get_current_id().unwrap(),
                         status: meta.status, length: meta.len as u16, data: data_slice
-                    })?
+                    }, _routing_table, *rank, *self_destination);
                 } else {
                     error!("Error receiving message slice");
                 }
@@ -453,18 +498,19 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
 
 fn process_aux_packets(dma_manager: &mut DmaManager, analyzer: &mut Analyzer,
         kernelmgr: &mut KernelManager, repeaters: &mut [repeater::Repeater],
-        routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8) {
+        routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8, router: &mut routing::Router,
+        destination: &mut u8) {
     let result =
         drtioaux::recv(0).and_then(|packet| {
-            if let Some(packet) = packet {
-                process_aux_packet(dma_manager, analyzer, kernelmgr, repeaters, routing_table, rank, packet)
+            if let Some(packet) = packet.or_else(|| router.get_local_packet()) {
+                process_aux_packet(dma_manager, analyzer, kernelmgr, 
+                    repeaters, routing_table, rank, router, destination, packet)
             } else {
                 Ok(())
             }
         });
-    match result {
-        Ok(()) => (),
-        Err(e) => warn!("aux packet error ({})", e)
+    if let Err(e) = result {
+        warn!("aux packet error ({})", e);
     }
 }
 
@@ -557,6 +603,36 @@ const SI5324_SETTINGS: si5324::FrequencySettings
     crystal_as_ckin2: true
 };
 
+#[cfg(all(has_si549, rtio_frequency = "125.0"))]
+const SI549_SETTINGS: si549::FrequencySetting = si549::FrequencySetting {
+    main: si549::DividerConfig {
+        hsdiv: 0x058,
+        lsdiv: 0,
+        fbdiv: 0x04815791F25,
+    },
+    helper: si549::DividerConfig {
+        // 125MHz*32767/32768
+        hsdiv: 0x058,
+        lsdiv: 0,
+        fbdiv: 0x04814E8F442,
+    },
+};
+
+#[cfg(all(has_si549, rtio_frequency = "100.0"))]
+const SI549_SETTINGS: si549::FrequencySetting = si549::FrequencySetting {
+    main: si549::DividerConfig {
+        hsdiv: 0x06C,
+        lsdiv: 0,
+        fbdiv: 0x046C5F49797,
+    },
+    helper: si549::DividerConfig {
+        // 100MHz*32767/32768
+        hsdiv: 0x06C,
+        lsdiv: 0,
+        fbdiv: 0x046C5670BBD,
+    },
+};
+
 #[cfg(not(soc_platform = "efc"))]
 fn sysclk_setup() {
     let switched = unsafe {
@@ -569,6 +645,9 @@ fn sysclk_setup() {
     else {
         #[cfg(has_si5324)]
         si5324::setup(&SI5324_SETTINGS, si5324::Input::Ckin1).expect("cannot initialize Si5324");
+        #[cfg(has_si549)]
+        si549::main_setup(&SI549_SETTINGS).expect("cannot initialize main Si549");
+
         info!("Switching sys clock, rebooting...");
         // delay for clean UART log, wait until UART FIFO is empty
         clock::spin_us(3000);
@@ -592,6 +671,10 @@ pub extern fn main() -> i32 {
         ALLOC.add_range(&mut _fheap, &mut _eheap);
         pmp::init_stack_guard(&_sstack_guard as *const u8 as usize);
     }
+    #[cfg(soc_platform = "kasli")]
+    irq::enable_interrupts();
+    #[cfg(has_wrpll)]
+    irq::enable(csr::WRPLL_INTERRUPT);
 
     clock::init();
     uart_logger::ConsoleLogger::register();
@@ -627,10 +710,27 @@ pub extern fn main() -> i32 {
     #[cfg(not(soc_platform = "efc"))]
     sysclk_setup();
 
+    #[cfg(has_si549)]
+    si549::helper_setup(&SI549_SETTINGS).expect("cannot initialize helper Si549");    
+
     #[cfg(soc_platform = "efc")]
     let mut io_expander;
     #[cfg(soc_platform = "efc")]
     {
+        let p3v3_fmc_en_pin;
+        let vadj_fmc_en_pin;
+
+        #[cfg(hw_rev = "v1.0")]
+        {
+            p3v3_fmc_en_pin = 0;
+            vadj_fmc_en_pin = 1;
+        }
+        #[cfg(hw_rev = "v1.1")]
+        {
+            p3v3_fmc_en_pin = 1;
+            vadj_fmc_en_pin = 7;
+        }
+
         io_expander = board_misoc::io_expander::IoExpander::new().unwrap();
         io_expander.init().expect("I2C I/O expander initialization failed");
 
@@ -638,15 +738,15 @@ pub extern fn main() -> i32 {
         io_expander.set_oe(0, 1 << 5 | 1 << 6 | 1 << 7).unwrap();
         
         // Enable VADJ and P3V3_FMC
-        io_expander.set_oe(1, 1 << 0 | 1 << 1).unwrap();
+        io_expander.set_oe(1, 1 << p3v3_fmc_en_pin | 1 << vadj_fmc_en_pin).unwrap();
 
-        io_expander.set(1, 0, true);
-        io_expander.set(1, 1, true);
+        io_expander.set(1, p3v3_fmc_en_pin, true);
+        io_expander.set(1, vadj_fmc_en_pin, true);
 
         io_expander.service().unwrap();
     }
 
-    #[cfg(not(has_drtio_eem))]
+    #[cfg(not(soc_platform = "efc"))]
     unsafe {
         csr::gt_drtio::txenable_write(0xffffffffu32 as _);
     }
@@ -658,6 +758,18 @@ pub extern fn main() -> i32 {
 
     init_rtio_crg();
 
+    config::read_str("sed_spread_enable", |r| {
+        match r {
+            Ok("1") => { info!("SED spreading enabled"); toggle_sed_spread(1); },
+            Ok("0") => { info!("SED spreading disabled"); toggle_sed_spread(0); },
+            Ok(_) => { 
+                warn!("sed_spread_enable value not supported (only 1, 0 allowed), disabling by default");
+                toggle_sed_spread(0);
+            },
+            Err(_) => { info!("SED spreading disabled by default"); toggle_sed_spread(0) },
+        }
+    });
+
     #[cfg(has_drtio_eem)]
     drtio_eem::init();
 
@@ -670,6 +782,7 @@ pub extern fn main() -> i32 {
     } 
     let mut routing_table = drtio_routing::RoutingTable::default_empty();
     let mut rank = 1;
+    let mut destination = 1;
 
     let mut hardware_tick_ts = 0;
 
@@ -677,10 +790,12 @@ pub extern fn main() -> i32 {
     ad9117::init().expect("AD9117 initialization failed");
     
     loop {
+        let mut router = routing::Router::new();
+
         while !drtiosat_link_rx_up() {
             drtiosat_process_errors();
             for rep in repeaters.iter_mut() {
-                rep.service(&routing_table, rank);
+                rep.service(&routing_table, rank, destination, &mut router);
             }
             #[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
             {
@@ -699,6 +814,9 @@ pub extern fn main() -> i32 {
             si5324::siphaser::calibrate_skew().expect("failed to calibrate skew");
         }
 
+        #[cfg(has_wrpll)]
+        si549::wrpll::select_recovered_clock(true);
+
         // various managers created here, so when link is dropped, DMA traces,
         // analyzer logs, kernels are cleared and/or stopped for a clean slate
         // on subsequent connections, without a manual intervention.
@@ -714,10 +832,10 @@ pub extern fn main() -> i32 {
         while drtiosat_link_rx_up() {
             drtiosat_process_errors();
             process_aux_packets(&mut dma_manager, &mut analyzer, 
-                &mut kernelmgr, &mut repeaters, 
-                &mut routing_table, &mut rank);
+                &mut kernelmgr, &mut repeaters, &mut routing_table,
+                &mut rank, &mut router, &mut destination);
             for rep in repeaters.iter_mut() {
-                rep.service(&routing_table, rank);
+                rep.service(&routing_table, rank, destination, &mut router);
             }
             #[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
             {
@@ -738,7 +856,26 @@ pub extern fn main() -> i32 {
                     error!("aux packet error: {}", e);
                 }
             }
-            kernelmgr.process_kern_requests(rank);
+            if let Some(status) = dma_manager.get_status() {
+                info!("playback done, error: {}, channel: {}, timestamp: {}", status.error, status.channel, status.timestamp);
+                router.route(drtioaux::Packet::DmaPlaybackStatus { 
+                    source: destination, destination: status.source, id: status.id,
+                    error: status.error, channel: status.channel, timestamp: status.timestamp 
+                }, &routing_table, rank, destination);
+            }
+
+            kernelmgr.process_kern_requests(&mut router, &routing_table, rank, destination, &mut dma_manager);
+            
+            #[cfg(has_drtio_routing)]
+            if let Some((repno, packet)) = router.get_downstream_packet() {
+                if let Err(e) = repeaters[repno].aux_send(&packet) {
+                    warn!("[REP#{}] Error when sending packet to satellite ({:?})", repno, e)
+                }
+            }
+
+            if let Some(packet) = router.get_upstream_packet() {
+                drtioaux::send(0, &packet).unwrap();
+            }
         }
 
         drtiosat_reset_phy(true);
@@ -747,11 +884,27 @@ pub extern fn main() -> i32 {
         info!("uplink is down, switching to local oscillator clock");
         #[cfg(has_si5324)]
         si5324::siphaser::select_recovered_clock(false).expect("failed to switch clocks");
+        #[cfg(has_wrpll)]
+        si549::wrpll::select_recovered_clock(false);
     }
 }
 
 #[cfg(soc_platform = "efc")]
 fn enable_error_led() {
+    let p3v3_fmc_en_pin;
+    let vadj_fmc_en_pin;
+
+    #[cfg(hw_rev = "v1.0")]
+    {
+        p3v3_fmc_en_pin = 0;
+        vadj_fmc_en_pin = 1;
+    }
+    #[cfg(hw_rev = "v1.1")]
+    {
+        p3v3_fmc_en_pin = 1;
+        vadj_fmc_en_pin = 7;
+    }
+
     let mut io_expander = board_misoc::io_expander::IoExpander::new().unwrap();
 
     // Keep LEDs enabled
@@ -760,10 +913,10 @@ fn enable_error_led() {
     io_expander.set(0, 7, true);
 
     // Keep VADJ and P3V3_FMC enabled
-    io_expander.set_oe(1, 1 << 0 | 1 << 1).unwrap();
+    io_expander.set_oe(1, 1 << p3v3_fmc_en_pin | 1 << vadj_fmc_en_pin).unwrap();
 
-    io_expander.set(1, 0, true);
-    io_expander.set(1, 1, true);
+    io_expander.set(1, p3v3_fmc_en_pin, true);
+    io_expander.set(1, vadj_fmc_en_pin, true);
 
     io_expander.service().unwrap();
 }
@@ -772,7 +925,15 @@ fn enable_error_led() {
 pub extern fn exception(_regs: *const u32) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
+    match cause {
+        mcause::Trap::Interrupt(_source) => {
+            #[cfg(has_wrpll)]
+            if irq::is_pending(csr::WRPLL_INTERRUPT) {
+                si549::wrpll::interrupt_handler();
+            }
+        },
 
+        mcause::Trap::Exception(e) => {
             fn hexdump(addr: u32) {
                 let addr = (addr - addr % 4) as *const u32;
                 let mut ptr  = addr;
@@ -788,7 +949,9 @@ pub extern fn exception(_regs: *const u32) {
 
             hexdump(u32::try_from(pc).unwrap());
             let mtval = mtval::read();
-    panic!("exception {:?} at PC 0x{:x}, trap value 0x{:x}", cause, u32::try_from(pc).unwrap(), mtval)
+            panic!("exception {:?} at PC 0x{:x}, trap value 0x{:x}", e, u32::try_from(pc).unwrap(), mtval)
+        }
+    }
 }
 
 #[no_mangle]

artiq/firmware/satman/repeater.rs

@@ -1,6 +1,7 @@
 use board_artiq::{drtioaux, drtio_routing};
 #[cfg(has_drtio_routing)]
 use board_misoc::{csr, clock};
+use routing::Router;
 
 #[cfg(has_drtio_routing)]
 fn rep_link_rx_up(repno: u8) -> bool {
@@ -48,7 +49,7 @@ impl Repeater {
         self.state == RepeaterState::Up
     }
 
-    pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8) {
+    pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8, self_destination: u8, router: &mut Router) {
         self.process_local_errors();
 
         match self.state {
@@ -74,6 +75,11 @@ impl Repeater {
                 if rep_link_rx_up(self.repno) {
                     if let Ok(Some(drtioaux::Packet::EchoReply)) = drtioaux::recv(self.auxno) {
                         info!("[REP#{}] remote replied after {} packets", self.repno, ping_count);
+                        // clear the aux buffer
+                        let max_time = clock::get_ms() + 200;
+                        while clock::get_ms() < max_time {
+                            let _ = drtioaux::recv(self.auxno);
+                        }
                         self.state = RepeaterState::Up;
                         if let Err(e) = self.sync_tsc() {
                             error!("[REP#{}] failed to sync TSC ({})", self.repno, e);
@@ -106,7 +112,7 @@ impl Repeater {
                 }
             }
             RepeaterState::Up => {
-                self.process_unsolicited_aux();
+                self.process_unsolicited_aux(routing_table, rank, self_destination, router);
                 if !rep_link_rx_up(self.repno) {
                     info!("[REP#{}] link is down", self.repno);
                     self.state = RepeaterState::Down;
@@ -121,9 +127,10 @@ impl Repeater {
         }
     }
 
-    fn process_unsolicited_aux(&self) {
+    fn process_unsolicited_aux(&self, routing_table: &drtio_routing::RoutingTable, 
+        rank: u8, self_destination: u8, router: &mut Router) {
         match drtioaux::recv(self.auxno) {
-            Ok(Some(packet)) => warn!("[REP#{}] unsolicited aux packet: {:?}", self.repno, packet),
+            Ok(Some(packet)) => router.route(packet, routing_table, rank, self_destination),
             Ok(None) => (),
             Err(_) => warn!("[REP#{}] aux packet error", self.repno)
         }
@@ -179,14 +186,39 @@ impl Repeater {
         }
     }
 
-    pub fn aux_forward(&self, request: &drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
+    pub fn aux_forward(&self, request: &drtioaux::Packet, router: &mut Router, 
+        routing_table: &drtio_routing::RoutingTable, rank: u8,
+        self_destination: u8) -> Result<(), drtioaux::Error<!>> {
+        self.aux_send(request)?;
+        loop {
+            let reply = self.recv_aux_timeout(200)?;
+            match reply {
+                // async/locally requested packets to be consumed or routed
+                // these may come while a packet would be forwarded
+                drtioaux::Packet::DmaPlaybackStatus { .. } |
+                drtioaux::Packet::SubkernelFinished { .. } |
+                drtioaux::Packet::SubkernelMessage  { .. } | 
+                drtioaux::Packet::SubkernelMessageAck { .. } | 
+                drtioaux::Packet::SubkernelLoadRunReply { .. } |
+                drtioaux::Packet::SubkernelException { .. } |
+                drtioaux::Packet::DmaAddTraceReply { .. } |
+                drtioaux::Packet::DmaPlaybackReply { .. } => {
+                    router.route(reply, routing_table, rank, self_destination);
+                }
+                _ => {
+                    drtioaux::send(0, &reply).unwrap();
+                    break;
+                }
+            }
+        }
+        Ok(())
+    }
+
+    pub fn aux_send(&self, request: &drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
         if self.state != RepeaterState::Up {
             return Err(drtioaux::Error::LinkDown);
         }
-        drtioaux::send(self.auxno, request).unwrap();
-        let reply = self.recv_aux_timeout(200)?;
-        drtioaux::send(0, &reply).unwrap();
-        Ok(())
+        drtioaux::send(self.auxno, request)
     }
 
     pub fn sync_tsc(&self) -> Result<(), drtioaux::Error<!>> {
@@ -199,7 +231,6 @@ impl Repeater {
             (csr::DRTIOREP[repno].set_time_write)(1);
             while (csr::DRTIOREP[repno].set_time_read)() == 1 {}
         }
-
         // TSCAck is the only aux packet that is sent spontaneously
         // by the satellite, in response to a TSC set on the RT link.
         let reply = self.recv_aux_timeout(10000)?;
@@ -275,7 +306,7 @@ pub struct Repeater {
 impl Repeater {
     pub fn new(_repno: u8) -> Repeater { Repeater::default() }
 
-    pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8) { }
+    pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8, _destination: u8, _router: &mut Router) { }
 
     pub fn sync_tsc(&self) -> Result<(), drtioaux::Error<!>> { Ok(()) }
 

artiq/firmware/satman/routing.rs

@@ -0,0 +1,169 @@
+use alloc::{vec::Vec, collections::vec_deque::VecDeque};
+use board_artiq::{drtioaux, drtio_routing};
+#[cfg(has_drtio_routing)]
+use board_misoc::csr;
+use core::cmp::min;
+use proto_artiq::drtioaux_proto::PayloadStatus;
+use SAT_PAYLOAD_MAX_SIZE;
+use MASTER_PAYLOAD_MAX_SIZE;
+
+/* represents data that has to be sent with the aux protocol */
+#[derive(Debug)]
+pub struct Sliceable {
+    it: usize,
+    data: Vec<u8>,
+    destination: u8
+}
+
+pub struct SliceMeta {
+    pub destination: u8,
+    pub len: u16,
+    pub status: PayloadStatus
+}
+
+macro_rules! get_slice_fn {
+    ( $name:tt, $size:expr ) => {
+        pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
+            let first = self.it == 0;
+            let len = min($size, self.data.len() - self.it);
+            let last = self.it + len == self.data.len();
+            let status = PayloadStatus::from_status(first, last);
+            data_slice[..len].clone_from_slice(&self.data[self.it..self.it+len]);
+            self.it += len;
+    
+            SliceMeta {
+                destination: self.destination,
+                len: len as u16,
+                status: status
+            }
+        }
+    };
+}
+
+impl Sliceable {
+    pub fn new(destination: u8, data: Vec<u8>) -> Sliceable {
+        Sliceable {
+            it: 0,
+            data: data,
+            destination: destination
+        }
+    }
+
+    pub fn at_end(&self) -> bool {
+        self.it == self.data.len()
+    }
+
+    pub fn extend(&mut self, data: &[u8]) {
+        self.data.extend(data);
+    }
+
+    get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
+    get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
+}
+
+// Packets from downstream (further satellites) are received and routed appropriately.
+// they're passed as soon as possible downstream (within the subtree), or sent upstream,
+// which is notified about pending packets.
+// for rank 1 (connected to master) satellites, these packets are passed as an answer to DestinationStatusRequest;
+// for higher ranks, after getting a notification, it will transact with downstream to get the pending packets.
+
+// forward! macro is not deprecated, as routable packets are only these that can originate
+// from both master and satellite, e.g. DDMA and Subkernel.
+
+pub struct Router {
+    upstream_queue: VecDeque<drtioaux::Packet>, 
+    local_queue: VecDeque<drtioaux::Packet>,
+    #[cfg(has_drtio_routing)]
+    downstream_queue: VecDeque<(usize, drtioaux::Packet)>,
+}
+
+impl Router {
+    pub fn new() -> Router {
+        Router {
+            upstream_queue: VecDeque::new(),
+            local_queue: VecDeque::new(),
+            #[cfg(has_drtio_routing)]
+            downstream_queue: VecDeque::new(),
+        }
+    }
+
+    // called by local sources (DDMA, kernel) and by repeaters on receiving async data
+    // messages are always buffered for both upstream and downstream
+    pub fn route(&mut self, packet: drtioaux::Packet,
+        _routing_table: &drtio_routing::RoutingTable, _rank: u8,
+        self_destination: u8
+    ) {
+        let destination = packet.routable_destination();
+        #[cfg(has_drtio_routing)]
+        {
+            if let Some(destination) = destination {
+                let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
+                if destination == self_destination {
+                    self.local_queue.push_back(packet);
+                } else if hop > 0 && hop < csr::DRTIOREP.len() {
+                    let repno = (hop - 1) as usize;
+                    self.downstream_queue.push_back((repno, packet));
+                } else {
+                    self.upstream_queue.push_back(packet);
+                }
+            } else {
+                error!("Received an unroutable packet: {:?}", packet);
+            }
+        }
+        #[cfg(not(has_drtio_routing))]
+        {
+            if destination == Some(self_destination) {
+                self.local_queue.push_back(packet);
+            } else {
+                self.upstream_queue.push_back(packet);
+            }
+        }
+    }
+
+    // Sends a packet to a required destination, routing if it's necessary
+    pub fn send(&mut self, packet: drtioaux::Packet,
+        _routing_table: &drtio_routing::RoutingTable,
+        _rank: u8, _destination: u8
+    ) -> Result<(), drtioaux::Error<!>> {
+        #[cfg(has_drtio_routing)]
+        {
+            let destination = packet.routable_destination();
+            if let Some(destination) = destination {
+                let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
+                if destination == 0 {
+                    // response is needed immediately if master required it
+                    drtioaux::send(0, &packet)?;
+                } else if !(hop > 0 && hop < csr::DRTIOREP.len()) {
+                    // higher rank can wait
+                    self.upstream_queue.push_back(packet);
+                } else {
+                    let repno = (hop - 1) as usize;
+                    // transaction will occur at closest possible opportunity
+                    self.downstream_queue.push_back((repno, packet));
+                }
+                Ok(())
+            } else {
+                // packet not supported in routing, fallback - sent directly
+                drtioaux::send(0, &packet)
+            }
+        }
+        #[cfg(not(has_drtio_routing))]
+        {
+            drtioaux::send(0, &packet)
+        }
+    }
+
+    pub fn get_upstream_packet(&mut self) -> Option<drtioaux::Packet> {
+        let packet = self.upstream_queue.pop_front();
+        packet
+    }
+
+    #[cfg(has_drtio_routing)]
+    pub fn get_downstream_packet(&mut self) -> Option<(usize, drtioaux::Packet)> {
+        self.downstream_queue.pop_front()
+    }
+
+    pub fn get_local_packet(&mut self) -> Option<drtioaux::Packet> {
+        self.local_queue.pop_front()
+    }
+}

artiq/frontend/afws_client.py

@@ -4,6 +4,7 @@ import sys
 import argparse
 import os
 import socket
+import asyncio
 import ssl
 import io
 import zipfile
@@ -41,22 +42,27 @@ def zip_unarchive(data, directory):
 
 class Client:
     def __init__(self, server, port, cafile):
+        self.server = server
+        self.port = port
         self.ssl_context = ssl.create_default_context(cafile=cafile)
-        self.raw_socket = socket.create_connection((server, port))
-        self.init_websocket(server)
-        try:
-            self.socket = self.ssl_context.wrap_socket(self.raw_socket, server_hostname=server)
-        except:
-            self.raw_socket.close()
-            raise
-        self.fsocket = self.socket.makefile("rwb")
+        self.reader = None
+        self.writer = None
 
-    def init_websocket(self, server):
-        self.raw_socket.sendall("GET / HTTP/1.1\r\nHost: {}\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n"
-            .format(server).encode())
+    async def connect(self):
+        self.reader, self.writer = await asyncio.open_connection(
+            host=self.server,
+            port=self.port,
+            happy_eyeballs_delay=0.25
+        )
+        await self.init_websocket()
+        await self.writer.start_tls(self.ssl_context)
+
+    async def init_websocket(self):
+        self.writer.write("GET / HTTP/1.1\r\nHost: {}\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n"
+            .format(self.server).encode())
         crlf_count = 0
         while crlf_count < 4:
-            char = self.raw_socket.recv(1)
+            char = await self.reader.read(1)
             if not char:
                 return ValueError("Connection closed during WebSocket initialization")
             if char == b"\r" or char == b"\n":
@@ -64,30 +70,30 @@ class Client:
             else:
                 crlf_count = 0
 
-    def close(self):
-        self.socket.close()
-        self.raw_socket.close()
+    async def close(self):
+        if self.writer:
+            self.writer.close()
+            await self.writer.wait_closed()
 
-    def send_command(self, *command):
-        self.fsocket.write((" ".join(command) + "\n").encode())
-        self.fsocket.flush()
+    async def send_command(self, *command):
+        self.writer.write((" ".join(command) + "\n").encode())
 
-    def read_line(self):
-        return self.fsocket.readline().decode("ascii")
+    async def read_line(self):
+        return (await self.reader.readline()).decode("ascii")
 
-    def read_reply(self):
-        return self.fsocket.readline().decode("ascii").split()
+    async def read_reply(self):
+        return (await self.reader.readline()).decode("ascii").split()
 
-    def read_json(self):
-        return json.loads(self.fsocket.readline().decode("ascii"))
+    async def read_json(self):
+        return json.loads(await self.reader.readline().decode("ascii"))
 
-    def login(self, username, password):
-        self.send_command("LOGIN", username, password)
-        return self.read_reply() == ["HELLO"]
+    async def login(self, username, password):
+        await self.send_command("LOGIN", username, password)
+        return await self.read_reply() == ["HELLO"]
 
-    def build(self, major_ver, rev, variant, log, experimental_features):
+    async def build(self, major_ver, rev, variant, log, experimental_features):
         if not variant:
-            variant = self.get_single_variant(error_msg="User can build more than 1 variant - need to specify")
+            variant = await self.get_single_variant(error_msg="User can build more than 1 variant - need to specify")
         print("Building variant: {}".format(variant))
         build_args = (
             rev,
@@ -96,25 +102,25 @@ class Client:
             major_ver,
             *experimental_features,
         )
-        self.send_command("BUILD", *build_args)
-        reply = self.read_reply()[0]
+        await self.send_command("BUILD", *build_args)
+        reply = (await self.read_reply())[0]
         if reply != "BUILDING":
             return reply, None
         print("Build in progress. This may take 10-15 minutes.")
         if log:
-            line = self.read_line()
+            line = await self.read_line()
             while line != "" and line.startswith("LOG"):
                 print(line[4:], end="")
-                line = self.read_line()
+                line = await self.read_line()
             reply, status = line.split()
         else:
-            reply, status = self.read_reply()
+            reply, status = await self.read_reply()
         if reply != "DONE":
             raise ValueError("Unexpected server reply: expected 'DONE', got '{}'".format(reply))
         if status != "done":
             return status, None
         print("Build completed. Downloading...")
-        reply, length = self.read_reply()
+        reply, length = await self.read_reply()
         if reply != "PRODUCT":
             raise ValueError("Unexpected server reply: expected 'PRODUCT', got '{}'".format(reply))
         length = int(length)
@@ -123,25 +129,25 @@ class Client:
             total = 0
             while total != length:
                 chunk_len = min(4096, length-total)
-                contents += self.fsocket.read(chunk_len)
+                contents += await self.reader.read(chunk_len)
                 total += chunk_len
                 progress_bar.update(chunk_len)
         print("Download completed.")
         return "OK", contents
 
-    def passwd(self, password):
-        self.send_command("PASSWD", password)
-        return self.read_reply() == ["OK"]
+    async def passwd(self, password):
+        await self.send_command("PASSWD", password)
+        return (await self.read_reply()) == ["OK"]
     
-    def get_variants(self):
-        self.send_command("GET_VARIANTS")
-        reply = self.read_reply()[0]
+    async def get_variants(self):
+        await self.send_command("GET_VARIANTS")
+        reply = (await self.read_reply())[0]
         if reply != "OK":
             raise ValueError("Unexpected server reply: expected 'OK', got '{}'".format(reply))
-        return self.read_json()
+        return await self.read_json()
 
-    def get_single_variant(self, error_msg):
-        variants = self.get_variants()
+    async def get_single_variant(self, error_msg):
+        variants = await self.get_variants()
         if len(variants) != 1:
             print(error_msg)
             table = PrettyTable()
@@ -152,17 +158,17 @@ class Client:
             sys.exit(1)
         return variants[0][0]
 
-    def get_json(self, variant):
-        self.send_command("GET_JSON", variant)
-        reply = self.read_reply()
+    async def get_json(self, variant):
+        await self.send_command("GET_JSON", variant)
+        reply = await self.read_reply()
         if reply[0] != "OK":
             return reply[0], None
         length = int(reply[1])
-        json_str = self.fsocket.read(length).decode("ascii")
+        json_str = (await self.reader.read(length)).decode("ascii")
         return "OK", json_str
 
 
-def main():
+async def main_async():
     parser = argparse.ArgumentParser()
     parser.add_argument("--server", default="afws.m-labs.hk", help="server to connect to (default: %(default)s)")
     parser.add_argument("--port", default=80, type=int, help="port to connect to (default: %(default)d)")
@@ -186,6 +192,7 @@ def main():
     args = parser.parse_args()
 
     client = Client(args.server, args.port, args.cert)
+    await client.connect()
     try:
         if args.action == "build":
             # do this before user enters password so errors are reported without unnecessary user action
@@ -216,7 +223,7 @@ def main():
             password = getpass("Current password: ")
         else:
             password = getpass()
-        if not client.login(args.username, password):
+        if not await client.login(args.username, password):
             print("Login failed")
             sys.exit(1)
 
@@ -229,12 +236,12 @@ def main():
                 print("Passwords do not match")
                 password = getpass("New password: ")
                 password_confirm = getpass("New password (again): ")
-            if not client.passwd(password):
+            if not await client.passwd(password):
                 print("Failed to change password")
                 sys.exit(1)
         elif args.action == "build":
             # build dir and version variables set up above
-            result, contents = client.build(major_ver, rev, args.variant, args.log, args.experimental)
+            result, contents = await client.build(major_ver, rev, args.variant, args.log, args.experimental)
             if result != "OK":
                 if result == "UNAUTHORIZED":
                     print("You are not authorized to build this variant. Your firmware subscription may have expired. Contact helpdesk\x40m-labs.hk.")
@@ -245,7 +252,7 @@ def main():
                 sys.exit(1)
             zip_unarchive(contents, args.directory)
         elif args.action == "get_variants":
-            variants = client.get_variants()
+            variants = await client.get_variants()
             table = PrettyTable()
             table.field_names = ["Variant", "Expiry date"]
             for variant in variants:
@@ -255,8 +262,8 @@ def main():
             if args.variant:
                 variant = args.variant
             else:
-                variant = client.get_single_variant(error_msg="User can get JSON of more than 1 variant - need to specify")
-            result, json_str = client.get_json(variant)
+                variant = await client.get_single_variant(error_msg="User can get JSON of more than 1 variant - need to specify")
+            result, json_str = await client.get_json(variant)
             if result != "OK":
                 if result == "UNAUTHORIZED":
                     print(f"You are not authorized to get JSON of variant {variant}. Your firmware subscription may have expired. Contact helpdesk\x40m-labs.hk.")
@@ -272,8 +279,10 @@ def main():
         else:
             raise ValueError
     finally:
-        client.close()
+        await client.close()
 
+def main():
+    asyncio.run(main_async())
 
 if __name__ == "__main__":
     main()

artiq/frontend/aqctl_coreanalyzer_proxy.py

@@ -9,7 +9,7 @@ from sipyco.asyncio_tools import AsyncioServer, SignalHandler, atexit_register_c
 from sipyco.pc_rpc import Server
 from sipyco import common_args
 
-from artiq.coredevice.comm_analyzer import get_analyzer_dump
+from artiq.coredevice.comm_analyzer import get_analyzer_dump, ANALYZER_MAGIC
 
 
 logger = logging.getLogger(__name__)
@@ -24,6 +24,7 @@ class ProxyServer(AsyncioServer):
 
     async def _handle_connection_cr(self, reader, writer):
         try:
+            writer.write(ANALYZER_MAGIC)
             queue = asyncio.Queue(self._queue_limit)
             self._recipients.add(queue)
             try:
@@ -60,9 +61,7 @@ class ProxyControl:
             self.distribute_cb(dump)
         except:
             logger.warning("Trigger failed:", exc_info=True)
-            return False
-        else:
-            return True
+            raise
 
 
 def get_argparser():

artiq/frontend/artiq_client.py

@@ -22,6 +22,7 @@ from sipyco.pc_rpc import Client
 from sipyco.sync_struct import Subscriber
 from sipyco.broadcast import Receiver
 from sipyco import common_args, pyon
+from sipyco.asyncio_tools import SignalHandler
 
 from artiq.tools import (scale_from_metadata, short_format, parse_arguments,
                          parse_devarg_override)
@@ -112,11 +113,25 @@ def get_argparser():
         "del-dataset", help="delete a dataset")
     parser_del_dataset.add_argument("name", help="name of the dataset")
 
+    parser_supply_interactive = subparsers.add_parser(
+        "supply-interactive", help="supply interactive arguments")
+    parser_supply_interactive.add_argument(
+        "rid", metavar="RID", type=int, help="RID of target experiment")
+    parser_supply_interactive.add_argument(
+        "arguments", metavar="ARGUMENTS", nargs="*",
+        help="interactive arguments")
+
+    parser_cancel_interactive = subparsers.add_parser(
+        "cancel-interactive", help="cancel interactive arguments")
+    parser_cancel_interactive.add_argument(
+        "rid", metavar="RID", type=int, help="RID of target experiment")
+
     parser_show = subparsers.add_parser(
         "show", help="show schedule, log, devices or datasets")
     parser_show.add_argument(
         "what", metavar="WHAT",
-        choices=["schedule", "log", "ccb", "devices", "datasets"],
+        choices=["schedule", "log", "ccb", "devices", "datasets",
+                 "interactive-args"],
         help="select object to show: %(choices)s")
 
     subparsers.add_parser(
@@ -135,8 +150,7 @@ def get_argparser():
         "ls", help="list a directory on the master")
     parser_ls.add_argument("directory", default="", nargs="?")
 
-    subparsers.add_parser(
-        "terminate", help="terminate the ARTIQ master")
+    subparsers.add_parser("terminate", help="terminate the ARTIQ master")
 
     common_args.verbosity_args(parser)
     return parser
@@ -208,6 +222,15 @@ def _action_scan_devices(remote, args):
     remote.scan()
 
 
+def _action_supply_interactive(remote, args):
+    arguments = parse_arguments(args.arguments)
+    remote.supply(args.rid, arguments)
+
+
+def _action_cancel_interactive(remote, args):
+    remote.cancel(args.rid)
+
+
 def _action_scan_repository(remote, args):
     if getattr(args, "async"):
         remote.scan_repository_async(args.revision)
@@ -274,17 +297,34 @@ def _show_datasets(datasets):
     print(table)
 
 
+def _show_interactive_args(interactive_args):
+    clear_screen()
+    table = PrettyTable(["RID", "Title", "Key", "Type", "Group", "Tooltip"])
+    for rid, input_request in sorted(interactive_args.items(), key=itemgetter(0)):
+        title = input_request["title"]
+        for key, procdesc, group, tooltip in input_request["arglist_desc"]:
+            table.add_row([rid, title, key, procdesc["ty"], group, tooltip])
+    print(table)
+
+
 def _run_subscriber(host, port, subscriber):
     loop = asyncio.new_event_loop()
     asyncio.set_event_loop(loop)
+    try:
+        signal_handler = SignalHandler()
+        signal_handler.setup()
         try:
             loop.run_until_complete(subscriber.connect(host, port))
             try:
-            loop.run_until_complete(asyncio.wait_for(subscriber.receive_task,
-                                                     None))
-            print("Connection to master lost")
+                _, pending = loop.run_until_complete(asyncio.wait(
+                    [loop.create_task(signal_handler.wait_terminate()), subscriber.receive_task],
+                    return_when=asyncio.FIRST_COMPLETED))
+                for task in pending:
+                    task.cancel()
             finally:
                 loop.run_until_complete(subscriber.close())
+        finally:
+            signal_handler.teardown()
     finally:
         loop.close()
 
@@ -338,18 +378,22 @@ def main():
             _show_dict(args, "devices", _show_devices)
         elif args.what == "datasets":
             _show_dict(args, "datasets", _show_datasets)
+        elif args.what == "interactive-args":
+            _show_dict(args, "interactive_args", _show_interactive_args)
         else:
             raise ValueError
     else:
         port = 3251 if args.port is None else args.port
         target_name = {
-            "submit": "master_schedule",
-            "delete": "master_schedule",
-            "set_dataset": "master_dataset_db",
-            "del_dataset": "master_dataset_db",
-            "scan_devices": "master_device_db",
-            "scan_repository": "master_experiment_db",
-            "ls": "master_experiment_db",
+            "submit": "schedule",
+            "delete": "schedule",
+            "set_dataset": "dataset_db",
+            "del_dataset": "dataset_db",
+            "scan_devices": "device_db",
+            "supply_interactive": "interactive_arg_db",
+            "cancel_interactive": "interactive_arg_db",
+            "scan_repository": "experiment_db",
+            "ls": "experiment_db",
             "terminate": "master_management",
         }[action]
         remote = Client(args.server, port, target_name)

artiq/frontend/artiq_compile.py

@@ -67,12 +67,21 @@ def main():
                 core.compile(exp.run, [exp_inst], {},
                              attribute_writeback=False, print_as_rpc=False)
 
-            subkernels = {}
-            for sid, subkernel_fn in object_map.subkernels().items():
-                destination, subkernel_library = core.compile_subkernel(
+            subkernels = object_map.subkernels()
+            compiled_subkernels = {}
+            while True:
+                new_subkernels = {}
+                for sid, subkernel_fn in subkernels.items():
+                    if sid in compiled_subkernels.keys():
+                        continue
+                    destination, subkernel_library, embedding_map = core.compile_subkernel(
                         sid, subkernel_fn, object_map, 
-                    [exp_inst], subkernel_arg_types)
-                subkernels[sid] = (destination, subkernel_library)
+                        [exp_inst], subkernel_arg_types, subkernels)
+                    compiled_subkernels[sid] = (destination, subkernel_library)
+                    new_subkernels.update(embedding_map.subkernels())
+                if new_subkernels == subkernels:
+                    break
+                subkernels.update(new_subkernels)
         except CompileError as error:
             return
         finally:
@@ -107,7 +116,7 @@ def main():
             tar.addfile(main_kernel_info, fileobj=main_kernel_fileobj)
 
             # subkernels as "<sid> <destination>.elf"
-            for sid, (destination, subkernel_library) in subkernels.items():
+            for sid, (destination, subkernel_library) in compiled_subkernels.items():
                 subkernel_fileobj = io.BytesIO(subkernel_library)
                 subkernel_info = tarfile.TarInfo(name="{} {}.elf".format(sid, destination))
                 subkernel_info.size = len(subkernel_library)

artiq/frontend/artiq_dashboard.py

@@ -6,7 +6,6 @@ import atexit
 import importlib
 import os
 import logging
-import sys
 
 from PyQt5 import QtCore, QtGui, QtWidgets
 from qasync import QEventLoop
@@ -15,13 +14,15 @@ from sipyco.pc_rpc import AsyncioClient, Client
 from sipyco.broadcast import Receiver
 from sipyco import common_args
 from sipyco.asyncio_tools import atexit_register_coroutine
+from sipyco.sync_struct import Subscriber
 
 from artiq import __artiq_dir__ as artiq_dir, __version__ as artiq_version
 from artiq.tools import get_user_config_dir
 from artiq.gui.models import ModelSubscriber
 from artiq.gui import state, log
 from artiq.dashboard import (experiments, shortcuts, explorer,
-                             moninj, datasets, schedule, applets_ccb)
+                             moninj, datasets, schedule, applets_ccb,
+                             waveform, interactive_args)
 
 
 def get_argparser():
@@ -47,6 +48,15 @@ def get_argparser():
     parser.add_argument(
         "-p", "--load-plugin", dest="plugin_modules", action="append",
         help="Python module to load on startup")
+    parser.add_argument(
+        "--analyzer-proxy-timeout", default=5, type=float,
+        help="connection timeout to core analyzer proxy")
+    parser.add_argument(
+        "--analyzer-proxy-timer", default=5, type=float,
+        help="retry timer to core analyzer proxy")
+    parser.add_argument(
+        "--analyzer-proxy-timer-backoff", default=1.1, type=float,
+        help="retry timer backoff multiplier to core analyzer proxy")
     common_args.verbosity_args(parser)
     return parser
 
@@ -130,10 +140,10 @@ def main():
 
     # create connections to master
     rpc_clients = dict()
-    for target in "schedule", "experiment_db", "dataset_db", "device_db":
+    for target in "schedule", "experiment_db", "dataset_db", "device_db", "interactive_arg_db":
         client = AsyncioClient()
         loop.run_until_complete(client.connect_rpc(
-            args.server, args.port_control, "master_" + target))
+            args.server, args.port_control, target))
         atexit.register(client.close_rpc)
         rpc_clients[target] = client
 
@@ -144,6 +154,7 @@ def main():
         master_management.close_rpc()
 
     disconnect_reported = False
+
     def report_disconnect():
         nonlocal disconnect_reported
         if not disconnect_reported:
@@ -155,9 +166,9 @@ def main():
     for notifier_name, modelf in (("explist", explorer.Model),
                                   ("explist_status", explorer.StatusUpdater),
                                   ("datasets", datasets.Model),
-                                  ("schedule", schedule.Model)):
-        subscriber = ModelSubscriber(notifier_name, modelf,
-            report_disconnect)
+                                  ("schedule", schedule.Model),
+                                  ("interactive_args", interactive_args.Model)):
+        subscriber = ModelSubscriber(notifier_name, modelf, report_disconnect)
         loop.run_until_complete(subscriber.connect(
             args.server, args.port_notify))
         atexit_register_coroutine(subscriber.close, loop=loop)
@@ -215,10 +226,22 @@ def main():
     smgr.register(d_applets)
     broadcast_clients["ccb"].notify_cbs.append(d_applets.ccb_notify)
 
-    d_ttl_dds = moninj.MonInj(rpc_clients["schedule"])
-    loop.run_until_complete(d_ttl_dds.start(args.server, args.port_notify))
+    d_ttl_dds = moninj.MonInj(rpc_clients["schedule"], main_window)
+    smgr.register(d_ttl_dds)
     atexit_register_coroutine(d_ttl_dds.stop, loop=loop)
 
+    d_waveform = waveform.WaveformDock(
+        args.analyzer_proxy_timeout,
+        args.analyzer_proxy_timer,
+        args.analyzer_proxy_timer_backoff
+    )
+    atexit_register_coroutine(d_waveform.stop, loop=loop)
+
+    d_interactive_args = interactive_args.InteractiveArgsDock(
+        sub_clients["interactive_args"],
+        rpc_clients["interactive_arg_db"]
+    )
+
     d_schedule = schedule.ScheduleDock(
         rpc_clients["schedule"], sub_clients["schedule"])
     smgr.register(d_schedule)
@@ -231,8 +254,8 @@ def main():
     # lay out docks
     right_docks = [
         d_explorer, d_shortcuts,
-        d_ttl_dds.ttl_dock, d_ttl_dds.dds_dock, d_ttl_dds.dac_dock,
-        d_datasets, d_applets
+        d_datasets, d_applets,
+        d_waveform, d_interactive_args
     ]
     main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, right_docks[0])
     for d1, d2 in zip(right_docks, right_docks[1:]):
@@ -246,18 +269,25 @@ def main():
         # QDockWidgets fail to be embedded.
         main_window.show()
     smgr.load()
+
+    def init_cbs(ddb):
+        d_ttl_dds.dm.init_ddb(ddb)
+        d_waveform.init_ddb(ddb)
+        return ddb
+    devices_sub = Subscriber("devices", init_cbs, [d_ttl_dds.dm.notify_ddb, d_waveform.notify_ddb])
+    loop.run_until_complete(devices_sub.connect(args.server, args.port_notify))
+    atexit_register_coroutine(devices_sub.close, loop=loop)
+
     smgr.start(loop=loop)
     atexit_register_coroutine(smgr.stop, loop=loop)
 
-    # work around for https://github.com/m-labs/artiq/issues/1307
-    d_ttl_dds.ttl_dock.show()
-    d_ttl_dds.dds_dock.show()
-
     # create first log dock if not already in state
     d_log0 = logmgr.first_log_dock()
     if d_log0 is not None:
         main_window.tabifyDockWidget(d_schedule, d_log0)
-
+    d_moninj0 = d_ttl_dds.first_moninj_dock()
+    if d_moninj0 is not None:
+        main_window.tabifyDockWidget(right_docks[-1], d_moninj0)
 
     if server_name is not None:
         server_description = server_name + " ({})".format(args.server)
@@ -269,5 +299,6 @@ def main():
     main_window.show()
     loop.run_until_complete(main_window.exit_request.wait())
 
+
 if __name__ == "__main__":
     main()

artiq/frontend/artiq_ddb_template.py

@@ -211,6 +211,11 @@ class PeripheralManager:
         urukul_name = self.get_name("urukul")
         synchronization = peripheral["synchronization"]
         channel = count(0)
+        pll_en = peripheral["pll_en"]
+        clk_div = peripheral.get("clk_div")
+        if clk_div is None:
+            clk_div = 0 if pll_en else 1
+
         self.gen("""
             device_db["eeprom_{name}"] = {{
                 "type": "local",
@@ -277,7 +282,7 @@ class PeripheralManager:
             sync_device="\"ttl_{name}_sync\"".format(name=urukul_name) if synchronization else "None",
             refclk=peripheral.get("refclk", self.primary_description["rtio_frequency"]),
             clk_sel=peripheral["clk_sel"],
-            clk_div=peripheral["clk_div"])
+            clk_div=clk_div)
         dds = peripheral["dds"]
         pll_vco = peripheral.get("pll_vco")
         for i in range(4):
@@ -299,7 +304,7 @@ class PeripheralManager:
                     uchn=i,
                     sw=",\n        \"sw_device\": \"ttl_{name}_sw{uchn}\"".format(name=urukul_name, uchn=i) if len(peripheral["ports"]) > 1 else "",
                     pll_vco=",\n        \"pll_vco\": {}".format(pll_vco) if pll_vco is not None else "",
-                    pll_n=peripheral.get("pll_n", 32), pll_en=peripheral["pll_en"],
+                    pll_n=peripheral.get("pll_n", 32), pll_en=pll_en,
                     sync_delay_seed=",\n        \"sync_delay_seed\": \"eeprom_{}:{}\"".format(urukul_name, 64 + 4*i) if synchronization else "",
                     io_update_delay=",\n        \"io_update_delay\": \"eeprom_{}:{}\"".format(urukul_name, 64 + 4*i) if synchronization else "")
             elif dds == "ad9912":
@@ -320,7 +325,7 @@ class PeripheralManager:
                     uchn=i,
                     sw=",\n        \"sw_device\": \"ttl_{name}_sw{uchn}\"".format(name=urukul_name, uchn=i) if len(peripheral["ports"]) > 1 else "",
                     pll_vco=",\n        \"pll_vco\": {}".format(pll_vco) if pll_vco is not None else "",
-                    pll_n=peripheral.get("pll_n", 8), pll_en=peripheral["pll_en"])
+                    pll_n=peripheral.get("pll_n", 8), pll_en=pll_en)
             else:
                 raise ValueError
         return next(channel)

artiq/frontend/artiq_flash.py

@@ -261,12 +261,19 @@ def main():
             "storage":      ("spi0", 0x440000),
             "firmware":     ("spi0", 0x450000),
         },
-        "efc": {
+        "efc1v0": {
             "programmer":   partial(ProgrammerXC7, board="efc", proxy="bscan_spi_xc7a100t.bit"),
             "gateware":     ("spi0", 0x000000),
-            "bootloader":   ("spi0", 0x400000),
-            "storage":      ("spi0", 0x440000),
-            "firmware":     ("spi0", 0x450000),
+            "bootloader":   ("spi0", 0x600000),
+            "storage":      ("spi0", 0x640000),
+            "firmware":     ("spi0", 0x650000),
+        },
+        "efc1v1": {
+            "programmer":   partial(ProgrammerXC7, board="efc", proxy="bscan_spi_xc7a200t.bit"),
+            "gateware":     ("spi0", 0x000000),
+            "bootloader":   ("spi0", 0x600000),
+            "storage":      ("spi0", 0x640000),
+            "firmware":     ("spi0", 0x650000),
         },
         "kc705": {
             "programmer":   partial(ProgrammerXC7, board="kc705", proxy="bscan_spi_xc7k325t.bit"),

artiq/frontend/artiq_master.py

@@ -15,7 +15,8 @@ from sipyco.asyncio_tools import atexit_register_coroutine, SignalHandler
 
 from artiq import __version__ as artiq_version
 from artiq.master.log import log_args, init_log
-from artiq.master.databases import DeviceDB, DatasetDB
+from artiq.master.databases import (DeviceDB, DatasetDB,
+                                    InteractiveArgDB)
 from artiq.master.scheduler import Scheduler
 from artiq.master.rid_counter import RIDCounter
 from artiq.master.experiments import (FilesystemBackend, GitBackend,
@@ -60,7 +61,7 @@ def get_argparser():
                         help="friendly name, displayed in dashboards "
                              "to identify master instead of server address")
     parser.add_argument("--log-submissions", default=None,
-        help="set the filename to create the experiment subimission")
+                        help="log experiment submissions to specified file")
 
     return parser
 
@@ -81,8 +82,7 @@ def main():
         bind, args.port_broadcast))
     atexit_register_coroutine(server_broadcast.stop, loop=loop)
 
-    log_forwarder.callback = (lambda msg:
-        server_broadcast.broadcast("log", msg))
+    log_forwarder.callback = lambda msg: server_broadcast.broadcast("log", msg)
     def ccb_issue(service, *args, **kwargs):
         msg = {
             "service": service,
@@ -96,6 +96,7 @@ def main():
     atexit.register(dataset_db.close_db)
     dataset_db.start(loop=loop)
     atexit_register_coroutine(dataset_db.stop, loop=loop)
+    interactive_arg_db = InteractiveArgDB()
     worker_handlers = dict()
 
     if args.git:
@@ -106,15 +107,22 @@ def main():
         repo_backend, worker_handlers, args.experiment_subdir)
     atexit.register(experiment_db.close)
 
-    scheduler = Scheduler(RIDCounter(), worker_handlers, experiment_db, args.log_submissions)
+    scheduler = Scheduler(RIDCounter(), worker_handlers, experiment_db,
+                          args.log_submissions)
     scheduler.start(loop=loop)
     atexit_register_coroutine(scheduler.stop, loop=loop)
 
+    # Python doesn't allow writing attributes to bound methods.
+    def get_interactive_arguments(*args, **kwargs):
+        return interactive_arg_db.get(*args, **kwargs)
+    get_interactive_arguments._worker_pass_rid = True
     worker_handlers.update({
         "get_device_db": device_db.get_device_db,
         "get_device": device_db.get,
         "get_dataset": dataset_db.get,
+        "get_dataset_metadata": dataset_db.get_metadata,
         "update_dataset": dataset_db.update,
+        "get_interactive_arguments": get_interactive_arguments,
         "scheduler_submit": scheduler.submit,
         "scheduler_delete": scheduler.delete,
         "scheduler_request_termination": scheduler.request_termination,
@@ -133,10 +141,11 @@ def main():
 
     server_control = RPCServer({
         "master_management": master_management,
-        "master_device_db": device_db,
-        "master_dataset_db": dataset_db,
-        "master_schedule": scheduler,
-        "master_experiment_db": experiment_db,
+        "device_db": device_db,
+        "dataset_db": dataset_db,
+        "interactive_arg_db": interactive_arg_db,
+        "schedule": scheduler,
+        "experiment_db": experiment_db,
     }, allow_parallel=True)
     loop.run_until_complete(server_control.start(
         bind, args.port_control))
@@ -146,8 +155,9 @@ def main():
         "schedule": scheduler.notifier,
         "devices": device_db.data,
         "datasets": dataset_db.data,
+        "interactive_args": interactive_arg_db.pending,
         "explist": experiment_db.explist,
-        "explist_status": experiment_db.status
+        "explist_status": experiment_db.status,
     })
     loop.run_until_complete(server_notify.start(
         bind, args.port_notify))

artiq/frontend/artiq_run.py

@@ -13,7 +13,7 @@ import h5py
 
 from llvmlite import binding as llvm
 
-from sipyco import common_args
+from sipyco import common_args, pyon
 
 from artiq import __version__ as artiq_version
 from artiq.language.environment import EnvExperiment, ProcessArgumentManager
@@ -166,9 +166,30 @@ def get_argparser(with_file=True):
     return parser
 
 
+class ArgumentManager(ProcessArgumentManager):
+    def get_interactive(self, interactive_arglist, title):
+        print(title)
+        result = dict()
+        for key, processor, group, tooltip in interactive_arglist:
+            success = False
+            while not success:
+                user_input = input("{}:{} (group={}, tooltip={}): ".format(
+                    key, type(processor).__name__, group, tooltip))
+                try:
+                    user_input_deser = pyon.decode(user_input)
+                    value = processor.process(user_input_deser)
+                except:
+                    logger.error("failed to process user input, retrying",
+                                 exc_info=True)
+                else:
+                    success = True
+            result[key] = value
+        return result
+
+
 def _build_experiment(device_mgr, dataset_mgr, args):
     arguments = parse_arguments(args.arguments)
-    argument_mgr = ProcessArgumentManager(arguments)
+    argument_mgr = ArgumentManager(arguments)
     managers = (device_mgr, dataset_mgr, argument_mgr, {})
     if hasattr(args, "file"):
         is_tar = tarfile.is_tarfile(args.file)

artiq/frontend/artiq_sinara_tester.py

@@ -9,6 +9,7 @@ import sys
 from artiq.experiment import *
 from artiq.coredevice.ad9910 import AD9910, SyncDataEeprom
 from artiq.coredevice.phaser import PHASER_GW_BASE, PHASER_GW_MIQRO
+from artiq.coredevice.shuttler import shuttler_volt_to_mu
 from artiq.master.databases import DeviceDB
 from artiq.master.worker_db import DeviceManager
 
@@ -60,7 +61,9 @@ class SinaraTester(EnvExperiment):
         self.mirnies = dict()
         self.suservos = dict()
         self.suschannels = dict()
+        self.legacy_almaznys = dict()
         self.almaznys = dict()
+        self.shuttler = dict()
 
         ddb = self.get_device_db()
         for name, desc in ddb.items():
@@ -99,7 +102,20 @@ class SinaraTester(EnvExperiment):
                 elif (module, cls) == ("artiq.coredevice.suservo", "Channel"):
                     self.suschannels[name] = self.get_device(name)
                 elif (module, cls) == ("artiq.coredevice.almazny", "AlmaznyLegacy"):
+                    self.legacy_almaznys[name] = self.get_device(name)
+                elif (module, cls) == ("artiq.coredevice.almazny", "AlmaznyChannel"):
                     self.almaznys[name] = self.get_device(name)
+                elif (module, cls) == ("artiq.coredevice.shuttler", "Config"):
+                    shuttler_name  = name.replace("_config", "")
+                    self.shuttler[shuttler_name] = ({
+                        "config": self.get_device(name),
+                        "trigger": self.get_device("{}_trigger".format(shuttler_name)),
+                        "leds": [self.get_device("{}_led{}".format(shuttler_name, i)) for i in range(2)],
+                        "dcbias": [self.get_device("{}_dcbias{}".format(shuttler_name, i)) for i in range(16)],
+                        "dds": [self.get_device("{}_dds{}".format(shuttler_name, i)) for i in range(16)],
+                        "relay": self.get_device("{}_relay".format(shuttler_name)),
+                        "adc": self.get_device("{}_adc".format(shuttler_name)),
+                    })
 
         # Remove Urukul, Sampler, Zotino and Mirny control signals
         # from TTL outs (tested separately) and remove Urukuls covered by
@@ -148,6 +164,7 @@ class SinaraTester(EnvExperiment):
         self.mirnies = sorted(self.mirnies.items(), key=lambda x: (x[1].cpld.bus.channel, x[1].channel))
         self.suservos = sorted(self.suservos.items(), key=lambda x: x[1].channel)
         self.suschannels = sorted(self.suschannels.items(), key=lambda x: x[1].channel)
+        self.shuttler = sorted(self.shuttler.items(), key=lambda x: x[1]["leds"][0].channel)
 
     @kernel
     def test_led(self, led):
@@ -356,67 +373,111 @@ class SinaraTester(EnvExperiment):
                 channel.pulse(100*ms)
                 delay(100*ms)
     @kernel
-    def init_almazny(self, almazny):
+    def init_legacy_almazny(self, almazny):
         self.core.break_realtime()
         almazny.init()
         almazny.output_toggle(True)
 
     @kernel
-    def almazny_set_attenuators_mu(self, almazny, ch, atts):
+    def legacy_almazny_set_attenuators_mu(self, almazny, ch, atts):
         self.core.break_realtime()
         almazny.set_att_mu(ch, atts)
 
     @kernel
-    def almazny_set_attenuators(self, almazny, ch, atts):
+    def legacy_almazny_att_test(self, almazny):
+        # change attenuation bit by bit over time for all channels
+        att_mu = 0
+        while not is_enter_pressed():
             self.core.break_realtime()
-        almazny.set_att(ch, atts)
+            t = now_mu() - self.core.seconds_to_mu(0.5)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in range(4):
+                almazny.set_att_mu(ch, att_mu)
+            delay(250*ms)
+            if att_mu == 0:
+                att_mu = 1
+            else:
+                att_mu = (att_mu << 1) & 0x3F
     
     @kernel
-    def almazny_toggle_output(self, almazny, rf_on):
+    def legacy_almazny_toggle_output(self, almazny, rf_on):
         self.core.break_realtime()
         almazny.output_toggle(rf_on)
 
-    def test_almaznys(self):
-        print("*** Testing Almaznys.")
-        for name, almazny in sorted(self.almaznys.items(), key=lambda x: x[0]):
+    def test_legacy_almaznys(self):
+        print("*** Testing legacy Almaznys (v1.1 or older).")
+        for name, almazny in sorted(self.legacy_almaznys.items(), key=lambda x: x[0]):
             print(name + "...")
             print("Initializing Mirny CPLDs...")
             for name, cpld in sorted(self.mirny_cplds.items(), key=lambda x: x[0]):
                 print(name + "...")
                 self.init_mirny(cpld)
             print("...done")
-
             print("Testing attenuators. Frequencies:")
             for card_n, channels in enumerate(chunker(self.mirnies, 4)):
                 for channel_n, (channel_name, channel_dev) in enumerate(channels):
                     frequency = 2000 + card_n * 250 + channel_n * 50
                     print("{}\t{}MHz".format(channel_name, frequency*2))
                     self.setup_mirny(channel_dev, frequency)
-                    print("{} info: {}".format(channel_name, channel_dev.info()))
-            self.init_almazny(almazny)
-            print("RF ON, all attenuators ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators_mu(almazny, i, 63)
-            input()
-            print("RF ON, half power attenuators ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 15.5)
-            input()
-            print("RF ON, all attenuators OFF. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 0)
-            input()
+            self.init_legacy_almazny(almazny)
             print("SR outputs are OFF. Press ENTER when done.")
-            self.almazny_toggle_output(almazny, False)
-            input()
-            print("RF ON, all attenuators are ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 31.5)
-            self.almazny_toggle_output(almazny, True)
-            input()
-            print("RF OFF. Press ENTER when done.")
-            self.almazny_toggle_output(almazny, False)
+            self.legacy_almazny_toggle_output(almazny, False)
             input()
+            print("RF ON, attenuators are tested. Press ENTER when done.")
+            self.legacy_almazny_toggle_output(almazny, True)
+            self.legacy_almazny_att_test(almazny)
+            self.legacy_almazny_toggle_output(almazny, False)
+
+    @kernel
+    def almazny_led_wave(self, almaznys):
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            # do not fill the FIFOs too much to avoid long response times
+            t = now_mu() - self.core.seconds_to_mu(0.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in almaznys:
+                ch.set(31.5, False, True)
+                delay(100*ms)
+                ch.set(31.5, False, False)
+    
+    @kernel
+    def almazny_att_test(self, almaznys):
+        rf_en = 1
+        led = 1
+        att_mu = 0
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            t = now_mu() - self.core.seconds_to_mu(0.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            setting = led << 7 | rf_en << 6 | (att_mu & 0x3F)
+            for ch in almaznys:
+                ch.set_mu(setting)
+            delay(250*ms)
+            if att_mu == 0:
+                att_mu = 1
+            else:
+                att_mu = (att_mu << 1) & 0x3F
+
+    def test_almaznys(self):
+        print("*** Testing Almaznys (v1.2+).")
+        print("Initializing Mirny CPLDs...")
+        for name, cpld in sorted(self.mirny_cplds.items(), key=lambda x: x[0]):
+            print(name + "...")
+            self.init_mirny(cpld)
+        print("...done")
+        print("Frequencies:")
+        for card_n, channels in enumerate(chunker(self.mirnies, 4)):
+            for channel_n, (channel_name, channel_dev) in enumerate(channels):
+                frequency = 2000 + card_n * 250 + channel_n * 50
+                print("{}\t{}MHz".format(channel_name, frequency*2))
+                self.setup_mirny(channel_dev, frequency)
+        print("RF ON, attenuators are tested. Press ENTER when done.")
+        self.almazny_att_test([ch for _, ch in self.almaznys.items()])
+        print("RF OFF, testing LEDs. Press ENTER when done.")
+        self.almazny_led_wave([ch for _, ch in self.almaznys.items()])
 
     def test_mirnies(self):
         print("*** Testing Mirny PLLs.")
@@ -747,6 +808,125 @@ class SinaraTester(EnvExperiment):
         print("Press ENTER when done.")
         input()
 
+    @kernel
+    def setup_shuttler_init(self, relay, adc, dcbias, dds, trigger, config):
+        self.core.break_realtime()
+        # Reset Shuttler Output Relay
+        relay.init()
+        delay_mu(int64(self.core.ref_multiplier))
+
+        relay.enable(0x0000)
+        delay_mu(int64(self.core.ref_multiplier))
+
+        # Setup ADC and and Calibration
+        delay_mu(int64(self.core.ref_multiplier))
+        adc.power_up()
+
+        delay_mu(int64(self.core.ref_multiplier))
+        if adc.read_id() >> 4 != 0x038d:
+            print("Remote AFE Board's ADC is not found. Check Remote AFE Board's Cables Connections")
+            assert adc.read_id() >> 4 == 0x038d
+
+        delay_mu(int64(self.core.ref_multiplier))
+        adc.calibrate(dcbias, trigger, config)
+
+        #Reset Shuttler DAC Output
+        for ch in range(16):
+            self.setup_shuttler_set_output(dcbias, dds, trigger, ch, 0.0)
+
+    @kernel 
+    def set_shuttler_relay(self, relay, val):
+        self.core.break_realtime()
+        relay.enable(val)
+
+    @kernel
+    def get_shuttler_output_voltage(self, adc, ch, cb):
+        self.core.break_realtime()
+        cb(adc.read_ch(ch))
+
+    @kernel
+    def setup_shuttler_set_output(self, dcbias, dds, trigger, ch, volt):
+        self.core.break_realtime()
+        dcbias[ch].set_waveform(
+            a0=shuttler_volt_to_mu(volt),
+            a1=0,
+            a2=0,
+            a3=0,
+        )
+        delay_mu(int64(self.core.ref_multiplier))
+
+        dds[ch].set_waveform(
+            b0=0,
+            b1=0,
+            b2=0,
+            b3=0,
+            c0=0,
+            c1=0,
+            c2=0,
+        )
+        delay_mu(int64(self.core.ref_multiplier))
+
+        trigger.trigger(1 << ch)
+        delay_mu(int64(self.core.ref_multiplier))
+
+    @kernel
+    def shuttler_relay_led_wave(self, relay):
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            # do not fill the FIFOs too much to avoid long response times
+            t = now_mu() - self.core.seconds_to_mu(.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in range(16):
+                relay.enable(1 << ch)
+                delay(100*ms)
+            relay.enable(0x0000)
+            delay(100*ms)
+
+    def test_shuttler(self):
+        print("*** Testing Shuttler.")
+
+        for card_n, (card_name, card_dev) in enumerate(self.shuttler):
+            print("Testing: ", card_name)
+
+            output_voltage = 0.0
+            def setv(x):
+                nonlocal output_voltage
+                output_voltage = x
+
+            self.setup_shuttler_init(card_dev["relay"], card_dev["adc"], card_dev["dcbias"], card_dev["dds"], card_dev["trigger"], card_dev["config"])
+            
+            print("Check Remote AFE Board Relay LED Indicators.")
+            print("Press Enter to Continue.")
+            self.shuttler_relay_led_wave(card_dev["relay"])
+
+            self.set_shuttler_relay(card_dev["relay"], 0xFFFF)
+            
+            passed = True
+            adc_readings = []
+            volt_set = [(-1)**i*(2.*card_n + .1*(i//2 + 1)) for i in range(16)]
+
+            print("Testing Shuttler DAC")
+            print("Voltages:", " ".join(["{:.1f}".format(x) for x in volt_set]))
+
+            for ch, volt in enumerate(volt_set):
+                self.setup_shuttler_set_output(card_dev["dcbias"], card_dev["dds"], card_dev["trigger"], ch, volt)
+                self.get_shuttler_output_voltage(card_dev["adc"], ch, setv)
+                if (abs(volt) - abs(output_voltage)) > 0.1:
+                    passed = False
+                adc_readings.append(output_voltage)
+
+            print("Press Enter to Continue.")
+            input()
+            self.set_shuttler_relay(card_dev["relay"], 0x0000)
+
+            if passed:
+                print("PASSED")
+            else:
+                print("FAILED")
+                print("Shuttler Remote AFE Board ADC has abnormal readings.")
+                print(f"ADC Readings:", " ".join(["{:.2f}".format(x) for x in adc_readings]))
+               
     def run(self, tests):
         print("****** Sinara system tester ******")
         print("")

artiq/gateware/drtio/aux_controller.py

@@ -10,6 +10,7 @@ from misoc.interconnect import wishbone
 
 
 max_packet = 1024
+aux_buffer_count = 8
 
 
 class Transmitter(Module, AutoCSR):
@@ -95,13 +96,13 @@ class Transmitter(Module, AutoCSR):
 
 class Receiver(Module, AutoCSR):
     def __init__(self, link_layer, min_mem_dw):
-        self.aux_rx_length = CSRStatus(bits_for(max_packet))
         self.aux_rx_present = CSR()
         self.aux_rx_error = CSR()
+        self.aux_read_pointer = CSR(log2_int(aux_buffer_count))
 
         ll_dw = len(link_layer.rx_aux_data)
         mem_dw = max(min_mem_dw, ll_dw)
-        self.specials.mem = Memory(mem_dw, max_packet//(mem_dw//8))
+        self.specials.mem = Memory(mem_dw, aux_buffer_count*max_packet//(mem_dw//8))
 
         converter = ClockDomainsRenamer("rtio_rx")(
             stream.Converter(ll_dw, mem_dw))
@@ -123,30 +124,45 @@ class Receiver(Module, AutoCSR):
         mem_port = self.mem.get_port(write_capable=True, clock_domain="rtio_rx")
         self.specials += mem_port
 
+        # write pointer represents where the gateware is
+        write_pointer = Signal(log2_int(aux_buffer_count))
+        write_pointer_sys = Signal.like(write_pointer)
+        # read pointer represents where CPU is
+        # write reaching read is an error, read reaching write is buffer clear
+        read_pointer = Signal.like(write_pointer)
+        read_pointer_rx = Signal.like(write_pointer)
+
+        read_pointer.attr.add("no_retiming")
+        write_pointer.attr.add("no_retiming")
+        signal_error = PulseSynchronizer("rtio_rx", "sys")
+
+        self.specials += [
+            MultiReg(read_pointer, read_pointer_rx, "rtio_rx"),
+            MultiReg(write_pointer, write_pointer_sys)
+        ]
+
         frame_counter_nbits = bits_for(max_packet) - log2_int(mem_dw//8)
         frame_counter = Signal(frame_counter_nbits)
+        # frame counter requires one more bit to represent overflow (bits_for)
+        # actual valid packet will be addressed with one bit less
+        packet_nbits = frame_counter_nbits - 1
         self.comb += [
-            mem_port.adr.eq(frame_counter),
+            mem_port.adr[:packet_nbits].eq(frame_counter),
+            # bits above the frame counter point to current frame
+            mem_port.adr[packet_nbits:].eq(write_pointer),
             mem_port.dat_w.eq(converter.source.data),
-            converter.source.ack.eq(1)
+            converter.source.ack.eq(1),
+            self.aux_read_pointer.w.eq(read_pointer)
         ]
 
-        frame_counter.attr.add("no_retiming")
-        frame_counter_sys = Signal(frame_counter_nbits)
-        self.specials += MultiReg(frame_counter, frame_counter_sys)
-        self.comb += self.aux_rx_length.status.eq(frame_counter_sys << log2_int(mem_dw//8))
-
-        signal_frame = PulseSynchronizer("rtio_rx", "sys")
-        frame_ack = PulseSynchronizer("sys", "rtio_rx")
-        signal_error = PulseSynchronizer("rtio_rx", "sys")
-        self.submodules += signal_frame, frame_ack, signal_error
+        self.submodules += signal_error
         self.sync += [
-            If(self.aux_rx_present.re, self.aux_rx_present.w.eq(0)),
-            If(signal_frame.o, self.aux_rx_present.w.eq(1)),
             If(self.aux_rx_error.re, self.aux_rx_error.w.eq(0)),
-            If(signal_error.o, self.aux_rx_error.w.eq(1))
+            If(signal_error.o, self.aux_rx_error.w.eq(1)),
+            self.aux_rx_present.w.eq(~(read_pointer == write_pointer_sys)),
+            If(self.aux_rx_present.re & self.aux_rx_present.w,
+                read_pointer.eq(read_pointer + 1)),
         ]
-        self.comb += frame_ack.i.eq(self.aux_rx_present.re)
 
         fsm = ClockDomainsRenamer("rtio_rx")(FSM(reset_state="IDLE"))
         self.submodules += fsm
@@ -171,7 +187,7 @@ class Receiver(Module, AutoCSR):
                     NextState("FRAME")
                 )
             ).Else(
-                NextValue(frame_counter, 0)
+                NextValue(frame_counter, 0),
             )
         )
         fsm.act("FRAME",
@@ -189,14 +205,12 @@ class Receiver(Module, AutoCSR):
             )
         )
         fsm.act("SIGNAL_FRAME",
-            signal_frame.i.eq(1),
-            NextState("WAIT_ACK"),
-            If(converter.source.stb, signal_error.i.eq(1))
-        )
-        fsm.act("WAIT_ACK",
-            If(frame_ack.o,
-                NextValue(frame_counter, 0), 
-                NextState("IDLE")
+            NextState("IDLE"),
+            If((write_pointer + 1) == read_pointer_rx,
+                # on full buffer, overwrite only current frame
+                signal_error.i.eq(1)
+            ).Else(
+                NextValue(write_pointer, write_pointer + 1)
             ),
             If(converter.source.stb, signal_error.i.eq(1))
         )
@@ -215,8 +229,8 @@ class DRTIOAuxController(Module):
         tx_sdram_if = wishbone.SRAM(self.transmitter.mem, read_only=False, data_width=dw)
         rx_sdram_if = wishbone.SRAM(self.receiver.mem, read_only=True, data_width=dw)
         decoder = wishbone.Decoder(self.bus,
-            [(lambda a: a[log2_int(max_packet)-wsb] == 0, tx_sdram_if.bus),
-             (lambda a: a[log2_int(max_packet)-wsb] == 1, rx_sdram_if.bus)],
+            [(lambda a: a[log2_int(max_packet*aux_buffer_count)-wsb] == 0, tx_sdram_if.bus),
+             (lambda a: a[log2_int(max_packet*aux_buffer_count)-wsb] == 1, rx_sdram_if.bus)],
             register=True)
         self.submodules += tx_sdram_if, rx_sdram_if, decoder
 

artiq/gateware/drtio/core.py

@@ -1,6 +1,7 @@
 from types import SimpleNamespace
 
 from migen import *
+from migen.genlib.cdc import MultiReg
 from migen.genlib.resetsync import AsyncResetSynchronizer
 from misoc.interconnect.csr import *
 
@@ -51,6 +52,7 @@ class SyncRTIO(Module):
     def __init__(self, tsc, channels, lane_count=8, fifo_depth=128):
         self.cri = cri.Interface()
         self.async_errors = Record(async_errors_layout)
+        self.sed_spread_enable = Signal()
 
         chan_fine_ts_width = max(max(rtlink.get_fine_ts_width(channel.interface.o)
                                      for channel in channels),
@@ -61,10 +63,11 @@ class SyncRTIO(Module):
         self.submodules.outputs = ClockDomainsRenamer("rio")(
             SED(channels, tsc.glbl_fine_ts_width,
                 lane_count=lane_count, fifo_depth=fifo_depth,
-                enable_spread=False, report_buffer_space=True,
+                fifo_high_watermark=0.75, report_buffer_space=True,
                 interface=self.cri))
         self.comb += self.outputs.coarse_timestamp.eq(tsc.coarse_ts)
         self.sync += self.outputs.minimum_coarse_timestamp.eq(tsc.coarse_ts + 16)
+        self.specials += MultiReg(self.sed_spread_enable, self.outputs.enable_spread, "rio")
 
         self.submodules.inputs = ClockDomainsRenamer("rio")(
             InputCollector(tsc, channels, interface=self.cri))
@@ -78,6 +81,7 @@ class DRTIOSatellite(Module):
         self.reset = CSRStorage(reset=1)
         self.reset_phy = CSRStorage(reset=1)
         self.tsc_loaded = CSR()
+        self.sed_spread_enable = CSRStorage()
         # master interface in the sys domain
         self.cri = cri.Interface()
         self.async_errors = Record(async_errors_layout)
@@ -136,14 +140,14 @@ class DRTIOSatellite(Module):
 
         self.sync += [
             If(self.tsc_loaded.re, self.tsc_loaded.w.eq(0)),
-            If(self.rt_packet.tsc_load, self.tsc_loaded.w.eq(1))
+            If(self.rt_packet.tsc_load, self.tsc_loaded.w.eq(1)),
         ]
 
         self.submodules.rt_errors = rt_errors_satellite.RTErrorsSatellite(
             self.rt_packet, tsc, self.async_errors)
 
     def get_csrs(self):
-        return ([self.reset, self.reset_phy, self.tsc_loaded] +
+        return ([self.reset, self.sed_spread_enable, self.reset_phy, self.tsc_loaded] +
                 self.link_layer.get_csrs() + self.link_stats.get_csrs() +
                 self.rt_errors.get_csrs())
 

artiq/gateware/rtio/core.py

@@ -3,7 +3,7 @@ from operator import and_
 
 from migen import *
 from migen.genlib.resetsync import AsyncResetSynchronizer
-from migen.genlib.cdc import BlindTransfer
+from migen.genlib.cdc import MultiReg, BlindTransfer
 from misoc.interconnect.csr import *
 
 from artiq.gateware.rtio import cri
@@ -18,6 +18,7 @@ class Core(Module, AutoCSR):
         self.cri = cri.Interface()
         self.reset = CSR()
         self.reset_phy = CSR()
+        self.sed_spread_enable = CSRStorage()
         self.async_error = CSR(3)
         self.collision_channel = CSRStatus(16)
         self.busy_channel = CSRStatus(16)
@@ -67,6 +68,7 @@ class Core(Module, AutoCSR):
         self.submodules += outputs
         self.comb += outputs.coarse_timestamp.eq(tsc.coarse_ts)
         self.sync += outputs.minimum_coarse_timestamp.eq(tsc.coarse_ts + 12)
+        self.specials += MultiReg(self.sed_spread_enable.storage, outputs.enable_spread, "rio")
 
         inputs = ClockDomainsRenamer("rio")(InputCollector(tsc, channels,
             quash_channels=quash_channels,

artiq/gateware/rtio/phy/fastino.py

@@ -27,6 +27,11 @@ class Fastino(Module):
 
         # dac data words
         dacs = [Signal(16) for i in range(32)]
+        self.probes = dacs
+        staged_dacs = [Signal(16) for i in range(32)]
+        override_en = [Signal() for i in range(32)]
+        override_o = [Signal(16) for i in range(32)]
+        self.overrides = override_en + override_o
 
         header = Record([
             ("cfg", 4),
@@ -92,13 +97,21 @@ class Fastino(Module):
                 header.typ.eq(1),
             ],
         }
+        overrides = {
+            "default": []
+        }
         for i in range(0, len(dacs), width):
             cases[i] = [
-                Cat(dacs[i:i + width]).eq(self.rtlink.o.data),
+                Cat(staged_dacs[i:i + width]).eq(self.rtlink.o.data),
                 [If(~hold[i + j] & (header.typ == 0),
                     header.enable[i + j].eq(1),
                 ) for j in range(width)]
             ]
+            overrides[i] = If(override_en[i],
+                              Cat(staged_dacs[i:i + width]).eq(Replicate(override_o[i], width))
+                           ).Else(
+                              Cat(staged_dacs[i:i + width]).eq(self.rtlink.o.data)
+                           )
 
         self.comb += [
             If(header.typ == 0,
@@ -118,6 +131,11 @@ class Fastino(Module):
             If(self.rtlink.o.stb,
                 Case(self.rtlink.o.address, cases),
             ),
+            [If(override_en[i],
+                Cat(dacs[i:i + width]).eq(Cat(override_o[i:i + width]))
+             ).Else(
+                Cat(dacs[i:i + width]).eq(Cat(staged_dacs[i: i + width]))
+             ) for i in range(0, len(dacs), width)]
         ]
 
         self.sync += [

artiq/gateware/rtio/sed/core.py

@@ -12,18 +12,20 @@ __all__ = ["SED"]
 
 class SED(Module):
     def __init__(self, channels, glbl_fine_ts_width,
-                 lane_count=8, fifo_depth=128, enable_spread=True,
+                 lane_count=8, fifo_depth=128, fifo_high_watermark=1.0,
                  quash_channels=[], report_buffer_space=False, interface=None):
         seqn_width = layouts.seqn_width(lane_count, fifo_depth)
 
+        fifo_high_watermark = int(fifo_high_watermark * fifo_depth)
+        assert fifo_depth >= fifo_high_watermark
+
         self.submodules.lane_dist = LaneDistributor(lane_count, seqn_width,
             layouts.fifo_payload(channels),
             [channel.interface.o.delay for channel in channels],
             glbl_fine_ts_width,
-            enable_spread=enable_spread,
             quash_channels=quash_channels,
             interface=interface)
-        self.submodules.fifos = FIFOs(lane_count, fifo_depth,
+        self.submodules.fifos = FIFOs(lane_count, fifo_depth, fifo_high_watermark,
             layouts.fifo_payload(channels), report_buffer_space)
         self.submodules.gates = Gates(lane_count, seqn_width,
             layouts.fifo_payload(channels),
@@ -44,6 +46,10 @@ class SED(Module):
                 self.cri.o_buffer_space.eq(self.fifos.buffer_space)
             ]
 
+    @property
+    def enable_spread(self):
+        return self.lane_dist.enable_spread
+
     @property
     def cri(self):
         return self.lane_dist.cri

artiq/gateware/rtio/sed/fifos.py

@@ -11,7 +11,7 @@ __all__ = ["FIFOs"]
 
 
 class FIFOs(Module):
-    def __init__(self, lane_count, fifo_depth, layout_payload, report_buffer_space=False):
+    def __init__(self, lane_count, fifo_depth, high_watermark, layout_payload, report_buffer_space=False):
         seqn_width = layouts.seqn_width(lane_count, fifo_depth)
         self.input = [Record(layouts.fifo_ingress(seqn_width, layout_payload))
                       for _ in range(lane_count)]
@@ -33,6 +33,7 @@ class FIFOs(Module):
                 fifo.din.eq(Cat(input.seqn, input.payload.raw_bits())),
                 fifo.we.eq(input.we),
                 input.writable.eq(fifo.writable),
+                input.high_watermark.eq(fifo.level >= high_watermark),
 
                 Cat(output.seqn, output.payload.raw_bits()).eq(fifo.dout),
                 output.readable.eq(fifo.readable),

artiq/gateware/rtio/sed/lane_distributor.py

@@ -10,7 +10,7 @@ __all__ = ["LaneDistributor"]
 class LaneDistributor(Module):
     def __init__(self, lane_count, seqn_width, layout_payload,
                  compensation, glbl_fine_ts_width,
-                 enable_spread=True, quash_channels=[], interface=None):
+                 quash_channels=[], interface=None):
         if lane_count & (lane_count - 1):
             raise NotImplementedError("lane count must be a power of 2")
 
@@ -28,6 +28,8 @@ class LaneDistributor(Module):
         self.output = [Record(layouts.fifo_ingress(seqn_width, layout_payload))
                        for _ in range(lane_count)]
 
+        self.enable_spread = Signal()
+
         # # #
 
         o_status_wait = Signal()
@@ -154,8 +156,10 @@ class LaneDistributor(Module):
             self.comb += lio.payload.timestamp.eq(compensated_timestamp)
 
         # cycle #3, read status
+        current_lane_high_watermark = Signal()
         current_lane_writable = Signal()
         self.comb += [
+            current_lane_high_watermark.eq(Array(lio.high_watermark for lio in self.output)[current_lane]),
             current_lane_writable.eq(Array(lio.writable for lio in self.output)[current_lane]),
             o_status_wait.eq(~current_lane_writable)
         ]
@@ -170,12 +174,9 @@ class LaneDistributor(Module):
             self.sequence_error_channel.eq(self.cri.chan_sel[:16])
         ]
 
-        # current lane has been full, spread events by switching to the next.
-        if enable_spread:
-            current_lane_writable_r = Signal(reset=1)
+        # current lane has reached high watermark, spread events by switching to the next.
         self.sync += [
-                current_lane_writable_r.eq(current_lane_writable),
-                If(~current_lane_writable_r & current_lane_writable,
+            If(self.enable_spread & (current_lane_high_watermark | ~current_lane_writable),
                 force_laneB.eq(1)
             ),
             If(do_write,

artiq/gateware/rtio/sed/layouts.py

@@ -31,6 +31,7 @@ def fifo_ingress(seqn_width, layout_payload):
     return [
         ("we", 1, DIR_M_TO_S),
         ("writable", 1, DIR_S_TO_M),
+        ("high_watermark", 1, DIR_S_TO_M),
         ("seqn", seqn_width, DIR_M_TO_S),
         ("payload", [(a, b, DIR_M_TO_S) for a, b in layout_payload])
     ]

artiq/gateware/targets/efc.py

@@ -29,9 +29,10 @@ class Satellite(BaseSoC, AMPSoC):
     }
     mem_map.update(BaseSoC.mem_map)
 
-    def __init__(self, gateware_identifier_str=None, **kwargs):
+    def __init__(self, gateware_identifier_str=None, hw_rev="v1.1", **kwargs):
         BaseSoC.__init__(self,
                  cpu_type="vexriscv",
+                 hw_rev=hw_rev,
                  cpu_bus_width=64,
                  sdram_controller_type="minicon",
                  l2_size=128*1024,
@@ -82,9 +83,10 @@ class Satellite(BaseSoC, AMPSoC):
             core.link_layer, self.cpu_dw))
         self.csr_devices.append("drtioaux0")
 
+        drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
         memory_address = self.mem_map["drtioaux"]
-        self.add_wb_slave(memory_address, 0x800, self.drtioaux0.bus)
-        self.add_memory_region("drtioaux0_mem", memory_address | self.shadow_base, 0x800)
+        self.add_wb_slave(memory_address, drtio_aux_mem_size, self.drtioaux0.bus)
+        self.add_memory_region("drtioaux0_mem", memory_address | self.shadow_base, drtio_aux_mem_size)
 
         self.config["HAS_DRTIO"] = None
         self.add_csr_group("drtioaux", ["drtioaux0"])
@@ -215,7 +217,10 @@ class Satellite(BaseSoC, AMPSoC):
 
         # satellite (master-controlled) RTIO
         self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels, lane_count=sed_lanes)
-        self.comb += self.drtiosat.async_errors.eq(self.local_io.async_errors)
+        self.comb += [ 
+            self.drtiosat.async_errors.eq(self.local_io.async_errors),
+            self.local_io.sed_spread_enable.eq(self.drtiosat.sed_spread_enable.storage)
+        ]
 
         # subkernel RTIO
         self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc)
@@ -242,12 +247,15 @@ def main():
     builder_args(parser)
     parser.set_defaults(output_dir="artiq_efc")
     parser.add_argument("-V", "--variant", default="shuttler")
+    parser.add_argument("--hw-rev", choices=["v1.0", "v1.1"], default="v1.1", 
+                        help="Hardware revision")
     parser.add_argument("--gateware-identifier-str", default=None,
                         help="Override ROM identifier")
     args = parser.parse_args()
 
     argdict = dict()
     argdict["gateware_identifier_str"] = args.gateware_identifier_str
+    argdict["hw_rev"] = args.hw_rev
 
     soc = Satellite(**argdict)
     build_artiq_soc(soc, builder_argdict(args))