moninj: display voltage in DACWidget

Signed-off-by: Egor Savkin <es@m-labs.hk>

README.rst

@@ -5,7 +5,7 @@
   :target: https://m-labs.hk/artiq
 
 ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) is a leading-edge control and data acquisition system for quantum information experiments.
-It is maintained and developed by `M-Labs <https://m-labs.hk>`_ and the initial development was for and in partnership with the `Ion Storage Group at NIST <https://www.nist.gov/pml/time-and-frequency-division/ion-storage>`_. ARTIQ is free software and offered to the entire research community as a solution equally applicable to other challenging control tasks, including outside the field of ion trapping. Many laboratories around the world have adopted ARTIQ as their control system, with over a hundred Sinara hardware crates deployed, and some have `contributed <https://m-labs.hk/experiment-control/funding/>`_ to it.
+It is maintained and developed by `M-Labs <https://m-labs.hk>`_ and the initial development was for and in partnership with the `Ion Storage Group at NIST <https://www.nist.gov/pml/time-and-frequency-division/ion-storage>`_. ARTIQ is free software and offered to the entire research community as a solution equally applicable to other challenging control tasks, including outside the field of ion trapping. Many laboratories around the world have adopted ARTIQ as their control system and some have `contributed <https://m-labs.hk/experiment-control/funding/>`_ to it.
 
 The system features a high-level programming language that helps describing complex experiments, which is compiled and executed on dedicated hardware with nanosecond timing resolution and sub-microsecond latency. It includes graphical user interfaces to parametrize and schedule experiments and to visualize and explore the results.
 
@@ -29,7 +29,7 @@ Website: https://m-labs.hk/artiq
 License
 =======
 
-Copyright (C) 2014-2023 M-Labs Limited.
+Copyright (C) 2014-2024 M-Labs Limited.
 
 ARTIQ is free software: you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as published by

RELEASE_NOTES.rst

@@ -3,9 +3,14 @@
 Release notes
 =============
 
-ARTIQ-8 (Unreleased)
+ARTIQ-9 (Unreleased)
 --------------------
 
+* Zadig Driver Installer was added to the MSYS2 offline installer.
+
+ARTIQ-8
+-------
+
 Highlights:
 
 * New hardware support:
@@ -28,23 +33,38 @@ Highlights:
   clock, to facilitate implementation of local processing on DRTIO satellites, and to slightly
   reduce RTIO latency.
 * Support for DRTIO-over-EEM, used with Shuttler.
+* Support for WRPLL low-noise clock recovery.
+* Enabled event spreading on DRTIO satellites, using high watermark for lane switching.
 * Added channel names to RTIO error messages.
+* The RTIO analyzer is now proxied by ``aqctl_coreanalyzer_proxy`` typically running on the master
+  machine, similarly to ``aqctl_moninj_proxy``.
 * GUI:
+   - Integrated waveform analyzer, removing the need for external VCD viewers such as GtkWave.
    - Implemented Applet Request Interfaces which allow applets to modify datasets and set the
      current values of widgets in the dashboard's experiment windows.
-   - Implemented a new EntryArea widget which allows argument entry widgets to be used in applets.
+   - Implemented a new ``EntryArea`` widget which allows argument entry widgets to be used in applets.
    - The "Close all applets" command (shortcut: Ctrl-Alt-W) now ignores docked applets,
      making it a convenient way to clean up after exploratory work without destroying a
      carefully arranged default workspace.
    - Hotkeys now organize experiment windows in the order they were last interacted with:
       + CTRL+SHIFT+T tiles experiment windows
       + CTRL+SHIFT+C cascades experiment windows
+   - By enabling the ``quickstyle`` option, ``EnumerationValue`` entry widgets can now alternatively display 
+     its choices as buttons that submit the experiment on click.
+* Datasets can now be associated with units and scale factors, and displayed accordingly in the dashboard
+  including applets, like widgets such as ``NumberValue`` already did in earlier ARTIQ versions.
+* Experiments can now request arguments interactively from the user at any time.
 * Persistent datasets are now stored in a LMDB database for improved performance.
 * Python's built-in types (such as ``float``, or ``List[...]``) can now be used in type annotations on
   kernel functions.
-* Full Python 3.10 support.
 * MSYS2 packaging for Windows, which replaces Conda. Conda packages are still available to
   support legacy installations, but may be removed in a future release.
+* Experiments can now be submitted with revisions set to a branch / tag name instead of only git hashes.
+* Grabber image input now has an optional timeout.
+* On NAR3-supported devices (Kasli-SoC, ZC706), when a Rust panic occurs, a minimal environment is started
+  where the network and ``artiq_coremgmt`` can be used. This allows the user to inspect logs, change
+  configuration options, update the firmware, and reboot the device.
+* Full Python 3.11 support.
 
 Breaking changes:
 

artiq/applets/plot_xy.py

@@ -24,11 +24,11 @@ class XYPlot(pyqtgraph.PlotWidget):
             y = value[self.args.y]
         except KeyError:
             return
-        x = value.get(self.args.x, (False, None))
+        x = value.get(self.args.x)
         if x is None:
             x = np.arange(len(y))
-        error = value.get(self.args.error, (False, None))
-        fit = value.get(self.args.fit, (False, None))
+        error = value.get(self.args.error)
+        fit = value.get(self.args.fit)
 
         if not len(y) or len(y) != len(x):
             self.mismatch['X values'] = True

artiq/applets/simple.py

@@ -42,13 +42,13 @@ class _AppletRequestInterface:
         """
         raise NotImplementedError
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         """
         Temporarily set the value of an argument in a experiment in the dashboard.
         The value resets to default value when recomputing the argument.
 
         :param expurl: Experiment URL identifying the experiment in the dashboard. Example: 'repo:ArgumentsDemo'.
-        :param name: Name of the argument in the experiment.
+        :param key: Name of the argument in the experiment.
         :param value: Object representing the new temporary value of the argument. For ``Scannable`` arguments, this parameter
             should be a ``ScanObject``. The type of the ``ScanObject`` will be set as the selected type when this function is called. 
         """
@@ -77,10 +77,10 @@ class AppletRequestIPC(_AppletRequestInterface):
         mod = {"action": "append", "path": [key, 1], "x": value}
         self.ipc.update_dataset(mod)
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         if isinstance(value, ScanObject):
             value = value.describe()
-        self.ipc.set_argument_value(expurl, name, value)
+        self.ipc.set_argument_value(expurl, key, value)
 
 
 class AppletRequestRPC(_AppletRequestInterface):
@@ -182,10 +182,10 @@ class AppletIPCClient(AsyncioChildComm):
         self.write_pyon({"action": "update_dataset",
                          "mod": mod})
 
-    def set_argument_value(self, expurl, name, value):
+    def set_argument_value(self, expurl, key, value):
         self.write_pyon({"action": "set_argument_value",
                          "expurl": expurl,
-                         "name": name,
+                         "key": key,
                          "value": value})
 
 
@@ -255,7 +255,7 @@ class SimpleApplet:
         if self.embed is None:
             dataset_ctl = RPCClient()
             self.loop.run_until_complete(dataset_ctl.connect_rpc(
-                self.args.server, self.args.port_control, "master_dataset_db"))
+                self.args.server, self.args.port_control, "dataset_db"))
             self.req = AppletRequestRPC(self.loop, dataset_ctl)
         else:
             self.req = AppletRequestIPC(self.ipc)

artiq/browser/datasets.py

@@ -33,7 +33,7 @@ class DatasetCtl:
         try:
             remote = RPCClient()
             await remote.connect_rpc(self.master_host, self.master_port,
-                                     "master_dataset_db")
+                                     "dataset_db")
             try:
                 if op_name == "set":
                     await remote.set(key_or_mod, value, persist, metadata)

artiq/browser/experiments.py

@@ -10,87 +10,26 @@ import h5py
 from sipyco import pyon
 
 from artiq import __artiq_dir__ as artiq_dir
-from artiq.gui.tools import (LayoutWidget, WheelFilter, 
-                             log_level_to_name, get_open_file_name)
-from artiq.gui.entries import procdesc_to_entry
+from artiq.gui.tools import (LayoutWidget, log_level_to_name, get_open_file_name)
+from artiq.gui.entries import procdesc_to_entry, EntryTreeWidget
 from artiq.master.worker import Worker, log_worker_exception
 
 logger = logging.getLogger(__name__)
 
 
-class _ArgumentEditor(QtWidgets.QTreeWidget):
+class _ArgumentEditor(EntryTreeWidget):
     def __init__(self, dock):
-        QtWidgets.QTreeWidget.__init__(self)
-        self.setColumnCount(3)
-        self.header().setStretchLastSection(False)
-        try:
-            set_resize_mode = self.header().setSectionResizeMode
-        except AttributeError:
-            set_resize_mode = self.header().setResizeMode
-        set_resize_mode(0, QtWidgets.QHeaderView.ResizeToContents)
-        set_resize_mode(1, QtWidgets.QHeaderView.Stretch)
-        set_resize_mode(2, QtWidgets.QHeaderView.ResizeToContents)
-        self.header().setVisible(False)
-        self.setSelectionMode(self.NoSelection)
-        self.setHorizontalScrollMode(self.ScrollPerPixel)
-        self.setVerticalScrollMode(self.ScrollPerPixel)
-
-        self.setStyleSheet("QTreeWidget {background: " +
-                           self.palette().midlight().color().name() + " ;}")
-
-        self.viewport().installEventFilter(WheelFilter(self.viewport(), True))
-
-        self._groups = dict()
-        self._arg_to_widgets = dict()
+        EntryTreeWidget.__init__(self)
         self._dock = dock
 
         if not self._dock.arguments:
-            self.addTopLevelItem(QtWidgets.QTreeWidgetItem(["No arguments"]))
-        gradient = QtGui.QLinearGradient(
-            0, 0, 0, QtGui.QFontMetrics(self.font()).lineSpacing()*2.5)
-        gradient.setColorAt(0, self.palette().base().color())
-        gradient.setColorAt(1, self.palette().midlight().color())
+            self.insertTopLevelItem(0, QtWidgets.QTreeWidgetItem(["No arguments"]))
 
         for name, argument in self._dock.arguments.items():
-            widgets = dict()
-            self._arg_to_widgets[name] = widgets
+            self.set_argument(name, argument)
 
-            entry = procdesc_to_entry(argument["desc"])(argument)
-            widget_item = QtWidgets.QTreeWidgetItem([name])
-            if argument["tooltip"]:
-                widget_item.setToolTip(0, argument["tooltip"])
-            widgets["entry"] = entry
-            widgets["widget_item"] = widget_item
+        self.quickStyleClicked.connect(self._dock._run_clicked)
 
-            for col in range(3):
-                widget_item.setBackground(col, gradient)
-            font = widget_item.font(0)
-            font.setBold(True)
-            widget_item.setFont(0, font)
-
-            if argument["group"] is None:
-                self.addTopLevelItem(widget_item)
-            else:
-                self._get_group(argument["group"]).addChild(widget_item)
-            fix_layout = LayoutWidget()
-            widgets["fix_layout"] = fix_layout
-            fix_layout.addWidget(entry)
-            self.setItemWidget(widget_item, 1, fix_layout)
-
-            recompute_argument = QtWidgets.QToolButton()
-            recompute_argument.setToolTip("Re-run the experiment's build "
-                                          "method and take the default value")
-            recompute_argument.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_BrowserReload))
-            recompute_argument.clicked.connect(
-                partial(self._recompute_argument_clicked, name))
-            fix_layout = LayoutWidget()
-            fix_layout.addWidget(recompute_argument)
-            self.setItemWidget(widget_item, 2, fix_layout)
-
-        widget_item = QtWidgets.QTreeWidgetItem()
-        self.addTopLevelItem(widget_item)
         recompute_arguments = QtWidgets.QPushButton("Recompute all arguments")
         recompute_arguments.setIcon(
             QtWidgets.QApplication.style().standardIcon(
@@ -100,7 +39,7 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         load = QtWidgets.QPushButton("Set arguments from HDF5")
         load.setToolTip("Set arguments from currently selected HDF5 file")
         load.setIcon(QtWidgets.QApplication.style().standardIcon(
-                QtWidgets.QStyle.SP_DialogApplyButton))
+            QtWidgets.QStyle.SP_DialogApplyButton))
         load.clicked.connect(self._load_clicked)
 
         buttons = LayoutWidget()
@@ -108,21 +47,7 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         buttons.addWidget(load, 1, 2)
         for i, s in enumerate((1, 0, 0, 1)):
             buttons.layout.setColumnStretch(i, s)
-        self.setItemWidget(widget_item, 1, buttons)
-
-    def _get_group(self, name):
-        if name in self._groups:
-            return self._groups[name]
-        group = QtWidgets.QTreeWidgetItem([name])
-        for col in range(3):
-            group.setBackground(col, self.palette().mid())
-            group.setForeground(col, self.palette().brightText())
-            font = group.font(col)
-            font.setBold(True)
-            group.setFont(col, font)
-        self.addTopLevelItem(group)
-        self._groups[name] = group
-        return group
+        self.setItemWidget(self.bottom_item, 1, buttons)
 
     def _load_clicked(self):
         asyncio.ensure_future(self._dock.load_hdf5_task())
@@ -130,8 +55,8 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
     def _recompute_arguments_clicked(self):
         asyncio.ensure_future(self._dock._recompute_arguments())
 
-    def _recompute_argument_clicked(self, name):
-        asyncio.ensure_future(self._recompute_argument(name))
+    def reset_entry(self, key):
+        asyncio.ensure_future(self._recompute_argument(key))
 
     async def _recompute_argument(self, name):
         try:
@@ -146,29 +71,7 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         state = procdesc_to_entry(procdesc).default_state(procdesc)
         argument["desc"] = procdesc
         argument["state"] = state
-
-        widgets = self._arg_to_widgets[name]
-
-        widgets["entry"].deleteLater()
-        widgets["entry"] = procdesc_to_entry(procdesc)(argument)
-        widgets["fix_layout"] = LayoutWidget()
-        widgets["fix_layout"].addWidget(widgets["entry"])
-        self.setItemWidget(widgets["widget_item"], 1, widgets["fix_layout"])
-        self.updateGeometries()
-
-    def save_state(self):
-        expanded = []
-        for k, v in self._groups.items():
-            if v.isExpanded():
-                expanded.append(k)
-        return {"expanded": expanded}
-
-    def restore_state(self, state):
-        for e in state["expanded"]:
-            try:
-                self._groups[e].setExpanded(True)
-            except KeyError:
-                pass
+        self.update_argument(name, argument)
 
 
 log_levels = ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]
@@ -277,8 +180,8 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
         state = self.argeditor.save_state()
         self.argeditor.deleteLater()
         self.argeditor = _ArgumentEditor(self)
-        self.argeditor.restore_state(state)
         self.layout.addWidget(self.argeditor, 0, 0, 1, 5)
+        self.argeditor.restore_state(state)
 
     async def load_hdf5_task(self, filename=None):
         if filename is None:
@@ -466,6 +369,8 @@ class ExperimentsArea(QtWidgets.QMdiArea):
     def initialize_submission_arguments(self, arginfo):
         arguments = OrderedDict()
         for name, (procdesc, group, tooltip) in arginfo.items():
+            if procdesc["ty"] == "EnumerationValue" and procdesc["quickstyle"]:
+                procdesc["quickstyle"] = False
             state = procdesc_to_entry(procdesc).default_state(procdesc)
             arguments[name] = {
                 "desc": procdesc,

artiq/compiler/builtins.py

@@ -253,6 +253,12 @@ def fn_subkernel_await():
 def fn_subkernel_preload():
     return types.TBuiltinFunction("subkernel_preload")
 
+def fn_subkernel_send():
+    return types.TBuiltinFunction("subkernel_send")
+
+def fn_subkernel_recv():
+    return types.TBuiltinFunction("subkernel_recv")
+
 # Accessors
 
 def is_none(typ):

artiq/compiler/embedding.py

@@ -47,8 +47,15 @@ class SpecializedFunction:
         return hash((self.instance_type, self.host_function))
 
 
+class SubkernelMessageType:
+    def __init__(self, name, value_type):
+        self.name = name
+        self.value_type = value_type
+        self.send_loc = None
+        self.recv_loc = None
+
 class EmbeddingMap:
-    def __init__(self):
+    def __init__(self, old_embedding_map=None):
         self.object_current_key = 0
         self.object_forward_map = {}
         self.object_reverse_map = {}
@@ -65,6 +72,22 @@ class EmbeddingMap:
         self.str_forward_map = {}
         self.str_reverse_map = {}
 
+        # mapping `name` to object ID
+        self.subkernel_message_map = {}
+
+        # subkernels: dict of ID: function, just like object_forward_map
+        # allow the embedding map to be aware of subkernels from other kernels
+        if not old_embedding_map is None:
+            for key, obj_ref in old_embedding_map.subkernels().items():
+                self.object_forward_map[key] = obj_ref
+                obj_id = id(obj_ref)
+                self.object_reverse_map[obj_id] = key
+            for msg_id, msg_type in old_embedding_map.subkernel_messages().items():
+                self.object_forward_map[msg_id] = msg_type
+                obj_id = id(msg_type)
+                self.subkernel_message_map[msg_type.name] = msg_id
+                self.object_reverse_map[obj_id] = msg_id
+
         self.preallocate_runtime_exception_names(["RuntimeError",
                                                   "RTIOUnderflow",
                                                   "RTIOOverflow",
@@ -165,6 +188,11 @@ class EmbeddingMap:
             return self.object_reverse_map[obj_id]
 
         self.object_current_key += 1
+        while self.object_forward_map.get(self.object_current_key):
+            # make sure there's no collisions with previously inserted subkernels
+            # their identifiers must be consistent across all kernels/subkernels
+            self.object_current_key += 1
+        
         self.object_forward_map[self.object_current_key] = obj_ref
         self.object_reverse_map[obj_id] = self.object_current_key
         return self.object_current_key
@@ -177,7 +205,7 @@ class EmbeddingMap:
             obj_ref = self.object_forward_map[obj_id]
             if isinstance(obj_ref, (pytypes.FunctionType, pytypes.MethodType,
                                     pytypes.BuiltinFunctionType, pytypes.ModuleType,
-                                    SpecializedFunction)):
+                                    SpecializedFunction, SubkernelMessageType)):
                 continue
             elif isinstance(obj_ref, type):
                 _, obj_typ = self.type_map[obj_ref]
@@ -193,6 +221,35 @@ class EmbeddingMap:
                     subkernels[k] = v
         return subkernels
 
+    def store_subkernel_message(self, name, value_type, function_type, function_loc):
+        if name in self.subkernel_message_map:
+            msg_id = self.subkernel_message_map[name]
+        else:
+            msg_id = self.store_object(SubkernelMessageType(name, value_type))
+            self.subkernel_message_map[name] = msg_id
+        subkernel_msg = self.retrieve_object(msg_id)
+        if function_type == "send":
+            subkernel_msg.send_loc = function_loc
+        elif function_type == "recv":
+            subkernel_msg.recv_loc = function_loc
+        else:
+            assert False
+        return msg_id, subkernel_msg
+
+    def subkernel_messages(self):
+        messages = {}
+        for msg_id in self.subkernel_message_map.values():
+            messages[msg_id] = self.retrieve_object(msg_id)
+        return messages
+
+    def subkernel_messages_unpaired(self):
+        unpaired = []
+        for msg_id in self.subkernel_message_map.values():
+            msg_obj = self.retrieve_object(msg_id)
+            if msg_obj.send_loc is None or msg_obj.recv_loc is None:
+                unpaired.append(msg_obj)
+        return unpaired
+
     def has_rpc(self):
         return any(filter(
                 lambda x: (inspect.isfunction(x) or inspect.ismethod(x)) and \
@@ -200,10 +257,6 @@ class EmbeddingMap:
                 self.object_forward_map.values()
             ))
 
-    def has_rpc_or_subkernel(self):
-        return any(filter(lambda x: inspect.isfunction(x) or inspect.ismethod(x),
-                          self.object_forward_map.values()))
-
 
 class ASTSynthesizer:
     def __init__(self, embedding_map, value_map, quote_function=None, expanded_from=None):
@@ -694,9 +747,9 @@ class StitchingInferencer(Inferencer):
                 if elt.__class__ == float:
                     state |= IS_FLOAT
                 elif elt.__class__ == int:
-                    if -2**31 < elt < 2**31-1:
+                    if -2**31 <= elt <= 2**31-1:
                         state |= IS_INT32
-                    elif -2**63 < elt < 2**63-1:
+                    elif -2**63 <= elt <= 2**63-1:
                         state |= IS_INT64
                     else:
                         state = -1
@@ -794,7 +847,7 @@ class TypedtreeHasher(algorithm.Visitor):
         return hash(tuple(freeze(getattr(node, field_name)) for field_name in fields))
 
 class Stitcher:
-    def __init__(self, core, dmgr, engine=None, print_as_rpc=True, destination=0, subkernel_arg_types=[]):
+    def __init__(self, core, dmgr, engine=None, print_as_rpc=True, destination=0, subkernel_arg_types=[], old_embedding_map=None):
         self.core = core
         self.dmgr = dmgr
         if engine is None:
@@ -816,7 +869,7 @@ class Stitcher:
 
         self.functions = {}
 
-        self.embedding_map = EmbeddingMap()
+        self.embedding_map = EmbeddingMap(old_embedding_map)
         self.value_map = defaultdict(lambda: [])
         self.definitely_changed = False
 

artiq/compiler/prelude.py

@@ -59,4 +59,6 @@ def globals():
         # ARTIQ subkernel utility functions
         "subkernel_await":     builtins.fn_subkernel_await(),
         "subkernel_preload":   builtins.fn_subkernel_preload(),
+        "subkernel_send":      builtins.fn_subkernel_send(),
+        "subkernel_recv":      builtins.fn_subkernel_recv(), 
     }

artiq/compiler/transforms/artiq_ir_generator.py

@@ -2537,7 +2537,7 @@ class ARTIQIRGenerator(algorithm.Visitor):
                 timeout = self.visit(node.args[1])
             elif len(node.args) == 1 and len(node.keywords) == 0:
                 fn = node.args[0].type
-                timeout = ir.Constant(10_000, builtins.TInt64())
+                timeout = ir.Constant(-1, builtins.TInt64())
             else:
                 assert False
             if types.is_method(fn):
@@ -2557,7 +2557,44 @@ class ARTIQIRGenerator(algorithm.Visitor):
             if types.is_method(fn):
                 fn = types.get_method_function(fn)
             sid = ir.Constant(fn.sid, builtins.TInt32())
-            return self.append(ir.Builtin("subkernel_preload", [sid], builtins.TNone()))
+            dest = ir.Constant(fn.destination, builtins.TInt32())
+            return self.append(ir.Builtin("subkernel_preload", [sid, dest], builtins.TNone()))
+        elif types.is_builtin(typ, "subkernel_send"):
+            if len(node.args) == 3 and len(node.keywords) == 0:
+                dest = self.visit(node.args[0])
+                name = node.args[1].s
+                value = self.visit(node.args[2])
+            else:
+                assert False
+            msg_id, msg = self.embedding_map.store_subkernel_message(name, value.type, "send", node.loc)
+            msg_id = ir.Constant(msg_id, builtins.TInt32())
+            if value.type != msg.value_type:
+                diag = diagnostic.Diagnostic("error",
+                    "type mismatch for subkernel message '{name}', receiver expects {recv} while sending {send}",
+                    {"name": name, "recv": msg.value_type, "send": value.type},
+                    node.loc)
+                self.engine.process(diag)
+            return self.append(ir.Builtin("subkernel_send", [msg_id, dest, value], builtins.TNone()))
+        elif types.is_builtin(typ, "subkernel_recv"):
+            if len(node.args) == 2 and len(node.keywords) == 0:
+                name = node.args[0].s
+                vartype = node.args[1].value
+                timeout = ir.Constant(-1, builtins.TInt64())
+            elif len(node.args) == 3 and len(node.keywords) == 0:
+                name = node.args[0].s
+                vartype = node.args[1].value
+                timeout = self.visit(node.args[2])
+            else:
+                assert False
+            msg_id, msg = self.embedding_map.store_subkernel_message(name, vartype, "recv", node.loc)
+            msg_id = ir.Constant(msg_id, builtins.TInt32())
+            if vartype != msg.value_type:
+                diag = diagnostic.Diagnostic("error",
+                    "type mismatch for subkernel message '{name}', receiver expects {recv} while sending {send}",
+                    {"name": name, "recv": vartype, "send": msg.value_type},
+                    node.loc)
+                self.engine.process(diag)
+            return self.append(ir.Builtin("subkernel_recv", [msg_id, timeout], vartype))
         elif types.is_exn_constructor(typ):
             return self.alloc_exn(node.type, *[self.visit(arg_node) for arg_node in node.args])
         elif types.is_constructor(typ):

artiq/compiler/transforms/inferencer.py

@@ -1343,6 +1343,57 @@ class Inferencer(algorithm.Visitor):
                                 node.loc, None)
             else:
                 diagnose(valid_forms())
+        elif types.is_builtin(typ, "subkernel_send"):
+            valid_forms = lambda: [
+                valid_form("subkernel_send(dest: numpy.int?, name: str, value: V) -> None"),
+            ]
+            self._unify(node.type, builtins.TNone(),
+                        node.loc, None)
+            if len(node.args) == 3:
+                arg0 = node.args[0]
+                if types.is_var(arg0.type):
+                    pass  # undetermined yet
+                else:
+                    if builtins.is_int(arg0.type):
+                        self._unify(arg0.type, builtins.TInt8(),
+                                    arg0.loc, None)
+                    else:
+                        diagnose(valid_forms())
+                arg1 = node.args[1]
+                self._unify(arg1.type, builtins.TStr(),
+                        arg1.loc, None)
+            else:
+                diagnose(valid_forms())
+        elif types.is_builtin(typ, "subkernel_recv"):
+            valid_forms = lambda: [
+                valid_form("subkernel_recv(name: str, value_type: type) -> value_type"),
+                valid_form("subkernel_recv(name: str, value_type: type, timeout: numpy.int64) -> value_type"),
+            ]
+            if 2 <= len(node.args) <= 3:
+                arg0 = node.args[0]
+                if types.is_var(arg0.type):
+                    pass
+                else:
+                    self._unify(arg0.type, builtins.TStr(),
+                            arg0.loc, None)
+                arg1 = node.args[1]
+                if types.is_var(arg1.type):
+                    pass
+                else:
+                    self._unify(node.type, arg1.value,
+                                node.loc, None)
+                if len(node.args) == 3:
+                    arg2 = node.args[2]
+                    if types.is_var(arg2.type):
+                        pass
+                    elif builtins.is_int(arg2.type):
+                        # promote to TInt64
+                        self._unify(arg2.type, builtins.TInt64(),
+                                    arg2.loc, None)
+                    else:
+                        diagnose(valid_forms())
+            else:
+                diagnose(valid_forms())
         else:
             assert False
 

artiq/compiler/transforms/int_monomorphizer.py

@@ -14,9 +14,9 @@ class IntMonomorphizer(algorithm.Visitor):
     def visit_NumT(self, node):
         if builtins.is_int(node.type):
             if types.is_var(node.type["width"]):
-                if -2**31 < node.n < 2**31-1:
+                if -2**31 <= node.n <= 2**31-1:
                     width = 32
-                elif -2**63 < node.n < 2**63-1:
+                elif -2**63 <= node.n <= 2**63-1:
                     width = 64
                 else:
                     diag = diagnostic.Diagnostic("error",

artiq/compiler/transforms/llvm_ir_generator.py

@@ -399,9 +399,9 @@ class LLVMIRGenerator:
             llty = ll.FunctionType(lli32, [llptr])
 
         elif name == "subkernel_send_message":
-            llty = ll.FunctionType(llvoid, [lli32, lli8, llsliceptr, llptrptr])
+            llty = ll.FunctionType(llvoid, [lli32, lli1, lli8, lli8, llsliceptr, llptrptr])
         elif name == "subkernel_load_run":
-            llty = ll.FunctionType(llvoid, [lli32, lli1])
+            llty = ll.FunctionType(llvoid, [lli32, lli8, lli1])
         elif name == "subkernel_await_finish":
             llty = ll.FunctionType(llvoid, [lli32, lli64])
         elif name == "subkernel_await_message":
@@ -1417,8 +1417,23 @@ class LLVMIRGenerator:
             return self._build_rpc_recv(insn.type, llstackptr)
         elif insn.op == "subkernel_preload":
             llsid = self.map(insn.operands[0])
-            return self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, ll.Constant(lli1, 0)], 
+            lldest = ll.Constant(lli8, insn.operands[1].value)
+            return self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, lldest, ll.Constant(lli1, 0)], 
                                 name="subkernel.preload")
+        elif insn.op == "subkernel_send":
+            llmsgid = self.map(insn.operands[0])
+            lldest = self.map(insn.operands[1])
+            return self._build_subkernel_message(llmsgid, lldest, [insn.operands[2]])
+        elif insn.op == "subkernel_recv":
+            llmsgid = self.map(insn.operands[0])
+            lltimeout = self.map(insn.operands[1])
+            lltagptr = self._build_subkernel_tags([insn.type])
+            self.llbuilder.call(self.llbuiltin("subkernel_await_message"), 
+                                [llmsgid, lltimeout, lltagptr, ll.Constant(lli8, 1), ll.Constant(lli8, 1)],
+                                name="subkernel.await.message")
+            llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [],
+                                             name="subkernel.arg.stack")
+            return self._build_rpc_recv(insn.type, llstackptr)
         else:
             assert False
 
@@ -1427,7 +1442,7 @@ class LLVMIRGenerator:
         llmax = self.map(insn.operands[1])
         lltagptr = self._build_subkernel_tags(insn.arg_types)
         return self.llbuilder.call(self.llbuiltin("subkernel_await_message"), 
-                                    [ll.Constant(lli32, 0), ll.Constant(lli64, 10_000), lltagptr, llmin, llmax],
+                                    [ll.Constant(lli32, -1), ll.Constant(lli64, 10_000), lltagptr, llmin, llmax],
                                     name="subkernel.await.args")
 
     def process_Closure(self, insn):
@@ -1579,11 +1594,8 @@ class LLVMIRGenerator:
             self.llbuilder.branch(llnormalblock)
         return llret
 
-    def _build_rpc(self, fun_loc, fun_type, args, llnormalblock, llunwindblock):
-        llservice = ll.Constant(lli32, fun_type.service)
-
+    def _build_arg_tag(self, args, call_type):
         tag = b""
-
         for arg in args:
             def arg_error_handler(typ):
                 printer = types.TypePrinter()
@@ -1592,12 +1604,18 @@ class LLVMIRGenerator:
                     {"type": printer.name(typ)},
                     arg.loc)
                 diag = diagnostic.Diagnostic("error",
-                    "type {type} is not supported in remote procedure calls",
-                    {"type": printer.name(arg.type)},
+                    "type {type} is not supported in {call_type} calls",
+                    {"type": printer.name(arg.type), "call_type": call_type},
                     arg.loc, notes=[note])
                 self.engine.process(diag)
             tag += ir.rpc_tag(arg.type, arg_error_handler)
         tag += b":"
+        return tag
+
+    def _build_rpc(self, fun_loc, fun_type, args, llnormalblock, llunwindblock):
+        llservice = ll.Constant(lli32, fun_type.service)
+
+        tag = self._build_arg_tag(args, call_type="remote procedure")
 
         def ret_error_handler(typ):
             printer = types.TypePrinter()
@@ -1660,59 +1678,48 @@ class LLVMIRGenerator:
 
     def _build_subkernel_call(self, fun_loc, fun_type, args):
         llsid = ll.Constant(lli32, fun_type.sid)
-        tag = b""
-
-        for arg in args:
-            def arg_error_handler(typ):
-                printer = types.TypePrinter()
-                note = diagnostic.Diagnostic("note",
-                    "value of type {type}",
-                    {"type": printer.name(typ)},
-                    arg.loc)
-                diag = diagnostic.Diagnostic("error",
-                    "type {type} is not supported in subkernel calls",
-                    {"type": printer.name(arg.type)},
-                    arg.loc, notes=[note])
-                self.engine.process(diag)
-            tag += ir.rpc_tag(arg.type, arg_error_handler)
-        tag += b":"
-
+        lldest = ll.Constant(lli8, fun_type.destination)
         # run the kernel first
-        self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, ll.Constant(lli1, 1)])
+        self.llbuilder.call(self.llbuiltin("subkernel_load_run"), [llsid, lldest, ll.Constant(lli1, 1)])
+
+        if args:
+            # only send args if there's anything to send, 'self' is excluded
+            self._build_subkernel_message(llsid, lldest, args)
+
+        return llsid
+
+    def _build_subkernel_message(self, llid, lldest, args):
+        # args (or messages) are sent in the same vein as RPC
+        tag = self._build_arg_tag(args, call_type="subkernel")
 
-        # arg sent in the same vein as RPC
         llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [],
-                                         name="subkernel.stack")
-
+                                            name="subkernel.stack")
         lltag = self.llconst_of_const(ir.Constant(tag, builtins.TStr()))
         lltagptr = self.llbuilder.alloca(lltag.type)
         self.llbuilder.store(lltag, lltagptr)
 
-        if args:
-            # only send args if there's anything to send, 'self' is excluded
-            llargs = self.llbuilder.alloca(llptr, ll.Constant(lli32, len(args)),
-                                        name="subkernel.args")
-            for index, arg in enumerate(args):
-                if builtins.is_none(arg.type):
-                    llargslot = self.llbuilder.alloca(llunit,
-                                                    name="subkernel.arg{}".format(index))
-                else:
-                    llarg = self.map(arg)
-                    llargslot = self.llbuilder.alloca(llarg.type,
-                                                    name="subkernel.arg{}".format(index))
-                    self.llbuilder.store(llarg, llargslot)
-                llargslot = self.llbuilder.bitcast(llargslot, llptr)
+        llargs = self.llbuilder.alloca(llptr, ll.Constant(lli32, len(args)),
+                                    name="subkernel.args")
+        for index, arg in enumerate(args):
+            if builtins.is_none(arg.type):
+                llargslot = self.llbuilder.alloca(llunit,
+                                                name="subkernel.arg{}".format(index))
+            else:
+                llarg = self.map(arg)
+                llargslot = self.llbuilder.alloca(llarg.type,
+                                                name="subkernel.arg{}".format(index))
+                self.llbuilder.store(llarg, llargslot)
+            llargslot = self.llbuilder.bitcast(llargslot, llptr)
 
-                llargptr = self.llbuilder.gep(llargs, [ll.Constant(lli32, index)])
-                self.llbuilder.store(llargslot, llargptr)
+            llargptr = self.llbuilder.gep(llargs, [ll.Constant(lli32, index)])
+            self.llbuilder.store(llargslot, llargptr)
 
-            llargcount = ll.Constant(lli8, len(args))
+        llargcount = ll.Constant(lli8, len(args))
 
-            self.llbuilder.call(self.llbuiltin("subkernel_send_message"),
-                                [llsid, llargcount, lltagptr, llargs])
-            self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
-
-        return llsid
+        llisreturn = ll.Constant(lli1, False)
+        self.llbuilder.call(self.llbuiltin("subkernel_send_message"),
+                            [llid, llisreturn, lldest, llargcount, lltagptr, llargs])
+        return self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
 
     def _build_subkernel_return(self, insn):
         # builds a remote return.
@@ -1746,10 +1753,12 @@ class LLVMIRGenerator:
         llretslot = self.llbuilder.bitcast(llretslot, llptr)
         self.llbuilder.store(llretslot, llrets)
 
-        llsid = ll.Constant(lli32, 0)  # return goes back to master, sid is ignored
+        llsid = ll.Constant(lli32, 0)  # return goes back to the caller, sid is ignored
         lltagcount = ll.Constant(lli8, 1)  # only one thing is returned
+        llisreturn = ll.Constant(lli1, True)  # it's a return, so destination is ignored
+        lldest = ll.Constant(lli8, 0)
         self.llbuilder.call(self.llbuiltin("subkernel_send_message"),
-                            [llsid, lltagcount, lltagptr, llrets])
+                            [llsid, llisreturn, lldest, lltagcount, lltagptr, llrets])
 
     def process_Call(self, insn):
         functiontyp = insn.target_function().type

artiq/coredevice/ad9910.py

@@ -999,7 +999,7 @@ class AD9910:
         """
         if not self.cpld.sync_div:
             raise ValueError("parent cpld does not drive SYNC")
-        search_span = 31
+        search_span = 13
         # FIXME https://github.com/sinara-hw/Urukul/issues/16
         # should both be 2-4 once kasli sync_in jitter is identified
         min_window = 0

artiq/coredevice/adf5356.py

@@ -84,10 +84,11 @@ class ADF5356:
 
         :param blind: Do not attempt to verify presence.
         """
+        self.sync()
         if not blind:
             # MUXOUT = VDD
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 1)
-            self.sync()
+            self.write(self.regs[4])
             delay(1000 * us)
             if not self.read_muxout():
                 raise ValueError("MUXOUT not high")
@@ -95,7 +96,7 @@ class ADF5356:
 
             # MUXOUT = DGND
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 2)
-            self.sync()
+            self.write(self.regs[4])
             delay(1000 * us)
             if self.read_muxout():
                 raise ValueError("MUXOUT not low")
@@ -103,8 +104,7 @@ class ADF5356:
 
             # MUXOUT = digital lock-detect
             self.regs[4] = ADF5356_REG4_MUXOUT_UPDATE(self.regs[4], 6)
-        else:
-            self.sync()
+            self.write(self.regs[4])
 
     @kernel
     def set_att(self, att):

artiq/coredevice/almazny.py

@@ -1,4 +1,5 @@
 from artiq.language.core import kernel, portable
+from artiq.language.units import us
 
 from numpy import int32
 
@@ -117,11 +118,6 @@ class AlmaznyLegacy:
         )
         delay(100 * us)
 
-    @kernel
-    def _update_all_registers(self):
-        for i in range(4):
-            self._update_register(i)
-
 
 class AlmaznyChannel:
     """

artiq/coredevice/comm_analyzer.py

@@ -2,15 +2,22 @@ from operator import itemgetter
 from collections import namedtuple
 from itertools import count
 from contextlib import contextmanager
+from sipyco import keepalive
+import asyncio
 from enum import Enum
 import struct
 import logging
 import socket
+import math
 
 
 logger = logging.getLogger(__name__)
 
 
+DEFAULT_REF_PERIOD = 1e-9
+ANALYZER_MAGIC = b"ARTIQ Analyzer Proxy\n"
+
+
 class MessageType(Enum):
     output = 0b00
     input = 0b01
@@ -34,6 +41,13 @@ class ExceptionType(Enum):
     i_overflow = 0b100001
 
 
+class WaveformType(Enum):
+    ANALOG = 0
+    BIT = 1
+    VECTOR = 2
+    LOG = 3
+
+
 def get_analyzer_dump(host, port=1382):
     sock = socket.create_connection((host, port))
     try:
@@ -104,6 +118,8 @@ def decode_dump(data):
     (sent_bytes, total_byte_count,
      error_occurred, log_channel, dds_onehot_sel) = parts
 
+    logger.debug("analyzer dump has length %d", sent_bytes)
+
     expected_len = sent_bytes + 15
     if expected_len != len(data):
         raise ValueError("analyzer dump has incorrect length "
@@ -115,15 +131,83 @@ def decode_dump(data):
     if total_byte_count > sent_bytes:
         logger.info("analyzer ring buffer has wrapped %d times",
                     total_byte_count//sent_bytes)
+    if sent_bytes == 0:
+        logger.warning("analyzer dump is empty")
 
     position = 15
     messages = []
     for _ in range(sent_bytes//32):
         messages.append(decode_message(data[position:position+32]))
         position += 32
+
+    if len(messages) == 1 and isinstance(messages[0], StoppedMessage):
+        logger.warning("analyzer dump is empty aside from stop message")
+
     return DecodedDump(log_channel, bool(dds_onehot_sel), messages)
 
 
+# simplified from sipyco broadcast Receiver
+class AnalyzerProxyReceiver:
+    def __init__(self, receive_cb, disconnect_cb=None):
+        self.receive_cb = receive_cb
+        self.disconnect_cb = disconnect_cb
+
+    async def connect(self, host, port):
+        self.reader, self.writer = \
+            await keepalive.async_open_connection(host, port)
+        try:
+            line = await self.reader.readline()
+            assert line == ANALYZER_MAGIC
+            self.receive_task = asyncio.create_task(self._receive_cr())
+        except:
+            self.writer.close()
+            del self.reader
+            del self.writer
+            raise
+
+    async def close(self):
+        self.disconnect_cb = None
+        try:
+            self.receive_task.cancel()
+            try:
+                await self.receive_task
+            except asyncio.CancelledError:
+                pass
+        finally:
+            self.writer.close()
+            del self.reader
+            del self.writer
+
+    async def _receive_cr(self):
+        try:
+            while True:
+                endian_byte = await self.reader.read(1)
+                if endian_byte == b"E":
+                    endian = '>'
+                elif endian_byte == b"e":
+                    endian = '<'
+                elif endian_byte == b"":
+                    # EOF reached, connection lost
+                    return
+                else:
+                    raise ValueError
+                payload_length_word = await self.reader.readexactly(4)
+                payload_length = struct.unpack(endian + "I", payload_length_word)[0]
+                if payload_length > 10 * 512 * 1024:
+                    # 10x buffer size of firmware
+                    raise ValueError
+
+                # The remaining header length is 11 bytes.
+                remaining_data = await self.reader.readexactly(payload_length + 11)
+                data = endian_byte + payload_length_word + remaining_data
+                self.receive_cb(data)
+        except Exception:
+            logger.error("analyzer receiver connection terminating with exception", exc_info=True)
+        finally:
+            if self.disconnect_cb is not None:
+                self.disconnect_cb()
+
+
 def vcd_codes():
     codechars = [chr(i) for i in range(33, 127)]
     for n in count():
@@ -150,38 +234,129 @@ class VCDChannel:
         integer_cast = struct.unpack(">Q", struct.pack(">d", x))[0]
         self.set_value("{:064b}".format(integer_cast))
 
+    def set_log(self, log_message):
+        value = ""
+        for c in log_message:
+            value += "{:08b}".format(ord(c))
+        self.set_value(value)
+
 
 class VCDManager:
     def __init__(self, fileobj):
         self.out = fileobj
         self.codes = vcd_codes()
         self.current_time = None
+        self.start_time = 0
 
     def set_timescale_ps(self, timescale):
         self.out.write("$timescale {}ps $end\n".format(round(timescale)))
 
-    def get_channel(self, name, width):
+    def get_channel(self, name, width, ty, precision=0, unit=""):
         code = next(self.codes)
         self.out.write("$var wire {width} {code} {name} $end\n"
                        .format(name=name, code=code, width=width))
         return VCDChannel(self.out, code)
 
     @contextmanager
-    def scope(self, name):
-        self.out.write("$scope module {} $end\n".format(name))
+    def scope(self, scope, name):
+        self.out.write("$scope module {}/{} $end\n".format(scope, name))
         yield
         self.out.write("$upscope $end\n")
 
     def set_time(self, time):
+        time -= self.start_time
         if time != self.current_time:
             self.out.write("#{}\n".format(time))
             self.current_time = time
 
+    def set_start_time(self, time):
+        self.start_time = time
+
+    def set_end_time(self, time):
+        pass
+
+
+class WaveformManager:
+    def __init__(self):
+        self.current_time = 0
+        self.start_time = 0
+        self.end_time = 0
+        self.channels = list()
+        self.current_scope = ""
+        self.trace = {"timescale": 1, "stopped_x": None, "logs": dict(), "data": dict()}
+
+    def set_timescale_ps(self, timescale):
+        self.trace["timescale"] = int(timescale)
+
+    def get_channel(self, name, width, ty, precision=0, unit=""):
+        if ty == WaveformType.LOG:
+            self.trace["logs"][self.current_scope + name] = (ty, width, precision, unit)
+        data = self.trace["data"][self.current_scope + name] = list()
+        channel = WaveformChannel(data, self.current_time)
+        self.channels.append(channel)
+        return channel
+
+    @contextmanager
+    def scope(self, scope, name):
+        old_scope = self.current_scope
+        self.current_scope = scope + "/"
+        yield
+        self.current_scope = old_scope
+
+    def set_time(self, time):
+        time -= self.start_time
+        for channel in self.channels:
+            channel.set_time(time)
+
+    def set_start_time(self, time):
+        self.start_time = time
+        if self.trace["stopped_x"] is not None:
+            self.trace["stopped_x"] = self.end_time - self.start_time
+
+    def set_end_time(self, time):
+        self.end_time = time
+        self.trace["stopped_x"] = self.end_time - self.start_time
+
+
+class WaveformChannel:
+    def __init__(self, data, current_time):
+        self.data = data
+        self.current_time = current_time
+
+    def set_value(self, value):
+        self.data.append((self.current_time, value))
+
+    def set_value_double(self, x):
+        self.data.append((self.current_time, x))
+
+    def set_time(self, time):
+        self.current_time = time
+
+    def set_log(self, log_message):
+        self.data.append((self.current_time, log_message))
+
+
+class ChannelSignatureManager:
+    def __init__(self):
+        self.current_scope = ""
+        self.channels = dict()
+
+    def get_channel(self, name, width, ty, precision=0, unit=""):
+        self.channels[self.current_scope + name] = (ty, width, precision, unit)
+        return None
+
+    @contextmanager
+    def scope(self, scope, name):
+        old_scope = self.current_scope
+        self.current_scope = scope + "/"
+        yield
+        self.current_scope = old_scope
+
 
 class TTLHandler:
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self.name = name
-        self.channel_value = vcd_manager.get_channel("ttl/" + name, 1)
+        self.channel_value = manager.get_channel("ttl/" + name, 1, ty=WaveformType.BIT)
         self.last_value = "X"
         self.oe = True
 
@@ -206,11 +381,12 @@ class TTLHandler:
 
 
 class TTLClockGenHandler:
-    def __init__(self, vcd_manager, name, ref_period):
+    def __init__(self, manager, name, ref_period):
         self.name = name
         self.ref_period = ref_period
-        self.channel_frequency = vcd_manager.get_channel(
-            "ttl_clkgen/" + name, 64)
+        precision = max(0, math.ceil(math.log10(2**24 * ref_period) + 6))
+        self.channel_frequency = manager.get_channel(
+            "ttl_clkgen/" + name, 64, ty=WaveformType.ANALOG, precision=precision, unit="MHz")
 
     def process_message(self, message):
         if isinstance(message, OutputMessage):
@@ -221,8 +397,8 @@ class TTLClockGenHandler:
 
 
 class DDSHandler:
-    def __init__(self, vcd_manager, onehot_sel, sysclk):
-        self.vcd_manager = vcd_manager
+    def __init__(self, manager, onehot_sel, sysclk):
+        self.manager = manager
         self.onehot_sel = onehot_sel
         self.sysclk = sysclk
 
@@ -231,11 +407,18 @@ class DDSHandler:
 
     def add_dds_channel(self, name, dds_channel_nr):
         dds_channel = dict()
-        with self.vcd_manager.scope("dds/{}".format(name)):
+        frequency_precision = max(0, math.ceil(math.log10(2**32 / self.sysclk) + 6))
+        phase_precision = max(0, math.ceil(math.log10(2**16)))
+        with self.manager.scope("dds", name):
             dds_channel["vcd_frequency"] = \
-                self.vcd_manager.get_channel(name + "/frequency", 64)
+                self.manager.get_channel(name + "/frequency", 64, 
+                                         ty=WaveformType.ANALOG, 
+                                         precision=frequency_precision,
+                                         unit="MHz")
             dds_channel["vcd_phase"] = \
-                self.vcd_manager.get_channel(name + "/phase", 64)
+                self.manager.get_channel(name + "/phase", 64, 
+                                         ty=WaveformType.ANALOG,
+                                         precision=phase_precision)
         dds_channel["ftw"] = [None, None]
         dds_channel["pow"] = None
         self.dds_channels[dds_channel_nr] = dds_channel
@@ -285,10 +468,10 @@ class DDSHandler:
 
 
 class WishboneHandler:
-    def __init__(self, vcd_manager, name, read_bit):
+    def __init__(self, manager, name, read_bit):
         self._reads = []
         self._read_bit = read_bit
-        self.stb = vcd_manager.get_channel("{}/{}".format(name, "stb"), 1)
+        self.stb = manager.get_channel(name + "/stb", 1, ty=WaveformType.BIT)
 
     def process_message(self, message):
         self.stb.set_value("1")
@@ -318,16 +501,17 @@ class WishboneHandler:
 
 
 class SPIMasterHandler(WishboneHandler):
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self.channels = {}
-        with vcd_manager.scope("spi/{}".format(name)):
-            super().__init__(vcd_manager, name, read_bit=0b100)
+        self.scope = "spi"
+        with manager.scope("spi", name):
+            super().__init__(manager, name, read_bit=0b100)
             for reg_name, reg_width in [
                     ("config", 32), ("chip_select", 16),
                     ("write_length", 8), ("read_length", 8),
                     ("write", 32), ("read", 32)]:
-                self.channels[reg_name] = vcd_manager.get_channel(
-                        "{}/{}".format(name, reg_name), reg_width)
+                self.channels[reg_name] = manager.get_channel(
+                    "{}/{}".format(name, reg_name), reg_width, ty=WaveformType.VECTOR)
 
     def process_write(self, address, data):
         if address == 0:
@@ -352,11 +536,12 @@ class SPIMasterHandler(WishboneHandler):
 
 
 class SPIMaster2Handler(WishboneHandler):
-    def __init__(self, vcd_manager, name):
+    def __init__(self, manager, name):
         self._reads = []
         self.channels = {}
-        with vcd_manager.scope("spi2/{}".format(name)):
-            self.stb = vcd_manager.get_channel("{}/{}".format(name, "stb"), 1)
+        self.scope = "spi2"
+        with manager.scope("spi2", name):
+            self.stb = manager.get_channel(name + "/stb", 1, ty=WaveformType.BIT)
             for reg_name, reg_width in [
                     ("flags", 8),
                     ("length", 5),
@@ -364,8 +549,8 @@ class SPIMaster2Handler(WishboneHandler):
                     ("chip_select", 8),
                     ("write", 32),
                     ("read", 32)]:
-                self.channels[reg_name] = vcd_manager.get_channel(
-                        "{}/{}".format(name, reg_name), reg_width)
+                self.channels[reg_name] = manager.get_channel(
+                    "{}/{}".format(name, reg_name), reg_width, ty=WaveformType.VECTOR)
 
     def process_message(self, message):
         self.stb.set_value("1")
@@ -413,11 +598,12 @@ def _extract_log_chars(data):
 
 
 class LogHandler:
-    def __init__(self, vcd_manager, vcd_log_channels):
-        self.vcd_channels = dict()
-        for name, maxlength in vcd_log_channels.items():
-            self.vcd_channels[name] = vcd_manager.get_channel("log/" + name,
-                                                              maxlength*8)
+    def __init__(self, manager, log_channels):
+        self.channels = dict()
+        for name, maxlength in log_channels.items():
+            self.channels[name] = manager.get_channel("logs/" + name,
+                                                      maxlength * 8,
+                                                      ty=WaveformType.LOG)
         self.current_entry = ""
 
     def process_message(self, message):
@@ -425,15 +611,12 @@ class LogHandler:
             self.current_entry += _extract_log_chars(message.data)
             if len(self.current_entry) > 1 and self.current_entry[-1] == "\x1D":
                 channel_name, log_message = self.current_entry[:-1].split("\x1E", maxsplit=1)
-                vcd_value = ""
-                for c in log_message:
-                    vcd_value += "{:08b}".format(ord(c))
-                self.vcd_channels[channel_name].set_value(vcd_value)
+                self.channels[channel_name].set_log(log_message)
                 self.current_entry = ""
 
 
-def get_vcd_log_channels(log_channel, messages):
-    vcd_log_channels = dict()
+def get_log_channels(log_channel, messages):
+    log_channels = dict()
     log_entry = ""
     for message in messages:
         if (isinstance(message, OutputMessage)
@@ -442,13 +625,13 @@ def get_vcd_log_channels(log_channel, messages):
             if len(log_entry) > 1 and log_entry[-1] == "\x1D":
                 channel_name, log_message = log_entry[:-1].split("\x1E", maxsplit=1)
                 l = len(log_message)
-                if channel_name in vcd_log_channels:
-                    if vcd_log_channels[channel_name] < l:
-                        vcd_log_channels[channel_name] = l
+                if channel_name in log_channels:
+                    if log_channels[channel_name] < l:
+                        log_channels[channel_name] = l
                 else:
-                    vcd_log_channels[channel_name] = l
+                    log_channels[channel_name] = l
                 log_entry = ""
-    return vcd_log_channels
+    return log_channels
 
 
 def get_single_device_argument(devices, module, cls, argument):
@@ -475,7 +658,7 @@ def get_dds_sysclk(devices):
                                       ("AD9914",), "sysclk")
 
 
-def create_channel_handlers(vcd_manager, devices, ref_period,
+def create_channel_handlers(manager, devices, ref_period,
                             dds_sysclk, dds_onehot_sel):
     channel_handlers = dict()
     for name, desc in sorted(devices.items(), key=itemgetter(0)):
@@ -483,11 +666,11 @@ def create_channel_handlers(vcd_manager, devices, ref_period,
             if (desc["module"] == "artiq.coredevice.ttl"
                     and desc["class"] in {"TTLOut", "TTLInOut"}):
                 channel = desc["arguments"]["channel"]
-                channel_handlers[channel] = TTLHandler(vcd_manager, name)
+                channel_handlers[channel] = TTLHandler(manager, name)
             if (desc["module"] == "artiq.coredevice.ttl"
                     and desc["class"] == "TTLClockGen"):
                 channel = desc["arguments"]["channel"]
-                channel_handlers[channel] = TTLClockGenHandler(vcd_manager, name, ref_period)
+                channel_handlers[channel] = TTLClockGenHandler(manager, name, ref_period)
             if (desc["module"] == "artiq.coredevice.ad9914"
                     and desc["class"] == "AD9914"):
                 dds_bus_channel = desc["arguments"]["bus_channel"]
@@ -495,37 +678,60 @@ def create_channel_handlers(vcd_manager, devices, ref_period,
                 if dds_bus_channel in channel_handlers:
                     dds_handler = channel_handlers[dds_bus_channel]
                 else:
-                    dds_handler = DDSHandler(vcd_manager, dds_onehot_sel, dds_sysclk)
+                    dds_handler = DDSHandler(manager, dds_onehot_sel, dds_sysclk)
                     channel_handlers[dds_bus_channel] = dds_handler
                 dds_handler.add_dds_channel(name, dds_channel)
             if (desc["module"] == "artiq.coredevice.spi2" and
                     desc["class"] == "SPIMaster"):
                 channel = desc["arguments"]["channel"]
                 channel_handlers[channel] = SPIMaster2Handler(
-                        vcd_manager, name)
+                        manager, name)
     return channel_handlers
 
 
+def get_channel_list(devices):
+    manager = ChannelSignatureManager()
+    create_channel_handlers(manager, devices, 1e-9, 3e9, False)
+    ref_period = get_ref_period(devices)
+    if ref_period is None:
+        ref_period = DEFAULT_REF_PERIOD
+    precision = max(0, math.ceil(math.log10(1 / ref_period) - 6))
+    manager.get_channel("rtio_slack", 64, ty=WaveformType.ANALOG, precision=precision, unit="us")
+    return manager.channels
+
+
 def get_message_time(message):
     return getattr(message, "timestamp", message.rtio_counter)
 
 
 def decoded_dump_to_vcd(fileobj, devices, dump, uniform_interval=False):
     vcd_manager = VCDManager(fileobj)
+    decoded_dump_to_target(vcd_manager, devices, dump, uniform_interval)
+
+
+def decoded_dump_to_waveform_data(devices, dump, uniform_interval=False):
+    manager = WaveformManager()
+    decoded_dump_to_target(manager, devices, dump, uniform_interval)
+    return manager.trace
+
+
+def decoded_dump_to_target(manager, devices, dump, uniform_interval):
     ref_period = get_ref_period(devices)
 
-    if ref_period is not None:
-        if not uniform_interval:
-            vcd_manager.set_timescale_ps(ref_period*1e12)
-    else:
+    if ref_period is None:
         logger.warning("unable to determine core device ref_period")
-        ref_period = 1e-9  # guess
+        ref_period = DEFAULT_REF_PERIOD
+    if not uniform_interval:
+        manager.set_timescale_ps(ref_period*1e12)
     dds_sysclk = get_dds_sysclk(devices)
     if dds_sysclk is None:
         logger.warning("unable to determine DDS sysclk")
         dds_sysclk = 3e9  # guess
 
     if isinstance(dump.messages[-1], StoppedMessage):
+        m = dump.messages[-1]
+        end_time = get_message_time(m)
+        manager.set_end_time(end_time)
         messages = dump.messages[:-1]
     else:
         logger.warning("StoppedMessage missing")
@@ -533,38 +739,39 @@ def decoded_dump_to_vcd(fileobj, devices, dump, uniform_interval=False):
     messages = sorted(messages, key=get_message_time)
 
     channel_handlers = create_channel_handlers(
-        vcd_manager, devices, ref_period,
+        manager, devices, ref_period,
         dds_sysclk, dump.dds_onehot_sel)
-    vcd_log_channels = get_vcd_log_channels(dump.log_channel, messages)
+    log_channels = get_log_channels(dump.log_channel, messages)
     channel_handlers[dump.log_channel] = LogHandler(
-        vcd_manager, vcd_log_channels)
+        manager, log_channels)
     if uniform_interval:
         # RTIO event timestamp in machine units
-        timestamp = vcd_manager.get_channel("timestamp", 64)
+        timestamp = manager.get_channel("timestamp", 64, ty=WaveformType.VECTOR)
         # RTIO time interval between this and the next timed event
         # in SI seconds
-        interval = vcd_manager.get_channel("interval", 64)
-    slack = vcd_manager.get_channel("rtio_slack", 64)
+        interval = manager.get_channel("interval", 64, ty=WaveformType.ANALOG)
+    slack = manager.get_channel("rtio_slack", 64, ty=WaveformType.ANALOG)
 
-    vcd_manager.set_time(0)
+    manager.set_time(0)
     start_time = 0
     for m in messages:
         start_time = get_message_time(m)
         if start_time:
             break
-
-    t0 = 0
+    if not uniform_interval:
+        manager.set_start_time(start_time)
+    t0 = start_time
     for i, message in enumerate(messages):
         if message.channel in channel_handlers:
-            t = get_message_time(message) - start_time
+            t = get_message_time(message)
             if t >= 0:
                 if uniform_interval:
                     interval.set_value_double((t - t0)*ref_period)
-                    vcd_manager.set_time(i)
+                    manager.set_time(i)
                     timestamp.set_value("{:064b}".format(t))
                     t0 = t
                 else:
-                    vcd_manager.set_time(t)
+                    manager.set_time(t)
             channel_handlers[message.channel].process_message(message)
             if isinstance(message, OutputMessage):
                 slack.set_value_double(

artiq/coredevice/comm_kernel.py

@@ -465,12 +465,12 @@ class CommKernel:
             self._write_bool(value)
         elif tag == "i":
             check(isinstance(value, (int, numpy.int32)) and
-                  (-2**31 <= value < 2**31),
+                  (-2**31 <= value <= 2**31-1),
                   lambda: "32-bit int")
             self._write_int32(value)
         elif tag == "I":
             check(isinstance(value, (int, numpy.int32, numpy.int64)) and
-                  (-2**63 <= value < 2**63),
+                  (-2**63 <= value <= 2**63-1),
                   lambda: "64-bit int")
             self._write_int64(value)
         elif tag == "f":
@@ -479,8 +479,8 @@ class CommKernel:
             self._write_float64(value)
         elif tag == "F":
             check(isinstance(value, Fraction) and
-                  (-2**63 <= value.numerator < 2**63) and
-                  (-2**63 <= value.denominator < 2**63),
+                  (-2**63 <= value.numerator <= 2**63-1) and
+                  (-2**63 <= value.denominator <= 2**63-1),
                   lambda: "64-bit Fraction")
             self._write_int64(value.numerator)
             self._write_int64(value.denominator)

artiq/coredevice/comm_moninj.py

@@ -94,9 +94,7 @@ class CommMonInj:
                     self.injection_status_cb(channel, override, value)
                 else:
                     raise ValueError("Unknown packet type", ty)
-        except asyncio.CancelledError:
-            raise
-        except:
+        except Exception:
             logger.error("Moninj connection terminating with exception", exc_info=True)
         finally:
             if self.disconnect_cb is not None:

artiq/coredevice/core.py

@@ -120,13 +120,15 @@ class Core:
 
     def compile(self, function, args, kwargs, set_result=None,
                 attribute_writeback=True, print_as_rpc=True,
-                target=None, destination=0, subkernel_arg_types=[]):
+                target=None, destination=0, subkernel_arg_types=[],
+                old_embedding_map=None):
         try:
             engine = _DiagnosticEngine(all_errors_are_fatal=True)
 
             stitcher = Stitcher(engine=engine, core=self, dmgr=self.dmgr,
                                 print_as_rpc=print_as_rpc,
-                                destination=destination, subkernel_arg_types=subkernel_arg_types)
+                                destination=destination, subkernel_arg_types=subkernel_arg_types,
+                                old_embedding_map=old_embedding_map)
             stitcher.stitch_call(function, args, kwargs, set_result)
             stitcher.finalize()
 
@@ -165,7 +167,7 @@ class Core:
         self._run_compiled(kernel_library, embedding_map, symbolizer, demangler)
         return result
 
-    def compile_subkernel(self, sid, subkernel_fn, embedding_map, args, subkernel_arg_types):
+    def compile_subkernel(self, sid, subkernel_fn, embedding_map, args, subkernel_arg_types, subkernels):
         # pass self to subkernels (if applicable)
         # assuming the first argument is self
         subkernel_args = getfullargspec(subkernel_fn.artiq_embedded.function)
@@ -179,17 +181,49 @@ class Core:
         object_map, kernel_library, _, _, _ = \
             self.compile(subkernel_fn, self_arg, {}, attribute_writeback=False,
                         print_as_rpc=False, target=target, destination=destination, 
-                        subkernel_arg_types=subkernel_arg_types.get(sid, []))
-        if object_map.has_rpc_or_subkernel():
-            raise ValueError("Subkernel must not use RPC or subkernels in other destinations")
-        return destination, kernel_library
+                        subkernel_arg_types=subkernel_arg_types.get(sid, []),
+                        old_embedding_map=embedding_map)
+        if object_map.has_rpc():
+            raise ValueError("Subkernel must not use RPC")
+        return destination, kernel_library, object_map
 
     def compile_and_upload_subkernels(self, embedding_map, args, subkernel_arg_types):
-        for sid, subkernel_fn in embedding_map.subkernels().items():
-            destination, kernel_library = \
-                self.compile_subkernel(sid, subkernel_fn, embedding_map,
-                                       args, subkernel_arg_types)
-            self.comm.upload_subkernel(kernel_library, sid, destination)
+        subkernels = embedding_map.subkernels()
+        subkernels_compiled = []
+        while True:
+            new_subkernels = {}
+            for sid, subkernel_fn in subkernels.items():
+                if sid in subkernels_compiled:
+                    continue
+                destination, kernel_library, embedding_map = \
+                    self.compile_subkernel(sid, subkernel_fn, embedding_map,
+                                        args, subkernel_arg_types, subkernels)
+                self.comm.upload_subkernel(kernel_library, sid, destination)
+                new_subkernels.update(embedding_map.subkernels())
+                subkernels_compiled.append(sid)
+            if new_subkernels == subkernels:
+                break
+            subkernels.update(new_subkernels)
+        # check for messages without a send/recv pair
+        unpaired_messages = embedding_map.subkernel_messages_unpaired()
+        if unpaired_messages:
+            for unpaired_message in unpaired_messages:
+                engine = _DiagnosticEngine(all_errors_are_fatal=False)
+                # errors are non-fatal in order to display
+                # all unpaired message errors before raising an excption
+                if unpaired_message.send_loc is None:
+                    diag = diagnostic.Diagnostic("error",
+                        "subkernel message '{name}' only has a receiver but no sender",
+                        {"name": unpaired_message.name},
+                        unpaired_message.recv_loc)
+                else:
+                    diag = diagnostic.Diagnostic("error",
+                        "subkernel message '{name}' only has a sender but no receiver",
+                        {"name": unpaired_message.name},
+                        unpaired_message.send_loc)
+                engine.process(diag)
+            raise ValueError("Found subkernel message(s) without a full send/recv pair")
+
 
     def precompile(self, function, *args, **kwargs):
         """Precompile a kernel and return a callable that executes it on the core device
@@ -319,6 +353,4 @@ class Core:
         if self.analyzer_proxy is None:
             raise IOError("No analyzer proxy configured")
         else:
-            success = self.analyzer_proxy.trigger()
-            if not success:
-                raise IOError("Analyzer proxy reported failure")
+            self.analyzer_proxy.trigger()

artiq/coredevice/coredevice_generic.schema.json

@@ -49,6 +49,10 @@
             "default": 125e6,
             "description": "RTIO frequency"
         },
+        "enable_wrpll": {
+            "type": "boolean",
+            "default": false
+        },
         "core_addr": {
             "type": "string",
             "format": "ipv4",
@@ -308,8 +312,7 @@
                         "clk_div": {
                             "type": "integer",
                             "minimum": 0,
-                            "maximum": 3,
-                            "default": 0
+                            "maximum": 3
                         },
                         "pll_n": {
                             "type": "integer"
@@ -631,6 +634,10 @@
                         },
                         "drtio_destination": {
                             "type": "integer"
+                        },
+                        "hw_rev": {
+                            "type": "string",
+                            "enum": ["v1.0", "v1.1"]
                         }
                     },
                     "required": ["ports"]

artiq/coredevice/grabber.py

@@ -2,7 +2,7 @@ from numpy import int32, int64
 
 from artiq.language.core import *
 from artiq.language.types import *
-from artiq.coredevice.rtio import rtio_output, rtio_input_data
+from artiq.coredevice.rtio import rtio_output, rtio_input_timestamped_data
 
 
 class OutOfSyncException(Exception):
@@ -11,6 +11,11 @@ class OutOfSyncException(Exception):
     pass
 
 
+class GrabberTimeoutException(Exception):
+    """Raised when a timeout occurs while attempting to read Grabber RTIO input events."""
+    pass
+
+
 class Grabber:
     """Driver for the Grabber camera interface."""
     kernel_invariants = {"core", "channel_base", "sentinel"}
@@ -82,10 +87,10 @@ class Grabber:
         self.gate_roi(0)
 
     @kernel
-    def input_mu(self, data):
+    def input_mu(self, data, timeout_mu=-1):
         """
         Retrieves the accumulated values for one frame from the ROI engines.
-        Blocks until values are available.
+        Blocks until values are available or timeout is reached.
 
         The input list must be a list of integers of the same length as there
         are enabled ROI engines. This method replaces the elements of the
@@ -95,15 +100,26 @@ class Grabber:
         If the number of elements in the list does not match the number of
         ROI engines that produced output, an exception will be raised during
         this call or the next.
+        
+        If the timeout is reached before data is available, the exception
+        GrabberTimeoutException is raised.
+
+        :param timeout_mu: Timestamp at which a timeout will occur. Set to -1
+                           (default) to disable timeout. 
         """
         channel = self.channel_base + 1
 
-        sentinel = rtio_input_data(channel)
+        timestamp, sentinel = rtio_input_timestamped_data(timeout_mu, channel)
+        if timestamp == -1:
+            raise GrabberTimeoutException("Timeout before Grabber frame available")
         if sentinel != self.sentinel:
             raise OutOfSyncException
 
         for i in range(len(data)):
-            roi_output = rtio_input_data(channel)
+            timestamp, roi_output = rtio_input_timestamped_data(timeout_mu, channel)
             if roi_output == self.sentinel:
                 raise OutOfSyncException
+            if timestamp == -1:
+                raise GrabberTimeoutException(
+                    "Timeout retrieving ROIs (attempting to read more ROIs than enabled?)")
             data[i] = roi_output

artiq/coredevice/kasli_i2c.py

@@ -25,14 +25,14 @@ port_mapping = {
 
 
 class KasliEEPROM:
-    def __init__(self, dmgr, port, busno=0,
+    def __init__(self, dmgr, port, address=0xa0, busno=0,
             core_device="core", sw0_device="i2c_switch0", sw1_device="i2c_switch1"):
         self.core = dmgr.get(core_device)
         self.sw0 = dmgr.get(sw0_device)
         self.sw1 = dmgr.get(sw1_device)
         self.busno = busno
         self.port = port_mapping[port]
-        self.address = 0xa0  # i2c 8 bit
+        self.address = address  # i2c 8 bit
 
     @kernel
     def select(self):

artiq/coredevice/sampler.py

@@ -21,7 +21,7 @@ def adc_mu_to_volt(data, gain=0, corrected_fs=True):
     :param data: 16 bit signed ADC word
     :param gain: PGIA gain setting (0: 1, ..., 3: 1000)
     :param corrected_fs: use corrected ADC FS reference.
-    Should be True for Samplers' revisions after v2.1. False for v2.1 and earlier.
+        Should be True for Samplers' revisions after v2.1. False for v2.1 and earlier.
     :return: Voltage in Volts
     """
     if gain == 0:

artiq/dashboard/applets_ccb.py

@@ -36,7 +36,7 @@ class AppletsCCBDock(applets.AppletsDock):
         ccbp_group_menu.addAction(self.ccbp_group_create)
         actiongroup.addAction(self.ccbp_group_create)
         self.ccbp_group_enable = QtWidgets.QAction("Create and enable/disable applets",
-                                                    self.table)
+                                                   self.table)
         self.ccbp_group_enable.setCheckable(True)
         self.ccbp_group_enable.triggered.connect(lambda: self.set_ccbp("enable"))
         ccbp_group_menu.addAction(self.ccbp_group_enable)

artiq/dashboard/datasets.py

@@ -8,7 +8,6 @@ from sipyco import pyon
 from artiq.tools import scale_from_metadata, short_format, exc_to_warning
 from artiq.gui.tools import LayoutWidget, QRecursiveFilterProxyModel
 from artiq.gui.models import DictSyncTreeSepModel
-from artiq.gui.scientific_spinbox import ScientificSpinBox
 
 
 logger = logging.getLogger(__name__)
@@ -81,8 +80,7 @@ class CreateEditDialog(QtWidgets.QDialog):
             t = value.dtype if value is np.ndarray else type(value)
             if scale != 1 and np.issubdtype(t, np.number):
                 # degenerates to float type
-                value_edit_string = self.value_to_edit_string(
-                        np.float64(value / scale))
+                value_edit_string = self.value_to_edit_string(value / scale)
             self.unit_widget.setText(metadata.get('unit', ''))
             self.scale_widget.setText(str(metadata.get('scale', '')))
             self.precision_widget.setText(str(metadata.get('precision', '')))
@@ -109,11 +107,13 @@ class CreateEditDialog(QtWidgets.QDialog):
         t = value.dtype if value is np.ndarray else type(value)
         if scale != 1 and np.issubdtype(t, np.number):
             # degenerates to float type
-            value = np.float64(value * scale)
+            value = float(value * scale)
         if self.key and self.key != key:
-            asyncio.ensure_future(exc_to_warning(rename(self.key, key, value, metadata, persist, self.dataset_ctl)))
+            asyncio.ensure_future(exc_to_warning(rename(self.key, key, value, metadata, persist,
+                                                        self.dataset_ctl)))
         else:
-            asyncio.ensure_future(exc_to_warning(self.dataset_ctl.set(key, value, metadata=metadata, persist=persist)))
+            asyncio.ensure_future(exc_to_warning(self.dataset_ctl.set(key, value, metadata=metadata,
+                                                                      persist=persist)))
         self.key = key
         QtWidgets.QDialog.accept(self)
 
@@ -163,7 +163,7 @@ class CreateEditDialog(QtWidgets.QDialog):
 
 
 class Model(DictSyncTreeSepModel):
-    def __init__(self,  init):
+    def __init__(self, init):
         DictSyncTreeSepModel.__init__(self, ".",
                                       ["Dataset", "Persistent", "Value"],
                                       init)

artiq/dashboard/experiments.py

@@ -9,10 +9,9 @@ import h5py
 
 from sipyco import pyon
 
-from artiq.gui.entries import procdesc_to_entry, ScanEntry
+from artiq.gui.entries import procdesc_to_entry, EntryTreeWidget
 from artiq.gui.fuzzy_select import FuzzySelectWidget
-from artiq.gui.tools import (LayoutWidget, WheelFilter, 
-                             log_level_to_name, get_open_file_name)
+from artiq.gui.tools import (LayoutWidget, log_level_to_name, get_open_file_name)
 from artiq.tools import parse_devarg_override, unparse_devarg_override
 
 
@@ -25,99 +24,23 @@ logger = logging.getLogger(__name__)
 # 2. file:<class name>@<file name>
 
 
-class _ArgumentEditor(QtWidgets.QTreeWidget):
+class _ArgumentEditor(EntryTreeWidget):
     def __init__(self, manager, dock, expurl):
         self.manager = manager
         self.expurl = expurl
 
-        QtWidgets.QTreeWidget.__init__(self)
-        self.setColumnCount(3)
-        self.header().setStretchLastSection(False)
-        if hasattr(self.header(), "setSectionResizeMode"):
-            set_resize_mode = self.header().setSectionResizeMode
-        else:
-            set_resize_mode = self.header().setResizeMode
-        set_resize_mode(0, QtWidgets.QHeaderView.ResizeToContents)
-        set_resize_mode(1, QtWidgets.QHeaderView.Stretch)
-        set_resize_mode(2, QtWidgets.QHeaderView.ResizeToContents)
-        self.header().setVisible(False)
-        self.setSelectionMode(self.NoSelection)
-        self.setHorizontalScrollMode(self.ScrollPerPixel)
-        self.setVerticalScrollMode(self.ScrollPerPixel)
-
-        self.setStyleSheet("QTreeWidget {background: " +
-                           self.palette().midlight().color().name() + " ;}")
-
-        self.viewport().installEventFilter(WheelFilter(self.viewport(), True))
-
-        self._groups = dict()
-        self._arg_to_widgets = dict()
+        EntryTreeWidget.__init__(self)
 
         arguments = self.manager.get_submission_arguments(self.expurl)
 
         if not arguments:
-            self.addTopLevelItem(QtWidgets.QTreeWidgetItem(["No arguments"]))
+            self.insertTopLevelItem(0, QtWidgets.QTreeWidgetItem(["No arguments"]))
 
-        gradient = QtGui.QLinearGradient(
-            0, 0, 0, QtGui.QFontMetrics(self.font()).lineSpacing()*2.5)
-        gradient.setColorAt(0, self.palette().base().color())
-        gradient.setColorAt(1, self.palette().midlight().color())
         for name, argument in arguments.items():
-            widgets = dict()
-            self._arg_to_widgets[name] = widgets
+            self.set_argument(name, argument)
 
-            entry = procdesc_to_entry(argument["desc"])(argument)
-            widget_item = QtWidgets.QTreeWidgetItem([name])
-            if argument["tooltip"]:
-                widget_item.setToolTip(0, argument["tooltip"])
-            widgets["entry"] = entry
-            widgets["widget_item"] = widget_item
+        self.quickStyleClicked.connect(dock.submit_clicked)
 
-            for col in range(3):
-                widget_item.setBackground(col, gradient)
-            font = widget_item.font(0)
-            font.setBold(True)
-            widget_item.setFont(0, font)
-
-            if argument["group"] is None:
-                self.addTopLevelItem(widget_item)
-            else:
-                self._get_group(argument["group"]).addChild(widget_item)
-            fix_layout = LayoutWidget()
-            widgets["fix_layout"] = fix_layout
-            fix_layout.addWidget(entry)
-            self.setItemWidget(widget_item, 1, fix_layout)
-            recompute_argument = QtWidgets.QToolButton()
-            recompute_argument.setToolTip("Re-run the experiment's build "
-                                          "method and take the default value")
-            recompute_argument.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_BrowserReload))
-            recompute_argument.clicked.connect(
-                partial(self._recompute_argument_clicked, name))
-
-            tool_buttons = LayoutWidget()
-            tool_buttons.addWidget(recompute_argument, 1)
-
-            disable_other_scans = QtWidgets.QToolButton()
-            widgets["disable_other_scans"] = disable_other_scans
-            disable_other_scans.setIcon(
-                QtWidgets.QApplication.style().standardIcon(
-                    QtWidgets.QStyle.SP_DialogResetButton))
-            disable_other_scans.setToolTip("Disable all other scans in "
-                                           "this experiment")
-            disable_other_scans.clicked.connect(
-                partial(self._disable_other_scans, name))
-            tool_buttons.layout.setRowStretch(0, 1)
-            tool_buttons.layout.setRowStretch(3, 1)
-            tool_buttons.addWidget(disable_other_scans, 2)
-            if not isinstance(entry, ScanEntry):
-                disable_other_scans.setVisible(False)
-
-            self.setItemWidget(widget_item, 2, tool_buttons)
-
-        widget_item = QtWidgets.QTreeWidgetItem()
-        self.addTopLevelItem(widget_item)
         recompute_arguments = QtWidgets.QPushButton("Recompute all arguments")
         recompute_arguments.setIcon(
             QtWidgets.QApplication.style().standardIcon(
@@ -136,41 +59,10 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         buttons.layout.setColumnStretch(1, 0)
         buttons.layout.setColumnStretch(2, 0)
         buttons.layout.setColumnStretch(3, 1)
-        self.setItemWidget(widget_item, 1, buttons)
+        self.setItemWidget(self.bottom_item, 1, buttons)
 
-    def _get_group(self, name):
-        if name in self._groups:
-            return self._groups[name]
-        group = QtWidgets.QTreeWidgetItem([name])
-        for col in range(3):
-            group.setBackground(col, self.palette().mid())
-            group.setForeground(col, self.palette().brightText())
-            font = group.font(col)
-            font.setBold(True)
-            group.setFont(col, font)
-        self.addTopLevelItem(group)
-        self._groups[name] = group
-        return group
-
-    def update_argument(self, name, argument):
-        widgets = self._arg_to_widgets[name]
-
-        # Qt needs a setItemWidget() to handle layout correctly,
-        # simply replacing the entry inside the LayoutWidget
-        # results in a bug.
-
-        widgets["entry"].deleteLater()
-        widgets["entry"] = procdesc_to_entry(argument["desc"])(argument)
-        widgets["disable_other_scans"].setVisible(
-            isinstance(widgets["entry"], ScanEntry))
-        widgets["fix_layout"].deleteLater()
-        widgets["fix_layout"] = LayoutWidget()
-        widgets["fix_layout"].addWidget(widgets["entry"])
-        self.setItemWidget(widgets["widget_item"], 1, widgets["fix_layout"])
-        self.updateGeometries()
-
-    def _recompute_argument_clicked(self, name):
-        asyncio.ensure_future(self._recompute_argument(name))
+    def reset_entry(self, key):
+        asyncio.ensure_future(self._recompute_argument(key))
 
     async def _recompute_argument(self, name):
         try:
@@ -187,30 +79,6 @@ class _ArgumentEditor(QtWidgets.QTreeWidget):
         argument["state"] = state
         self.update_argument(name, argument)
 
-    def _disable_other_scans(self, current_name):
-        for name, widgets in self._arg_to_widgets.items():
-            if (name != current_name
-                    and isinstance(widgets["entry"], ScanEntry)):
-                widgets["entry"].disable()
-
-    def save_state(self):
-        expanded = []
-        for k, v in self._groups.items():
-            if v.isExpanded():
-                expanded.append(k)
-        return {
-            "expanded": expanded,
-            "scroll": self.verticalScrollBar().value()
-        }
-
-    def restore_state(self, state):
-        for e in state["expanded"]:
-            try:
-                self._groups[e].setExpanded(True)
-            except KeyError:
-                pass
-        self.verticalScrollBar().setValue(state["scroll"])
-
     # Hooks that allow user-supplied argument editors to react to imminent user
     # actions. Here, we always keep the manager-stored submission arguments
     # up-to-date, so no further action is required.
@@ -230,7 +98,7 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
     def __init__(self, manager, expurl):
         QtWidgets.QMdiSubWindow.__init__(self)
         qfm = QtGui.QFontMetrics(self.font())
-        self.resize(100*qfm.averageCharWidth(), 30*qfm.lineSpacing())
+        self.resize(100 * qfm.averageCharWidth(), 30 * qfm.lineSpacing())
         self.setWindowTitle(expurl)
         self.setWindowIcon(QtWidgets.QApplication.style().standardIcon(
             QtWidgets.QStyle.SP_FileDialogContentsView))
@@ -263,17 +131,17 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
             datetime.setDate(QtCore.QDate.currentDate())
         else:
             datetime.setDateTime(QtCore.QDateTime.fromMSecsSinceEpoch(
-                int(scheduling["due_date"]*1000)))
+                int(scheduling["due_date"] * 1000)))
         datetime_en.setChecked(scheduling["due_date"] is not None)
 
         def update_datetime(dt):
-            scheduling["due_date"] = dt.toMSecsSinceEpoch()/1000
+            scheduling["due_date"] = dt.toMSecsSinceEpoch() / 1000
             datetime_en.setChecked(True)
         datetime.dateTimeChanged.connect(update_datetime)
 
         def update_datetime_en(checked):
             if checked:
-                due_date = datetime.dateTime().toMSecsSinceEpoch()/1000
+                due_date = datetime.dateTime().toMSecsSinceEpoch() / 1000
             else:
                 due_date = None
             scheduling["due_date"] = due_date
@@ -349,9 +217,10 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
             repo_rev = QtWidgets.QLineEdit()
             repo_rev.setPlaceholderText("current")
             repo_rev.setClearButtonEnabled(True)
-            repo_rev_label = QtWidgets.QLabel("Revision:")
+            repo_rev_label = QtWidgets.QLabel("Rev / ref:")
             repo_rev_label.setToolTip("Experiment repository revision "
-                                      "(commit ID) to use")
+                                      "(commit ID) or reference (branch "
+                                      "or tag) to use")
             self.layout.addWidget(repo_rev_label, 3, 2)
             self.layout.addWidget(repo_rev, 3, 3)
 
@@ -368,7 +237,7 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
 
         submit = QtWidgets.QPushButton("Submit")
         submit.setIcon(QtWidgets.QApplication.style().standardIcon(
-                QtWidgets.QStyle.SP_DialogOkButton))
+                       QtWidgets.QStyle.SP_DialogOkButton))
         submit.setToolTip("Schedule the experiment (Ctrl+Return)")
         submit.setShortcut("CTRL+RETURN")
         submit.setSizePolicy(QtWidgets.QSizePolicy.Expanding,
@@ -378,7 +247,7 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
 
         reqterm = QtWidgets.QPushButton("Terminate instances")
         reqterm.setIcon(QtWidgets.QApplication.style().standardIcon(
-                QtWidgets.QStyle.SP_DialogCancelButton))
+                        QtWidgets.QStyle.SP_DialogCancelButton))
         reqterm.setToolTip("Request termination of instances (Ctrl+Backspace)")
         reqterm.setShortcut("CTRL+BACKSPACE")
         reqterm.setSizePolicy(QtWidgets.QSizePolicy.Expanding,
@@ -420,8 +289,7 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
         arginfo = expdesc["arginfo"]
         for k, v in overrides.items():
             # Some values (e.g. scans) may have multiple defaults in a list
-            if ("default" in arginfo[k][0]
-                    and isinstance(arginfo[k][0]["default"], list)):
+            if ("default" in arginfo[k][0] and isinstance(arginfo[k][0]["default"], list)):
                 arginfo[k][0]["default"].insert(0, v)
             else:
                 arginfo[k][0]["default"] = v
@@ -432,8 +300,8 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
 
         editor_class = self.manager.get_argument_editor_class(self.expurl)
         self.argeditor = editor_class(self.manager, self, self.expurl)
-        self.argeditor.restore_state(argeditor_state)
         self.layout.addWidget(self.argeditor, 0, 0, 1, 5)
+        self.argeditor.restore_state(argeditor_state)
 
     def contextMenuEvent(self, event):
         menu = QtWidgets.QMenu(self)
@@ -486,9 +354,9 @@ class _ExperimentDock(QtWidgets.QMdiSubWindow):
                     unparse_devarg_override(expid["devarg_override"]))
             self.log_level.setCurrentIndex(log_levels.index(
                 log_level_to_name(expid["log_level"])))
-            if ("repo_rev" in expid and
-                    expid["repo_rev"] != "N/A" and
-                    hasattr(self, "repo_rev")):
+            if "repo_rev" in expid and \
+               expid["repo_rev"] != "N/A" and \
+               hasattr(self, "repo_rev"):
                 self.repo_rev.setText(expid["repo_rev"])
         except:
             logger.error("Could not set submission options from HDF5 expid",
@@ -678,7 +546,7 @@ class ExperimentManager:
         self.submission_arguments[expurl] = arguments
         self.argument_ui_names[expurl] = ui_name
         return arguments
-    
+
     def set_argument_value(self, expurl, name, value):
         try:
             argument = self.submission_arguments[expurl][name]
@@ -691,7 +559,8 @@ class ExperimentManager:
             if expurl in self.open_experiments.keys():
                 self.open_experiments[expurl].argeditor.update_argument(name, argument)
         except:
-            logger.warn("Failed to set value for argument \"{}\" in experiment: {}.".format(name, expurl), exc_info=1)
+            logger.warn("Failed to set value for argument \"{}\" in experiment: {}."
+                        .format(name, expurl), exc_info=1)
 
     def get_submission_arguments(self, expurl):
         if expurl in self.submission_arguments:
@@ -701,8 +570,8 @@ class ExperimentManager:
                 raise ValueError("Submission arguments must be preinitialized "
                                  "when not using repository")
             class_desc = self.explist[expurl[5:]]
-            return self.initialize_submission_arguments(expurl,
-                class_desc["arginfo"], class_desc.get("argument_ui", None))
+            return self.initialize_submission_arguments(expurl, class_desc["arginfo"],
+                                                        class_desc.get("argument_ui", None))
 
     def open_experiment(self, expurl):
         if expurl in self.open_experiments:
@@ -739,8 +608,13 @@ class ExperimentManager:
         del self.open_experiments[expurl]
 
     async def _submit_task(self, expurl, *args):
-        rid = await self.schedule_ctl.submit(*args)
-        logger.info("Submitted '%s', RID is %d", expurl, rid)
+        try:
+            rid = await self.schedule_ctl.submit(*args)
+        except KeyError:
+            expid = args[1]
+            logger.error("Submission failed - revision \"%s\" was not found", expid["repo_rev"])
+        else:
+            logger.info("Submitted '%s', RID is %d", expurl, rid)
 
     def submit(self, expurl):
         file, class_name, _ = self.resolve_expurl(expurl)
@@ -797,9 +671,9 @@ class ExperimentManager:
                 repo_match = "repo_rev" in expid
             else:
                 repo_match = "repo_rev" not in expid
-            if (repo_match and
-                    ("file" in expid and expid["file"] == file) and
-                    expid["class_name"] == class_name):
+            if repo_match and \
+               ("file" in expid and expid["file"] == file) and \
+               expid["class_name"] == class_name:
                 rids.append(rid)
         asyncio.ensure_future(self._request_term_multiple(rids))
 
@@ -819,7 +693,7 @@ class ExperimentManager:
         for class_name, class_desc in description.items():
             expurl = "file:{}@{}".format(class_name, file)
             self.initialize_submission_arguments(expurl, class_desc["arginfo"],
-                class_desc.get("argument_ui", None))
+                                                 class_desc.get("argument_ui", None))
             if expurl in self.open_experiments:
                 self.open_experiments[expurl].close()
             self.open_experiment(expurl)
@@ -853,6 +727,7 @@ class ExperimentManager:
 
         self.is_quick_open_shown = True
         dialog = _QuickOpenDialog(self)
+
         def closed():
             self.is_quick_open_shown = False
         dialog.closed.connect(closed)

artiq/dashboard/explorer.py

@@ -94,7 +94,7 @@ class _OpenFileDialog(QtWidgets.QDialog):
                         else:
                             break
                     self.explorer.current_directory = \
-                        self.explorer.current_directory[:idx+1]
+                        self.explorer.current_directory[:idx + 1]
                     if self.explorer.current_directory == "/":
                         self.explorer.current_directory = ""
                 asyncio.ensure_future(self.refresh_view())
@@ -103,6 +103,7 @@ class _OpenFileDialog(QtWidgets.QDialog):
                 asyncio.ensure_future(self.refresh_view())
             else:
                 file = self.explorer.current_directory + selected
+
                 async def open_task():
                     try:
                         await self.exp_manager.open_file(file)
@@ -232,7 +233,7 @@ class ExplorerDock(QtWidgets.QDockWidget):
 
         set_shortcut_menu = QtWidgets.QMenu()
         for i in range(12):
-            action = QtWidgets.QAction("F" + str(i+1), self.el)
+            action = QtWidgets.QAction("F" + str(i + 1), self.el)
             action.triggered.connect(partial(self.set_shortcut, i))
             set_shortcut_menu.addAction(action)
 
@@ -246,12 +247,14 @@ class ExplorerDock(QtWidgets.QDockWidget):
 
         scan_repository_action = QtWidgets.QAction("Scan repository HEAD",
                                                    self.el)
+
         def scan_repository():
             asyncio.ensure_future(experiment_db_ctl.scan_repository_async())
         scan_repository_action.triggered.connect(scan_repository)
         self.el.addAction(scan_repository_action)
 
         scan_ddb_action = QtWidgets.QAction("Scan device database", self.el)
+
         def scan_ddb():
             asyncio.ensure_future(device_db_ctl.scan())
         scan_ddb_action.triggered.connect(scan_ddb)
@@ -292,7 +295,7 @@ class ExplorerDock(QtWidgets.QDockWidget):
         if expname is not None:
             expurl = "repo:" + expname
             self.d_shortcuts.set_shortcut(nr, expurl)
-            logger.info("Set shortcut F%d to '%s'", nr+1, expurl)
+            logger.info("Set shortcut F%d to '%s'", nr + 1, expurl)
 
     def update_scanning(self, scanning):
         if scanning:

artiq/dashboard/interactive_args.py

@@ -0,0 +1,155 @@
+import logging
+import asyncio
+
+from PyQt5 import QtCore, QtWidgets, QtGui
+
+from artiq.gui.models import DictSyncModel
+from artiq.gui.entries import EntryTreeWidget, procdesc_to_entry
+from artiq.gui.tools import LayoutWidget
+
+
+logger = logging.getLogger(__name__)
+
+
+class Model(DictSyncModel):
+    def __init__(self, init):
+        DictSyncModel.__init__(self, ["RID", "Title", "Args"], init)
+
+    def convert(self, k, v, column):
+        if column == 0:
+            return k
+        elif column == 1:
+            txt = ": " + v["title"] if v["title"] != "" else ""
+            return str(k) + txt
+        elif column == 2:
+            return v["arglist_desc"]
+        else:
+            raise ValueError
+
+    def sort_key(self, k, v):
+        return k
+
+
+class _InteractiveArgsRequest(EntryTreeWidget):
+    supplied = QtCore.pyqtSignal(int, dict)
+    cancelled = QtCore.pyqtSignal(int)
+
+    def __init__(self, rid, arglist_desc):
+        EntryTreeWidget.__init__(self)
+        self.rid = rid
+        self.arguments = dict()
+        for key, procdesc, group, tooltip in arglist_desc:
+            self.arguments[key] = {"desc": procdesc, "group": group, "tooltip": tooltip}
+            self.set_argument(key, self.arguments[key])
+        self.quickStyleClicked.connect(self.supply)
+        cancel_btn = QtWidgets.QPushButton("Cancel")
+        cancel_btn.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_DialogCancelButton))
+        cancel_btn.clicked.connect(self.cancel)
+        supply_btn = QtWidgets.QPushButton("Supply")
+        supply_btn.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_DialogOkButton))
+        supply_btn.clicked.connect(self.supply)
+        buttons = LayoutWidget()
+        buttons.addWidget(cancel_btn, 1, 1)
+        buttons.addWidget(supply_btn, 1, 2)
+        buttons.layout.setColumnStretch(0, 1)
+        buttons.layout.setColumnStretch(1, 0)
+        buttons.layout.setColumnStretch(2, 0)
+        buttons.layout.setColumnStretch(3, 1)
+        self.setItemWidget(self.bottom_item, 1, buttons)
+
+    def supply(self):
+        argument_values = dict()
+        for key, argument in self.arguments.items():
+            entry_cls = procdesc_to_entry(argument["desc"])
+            argument_values[key] = entry_cls.state_to_value(argument["state"])
+        self.supplied.emit(self.rid, argument_values)
+
+    def cancel(self):
+        self.cancelled.emit(self.rid)
+
+
+class _InteractiveArgsView(QtWidgets.QStackedWidget):
+    supplied = QtCore.pyqtSignal(int, dict)
+    cancelled = QtCore.pyqtSignal(int)
+
+    def __init__(self):
+        QtWidgets.QStackedWidget.__init__(self)
+        self.tabs = QtWidgets.QTabWidget()
+        self.default_label = QtWidgets.QLabel("No pending interactive arguments requests.")
+        self.default_label.setAlignment(QtCore.Qt.AlignCenter)
+        font = QtGui.QFont(self.default_label.font())
+        font.setItalic(True)
+        self.default_label.setFont(font)
+        self.addWidget(self.tabs)
+        self.addWidget(self.default_label)
+        self.model = Model({})
+
+    def setModel(self, model):
+        self.setCurrentIndex(1)
+        for i in range(self.tabs.count()):
+            widget = self.tabs.widget(i)
+            self.tabs.removeTab(i)
+            widget.deleteLater()
+        self.model = model
+        self.model.rowsInserted.connect(self.rowsInserted)
+        self.model.rowsRemoved.connect(self.rowsRemoved)
+        for i in range(self.model.rowCount(QtCore.QModelIndex())):
+            self._insert_widget(i)
+
+    def _insert_widget(self, row):
+        rid = self.model.data(self.model.index(row, 0), QtCore.Qt.DisplayRole)
+        title = self.model.data(self.model.index(row, 1), QtCore.Qt.DisplayRole)
+        arglist_desc = self.model.data(self.model.index(row, 2), QtCore.Qt.DisplayRole)
+        inter_args_request = _InteractiveArgsRequest(rid, arglist_desc)
+        inter_args_request.supplied.connect(self.supplied)
+        inter_args_request.cancelled.connect(self.cancelled)
+        self.tabs.insertTab(row, inter_args_request, title)
+
+    def rowsInserted(self, parent, first, last):
+        assert first == last
+        self.setCurrentIndex(0)
+        self._insert_widget(first)
+
+    def rowsRemoved(self, parent, first, last):
+        assert first == last
+        widget = self.tabs.widget(first)
+        self.tabs.removeTab(first)
+        widget.deleteLater()
+        if self.tabs.count() == 0:
+            self.setCurrentIndex(1)
+
+
+class InteractiveArgsDock(QtWidgets.QDockWidget):
+    def __init__(self, interactive_args_sub, interactive_args_rpc):
+        QtWidgets.QDockWidget.__init__(self, "Interactive Args")
+        self.setObjectName("Interactive Args")
+        self.setFeatures(
+            QtWidgets.QDockWidget.DockWidgetMovable | QtWidgets.QDockWidget.DockWidgetFloatable)
+        self.interactive_args_rpc = interactive_args_rpc
+        self.request_view = _InteractiveArgsView()
+        self.request_view.supplied.connect(self.supply)
+        self.request_view.cancelled.connect(self.cancel)
+        self.setWidget(self.request_view)
+        interactive_args_sub.add_setmodel_callback(self.request_view.setModel)
+
+    def supply(self, rid, values):
+        asyncio.ensure_future(self._supply_task(rid, values))
+
+    async def _supply_task(self, rid, values):
+        try:
+            await self.interactive_args_rpc.supply(rid, values)
+        except Exception:
+            logger.error("failed to supply interactive arguments for experiment: %d",
+                         rid, exc_info=True)
+
+    def cancel(self, rid):
+        asyncio.ensure_future(self._cancel_task(rid))
+
+    async def _cancel_task(self, rid):
+        try:
+            await self.interactive_args_rpc.cancel(rid)
+        except Exception:
+            logger.error("failed to cancel interactive args request for experiment: %d",
+                         rid, exc_info=True)

artiq/dashboard/moninj.py

@@ -2,23 +2,20 @@ import asyncio
 import logging
 import textwrap
 from collections import namedtuple
+from functools import partial
 
-from PyQt5 import QtCore, QtWidgets, QtGui
+from PyQt5 import QtCore, QtWidgets
 
-from sipyco.sync_struct import Subscriber
-
-from artiq.coredevice.comm_moninj import *
-from artiq.coredevice.ad9910 import (
-    _AD9910_REG_PROFILE0, _AD9910_REG_PROFILE7, 
-    _AD9910_REG_FTW, _AD9910_REG_CFR1
-)
-from artiq.coredevice.ad9912_reg import AD9912_POW1, AD9912_SER_CONF
-from artiq.gui.tools import LayoutWidget
-from artiq.gui.flowlayout import FlowLayout
+from artiq.coredevice.comm_moninj import CommMonInj, TTLOverride, TTLProbe
+from artiq.coredevice.ad9912_reg import AD9912_SER_CONF
+from artiq.gui.tools import LayoutWidget, QDockWidgetCloseDetect, DoubleClickLineEdit
+from artiq.gui.dndwidgets import VDragScrollArea, DragDropFlowLayoutWidget
+from artiq.gui.models import DictSyncTreeSepModel
 
 
 logger = logging.getLogger(__name__)
 
+
 class _CancellableLineEdit(QtWidgets.QLineEdit):
     def escapePressedConnect(self, cb):
         self.esc_cb = cb
@@ -37,6 +34,7 @@ class _TTLWidget(QtWidgets.QFrame):
         self.channel = channel
         self.set_mode = dm.ttl_set_mode
         self.force_out = force_out
+        self.title = title
 
         self.setFrameShape(QtWidgets.QFrame.Box)
         self.setFrameShadow(QtWidgets.QFrame.Raised)
@@ -133,7 +131,7 @@ class _TTLWidget(QtWidgets.QFrame):
         else:
             color = ""
         self.value.setText("<font size=\"5\"{}>{}</font>".format(
-                            color, value_s))
+                           color, value_s))
         oe = self.cur_oe or self.force_out
         direction = "OUT" if oe else "IN"
         self.direction.setText("<font size=\"2\">" + direction + "</font>")
@@ -148,40 +146,13 @@ class _TTLWidget(QtWidgets.QFrame):
             self.programmatic_change = False
 
     def sort_key(self):
-        return self.channel
+        return (0, self.channel, 0)
 
+    def uid(self):
+        return self.title
 
-class _SimpleDisplayWidget(QtWidgets.QFrame):
-    def __init__(self, title):
-        QtWidgets.QFrame.__init__(self)
-
-        self.setFrameShape(QtWidgets.QFrame.Box)
-        self.setFrameShadow(QtWidgets.QFrame.Raised)
-
-        grid = QtWidgets.QGridLayout()
-        grid.setContentsMargins(0, 0, 0, 0)
-        grid.setHorizontalSpacing(0)
-        grid.setVerticalSpacing(0)
-        self.setLayout(grid)
-        label = QtWidgets.QLabel(title)
-        label.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(label, 1, 1)
-
-        self.value = QtWidgets.QLabel()
-        self.value.setAlignment(QtCore.Qt.AlignCenter)
-        grid.addWidget(self.value, 2, 1, 6, 1)
-
-        grid.setRowStretch(1, 1)
-        grid.setRowStretch(2, 0)
-        grid.setRowStretch(3, 1)
-
-        self.refresh_display()
-
-    def refresh_display(self):
-        raise NotImplementedError
-
-    def sort_key(self):
-        raise NotImplementedError
+    def to_model_path(self):
+        return "ttl/{}".format(self.title)
 
 
 class _DDSModel:
@@ -191,7 +162,7 @@ class _DDSModel:
         self.cur_reg = 0
         self.dds_type = dds_type
         self.is_urukul = dds_type in ["AD9910", "AD9912"]
-    
+
         if dds_type == "AD9914":
             self.ftw_per_hz = 2**32 / ref_clk
         else:
@@ -216,13 +187,15 @@ class _DDSModel:
 
 
 class _DDSWidget(QtWidgets.QFrame):
-    def __init__(self, dm, title, bus_channel=0, channel=0, dds_model=None):
+    def __init__(self, dm, title, bus_channel, channel,
+                 dds_type, ref_clk, cpld=None, pll=1, clk_div=0):
         self.dm = dm
         self.bus_channel = bus_channel
         self.channel = channel
         self.dds_name = title
         self.cur_frequency = 0
-        self.dds_model = dds_model
+        self.dds_model = _DDSModel(dds_type, ref_clk, cpld, pll, clk_div)
+        self.title = title
 
         QtWidgets.QFrame.__init__(self)
 
@@ -283,7 +256,7 @@ class _DDSWidget(QtWidgets.QFrame):
         set_btn.setText("Set")
         set_btn.setToolTip("Set frequency")
         set_grid.addWidget(set_btn, 0, 1, 1, 1)
-        
+
         # for urukuls also allow switching off RF
         if self.dds_model.is_urukul:
             off_btn = QtWidgets.QToolButton()
@@ -327,18 +300,17 @@ class _DDSWidget(QtWidgets.QFrame):
     def set_clicked(self, set):
         self.data_stack.setCurrentIndex(1)
         self.button_stack.setCurrentIndex(1)
-        self.value_edit.setText("{:.7f}"
-                .format(self.cur_frequency/1e6))
+        self.value_edit.setText("{:.7f}".format(self.cur_frequency / 1e6))
         self.value_edit.setFocus()
         self.value_edit.selectAll()
 
     def off_clicked(self, set):
         self.dm.dds_channel_toggle(self.dds_name, self.dds_model, sw=False)
-    
+
     def apply_changes(self, apply):
         self.data_stack.setCurrentIndex(0)
         self.button_stack.setCurrentIndex(0)
-        frequency = float(self.value_edit.text())*1e6
+        frequency = float(self.value_edit.text()) * 1e6
         self.dm.dds_set_frequency(self.dds_name, self.dds_model, frequency)
 
     def cancel_changes(self, cancel):
@@ -347,28 +319,67 @@ class _DDSWidget(QtWidgets.QFrame):
 
     def refresh_display(self):
         self.cur_frequency = self.dds_model.cur_frequency
-        self.value_label.setText("<font size=\"4\">{:.7f}</font>"
-                           .format(self.cur_frequency/1e6))
-        self.value_edit.setText("{:.7f}"
-                           .format(self.cur_frequency/1e6))
+        self.value_label.setText("<font size=\"4\">{:.7f}</font>".format(self.cur_frequency / 1e6))
+        self.value_edit.setText("{:.7f}".format(self.cur_frequency / 1e6))
 
     def sort_key(self):
-        return (self.bus_channel, self.channel)
+        return (1, self.bus_channel, self.channel)
+
+    def uid(self):
+        return self.title
+
+    def to_model_path(self):
+        return "dds/{}".format(self.title)
 
 
-class _DACWidget(_SimpleDisplayWidget):
-    def __init__(self, dm, spi_channel, channel, title):
+class _DACWidget(QtWidgets.QFrame):
+    def __init__(self, dm, spi_channel, channel, title, vref, offset_dacs):
+        QtWidgets.QFrame.__init__(self)
         self.spi_channel = spi_channel
         self.channel = channel
-        self.cur_value = 0
-        _SimpleDisplayWidget.__init__(self, "{} ch{}".format(title, channel))
+        self.cur_value = 0x8000;
+        self.title = title
+        self.vref = vref
+        self.offset_dacs = offset_dacs
+
+        self.setFrameShape(QtWidgets.QFrame.Box)
+        self.setFrameShadow(QtWidgets.QFrame.Raised)
+
+        grid = QtWidgets.QGridLayout()
+        grid.setContentsMargins(5, 5, 5, 5)
+        grid.setHorizontalSpacing(5)
+        grid.setVerticalSpacing(5)
+        self.setLayout(grid)
+        label = QtWidgets.QLabel("{}[{}]".format(title, channel))
+        label.setAlignment(QtCore.Qt.AlignCenter)
+        grid.addWidget(label, 1, 1)
+
+        self.value = QtWidgets.QLabel()
+        self.value.setAlignment(QtCore.Qt.AlignCenter)
+        grid.addWidget(self.value, 2, 1, 6, 1)
+
+        grid.setRowStretch(1, 1)
+        grid.setRowStretch(2, 0)
+        grid.setRowStretch(3, 1)
+
+        self.refresh_display()
+
+    def mu_to_voltage(self, code):
+        voltage = ((code - self.offset_dacs * 0x4) / (1 << 16)) * (4. * self.vref)
+        return voltage
 
     def refresh_display(self):
-        self.value.setText("<font size=\"4\">{:.3f}</font><font size=\"2\"> %</font>"
-                           .format(self.cur_value*100/2**16))
+        self.value.setText("<font size=\"4\">{:+.3f} V</font>"
+                           .format(self.mu_to_voltage(self.cur_value)))
 
     def sort_key(self):
-        return (self.spi_channel, self.channel)
+        return (2, self.spi_channel, self.channel)
+
+    def uid(self):
+        return (self.title, self.channel)
+
+    def to_model_path(self):
+        return "dac/{}[{}]".format(self.title, self.channel)
 
 
 _WidgetDesc = namedtuple("_WidgetDesc", "uid comment cls arguments")
@@ -392,18 +403,15 @@ def setup_from_ddb(ddb):
                         force_out = v["class"] == "TTLOut"
                         widget = _WidgetDesc(k, comment, _TTLWidget, (channel, force_out, k))
                         description.add(widget)
-                    elif (v["module"] == "artiq.coredevice.ad9914"
-                            and v["class"] == "AD9914"):
+                    elif (v["module"] == "artiq.coredevice.ad9914" and v["class"] == "AD9914"):
                         bus_channel = v["arguments"]["bus_channel"]
                         channel = v["arguments"]["channel"]
                         dds_sysclk = v["arguments"]["sysclk"]
-                        model = _DDSModel(v["class"], dds_sysclk)
-                        widget = _WidgetDesc(k, comment, _DDSWidget, (k, bus_channel, channel, model))
+                        widget = _WidgetDesc(k, comment, _DDSWidget,
+                                             (k, bus_channel, channel, v["class"], dds_sysclk))
                         description.add(widget)
-                    elif (v["module"] == "artiq.coredevice.ad9910"
-                                and v["class"] == "AD9910") or \
-                            (v["module"] == "artiq.coredevice.ad9912"
-                                and v["class"] == "AD9912"):
+                    elif (v["module"] == "artiq.coredevice.ad9910" and v["class"] == "AD9910") or \
+                         (v["module"] == "artiq.coredevice.ad9912" and v["class"] == "AD9912"):
                         channel = v["arguments"]["chip_select"] - 4
                         if channel < 0:
                             continue
@@ -413,18 +421,22 @@ def setup_from_ddb(ddb):
                         pll = v["arguments"]["pll_n"]
                         refclk = ddb[dds_cpld]["arguments"]["refclk"]
                         clk_div = v["arguments"].get("clk_div", 0)
-                        model = _DDSModel( v["class"], refclk, dds_cpld, pll, clk_div)
-                        widget = _WidgetDesc(k, comment, _DDSWidget, (k, bus_channel, channel, model))
-                        description.add(widget)       
-                    elif (   (v["module"] == "artiq.coredevice.ad53xx" and v["class"] == "AD53xx")
-                          or (v["module"] == "artiq.coredevice.zotino" and v["class"] == "Zotino")):
+                        widget = _WidgetDesc(k, comment, _DDSWidget,
+                                             (k, bus_channel, channel, v["class"],
+                                              refclk, dds_cpld, pll, clk_div))
+                        description.add(widget)
+                    elif (v["module"] == "artiq.coredevice.ad53xx" and v["class"] == "AD53xx") or \
+                         (v["module"] == "artiq.coredevice.zotino" and v["class"] == "Zotino"):
                         spi_device = v["arguments"]["spi_device"]
                         spi_device = ddb[spi_device]
                         while isinstance(spi_device, str):
                             spi_device = ddb[spi_device]
                         spi_channel = spi_device["arguments"]["channel"]
+                        vref = v["arguments"].get("vref", 5.)
+                        offset_dacs = v["arguments"].get("offset_dacs", 8192)
                         for channel in range(32):
-                            widget = _WidgetDesc((k, channel), comment, _DACWidget, (spi_channel, channel, k))
+                            widget = _WidgetDesc((k, channel), comment, _DACWidget,
+                                                 (spi_channel, channel, k, vref, offset_dacs))
                             description.add(widget)
                 elif v["type"] == "controller" and k == "core_moninj":
                     mi_addr = v["host"]
@@ -449,18 +461,15 @@ class _DeviceManager:
         self.widgets_by_uid = dict()
 
         self.dds_sysclk = 0
-        self.ttl_cb = lambda: None
         self.ttl_widgets = dict()
-        self.dds_cb = lambda: None
         self.dds_widgets = dict()
-        self.dac_cb = lambda: None
         self.dac_widgets = dict()
+        self.channels_cb = lambda: None
 
     def init_ddb(self, ddb):
         self.ddb = ddb
-        return ddb
 
-    def notify(self, mod):
+    def notify_ddb(self, mod):
         mi_addr, mi_port, description = setup_from_ddb(self.ddb)
 
         if (mi_addr, mi_port) != (self.mi_addr, self.mi_port):
@@ -471,24 +480,8 @@ class _DeviceManager:
         for to_remove in self.description - description:
             widget = self.widgets_by_uid[to_remove.uid]
             del self.widgets_by_uid[to_remove.uid]
-
-            if isinstance(widget, _TTLWidget):
-                self.setup_ttl_monitoring(False, widget.channel)
-                widget.deleteLater()
-                del self.ttl_widgets[widget.channel]
-                self.ttl_cb()
-            elif isinstance(widget, _DDSWidget):
-                self.setup_dds_monitoring(False, widget.bus_channel, widget.channel)
-                widget.deleteLater()
-                del self.dds_widgets[(widget.bus_channel, widget.channel)]
-                self.dds_cb()
-            elif isinstance(widget, _DACWidget):
-                self.setup_dac_monitoring(False, widget.spi_channel, widget.channel)
-                widget.deleteLater()
-                del self.dac_widgets[(widget.spi_channel, widget.channel)]
-                self.dac_cb()     
-            else:
-                raise ValueError
+            self.setup_monitoring(False, widget)
+            widget.deleteLater()
 
         for to_add in description - self.description:
             widget = to_add.cls(self, *to_add.arguments)
@@ -496,26 +489,15 @@ class _DeviceManager:
                 widget.setToolTip(to_add.comment)
             self.widgets_by_uid[to_add.uid] = widget
 
-            if isinstance(widget, _TTLWidget):
-                self.ttl_widgets[widget.channel] = widget
-                self.ttl_cb()
-                self.setup_ttl_monitoring(True, widget.channel)
-            elif isinstance(widget, _DDSWidget):
-                self.dds_widgets[(widget.bus_channel, widget.channel)] = widget
-                self.dds_cb()
-                self.setup_dds_monitoring(True, widget.bus_channel, widget.channel)
-            elif isinstance(widget, _DACWidget):
-                self.dac_widgets[(widget.spi_channel, widget.channel)] = widget
-                self.dac_cb()
-                self.setup_dac_monitoring(True, widget.spi_channel, widget.channel)
-            else:
-                raise ValueError
+        if description != self.description:
+            self.channels_cb()
 
         self.description = description
 
     def ttl_set_mode(self, channel, mode):
         if self.mi_connection is not None:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if mode == "0":
                 widget.cur_override = True
                 widget.cur_level = False
@@ -565,7 +547,7 @@ class _DeviceManager:
             cpld_dev = """self.setattr_device("core_cache")
                 self.setattr_device("{}")""".format(dds_model.cpld)
 
-            # `sta`/`rf_sw`` variables are guaranteed for urukuls 
+            # `sta`/`rf_sw`` variables are guaranteed for urukuls
             # so {action} can use it
             # if there's no RF enabled, CPLD may have not been initialized
             # but if there is, it has been initialised - no need to do again
@@ -624,8 +606,8 @@ class _DeviceManager:
                    channel_init=channel_init))
         asyncio.ensure_future(
             self._submit_by_content(
-                dds_exp, 
-                title, 
+                dds_exp,
+                title,
                 log_msg))
 
     def dds_set_frequency(self, dds_channel, dds_model, freq):
@@ -640,8 +622,8 @@ class _DeviceManager:
             dds_channel,
             dds_model,
             action,
-            "SetDDS", 
-            "Set DDS {} {}MHz".format(dds_channel, freq/1e6))
+            "SetDDS",
+            "Set DDS {} {}MHz".format(dds_channel, freq / 1e6))
 
     def dds_channel_toggle(self, dds_channel, dds_model, sw=True):
         # urukul only
@@ -661,9 +643,34 @@ class _DeviceManager:
             dds_channel,
             dds_model,
             action,
-            "ToggleDDS", 
+            "ToggleDDS",
             "Toggle DDS {} {}".format(dds_channel, "on" if sw else "off"))
 
+    def setup_monitoring(self, enable, widget):
+        if isinstance(widget, _TTLWidget):
+            key = widget.channel
+            args = (key,)
+            subscribers = self.ttl_widgets
+            subscribe_func = self.setup_ttl_monitoring
+        elif isinstance(widget, _DDSWidget):
+            key = (widget.bus_channel, widget.channel)
+            args = key
+            subscribers = self.dds_widgets
+            subscribe_func = self.setup_dds_monitoring
+        elif isinstance(widget, _DACWidget):
+            key = (widget.spi_channel, widget.channel)
+            args = key
+            subscribers = self.dac_widgets
+            subscribe_func = self.setup_dac_monitoring
+        else:
+            raise ValueError
+        if enable and key not in subscribers:
+            subscribers[key] = widget.uid()
+            subscribe_func(enable, *args)
+        elif not enable and key in subscribers:
+            subscribe_func(enable, *args)
+            del subscribers[key]
+
     def setup_ttl_monitoring(self, enable, channel):
         if self.mi_connection is not None:
             self.mi_connection.monitor_probe(enable, channel, TTLProbe.level.value)
@@ -683,24 +690,28 @@ class _DeviceManager:
 
     def monitor_cb(self, channel, probe, value):
         if channel in self.ttl_widgets:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if probe == TTLProbe.level.value:
                 widget.cur_level = bool(value)
             elif probe == TTLProbe.oe.value:
                 widget.cur_oe = bool(value)
             widget.refresh_display()
         elif (channel, probe) in self.dds_widgets:
-            widget = self.dds_widgets[(channel, probe)]
+            widget_uid = self.dds_widgets[(channel, probe)]
+            widget = self.widgets_by_uid[widget_uid]
             widget.dds_model.monitor_update(probe, value)
             widget.refresh_display()
         elif (channel, probe) in self.dac_widgets:
-            widget = self.dac_widgets[(channel, probe)]
+            widget_uid = self.dac_widgets[(channel, probe)]
+            widget = self.widgets_by_uid[widget_uid]
             widget.cur_value = value
             widget.refresh_display()
 
     def injection_status_cb(self, channel, override, value):
         if channel in self.ttl_widgets:
-            widget = self.ttl_widgets[channel]
+            widget_uid = self.ttl_widgets[channel]
+            widget = self.widgets_by_uid[widget_uid]
             if override == TTLOverride.en.value:
                 widget.cur_override = bool(value)
             if override == TTLOverride.level.value:
@@ -719,14 +730,12 @@ class _DeviceManager:
                 await self.mi_connection.close()
                 self.mi_connection = None
             new_mi_connection = CommMonInj(self.monitor_cb, self.injection_status_cb,
-                    self.disconnect_cb)
+                                           self.disconnect_cb)
             try:
                 await new_mi_connection.connect(self.mi_addr, self.mi_port)
-            except asyncio.CancelledError:
-                logger.info("cancelled connection to moninj")
-                break
-            except:
-                logger.error("failed to connect to moninj. Is aqctl_moninj_proxy running?", exc_info=True)
+            except Exception:
+                logger.error("failed to connect to moninj. Is aqctl_moninj_proxy running?",
+                             exc_info=True)
                 await asyncio.sleep(10.)
                 self.reconnect_mi.set()
             else:
@@ -736,9 +745,9 @@ class _DeviceManager:
                 for ttl_channel in self.ttl_widgets.keys():
                     self.setup_ttl_monitoring(True, ttl_channel)
                 for bus_channel, channel in self.dds_widgets.keys():
-                    self.setup_dds_monitoring(True, bus_channel, channel)
+                     self.setup_dds_monitoring(True, bus_channel, channel)
                 for spi_channel, channel in self.dac_widgets.keys():
-                    self.setup_dac_monitoring(True, spi_channel, channel)
+                     self.setup_dac_monitoring(True, spi_channel, channel)
 
     async def close(self):
         self.mi_connector_task.cancel()
@@ -750,48 +759,233 @@ class _DeviceManager:
             await self.mi_connection.close()
 
 
-class _MonInjDock(QtWidgets.QDockWidget):
-    def __init__(self, name):
-        QtWidgets.QDockWidget.__init__(self, name)
+class Model(DictSyncTreeSepModel):
+    def __init__(self, init):
+        DictSyncTreeSepModel.__init__(self, "/", ["Channels"], init)
+
+    def clear(self):
+        for k in self.backing_store:
+            self._del_item(self, k.split(self.separator))
+        self.backing_store.clear()
+
+    def update(self, d):
+        for k, v in d.items():
+            self[v.to_model_path()] = v
+
+
+class _AddChannelDialog(QtWidgets.QDialog):
+    def __init__(self, parent, model):
+        QtWidgets.QDialog.__init__(self, parent=parent)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+        self.setWindowTitle("Add channels")
+
+        layout = QtWidgets.QVBoxLayout()
+        self.setLayout(layout)
+
+        self._model = model
+        self._tree_view = QtWidgets.QTreeView()
+        self._tree_view.setHeaderHidden(True)
+        self._tree_view.setSelectionBehavior(
+            QtWidgets.QAbstractItemView.SelectItems)
+        self._tree_view.setSelectionMode(
+            QtWidgets.QAbstractItemView.ExtendedSelection)
+        self._tree_view.setModel(self._model)
+        layout.addWidget(self._tree_view)
+
+        self._button_box = QtWidgets.QDialogButtonBox(
+            QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel
+        )
+        self._button_box.setCenterButtons(True)
+        self._button_box.accepted.connect(self.add_channels)
+        self._button_box.rejected.connect(self.reject)
+        layout.addWidget(self._button_box)
+
+    def add_channels(self):
+        selection = self._tree_view.selectedIndexes()
+        channels = []
+        for select in selection:
+            key = self._model.index_to_key(select)
+            if key is not None:
+                channels.append(self._model[key].ref)
+        self.channels = channels
+        self.accept()
+
+
+class _MonInjDock(QDockWidgetCloseDetect):
+    def __init__(self, name, manager):
+        QtWidgets.QDockWidget.__init__(self, "MonInj")
         self.setObjectName(name)
         self.setFeatures(QtWidgets.QDockWidget.DockWidgetMovable |
                          QtWidgets.QDockWidget.DockWidgetFloatable)
+        grid = LayoutWidget()
+        self.setWidget(grid)
+        self.manager = manager
+        self.widget_uids = None
+
+        newdock = QtWidgets.QToolButton()
+        newdock.setToolTip("Create new moninj dock")
+        newdock.setIcon(QtWidgets.QApplication.style().standardIcon(
+            QtWidgets.QStyle.SP_FileDialogNewFolder))
+        newdock.clicked.connect(lambda: self.manager.create_new_dock())
+        grid.addWidget(newdock, 0, 0)
+
+        self.channel_dialog = _AddChannelDialog(self, self.manager.channel_model)
+        self.channel_dialog.accepted.connect(self.add_channels)
+
+        dialog_btn = QtWidgets.QToolButton()
+        dialog_btn.setToolTip("Add channels")
+        dialog_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogListView))
+        dialog_btn.clicked.connect(self.channel_dialog.open)
+        grid.addWidget(dialog_btn, 0, 1)
+
+        self.label = DoubleClickLineEdit(name)
+        self.label.setStyleSheet("background:transparent;")
+        grid.addWidget(self.label, 0, 2)
+
+        scroll_area = VDragScrollArea(self)
+        grid.addWidget(scroll_area, 1, 0, 1, 10)
+        self.flow = DragDropFlowLayoutWidget()
+        scroll_area.setWidgetResizable(True)
+        scroll_area.setWidget(self.flow)
+        self.flow.setContextMenuPolicy(QtCore.Qt.CustomContextMenu)
+        self.flow.customContextMenuRequested.connect(self.custom_context_menu)
+
+    def custom_context_menu(self, pos):
+        index = self.flow._get_index(pos)
+        if index == -1:
+            return
+        menu = QtWidgets.QMenu()
+        delete_action = QtWidgets.QAction("Delete widget", menu)
+        delete_action.triggered.connect(partial(self.delete_widget, index))
+        menu.addAction(delete_action)
+        menu.exec_(self.flow.mapToGlobal(pos))
+
+    def delete_all_widgets(self):
+        for index in reversed(range(self.flow.count())):
+            self.delete_widget(index, True)
+
+
+    def delete_widget(self, index, checked):
+        widget = self.flow.itemAt(index).widget()
+        widget.hide()
+        self.manager.dm.setup_monitoring(False, widget)
+        self.flow.layout.takeAt(index)
+
+    def add_channels(self):
+        channels = self.channel_dialog.channels
+        self.layout_widgets(channels)
 
     def layout_widgets(self, widgets):
-        scroll_area = QtWidgets.QScrollArea()
-        self.setWidget(scroll_area)
-
-        grid = FlowLayout()
-        grid_widget = QtWidgets.QWidget()
-        grid_widget.setLayout(grid)
-
         for widget in sorted(widgets, key=lambda w: w.sort_key()):
-            grid.addWidget(widget)
+            widget.show()
+            self.manager.dm.setup_monitoring(True, widget)
+            self.flow.addWidget(widget)
 
-        scroll_area.setWidgetResizable(True)
-        scroll_area.setWidget(grid_widget)
+    def restore_widgets(self):
+        if self.widget_uids is not None:
+            widgets_by_uid = self.manager.dm.widgets_by_uid
+            widgets = list()
+            for uid in self.widget_uids:
+                if uid in widgets_by_uid:
+                    widgets.append(widgets_by_uid[uid])
+                else:
+                    logger.warning("removing moninj widget {}".format(uid))
+            self.layout_widgets(widgets)
+            self.widget_uids = None
+
+    def _save_widget_uids(self):
+        uids = []
+        for i in range(self.flow.count()):
+            uids.append(self.flow.itemAt(i).widget().uid())
+        return uids
+
+    def save_state(self):
+        return {
+            "dock_label": self.label.text(),
+            "widget_uids": self._save_widget_uids()
+        }
+
+    def restore_state(self, state):
+        try:
+            label = state["dock_label"]
+        except KeyError:
+            pass
+        else:
+            self.label._text = label
+            self.label.setText(label)
+        try:
+            self.widget_uids = state["widget_uids"]
+        except KeyError:
+            pass
 
 
 class MonInj:
-    def __init__(self, schedule_ctl):
-        self.ttl_dock = _MonInjDock("TTL")
-        self.dds_dock = _MonInjDock("DDS")
-        self.dac_dock = _MonInjDock("DAC")
-
+    def __init__(self, schedule_ctl, main_window):
+        self.docks = dict()
+        self.main_window = main_window
         self.dm = _DeviceManager(schedule_ctl)
-        self.dm.ttl_cb = lambda: self.ttl_dock.layout_widgets(
-                            self.dm.ttl_widgets.values())
-        self.dm.dds_cb = lambda: self.dds_dock.layout_widgets(
-                            self.dm.dds_widgets.values())
-        self.dm.dac_cb = lambda: self.dac_dock.layout_widgets(
-                            self.dm.dac_widgets.values())
+        self.dm.channels_cb = self.add_channels
+        self.channel_model = Model({})
 
-        self.subscriber = Subscriber("devices", self.dm.init_ddb, self.dm.notify)
+    def add_channels(self):
+        self.channel_model.clear()
+        self.channel_model.update(self.dm.widgets_by_uid)
+        for dock in self.docks.values():
+            dock.restore_widgets()
 
-    async def start(self, server, port):
-        await self.subscriber.connect(server, port)
+    def create_new_dock(self, add_to_area=True):
+        n = 0
+        name = "moninj0"
+        while name in self.docks:
+            n += 1
+            name = "moninj" + str(n)
+
+        dock = _MonInjDock(name, self)
+        self.docks[name] = dock
+        if add_to_area:
+            self.main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
+            dock.setFloating(True)
+        dock.sigClosed.connect(partial(self.on_dock_closed, name))
+        self.update_closable()
+        return dock
+
+    def on_dock_closed(self, name):
+        dock = self.docks[name]
+        del self.docks[name]
+        self.update_closable()
+        dock.delete_all_widgets()
+        dock.hide()  # dock may be parent, only delete on exit
+
+    def update_closable(self):
+        flags = (QtWidgets.QDockWidget.DockWidgetMovable |
+                 QtWidgets.QDockWidget.DockWidgetFloatable)
+        if len(self.docks) > 1:
+            flags |= QtWidgets.QDockWidget.DockWidgetClosable
+        for dock in self.docks.values():
+            dock.setFeatures(flags)
+
+    def first_moninj_dock(self):
+        if self.docks:
+            return None
+        dock = self.create_new_dock(False)
+        return dock
+
+    def save_state(self):
+        return {name: dock.save_state() for name, dock in self.docks.items()}
+
+    def restore_state(self, state):
+        if self.docks:
+            raise NotImplementedError
+        for name, dock_state in state.items():
+            dock = _MonInjDock(name, self)
+            self.docks[name] = dock
+            dock.restore_state(dock_state)
+            self.main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, dock)
+            dock.sigClosed.connect(partial(self.on_dock_closed, name))
+        self.update_closable()
 
     async def stop(self):
-        await self.subscriber.close()
         if self.dm is not None:
             await self.dm.close()

artiq/dashboard/schedule.py

@@ -15,9 +15,8 @@ logger = logging.getLogger(__name__)
 class Model(DictSyncModel):
     def __init__(self, init):
         DictSyncModel.__init__(self,
-            ["RID", "Pipeline", "Status", "Prio", "Due date",
-             "Revision", "File", "Class name"],
-            init)
+                               ["RID", "Pipeline", "Status", "Prio", "Due date",
+                                "Revision", "File", "Class name"], init)
 
     def sort_key(self, k, v):
         # order by priority, and then by due date and RID
@@ -96,14 +95,14 @@ class ScheduleDock(QtWidgets.QDockWidget):
 
         cw = QtGui.QFontMetrics(self.font()).averageCharWidth()
         h = self.table.horizontalHeader()
-        h.resizeSection(0, 7*cw)
-        h.resizeSection(1, 12*cw)
-        h.resizeSection(2, 16*cw)
-        h.resizeSection(3, 6*cw)
-        h.resizeSection(4, 16*cw)
-        h.resizeSection(5, 30*cw)
-        h.resizeSection(6, 20*cw)
-        h.resizeSection(7, 20*cw)
+        h.resizeSection(0, 7 * cw)
+        h.resizeSection(1, 12 * cw)
+        h.resizeSection(2, 16 * cw)
+        h.resizeSection(3, 6 * cw)
+        h.resizeSection(4, 16 * cw)
+        h.resizeSection(5, 30 * cw)
+        h.resizeSection(6, 20 * cw)
+        h.resizeSection(7, 20 * cw)
 
     def set_model(self, model):
         self.table_model = model
@@ -143,7 +142,7 @@ class ScheduleDock(QtWidgets.QDockWidget):
             selected_rid = self.table_model.row_to_key[row]
             pipeline = self.table_model.backing_store[selected_rid]["pipeline"]
             logger.info("Requesting termination of all "
-                "experiments in pipeline '%s'", pipeline)
+                        "experiments in pipeline '%s'", pipeline)
 
             rids = set()
             for rid, info in self.table_model.backing_store.items():
@@ -151,7 +150,6 @@ class ScheduleDock(QtWidgets.QDockWidget):
                     rids.add(rid)
             asyncio.ensure_future(self.request_term_multiple(rids))
 
-
     def save_state(self):
         return bytes(self.table.horizontalHeader().saveState())
 

artiq/dashboard/shortcuts.py

@@ -3,8 +3,6 @@ from functools import partial
 
 from PyQt5 import QtCore, QtWidgets
 
-from artiq.gui.tools import LayoutWidget
-
 
 logger = logging.getLogger(__name__)
 
@@ -35,7 +33,7 @@ class ShortcutsDock(QtWidgets.QDockWidget):
         for i in range(12):
             row = i + 1
 
-            layout.addWidget(QtWidgets.QLabel("F" + str(i+1)), row, 0)
+            layout.addWidget(QtWidgets.QLabel("F" + str(i + 1)), row, 0)
 
             label = QtWidgets.QLabel()
             label.setSizePolicy(QtWidgets.QSizePolicy.Ignored,
@@ -70,7 +68,7 @@ class ShortcutsDock(QtWidgets.QDockWidget):
                 "open": open,
                 "submit": submit
             }
-            shortcut = QtWidgets.QShortcut("F" + str(i+1), main_window)
+            shortcut = QtWidgets.QShortcut("F" + str(i + 1), main_window)
             shortcut.setContext(QtCore.Qt.ApplicationShortcut)
             shortcut.activated.connect(partial(self._activated, i))
 

artiq/dashboard/waveform.py

@@ -0,0 +1,914 @@
+import os
+import asyncio
+import logging
+import bisect
+import itertools
+import math
+
+from PyQt5 import QtCore, QtWidgets, QtGui
+
+import pyqtgraph as pg
+import numpy as np
+
+from sipyco.pc_rpc import AsyncioClient
+from sipyco import pyon
+
+from artiq.tools import exc_to_warning, short_format
+from artiq.coredevice import comm_analyzer
+from artiq.coredevice.comm_analyzer import WaveformType
+from artiq.gui.tools import LayoutWidget, get_open_file_name, get_save_file_name
+from artiq.gui.models import DictSyncTreeSepModel
+from artiq.gui.dndwidgets import VDragScrollArea, VDragDropSplitter
+
+
+logger = logging.getLogger(__name__)
+
+WAVEFORM_MIN_HEIGHT = 50
+WAVEFORM_MAX_HEIGHT = 200
+
+
+class ProxyClient():
+    def __init__(self, receive_cb, timeout=5, timer=5, timer_backoff=1.1):
+        self.receive_cb = receive_cb
+        self.receiver = None
+        self.addr = None
+        self.port_proxy = None
+        self.port = None
+        self._reconnect_event = asyncio.Event()
+        self.timeout = timeout
+        self.timer = timer
+        self.timer_cur = timer
+        self.timer_backoff = timer_backoff
+        self._reconnect_task = asyncio.ensure_future(self._reconnect())
+
+    def update_address(self, addr, port, port_proxy):
+        self.addr = addr
+        self.port = port
+        self.port_proxy = port_proxy
+        self._reconnect_event.set()
+
+    async def trigger_proxy_task(self):
+        remote = AsyncioClient()
+        try:
+            try:
+                if self.addr is None:
+                    logger.error("missing core_analyzer host in device db")
+                    return
+                await remote.connect_rpc(self.addr, self.port, "coreanalyzer_proxy_control")
+            except:
+                logger.error("error connecting to analyzer proxy control", exc_info=True)
+                return
+            await remote.trigger()
+        except:
+            logger.error("analyzer proxy reported failure", exc_info=True)
+        finally:
+            remote.close_rpc()
+
+    async def _reconnect(self):
+        while True:
+            await self._reconnect_event.wait()
+            self._reconnect_event.clear()
+            if self.receiver is not None:
+                await self.receiver.close()
+                self.receiver = None
+            new_receiver = comm_analyzer.AnalyzerProxyReceiver(
+                self.receive_cb, self.disconnect_cb)
+            try:
+                if self.addr is not None:
+                    await asyncio.wait_for(new_receiver.connect(self.addr, self.port_proxy),
+                                           self.timeout)
+                    logger.info("ARTIQ dashboard connected to analyzer proxy (%s)", self.addr)
+                    self.timer_cur = self.timer
+                    self.receiver = new_receiver
+                continue
+            except Exception:
+                logger.error("error connecting to analyzer proxy", exc_info=True)
+            try:
+                await asyncio.wait_for(self._reconnect_event.wait(), self.timer_cur)
+            except asyncio.TimeoutError:
+                self.timer_cur *= self.timer_backoff
+                self._reconnect_event.set()
+            else:
+                self.timer_cur = self.timer
+
+    async def close(self):
+        self._reconnect_task.cancel()
+        try:
+            await asyncio.wait_for(self._reconnect_task, None)
+        except asyncio.CancelledError:
+            pass
+        if self.receiver is not None:
+            await self.receiver.close()
+
+    def disconnect_cb(self):
+        logger.error("lost connection to analyzer proxy")
+        self._reconnect_event.set()
+
+
+class _BackgroundItem(pg.GraphicsWidgetAnchor, pg.GraphicsWidget):
+    def __init__(self, parent, rect):
+        pg.GraphicsWidget.__init__(self, parent)
+        pg.GraphicsWidgetAnchor.__init__(self)
+        self.item = QtWidgets.QGraphicsRectItem(rect, self)
+        brush = QtGui.QBrush(QtGui.QColor(10, 10, 10, 140))
+        self.item.setBrush(brush)
+
+
+class _BaseWaveform(pg.PlotWidget):
+    cursorMove = QtCore.pyqtSignal(float)
+
+    def __init__(self, name, width, precision, unit,
+                 parent=None, pen="r", stepMode="right", connect="finite"):
+        pg.PlotWidget.__init__(self,
+                               parent=parent,
+                               x=None,
+                               y=None,
+                               pen=pen,
+                               stepMode=stepMode,
+                               connect=connect)
+
+        self.setMinimumHeight(WAVEFORM_MIN_HEIGHT)
+        self.setMaximumHeight(WAVEFORM_MAX_HEIGHT)
+        self.setMenuEnabled(False)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+
+        self.name = name
+        self.width = width
+        self.precision = precision
+        self.unit = unit
+
+        self.x_data = []
+        self.y_data = []
+
+        self.plot_item = self.getPlotItem()
+        self.plot_item.hideButtons()
+        self.plot_item.hideAxis("top")
+        self.plot_item.getAxis("bottom").setStyle(showValues=False, tickLength=0)
+        self.plot_item.getAxis("left").setStyle(showValues=False, tickLength=0)
+        self.plot_item.setRange(yRange=(0, 1), padding=0.1)
+        self.plot_item.showGrid(x=True, y=True)
+
+        self.plot_data_item = self.plot_item.listDataItems()[0]
+        self.plot_data_item.setClipToView(True)
+
+        self.view_box = self.plot_item.getViewBox()
+        self.view_box.setMouseEnabled(x=True, y=False)
+        self.view_box.disableAutoRange(axis=pg.ViewBox.YAxis)
+        self.view_box.setLimits(xMin=0, minXRange=20)
+
+        self.title_label = pg.LabelItem(self.name, parent=self.plot_item)
+        self.title_label.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 0))
+        self.title_label.setAttr('justify', 'left')
+        self.title_label.setZValue(10)
+
+        rect = self.title_label.boundingRect()
+        rect.setHeight(rect.height() * 2)
+        rect.setWidth(225)
+        self.label_bg = _BackgroundItem(parent=self.plot_item, rect=rect)
+        self.label_bg.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 0))
+
+        self.cursor = pg.InfiniteLine()
+        self.cursor_y = None
+        self.addItem(self.cursor)
+
+        self.cursor_label = pg.LabelItem('', parent=self.plot_item)
+        self.cursor_label.anchor(itemPos=(0, 0), parentPos=(0, 0), offset=(0, 20))
+        self.cursor_label.setAttr('justify', 'left')
+        self.cursor_label.setZValue(10)
+
+    def setStoppedX(self, stopped_x):
+        self.stopped_x = stopped_x
+        self.view_box.setLimits(xMax=stopped_x)
+
+    def setData(self, data):
+        if len(data) == 0:
+            self.x_data, self.y_data = [], []
+        else:
+            self.x_data, self.y_data = zip(*data)
+
+    def onDataChange(self, data):
+        raise NotImplementedError
+
+    def onCursorMove(self, x):
+        self.cursor.setValue(x)
+        if len(self.x_data) < 1:
+            return
+        ind = bisect.bisect_left(self.x_data, x) - 1
+        dr = self.plot_data_item.dataRect()
+        self.cursor_y = None
+        if dr is not None and 0 <= ind < len(self.y_data):
+            self.cursor_y = self.y_data[ind]
+
+    def mouseMoveEvent(self, e):
+        if e.buttons() == QtCore.Qt.LeftButton \
+           and e.modifiers() == QtCore.Qt.ShiftModifier:
+            drag = QtGui.QDrag(self)
+            mime = QtCore.QMimeData()
+            drag.setMimeData(mime)
+            pixmapi = QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileIcon)
+            drag.setPixmap(pixmapi.pixmap(32))
+            drag.exec_(QtCore.Qt.MoveAction)
+        else:
+            super().mouseMoveEvent(e)
+
+    def wheelEvent(self, e):
+        if e.modifiers() & QtCore.Qt.ControlModifier:
+            super().wheelEvent(e)
+
+    def mouseDoubleClickEvent(self, e):
+        pos = self.view_box.mapSceneToView(e.pos())
+        self.cursorMove.emit(pos.x())
+
+
+class BitWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, unit, parent)
+        self.plot_item.showGrid(x=True, y=False)
+        self._arrows = []
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for arw in self._arrows:
+                self.removeItem(arw)
+            self._arrows = []
+            l = len(data)
+            display_y = np.empty(l)
+            display_x = np.empty(l)
+            display_map = {
+                "X": 0.5,
+                "1": 1,
+                "0": 0
+            }
+            previous_y = None
+            for i, coord in enumerate(data):
+                x, y = coord
+                dis_y = display_map[y]
+                if previous_y == y:
+                    arw = pg.ArrowItem(pxMode=True, angle=90)
+                    self.addItem(arw)
+                    self._arrows.append(arw)
+                    arw.setPos(x, dis_y)
+                display_y[i] = dis_y
+                display_x[i] = x
+                previous_y = y
+            self.plot_data_item.setData(x=display_x, y=display_y)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for arw in self._arrows:
+                self.removeItem(arw)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            self.cursor_label.setText(self.cursor_y)
+        else:
+            self.cursor_label.setText("")
+
+
+class AnalogWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, unit, parent)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            self.plot_data_item.setData(x=self.x_data, y=self.y_data)
+            if len(data) > 0:
+                max_y = max(self.y_data)
+                min_y = min(self.y_data)
+                self.plot_item.setRange(yRange=(min_y, max_y), padding=0.1)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            t = short_format(self.cursor_y, {"precision": self.precision, "unit": self.unit})
+        else:
+            t = ""
+        self.cursor_label.setText(t)
+
+
+class BitVectorWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, parent)
+        self._labels = []
+        self._format_string = "{:0=" + str(math.ceil(width / 4)) + "X}"
+        self.view_box.sigTransformChanged.connect(self._update_labels)
+        self.plot_item.showGrid(x=True, y=False)
+
+    def _update_labels(self):
+        for label in self._labels:
+            self.removeItem(label)
+        xmin, xmax = self.view_box.viewRange()[0]
+        left_label_i = bisect.bisect_left(self.x_data, xmin)
+        right_label_i = bisect.bisect_right(self.x_data, xmax) + 1
+        for i, j in itertools.pairwise(range(left_label_i, right_label_i)):
+            x1 = self.x_data[i]
+            x2 = self.x_data[j] if j < len(self.x_data) else self.stopped_x
+            lbl = self._labels[i]
+            bounds = lbl.boundingRect()
+            bounds_view = self.view_box.mapSceneToView(bounds)
+            if bounds_view.boundingRect().width() < x2 - x1:
+                self.addItem(lbl)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self._labels = []
+            l = len(data)
+            display_x = np.empty(l * 2)
+            display_y = np.empty(l * 2)
+            for i, coord in enumerate(data):
+                x, y = coord
+                display_x[i * 2] = x
+                display_x[i * 2 + 1] = x
+                display_y[i * 2] = 0
+                display_y[i * 2 + 1] = int(int(y) != 0)
+                lbl = pg.TextItem(
+                    self._format_string.format(int(y, 2)), anchor=(0, 0.5))
+                lbl.setPos(x, 0.5)
+                lbl.setTextWidth(100)
+                self._labels.append(lbl)
+            self.plot_data_item.setData(x=display_x, y=display_y)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self.plot_data_item.setData(x=[], y=[])
+
+    def onCursorMove(self, x):
+        _BaseWaveform.onCursorMove(self, x)
+        if self.cursor_y is not None:
+            t = self._format_string.format(int(self.cursor_y, 2))
+        else:
+            t = ""
+        self.cursor_label.setText(t)
+
+
+class LogWaveform(_BaseWaveform):
+    def __init__(self, name, width, precision, unit, parent=None):
+        _BaseWaveform.__init__(self, name, width, precision, parent)
+        self.plot_data_item.opts['pen'] = None
+        self.plot_data_item.opts['symbol'] = 'x'
+        self._labels = []
+        self.plot_item.showGrid(x=True, y=False)
+
+    def onDataChange(self, data):
+        try:
+            self.setData(data)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self._labels = []
+            self.plot_data_item.setData(
+                x=self.x_data, y=np.ones(len(self.x_data)))
+            if len(data) == 0:
+                return
+            old_x = data[0][0]
+            old_msg = data[0][1]
+            for x, msg in data[1:]:
+                if x == old_x:
+                    old_msg += "\n" + msg
+                else:
+                    lbl = pg.TextItem(old_msg)
+                    self.addItem(lbl)
+                    self._labels.append(lbl)
+                    lbl.setPos(old_x, 1)
+                    old_msg = msg
+                    old_x = x
+            lbl = pg.TextItem(old_msg)
+            self.addItem(lbl)
+            self._labels.append(lbl)
+            lbl.setPos(old_x, 1)
+        except:
+            logger.error("Error when displaying waveform: %s", self.name, exc_info=True)
+            for lbl in self._labels:
+                self.plot_item.removeItem(lbl)
+            self.plot_data_item.setData(x=[], y=[])
+
+
+class _WaveformView(QtWidgets.QWidget):
+    cursorMove = QtCore.pyqtSignal(float)
+
+    def __init__(self, parent):
+        QtWidgets.QWidget.__init__(self, parent=parent)
+
+        self._stopped_x = None
+        self._timescale = 1
+        self._cursor_x = 0
+
+        layout = QtWidgets.QVBoxLayout()
+        layout.setContentsMargins(0, 0, 0, 0)
+        layout.setSpacing(0)
+        self.setLayout(layout)
+
+        self._ref_axis = pg.PlotWidget()
+        self._ref_axis.hideAxis("bottom")
+        self._ref_axis.hideAxis("left")
+        self._ref_axis.hideButtons()
+        self._ref_axis.setFixedHeight(45)
+        self._ref_axis.setMenuEnabled(False)
+        self._top = pg.AxisItem("top")
+        self._top.setScale(1e-12)
+        self._top.setLabel(units="s")
+        self._ref_axis.setAxisItems({"top": self._top})
+        layout.addWidget(self._ref_axis)
+
+        self._ref_vb = self._ref_axis.getPlotItem().getViewBox()
+        self._ref_vb.setFixedHeight(0)
+        self._ref_vb.setMouseEnabled(x=True, y=False)
+        self._ref_vb.setLimits(xMin=0)
+
+        scroll_area = VDragScrollArea(self)
+        scroll_area.setWidgetResizable(True)
+        scroll_area.setContentsMargins(0, 0, 0, 0)
+        scroll_area.setFrameShape(QtWidgets.QFrame.NoFrame)
+        scroll_area.setVerticalScrollBarPolicy(QtCore.Qt.ScrollBarAlwaysOff)
+        layout.addWidget(scroll_area)
+
+        self._splitter = VDragDropSplitter(parent=scroll_area)
+        self._splitter.setHandleWidth(1)
+        scroll_area.setWidget(self._splitter)
+
+        self.cursorMove.connect(self.onCursorMove)
+
+        self.confirm_delete_dialog = QtWidgets.QMessageBox(self)
+        self.confirm_delete_dialog.setIcon(
+            QtWidgets.QMessageBox.Icon.Warning
+        )
+        self.confirm_delete_dialog.setText("Delete all waveforms?")
+        self.confirm_delete_dialog.setStandardButtons(
+            QtWidgets.QMessageBox.Ok | QtWidgets.QMessageBox.Cancel
+        )
+        self.confirm_delete_dialog.setDefaultButton(
+            QtWidgets.QMessageBox.Ok
+        )
+
+    def setModel(self, model):
+        self._model = model
+        self._model.dataChanged.connect(self.onDataChange)
+        self._model.rowsInserted.connect(self.onInsert)
+        self._model.rowsRemoved.connect(self.onRemove)
+        self._model.rowsMoved.connect(self.onMove)
+        self._splitter.dropped.connect(self._model.move)
+        self.confirm_delete_dialog.accepted.connect(self._model.clear)
+
+    def setTimescale(self, timescale):
+        self._timescale = timescale
+        self._top.setScale(1e-12 * timescale)
+
+    def setStoppedX(self, stopped_x):
+        self._stopped_x = stopped_x
+        self._ref_vb.setLimits(xMax=stopped_x)
+        self._ref_vb.setRange(xRange=(0, stopped_x))
+        for i in range(self._model.rowCount()):
+            self._splitter.widget(i).setStoppedX(stopped_x)
+
+    def resetZoom(self):
+        if self._stopped_x is not None:
+            self._ref_vb.setRange(xRange=(0, self._stopped_x))
+
+    def onDataChange(self, top, bottom, roles):
+        self.cursorMove.emit(0)
+        first = top.row()
+        last = bottom.row()
+        data_row = self._model.headers.index("data")
+        for i in range(first, last + 1):
+            data = self._model.data(self._model.index(i, data_row))
+            self._splitter.widget(i).onDataChange(data)
+
+    def onInsert(self, parent, first, last):
+        for i in range(first, last + 1):
+            w = self._create_waveform(i)
+            self._splitter.insertWidget(i, w)
+        self._resize()
+
+    def onRemove(self, parent, first, last):
+        for i in reversed(range(first, last + 1)):
+            w = self._splitter.widget(i)
+            w.deleteLater()
+        self._splitter.refresh()
+        self._resize()
+
+    def onMove(self, src_parent, src_start, src_end, dest_parent, dest_row):
+        w = self._splitter.widget(src_start)
+        self._splitter.insertWidget(dest_row, w)
+
+    def onCursorMove(self, x):
+        self._cursor_x = x
+        for i in range(self._model.rowCount()):
+            self._splitter.widget(i).onCursorMove(x)
+
+    def _create_waveform(self, row):
+        name, ty, width, precision, unit = (
+            self._model.data(self._model.index(row, i)) for i in range(5))
+        waveform_cls = {
+            WaveformType.BIT: BitWaveform,
+            WaveformType.VECTOR: BitVectorWaveform,
+            WaveformType.ANALOG: AnalogWaveform,
+            WaveformType.LOG: LogWaveform
+        }[ty]
+        w = waveform_cls(name, width, precision, unit, parent=self._splitter)
+        w.setXLink(self._ref_vb)
+        w.setStoppedX(self._stopped_x)
+        w.cursorMove.connect(self.cursorMove)
+        w.onCursorMove(self._cursor_x)
+        action = QtWidgets.QAction("Delete waveform", w)
+        action.triggered.connect(lambda: self._delete_waveform(w))
+        w.addAction(action)
+        action = QtWidgets.QAction("Delete all waveforms", w)
+        action.triggered.connect(self.confirm_delete_dialog.open)
+        w.addAction(action)
+        return w
+
+    def _delete_waveform(self, waveform):
+        row = self._splitter.indexOf(waveform)
+        self._model.pop(row)
+
+    def _resize(self):
+        self._splitter.setFixedHeight(
+            int((WAVEFORM_MIN_HEIGHT + WAVEFORM_MAX_HEIGHT) * self._model.rowCount() / 2))
+
+
+class _WaveformModel(QtCore.QAbstractTableModel):
+    def __init__(self):
+        self.backing_struct = []
+        self.headers = ["name", "type", "width", "precision", "unit", "data"]
+        QtCore.QAbstractTableModel.__init__(self)
+
+    def rowCount(self, parent=QtCore.QModelIndex()):
+        return len(self.backing_struct)
+
+    def columnCount(self, parent=QtCore.QModelIndex()):
+        return len(self.headers)
+
+    def data(self, index, role=QtCore.Qt.DisplayRole):
+        if index.isValid():
+            return self.backing_struct[index.row()][index.column()]
+        return None
+
+    def extend(self, data):
+        length = len(self.backing_struct)
+        len_data = len(data)
+        self.beginInsertRows(QtCore.QModelIndex(), length, length + len_data - 1)
+        self.backing_struct.extend(data)
+        self.endInsertRows()
+
+    def pop(self, row):
+        self.beginRemoveRows(QtCore.QModelIndex(), row, row)
+        self.backing_struct.pop(row)
+        self.endRemoveRows()
+
+    def move(self, src, dest):
+        if src == dest:
+            return
+        if src < dest:
+            dest, src = src, dest
+        self.beginMoveRows(QtCore.QModelIndex(), src, src, QtCore.QModelIndex(), dest)
+        self.backing_struct.insert(dest, self.backing_struct.pop(src))
+        self.endMoveRows()
+
+    def clear(self):
+        self.beginRemoveRows(QtCore.QModelIndex(), 0, len(self.backing_struct) - 1)
+        self.backing_struct.clear()
+        self.endRemoveRows()
+
+    def export_list(self):
+        return [[row[0], row[1].value, *row[2:5]] for row in self.backing_struct]
+
+    def import_list(self, channel_list):
+        self.clear()
+        data = [[row[0], WaveformType(row[1]), *row[2:5], []] for row in channel_list]
+        self.extend(data)
+
+    def update_data(self, waveform_data, top, bottom):
+        name_col = self.headers.index("name")
+        data_col = self.headers.index("data")
+        for i in range(top, bottom):
+            name = self.data(self.index(i, name_col))
+            self.backing_struct[i][data_col] = waveform_data.get(name, [])
+            self.dataChanged.emit(self.index(i, data_col),
+                                  self.index(i, data_col))
+
+    def update_all(self, waveform_data):
+        self.update_data(waveform_data, 0, self.rowCount())
+
+
+class _CursorTimeControl(QtWidgets.QLineEdit):
+    submit = QtCore.pyqtSignal(float)
+
+    def __init__(self, parent):
+        QtWidgets.QLineEdit.__init__(self, parent=parent)
+        self._text = ""
+        self._value = 0
+        self._timescale = 1
+        self.setDisplayValue(0)
+        self.textChanged.connect(self._onTextChange)
+        self.returnPressed.connect(self._onReturnPress)
+
+    def setTimescale(self, timescale):
+        self._timescale = timescale
+
+    def _onTextChange(self, text):
+        self._text = text
+
+    def setDisplayValue(self, value):
+        self._value = value
+        self._text = pg.siFormat(value * 1e-12 * self._timescale,
+                                 suffix="s",
+                                 allowUnicode=False,
+                                 precision=15)
+        self.setText(self._text)
+
+    def _setValueFromText(self, text):
+        try:
+            self._value = pg.siEval(text) * (1e12 / self._timescale)
+        except:
+            logger.error("Error when parsing cursor time input", exc_info=True)
+
+    def _onReturnPress(self):
+        self._setValueFromText(self._text)
+        self.setDisplayValue(self._value)
+        self.submit.emit(self._value)
+        self.clearFocus()
+
+
+class Model(DictSyncTreeSepModel):
+    def __init__(self, init):
+        DictSyncTreeSepModel.__init__(self, "/", ["Channels"], init)
+
+    def clear(self):
+        for k in self.backing_store:
+            self._del_item(self, k.split(self.separator))
+        self.backing_store.clear()
+
+    def update(self, d):
+        for k, v in d.items():
+            self[k] = v
+
+
+class _AddChannelDialog(QtWidgets.QDialog):
+    def __init__(self, parent, model):
+        QtWidgets.QDialog.__init__(self, parent=parent)
+        self.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+        self.setWindowTitle("Add channels")
+
+        layout = QtWidgets.QVBoxLayout()
+        self.setLayout(layout)
+
+        self._model = model
+        self._tree_view = QtWidgets.QTreeView()
+        self._tree_view.setHeaderHidden(True)
+        self._tree_view.setSelectionBehavior(
+            QtWidgets.QAbstractItemView.SelectItems)
+        self._tree_view.setSelectionMode(
+            QtWidgets.QAbstractItemView.ExtendedSelection)
+        self._tree_view.setModel(self._model)
+        layout.addWidget(self._tree_view)
+
+        self._button_box = QtWidgets.QDialogButtonBox(
+            QtWidgets.QDialogButtonBox.Ok | QtWidgets.QDialogButtonBox.Cancel
+        )
+        self._button_box.setCenterButtons(True)
+        self._button_box.accepted.connect(self.add_channels)
+        self._button_box.rejected.connect(self.reject)
+        layout.addWidget(self._button_box)
+
+    def add_channels(self):
+        selection = self._tree_view.selectedIndexes()
+        channels = []
+        for select in selection:
+            key = self._model.index_to_key(select)
+            if key is not None:
+                channels.append([key, *self._model[key].ref, []])
+        self.channels = channels
+        self.accept()
+
+
+class WaveformDock(QtWidgets.QDockWidget):
+    def __init__(self, timeout, timer, timer_backoff):
+        QtWidgets.QDockWidget.__init__(self, "Waveform")
+        self.setObjectName("Waveform")
+        self.setFeatures(
+            QtWidgets.QDockWidget.DockWidgetMovable | QtWidgets.QDockWidget.DockWidgetFloatable)
+
+        self._channel_model = Model({})
+        self._waveform_model = _WaveformModel()
+
+        self._ddb = None
+        self._dump = None
+
+        self._waveform_data = {
+            "timescale": 1,
+            "stopped_x": None,
+            "logs": dict(),
+            "data": dict(),
+        }
+
+        self._current_dir = os.getcwd()
+
+        self.proxy_client = ProxyClient(self.on_dump_receive,
+                                        timeout,
+                                        timer,
+                                        timer_backoff)
+
+        grid = LayoutWidget()
+        self.setWidget(grid)
+
+        self._menu_btn = QtWidgets.QPushButton()
+        self._menu_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogStart))
+        grid.addWidget(self._menu_btn, 0, 0)
+
+        self._request_dump_btn = QtWidgets.QToolButton()
+        self._request_dump_btn.setToolTip("Fetch analyzer data from device")
+        self._request_dump_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_BrowserReload))
+        self._request_dump_btn.clicked.connect(
+            lambda: asyncio.ensure_future(exc_to_warning(self.proxy_client.trigger_proxy_task())))
+        grid.addWidget(self._request_dump_btn, 0, 1)
+
+        self._add_channel_dialog = _AddChannelDialog(self, self._channel_model)
+        self._add_channel_dialog.accepted.connect(self._add_channels)
+
+        self._add_btn = QtWidgets.QToolButton()
+        self._add_btn.setToolTip("Add channels...")
+        self._add_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_FileDialogListView))
+        self._add_btn.clicked.connect(self._add_channel_dialog.open)
+        grid.addWidget(self._add_btn, 0, 2)
+
+        self._file_menu = QtWidgets.QMenu()
+        self._add_async_action("Open trace...", self.load_trace)
+        self._add_async_action("Save trace...", self.save_trace)
+        self._add_async_action("Save trace as VCD...", self.save_vcd)
+        self._add_async_action("Open channel list...", self.load_channels)
+        self._add_async_action("Save channel list...", self.save_channels)
+        self._menu_btn.setMenu(self._file_menu)
+
+        self._waveform_view = _WaveformView(self)
+        self._waveform_view.setModel(self._waveform_model)
+        grid.addWidget(self._waveform_view, 1, 0, colspan=12)
+
+        self._reset_zoom_btn = QtWidgets.QToolButton()
+        self._reset_zoom_btn.setToolTip("Reset zoom")
+        self._reset_zoom_btn.setIcon(
+            QtWidgets.QApplication.style().standardIcon(
+                QtWidgets.QStyle.SP_TitleBarMaxButton))
+        self._reset_zoom_btn.clicked.connect(self._waveform_view.resetZoom)
+        grid.addWidget(self._reset_zoom_btn, 0, 3)
+
+        self._cursor_control = _CursorTimeControl(self)
+        self._waveform_view.cursorMove.connect(self._cursor_control.setDisplayValue)
+        self._cursor_control.submit.connect(self._waveform_view.onCursorMove)
+        grid.addWidget(self._cursor_control, 0, 4, colspan=6)
+
+    def _add_async_action(self, label, coro):
+        action = QtWidgets.QAction(label, self)
+        action.triggered.connect(
+            lambda: asyncio.ensure_future(exc_to_warning(coro())))
+        self._file_menu.addAction(action)
+
+    def _add_channels(self):
+        channels = self._add_channel_dialog.channels
+        count = self._waveform_model.rowCount()
+        self._waveform_model.extend(channels)
+        self._waveform_model.update_data(self._waveform_data['data'],
+                                         count,
+                                         count + len(channels))
+
+    def on_dump_receive(self, dump):
+        self._dump = dump
+        decoded_dump = comm_analyzer.decode_dump(dump)
+        waveform_data = comm_analyzer.decoded_dump_to_waveform_data(self._ddb, decoded_dump)
+        self._waveform_data.update(waveform_data)
+        self._channel_model.update(self._waveform_data['logs'])
+        self._waveform_model.update_all(self._waveform_data['data'])
+        self._waveform_view.setStoppedX(self._waveform_data['stopped_x'])
+        self._waveform_view.setTimescale(self._waveform_data['timescale'])
+        self._cursor_control.setTimescale(self._waveform_data['timescale'])
+
+    async def load_trace(self):
+        try:
+            filename = await get_open_file_name(
+                self,
+                "Load Analyzer Trace",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            with open(filename, 'rb') as f:
+                dump = f.read()
+            self.on_dump_receive(dump)
+        except:
+            logger.error("Failed to open analyzer trace", exc_info=True)
+
+    async def save_trace(self):
+        if self._dump is None:
+            logger.error("No analyzer trace stored in dashboard, "
+                         "try loading from file or fetching from device")
+            return
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save Analyzer Trace",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            with open(filename, 'wb') as f:
+                f.write(self._dump)
+        except:
+            logger.error("Failed to save analyzer trace", exc_info=True)
+
+    async def save_vcd(self):
+        if self._dump is None:
+            logger.error("No analyzer trace stored in dashboard, "
+                         "try loading from file or fetching from device")
+            return
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save VCD",
+                self._current_dir,
+                "All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            decoded_dump = comm_analyzer.decode_dump(self._dump)
+            with open(filename, 'w') as f:
+                comm_analyzer.decoded_dump_to_vcd(f, self._ddb, decoded_dump)
+        except:
+            logger.error("Failed to save trace as VCD", exc_info=True)
+
+    async def load_channels(self):
+        try:
+            filename = await get_open_file_name(
+                self,
+                "Open channel list",
+                self._current_dir,
+                "PYON files (*.pyon);;All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            channel_list = pyon.load_file(filename)
+            self._waveform_model.import_list(channel_list)
+            self._waveform_model.update_all(self._waveform_data['data'])
+        except:
+            logger.error("Failed to open channel list", exc_info=True)
+
+    async def save_channels(self):
+        try:
+            filename = await get_save_file_name(
+                self,
+                "Save channel list",
+                self._current_dir,
+                "PYON files (*.pyon);;All files (*.*)")
+        except asyncio.CancelledError:
+            return
+        self._current_dir = os.path.dirname(filename)
+        try:
+            channel_list = self._waveform_model.export_list()
+            pyon.store_file(filename, channel_list)
+        except:
+            logger.error("Failed to save channel list", exc_info=True)
+
+    def _process_ddb(self):
+        channel_list = comm_analyzer.get_channel_list(self._ddb)
+        self._channel_model.clear()
+        self._channel_model.update(channel_list)
+        desc = self._ddb.get("core_analyzer")
+        if desc is not None:
+            addr = desc["host"]
+            port_proxy = desc.get("port_proxy", 1385)
+            port = desc.get("port", 1386)
+            self.proxy_client.update_address(addr, port, port_proxy)
+        else:
+            self.proxy_client.update_address(None, None, None)
+
+    def init_ddb(self, ddb):
+        self._ddb = ddb
+        self._process_ddb()
+        return ddb
+
+    def notify_ddb(self, mod):
+        self._process_ddb()
+
+    async def stop(self):
+        if self.proxy_client is not None:
+            await self.proxy_client.close()

artiq/examples/artiq_ipython_notebook.ipynb

@@ -127,7 +127,7 @@
     "# let's connect to the master\n",
     "\n",
     "schedule, exps, datasets = [\n",
-    "    Client(\"::1\", 3251, \"master_\" + i) for i in\n",
+    "    Client(\"::1\", 3251, i) for i in\n",
     "    \"schedule experiment_db dataset_db\".split()]\n",
     "\n",
     "print(\"current schedule\")\n",

artiq/examples/kasli_shuttler/kasli_shuttler.json

@@ -6,6 +6,7 @@
     "peripherals": [
         {
             "type": "shuttler",
+            "hw_rev": "v1.1",
             "ports": [0]
         },
         {

artiq/examples/no_hardware/repository/interactive.py

@@ -0,0 +1,34 @@
+from artiq.experiment import *
+
+
+class InteractiveDemo(EnvExperiment):
+    def build(self):
+        pass
+
+    def run(self):
+        print("Waiting for user input...")
+        with self.interactive(title="Interactive Demo") as interactive:
+            interactive.setattr_argument("pyon_value",
+                                         PYONValue(self.get_dataset("foo", default=42)))
+            interactive.setattr_argument("number", NumberValue(42e-6,
+                                         unit="us",
+                                         precision=4))
+            interactive.setattr_argument("integer", NumberValue(42,
+                                         step=1, precision=0))
+            interactive.setattr_argument("string", StringValue("Hello World"))
+            interactive.setattr_argument("scan", Scannable(global_max=400,
+                                         default=NoScan(325),
+                                         precision=6))
+            interactive.setattr_argument("boolean", BooleanValue(True), "Group")
+            interactive.setattr_argument("enum",
+                                         EnumerationValue(["foo", "bar", "quux"], "foo"),
+                                         "Group")
+        print("Done! Values:")
+        print(interactive.pyon_value)
+        print(interactive.boolean)
+        print(interactive.enum)
+        print(interactive.number, type(interactive.number))
+        print(interactive.integer, type(interactive.integer))
+        print(interactive.string)
+        for i in interactive.scan:
+            print(i)

artiq/firmware/Cargo.lock

@@ -1,10 +1,50 @@
 # This file is automatically @generated by Cargo.
 # It is not intended for manual editing.
+# And yet, manual edits have been made. Crate yanking should be illegal.
+version = 3
+
+[[package]]
+name = "addr2line"
+version = "0.16.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3e61f2b7f93d2c7d2b08263acaa4a363b3e276806c68af6134c44f523bf1aacd"
+dependencies = [
+ "cpp_demangle",
+ "fallible-iterator",
+ "gimli",
+ "object",
+ "rustc-demangle",
+ "smallvec",
+]
+
+[[package]]
+name = "adler"
+version = "0.2.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ee2a4ec343196209d6594e19543ae87a39f96d5534d7174822a3ad825dd6ed7e"
+
+[[package]]
+name = "adler"
+version = "1.0.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f26201604c87b1e01bd3d98f8d5d9a8fcbb815e8cedb41ffccbeb4bf593a35fe"
+
+[[package]]
+name = "ahash"
+version = "0.7.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "efa60d2eadd8b12a996add391db32bd1153eac697ba4869660c0016353611426"
+dependencies = [
+ "getrandom",
+ "once_cell",
+ "version_check",
+]
+
 [[package]]
 name = "aho-corasick"
-version = "0.7.18"
+version = "0.7.20"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1e37cfd5e7657ada45f742d6e99ca5788580b5c529dc78faf11ece6dc702656f"
+checksum = "cc936419f96fa211c1b9166887b38e5e40b19958e5b895be7c1f93adec7071ac"
 dependencies = [
  "memchr",
 ]
@@ -30,9 +70,9 @@ dependencies = [
 
 [[package]]
 name = "bit_field"
-version = "0.10.1"
+version = "0.10.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dcb6dd1c2376d2e096796e234a70e17e94cc2d5d54ff8ce42b28cef1d0d359a4"
+checksum = "dc827186963e592360843fb5ba4b973e145841266c1357f7180c43526f2e5b61"
 
 [[package]]
 name = "bitflags"
@@ -72,12 +112,30 @@ dependencies = [
 name = "bootloader"
 version = "0.0.0"
 dependencies = [
+ "addr2line",
+ "ahash",
  "board_misoc",
  "build_misoc",
  "byteorder",
+ "cc",
+ "compiler_builtins",
  "crc",
+ "dlmalloc",
+ "fortanix-sgx-abi",
+ "getopts",
+ "getrandom",
+ "hashbrown",
+ "hermit-abi",
+ "libc 0.2.99",
+ "miniz_oxide 0.4.0",
+ "object",
+ "once_cell",
  "riscv",
+ "rustc-demangle",
  "smoltcp",
+ "unicode-width",
+ "version_check",
+ "wasi",
 ]
 
 [[package]]
@@ -98,9 +156,9 @@ checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
 
 [[package]]
 name = "cc"
-version = "1.0.70"
+version = "1.0.69"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d26a6ce4b6a484fa3edb70f7efa6fc430fd2b87285fe8b84304fd0936faa0dc0"
+checksum = "e70cc2f62c6ce1868963827bd677764c62d07c3d9a3e1fb1177ee1a9ab199eb2"
 
 [[package]]
 name = "cfg-if"
@@ -116,9 +174,18 @@ checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
 
 [[package]]
 name = "compiler_builtins"
-version = "0.1.39"
+version = "0.1.49"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3748f82c7d366a0b4950257d19db685d4958d2fa27c6d164a3f069fec42b748b"
+checksum = "20b1438ef42c655665a8ab2c1c6d605a305f031d38d9be689ddfef41a20f3aa2"
+
+[[package]]
+name = "cpp_demangle"
+version = "0.3.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "eeaa953eaad386a53111e47172c2fedba671e5684c8dd601a5f474f4f118710f"
+dependencies = [
+ "cfg-if 1.0.0",
+]
 
 [[package]]
 name = "crc"
@@ -129,12 +196,30 @@ dependencies = [
  "build_const",
 ]
 
+[[package]]
+name = "crc32fast"
+version = "1.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b3855a8a784b474f333699ef2bbca9db2c4a1f6d9088a90a2d25b1eb53111eaa"
+dependencies = [
+ "cfg-if 1.0.0",
+]
+
 [[package]]
 name = "cslice"
 version = "0.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "0f8cb7306107e4b10e64994de6d3274bd08996a7c1322a27b86482392f96be0a"
 
+[[package]]
+name = "dlmalloc"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "332570860c2edf2d57914987bf9e24835425f75825086b6ba7d1e6a3e4f1f254"
+dependencies = [
+ "libc 0.2.99",
+]
+
 [[package]]
 name = "dyld"
 version = "0.0.0"
@@ -143,7 +228,6 @@ version = "0.0.0"
 name = "eh"
 version = "0.0.0"
 dependencies = [
- "compiler_builtins",
  "cslice",
  "libc 0.1.0",
  "unwind",
@@ -166,12 +250,82 @@ dependencies = [
  "synstructure",
 ]
 
+[[package]]
+name = "fallible-iterator"
+version = "0.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4443176a9f2c162692bd3d352d745ef9413eec5782a80d8fd6f8a1ac692a07f7"
+
+[[package]]
+name = "flate2"
+version = "1.0.30"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5f54427cfd1c7829e2a139fcefea601bf088ebca651d2bf53ebc600eac295dae"
+dependencies = [
+ "crc32fast",
+ "miniz_oxide 0.7.2",
+]
+
+[[package]]
+name = "fortanix-sgx-abi"
+version = "0.3.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c56c422ef86062869b2d57ae87270608dc5929969dd130a6e248979cf4fb6ca6"
+
 [[package]]
 name = "fringe"
 version = "1.2.1"
-source = "git+https://git.m-labs.hk/M-Labs/libfringe.git?rev=3ecbe5#3ecbe53f7644b18ee46ebd5b2ca12c9cbceec43a"
+source = "git+https://git.m-labs.hk/M-Labs/libfringe.git?rev=53a964#53a964a63d2d384b22ae1949a471a732003a30b9"
 dependencies = [
- "libc 0.2.101",
+ "libc 0.2.99",
+]
+
+[[package]]
+name = "getopts"
+version = "0.2.21"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "14dbbfd5c71d70241ecf9e6f13737f7b5ce823821063188d7e46c41d371eebd5"
+dependencies = [
+ "unicode-width",
+]
+
+[[package]]
+name = "getrandom"
+version = "0.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ee8025cf36f917e6a52cce185b7c7177689b838b7ec138364e50cc2277a56cf4"
+dependencies = [
+ "cfg-if 0.1.10",
+ "libc 0.2.99",
+ "wasi",
+]
+
+[[package]]
+name = "gimli"
+version = "0.25.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0a01e0497841a3b2db4f8afa483cce65f7e96a3498bd6c541734792aeac8fe7"
+dependencies = [
+ "fallible-iterator",
+ "stable_deref_trait",
+]
+
+[[package]]
+name = "hashbrown"
+version = "0.11.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "362385356d610bd1e5a408ddf8d022041774b683f345a1d2cfcb4f60f8ae2db5"
+dependencies = [
+ "ahash",
+]
+
+[[package]]
+name = "hermit-abi"
+version = "0.1.19"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "62b467343b94ba476dcb2500d242dadbb39557df889310ac77c5d99100aaac33"
+dependencies = [
+ "libc 0.2.99",
 ]
 
 [[package]]
@@ -212,9 +366,9 @@ version = "0.1.0"
 
 [[package]]
 name = "libc"
-version = "0.2.101"
+version = "0.2.99"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3cb00336871be5ed2c8ed44b60ae9959dc5b9f08539422ed43f09e34ecaeba21"
+checksum = "a7f823d141fe0a24df1e23b4af4e3c7ba9e5966ec514ea068c93024aa7deb765"
 
 [[package]]
 name = "log"
@@ -258,6 +412,40 @@ version = "2.4.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "308cc39be01b73d0d18f82a0e7b2a3df85245f84af96fdddc5d202d27e47b86a"
 
+[[package]]
+name = "miniz_oxide"
+version = "0.4.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "be0f75932c1f6cfae3c04000e40114adf955636e19040f9c0a2c380702aa1c7f"
+dependencies = [
+ "adler 0.2.3",
+]
+
+[[package]]
+name = "miniz_oxide"
+version = "0.7.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9d811f3e15f28568be3407c8e7fdb6514c1cda3cb30683f15b6a1a1dc4ea14a7"
+dependencies = [
+ "adler 1.0.2",
+]
+
+[[package]]
+name = "object"
+version = "0.26.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39f37e50073ccad23b6d09bcb5b263f4e76d3bb6038e4a3c08e52162ffa8abc2"
+dependencies = [
+ "flate2",
+ "memchr",
+]
+
+[[package]]
+name = "once_cell"
+version = "1.8.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "692fcb63b64b1758029e0a96ee63e049ce8c5948587f2f7208df04625e5f6b56"
+
 [[package]]
 name = "proto_artiq"
 version = "0.0.0"
@@ -280,9 +468,9 @@ checksum = "7a6e920b65c65f10b2ae65c831a81a073a89edd28c7cce89475bff467ab4167a"
 
 [[package]]
 name = "regex"
-version = "1.5.4"
+version = "1.7.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d07a8629359eb56f1e2fb1652bb04212c072a87ba68546a04065d525673ac461"
+checksum = "8b1f693b24f6ac912f4893ef08244d70b6067480d2f1a46e950c9691e6749d1d"
 dependencies = [
  "aho-corasick",
  "memchr",
@@ -291,9 +479,9 @@ dependencies = [
 
 [[package]]
 name = "regex-syntax"
-version = "0.6.25"
+version = "0.6.29"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f497285884f3fcff424ffc933e56d7cbca511def0c9831a7f9b5f6153e3cc89b"
+checksum = "f162c6dd7b008981e4d40210aca20b4bd0f9b60ca9271061b07f78537722f2e1"
 
 [[package]]
 name = "riscv"
@@ -342,6 +530,12 @@ dependencies = [
  "unwind_backtrace",
 ]
 
+[[package]]
+name = "rustc-demangle"
+version = "0.1.21"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7ef03e0a2b150c7a90d01faf6254c9c48a41e95fb2a8c2ac1c6f0d2b9aefc342"
+
 [[package]]
 name = "rustc_version"
 version = "0.2.3"
@@ -382,6 +576,12 @@ version = "0.7.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3"
 
+[[package]]
+name = "smallvec"
+version = "1.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3c5e1a9a646d36c3599cd173a41282daf47c44583ad367b8e6837255952e5c67"
+
 [[package]]
 name = "smoltcp"
 version = "0.8.2"
@@ -393,6 +593,12 @@ dependencies = [
  "managed 0.8.0",
 ]
 
+[[package]]
+name = "stable_deref_trait"
+version = "1.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3"
+
 [[package]]
 name = "syn"
 version = "0.11.11"
@@ -433,6 +639,12 @@ dependencies = [
  "log",
 ]
 
+[[package]]
+name = "unicode-width"
+version = "0.1.8"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9337591893a19b88d8d87f2cec1e73fad5cdfd10e5a6f349f498ad6ea2ffb1e3"
+
 [[package]]
 name = "unicode-xid"
 version = "0.0.4"
@@ -454,3 +666,15 @@ dependencies = [
  "libc 0.1.0",
  "unwind",
 ]
+
+[[package]]
+name = "version_check"
+version = "0.9.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5fecdca9a5291cc2b8dcf7dc02453fee791a280f3743cb0905f8822ae463b3fe"
+
+[[package]]
+name = "wasi"
+version = "0.9.0+wasi-snapshot-preview1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

artiq/firmware/bootloader/Cargo.toml

@@ -13,8 +13,30 @@ path = "main.rs"
 build_misoc = { path = "../libbuild_misoc" }
 
 [dependencies]
-byteorder = { version = "1.0", default-features = false }
+byteorder = { version = "=1.4.3", default-features = false }
 crc = { version = "1.7", default-features = false }
 board_misoc = { path = "../libboard_misoc", features = ["uart_console", "smoltcp"] }
 smoltcp = { version = "0.8.2", default-features = false, features = ["medium-ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"] }
 riscv = { version = "0.6.0", features = ["inline-asm"] }
+
+# insanity required by using cargo build-std over xbuild with nix
+# cargo update does not work thanks to ahash 0.7 problems
+[dev-dependencies]
+getopts = "=0.2.21"
+libc = "=0.2.99"
+unicode-width = "=0.1.8"
+addr2line = "=0.16.0"
+hashbrown = "=0.11.0" # must be injected into lockfile manually
+ahash = "=0.7.0" # must be injected into lockfile manually
+miniz_oxide = "=0.4.0"
+rustc-demangle = "=0.1.21"
+hermit-abi = "=0.1.19"
+dlmalloc = "=0.2.1"
+fortanix-sgx-abi = "=0.3.3"
+cc = "=1.0.69"
+compiler_builtins = "=0.1.49"
+version_check = "=0.9.3"
+once_cell = "=1.8.0"
+wasi = "=0.9.0"
+getrandom = "=0.2.0"
+object = "=0.26.2"

artiq/firmware/bootloader/Makefile

@@ -3,8 +3,6 @@ include $(MISOC_DIRECTORY)/software/common.mak
 
 RUSTFLAGS += -Cpanic=abort
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: bootloader.bin
 
 .PHONY: $(RUSTOUT)/libbootloader.a

artiq/firmware/bootloader/main.rs

@@ -500,7 +500,7 @@ pub extern fn main() -> i32 {
     println!(r"|_|  |_|_|____/ \___/ \____|");
     println!("");
     println!("MiSoC Bootloader");
-    println!("Copyright (c) 2017-2023 M-Labs Limited");
+    println!("Copyright (c) 2017-2024 M-Labs Limited");
     println!("");
 
     #[cfg(has_ethmac)]

artiq/firmware/ksupport/Makefile

@@ -14,8 +14,6 @@ LDFLAGS += --eh-frame-hdr \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: ksupport.elf
 
 .PHONY: $(RUSTOUT)/libksupport.a

artiq/firmware/ksupport/eh_artiq.rs

@@ -55,12 +55,14 @@ struct ExceptionBuffer {
     exception_count: usize,
 }
 
+const EXCEPTION: uw::_Unwind_Exception = uw::_Unwind_Exception {
+    exception_class:   EXCEPTION_CLASS,
+    exception_cleanup: cleanup,
+    private:           [0; uw::unwinder_private_data_size],
+};
+
 static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
-    uw_exceptions: [uw::_Unwind_Exception {
-        exception_class:   EXCEPTION_CLASS,
-        exception_cleanup: cleanup,
-        private:           [0; uw::unwinder_private_data_size],
-    }; MAX_INFLIGHT_EXCEPTIONS],
+    uw_exceptions: [EXCEPTION; MAX_INFLIGHT_EXCEPTIONS],
     exceptions: [None; MAX_INFLIGHT_EXCEPTIONS + 1],
     exception_stack: [-1; MAX_INFLIGHT_EXCEPTIONS + 1],
     backtrace: [(0, 0); MAX_BACKTRACE_SIZE],
@@ -74,11 +76,7 @@ static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
 };
 
 pub unsafe extern fn reset_exception_buffer(payload_addr: usize) {
-    EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception {
-        exception_class:   EXCEPTION_CLASS,
-        exception_cleanup: cleanup,
-        private:           [0; uw::unwinder_private_data_size],
-    }; MAX_INFLIGHT_EXCEPTIONS];
+    EXCEPTION_BUFFER.uw_exceptions = [EXCEPTION; MAX_INFLIGHT_EXCEPTIONS];
     EXCEPTION_BUFFER.exceptions = [None; MAX_INFLIGHT_EXCEPTIONS + 1];
     EXCEPTION_BUFFER.exception_stack = [-1; MAX_INFLIGHT_EXCEPTIONS + 1];
     EXCEPTION_BUFFER.backtrace_size = 0;
@@ -151,8 +149,7 @@ pub extern fn personality(version: c_int,
 }
 
 #[export_name="__artiq_raise"]
-#[unwind(allowed)]
-pub unsafe extern fn raise(exception: *const Exception) -> ! {
+pub unsafe extern "C-unwind" fn raise(exception: *const Exception) -> ! {
     let count = EXCEPTION_BUFFER.exception_count;
     let stack = &mut EXCEPTION_BUFFER.exception_stack;
     let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize;
@@ -222,8 +219,7 @@ pub unsafe extern fn raise(exception: *const Exception) -> ! {
 
 
 #[export_name="__artiq_resume"]
-#[unwind(allowed)]
-pub unsafe extern fn resume() -> ! {
+pub unsafe extern "C-unwind" fn resume() -> ! {
     assert!(EXCEPTION_BUFFER.exception_count != 0);
     let i = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
     assert!(i != -1);
@@ -233,8 +229,7 @@ pub unsafe extern fn resume() -> ! {
 }
 
 #[export_name="__artiq_end_catch"]
-#[unwind(allowed)]
-pub unsafe extern fn end_catch() {
+pub unsafe extern "C-unwind" fn end_catch() {
     let mut count = EXCEPTION_BUFFER.exception_count;
     assert!(count != 0);
     // we remove all exceptions with SP <= current exception SP

artiq/firmware/ksupport/lib.rs

@@ -1,5 +1,5 @@
-#![feature(lang_items, llvm_asm, panic_unwind, libc, unwind_attributes,
-           panic_info_message, nll, const_in_array_repeat_expressions)]
+#![feature(lang_items, asm, panic_unwind, libc,
+           panic_info_message, nll, c_unwind)]
 #![no_std]
 
 extern crate libc;
@@ -30,8 +30,9 @@ fn send(request: &Message) {
 }
 
 fn recv<R, F: FnOnce(&Message) -> R>(f: F) -> R {
-    while mailbox::receive() == 0 {}
-    let result = f(unsafe { &*(mailbox::receive() as *const Message) });
+    let mut msg_ptr = 0;
+    while msg_ptr == 0 { msg_ptr = mailbox::receive(); }
+    let result = f(unsafe { &*(msg_ptr as *const Message) });
     mailbox::acknowledge();
     result
 }
@@ -121,7 +122,6 @@ pub extern fn send_to_rtio_log(text: CSlice<u8>) {
     rtio::log(text.as_ref())
 }
 
-#[unwind(aborts)]
 extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while !rpc_queue::empty() {}
     send(&RpcSend {
@@ -132,7 +132,6 @@ extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     })
 }
 
-#[unwind(aborts)]
 extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while rpc_queue::full() {}
     rpc_queue::enqueue(|mut slice| {
@@ -170,8 +169,7 @@ extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ())
 /// to the maximum required for any of the possible types according to the target ABI).
 ///
 /// If the RPC call resulted in an exception, it is reconstructed and raised.
-#[unwind(allowed)]
-extern fn rpc_recv(slot: *mut ()) -> usize {
+extern "C-unwind" fn rpc_recv(slot: *mut ()) -> usize {
     send(&RpcRecvRequest(slot));
     recv!(&RpcRecvReply(ref result) => {
         match result {
@@ -203,7 +201,6 @@ fn terminate(exceptions: &'static [Option<eh_artiq::Exception<'static>>],
     loop {}
 }
 
-#[unwind(aborts)]
 extern fn cache_get<'a>(key: &CSlice<u8>) -> *const CSlice<'a, i32> {
     send(&CacheGetRequest {
         key:   str::from_utf8(key.as_ref()).unwrap()
@@ -213,8 +210,7 @@ extern fn cache_get<'a>(key: &CSlice<u8>) -> *const CSlice<'a, i32> {
     })
 }
 
-#[unwind(allowed)]
-extern fn cache_put(key: &CSlice<u8>, list: &CSlice<i32>) {
+extern "C-unwind" fn cache_put(key: &CSlice<u8>, list: &CSlice<i32>) {
     send(&CachePutRequest {
         key:   str::from_utf8(key.as_ref()).unwrap(),
         value: list.as_ref()
@@ -247,8 +243,7 @@ fn dma_record_flush() {
     }
 }
 
-#[unwind(allowed)]
-extern fn dma_record_start(name: &CSlice<u8>) {
+extern "C-unwind" fn dma_record_start(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     unsafe {
@@ -267,8 +262,7 @@ extern fn dma_record_start(name: &CSlice<u8>) {
     }
 }
 
-#[unwind(allowed)]
-extern fn dma_record_stop(duration: i64, enable_ddma: bool) {
+extern "C-unwind" fn dma_record_stop(duration: i64, enable_ddma: bool) {
     unsafe {
         dma_record_flush();
 
@@ -290,7 +284,6 @@ extern fn dma_record_stop(duration: i64, enable_ddma: bool) {
     }
 }
 
-#[unwind(aborts)]
 #[inline(always)]
 unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
                                     words: usize) -> &'static mut [u8] {
@@ -327,7 +320,6 @@ unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
     data
 }
 
-#[unwind(aborts)]
 extern fn dma_record_output(target: i32, word: i32) {
     unsafe {
         let timestamp = ((csr::rtio::now_hi_read() as i64) << 32) | (csr::rtio::now_lo_read() as i64);
@@ -341,7 +333,6 @@ extern fn dma_record_output(target: i32, word: i32) {
     }
 }
 
-#[unwind(aborts)]
 extern fn dma_record_output_wide(target: i32, words: &CSlice<i32>) {
     assert!(words.len() <= 16); // enforce the hardware limit
 
@@ -360,7 +351,6 @@ extern fn dma_record_output_wide(target: i32, words: &CSlice<i32>) {
     }
 }
 
-#[unwind(aborts)]
 extern fn dma_erase(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
@@ -374,8 +364,7 @@ struct DmaTrace {
     uses_ddma: bool,
 }
 
-#[unwind(allowed)]
-extern fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
+extern "C-unwind" fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     send(&DmaRetrieveRequest { name: name });
@@ -396,8 +385,7 @@ extern fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
 }
 
 #[cfg(kernel_has_rtio_dma)]
-#[unwind(allowed)]
-extern fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
+extern "C-unwind" fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
     assert!(ptr % 64 == 0);
 
     unsafe {
@@ -453,15 +441,39 @@ extern fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
     }
 }
 
-#[cfg(not(kernel_has_rtio_dma))]
-#[unwind(allowed)]
-extern fn dma_playback(_timestamp: i64, _ptr: i32, _uses_ddma: bool) {
+#[cfg(all(not(kernel_has_rtio_dma), not(has_rtio_dma)))]
+extern "C-unwind" fn dma_playback(_timestamp: i64, _ptr: i32, _uses_ddma: bool) {
     unimplemented!("not(kernel_has_rtio_dma)")
 }
 
-#[unwind(allowed)]
-extern fn subkernel_load_run(id: u32, run: bool) {
-    send(&SubkernelLoadRunRequest { id: id, run: run });
+// for satellite (has_rtio_dma but not in kernel)
+#[cfg(all(not(kernel_has_rtio_dma), has_rtio_dma))]
+extern "C-unwind" fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
+    // DDMA is always used on satellites, so the `uses_ddma` setting is ignored
+    // StartRemoteRequest reused as "normal" start request
+    send(&DmaStartRemoteRequest { id: ptr as i32, timestamp: timestamp });
+    // skip awaitremoterequest - it's a given
+    recv!(&DmaAwaitRemoteReply { timeout, error, channel, timestamp } => {
+        if timeout {
+            raise!("DMAError",
+                "Error running DMA on satellite device, timed out waiting for results");
+        }
+        if error & 1 != 0 {
+            raise!("RTIOUnderflow",
+                "RTIO underflow at channel {rtio_channel_info:0}, {1} mu",
+                channel as i64, timestamp as i64, 0);
+        }
+        if error & 2 != 0 {
+            raise!("RTIODestinationUnreachable",
+                "RTIO destination unreachable, output, at channel {rtio_channel_info:0}, {1} mu",
+                channel as i64, timestamp as i64, 0);
+        }
+    });
+}
+
+
+extern "C-unwind" fn subkernel_load_run(id: u32, destination: u8, run: bool) {
+    send(&SubkernelLoadRunRequest { id: id, destination: destination, run: run });
     recv!(&SubkernelLoadRunReply { succeeded } => {
         if !succeeded {
             raise!("SubkernelError",
@@ -470,8 +482,7 @@ extern fn subkernel_load_run(id: u32, run: bool) {
     });
 }
 
-#[unwind(allowed)]
-extern fn subkernel_await_finish(id: u32, timeout: u64) {
+extern "C-unwind" fn subkernel_await_finish(id: u32, timeout: i64) {
     send(&SubkernelAwaitFinishRequest { id: id, timeout: timeout });
     recv!(SubkernelAwaitFinishReply { status } => {
         match status {
@@ -488,18 +499,18 @@ extern fn subkernel_await_finish(id: u32, timeout: u64) {
     })
 }
 
-#[unwind(aborts)]
-extern fn subkernel_send_message(id: u32, count: u8, tag: &CSlice<u8>, data: *const *const ()) {
+extern fn subkernel_send_message(id: u32, is_return: bool, destination: u8, 
+    count: u8, tag: &CSlice<u8>, data: *const *const ()) {
     send(&SubkernelMsgSend { 
         id: id,
+        destination: if is_return { None } else { Some(destination) },
         count: count,
         tag: tag.as_ref(),
         data: data 
     });
 }
 
-#[unwind(allowed)]
-extern fn subkernel_await_message(id: u32, timeout: u64, tags: &CSlice<u8>, min: u8, max: u8) -> u8 {
+extern "C-unwind" fn subkernel_await_message(id: i32, timeout: i64, tags: &CSlice<u8>, min: u8, max: u8) -> u8 {
     send(&SubkernelMsgRecvRequest { id: id, timeout: timeout, tags: tags.as_ref() });
     recv!(SubkernelMsgRecvReply { status, count } => {
         match status {
@@ -624,8 +635,7 @@ pub unsafe fn main() {
 }
 
 #[no_mangle]
-#[unwind(allowed)]
-pub unsafe extern fn exception(_regs: *const u32) {
+pub unsafe extern "C-unwind" fn exception(_regs: *const u32) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
     let mtval = mtval::read();
@@ -643,7 +653,6 @@ pub unsafe extern fn exception(_regs: *const u32) {
 }
 
 #[no_mangle]
-#[unwind(allowed)]
-pub extern fn abort() {
+pub extern "C-unwind" fn abort() {
     panic!("aborted")
 }

artiq/firmware/libboard_artiq/Cargo.toml

@@ -25,3 +25,4 @@ proto_artiq = { path = "../libproto_artiq" }
 [features]
 uart_console = []
 alloc = []
+calibrate_wrpll_skew = []

artiq/firmware/libboard_artiq/drtioaux.rs

@@ -69,9 +69,9 @@ fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error<!>>
     let linkidx = linkno as usize;
     unsafe {
         if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
-            let ptr = DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2;
-            let len = (DRTIOAUX[linkidx].aux_rx_length_read)();
-            let result = f(slice::from_raw_parts(ptr as *mut u8, len as usize));
+            let read_ptr = (DRTIOAUX[linkidx].aux_read_pointer_read)() as usize;
+            let ptr = DRTIOAUX_MEM[linkidx].base + (DRTIOAUX_MEM[linkidx].size / 2) + (read_ptr * 0x400);
+            let result = f(slice::from_raw_parts(ptr as *mut u8, 0x400 as usize));
             (DRTIOAUX[linkidx].aux_rx_present_write)(1);
             Ok(Some(result?))
         } else {
@@ -86,21 +86,17 @@ pub fn recv(linkno: u8) -> Result<Option<Packet>, Error<!>> {
     }
 
     receive(linkno, |buffer| {
-        if buffer.len() < 8 {
-            return Err(IoError::UnexpectedEnd.into())
-        }
-
         let mut reader = Cursor::new(buffer);
 
-        let checksum_at = buffer.len() - 4;
+        let packet = Packet::read_from(&mut reader)?;
+        let padding = (12 - (reader.position() % 8)) % 8;
+        let checksum_at = reader.position() + padding;
         let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
         reader.set_position(checksum_at);
         if reader.read_u32()? != checksum {
             return Err(Error::CorruptedPacket)
         }
-        reader.set_position(0);
-
-        Ok(Packet::read_from(&mut reader)?)
+        Ok(packet)
     })
 }
 

artiq/firmware/libboard_artiq/lib.rs

@@ -1,4 +1,4 @@
-#![feature(lang_items, never_type)]
+#![feature(asm, lang_items, never_type)]
 #![no_std]
 
 extern crate failure;
@@ -24,6 +24,8 @@ pub mod rpc_queue;
 
 #[cfg(has_si5324)]
 pub mod si5324;
+#[cfg(has_si549)]
+pub mod si549;
 
 #[cfg(has_grabber)]
 pub mod grabber;

artiq/firmware/libboard_artiq/mailbox.rs

@@ -6,7 +6,11 @@ static mut LAST: usize = 0;
 
 pub unsafe fn send(data: usize) {
     LAST = data;
-    write_volatile(MAILBOX, data)
+    // after Rust toolchain update to LLVM12, this empty asm! block is required
+    // to ensure that the compiler doesn't take any shortcuts
+    // otherwise, the comm CPU will read garbage data and crash
+    asm!("", options(preserves_flags, readonly, nostack));
+    write_volatile(MAILBOX, data);
 }
 
 pub fn acknowledged() -> bool {

artiq/firmware/libboard_artiq/si549.rs

@@ -0,0 +1,862 @@
+use board_misoc::{clock, csr};
+use log::info;
+
+const ADDRESS: u8 = 0x67;
+
+const ADPLL_MAX: i32 = (950.0 / 0.0001164) as i32;
+
+pub struct DividerConfig {
+    pub hsdiv: u16,
+    pub lsdiv: u8,
+    pub fbdiv: u64,
+}
+
+pub struct FrequencySetting {
+    pub main: DividerConfig,
+    pub helper: DividerConfig,
+}
+
+mod i2c {
+    use super::*;
+
+    #[derive(Clone, Copy)]
+    pub enum DCXO {
+        Main,
+        Helper,
+    }
+
+    fn half_period() {
+        clock::spin_us(1)
+    }
+
+    fn sda_i(dcxo: DCXO) -> bool {
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_in_read() == 1 },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_in_read() == 1 },
+        }
+    }
+
+    fn sda_oe(dcxo: DCXO, oe: bool) {
+        let val = if oe { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_oe_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_oe_write(val) },
+        };
+    }
+
+    fn sda_o(dcxo: DCXO, o: bool) {
+        let val = if o { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_out_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_out_write(val) },
+        };
+    }
+
+    fn scl_oe(dcxo: DCXO, oe: bool) {
+        let val = if oe { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_oe_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_oe_write(val) },
+        };
+    }
+
+    fn scl_o(dcxo: DCXO, o: bool) {
+        let val = if o { 1 } else { 0 };
+        match dcxo {
+            DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_out_write(val) },
+            DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_out_write(val) },
+        };
+    }
+
+    pub fn init(dcxo: DCXO) -> Result<(), &'static str> {
+        // Set SCL as output, and high level
+        scl_o(dcxo, true);
+        scl_oe(dcxo, true);
+        // Prepare a zero level on SDA so that sda_oe pulls it down
+        sda_o(dcxo, false);
+        // Release SDA
+        sda_oe(dcxo, false);
+
+        // Check the I2C bus is ready
+        half_period();
+        half_period();
+        if !sda_i(dcxo) {
+            // Try toggling SCL a few times
+            for _bit in 0..8 {
+                scl_o(dcxo, false);
+                half_period();
+                scl_o(dcxo, true);
+                half_period();
+            }
+        }
+
+        if !sda_i(dcxo) {
+            return Err("SDA is stuck low and doesn't get unstuck");
+        }
+        Ok(())
+    }
+
+    pub fn start(dcxo: DCXO) {
+        // Set SCL high then SDA low
+        scl_o(dcxo, true);
+        half_period();
+        sda_oe(dcxo, true);
+        half_period();
+    }
+
+    pub fn stop(dcxo: DCXO) {
+        // First, make sure SCL is low, so that the target releases the SDA line
+        scl_o(dcxo, false);
+        half_period();
+        // Set SCL high then SDA high
+        sda_oe(dcxo, true);
+        scl_o(dcxo, true);
+        half_period();
+        sda_oe(dcxo, false);
+        half_period();
+    }
+
+    pub fn write(dcxo: DCXO, data: u8) -> bool {
+        // MSB first
+        for bit in (0..8).rev() {
+            // Set SCL low and set our bit on SDA
+            scl_o(dcxo, false);
+            sda_oe(dcxo, data & (1 << bit) == 0);
+            half_period();
+            // Set SCL high ; data is shifted on the rising edge of SCL
+            scl_o(dcxo, true);
+            half_period();
+        }
+        // Check ack
+        // Set SCL low, then release SDA so that the I2C target can respond
+        scl_o(dcxo, false);
+        half_period();
+        sda_oe(dcxo, false);
+        // Set SCL high and check for ack
+        scl_o(dcxo, true);
+        half_period();
+        // returns true if acked (I2C target pulled SDA low)
+        !sda_i(dcxo)
+    }
+
+    pub fn read(dcxo: DCXO, ack: bool) -> u8 {
+        // Set SCL low first, otherwise setting SDA as input may cause a transition
+        // on SDA with SCL high which will be interpreted as START/STOP condition.
+        scl_o(dcxo, false);
+        half_period(); // make sure SCL has settled low
+        sda_oe(dcxo, false);
+
+        let mut data: u8 = 0;
+
+        // MSB first
+        for bit in (0..8).rev() {
+            scl_o(dcxo, false);
+            half_period();
+            // Set SCL high and shift data
+            scl_o(dcxo, true);
+            half_period();
+            if sda_i(dcxo) {
+                data |= 1 << bit
+            }
+        }
+        // Send ack
+        // Set SCL low and pull SDA low when acking
+        scl_o(dcxo, false);
+        if ack {
+            sda_oe(dcxo, true)
+        }
+        half_period();
+        // then set SCL high
+        scl_o(dcxo, true);
+        half_period();
+
+        data
+    }
+}
+
+fn write(dcxo: i2c::DCXO, reg: u8, val: u8) -> Result<(), &'static str> {
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, ADDRESS << 1) {
+        return Err("Si549 failed to ack write address");
+    }
+    if !i2c::write(dcxo, reg) {
+        return Err("Si549 failed to ack register");
+    }
+    if !i2c::write(dcxo, val) {
+        return Err("Si549 failed to ack value");
+    }
+    i2c::stop(dcxo);
+    Ok(())
+}
+
+fn read(dcxo: i2c::DCXO, reg: u8) -> Result<u8, &'static str> {
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, ADDRESS << 1) {
+        return Err("Si549 failed to ack write address");
+    }
+    if !i2c::write(dcxo, reg) {
+        return Err("Si549 failed to ack register");
+    }
+    i2c::stop(dcxo);
+
+    i2c::start(dcxo);
+    if !i2c::write(dcxo, (ADDRESS << 1) | 1) {
+        return Err("Si549 failed to ack read address");
+    }
+    let val = i2c::read(dcxo, false);
+    i2c::stop(dcxo);
+    Ok(val)
+}
+
+fn setup(dcxo: i2c::DCXO, config: &DividerConfig) -> Result<(), &'static str> {
+    i2c::init(dcxo)?;
+
+    write(dcxo, 255, 0x00)?; // PAGE
+    write(dcxo, 69, 0x00)?; // Disable FCAL override.
+    write(dcxo, 17, 0x00)?; // Synchronously disable output
+
+    // The Si549 has no ID register, so we check that it responds correctly
+    // by writing values to a RAM-like register and reading them back.
+    for test_value in 0..255 {
+        write(dcxo, 23, test_value)?;
+        let readback = read(dcxo, 23)?;
+        if readback != test_value {
+            return Err("Si549 detection failed");
+        }
+    }
+
+    write(dcxo, 23, config.hsdiv as u8)?;
+    write(dcxo, 24, (config.hsdiv >> 8) as u8 | (config.lsdiv << 4))?;
+    write(dcxo, 26, config.fbdiv as u8)?;
+    write(dcxo, 27, (config.fbdiv >> 8) as u8)?;
+    write(dcxo, 28, (config.fbdiv >> 16) as u8)?;
+    write(dcxo, 29, (config.fbdiv >> 24) as u8)?;
+    write(dcxo, 30, (config.fbdiv >> 32) as u8)?;
+    write(dcxo, 31, (config.fbdiv >> 40) as u8)?;
+
+    write(dcxo, 7, 0x08)?; // Start FCAL
+    clock::spin_us(30_000); // Internal FCAL VCO calibration
+    write(dcxo, 17, 0x01)?; // Synchronously enable output
+
+    Ok(())
+}
+
+pub fn main_setup(settings: &FrequencySetting) -> Result<(), &'static str> {
+    unsafe {
+        csr::wrpll::main_dcxo_bitbang_enable_write(1);
+        csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
+    }
+
+    setup(i2c::DCXO::Main, &settings.main)?;
+
+    // Si549 maximum settling time for large frequency change.
+    clock::spin_us(40_000);
+
+    unsafe {
+        csr::wrpll::main_dcxo_bitbang_enable_write(0);
+    }
+
+    info!("Main Si549 started");
+    Ok(())
+}
+
+pub fn helper_setup(settings: &FrequencySetting) -> Result<(), &'static str> {
+    unsafe {
+        csr::wrpll::helper_reset_write(1);
+        csr::wrpll::helper_dcxo_bitbang_enable_write(1);
+        csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
+    }
+
+    setup(i2c::DCXO::Helper, &settings.helper)?;
+
+    // Si549 maximum settling time for large frequency change.
+    clock::spin_us(40_000);
+
+    unsafe {
+        csr::wrpll::helper_reset_write(0);
+        csr::wrpll::helper_dcxo_bitbang_enable_write(0);
+    }
+    info!("Helper Si549 started");
+    Ok(())
+}
+
+fn set_adpll(dcxo: i2c::DCXO, adpll: i32) -> Result<(), &'static str> {
+    if adpll.abs() > ADPLL_MAX {
+        return Err("adpll is too large");
+    }
+
+    match dcxo {
+        i2c::DCXO::Main => unsafe {
+            if csr::wrpll::main_dcxo_bitbang_enable_read() == 1 {
+                return Err("Main si549 bitbang mode is active when using gateware i2c");
+            }
+
+            while csr::wrpll::main_dcxo_adpll_busy_read() == 1 {}
+            if csr::wrpll::main_dcxo_nack_read() == 1 {
+                return Err("Main si549 failed to ack adpll write");
+            }
+
+            csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
+            csr::wrpll::main_dcxo_adpll_write(adpll as u32);
+
+            csr::wrpll::main_dcxo_adpll_stb_write(1);
+        },
+        i2c::DCXO::Helper => unsafe {
+            if csr::wrpll::helper_dcxo_bitbang_enable_read() == 1 {
+                return Err("Helper si549 bitbang mode is active when using gateware i2c");
+            }
+
+            while csr::wrpll::helper_dcxo_adpll_busy_read() == 1 {}
+            if csr::wrpll::helper_dcxo_nack_read() == 1 {
+                return Err("Helper si549 failed to ack adpll write");
+            }
+
+            csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
+            csr::wrpll::helper_dcxo_adpll_write(adpll as u32);
+
+            csr::wrpll::helper_dcxo_adpll_stb_write(1);
+        },
+    };
+
+    Ok(())
+}
+
+#[cfg(has_wrpll)]
+pub mod wrpll {
+
+    use super::*;
+
+    const BEATING_PERIOD: i32 = 0x8000;
+    const BEATING_HALFPERIOD: i32 = 0x4000;
+    const COUNTER_WIDTH: u32 = 24;
+    const DIV_WIDTH: u32 = 2;
+
+    // y[n] = b0*x[n] + b1*x[n-1] + b2*x[n-2] - a1*y[n-1] - a2*y[n-2]
+    struct FilterParameters {
+        pub b0: i64,
+        pub b1: i64,
+        pub b2: i64,
+        pub a1: i64,
+        pub a2: i64,
+    }
+
+    #[cfg(rtio_frequency = "100.0")]
+    const LPF: FilterParameters = FilterParameters {
+        b0: 10905723400,   // 0.03967479060647884 * 1 << 38
+        b1: 21811446800,   // 0.07934958121295768 * 1 << 38
+        b2: 10905723400,   // 0.03967479060647884 * 1 << 38
+        a1: -381134538612, // -1.3865593741228928 * 1 << 38
+        a2: 149879525269,  // 0.5452585365488082  * 1 << 38
+    };
+
+    #[cfg(rtio_frequency = "125.0")]
+    const LPF: FilterParameters = FilterParameters {
+        b0: 19816511911,   // 0.07209205036273991  * 1 << 38
+        b1: 39633023822,   // 0.14418410072547982  * 1 << 38
+        b2: 19816511911,   // 0.07209205036273991  * 1 << 38
+        a1: -168062510414, // -0.6114078511562919  * 1 << 38
+        a2: -27549348884,  // -0.10022394739274834 * 1 << 38
+    };
+
+    static mut H_ADPLL1: i32 = 0;
+    static mut H_ADPLL2: i32 = 0;
+    static mut PERIOD_ERR1: i32 = 0;
+    static mut PERIOD_ERR2: i32 = 0;
+
+    static mut M_ADPLL1: i32 = 0;
+    static mut M_ADPLL2: i32 = 0;
+    static mut PHASE_ERR1: i32 = 0;
+    static mut PHASE_ERR2: i32 = 0;
+
+    static mut BASE_ADPLL: i32 = 0;
+
+    #[derive(Clone, Copy)]
+    pub enum ISR {
+        RefTag,
+        MainTag,
+    }
+
+    mod tag_collector {
+        use super::*;
+
+        #[cfg(wrpll_ref_clk = "GT_CDR")]
+        static mut TAG_OFFSET: u32 = 23890;
+        #[cfg(wrpll_ref_clk = "SMA_CLKIN")]
+        static mut TAG_OFFSET: u32 = 0;
+        static mut REF_TAG: u32 = 0;
+        static mut REF_TAG_READY: bool = false;
+        static mut MAIN_TAG: u32 = 0;
+        static mut MAIN_TAG_READY: bool = false;
+
+        pub fn reset() {
+            clear_phase_diff_ready();
+            unsafe {
+                REF_TAG = 0;
+                MAIN_TAG = 0;
+            }
+        }
+
+        pub fn clear_phase_diff_ready() {
+            unsafe {
+                REF_TAG_READY = false;
+                MAIN_TAG_READY = false;
+            }
+        }
+
+        pub fn collect_tags(interrupt: ISR) {
+            match interrupt {
+                ISR::RefTag => unsafe {
+                    REF_TAG = csr::wrpll::ref_tag_read();
+                    REF_TAG_READY = true;
+                },
+                ISR::MainTag => unsafe {
+                    MAIN_TAG = csr::wrpll::main_tag_read();
+                    MAIN_TAG_READY = true;
+                },
+            }
+        }
+
+        pub fn phase_diff_ready() -> bool {
+            unsafe { REF_TAG_READY && MAIN_TAG_READY }
+        }
+
+        #[cfg(feature = "calibrate_wrpll_skew")]
+        pub fn set_tag_offset(offset: u32) {
+            unsafe {
+                TAG_OFFSET = offset;
+            }
+        }
+
+        #[cfg(feature = "calibrate_wrpll_skew")]
+        pub fn get_tag_offset() -> u32 {
+            unsafe { TAG_OFFSET }
+        }
+
+        pub fn get_period_error() -> i32 {
+            // n * BEATING_PERIOD - REF_TAG(n) mod BEATING_PERIOD
+            let mut period_error = unsafe {
+                REF_TAG
+                    .overflowing_neg()
+                    .0
+                    .rem_euclid(BEATING_PERIOD as u32) as i32
+            };
+            // mapping tags from [0, 2π] -> [-π, π]
+            if period_error > BEATING_HALFPERIOD {
+                period_error -= BEATING_PERIOD
+            }
+            period_error
+        }
+
+        pub fn get_phase_error() -> i32 {
+            // MAIN_TAG(n) - REF_TAG(n) - TAG_OFFSET mod BEATING_PERIOD
+            let mut phase_error = unsafe {
+                MAIN_TAG
+                    .overflowing_sub(REF_TAG + TAG_OFFSET)
+                    .0
+                    .rem_euclid(BEATING_PERIOD as u32) as i32
+            };
+
+            // mapping tags from [0, 2π] -> [-π, π]
+            if phase_error > BEATING_HALFPERIOD {
+                phase_error -= BEATING_PERIOD
+            }
+            phase_error
+        }
+    }
+
+    fn set_isr(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll::ref_tag_ev_enable_write(val);
+            csr::wrpll::main_tag_ev_enable_write(val);
+        }
+    }
+
+    fn set_base_adpll() -> Result<(), &'static str> {
+        let count2adpll = |error: i32| {
+            ((error as f64 * 1e6) / (0.0001164 * (1 << (COUNTER_WIDTH - DIV_WIDTH)) as f64)) as i32
+        };
+
+        let (ref_count, main_count) = get_freq_counts();
+        unsafe {
+            BASE_ADPLL = count2adpll(ref_count as i32 - main_count as i32);
+            set_adpll(i2c::DCXO::Main, BASE_ADPLL)?;
+            set_adpll(i2c::DCXO::Helper, BASE_ADPLL)?;
+        }
+        Ok(())
+    }
+
+    fn get_freq_counts() -> (u32, u32) {
+        unsafe {
+            csr::wrpll::frequency_counter_update_write(1);
+            while csr::wrpll::frequency_counter_busy_read() == 1 {}
+            #[cfg(wrpll_ref_clk = "GT_CDR")]
+            let ref_count = csr::wrpll::frequency_counter_counter_rtio_rx0_read();
+            #[cfg(wrpll_ref_clk = "SMA_CLKIN")]
+            let ref_count = csr::wrpll::frequency_counter_counter_ref_read();
+            let main_count = csr::wrpll::frequency_counter_counter_sys_read();
+
+            (ref_count, main_count)
+        }
+    }
+
+    fn reset_plls() -> Result<(), &'static str> {
+        unsafe {
+            H_ADPLL1 = 0;
+            H_ADPLL2 = 0;
+            PERIOD_ERR1 = 0;
+            PERIOD_ERR2 = 0;
+            M_ADPLL1 = 0;
+            M_ADPLL2 = 0;
+            PHASE_ERR1 = 0;
+            PHASE_ERR2 = 0;
+        }
+        set_adpll(i2c::DCXO::Main, 0)?;
+        set_adpll(i2c::DCXO::Helper, 0)?;
+        // wait for adpll to transfer and DCXO to settle
+        clock::spin_us(200);
+        Ok(())
+    }
+
+    fn clear_pending(interrupt: ISR) {
+        match interrupt {
+            ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_write(1) },
+            ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_write(1) },
+        };
+    }
+
+    fn is_pending(interrupt: ISR) -> bool {
+        match interrupt {
+            ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_read() == 1 },
+            ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_read() == 1 },
+        }
+    }
+
+    pub fn interrupt_handler() {
+        if is_pending(ISR::RefTag) {
+            tag_collector::collect_tags(ISR::RefTag);
+            clear_pending(ISR::RefTag);
+            helper_pll().expect("failed to run helper DCXO PLL");
+        }
+
+        if is_pending(ISR::MainTag) {
+            tag_collector::collect_tags(ISR::MainTag);
+            clear_pending(ISR::MainTag);
+        }
+
+        if tag_collector::phase_diff_ready() {
+            main_pll().expect("failed to run main DCXO PLL");
+            tag_collector::clear_phase_diff_ready();
+        }
+    }
+
+    fn helper_pll() -> Result<(), &'static str> {
+        let period_err = tag_collector::get_period_error();
+        unsafe {
+            let adpll = (((LPF.b0 * period_err as i64)
+                + (LPF.b1 * PERIOD_ERR1 as i64)
+                + (LPF.b2 * PERIOD_ERR2 as i64)
+                - (LPF.a1 * H_ADPLL1 as i64)
+                - (LPF.a2 * H_ADPLL2 as i64))
+                >> 38) as i32;
+            set_adpll(i2c::DCXO::Helper, BASE_ADPLL + adpll)?;
+            H_ADPLL2 = H_ADPLL1;
+            PERIOD_ERR2 = PERIOD_ERR1;
+            H_ADPLL1 = adpll;
+            PERIOD_ERR1 = period_err;
+        };
+        Ok(())
+    }
+
+    fn main_pll() -> Result<(), &'static str> {
+        let phase_err = tag_collector::get_phase_error();
+        unsafe {
+            let adpll = (((LPF.b0 * phase_err as i64)
+                + (LPF.b1 * PHASE_ERR1 as i64)
+                + (LPF.b2 * PHASE_ERR2 as i64)
+                - (LPF.a1 * M_ADPLL1 as i64)
+                - (LPF.a2 * M_ADPLL2 as i64))
+                >> 38) as i32;
+            set_adpll(i2c::DCXO::Main, BASE_ADPLL + adpll)?;
+            M_ADPLL2 = M_ADPLL1;
+            PHASE_ERR2 = PHASE_ERR1;
+            M_ADPLL1 = adpll;
+            PHASE_ERR1 = phase_err;
+        };
+        Ok(())
+    }
+
+    #[cfg(wrpll_ref_clk = "GT_CDR")]
+    fn test_skew() -> Result<(), &'static str> {
+        // wait for PLL to stabilize
+        clock::spin_us(20_000);
+
+        info!("testing the skew of SYS CLK...");
+        if has_timing_error() {
+            return Err("the skew cannot satisfy setup/hold time constraint of RX synchronizer");
+        }
+        info!("the skew of SYS CLK met the timing constraint");
+        Ok(())
+    }
+
+    #[cfg(wrpll_ref_clk = "GT_CDR")]
+    fn has_timing_error() -> bool {
+        unsafe {
+            csr::wrpll_skewtester::error_write(1);
+        }
+        clock::spin_us(5_000);
+        unsafe { csr::wrpll_skewtester::error_read() == 1 }
+    }
+
+    #[cfg(feature = "calibrate_wrpll_skew")]
+    fn find_edge(target: bool) -> Result<u32, &'static str> {
+        const STEP: u32 = 8;
+        const STABLE_THRESHOLD: u32 = 10;
+
+        enum FSM {
+            Init,
+            WaitEdge,
+            GotEdge,
+        }
+
+        let mut state: FSM = FSM::Init;
+        let mut offset: u32 = tag_collector::get_tag_offset();
+        let mut median_edge: u32 = 0;
+        let mut stable_counter: u32 = 0;
+
+        for _ in 0..(BEATING_PERIOD as u32 / STEP) as usize {
+            tag_collector::set_tag_offset(offset);
+            offset += STEP;
+            // wait for PLL to stabilize
+            clock::spin_us(20_000);
+
+            let error = has_timing_error();
+            // A median edge deglitcher
+            match state {
+                FSM::Init => {
+                    if error != target {
+                        stable_counter += 1;
+                    } else {
+                        stable_counter = 0;
+                    }
+
+                    if stable_counter >= STABLE_THRESHOLD {
+                        state = FSM::WaitEdge;
+                        stable_counter = 0;
+                    }
+                }
+                FSM::WaitEdge => {
+                    if error == target {
+                        state = FSM::GotEdge;
+                        median_edge = offset;
+                    }
+                }
+                FSM::GotEdge => {
+                    if error != target {
+                        median_edge += STEP;
+                        stable_counter = 0;
+                    } else {
+                        stable_counter += 1;
+                    }
+
+                    if stable_counter >= STABLE_THRESHOLD {
+                        return Ok(median_edge);
+                    }
+                }
+            }
+        }
+        return Err("failed to find timing error edge");
+    }
+
+    #[cfg(feature = "calibrate_wrpll_skew")]
+    fn calibrate_skew() -> Result<(), &'static str> {
+        info!("calibrating skew to meet timing constraint...");
+
+        // clear calibrated value
+        tag_collector::set_tag_offset(0);
+        let rising = find_edge(true)? as i32;
+        let falling = find_edge(false)? as i32;
+
+        let width = BEATING_PERIOD - (falling - rising);
+        let result = falling + width / 2;
+        tag_collector::set_tag_offset(result as u32);
+
+        info!(
+            "calibration successful, error zone: {} -> {}, width: {} ({}deg), middle of working region: {}",
+            rising,
+            falling,
+            width,
+            360 * width / BEATING_PERIOD,
+            result,
+        );
+
+        Ok(())
+    }
+
+    pub fn select_recovered_clock(rc: bool) {
+        set_isr(false);
+
+        if rc {
+            tag_collector::reset();
+            reset_plls().expect("failed to reset main and helper PLL");
+
+            // get within capture range
+            set_base_adpll().expect("failed to set base adpll");
+
+            // clear gateware pending flag
+            clear_pending(ISR::RefTag);
+            clear_pending(ISR::MainTag);
+
+            // use nFIQ to avoid IRQ being disabled by mutex lock and mess up PLL
+            set_isr(true);
+            info!("WRPLL interrupt enabled");
+
+            #[cfg(feature = "calibrate_wrpll_skew")]
+            calibrate_skew().expect("failed to set the correct skew");
+
+            #[cfg(wrpll_ref_clk = "GT_CDR")]
+            test_skew().expect("skew test failed");
+        }
+    }
+}
+
+#[cfg(has_wrpll_refclk)]
+pub mod wrpll_refclk {
+    use super::*;
+
+    pub struct MmcmSetting {
+        pub clkout0_reg1: u16,  //0x08
+        pub clkout0_reg2: u16,  //0x09
+        pub clkfbout_reg1: u16, //0x14
+        pub clkfbout_reg2: u16, //0x15
+        pub div_reg: u16,       //0x16
+        pub lock_reg1: u16,     //0x18
+        pub lock_reg2: u16,     //0x19
+        pub lock_reg3: u16,     //0x1A
+        pub power_reg: u16,     //0x28
+        pub filt_reg1: u16,     //0x4E
+        pub filt_reg2: u16,     //0x4F
+    }
+
+    fn one_clock_cycle() {
+        unsafe {
+            csr::wrpll_refclk::mmcm_dclk_write(1);
+            csr::wrpll_refclk::mmcm_dclk_write(0);
+        }
+    }
+
+    fn set_addr(address: u8) {
+        unsafe {
+            csr::wrpll_refclk::mmcm_daddr_write(address);
+        }
+    }
+
+    fn set_data(value: u16) {
+        unsafe {
+            csr::wrpll_refclk::mmcm_din_write(value);
+        }
+    }
+
+    fn set_enable(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_den_write(val);
+        }
+    }
+
+    fn set_write_enable(en: bool) {
+        let val = if en { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_dwen_write(val);
+        }
+    }
+
+    fn get_data() -> u16 {
+        unsafe { csr::wrpll_refclk::mmcm_dout_read() }
+    }
+
+    fn drp_ready() -> bool {
+        unsafe { csr::wrpll_refclk::mmcm_dready_read() == 1 }
+    }
+
+    #[allow(dead_code)]
+    fn read(address: u8) -> u16 {
+        set_addr(address);
+        set_enable(true);
+        // Set DADDR on the mmcm and assert DEN for one clock cycle
+        one_clock_cycle();
+
+        set_enable(false);
+        while !drp_ready() {
+            // keep the clock signal until data is ready
+            one_clock_cycle();
+        }
+        get_data()
+    }
+
+    fn write(address: u8, value: u16) {
+        set_addr(address);
+        set_data(value);
+        set_write_enable(true);
+        set_enable(true);
+        // Set DADDR, DI on the mmcm and assert DWE, DEN for one clock cycle
+        one_clock_cycle();
+
+        set_write_enable(false);
+        set_enable(false);
+        while !drp_ready() {
+            // keep the clock signal until write is finished
+            one_clock_cycle();
+        }
+    }
+
+    fn reset(rst: bool) {
+        let val = if rst { 1 } else { 0 };
+        unsafe {
+            csr::wrpll_refclk::mmcm_reset_write(val)
+        }
+    }
+
+    pub fn setup(settings: MmcmSetting, mmcm_bypass: bool) -> Result<(), &'static str> {
+        unsafe {
+            csr::wrpll_refclk::refclk_reset_write(1);
+        }
+
+        if mmcm_bypass {
+            info!("Bypassing mmcm");
+            unsafe {
+                csr::wrpll_refclk::mmcm_bypass_write(1);
+            }
+        } else {
+            // Based on "DRP State Machine" from XAPP888
+            // hold reset HIGH during mmcm config
+            reset(true);
+            write(0x08, settings.clkout0_reg1);
+            write(0x09, settings.clkout0_reg2);
+            write(0x14, settings.clkfbout_reg1);
+            write(0x15, settings.clkfbout_reg2);
+            write(0x16, settings.div_reg);
+            write(0x18, settings.lock_reg1);
+            write(0x19, settings.lock_reg2);
+            write(0x1A, settings.lock_reg3);
+            write(0x28, settings.power_reg);
+            write(0x4E, settings.filt_reg1);
+            write(0x4F, settings.filt_reg2);
+            reset(false);
+
+            // wait for the mmcm to lock
+            clock::spin_us(100);
+
+            let locked = unsafe { csr::wrpll_refclk::mmcm_locked_read() == 1 };
+            if !locked {
+                return Err("mmcm failed to generate 125MHz ref clock from SMA CLKIN");
+            }
+        }
+
+        unsafe {
+            csr::wrpll_refclk::refclk_reset_write(0);
+        }
+
+        Ok(())
+    }
+}

artiq/firmware/libboard_misoc/io_expander.rs

@@ -27,6 +27,26 @@ impl IoExpander {
         const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 6), (1, 1, 6)];
         const VIRTUAL_LED_MAPPING1: [(u8, u8, u8); 2] = [(2, 0, 6), (3, 1, 6)];
 
+        #[cfg(has_si549)]
+        const IODIR_CLK_SEL: u8 = 0x80; // out
+        #[cfg(has_si5324)]
+        const IODIR_CLK_SEL: u8 = 0x00; // in
+        
+        #[cfg(has_si549)]
+        const CLK_SEL_OUT: u8 = 1 << 7;
+        #[cfg(has_si5324)]
+        const CLK_SEL_OUT: u8 = 0;
+        
+        const IODIR0 : [u8; 2] = [
+            0xFF,
+            0xFF & !IODIR_CLK_SEL
+        ];
+        
+        const OUT_TAR0 : [u8; 2] = [
+            0,
+            CLK_SEL_OUT
+        ];        
+
         // Both expanders on SHARED I2C bus
         let mut io_expander = match index {
             0 => IoExpander {
@@ -34,9 +54,9 @@ impl IoExpander {
                 port: 11,
                 address: 0x40,
                 virtual_led_mapping: &VIRTUAL_LED_MAPPING0,
-                iodir: [0xff; 2],
+                iodir: IODIR0,
                 out_current: [0; 2],
-                out_target: [0; 2],
+                out_target: OUT_TAR0,
                 registers: Registers {
                     iodira: 0x00,
                     iodirb: 0x01,
@@ -153,10 +173,10 @@ impl IoExpander {
         }
         self.update_iodir()?;
 
-        self.out_current[0] = 0x00;
-        self.write(self.registers.gpioa, 0x00)?;
-        self.out_current[1] = 0x00;
-        self.write(self.registers.gpiob, 0x00)?;
+        self.write(self.registers.gpioa, self.out_target[0])?;
+        self.out_current[0] = self.out_target[0];
+        self.write(self.registers.gpiob, self.out_target[1])?;
+        self.out_current[1] = self.out_target[1];
         Ok(())
     }
 

artiq/firmware/libboard_misoc/lib.rs

@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(llvm_asm)]
+#![feature(asm)]
 
 extern crate byteorder;
 #[cfg(feature = "log")]

artiq/firmware/libboard_misoc/riscv32/boot.rs

@@ -2,29 +2,26 @@ use super::{cache, pmp};
 use riscv::register::*;
 
 pub unsafe fn reset() -> ! {
-    llvm_asm!(r#"
-        j          _reset_handler
-         nop
-    "# : : : : "volatile");
-    loop {}
+    asm!("j          _reset_handler",
+         "nop",
+         options(nomem, nostack, noreturn)
+    );
 }
 
 pub unsafe fn jump(addr: usize) -> ! {
     cache::flush_cpu_icache();
-    llvm_asm!(r#"
-        jalr       x0, 0($0)
-         nop
-    "# : : "r"(addr) : : "volatile");
-    loop {}
+    asm!("jalr       x0, 0({0})",
+         "nop",
+         in(reg) addr,
+         options(nomem, nostack, noreturn)
+    );
 }
 
 pub unsafe fn start_user(addr: usize) -> ! {
     pmp::enable_user_memory();
     mstatus::set_mpp(mstatus::MPP::User);
     mepc::write(addr);
-    llvm_asm!(
-        "mret"
-        : : : : "volatile"
+    asm!("mret",
+        options(nomem, nostack, noreturn)
     );
-    unreachable!()
 }

artiq/firmware/libboard_misoc/riscv32/cache.rs

@@ -7,20 +7,24 @@ use mem;
 
 pub fn flush_cpu_icache() {
     unsafe {
-        llvm_asm!(r#"
-            fence.i
-            nop
-            nop
-            nop
-            nop
-            nop
-        "# : : : : "volatile");
+        asm!(
+            "fence.i",
+            "nop",
+            "nop",
+            "nop",
+            "nop",
+            "nop",
+            options(preserves_flags, nostack)
+        );
     }
 }
 
 pub fn flush_cpu_dcache() {
     unsafe {
-        llvm_asm!(".word(0x500F)" : : : : "volatile");
+        asm!(
+            ".word(0x500F)",
+            options(preserves_flags)
+        );
     }
 }
 

artiq/firmware/libboard_misoc/riscv32/irq.rs

@@ -0,0 +1,49 @@
+use riscv::register::{mie, mstatus};
+
+fn vmim_write(val: usize) {
+    unsafe {
+        asm!("csrw {csr}, {rs}", rs = in(reg) val, csr = const 0xBC0);
+    }
+}
+
+fn vmim_read() -> usize {
+    let r: usize;
+    unsafe {
+        asm!("csrr {rd}, {csr}", rd = out(reg) r, csr = const 0xBC0);
+    }
+    r
+}
+
+fn vmip_read() -> usize {
+    let r: usize;
+    unsafe {
+        asm!("csrr {rd}, {csr}", rd = out(reg) r, csr = const 0xFC0);
+    }
+    r
+}
+
+pub fn enable_interrupts() {
+    unsafe {
+        mstatus::set_mie();
+        mie::set_mext();
+    }
+}
+
+pub fn disable_interrupts() {
+    unsafe {
+        mstatus::clear_mie();
+        mie::clear_mext();
+    }
+}
+
+pub fn enable(id: u32) {
+    vmim_write(vmim_read() | (1 << id));
+}
+
+pub fn disable(id: u32) {
+    vmim_write(vmim_read() & !(1 << id));
+}
+
+pub fn is_pending(id: u32) -> bool {
+    (vmip_read() >> id) & 1 == 1
+}

artiq/firmware/libboard_misoc/riscv32/mod.rs

@@ -1,3 +1,4 @@
 pub mod cache;
 pub mod boot;
+pub mod irq;
 pub mod pmp;

artiq/firmware/libeh/Cargo.toml

@@ -10,5 +10,4 @@ path = "lib.rs"
 [dependencies]
 cslice = { version = "0.3" }
 libc = { path = "../libc" }
-unwind = { path = "../libunwind" }
-compiler_builtins = "=0.1.39"    # A dependency of alloc, libeh doesn't need it
+unwind = { path = "../libunwind" }

artiq/firmware/libeh/lib.rs

@@ -1,4 +1,4 @@
-#![feature(lang_items, panic_unwind, libc, unwind_attributes, int_bits_const)]
+#![feature(lang_items, panic_unwind, libc)]
 #![no_std]
 
 extern crate cslice;

artiq/firmware/libproto_artiq/drtioaux_proto.rs

@@ -16,9 +16,9 @@ impl<T> From<IoError<T>> for Error<T> {
 
 // maximum size of arbitrary payloads
 // used by satellite -> master analyzer, subkernel exceptions
-pub const SAT_PAYLOAD_MAX_SIZE: usize  = /*max size*/512 - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
+pub const SAT_PAYLOAD_MAX_SIZE: usize  = /*max size*/1024 - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
 // used by DDMA, subkernel program data (need to provide extra ID and destination)
-pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*destination*/1 - /*ID*/4;
+pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*source*/1 - /*destination*/1 - /*ID*/4;
 
 #[derive(PartialEq, Clone, Copy, Debug)]
 #[repr(u8)]
@@ -106,22 +106,26 @@ pub enum Packet {
     AnalyzerDataRequest { destination: u8 },
     AnalyzerData { last: bool, length: u16, data: [u8; SAT_PAYLOAD_MAX_SIZE]},
 
-    DmaAddTraceRequest { destination: u8, id: u32, status: PayloadStatus, length: u16, trace: [u8; MASTER_PAYLOAD_MAX_SIZE] },
-    DmaAddTraceReply { succeeded: bool },
-    DmaRemoveTraceRequest { destination: u8, id: u32 },
-    DmaRemoveTraceReply { succeeded: bool },
-    DmaPlaybackRequest { destination: u8, id: u32, timestamp: u64 },
-    DmaPlaybackReply { succeeded: bool },
-    DmaPlaybackStatus { destination: u8, id: u32, error: u8, channel: u32, timestamp: u64 },
+    DmaAddTraceRequest { 
+        source: u8, destination: u8, 
+        id: u32, status: PayloadStatus, 
+        length: u16, trace: [u8; MASTER_PAYLOAD_MAX_SIZE] 
+    },
+    DmaAddTraceReply { source: u8, destination: u8, id: u32, succeeded: bool },
+    DmaRemoveTraceRequest { source: u8, destination: u8, id: u32 },
+    DmaRemoveTraceReply { destination: u8, succeeded: bool },
+    DmaPlaybackRequest { source: u8, destination: u8, id: u32, timestamp: u64 },
+    DmaPlaybackReply { destination: u8, succeeded: bool },
+    DmaPlaybackStatus { source: u8, destination: u8, id: u32, error: u8, channel: u32, timestamp: u64 },
 
     SubkernelAddDataRequest { destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
     SubkernelAddDataReply { succeeded: bool },
-    SubkernelLoadRunRequest { destination: u8, id: u32, run: bool },
-    SubkernelLoadRunReply { succeeded: bool },
-    SubkernelFinished { id: u32, with_exception: bool },
+    SubkernelLoadRunRequest { source: u8, destination: u8, id: u32, run: bool },
+    SubkernelLoadRunReply { destination: u8, succeeded: bool },
+    SubkernelFinished { destination: u8, id: u32, with_exception: bool, exception_src: u8 },
     SubkernelExceptionRequest { destination: u8 },
     SubkernelException { last: bool, length: u16, data: [u8; SAT_PAYLOAD_MAX_SIZE] },
-    SubkernelMessage { destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
+    SubkernelMessage { source: u8, destination: u8, id: u32, status: PayloadStatus, length: u16, data: [u8; MASTER_PAYLOAD_MAX_SIZE] },
     SubkernelMessageAck { destination: u8 },
 }
 
@@ -277,7 +281,8 @@ impl Packet {
                 }
             },
 
-            0xb0 => { 
+            0xb0 => {
+                let source = reader.read_u8()?;
                 let destination = reader.read_u8()?;
                 let id = reader.read_u32()?;
                 let status = reader.read_u8()?;
@@ -285,6 +290,7 @@ impl Packet {
                 let mut trace: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 reader.read_exact(&mut trace[0..length as usize])?;
                 Packet::DmaAddTraceRequest {
+                    source: source,
                     destination: destination,
                     id: id,
                     status: PayloadStatus::from(status),
@@ -293,24 +299,32 @@ impl Packet {
                 }
             },
             0xb1 => Packet::DmaAddTraceReply {
+                source: reader.read_u8()?,
+                destination: reader.read_u8()?,
+                id: reader.read_u32()?,
                 succeeded: reader.read_bool()?
             },
             0xb2 => Packet::DmaRemoveTraceRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?
             },
             0xb3 => Packet::DmaRemoveTraceReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xb4 => Packet::DmaPlaybackRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 timestamp: reader.read_u64()?
             },
             0xb5 => Packet::DmaPlaybackReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xb6 => Packet::DmaPlaybackStatus {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 error: reader.read_u8()?,
@@ -337,16 +351,20 @@ impl Packet {
                 succeeded: reader.read_bool()?
             },
             0xc4 => Packet::SubkernelLoadRunRequest {
+                source: reader.read_u8()?,
                 destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 run: reader.read_bool()?
             },
             0xc5 => Packet::SubkernelLoadRunReply {
+                destination: reader.read_u8()?,
                 succeeded: reader.read_bool()?
             },
             0xc8 => Packet::SubkernelFinished {
+                destination: reader.read_u8()?,
                 id: reader.read_u32()?,
                 with_exception: reader.read_bool()?,
+                exception_src: reader.read_u8()?
             },
             0xc9 => Packet::SubkernelExceptionRequest {
                 destination: reader.read_u8()?
@@ -363,6 +381,7 @@ impl Packet {
                 }
             },
             0xcb => {
+                let source = reader.read_u8()?;
                 let destination = reader.read_u8()?;
                 let id = reader.read_u32()?;
                 let status = reader.read_u8()?;
@@ -370,6 +389,7 @@ impl Packet {
                 let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 reader.read_exact(&mut data[0..length as usize])?;
                 Packet::SubkernelMessage {
+                    source: source,
                     destination: destination,
                     id: id,
                     status: PayloadStatus::from(status),
@@ -561,8 +581,9 @@ impl Packet {
                 writer.write_all(&data[0..length as usize])?;
             },
 
-            Packet::DmaAddTraceRequest { destination, id, status, trace, length } => {
+            Packet::DmaAddTraceRequest { source, destination, id, status, trace, length } => {
                 writer.write_u8(0xb0)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(status as u8)?;
@@ -571,31 +592,39 @@ impl Packet {
                 writer.write_u16(length)?;
                 writer.write_all(&trace[0..length as usize])?;
             },
-            Packet::DmaAddTraceReply { succeeded } => {
+            Packet::DmaAddTraceReply { source, destination, id, succeeded } => {
                 writer.write_u8(0xb1)?;
+                writer.write_u8(source)?;
+                writer.write_u8(destination)?;
+                writer.write_u32(id)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaRemoveTraceRequest { destination, id } => {
+            Packet::DmaRemoveTraceRequest { source, destination, id } => {
                 writer.write_u8(0xb2)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
             },
-            Packet::DmaRemoveTraceReply { succeeded } => {
+            Packet::DmaRemoveTraceReply { destination, succeeded } => {
                 writer.write_u8(0xb3)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaPlaybackRequest { destination, id, timestamp } => {
+            Packet::DmaPlaybackRequest { source, destination, id, timestamp } => {
                 writer.write_u8(0xb4)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u64(timestamp)?;
             },
-            Packet::DmaPlaybackReply { succeeded } => {
+            Packet::DmaPlaybackReply { destination, succeeded } => {
                 writer.write_u8(0xb5)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::DmaPlaybackStatus { destination, id, error, channel, timestamp } => {
+            Packet::DmaPlaybackStatus { source, destination, id, error, channel, timestamp } => {
                 writer.write_u8(0xb6)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(error)?;
@@ -615,20 +644,24 @@ impl Packet {
                 writer.write_u8(0xc1)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::SubkernelLoadRunRequest { destination, id, run } => {
+            Packet::SubkernelLoadRunRequest { source, destination, id, run } => {
                 writer.write_u8(0xc4)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_bool(run)?;
             },
-            Packet::SubkernelLoadRunReply { succeeded } => {
+            Packet::SubkernelLoadRunReply { destination, succeeded } => {
                 writer.write_u8(0xc5)?;
+                writer.write_u8(destination)?;
                 writer.write_bool(succeeded)?;
             },
-            Packet::SubkernelFinished { id, with_exception } => {
+            Packet::SubkernelFinished { destination, id, with_exception, exception_src } => {
                 writer.write_u8(0xc8)?;
+                writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_bool(with_exception)?;
+                writer.write_u8(exception_src)?;
             },
             Packet::SubkernelExceptionRequest { destination } => {
                 writer.write_u8(0xc9)?;
@@ -640,8 +673,9 @@ impl Packet {
                 writer.write_u16(length)?;
                 writer.write_all(&data[0..length as usize])?;
             },
-            Packet::SubkernelMessage { destination, id, status, data, length } => {
+            Packet::SubkernelMessage { source, destination, id, status, data, length } => {
                 writer.write_u8(0xcb)?;
+                writer.write_u8(source)?;
                 writer.write_u8(destination)?;
                 writer.write_u32(id)?;
                 writer.write_u8(status as u8)?;
@@ -655,4 +689,36 @@ impl Packet {
         }
         Ok(())
     }
+
+    pub fn routable_destination(&self) -> Option<u8> {
+        // only for packets that could be re-routed, not only forwarded
+        match self {
+            Packet::DmaAddTraceRequest      { destination, .. } => Some(*destination),
+            Packet::DmaAddTraceReply        { destination, .. } => Some(*destination),
+            Packet::DmaRemoveTraceRequest   { destination, .. } => Some(*destination),
+            Packet::DmaRemoveTraceReply     { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackRequest      { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackReply        { destination, .. } => Some(*destination),
+            Packet::SubkernelLoadRunRequest { destination, .. } => Some(*destination),
+            Packet::SubkernelLoadRunReply   { destination, .. } => Some(*destination),
+            Packet::SubkernelMessage        { destination, .. } => Some(*destination),
+            Packet::SubkernelMessageAck     { destination, .. } => Some(*destination),
+            Packet::DmaPlaybackStatus       { destination, .. } => Some(*destination),
+            Packet::SubkernelFinished       { destination, .. } => Some(*destination),
+            _ => None
+        }
+    }
+
+    pub fn expects_response(&self) -> bool {
+        // returns true if the routable packet should elicit a response
+        // e.g. reply, ACK packets end a conversation,
+        // and firmware should not wait for response
+        match self {
+            Packet::DmaAddTraceReply { .. } | Packet::DmaRemoveTraceReply { .. } |
+                Packet::DmaPlaybackReply { .. } | Packet::SubkernelLoadRunReply { .. } |
+                Packet::SubkernelMessageAck { .. } | Packet::DmaPlaybackStatus { .. } |
+                Packet::SubkernelFinished { .. } => false,
+            _ => true
+        }
+    }
 }

artiq/firmware/libproto_artiq/kernel_proto.rs

@@ -103,12 +103,12 @@ pub enum Message<'a> {
     SpiReadReply { succeeded: bool, data: u32 },
     SpiBasicReply { succeeded: bool },
 
-    SubkernelLoadRunRequest { id: u32, run: bool },
+    SubkernelLoadRunRequest { id: u32, destination: u8, run: bool },
     SubkernelLoadRunReply { succeeded: bool },
-    SubkernelAwaitFinishRequest { id: u32, timeout: u64 },
+    SubkernelAwaitFinishRequest { id: u32, timeout: i64 },
     SubkernelAwaitFinishReply { status: SubkernelStatus },
-    SubkernelMsgSend { id: u32, count: u8, tag: &'a [u8], data: *const *const () },
-    SubkernelMsgRecvRequest { id: u32, timeout: u64, tags: &'a [u8] },
+    SubkernelMsgSend { id: u32, destination: Option<u8>, count: u8, tag: &'a [u8], data: *const *const () },
+    SubkernelMsgRecvRequest { id: i32, timeout: i64, tags: &'a [u8] },
     SubkernelMsgRecvReply { status: SubkernelStatus, count: u8 },
 
     Log(fmt::Arguments<'a>),

artiq/firmware/libunwind/src/lib.rs

@@ -3,8 +3,8 @@
 #![feature(link_cfg)]
 #![feature(nll)]
 #![feature(staged_api)]
-#![feature(unwind_attributes)]
 #![feature(static_nobundle)]
+#![feature(c_unwind)]
 #![cfg_attr(not(target_env = "msvc"), feature(libc))]
 
 mod libunwind;

artiq/firmware/libunwind/src/libunwind.rs

@@ -81,9 +81,14 @@ pub type _Unwind_Exception_Cleanup_Fn =
     all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
     link(name = "unwind", kind = "static")
 )]
-extern "C" {
-    #[unwind(allowed)]
+extern "C-unwind" {
     pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !;
+}
+#[cfg_attr(
+    all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
+    link(name = "unwind", kind = "static")
+)]
+extern "C" {
     pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception);
     pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void;
     pub fn _Unwind_GetRegionStart(ctx: *mut _Unwind_Context) -> _Unwind_Ptr;
@@ -230,9 +235,13 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
+    }
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
         pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn,
                                  trace_argument: *mut c_void)
                                  -> _Unwind_Reason_Code;
@@ -242,8 +251,7 @@ if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
     #[cfg_attr(all(feature = "llvm-libunwind",
                    any(target_os = "fuchsia", target_os = "linux")),
                link(name = "unwind", kind = "static"))]
-    extern "C" {
-        #[unwind(allowed)]
+    extern "C-unwind" {
         pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
     }
 

artiq/firmware/riscv32g-unknown-none-elf.json

@@ -21,20 +21,6 @@
   },
   "relro-level": "full",
   "target-family": "unix",
-  "target-pointer-width": "32",
-  "unsupported-abis": [
-    "cdecl",
-    "stdcall",
-    "fastcall",
-    "vectorcall",
-    "thiscall",
-    "aapcs",
-    "win64",
-    "sysv64",
-    "ptx-kernel",
-    "msp430-interrupt",
-    "x86-interrupt",
-    "amdgpu-kernel"
-  ]
+  "target-pointer-width": "32"
 }
   

artiq/firmware/riscv32ima-unknown-none-elf.json

@@ -13,19 +13,5 @@
   "max-atomic-width": 32,
   "panic-strategy": "unwind",
   "relocation-model": "static",
-  "target-pointer-width": "32",
-  "unsupported-abis": [
-    "cdecl",
-    "stdcall",
-    "fastcall",
-    "vectorcall",
-    "thiscall",
-    "aapcs",
-    "win64",
-    "sysv64",
-    "ptx-kernel",
-    "msp430-interrupt",
-    "x86-interrupt",
-    "amdgpu-kernel"
-  ]
+  "target-pointer-width": "32"
 }

artiq/firmware/runtime/Cargo.toml

@@ -17,7 +17,7 @@ failure = { version = "0.1", default-features = false }
 failure_derive = { version = "0.1", default-features = false }
 byteorder = { version = "1.0", default-features = false }
 cslice = { version = "0.3" }
-log = { version = "0.4", default-features = false }
+log = { version = "=0.4.14", default-features = false }
 managed = { version = "^0.7.1", default-features = false, features = ["alloc", "map"] }
 dyld = { path = "../libdyld" }
 eh = { path = "../libeh" }
@@ -37,7 +37,7 @@ features = ["alloc", "medium-ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"
 
 [dependencies.fringe]
 git = "https://git.m-labs.hk/M-Labs/libfringe.git"
-rev = "3ecbe5"
+rev = "53a964"
 default-features = false
 features = ["alloc"]
 

artiq/firmware/runtime/Makefile

@@ -10,8 +10,6 @@ LDFLAGS += \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: runtime.bin runtime.fbi
 
 .PHONY: $(RUSTOUT)/libruntime.a

artiq/firmware/runtime/analyzer.rs

@@ -52,7 +52,7 @@ pub mod remote_analyzer {
         pub data: Vec<u8>
     }
 
-    pub fn get_data(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
+    pub fn get_data(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
         up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
     ) -> Result<RemoteBuffer, drtio::Error> {
             // gets data from satellites and returns consolidated data
@@ -62,7 +62,7 @@ pub mod remote_analyzer {
             let mut remote_total_bytes = 0;
 
             let data_vec = drtio::analyzer_query(
-                io, aux_mutex, routing_table, up_destinations
+                io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, up_destinations
             )?;
             for data in data_vec {
                 remote_total_bytes += data.total_byte_count;
@@ -82,7 +82,8 @@ pub mod remote_analyzer {
 
 
 
-fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex, 
+fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
+    _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex,
     _routing_table: &drtio_routing::RoutingTable,
     _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
 ) -> Result<(), IoError<SchedError>> {
@@ -96,7 +97,7 @@ fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
 
     #[cfg(has_drtio)]
     let remote = remote_analyzer::get_data(
-        _io, _aux_mutex, _routing_table, _up_destinations);
+        _io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, _up_destinations);
     #[cfg(has_drtio)]
     let (header, remote_data) = match remote {
         Ok(remote) => (Header {
@@ -143,7 +144,7 @@ fn worker(stream: &mut TcpStream, _io: &Io, _aux_mutex: &Mutex,
     Ok(())
 }
 
-pub fn thread(io: Io, aux_mutex: &Mutex,
+pub fn thread(io: Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
     routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>,
     up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>) {
     let listener = TcpListener::new(&io, 65535);
@@ -158,7 +159,7 @@ pub fn thread(io: Io, aux_mutex: &Mutex,
         disarm();
 
         let routing_table = routing_table.borrow();
-        match worker(&mut stream, &io, aux_mutex, &routing_table, up_destinations) {
+        match worker(&mut stream, &io, aux_mutex, ddma_mutex, subkernel_mutex, &routing_table, up_destinations) {
             Ok(())   => (),
             Err(err) => error!("analyzer aborted: {}", err)
         }

artiq/firmware/runtime/kern_hwreq.rs

@@ -11,13 +11,15 @@ use board_artiq::spi as local_spi;
 #[cfg(has_drtio)]
 mod remote_i2c {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn start(io: &Io, aux_mutex: &Mutex,
+    pub fn start(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cStartRequest {
                 destination: destination,
                 busno: busno
@@ -37,10 +39,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn restart(io: &Io, aux_mutex: &Mutex, 
+    pub fn restart(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cRestartRequest {
                 destination: destination,
                 busno: busno
@@ -60,10 +63,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn stop(io: &Io, aux_mutex: &Mutex,
+    pub fn stop(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno,
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
             &drtioaux::Packet::I2cStopRequest  {
                 destination: destination,
                 busno: busno
@@ -83,10 +87,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn write(io: &Io, aux_mutex: &Mutex,
+    pub fn write(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, data: u8
     ) -> Result<bool, &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cWriteRequest {
                 destination: destination,
                 busno: busno,
@@ -107,10 +112,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn read(io: &Io, aux_mutex: &Mutex,
+    pub fn read(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, ack: bool
     ) -> Result<u8, &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cReadRequest {
                 destination: destination,
                 busno: busno,
@@ -131,10 +137,11 @@ mod remote_i2c {
         }
     }
 
-    pub fn switch_select(io: &Io, aux_mutex: &Mutex,
+    pub fn switch_select(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, address: u8, mask: u8
     ) -> Result<(), &'static str> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::I2cSwitchSelectRequest {
                 destination: destination,
                 busno: busno,
@@ -160,13 +167,15 @@ mod remote_i2c {
 #[cfg(has_drtio)]
 mod remote_spi {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn set_config(io: &Io, aux_mutex: &Mutex,
+    pub fn set_config(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, flags: u8, length: u8, div: u8, cs: u8
     ) -> Result<(), ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::SpiSetConfigRequest {
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, &drtioaux::Packet::SpiSetConfigRequest {
             destination: destination,
             busno: busno,
             flags: flags,
@@ -189,10 +198,11 @@ mod remote_spi {
         }
     }
 
-    pub fn write(io: &Io, aux_mutex: &Mutex,
+    pub fn write(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         linkno: u8, destination: u8, busno: u8, data: u32
     ) -> Result<(), ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::SpiWriteRequest {
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, &drtioaux::Packet::SpiWriteRequest {
             destination: destination,
             busno: busno,
             data: data
@@ -212,9 +222,10 @@ mod remote_spi {
         }
     }
 
-    pub fn read(io: &Io, aux_mutex: &Mutex, linkno: u8, destination: u8, busno: u8
+    pub fn read(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, linkno: u8, destination: u8, busno: u8
     ) -> Result<u32, ()> {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::SpiReadRequest {
                 destination: destination,
                 busno: busno
@@ -238,7 +249,7 @@ mod remote_spi {
 
 #[cfg(has_drtio)]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
         let destination = ($busno >> 16) as u8;
         let busno = $busno as u8;
         let hop = $routing_table.0[destination as usize][0];
@@ -246,27 +257,27 @@ macro_rules! dispatch {
             $mod_local::$func(busno, $($param, )*)
         } else {
             let linkno = hop - 1;
-            $mod_remote::$func($io, $aux_mutex, linkno, destination, busno, $($param, )*)
+            $mod_remote::$func($io, $aux_mutex, $ddma_mutex, $subkernel_mutex, $routing_table, linkno, destination, busno, $($param, )*)
         }
     }}
 }
 
 #[cfg(not(has_drtio))]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $mod_local:ident, $mod_remote:ident, $routing_table:ident, $busno:expr, $func:ident $(, $param:expr)*) => {{
         let busno = $busno as u8;
         $mod_local::$func(busno, $($param, )*)
     }}
 }
 
-pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex,
-        _routing_table: &drtio_routing::RoutingTable,
+pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable,
         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
         request: &kern::Message) -> Result<bool, Error<SchedError>> {
     match request {
         &kern::RtioInitRequest => {
             info!("resetting RTIO");
-            rtio_mgt::reset(io, aux_mutex);
+            rtio_mgt::reset(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table);
             kern_acknowledge()
         }
 
@@ -282,47 +293,47 @@ pub fn process_kern_hwreq(io: &Io, aux_mutex: &Mutex,
         }
 
         &kern::I2cStartRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, start).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, start).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cRestartRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, restart).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, restart).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cStopRequest { busno } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, stop).is_ok();
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, stop).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
         &kern::I2cWriteRequest { busno, data } => {
-            match dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, write, data) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, write, data) {
                 Ok(ack) => kern_send(io, &kern::I2cWriteReply { succeeded: true, ack: ack }),
                 Err(_) => kern_send(io, &kern::I2cWriteReply { succeeded: false, ack: false })
             }
         }
         &kern::I2cReadRequest { busno, ack } => {
-            match dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno, read, ack) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno, read, ack) {
                 Ok(data) => kern_send(io, &kern::I2cReadReply { succeeded: true, data: data }),
                 Err(_) => kern_send(io, &kern::I2cReadReply { succeeded: false, data: 0xff })
             }
         }
         &kern::I2cSwitchSelectRequest { busno, address, mask } => {
-            let succeeded = dispatch!(io, aux_mutex, local_i2c, remote_i2c, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_i2c, remote_i2c, routing_table, busno,
                 switch_select, address, mask).is_ok();
             kern_send(io, &kern::I2cBasicReply { succeeded: succeeded })
         }
 
         &kern::SpiSetConfigRequest { busno, flags, length, div, cs } => {
-            let succeeded = dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno,
                 set_config, flags, length, div, cs).is_ok();
             kern_send(io, &kern::SpiBasicReply { succeeded: succeeded })
         },
         &kern::SpiWriteRequest { busno, data } => {
-            let succeeded = dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno,
+            let succeeded = dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno,
                 write, data).is_ok();
             kern_send(io, &kern::SpiBasicReply { succeeded: succeeded })
         }
         &kern::SpiReadRequest { busno } => {
-            match dispatch!(io, aux_mutex, local_spi, remote_spi, _routing_table, busno, read) {
+            match dispatch!(io, aux_mutex, ddma_mutex, subkernel_mutex, local_spi, remote_spi, routing_table, busno, read) {
                 Ok(data) => kern_send(io, &kern::SpiReadReply { succeeded: true, data: data }),
                 Err(_) => kern_send(io, &kern::SpiReadReply { succeeded: false, data: 0 })
             }

artiq/firmware/runtime/kernel.rs

@@ -103,7 +103,7 @@ pub mod subkernel {
     pub enum FinishStatus {
         Ok,
         CommLost,
-        Exception
+        Exception(u8) // exception source
     }
 
     #[derive(Debug, PartialEq, Clone, Copy)]
@@ -181,17 +181,17 @@ pub mod subkernel {
         Ok(())
     }
 
-    pub fn upload(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex, 
+    pub fn upload(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
              routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
-        drtio::subkernel_upload(io, aux_mutex, routing_table, id, 
+        drtio::subkernel_upload(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, 
             subkernel.destination, &subkernel.data)?;
         subkernel.state = SubkernelState::Uploaded; 
         Ok(()) 
     }
 
-    pub fn load(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &RoutingTable,
+    pub fn load(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &RoutingTable,
             id: u32, run: bool) -> Result<(), Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
@@ -199,7 +199,7 @@ pub mod subkernel {
             error!("for id: {} expected Uploaded, got: {:?}", id, subkernel.state);
             return Err(Error::IncorrectState);
         }
-        drtio::subkernel_load(io, aux_mutex, routing_table, id, subkernel.destination, run)?;
+        drtio::subkernel_load(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, subkernel.destination, run)?;
         if run {
             subkernel.state = SubkernelState::Running;
         }
@@ -216,7 +216,7 @@ pub mod subkernel {
         Ok(())
     }
 
-    pub fn subkernel_finished(io: &Io, subkernel_mutex: &Mutex, id: u32, with_exception: bool) {
+    pub fn subkernel_finished(io: &Io, subkernel_mutex: &Mutex, id: u32, with_exception: bool, exception_src: u8) {
         // called upon receiving DRTIO SubkernelRunDone
         let _lock = subkernel_mutex.lock(io).unwrap();
         let subkernel = unsafe { SUBKERNELS.get_mut(&id) };
@@ -226,7 +226,7 @@ pub mod subkernel {
             if subkernel.state == SubkernelState::Running {
                 subkernel.state = SubkernelState::Finished {
                     status: match with_exception {
-                    true => FinishStatus::Exception,
+                    true => FinishStatus::Exception(exception_src),
                     false => FinishStatus::Ok,
                     }
                 }
@@ -234,14 +234,14 @@ pub mod subkernel {
         }
     }
 
-    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
+    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
              routing_table: &RoutingTable, destination: u8, up: bool) {
         let _lock = subkernel_mutex.lock(io).unwrap();
         let subkernels_iter = unsafe { SUBKERNELS.iter_mut() };
         for (id, subkernel) in subkernels_iter {
             if subkernel.destination == destination {
                 if up {
-                    match drtio::subkernel_upload(io, aux_mutex, routing_table, *id, destination, &subkernel.data)
+                    match drtio::subkernel_upload(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id, destination, &subkernel.data)
                     {
                         Ok(_) => subkernel.state = SubkernelState::Uploaded,
                         Err(e) => error!("Error adding subkernel on destination {}: {}", destination, e)
@@ -256,7 +256,7 @@ pub mod subkernel {
         }
     }
 
-    pub fn retrieve_finish_status(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
+    pub fn retrieve_finish_status(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &RoutingTable, id: u32) -> Result<SubkernelFinished, Error> {
         let _lock = subkernel_mutex.lock(io)?;
         let mut subkernel = unsafe { SUBKERNELS.get_mut(&id).unwrap() };
@@ -266,9 +266,9 @@ pub mod subkernel {
                 Ok(SubkernelFinished {
                     id: id,
                     comm_lost: status == FinishStatus::CommLost,
-                    exception: if status == FinishStatus::Exception { 
-                        Some(drtio::subkernel_retrieve_exception(io, aux_mutex,
-                            routing_table, subkernel.destination)?) 
+                    exception: if let FinishStatus::Exception(dest) = status { 
+                        Some(drtio::subkernel_retrieve_exception(io, aux_mutex, ddma_mutex, subkernel_mutex,
+                            routing_table, dest)?) 
                     } else { None }
                 })
             },
@@ -278,8 +278,8 @@ pub mod subkernel {
         }
     }
 
-    pub fn await_finish(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
-        routing_table: &RoutingTable, id: u32, timeout: u64) -> Result<SubkernelFinished, Error> {
+    pub fn await_finish(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &RoutingTable, id: u32, timeout: i64) -> Result<SubkernelFinished, Error> {
         {
             let _lock = subkernel_mutex.lock(io)?;
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
@@ -291,7 +291,7 @@ pub mod subkernel {
         }
         let max_time = clock::get_ms() + timeout as u64;
         let _res = io.until(|| {
-            if clock::get_ms() > max_time {
+            if timeout > 0 && clock::get_ms() > max_time {
                 return true;
             }
             if subkernel_mutex.test_lock() {
@@ -305,11 +305,11 @@ pub mod subkernel {
                 _ => false
             }
         })?;
-        if clock::get_ms() > max_time {
+        if timeout > 0 && clock::get_ms() > max_time {
             error!("Remote subkernel finish await timed out");
             return Err(Error::Timeout);
         }
-        retrieve_finish_status(io, aux_mutex, subkernel_mutex, routing_table, id)
+        retrieve_finish_status(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id)
     }
 
     pub struct Message {
@@ -332,8 +332,9 @@ pub mod subkernel {
             Err(_) => return,
         };
         let subkernel = unsafe { SUBKERNELS.get(&id) };
-        if subkernel.is_none() || subkernel.unwrap().state != SubkernelState::Running {
-            // do not add messages for non-existing, non-running or deleted subkernels
+        if subkernel.is_some() && subkernel.unwrap().state != SubkernelState::Running {
+            warn!("received a message for a non-running subkernel #{}", id);
+            // do not add messages for non-running or deleted subkernels
             return
         }
         if status.is_first() {
@@ -359,19 +360,26 @@ pub mod subkernel {
         }
     }
 
-    pub fn message_await(io: &Io, subkernel_mutex: &Mutex, id: u32, timeout: u64
+    pub fn message_await(io: &Io, subkernel_mutex: &Mutex, id: u32, timeout: i64
     ) -> Result<Message, Error> {
-        {
+        let is_subkernel = {
             let _lock = subkernel_mutex.lock(io)?;
+            let is_subkernel = unsafe { SUBKERNELS.get(&id).is_some() };
+            if is_subkernel {
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
-                SubkernelState::Finished { .. } => return Err(Error::SubkernelFinished),
+                SubkernelState::Finished { status: FinishStatus::Ok } |
                 SubkernelState::Running => (),
+                SubkernelState::Finished {
+                    status: FinishStatus::CommLost,
+                    } => return Err(Error::SubkernelFinished),
                 _ => return Err(Error::IncorrectState)
             }
         }
+            is_subkernel
+        };
         let max_time = clock::get_ms() + timeout as u64;
         let message = io.until_ok(|| {
-            if clock::get_ms() > max_time {
+            if timeout > 0 && clock::get_ms() > max_time {
                 return Ok(None);
             }
             if subkernel_mutex.test_lock() {
@@ -384,9 +392,12 @@ pub mod subkernel {
                     return Ok(Some(unsafe { MESSAGE_QUEUE.remove(i) }));
                 }
             }
+            if is_subkernel {
             match unsafe { SUBKERNELS.get(&id).unwrap().state } {
-                SubkernelState::Finished { .. } => return Ok(None),
+                SubkernelState::Finished { status: FinishStatus::CommLost } | 
+                    SubkernelState::Finished { status: FinishStatus::Exception(_) }  => return Ok(None),
                 _ => ()
+                }
             }
             Err(())
         });
@@ -407,20 +418,22 @@ pub mod subkernel {
         }
     }
 
-    pub fn message_send<'a>(io: &Io, aux_mutex: &Mutex, subkernel_mutex: &Mutex,
-        routing_table: &RoutingTable, id: u32, count: u8, tag: &'a [u8], message: *const *const ()
+    pub fn message_send<'a>(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &RoutingTable, id: u32, destination: Option<u8>, count: u8, tag: &'a [u8], message: *const *const ()
     ) -> Result<(), Error> {
         let mut writer = Cursor::new(Vec::new());
-        let _lock = subkernel_mutex.lock(io)?;
-        let destination = unsafe { SUBKERNELS.get(&id).unwrap().destination };
-
         // reuse rpc code for sending arbitrary data
         rpc::send_args(&mut writer, 0, tag, message, false)?;
         // skip service tag, but overwrite first byte with tag count
+        let destination = destination.unwrap_or_else(|| {
+                let _lock = subkernel_mutex.lock(io).unwrap();
+                unsafe { SUBKERNELS.get(&id).unwrap().destination }
+            }
+        );
         let data = &mut writer.into_inner()[3..];
         data[0] = count;
         Ok(drtio::subkernel_send_message(
-            io, aux_mutex, routing_table, id, destination, data
+            io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, destination, data
         )?)
     }
 }

artiq/firmware/runtime/main.rs

@@ -36,12 +36,16 @@ use smoltcp::wire::HardwareAddress;
 use board_misoc::{csr, ident, clock, spiflash, config, net_settings, pmp, boot};
 #[cfg(has_ethmac)]
 use board_misoc::ethmac;
+#[cfg(soc_platform = "kasli")]
+use board_misoc::irq;
 use board_misoc::net_settings::{Ipv4AddrConfig};
 #[cfg(has_drtio)]
 use board_artiq::drtioaux;
 use board_artiq::drtio_routing;
 use board_artiq::{mailbox, rpc_queue};
 use proto_artiq::{mgmt_proto, moninj_proto, rpc_proto, session_proto, kernel_proto};
+#[cfg(has_wrpll)]
+use board_artiq::si549;
 #[cfg(has_drtio_eem)]
 use board_artiq::drtio_eem;
 #[cfg(has_rtio_analyzer)]
@@ -214,15 +218,19 @@ fn startup() {
     #[cfg(any(has_rtio_moninj, has_drtio))]
     {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let drtio_routing_table = drtio_routing_table.clone();
-        io.spawn(4096, move |io| { moninj::thread(io, &aux_mutex, &drtio_routing_table) });
+        io.spawn(4096, move |io| { moninj::thread(io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &drtio_routing_table) });
     }
     #[cfg(has_rtio_analyzer)]
     {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let drtio_routing_table = drtio_routing_table.clone();
         let up_destinations = up_destinations.clone();
-        io.spawn(8192, move |io| { analyzer::thread(io, &aux_mutex, &drtio_routing_table, &up_destinations) });
+        io.spawn(8192, move |io| { analyzer::thread(io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &drtio_routing_table, &up_destinations) });
     }
 
     #[cfg(has_grabber)]
@@ -261,6 +269,11 @@ pub extern fn main() -> i32 {
 
         pmp::init_stack_guard(&_sstack_guard as *const u8 as usize);
 
+        #[cfg(soc_platform = "kasli")]
+        irq::enable_interrupts();
+        #[cfg(has_wrpll)]
+        irq::enable(csr::WRPLL_INTERRUPT);
+
         logger_artiq::BufferLogger::new(&mut LOG_BUFFER[..]).register(||
             boot::start_user(startup as usize)
         );
@@ -295,8 +308,11 @@ pub extern fn exception(regs: *const TrapFrame) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
     match cause {
-        mcause::Trap::Interrupt(source) => {
-            info!("Called interrupt with {:?}", source);
+        mcause::Trap::Interrupt(_source) => {
+            #[cfg(has_wrpll)]
+            if irq::is_pending(csr::WRPLL_INTERRUPT) {
+                si549::wrpll::interrupt_handler();
+            }
         },
 
         mcause::Trap::Exception(mcause::Exception::UserEnvCall) => {

artiq/firmware/runtime/moninj.rs

@@ -50,12 +50,15 @@ mod local_moninj {
 #[cfg(has_drtio)]
 mod remote_moninj {
     use drtioaux;
+    use drtio_routing;
     use rtio_mgt::drtio;
     use sched::{Io, Mutex};
 
-    pub fn read_probe(io: &Io, aux_mutex: &Mutex, linkno: u8,
+    pub fn read_probe(io: &Io, aux_mutex: &Mutex, 
+        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, probe: u8) -> u64 {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::MonitorRequest { 
                 destination: destination,
                 channel: channel,
@@ -69,7 +72,9 @@ mod remote_moninj {
         0
     }
 
-    pub fn inject(io: &Io, aux_mutex: &Mutex, linkno: u8, 
+    pub fn inject(io: &Io, aux_mutex: &Mutex,
+        _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, 
+        _routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, overrd: u8, value: u8) {
         let _lock = aux_mutex.lock(io).unwrap();
         drtioaux::send(linkno, &drtioaux::Packet::InjectionRequest {
@@ -80,9 +85,11 @@ mod remote_moninj {
         }).unwrap();
     }
 
-    pub fn read_injection_status(io: &Io, aux_mutex: &Mutex, linkno: u8,
+    pub fn read_injection_status(io: &Io, aux_mutex: &Mutex,
+        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8,
         destination: u8, channel: u16, overrd: u8) -> u8 {
-        let reply = drtio::aux_transact(io, aux_mutex, linkno, 
+        let reply = drtio::aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::InjectionStatusRequest {
                 destination: destination,
                 channel: channel,
@@ -99,7 +106,7 @@ mod remote_moninj {
 
 #[cfg(has_drtio)]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
         let destination = ($channel >> 16) as u8;
         let channel = $channel as u16;
         let hop = $routing_table.0[destination as usize][0];
@@ -107,21 +114,21 @@ macro_rules! dispatch {
             local_moninj::$func(channel, $($param, )*)
         } else {
             let linkno = hop - 1;
-            remote_moninj::$func($io, $aux_mutex, linkno, destination, channel, $($param, )*)
+            remote_moninj::$func($io, $aux_mutex, $ddma_mutex, $subkernel_mutex, $routing_table, linkno, destination, channel, $($param, )*)
         }
     }}
 }
 
 #[cfg(not(has_drtio))]
 macro_rules! dispatch {
-    ($io:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
+    ($io:ident, $aux_mutex:ident, $ddma_mutex:ident, $subkernel_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
         let channel = $channel as u16;
         local_moninj::$func(channel, $($param, )*)
     }}
 }
 
-fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing::RoutingTable,
-    mut stream: &mut TcpStream) -> Result<(), Error<SchedError>> {
+fn connection_worker(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex,
+    _routing_table: &drtio_routing::RoutingTable, mut stream: &mut TcpStream) -> Result<(), Error<SchedError>> {
     let mut probe_watch_list = BTreeMap::new();
     let mut inject_watch_list = BTreeMap::new();
     let mut next_check = 0;
@@ -150,9 +157,9 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
                     }
                 },
                 HostMessage::Inject { channel, overrd, value } => dispatch!(
-                    io, _aux_mutex, _routing_table, channel, inject, overrd, value),
+                    io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, inject, overrd, value),
                 HostMessage::GetInjectionStatus { channel, overrd } => {
-                    let value = dispatch!(io, _aux_mutex, _routing_table, channel, read_injection_status, overrd);
+                    let value = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_injection_status, overrd);
                     let reply = DeviceMessage::InjectionStatus {
                         channel: channel,
                         overrd: overrd,
@@ -169,7 +176,7 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
 
         if clock::get_ms() > next_check {
             for (&(channel, probe), previous) in probe_watch_list.iter_mut() {
-                let current = dispatch!(io, _aux_mutex, _routing_table, channel, read_probe, probe);
+                let current = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_probe, probe);
                 if previous.is_none() || previous.unwrap() != current {
                     let message = DeviceMessage::MonitorStatus {
                         channel: channel,
@@ -184,7 +191,7 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
                 }
             }
             for (&(channel, overrd), previous) in inject_watch_list.iter_mut() {
-                let current = dispatch!(io, _aux_mutex, _routing_table, channel, read_injection_status, overrd);
+                let current = dispatch!(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, channel, read_injection_status, overrd);
                 if previous.is_none() || previous.unwrap() != current {
                     let message = DeviceMessage::InjectionStatus {
                         channel: channel,
@@ -205,18 +212,20 @@ fn connection_worker(io: &Io, _aux_mutex: &Mutex, _routing_table: &drtio_routing
     }
 }
 
-pub fn thread(io: Io, aux_mutex: &Mutex, routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>) {
+pub fn thread(io: Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>) {
     let listener = TcpListener::new(&io, 2047);
     listener.listen(1383).expect("moninj: cannot listen");
 
     loop {
         let aux_mutex = aux_mutex.clone();
+        let ddma_mutex = ddma_mutex.clone();
+        let subkernel_mutex = subkernel_mutex.clone();
         let routing_table = routing_table.clone();
         let stream = listener.accept().expect("moninj: cannot accept").into_handle();
         io.spawn(16384, move |io| {
             let routing_table = routing_table.borrow();
             let mut stream = TcpStream::from_handle(&io, stream);
-            match connection_worker(&io, &aux_mutex, &routing_table, &mut stream) {
+            match connection_worker(&io, &aux_mutex, &ddma_mutex, &subkernel_mutex, &routing_table, &mut stream) {
                 Ok(()) => {},
                 Err(err) => error!("moninj aborted: {}", err)
             }

artiq/firmware/runtime/rtio_clocking.rs

@@ -1,8 +1,11 @@
 use board_misoc::config;
+#[cfg(has_si5324)]
 use board_artiq::si5324;
+#[cfg(has_si549)]
+use board_artiq::si549;
 use board_misoc::{csr, clock};
 
-#[derive(Debug, PartialEq)]
+#[derive(Debug, PartialEq, Copy, Clone)]
 #[allow(non_camel_case_types)]
 pub enum RtioClock {
     Default,
@@ -89,13 +92,14 @@ pub mod crg {
 
 // Si5324 input to select for locking to an external clock (as opposed to
 // a recovered link clock in DRTIO satellites, which is handled elsewhere).
-#[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
+#[cfg(all(has_si5324, soc_platform = "kasli", hw_rev = "v2.0"))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin1;
-#[cfg(all(soc_platform = "kasli", not(hw_rev = "v2.0")))]
+#[cfg(all(has_si5324, soc_platform = "kasli", not(hw_rev = "v2.0")))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin2;
-#[cfg(all(soc_platform = "kc705"))]
+#[cfg(all(has_si5324, soc_platform = "kc705"))]
 const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin2;
 
+#[cfg(has_si5324)]
 fn setup_si5324_pll(cfg: RtioClock) {
     let (si5324_settings, si5324_ref_input) = match cfg {
         RtioClock::Ext0_Synth0_10to125 => { // 125 MHz output from 10 MHz CLKINx reference, 504 Hz BW
@@ -214,7 +218,7 @@ fn setup_si5324_pll(cfg: RtioClock) {
     si5324::setup(&si5324_settings, si5324_ref_input).expect("cannot initialize Si5324");
 }
 
-fn setup_si5324(clock_cfg: RtioClock) {
+fn sysclk_setup(clock_cfg: RtioClock) {
     let switched = unsafe {
         csr::crg::switch_done_read()
     };
@@ -222,6 +226,8 @@ fn setup_si5324(clock_cfg: RtioClock) {
         info!("Clocking has already been set up.");
         return;
     }
+
+    #[cfg(has_si5324)]
     match clock_cfg {
         RtioClock::Ext0_Bypass => {
             info!("using external RTIO clock with PLL bypass");
@@ -230,7 +236,10 @@ fn setup_si5324(clock_cfg: RtioClock) {
         _ => setup_si5324_pll(clock_cfg),
     }
 
-    // switch sysclk source to si5324
+    #[cfg(has_si549)]
+    si549::main_setup(&get_si549_setting(clock_cfg)).expect("cannot initialize main Si549");
+
+    // switch sysclk source
     #[cfg(not(has_drtio))]
     {
         info!("Switching sys clock, rebooting...");
@@ -244,9 +253,153 @@ fn setup_si5324(clock_cfg: RtioClock) {
 }
 
 
+#[cfg(all(has_si549, has_wrpll))]
+fn wrpll_setup(clk: RtioClock, si549_settings: &si549::FrequencySetting) {
+    // register values are directly copied from preconfigured mmcm
+    let (mmcm_setting, mmcm_bypass) = match clk {
+        RtioClock::Ext0_Synth0_10to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 62.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 5
+                clkout0_reg1: 0x1083,
+                clkout0_reg2: 0x0080,
+                clkfbout_reg1: 0x179e,
+                clkfbout_reg2: 0x4c00,
+                div_reg: 0x1041,
+                lock_reg1: 0x00fa,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x1008,
+                filt_reg2: 0x8800,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_80to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 15.625, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x11c7,
+                clkfbout_reg2: 0x5880,
+                div_reg: 0x1041,
+                lock_reg1: 0x028a,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x9908,
+                filt_reg2: 0x8100,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_100to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 12.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x1145,
+                clkfbout_reg2: 0x4c00,
+                div_reg: 0x1041,
+                lock_reg1: 0x0339,
+                lock_reg2: 0x7c01,
+                lock_reg3: 0xffe9,
+                power_reg: 0x9900,
+                filt_reg1: 0x9108,
+                filt_reg2: 0x0100,
+            },
+            false,
+        ),
+        RtioClock::Ext0_Synth0_125to125 => (
+            si549::wrpll_refclk::MmcmSetting {
+                // CLKFBOUT_MULT = 10, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
+                clkout0_reg1: 0x1145,
+                clkout0_reg2: 0x0000,
+                clkfbout_reg1: 0x1145,
+                clkfbout_reg2: 0x0000,
+                div_reg: 0x1041,
+                lock_reg1: 0x03e8,
+                lock_reg2: 0x7001,
+                lock_reg3: 0xf3e9,
+                power_reg: 0x0100,
+                filt_reg1: 0x9908,
+                filt_reg2: 0x1100,
+            },
+            true,
+        ),
+        _ => unreachable!(),
+    };
+
+    si549::helper_setup(&si549_settings).expect("cannot initialize helper Si549");
+    si549::wrpll_refclk::setup(mmcm_setting, mmcm_bypass).expect("cannot initialize ref clk for wrpll");
+    si549::wrpll::select_recovered_clock(true);
+}
+
+#[cfg(has_si549)]
+fn get_si549_setting(clk: RtioClock) -> si549::FrequencySetting {
+    match clk {
+        RtioClock::Ext0_Synth0_10to125 => {
+            info!("using 10MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_80to125 => {
+            info!("using 80MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_100to125 => {
+            info!("using 100MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Ext0_Synth0_125to125 => {
+            info!("using 125MHz reference to make 125MHz RTIO clock with WRPLL");
+        }
+        RtioClock::Int_100 => {
+            info!("using internal 100MHz RTIO clock");
+        }
+        RtioClock::Int_125 => {
+            info!("using internal 125MHz RTIO clock");
+        }
+        _ => {
+            warn!(
+                "rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.",
+                clk
+            );
+        }
+    };
+
+    match clk {
+        RtioClock::Int_100 => {
+            si549::FrequencySetting {
+                main: si549::DividerConfig {
+                    hsdiv: 0x06C,
+                    lsdiv: 0,
+                    fbdiv: 0x046C5F49797,
+                },
+                helper: si549::DividerConfig {
+                    // 100MHz*32767/32768
+                    hsdiv: 0x06C,
+                    lsdiv: 0,
+                    fbdiv: 0x046C5670BBD,
+                },
+            }
+        }
+        _ => {
+            // Everything else use 125MHz
+            si549::FrequencySetting {
+                main: si549::DividerConfig {
+                    hsdiv: 0x058,
+                    lsdiv: 0,
+                    fbdiv: 0x04815791F25,
+                },
+                helper: si549::DividerConfig {
+                    // 125MHz*32767/32768
+                    hsdiv: 0x058,
+                    lsdiv: 0,
+                    fbdiv: 0x04814E8F442,
+                },
+            }
+        }
+    }
+}
+
 pub fn init() {
     let clock_cfg = get_rtio_clock_cfg();
-    setup_si5324(clock_cfg);
+    sysclk_setup(clock_cfg);
 
     #[cfg(has_drtio)]
     {
@@ -282,4 +435,18 @@ pub fn init() {
             error!("RTIO clock failed");
         }
     }
+
+    #[cfg(all(has_si549, has_wrpll))]
+    {
+        // SYS CLK switch will reset CSRs that are used by WRPLL
+        match clock_cfg {
+            RtioClock::Ext0_Synth0_10to125
+            | RtioClock::Ext0_Synth0_80to125
+            | RtioClock::Ext0_Synth0_100to125
+            | RtioClock::Ext0_Synth0_125to125 => {
+                wrpll_setup(clock_cfg, &get_si549_setting(clock_cfg));
+            }
+            _ => {}
+        }
+    }
 }

artiq/firmware/runtime/rtio_dma.rs

@@ -120,12 +120,12 @@ pub mod remote_dma {
         Ok(playback_state)
     }
 
-    pub fn erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = ddma_mutex.lock(io)?;
         let destinations = unsafe { TRACES.get(&id).unwrap() };
         for destination in destinations.keys() {
-            match drtio::ddma_send_erase(io, aux_mutex, routing_table, id, *destination) {
+            match drtio::ddma_send_erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination) {
                 Ok(_) => (),
                 Err(e) => error!("Error erasing trace on DMA: {}", e)
             } 
@@ -134,18 +134,18 @@ pub mod remote_dma {
         Ok(())
     }
 
-    pub fn upload_traces(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn upload_traces(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32) -> Result<(), Error> {
         let _lock = ddma_mutex.lock(io)?;
         let traces = unsafe { TRACES.get_mut(&id).unwrap() };
         for (destination, mut trace) in traces {
-            drtio::ddma_upload_trace(io, aux_mutex, routing_table, id, *destination, trace.get_trace())?;
+            drtio::ddma_upload_trace(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination, trace.get_trace())?;
             trace.state = RemoteState::Loaded;
         }
         Ok(())
     }
 
-    pub fn playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
             routing_table: &RoutingTable, id: u32, timestamp: u64) -> Result<(), Error>{
         // triggers playback on satellites
         let destinations = unsafe { 
@@ -161,16 +161,16 @@ pub mod remote_dma {
                     return Err(Error::IncorrectState);
                 }
             }
-            drtio::ddma_send_playback(io, aux_mutex, routing_table, id, *destination, timestamp)?;
+            drtio::ddma_send_playback(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, *destination, timestamp)?;
         }
         Ok(())
     }
 
     pub fn playback_done(io: &Io, ddma_mutex: &Mutex, 
-            id: u32, destination: u8, error: u8, channel: u32, timestamp: u64) {
+            id: u32, source: u8, error: u8, channel: u32, timestamp: u64) {
         // called upon receiving PlaybackDone aux packet
         let _lock = ddma_mutex.lock(io).unwrap();
-        let mut trace = unsafe { TRACES.get_mut(&id).unwrap().get_mut(&destination).unwrap() };
+        let mut trace = unsafe { TRACES.get_mut(&id).unwrap().get_mut(&source).unwrap() };
         trace.state = RemoteState::PlaybackEnded {
             error: error, 
             channel: channel, 
@@ -178,7 +178,7 @@ pub mod remote_dma {
         };
     }
 
-    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex,
+    pub fn destination_changed(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
         routing_table: &RoutingTable, destination: u8, up: bool) {
         // update state of the destination, resend traces if it's up
         let _lock = ddma_mutex.lock(io).unwrap();
@@ -186,7 +186,7 @@ pub mod remote_dma {
         for (id, dest_traces) in traces_iter {
             if let Some(trace) = dest_traces.get_mut(&destination) {
                 if up {
-                    match drtio::ddma_upload_trace(io, aux_mutex, routing_table, *id, destination, trace.get_trace())
+                    match drtio::ddma_upload_trace(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id, destination, trace.get_trace())
                     {
                         Ok(_) => trace.state = RemoteState::Loaded,
                         Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e)

artiq/firmware/runtime/rtio_mgt.rs

@@ -65,7 +65,7 @@ pub mod drtio {
         let up_destinations = up_destinations.clone();
         let ddma_mutex = ddma_mutex.clone();
         let subkernel_mutex = subkernel_mutex.clone();
-        io.spawn(8192, move |io| {
+        io.spawn(16384, move |io| {
             let routing_table = routing_table.borrow();
             link_thread(io, &aux_mutex, &routing_table, &up_destinations, &ddma_mutex, &subkernel_mutex);
         });
@@ -96,36 +96,76 @@ pub mod drtio {
         }
     }
 
-    fn process_async_packets(io: &Io, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, linkno: u8,
-            packet: drtioaux::Packet) -> Option<drtioaux::Packet> {
-        // returns None if an async packet has been consumed
+    fn process_async_packets(io: &Io, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+            routing_table: &drtio_routing::RoutingTable, linkno: u8, packet: &drtioaux::Packet
+    ) -> bool {
         match packet {
-            drtioaux::Packet::DmaPlaybackStatus { id, destination, error, channel, timestamp } => {
-                remote_dma::playback_done(io, ddma_mutex, id, destination, error, channel, timestamp);
-                None
+            // packets to be consumed locally
+            drtioaux::Packet::DmaPlaybackStatus { id, source, destination: 0, error, channel, timestamp } => {
+                remote_dma::playback_done(io, ddma_mutex, *id, *source, *error, *channel, *timestamp);
+                true
             },
-            drtioaux::Packet::SubkernelFinished { id, with_exception } => {
-                subkernel::subkernel_finished(io, subkernel_mutex, id, with_exception);
-                None
+            drtioaux::Packet::SubkernelFinished { id, destination: 0, with_exception, exception_src } => {
+                subkernel::subkernel_finished(io, subkernel_mutex, *id, *with_exception, *exception_src);
+                true
             },
-            drtioaux::Packet::SubkernelMessage { id, destination: from, status, length, data } => {
-                subkernel::message_handle_incoming(io, subkernel_mutex, id, status, length as usize, &data);
+            drtioaux::Packet::SubkernelMessage { id, source: from, destination: 0, status, length, data } => {
+                subkernel::message_handle_incoming(io, subkernel_mutex, *id, *status, *length as usize, data);
                 // acknowledge receiving part of the message
                 drtioaux::send(linkno, 
-                    &drtioaux::Packet::SubkernelMessageAck { destination: from }
+                    &drtioaux::Packet::SubkernelMessageAck { destination: *from }
                 ).unwrap();
-                None
+                true
+            },
+            // (potentially) routable packets
+            drtioaux::Packet::DmaAddTraceRequest      { destination, .. } |
+                drtioaux::Packet::DmaAddTraceReply        { destination, .. } |
+                drtioaux::Packet::DmaRemoveTraceRequest   { destination, .. } |
+                drtioaux::Packet::DmaRemoveTraceReply     { destination, .. } |
+                drtioaux::Packet::DmaPlaybackRequest      { destination, .. } |
+                drtioaux::Packet::DmaPlaybackReply        { destination, .. } |
+                drtioaux::Packet::SubkernelLoadRunRequest { destination, .. } |
+                drtioaux::Packet::SubkernelLoadRunReply   { destination, .. } |
+                drtioaux::Packet::SubkernelMessage        { destination, .. } |
+                drtioaux::Packet::SubkernelMessageAck     { destination, .. } |
+                drtioaux::Packet::DmaPlaybackStatus       { destination, .. } |
+                drtioaux::Packet::SubkernelFinished       { destination, .. } => {
+                if *destination == 0 {
+                    false
+                } else {
+                    let dest_link = routing_table.0[*destination as usize][0] - 1;
+                    if dest_link == linkno {
+                        warn!("[LINK#{}] Re-routed packet would return to the same link, dropping: {:?}", linkno, packet);
+                    } else {
+                        drtioaux::send(dest_link, &packet).unwrap();
+                    }
+                    true
+                }
             }
-            other => Some(other)
+            _ => false
         }
     }
 
-    pub fn aux_transact(io: &Io, aux_mutex: &Mutex, linkno: u8, request: &drtioaux::Packet
+    pub fn aux_transact(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, linkno: u8, request: &drtioaux::Packet
     ) -> Result<drtioaux::Packet, Error> {
         let _lock = aux_mutex.lock(io)?;
         drtioaux::send(linkno, request).unwrap();
-        let reply = recv_aux_timeout(io, linkno, 200)?;
-        Ok(reply)
+        loop {
+            let reply = recv_aux_timeout(io, linkno, 200)?;
+            if !process_async_packets(io, ddma_mutex, subkernel_mutex, routing_table, linkno, &reply) {
+                // returns none if it was an async packet
+                return Ok(reply);
+            }
+        }
+    }
+
+    fn setup_transact(io: &Io, aux_mutex: &Mutex, linkno: u8, request: &drtioaux::Packet) -> Result<drtioaux::Packet, Error> {
+        /* shorter aux_transact for setup purposes, no checking for async packets,
+           as they should not be generated yet */
+        let _lock = aux_mutex.lock(io)?;
+        drtioaux::send(linkno, request).unwrap();
+        recv_aux_timeout(io, linkno, 200)
     }
 
     pub fn clear_buffers(io: &Io, aux_mutex: &Mutex) {
@@ -148,7 +188,7 @@ pub mod drtio {
             if count > 100 {
                 return 0;
             }
-            let reply = aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::EchoRequest);
+            let reply = setup_transact(io, aux_mutex, linkno, &drtioaux::Packet::EchoRequest);
             match reply {
                 Ok(drtioaux::Packet::EchoReply) => {
                     // make sure receive buffer is drained
@@ -187,7 +227,7 @@ pub mod drtio {
     fn load_routing_table(io: &Io, aux_mutex: &Mutex,
         linkno: u8, routing_table: &drtio_routing::RoutingTable) -> Result<(), Error> {
         for i in 0..drtio_routing::DEST_COUNT {
-            let reply = aux_transact(io, aux_mutex, linkno, &drtioaux::Packet::RoutingSetPath {
+            let reply = setup_transact(io, aux_mutex, linkno, &drtioaux::Packet::RoutingSetPath {
                 destination: i as u8,
                 hops: routing_table.0[i]
             })?;
@@ -200,7 +240,7 @@ pub mod drtio {
 
     fn set_rank(io: &Io, aux_mutex: &Mutex,
         linkno: u8, rank: u8) -> Result<(), Error> {
-        let reply = aux_transact(io, aux_mutex, linkno,
+        let reply = setup_transact(io, aux_mutex, linkno,
             &drtioaux::Packet::RoutingSetRank {
                 rank: rank
             })?;
@@ -223,16 +263,19 @@ pub mod drtio {
         }
     }
 
-    fn process_unsolicited_aux(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, linkno: u8) {
+    fn process_unsolicited_aux(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+        routing_table: &drtio_routing::RoutingTable, linkno: u8) {
         let _lock = aux_mutex.lock(io).unwrap();
-        match drtioaux::recv(linkno) {
-            Ok(Some(packet)) => {
-                if let Some(packet) = process_async_packets(io, ddma_mutex, subkernel_mutex, linkno, packet) {
-                    warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet);
-                }
+        loop {
+            match drtioaux::recv(linkno) {
+                Ok(Some(packet)) => {
+                    if !process_async_packets(&io, ddma_mutex, subkernel_mutex, routing_table, linkno, &packet) {
+                        warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet);
+                    }
+                },
+                Ok(None) => return,
+                Err(_) => { warn!("[LINK#{}] aux packet error", linkno); return }
             }
-            Ok(None) => (),
-            Err(_) => warn!("[LINK#{}] aux packet error", linkno)
         }
     }
 
@@ -293,53 +336,46 @@ pub mod drtio {
                 let linkno = hop - 1;
                 if destination_up(up_destinations, destination) {
                     if up_links[linkno as usize] {
-                        loop {
-                            let reply = aux_transact(io, aux_mutex, linkno, 
-                                &drtioaux::Packet::DestinationStatusRequest {
-                                    destination: destination
-                                });
-                            if let Ok(reply) = reply {
-                                let reply = process_async_packets(io, ddma_mutex, subkernel_mutex, linkno, reply);
-                                match reply {
-                                    Some(drtioaux::Packet::DestinationDownReply) => {
-                                        destination_set_up(routing_table, up_destinations, destination, false);
-                                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, false);
-                                        subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, false);
-                                    }
-                                    Some(drtioaux::Packet::DestinationOkReply) => (),
-                                    Some(drtioaux::Packet::DestinationSequenceErrorReply { channel }) => {
-                                        error!("[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
-                                        unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
-                                    }
-                                    Some(drtioaux::Packet::DestinationCollisionReply { channel }) => {
-                                        error!("[DEST#{}] RTIO collision involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
-                                        unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
-                                    }
-                                    Some(drtioaux::Packet::DestinationBusyReply { channel }) => {
-                                        error!("[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
-                                        unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
-                                    }
-                                    Some(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
-                                    None => { 
-                                        // continue asking until we get Destination...Reply or error out
-                                        // wait a bit not to overwhelm the receiver causing gateway errors
-                                        io.sleep(10).unwrap(); 
-                                        continue;
-                                    }
+                        let reply = aux_transact(io, aux_mutex,
+                             ddma_mutex, subkernel_mutex, routing_table, linkno,
+                            &drtioaux::Packet::DestinationStatusRequest {
+                                destination: destination
+                            });
+                        if let Ok(reply) = reply {
+                            match reply {
+                                drtioaux::Packet::DestinationDownReply => {
+                                    destination_set_up(routing_table, up_destinations, destination, false);
+                                    remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
+                                    subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
                                 }
-                            } else { 
-                                error!("[DEST#{}] communication failed ({:?})", destination, reply.unwrap_err()); 
+                                drtioaux::Packet::DestinationOkReply => (),
+                                drtioaux::Packet::DestinationSequenceErrorReply { channel } => {
+                                    error!("[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
+                                    unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
+                                }
+                                drtioaux::Packet::DestinationCollisionReply { channel } => {
+                                    error!("[DEST#{}] RTIO collision involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
+                                    unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
+                                }
+                                drtioaux::Packet::DestinationBusyReply { channel } => {
+                                    error!("[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}", destination, channel, resolve_channel_name(channel as u32));
+                                    unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
+                                }
+                                packet => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
+                                
                             }
-                            break;
+                        } else { 
+                            error!("[DEST#{}] communication failed ({:?})", destination, reply.unwrap_err()); 
                         }
                     } else {
                         destination_set_up(routing_table, up_destinations, destination, false);
-                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, false);
-                        subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, false);
+                        remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
+                        subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, false);
                     }
                 } else {
                     if up_links[linkno as usize] {
-                        let reply = aux_transact(io, aux_mutex, linkno, 
+                        let reply = aux_transact(io, aux_mutex, ddma_mutex,
+                            subkernel_mutex, routing_table, linkno,
                             &drtioaux::Packet::DestinationStatusRequest {
                                 destination: destination
                             });
@@ -348,8 +384,8 @@ pub mod drtio {
                             Ok(drtioaux::Packet::DestinationOkReply) => {
                                 destination_set_up(routing_table, up_destinations, destination, true);
                                 init_buffer_space(destination as u8, linkno);
-                                remote_dma::destination_changed(io, aux_mutex, ddma_mutex, routing_table, destination, true);
-                                subkernel::destination_changed(io, aux_mutex, subkernel_mutex, routing_table, destination, true);
+                                remote_dma::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, true);
+                                subkernel::destination_changed(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, destination, true);
                             },
                             Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
                             Err(e) => error!("[DEST#{}] communication failed ({:?})", destination, e)
@@ -371,7 +407,7 @@ pub mod drtio {
                 if up_links[linkno as usize] {
                     /* link was previously up */
                     if link_rx_up(linkno) {
-                        process_unsolicited_aux(&io, aux_mutex, ddma_mutex, subkernel_mutex, linkno);
+                        process_unsolicited_aux(&io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno);
                         process_local_errors(linkno);
                     } else {
                         info!("[LINK#{}] link is down", linkno);
@@ -406,7 +442,7 @@ pub mod drtio {
         }
     }
 
-    pub fn reset(io: &Io, aux_mutex: &Mutex) {
+    pub fn reset(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable) {
         for linkno in 0..csr::DRTIO.len() {
             unsafe {
                 (csr::DRTIO[linkno].reset_write)(1);
@@ -422,7 +458,7 @@ pub mod drtio {
         for linkno in 0..csr::DRTIO.len() {
             let linkno = linkno as u8;
             if link_rx_up(linkno) {
-                let reply = aux_transact(io, aux_mutex, linkno,
+                let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
                     &drtioaux::Packet::ResetRequest);
                 match reply {
                     Ok(drtioaux::Packet::ResetAck) => (),
@@ -449,52 +485,52 @@ pub mod drtio {
             Ok(())
         }
 
-    pub fn ddma_upload_trace(io: &Io, aux_mutex: &Mutex,
-            routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, trace: &[u8]) -> Result<(), Error> {
+    pub fn ddma_upload_trace(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, trace: &[u8]
+    ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(trace, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno,
                 &drtioaux::Packet::DmaAddTraceRequest {
-                    id: id, destination: destination, status: status, length: len as u16, trace: *slice})?;
+                    id: id, source: 0, destination: destination, status: status, length: len as u16, trace: *slice})?;
             match reply {
-                drtioaux::Packet::DmaAddTraceReply { succeeded: true } => Ok(()),
-                drtioaux::Packet::DmaAddTraceReply { succeeded: false } => Err(Error::DmaAddTraceFail(destination)),
+                drtioaux::Packet::DmaAddTraceReply { destination: 0, succeeded: true, .. } => Ok(()),
+                drtioaux::Packet::DmaAddTraceReply { destination: 0, succeeded: false, .. } => Err(Error::DmaAddTraceFail(destination)),
                 packet => Err(Error::UnexpectedPacket(packet)),
             }
         })
     }
 
-    pub fn ddma_send_erase(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable, 
-            id: u32, destination: u8) -> Result<(), Error> {
+    pub fn ddma_send_erase(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::DmaRemoveTraceRequest { id: id, destination: destination })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::DmaRemoveTraceRequest { id: id, source: 0, destination: destination })?;
         match reply {
-            drtioaux::Packet::DmaRemoveTraceReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::DmaRemoveTraceReply { succeeded: false } => Err(Error::DmaEraseFail(destination)),
+            drtioaux::Packet::DmaRemoveTraceReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::DmaRemoveTraceReply { destination: 0, succeeded: false } => Err(Error::DmaEraseFail(destination)),
             packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
-    pub fn ddma_send_playback(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, timestamp: u64) -> Result<(), Error> {
+    pub fn ddma_send_playback(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, timestamp: u64) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::DmaPlaybackRequest{ id: id, destination: destination, timestamp: timestamp })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::DmaPlaybackRequest{ id: id, source: 0, destination: destination, timestamp: timestamp })?;
         match reply {
-            drtioaux::Packet::DmaPlaybackReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::DmaPlaybackReply { succeeded: false } =>
+            drtioaux::Packet::DmaPlaybackReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::DmaPlaybackReply { destination: 0, succeeded: false } =>
                     Err(Error::DmaPlaybackFail(destination)),
             packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
     #[cfg(has_rtio_analyzer)]
-    fn analyzer_get_data(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-        destination: u8) -> Result<RemoteBuffer, Error> {
+    fn analyzer_get_data(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, destination: u8) -> Result<RemoteBuffer, Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
             &drtioaux::Packet::AnalyzerHeaderRequest { destination: destination })?;
         let (sent, total, overflow) = match reply {
             drtioaux::Packet::AnalyzerHeader { sent_bytes, total_byte_count, overflow_occurred } => 
@@ -506,7 +542,7 @@ pub mod drtio {
         if sent > 0 {
             let mut last_packet = false;
             while !last_packet {
-                let reply = aux_transact(io, aux_mutex, linkno, 
+                let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                     &drtioaux::Packet::AnalyzerDataRequest { destination: destination })?;
                 match reply {
                     drtioaux::Packet::AnalyzerData { last, length, data } => { 
@@ -527,23 +563,23 @@ pub mod drtio {
     }
 
     #[cfg(has_rtio_analyzer)]
-    pub fn analyzer_query(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-        up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
+    pub fn analyzer_query(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+        routing_table: &drtio_routing::RoutingTable, up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>
     ) -> Result<Vec<RemoteBuffer>, Error> {
         let mut remote_buffers: Vec<RemoteBuffer> = Vec::new();
         for i in 1..drtio_routing::DEST_COUNT {
             if destination_up(up_destinations, i as u8) {
-                remote_buffers.push(analyzer_get_data(io, aux_mutex, routing_table, i as u8)?);
+                remote_buffers.push(analyzer_get_data(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, i as u8)?);
             }
         }
         Ok(remote_buffers)
     }
 
-    pub fn subkernel_upload(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, data: &Vec<u8>) -> Result<(), Error> {
+    pub fn subkernel_upload(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, data: &Vec<u8>) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(data, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelAddDataRequest {
                     id: id, destination: destination, status: status, length: len as u16, data: *slice})?;
             match reply {
@@ -555,26 +591,27 @@ pub mod drtio {
         })
     }
 
-    pub fn subkernel_load(io: &Io, aux_mutex: &Mutex, routing_table: &drtio_routing::RoutingTable,
-            id: u32, destination: u8, run: bool) -> Result<(), Error> {
+    pub fn subkernel_load(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex, 
+            routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, run: bool
+        ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
-        let reply = aux_transact(io, aux_mutex, linkno, 
-            &drtioaux::Packet::SubkernelLoadRunRequest{ id: id, destination: destination, run: run })?;
+        let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
+            &drtioaux::Packet::SubkernelLoadRunRequest{ id: id, source: 0, destination: destination, run: run })?;
         match reply {
-            drtioaux::Packet::SubkernelLoadRunReply { succeeded: true } => Ok(()),
-            drtioaux::Packet::SubkernelLoadRunReply { succeeded: false } =>
+            drtioaux::Packet::SubkernelLoadRunReply { destination: 0, succeeded: true } => Ok(()),
+            drtioaux::Packet::SubkernelLoadRunReply { destination: 0, succeeded: false } =>
                     Err(Error::SubkernelRunFail(destination)),
                 packet => Err(Error::UnexpectedPacket(packet)),
         }
     }
 
-    pub fn subkernel_retrieve_exception(io: &Io, aux_mutex: &Mutex,
+    pub fn subkernel_retrieve_exception(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &drtio_routing::RoutingTable, destination: u8
     ) -> Result<Vec<u8>, Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         let mut remote_data: Vec<u8> = Vec::new();
         loop {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelExceptionRequest { destination: destination })?;
             match reply {
                 drtioaux::Packet::SubkernelException { last, length, data } => { 
@@ -588,21 +625,21 @@ pub mod drtio {
         }
     }
 
-    pub fn subkernel_send_message(io: &Io, aux_mutex: &Mutex,
+    pub fn subkernel_send_message(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
         routing_table: &drtio_routing::RoutingTable, id: u32, destination: u8, message: &[u8]
     ) -> Result<(), Error> {
         let linkno = routing_table.0[destination as usize][0] - 1;
         partition_data(message, |slice, status, len: usize| {
-            let reply = aux_transact(io, aux_mutex, linkno, 
+            let reply = aux_transact(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, linkno, 
                 &drtioaux::Packet::SubkernelMessage {
-                    destination: destination, id: id, status: status, length: len as u16, data: *slice})?;
+                    source: 0, destination: destination,
+                    id: id, status: status, length: len as u16, data: *slice})?;
             match reply {
                 drtioaux::Packet::SubkernelMessageAck { .. } => Ok(()),
                 packet => Err(Error::UnexpectedPacket(packet)),
             }
         })
     }
-
 }
 
 #[cfg(not(has_drtio))]
@@ -613,7 +650,7 @@ pub mod drtio {
         _routing_table: &Urc<RefCell<drtio_routing::RoutingTable>>,
         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
         _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex) {}
-    pub fn reset(_io: &Io, _aux_mutex: &Mutex) {}
+    pub fn reset(_io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, _routing_table: &drtio_routing::RoutingTable) {}
 }
 
 static mut SEEN_ASYNC_ERRORS: u8 = 0;
@@ -685,9 +722,10 @@ pub fn startup(io: &Io, aux_mutex: &Mutex,
     io.spawn(4096, async_error_thread);
 }
 
-pub fn reset(io: &Io, aux_mutex: &Mutex) {
+pub fn reset(io: &Io, aux_mutex: &Mutex, ddma_mutex: &Mutex, subkernel_mutex: &Mutex,
+    routing_table: &drtio_routing::RoutingTable) {
     unsafe {
         csr::rtio_core::reset_write(1);
     }
-    drtio::reset(io, aux_mutex)
+    drtio::reset(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table)
 }

artiq/firmware/runtime/session.rs

@@ -243,12 +243,13 @@ pub fn kern_send(io: &Io, request: &kern::Message) -> Result<(), Error<SchedErro
 
 fn kern_recv_notrace<R, F>(io: &Io, f: F) -> Result<R, Error<SchedError>>
         where F: FnOnce(&kern::Message) -> Result<R, Error<SchedError>> {
-    io.until(|| mailbox::receive() != 0)?;
-    if !kernel::validate(mailbox::receive()) {
-        return Err(Error::InvalidPointer(mailbox::receive()))
+    let mut msg_ptr = 0;
+    io.until(|| { msg_ptr = mailbox::receive(); msg_ptr != 0 })?;
+    if !kernel::validate(msg_ptr) {
+        return Err(Error::InvalidPointer(msg_ptr))
     }
 
-    f(unsafe { &*(mailbox::receive() as *const kern::Message) })
+    f(unsafe { &*(msg_ptr as *const kern::Message) })
 }
 
 fn kern_recv_dotrace(reply: &kern::Message) {
@@ -316,7 +317,7 @@ fn kern_run(session: &mut Session) -> Result<(), Error<SchedError>> {
 }
 
 
-fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex, 
+fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex, _ddma_mutex: &Mutex,
                         _routing_table: &drtio_routing::RoutingTable,
                         _up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
                         session: &mut Session, kernel: &[u8]
@@ -355,7 +356,7 @@ fn process_flash_kernel(io: &Io, _aux_mutex: &Mutex, _subkernel_mutex: &Mutex,
                     if up {
                         let subkernel_lib = entry.data().to_vec();
                         subkernel::add_subkernel(io, _subkernel_mutex, sid, dest, subkernel_lib)?;
-                        subkernel::upload(io, _aux_mutex, _subkernel_mutex, _routing_table, sid)?;
+                        subkernel::upload(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, sid)?;
                     } else {
                         return Err(Error::DestinationDown);
                     }
@@ -468,9 +469,10 @@ fn process_host_message(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subke
             #[cfg(has_drtio)]
             {
                 subkernel::add_subkernel(io, _subkernel_mutex, _id, _dest, _kernel)?;
-                match subkernel::upload(io, _aux_mutex, _subkernel_mutex, _routing_table, _id) {
+                match subkernel::upload(io, _aux_mutex, _ddma_mutex, _subkernel_mutex, _routing_table, _id) {
                     Ok(_) => host_write(stream, host::Reply::LoadCompleted)?,
                     Err(error) => {
+                        subkernel::clear_subkernels(io, _subkernel_mutex)?;
                         let mut description = String::new();
                         write!(&mut description, "{}", error).unwrap();
                         host_write(stream, host::Reply::LoadFailed(&description))?
@@ -488,7 +490,7 @@ fn process_host_message(io: &Io, _aux_mutex: &Mutex, _ddma_mutex: &Mutex, _subke
 fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                         routing_table: &drtio_routing::RoutingTable,
                         up_destinations: &Urc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
-                        ddma_mutex: &Mutex, _subkernel_mutex: &Mutex, mut stream: Option<&mut TcpStream>,
+                        ddma_mutex: &Mutex, subkernel_mutex: &Mutex, mut stream: Option<&mut TcpStream>,
                         session: &mut Session) -> Result<bool, Error<SchedError>> {
     kern_recv_notrace(io, |request| {
         match (request, session.kernel_state) {
@@ -506,7 +508,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
 
         kern_recv_dotrace(request);
 
-        if kern_hwreq::process_kern_hwreq(io, aux_mutex, routing_table, up_destinations, request)? {
+        if kern_hwreq::process_kern_hwreq(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, up_destinations, request)? {
             return Ok(false)
         }
 
@@ -530,7 +532,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 if let Some(_id) = session.congress.dma_manager.record_start(name) {
                     // replace the record
                     #[cfg(has_drtio)]
-                    remote_dma::erase(io, aux_mutex, ddma_mutex, routing_table, _id)?;
+                    remote_dma::erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id)?;
                 }
                 kern_acknowledge()
             }
@@ -542,7 +544,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 let _id = session.congress.dma_manager.record_stop(duration, enable_ddma, io, ddma_mutex)?;
                 #[cfg(has_drtio)]
                 if enable_ddma {
-                    remote_dma::upload_traces(io, aux_mutex, ddma_mutex, routing_table, _id)?;
+                    remote_dma::upload_traces(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id)?;
                 }
                 cache::flush_l2_cache();
                 kern_acknowledge()
@@ -550,7 +552,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             &kern::DmaEraseRequest { name } => {
                 #[cfg(has_drtio)]
                 if let Some(id) = session.congress.dma_manager.get_id(name) {
-                    remote_dma::erase(io, aux_mutex, ddma_mutex, routing_table, *id)?;
+                    remote_dma::erase(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, *id)?;
                 }
                 session.congress.dma_manager.erase(name);
                 kern_acknowledge()
@@ -573,7 +575,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             }
             &kern::DmaStartRemoteRequest { id: _id, timestamp: _timestamp } => {
                 #[cfg(has_drtio)]
-                remote_dma::playback(io, aux_mutex, ddma_mutex, routing_table, _id as u32, _timestamp as u64)?;
+                remote_dma::playback(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, _id as u32, _timestamp as u64)?;
                 kern_acknowledge()
             }
             &kern::DmaAwaitRemoteRequest { id: _id } => {
@@ -631,6 +633,8 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 unsafe { kernel::stop() }
                 session.kernel_state = KernelState::Absent;
                 unsafe { session.congress.cache.unborrow() }
+                #[cfg(has_drtio)]
+                subkernel::clear_subkernels(io, subkernel_mutex)?;
 
                 match stream {
                     None => return Ok(true),
@@ -648,6 +652,8 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 unsafe { kernel::stop() }
                 session.kernel_state = KernelState::Absent;
                 unsafe { session.congress.cache.unborrow() }
+                #[cfg(has_drtio)]
+                subkernel::clear_subkernels(io, subkernel_mutex)?;
 
                 match stream {
                     None => {
@@ -668,9 +674,9 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 }
             }
             #[cfg(has_drtio)]
-            &kern::SubkernelLoadRunRequest { id, run } => {
+            &kern::SubkernelLoadRunRequest { id, destination: _, run } => {
                 let succeeded = match subkernel::load(
-                    io, aux_mutex, _subkernel_mutex, routing_table, id, run) {
+                    io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, run) {
                         Ok(()) => true,
                         Err(e) => { error!("Error loading subkernel: {}", e); false }
                     };
@@ -678,7 +684,7 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
             }
             #[cfg(has_drtio)]
             &kern::SubkernelAwaitFinishRequest{ id, timeout } => {
-                let res = subkernel::await_finish(io, aux_mutex, _subkernel_mutex, routing_table,
+                let res = subkernel::await_finish(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table,
                     id, timeout);
                 let status = match res {
                     Ok(ref res) => {
@@ -699,20 +705,20 @@ fn process_kern_message(io: &Io, aux_mutex: &Mutex,
                 kern_send(io, &kern::SubkernelAwaitFinishReply { status: status })
             }
             #[cfg(has_drtio)]
-            &kern::SubkernelMsgSend { id, count, tag, data } => {
-                subkernel::message_send(io, aux_mutex, _subkernel_mutex, routing_table, id, count, tag, data)?;
+            &kern::SubkernelMsgSend { id, destination, count, tag, data } => {
+                subkernel::message_send(io, aux_mutex, ddma_mutex, subkernel_mutex, routing_table, id, destination, count, tag, data)?;
                 kern_acknowledge()
             }
             #[cfg(has_drtio)]
             &kern::SubkernelMsgRecvRequest { id, timeout, tags } => {
-                let message_received = subkernel::message_await(io, _subkernel_mutex, id, timeout);
+                let message_received = subkernel::message_await(io, subkernel_mutex, id as u32, timeout);
                 let (status, count) = match message_received {
                     Ok(ref message) => (kern::SubkernelStatus::NoError, message.count),
                     Err(SubkernelError::Timeout) => (kern::SubkernelStatus::Timeout, 0),
                     Err(SubkernelError::IncorrectState) => (kern::SubkernelStatus::IncorrectState, 0),
                     Err(SubkernelError::SubkernelFinished) => {
-                        let res = subkernel::retrieve_finish_status(io, aux_mutex, _subkernel_mutex,
-                            routing_table, id)?;
+                        let res = subkernel::retrieve_finish_status(io, aux_mutex, ddma_mutex, subkernel_mutex,
+                            routing_table, id as u32)?;
                         if res.comm_lost {
                             (kern::SubkernelStatus::CommLost, 0)
                         } else if let Some(exception) = &res.exception {
@@ -842,7 +848,7 @@ fn flash_kernel_worker(io: &Io, aux_mutex: &Mutex,
         match result {
             Ok(kernel) => {
                 // process .ELF or .TAR kernels
-                let res = process_flash_kernel(io, aux_mutex, subkernel_mutex, routing_table, up_destinations, &mut session, kernel);
+                let res = process_flash_kernel(io, aux_mutex, subkernel_mutex, ddma_mutex, routing_table, up_destinations, &mut session, kernel);
                 #[cfg(has_drtio)]
                 match res {
                     // wait to establish the DRTIO connection
@@ -887,7 +893,7 @@ fn respawn<F>(io: &Io, handle: &mut Option<ThreadHandle>, f: F)
         }
     }
 
-    *handle = Some(io.spawn(24576, f))
+    *handle = Some(io.spawn(32768, f))
 }
 
 pub fn thread(io: Io, aux_mutex: &Mutex,
@@ -971,7 +977,13 @@ pub fn thread(io: Io, aux_mutex: &Mutex,
                         drtio::clear_buffers(&io, &aux_mutex);
                     }
                 }
-                stream.close().expect("session: close socket");
+                loop {
+                    match stream.close() {
+                        Ok(_) => break,
+                        Err(SchedError::Interrupted) => (),
+                        Err(e) => panic!("session: close socket: {:?}", e)
+                    };
+                }
             });
         }
 

artiq/firmware/satman/Makefile

@@ -10,8 +10,6 @@ LDFLAGS += \
 
 RUSTFLAGS += -Cpanic=unwind
 
-export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
-
 all:: satman.bin satman.fbi
 
 .PHONY: $(RUSTOUT)/libsatman.a

artiq/firmware/satman/dma.rs

@@ -1,41 +1,190 @@
+use alloc::{vec::Vec, collections::btree_map::BTreeMap, string::String};
+use core::mem;
+use board_artiq::{drtioaux, drtio_routing::RoutingTable};
 use board_misoc::{csr, cache::flush_l2_cache};
 use proto_artiq::drtioaux_proto::PayloadStatus;
-use alloc::{vec::Vec, collections::btree_map::BTreeMap};
-use ::{cricon_select, RtioMaster};
+use routing::{Router, Sliceable};
+use kernel::Manager as KernelManager;
+use ::{cricon_select, cricon_read, RtioMaster, MASTER_PAYLOAD_MAX_SIZE};
 
 const ALIGNMENT: usize = 64;
 
-#[derive(Debug, PartialEq)]
+#[derive(PartialEq)]
 enum ManagerState {
     Idle,
     Playback
 }
 
 pub struct RtioStatus {
+    pub source: u8,
     pub id: u32, 
-    pub error: u8, 
+    pub error: u8,
     pub channel: u32, 
     pub timestamp: u64
 }
 
+#[derive(Debug)]
 pub enum Error {
     IdNotFound,
     PlaybackInProgress,
-    EntryNotComplete
+    EntryNotComplete,
+    MasterDmaFound,
+    UploadFail,
 }
 
-#[derive(Debug)]
 struct Entry {
     trace: Vec<u8>,
     padding_len: usize,
-    complete: bool
+    complete: bool,
+    duration: u64, // relevant for locally ran DMA
+}
+
+impl Entry {
+    pub fn from_vec(data: Vec<u8>, duration: u64) -> Entry {
+        let mut entry = Entry {
+            trace: data,
+            padding_len: 0,
+            complete: true,
+            duration: duration,
+        };
+        entry.realign();
+        entry
+    }
+
+    pub fn id(&self) -> u32 {
+        self.trace[self.padding_len..].as_ptr() as u32
+    }
+
+    pub fn realign(&mut self) {
+        self.trace.push(0);
+        let data_len = self.trace.len();
+
+        self.trace.reserve(ALIGNMENT - 1);
+        let padding = ALIGNMENT - self.trace.as_ptr() as usize % ALIGNMENT;
+        let padding = if padding == ALIGNMENT { 0 } else { padding };
+        for _ in 0..padding {
+            // Vec guarantees that this will not reallocate
+            self.trace.push(0)
+        }
+        for i in 1..data_len + 1 {
+            self.trace[data_len + padding - i] = self.trace[data_len - i]
+        }
+        self.complete = true;
+        self.padding_len = padding;
+    }
+}
+
+enum RemoteTraceState {
+    Unsent,
+    Sending(usize),
+    Ready,
+    Running(usize),
+}
+
+struct RemoteTraces {
+    remote_traces: BTreeMap<u8, Sliceable>,
+    state: RemoteTraceState,
+}
+
+impl RemoteTraces {
+    pub fn new(traces: BTreeMap<u8, Sliceable>) -> RemoteTraces {
+        RemoteTraces {
+            remote_traces: traces,
+            state: RemoteTraceState::Unsent
+        }
+    }
+
+    // on subkernel request
+    pub fn upload_traces(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) -> usize {
+        let len = self.remote_traces.len();
+        if len > 0 {
+            self.state = RemoteTraceState::Sending(self.remote_traces.len());
+            for (dest, trace) in self.remote_traces.iter_mut() {
+                // queue up the first packet for all destinations, rest will be sent after first ACK
+                let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                let meta = trace.get_slice_master(&mut data_slice);
+                router.route(drtioaux::Packet::DmaAddTraceRequest {
+                    source: self_destination, destination: *dest, id: id,
+                    status: meta.status, length: meta.len, trace: data_slice
+                }, routing_table, rank, self_destination);
+            }
+        }
+        len
+    }
+
+    // on incoming Packet::DmaAddTraceReply
+    pub fn ack_upload(&mut self, kernel_manager: &mut KernelManager, source: u8, id: u32, succeeded: bool, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        if let RemoteTraceState::Sending(count) = self.state {
+            if let Some(trace) = self.remote_traces.get_mut(&source) {
+                if trace.at_end() {
+                        if count - 1 == 0 {
+                            self.state = RemoteTraceState::Ready;
+                            kernel_manager.ddma_remote_uploaded(succeeded);
+                        } else {
+                            self.state = RemoteTraceState::Sending(count - 1);
+                        }
+                } else {
+                    // send next slice
+                    let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+                    let meta = trace.get_slice_master(&mut data_slice);
+                    router.route(drtioaux::Packet::DmaAddTraceRequest {
+                        source: self_destination, destination: meta.destination, id: id,
+                        status: meta.status, length: meta.len, trace: data_slice
+                    }, routing_table, rank, self_destination);
+                }
+            }
+        }
+        
+    }
+
+    // on subkernel request
+    pub fn playback(&mut self, id: u32, timestamp: u64, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        // route all the playback requests
+        // remote traces + local trace
+        self.state = RemoteTraceState::Running(self.remote_traces.len() + 1);
+        for (dest, _) in self.remote_traces.iter() {
+            router.route(drtioaux::Packet::DmaPlaybackRequest {
+                source: self_destination, destination: *dest, id: id, timestamp: timestamp
+            }, routing_table, rank, self_destination);
+            // response will be ignored (succeeded = false handled by the main thread)
+        }
+    }
+
+    // on incoming Packet::DmaPlaybackDone
+    pub fn remote_finished(&mut self, kernel_manager: &mut KernelManager, error: u8, channel: u32, timestamp: u64) {
+        if let RemoteTraceState::Running(count) = self.state {
+            if error != 0 || count - 1 == 0 {
+                // notify the kernel about a DDMA error or finish
+                kernel_manager.ddma_finished(error, channel, timestamp);
+                self.state = RemoteTraceState::Ready;
+                // further messages will be ignored (if there was an error)
+            } else { // no error and not the last one awaited
+                self.state = RemoteTraceState::Running(count - 1);
+            }
+        }
+    }
+
+    pub fn erase(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        for (dest, _) in self.remote_traces.iter() {
+            router.route(drtioaux::Packet::DmaRemoveTraceRequest { 
+                source: self_destination, destination: *dest, id: id
+            }, routing_table, rank, self_destination);
+            // response will be ignored as this object will stop existing too
+        }
+    }
 }
 
-#[derive(Debug)]
 pub struct Manager {
-    entries: BTreeMap<u32, Entry>,
+    entries: BTreeMap<(u8, u32), Entry>,
     state: ManagerState,
-    currentid: u32
+    current_id: u32,
+    current_source: u8,
+    previous_cri_master: RtioMaster,
+
+    remote_entries: BTreeMap<u32, RemoteTraces>,
+    name_map: BTreeMap<String, u32>,
+    recording_trace: Vec<u8>,
+    recording_name: String
 }
 
 impl Manager {
@@ -47,74 +196,197 @@ impl Manager {
         }
         Manager {
             entries: BTreeMap::new(),
-            currentid: 0,
+            current_id: 0,
+            current_source: 0,
+            previous_cri_master: RtioMaster::Drtio,
             state: ManagerState::Idle,
+            remote_entries: BTreeMap::new(),
+            name_map: BTreeMap::new(),
+            recording_trace: Vec::new(),
+            recording_name: String::new(),
         }
     }
 
-    pub fn add(&mut self, id: u32, status: PayloadStatus, trace: &[u8], trace_len: usize) -> Result<(), Error> {
+    pub fn add(&mut self, source: u8, id: u32, status: PayloadStatus, trace: &[u8], trace_len: usize) -> Result<(), Error> {
         if status.is_first() {
-            self.entries.remove(&id);
+            self.entries.remove(&(source, id));
         }
-        let entry = match self.entries.get_mut(&id) {
+        let entry = match self.entries.get_mut(&(source, id)) {
             Some(entry) => {
                 if entry.complete {
                     // replace entry
-                    self.entries.remove(&id);
-                    self.entries.insert(id, Entry {
+                    self.entries.remove(&(source, id));
+                    self.entries.insert((source, id), Entry {
                         trace: Vec::new(),
                         padding_len: 0,
-                        complete: false });
-                    self.entries.get_mut(&id).unwrap()
+                        complete: false,
+                        duration: 0
+                    });
+                    self.entries.get_mut(&(source, id)).unwrap()
                 } else {
                     entry
                 }
             },
             None => {
-                self.entries.insert(id, Entry {
+                self.entries.insert((source, id), Entry {
                     trace: Vec::new(),
                     padding_len: 0,
-                    complete: false });
-                self.entries.get_mut(&id).unwrap()
+                    complete: false,
+                    duration: 0,
+                });
+                self.entries.get_mut(&(source, id)).unwrap()
             },
         };
         entry.trace.extend(&trace[0..trace_len]);
 
         if status.is_last() {
-            entry.trace.push(0);
-            let data_len = entry.trace.len();
-    
-            // Realign.
-            entry.trace.reserve(ALIGNMENT - 1);
-            let padding = ALIGNMENT - entry.trace.as_ptr() as usize % ALIGNMENT;
-            let padding = if padding == ALIGNMENT { 0 } else { padding };
-            for _ in 0..padding {
-                // Vec guarantees that this will not reallocate
-                entry.trace.push(0)
-            }
-            for i in 1..data_len + 1 {
-                entry.trace[data_len + padding - i] = entry.trace[data_len - i]
-            }
-            entry.complete = true;
-            entry.padding_len = padding;
+            entry.realign();
             flush_l2_cache();
         }
         Ok(())
     }
+    // API for subkernel
+    pub fn record_start(&mut self, name: &str) {
+        self.recording_name = String::from(name);
+        self.recording_trace = Vec::new();
+    }
 
-    pub fn erase(&mut self, id: u32) -> Result<(), Error> {
-        match self.entries.remove(&id) {
+    // API for subkernel
+    pub fn record_append(&mut self, data: &[u8]) {
+        self.recording_trace.extend_from_slice(data);
+    }
+
+    // API for subkernel
+    pub fn record_stop(&mut self, duration: u64, self_destination: u8) -> Result<u32, Error> {
+        let mut trace = Vec::new();
+        mem::swap(&mut self.recording_trace, &mut trace);
+        trace.push(0);
+        let mut local_trace = Vec::new();
+        let mut remote_traces: BTreeMap<u8, Sliceable> = BTreeMap::new();
+        // analyze each entry and put in proper buckets, as the kernel core
+        // sends whole chunks, to limit comms/kernel CPU communication,
+        // and as only comms core has access to varios DMA buffers.
+        let mut ptr = 0;
+        while trace[ptr] != 0 {
+            // ptr + 3 = tgt >> 24 (destination)
+            let len = trace[ptr] as usize;
+            let destination = trace[ptr+3];
+            if destination == 0 {
+                return Err(Error::MasterDmaFound);
+            } else if destination == self_destination {
+                local_trace.extend(&trace[ptr..ptr+len]);
+            }
+            else {
+                if let Some(remote_trace) = remote_traces.get_mut(&destination) {
+                    remote_trace.extend(&trace[ptr..ptr+len]);
+                } else {
+                    remote_traces.insert(destination, Sliceable::new(destination, trace[ptr..ptr+len].to_vec()));
+                }
+            }
+            // and jump to the next event
+            ptr += len;
+        }
+        let local_entry = Entry::from_vec(local_trace, duration);
+        let id = local_entry.id();
+
+        self.entries.insert((self_destination, id), local_entry);
+        self.remote_entries.insert(id, RemoteTraces::new(remote_traces));
+        let mut name = String::new();
+        mem::swap(&mut self.recording_name, &mut name);
+        self.name_map.insert(name, id);
+
+        flush_l2_cache();
+
+        Ok(id)
+    }
+
+    pub fn upload_traces(&mut self, id: u32, router: &mut Router, rank: u8, self_destination: u8,
+            routing_table: &RoutingTable) -> Result<usize, Error> {
+        let remote_traces = self.remote_entries.get_mut(&id);
+        let mut len = 0;
+        if let Some(traces) = remote_traces {
+            len = traces.upload_traces(id, router, rank, self_destination, routing_table);
+        }
+        Ok(len)
+    }
+
+    pub fn with_trace<F, R>(&self, self_destination: u8, name: &str, f: F) -> R
+        where F: FnOnce(Option<&[u8]>, u64) -> R {
+        if let Some(ptr) = self.name_map.get(name) {
+            match self.entries.get(&(self_destination, *ptr)) {
+                Some(entry) => f(Some(&entry.trace[entry.padding_len..]), entry.duration),
+                None => f(None, 0)
+            }
+        } else {
+            f(None, 0)
+        }
+    }
+
+    // API for subkernel
+    pub fn playback_remote(&mut self, id: u32, timestamp: u64, 
+        router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable
+    ) -> Result<(), Error> {
+        if let Some(traces) = self.remote_entries.get_mut(&id) {
+            traces.playback(id, timestamp, router, rank, self_destination, routing_table);
+            Ok(())
+        } else {
+            Err(Error::IdNotFound)
+        }
+    }
+
+    // API for subkernel
+    pub fn erase_name(&mut self, name: &str, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        if let Some(id) = self.name_map.get(name) {
+            if let Some(traces) = self.remote_entries.get_mut(&id) {
+                traces.erase(*id, router, rank, self_destination, routing_table);
+                self.remote_entries.remove(&id);
+            }
+            self.entries.remove(&(self_destination, *id));
+            self.name_map.remove(name);
+        }
+    }
+
+    // API for incoming DDMA (drtio)
+    pub fn erase(&mut self, source: u8, id: u32) -> Result<(), Error> {
+        match self.entries.remove(&(source, id)) {
             Some(_) => Ok(()),
             None => Err(Error::IdNotFound)
         }
     }
 
-    pub fn playback(&mut self, id: u32, timestamp: u64) -> Result<(), Error> {
+    pub fn remote_finished(&mut self, kernel_manager: &mut KernelManager, 
+        id: u32, error: u8, channel: u32, timestamp: u64) {
+        if let Some(entry) = self.remote_entries.get_mut(&id) {
+            entry.remote_finished(kernel_manager, error, channel, timestamp);
+        }
+    }
+
+    pub fn ack_upload(&mut self, kernel_manager: &mut KernelManager, source: u8, id: u32, succeeded: bool, 
+        router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+            if let Some(entry) = self.remote_entries.get_mut(&id) {
+                entry.ack_upload(kernel_manager, source, id, succeeded, router, rank, self_destination, routing_table);
+            }
+    }
+
+    pub fn cleanup(&mut self, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
+        // after subkernel ends, remove all self-generated traces
+        for (_, id) in self.name_map.iter_mut() {
+            if let Some(traces) = self.remote_entries.get_mut(&id) {
+                traces.erase(*id, router, rank, self_destination, routing_table);
+                self.remote_entries.remove(&id);
+            }
+            self.entries.remove(&(self_destination, *id));
+        }
+        self.name_map.clear();
+    }
+
+    // API for both incoming DDMA (drtio) and subkernel
+    pub fn playback(&mut self, source: u8, id: u32, timestamp: u64) -> Result<(), Error> {
         if self.state != ManagerState::Idle {
             return Err(Error::PlaybackInProgress);
         }
 
-        let entry = match self.entries.get(&id){
+        let entry = match self.entries.get(&(source, id)){
             Some(entry) => entry,
             None => { return Err(Error::IdNotFound); }
         };
@@ -125,7 +397,9 @@ impl Manager {
         assert!(ptr as u32 % 64 == 0);
 
         self.state = ManagerState::Playback;
-        self.currentid = id;
+        self.current_id = id;
+        self.current_source = source;
+        self.previous_cri_master = cricon_read();
 
         unsafe {
             csr::rtio_dma::base_address_write(ptr as u64);
@@ -149,15 +423,16 @@ impl Manager {
         } else {
             self.state = ManagerState::Idle;
             unsafe { 
-                cricon_select(RtioMaster::Drtio);
+                cricon_select(self.previous_cri_master);
                 let error = csr::rtio_dma::error_read();
                 let channel = csr::rtio_dma::error_channel_read();
                 let timestamp = csr::rtio_dma::error_timestamp_read();
                 if error != 0 {
                     csr::rtio_dma::error_write(1);
                 }
-                return Some(RtioStatus { 
-                    id: self.currentid, 
+                return Some(RtioStatus {
+                    source: self.current_source,
+                    id: self.current_id,
                     error: error,
                     channel: channel, 
                     timestamp: timestamp });

artiq/firmware/satman/kernel.rs

@@ -1,11 +1,11 @@
-use core::{mem, option::NoneError, cmp::min};
-use alloc::{string::String, format, vec::Vec, collections::{btree_map::BTreeMap, vec_deque::VecDeque}};
+use core::mem;
+use alloc::{string::String, format, vec::Vec, collections::btree_map::BTreeMap};
 use cslice::AsCSlice;
 
-use board_artiq::{mailbox, spi};
+use board_artiq::{drtioaux, drtio_routing::RoutingTable, mailbox, spi};
 use board_misoc::{csr, clock, i2c};
 use proto_artiq::{
-    drtioaux_proto::PayloadStatus, 
+    drtioaux_proto::PayloadStatus,
     kernel_proto as kern, 
     session_proto::Reply::KernelException as HostKernelException, 
     rpc_proto as rpc};
@@ -15,6 +15,8 @@ use kernel::eh_artiq::StackPointerBacktrace;
 
 use ::{cricon_select, RtioMaster};
 use cache::Cache;
+use dma::{Manager as DmaManager, Error as DmaError};
+use routing::{Router, Sliceable, SliceMeta};
 use SAT_PAYLOAD_MAX_SIZE;
 use MASTER_PAYLOAD_MAX_SIZE;
 
@@ -61,8 +63,12 @@ enum KernelState {
     Absent,
     Loaded,
     Running,
-    MsgAwait { max_time: u64, tags: Vec<u8> },
-    MsgSending
+    MsgAwait { id: u32, max_time: i64, tags: Vec<u8> },
+    MsgSending,
+    SubkernelAwaitLoad,
+    SubkernelAwaitFinish { max_time: i64, id: u32 },
+    DmaUploading { max_time: u64 },
+    DmaAwait { max_time: u64 },
 }
 
 #[derive(Debug)]
@@ -74,13 +80,9 @@ pub enum Error {
     NoMessage,
     AwaitingMessage,
     SubkernelIoError,
-    KernelException(Sliceable)
-}
-
-impl From<NoneError> for Error {
-    fn from(_: NoneError) -> Error {
-        Error::KernelNotFound
-    }
+    DrtioError,
+    KernelException(Sliceable),
+    DmaError(DmaError),
 }
 
 impl From<io::Error<!>> for Error {
@@ -89,19 +91,25 @@ impl From<io::Error<!>> for Error {
     }
 }
 
+impl From<drtioaux::Error<!>> for Error {
+    fn from(_value: drtioaux::Error<!>) -> Error {
+        Error::DrtioError
+    }
+}
+
+impl From<DmaError> for Error {
+    fn from(value: DmaError) -> Error {
+        Error::DmaError(value)
+    }
+}
+
 macro_rules! unexpected {
     ($($arg:tt)*) => (return Err(Error::Unexpected(format!($($arg)*))));
 }
 
-/* represents data that has to be sent to Master */
-#[derive(Debug)]
-pub struct Sliceable {
-    it: usize,
-    data: Vec<u8>
-}
-
 /* represents interkernel messages */
 struct Message {
+    id: u32,
     count: u8,
     data: Vec<u8>
 }
@@ -109,7 +117,6 @@ struct Message {
 #[derive(PartialEq)]
 enum OutMessageState {
     NoMessage,
-    MessageReady,
     MessageBeingSent,
     MessageSent,
     MessageAcknowledged
@@ -119,7 +126,7 @@ enum OutMessageState {
 struct MessageManager {
     out_message: Option<Sliceable>,
     out_state: OutMessageState,
-    in_queue: VecDeque<Message>,
+    in_queue: Vec<Message>,
     in_buffer: Option<Message>,
 }
 
@@ -128,7 +135,9 @@ struct Session {
     kernel_state: KernelState,
     log_buffer: String,
     last_exception: Option<Sliceable>,
-    messages: MessageManager
+    source: u8, // which destination requested running the kernel
+    messages: MessageManager,
+    subkernels_finished: Vec<u32> // ids of subkernels finished
 }
 
 #[derive(Debug)]
@@ -147,42 +156,9 @@ pub struct Manager {
 
 pub struct SubkernelFinished {
     pub id: u32,
-    pub with_exception: bool
-}
-
-pub struct SliceMeta {
-    pub len: u16,
-    pub status: PayloadStatus
-}
-
-macro_rules! get_slice_fn {
-    ( $name:tt, $size:expr ) => {
-        pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
-            let first = self.it == 0;
-            let len = min($size, self.data.len() - self.it);
-            let last = self.it + len == self.data.len();
-            let status = PayloadStatus::from_status(first, last);
-            data_slice[..len].clone_from_slice(&self.data[self.it..self.it+len]);
-            self.it += len;
-    
-            SliceMeta {
-                len: len as u16,
-                status: status
-            }
-        }
-    };
-}
-
-impl Sliceable {
-    pub fn new(data: Vec<u8>) -> Sliceable {
-        Sliceable {
-            it: 0,
-            data: data
-        }
-    }
-
-    get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
-    get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
+    pub with_exception: bool,
+    pub exception_source: u8,
+    pub source: u8
 }
 
 impl MessageManager {
@@ -190,12 +166,12 @@ impl MessageManager {
         MessageManager {
             out_message: None,
             out_state: OutMessageState::NoMessage,
-            in_queue: VecDeque::new(),
+            in_queue: Vec::new(),
             in_buffer: None
         }
     }
 
-    pub fn handle_incoming(&mut self, status: PayloadStatus, length: usize, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
+    pub fn handle_incoming(&mut self, status: PayloadStatus, length: usize, id: u32, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
         // called when receiving a message from master
         if status.is_first() {
             // clear the buffer for first message
@@ -205,6 +181,7 @@ impl MessageManager {
             Some(message) => message.data.extend(&data[..length]),
             None => {
                 self.in_buffer = Some(Message {
+                    id: id,
                     count: data[0],
                     data: data[1..length].to_vec()
                 });
@@ -212,18 +189,7 @@ impl MessageManager {
         };
         if status.is_last() {
             // when done, remove from working queue
-            self.in_queue.push_back(self.in_buffer.take().unwrap());
-        }
-    }
-
-    pub fn is_outgoing_ready(&mut self) -> bool {
-        // called by main loop, to see if there's anything to send, will send it afterwards
-        match self.out_state {
-            OutMessageState::MessageReady => {
-                self.out_state = OutMessageState::MessageBeingSent;
-                true
-            },
-            _ => false
+            self.in_queue.push(self.in_buffer.take().unwrap());
         }
     }
 
@@ -266,19 +232,42 @@ impl MessageManager {
         }
     }
 
-    pub fn accept_outgoing(&mut self, count: u8, tag: &[u8], data: *const *const ()) -> Result<(), Error>  {
+    pub fn accept_outgoing(&mut self, id: u32, self_destination: u8, destination: u8, 
+        count: u8, tag: &[u8], data: *const *const (), 
+        routing_table: &RoutingTable, rank: u8, router: &mut Router
+    ) -> Result<(), Error>  {
         let mut writer = Cursor::new(Vec::new());
         rpc::send_args(&mut writer, 0, tag, data, false)?;
         // skip service tag, but write the count
         let mut data = writer.into_inner().split_off(3);
         data[0] = count;
-        self.out_message = Some(Sliceable::new(data));
-        self.out_state = OutMessageState::MessageReady;
+        self.out_message = Some(Sliceable::new(destination, data));
+
+        let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
+        self.out_state = OutMessageState::MessageBeingSent;
+        let meta = self.get_outgoing_slice(&mut data_slice).unwrap();
+        router.route(drtioaux::Packet::SubkernelMessage {
+                source: self_destination, destination: destination, id: id,
+                status: meta.status, length: meta.len as u16, data: data_slice
+        }, routing_table, rank, self_destination);
         Ok(())
     }
 
-    pub fn get_incoming(&mut self) -> Option<Message> {
-        self.in_queue.pop_front()
+    pub fn get_incoming(&mut self, id: u32) -> Option<Message> {
+        for i in 0..self.in_queue.len() {
+            if self.in_queue[i].id == id {
+                return Some(self.in_queue.remove(i));
+            }
+        }
+        None
+    }
+
+    pub fn pending_ids(&self) -> Vec<u32> {
+        let mut pending_ids: Vec<u32> = Vec::new();
+        for msg in self.in_queue.iter() {
+            pending_ids.push(msg.id);
+        }
+        pending_ids
     }
 }
 
@@ -288,15 +277,16 @@ impl Session {
             kernel_state: KernelState::Absent,
             log_buffer: String::new(),
             last_exception: None,
-            messages: MessageManager::new()
+            source: 0,
+            messages: MessageManager::new(),
+            subkernels_finished: Vec::new()
         }
     }
 
     fn running(&self) -> bool {
         match self.kernel_state {
-            KernelState::Absent  | KernelState::Loaded  => false,
-            KernelState::Running | KernelState::MsgAwait { .. } |
-                KernelState::MsgSending => true
+            KernelState::Absent | KernelState::Loaded  => false,
+            _ => true
         }
     }
 
@@ -334,7 +324,7 @@ impl Manager {
                     self.kernels.insert(id, KernelLibrary {
                         library: Vec::new(),
                         complete: false });
-                    self.kernels.get_mut(&id)?
+                    self.kernels.get_mut(&id).unwrap()
                 } else {
                     kernel
                 }
@@ -343,7 +333,7 @@ impl Manager {
                 self.kernels.insert(id, KernelLibrary {
                     library: Vec::new(),
                     complete: false });
-                self.kernels.get_mut(&id)?
+                self.kernels.get_mut(&id).unwrap()
             },
         };
         kernel.library.extend(&data[0..data_len]);
@@ -369,23 +359,24 @@ impl Manager {
         unsafe { self.cache.unborrow() }
     }
 
-    pub fn run(&mut self, id: u32) -> Result<(), Error> {
+    pub fn run(&mut self, source: u8, id: u32) -> Result<(), Error> {
         info!("starting subkernel #{}", id);
         if self.session.kernel_state != KernelState::Loaded
             || self.current_id != id {
             self.load(id)?;
         }
+        self.session.source = source;
         self.session.kernel_state = KernelState::Running;
         cricon_select(RtioMaster::Kernel);
     
         kern_acknowledge()
     }
 
-    pub fn message_handle_incoming(&mut self, status: PayloadStatus, length: usize, slice: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
+    pub fn message_handle_incoming(&mut self, status: PayloadStatus, length: usize, id: u32, slice: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
         if !self.is_running() {
             return;
         }
-        self.session.messages.handle_incoming(status, length, slice);
+        self.session.messages.handle_incoming(status, length, id, slice);
     }
     
     pub fn message_get_slice(&mut self, slice: &mut [u8; MASTER_PAYLOAD_MAX_SIZE]) -> Option<SliceMeta> {
@@ -403,19 +394,11 @@ impl Manager {
         self.session.messages.ack_slice()
     }
 
-    pub fn message_is_ready(&mut self) -> bool {
-        self.session.messages.is_outgoing_ready()
-    }
-
-    pub fn get_last_finished(&mut self) -> Option<SubkernelFinished> {
-        self.last_finished.take()
-    }
-
     pub fn load(&mut self, id: u32) -> Result<(), Error> {
         if self.current_id == id && self.session.kernel_state == KernelState::Loaded {
             return Ok(())
         }
-        if !self.kernels.get(&id)?.complete {
+        if !self.kernels.get(&id).ok_or(Error::KernelNotFound)?.complete {
             return Err(Error::KernelNotFound)
         }
         self.current_id = id;
@@ -425,7 +408,7 @@ impl Manager {
         unsafe { 
             kernel_cpu::start();
 
-            kern_send(&kern::LoadRequest(&self.kernels.get(&id)?.library)).unwrap();
+            kern_send(&kern::LoadRequest(&self.kernels.get(&id).unwrap().library)).unwrap();
             kern_recv(|reply| {
                 match reply {
                     kern::LoadReply(Ok(())) => {
@@ -434,6 +417,7 @@ impl Manager {
                     }
                     kern::LoadReply(Err(error)) => {
                         kernel_cpu::stop();
+                        error!("load error: {:?}", error);
                         Err(Error::Load(format!("{}", error)))
                     }
                     other => {
@@ -447,7 +431,7 @@ impl Manager {
     pub fn exception_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
         match self.session.last_exception.as_mut() {
             Some(exception) => exception.get_slice_sat(data_slice),
-            None => SliceMeta { len: 0, status: PayloadStatus::FirstAndLast }
+            None => SliceMeta { destination: 0, len: 0, status: PayloadStatus::FirstAndLast }
         }
     }
 
@@ -472,12 +456,63 @@ impl Manager {
             backtrace: &[],
             async_errors: 0
         }).write_to(&mut writer) {
-            Ok(_) => self.session.last_exception = Some(Sliceable::new(writer.into_inner())),
+            Ok(_) => self.session.last_exception = Some(Sliceable::new(0, writer.into_inner())),
             Err(_) => error!("Error writing exception data")
         }
     }
 
-    pub fn process_kern_requests(&mut self, rank: u8) {
+    pub fn ddma_finished(&mut self, error: u8, channel: u32, timestamp: u64) {
+        if let KernelState::DmaAwait { .. } = self.session.kernel_state {
+            kern_send(&kern::DmaAwaitRemoteReply { 
+                timeout: false, error: error, channel: channel, timestamp: timestamp 
+            }).unwrap();
+            self.session.kernel_state = KernelState::Running;
+        }
+    }
+
+    pub fn ddma_nack(&mut self) {
+        // for simplicity treat it as a timeout for now...
+        if let KernelState::DmaAwait { .. } = self.session.kernel_state {
+            kern_send(&kern::DmaAwaitRemoteReply { 
+                timeout: true, error: 0, channel: 0, timestamp: 0
+            }).unwrap();
+            self.session.kernel_state = KernelState::Running;
+        }
+    }
+
+    pub fn ddma_remote_uploaded(&mut self, succeeded: bool) {
+        if let KernelState::DmaUploading { .. } = self.session.kernel_state {
+            if succeeded {
+                self.session.kernel_state = KernelState::Running;
+                kern_acknowledge().unwrap();
+            } else {
+                self.stop();
+                self.runtime_exception(Error::DmaError(DmaError::UploadFail));
+            }
+        }
+    }
+
+    pub fn process_kern_requests(&mut self, router: &mut Router, routing_table: &RoutingTable, rank: u8, destination: u8, dma_manager: &mut DmaManager) {
+        macro_rules! finished {
+            ($with_exception:expr) => {{ Some(SubkernelFinished { 
+                source: self.session.source, id: self.current_id, 
+                with_exception: $with_exception, exception_source: destination 
+            }) }}
+        }
+
+        if let Some(subkernel_finished) = self.last_finished.take() {
+            info!("subkernel {} finished, with exception: {}", subkernel_finished.id, subkernel_finished.with_exception);
+            let pending = self.session.messages.pending_ids();
+            if pending.len() > 0 {
+                warn!("subkernel terminated with messages still pending: {:?}", pending);
+            }
+            router.route(drtioaux::Packet::SubkernelFinished {
+                destination: subkernel_finished.source, id: subkernel_finished.id, 
+                with_exception: subkernel_finished.with_exception, exception_src: subkernel_finished.exception_source
+            }, &routing_table, rank, destination);
+            dma_manager.cleanup(router, rank, destination, routing_table);
+        }
+
         if !self.is_running() {
             return;
         }
@@ -490,39 +525,39 @@ impl Manager {
                 self.session.kernel_state = KernelState::Absent;
                 unsafe { self.cache.unborrow() }
                 self.session.last_exception = Some(exception);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
             },
             Err(e) => { 
                 error!("Error while running processing external messages: {:?}", e);
                 self.stop();
                 self.runtime_exception(e);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
              }
         }
 
-        match self.process_kern_message(rank) {
+        match self.process_kern_message(router, routing_table, rank, destination, dma_manager) {
             Ok(Some(with_exception)) => {
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: with_exception })
+                self.last_finished = finished!(with_exception)
             },
             Ok(None) | Err(Error::NoMessage) => (),
             Err(e) => { 
                 error!("Error while running kernel: {:?}", e); 
                 self.stop(); 
                 self.runtime_exception(e);
-                self.last_finished = Some(SubkernelFinished { id: self.current_id, with_exception: true })
+                self.last_finished = finished!(true);
             }
         }
     }
 
     fn process_external_messages(&mut self) -> Result<(), Error> {
         match &self.session.kernel_state {
-            KernelState::MsgAwait { max_time, tags } => {
-                if clock::get_ms() > *max_time {
+            KernelState::MsgAwait { id, max_time, tags } => {
+                if *max_time > 0 && clock::get_ms() > *max_time as u64 {
                     kern_send(&kern::SubkernelMsgRecvReply { status: kern::SubkernelStatus::Timeout, count: 0 })?;
                     self.session.kernel_state = KernelState::Running;
                     return Ok(())
                 }
-                if let Some(message) = self.session.messages.get_incoming() {
+                if let Some(message) = self.session.messages.get_incoming(*id) {
                     kern_send(&kern::SubkernelMsgRecvReply { status: kern::SubkernelStatus::NoError, count: message.count })?;
                     let tags = tags.clone();
                     self.session.kernel_state = KernelState::Running;
@@ -539,16 +574,88 @@ impl Manager {
                     Err(Error::AwaitingMessage)
                 }
             },
+            KernelState::SubkernelAwaitFinish { max_time, id } => {
+                if *max_time > 0 && clock::get_ms() > *max_time as u64 {
+                    kern_send(&kern::SubkernelAwaitFinishReply { status: kern::SubkernelStatus::Timeout })?;
+                    self.session.kernel_state = KernelState::Running;
+                } else {
+                    let mut i = 0;
+                    for status in &self.session.subkernels_finished {
+                        if *status == *id {
+                            kern_send(&kern::SubkernelAwaitFinishReply { status: kern::SubkernelStatus::NoError })?;
+                            self.session.kernel_state = KernelState::Running;
+                            self.session.subkernels_finished.swap_remove(i);
+                            break;
+                        }
+                        i += 1;
+                    }
+                }
+                Ok(())
+            }
+            KernelState::DmaAwait { max_time } => {
+                if clock::get_ms() > *max_time {
+                    kern_send(&kern::DmaAwaitRemoteReply { timeout: true, error: 0, channel: 0, timestamp: 0 })?;
+                    self.session.kernel_state = KernelState::Running;
+                }
+                // ddma_finished() and nack() covers the other case
+                Ok(())
+            }
+            KernelState::DmaUploading { max_time } => {
+                if clock::get_ms() > *max_time {
+                    unexpected!("DMAError: Timed out sending traces to remote");
+                }
+                Ok(())
+            }
             _ => Ok(())
         }
     }
 
-    fn process_kern_message(&mut self, rank: u8) -> Result<Option<bool>, Error> {
+    pub fn subkernel_load_run_reply(&mut self, succeeded: bool, self_destination: u8) {
+        if self.session.kernel_state == KernelState::SubkernelAwaitLoad {
+            if let Err(e) = kern_send(&kern::SubkernelLoadRunReply { succeeded: succeeded }) {
+                self.stop(); 
+                self.runtime_exception(e);
+                self.last_finished = Some(SubkernelFinished { 
+                    source: self.session.source, id: self.current_id, 
+                    with_exception: true, exception_source: self_destination 
+                })
+            } else {
+                self.session.kernel_state = KernelState::Running;
+            }
+        } else {
+            warn!("received unsolicited SubkernelLoadRunReply");
+        }
+    }
+
+    pub fn remote_subkernel_finished(&mut self, id: u32, with_exception: bool, exception_source: u8) {
+        if with_exception {
+            unsafe { kernel_cpu::stop() }
+            self.session.kernel_state = KernelState::Absent;
+            unsafe { self.cache.unborrow() }
+            self.last_finished = Some(SubkernelFinished {
+                source: self.session.source, id: self.current_id,
+                with_exception: true, exception_source: exception_source
+            })
+        } else {
+            self.session.subkernels_finished.push(id);
+        }
+    }
+
+    fn process_kern_message(&mut self, router: &mut Router, 
+        routing_table: &RoutingTable,
+        rank: u8, destination: u8,
+        dma_manager: &mut DmaManager
+    ) -> Result<Option<bool>, Error> {
         // returns Ok(with_exception) on finish
         // None if the kernel is still running
         kern_recv(|request| {
             match (request, &self.session.kernel_state) {
-                (&kern::LoadReply(_), KernelState::Loaded) => {
+                (&kern::LoadReply(_), KernelState::Loaded) |
+                    (_, KernelState::DmaUploading { .. }) |
+                    (_, KernelState::DmaAwait { .. }) |
+                    (_, KernelState::MsgSending) |
+                    (_, KernelState::SubkernelAwaitLoad) | 
+                    (_, KernelState::SubkernelAwaitFinish { .. }) => {
                     // We're standing by; ignore the message.
                     return Ok(None)
                 }
@@ -559,7 +666,7 @@ impl Manager {
                 },
             }
 
-            if process_kern_hwreq(request, rank)? {
+            if process_kern_hwreq(request, destination)? {
                 return Ok(None)
             }
 
@@ -613,19 +720,93 @@ impl Manager {
                     return Ok(Some(true))
                 }
 
-                &kern::SubkernelMsgSend { id: _, count, tag, data } => {
-                    self.session.messages.accept_outgoing(count, tag, data)?;
+                &kern::DmaRecordStart(name) => {
+                    dma_manager.record_start(name);
+                    kern_acknowledge()
+                }
+                &kern::DmaRecordAppend(data) => {
+                    dma_manager.record_append(data);
+                    kern_acknowledge()
+                }
+                &kern::DmaRecordStop { duration, enable_ddma: _ } => {
+                    // ddma is always used on satellites
+                    if let Ok(id) = dma_manager.record_stop(duration, destination) {
+                        let remote_count = dma_manager.upload_traces(id, router, rank, destination, routing_table)?;
+                        if remote_count > 0 {
+                            let max_time = clock::get_ms() + 10_000 as u64;
+                            self.session.kernel_state = KernelState::DmaUploading { max_time: max_time };
+                            Ok(())
+                        } else {
+                            kern_acknowledge()
+                        }
+                    } else {
+                        unexpected!("DMAError: found an unsupported call to RTIO devices on master") 
+                    }
+                }
+                &kern::DmaEraseRequest { name } => {
+                    dma_manager.erase_name(name, router, rank, destination, routing_table);
+                    kern_acknowledge()
+                }
+                &kern::DmaRetrieveRequest { name } => {
+                    dma_manager.with_trace(destination, name, |trace, duration| {
+                        kern_send(&kern::DmaRetrieveReply {
+                            trace:    trace,
+                            duration: duration,
+                            uses_ddma: true,
+                        })
+                    })
+                }
+                &kern::DmaStartRemoteRequest { id, timestamp } => {
+                    let max_time = clock::get_ms() + 10_000 as u64;
+                    self.session.kernel_state = KernelState::DmaAwait { max_time: max_time };
+                    dma_manager.playback_remote(id as u32, timestamp as u64, router, rank, destination, routing_table)?;
+                    dma_manager.playback(destination, id as u32, timestamp as u64)?;
+                    Ok(())
+                }
+
+                &kern::SubkernelMsgSend { id, destination: msg_dest, count, tag, data } => {
+                    let message_destination;
+                    let message_id;
+                    if let Some(dest) = msg_dest {
+                        message_destination = dest;
+                        message_id = id;
+                    } else {
+                        // return message, return to source
+                        message_destination = self.session.source;
+                        message_id = self.current_id;
+                    }
+                    self.session.messages.accept_outgoing(message_id, destination,
+                        message_destination, count, tag, data, 
+                        routing_table, rank, router)?;
                     // acknowledge after the message is sent
                     self.session.kernel_state = KernelState::MsgSending;
                     Ok(())
                 }
 
-                &kern::SubkernelMsgRecvRequest { id: _, timeout, tags } => {
-                    let max_time = clock::get_ms() + timeout as u64;
-                    self.session.kernel_state = KernelState::MsgAwait { max_time: max_time, tags: tags.to_vec() };
+                &kern::SubkernelMsgRecvRequest { id, timeout, tags } => {
+                    // negative timeout value means no timeout
+                    let max_time = if timeout > 0 { clock::get_ms() as i64 + timeout } else { timeout };
+                    // ID equal to -1 indicates wildcard for receiving arguments
+                    let id = if id == -1 { self.current_id } else { id as u32 };
+                    self.session.kernel_state = KernelState::MsgAwait { 
+                        id: id, max_time: max_time, tags: tags.to_vec() };
                     Ok(())
                 },
 
+                &kern::SubkernelLoadRunRequest { id, destination: sk_destination, run } => {
+                    self.session.kernel_state = KernelState::SubkernelAwaitLoad;
+                    router.route(drtioaux::Packet::SubkernelLoadRunRequest { 
+                        source: destination, destination: sk_destination, id: id, run: run 
+                    }, routing_table, rank, destination);
+                    Ok(())
+                }
+
+                &kern::SubkernelAwaitFinishRequest{ id, timeout } => {
+                    let max_time = if timeout > 0 { clock::get_ms() as i64 + timeout } else { timeout };
+                    self.session.kernel_state = KernelState::SubkernelAwaitFinish { max_time: max_time, id: id };
+                    Ok(())
+                }
+
                 request => unexpected!("unexpected request {:?} from kernel CPU", request)
             }.and(Ok(None))
         })
@@ -699,7 +880,7 @@ fn slice_kernel_exception(exceptions: &[Option<eh_artiq::Exception>],
         async_errors: 0
     }).write_to(&mut writer) {
         // save last exception data to be received by master
-        Ok(_) => Ok(Sliceable::new(writer.into_inner())),
+        Ok(_) => Ok(Sliceable::new(0, writer.into_inner())),
         Err(_) => Err(Error::SubkernelIoError)
     }
 }
@@ -759,7 +940,7 @@ fn pass_message_to_kernel(message: &Message, tags: &[u8]) -> Result<(), Error> {
     Ok(())
 }
 
-fn process_kern_hwreq(request: &kern::Message, rank: u8) -> Result<bool, Error> {
+fn process_kern_hwreq(request: &kern::Message, self_destination: u8) -> Result<bool, Error> {
     match request {
         &kern::RtioInitRequest => {
             unsafe {
@@ -774,7 +955,7 @@ fn process_kern_hwreq(request: &kern::Message, rank: u8) -> Result<bool, Error>
             // only local destination is considered "up"
             // no access to other DRTIO destinations
             kern_send(&kern::RtioDestinationStatusReply { 
-                up: destination == rank })
+                up: destination == self_destination })
         }
 
         &kern::I2cStartRequest { busno } => {

artiq/firmware/satman/main.rs

@@ -1,4 +1,4 @@
-#![feature(never_type, panic_info_message, llvm_asm, default_alloc_error_handler, try_trait)]
+#![feature(never_type, panic_info_message, asm, default_alloc_error_handler)]
 #![no_std]
 
 #[macro_use]
@@ -17,6 +17,10 @@ use core::convert::TryFrom;
 use board_misoc::{csr, ident, clock, uart_logger, i2c, pmp};
 #[cfg(has_si5324)]
 use board_artiq::si5324;
+#[cfg(has_si549)]
+use board_artiq::si549;
+#[cfg(soc_platform = "kasli")]
+use board_misoc::irq;
 use board_artiq::{spi, drtioaux, drtio_routing};
 #[cfg(soc_platform = "efc")]
 use board_artiq::ad9117;
@@ -32,6 +36,7 @@ use analyzer::Analyzer;
 static mut ALLOC: alloc_list::ListAlloc = alloc_list::EMPTY;
 
 mod repeater;
+mod routing;
 mod dma;
 mod analyzer;
 mod kernel;
@@ -65,6 +70,7 @@ fn drtiosat_tsc_loaded() -> bool {
     }
 }
 
+#[derive(Clone, Copy)]
 pub enum RtioMaster {
     Drtio,
     Dma,
@@ -82,6 +88,16 @@ pub fn cricon_select(master: RtioMaster) {
     }
 }
 
+pub fn cricon_read() -> RtioMaster {
+    let val = unsafe { csr::cri_con::selected_read() };
+    match val {
+        0 => RtioMaster::Drtio,
+        1 => RtioMaster::Dma,
+        2 => RtioMaster::Kernel,
+        _ => unreachable!()
+    }
+}
+
 #[cfg(has_drtio_routing)]
 macro_rules! forward {
     ($routing_table:expr, $destination:expr, $rank:expr, $repeaters:expr, $packet:expr) => {{
@@ -89,7 +105,14 @@ macro_rules! forward {
         if hop != 0 {
             let repno = (hop - 1) as usize;
             if repno < $repeaters.len() {
-                return $repeaters[repno].aux_forward($packet);
+                if $packet.expects_response() {
+                    return $repeaters[repno].aux_forward($packet);
+                } else {
+                    let res = $repeaters[repno].aux_send($packet);
+                    // allow the satellite to parse the packet before next
+                    clock::spin_us(10_000);
+                    return res;
+                }
             } else {
                 return Err(drtioaux::Error::RoutingError);
             }
@@ -103,8 +126,9 @@ macro_rules! forward {
 }
 
 fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmgr: &mut KernelManager,
-        _repeaters: &mut [repeater::Repeater], _routing_table: &mut drtio_routing::RoutingTable, _rank: &mut u8,
-        packet: drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
+        _repeaters: &mut [repeater::Repeater], _routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8,
+        router: &mut routing::Router, self_destination: &mut u8, packet: drtioaux::Packet
+) -> Result<(), drtioaux::Error<!>> {
     // In the code below, *_chan_sel_write takes an u8 if there are fewer than 256 channels,
     // and u16 otherwise; hence the `as _` conversion.
     match packet {
@@ -125,61 +149,43 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
 
         drtioaux::Packet::DestinationStatusRequest { destination } => {
             #[cfg(has_drtio_routing)]
-            let hop = _routing_table.0[destination as usize][*_rank as usize];
+            let hop = _routing_table.0[destination as usize][*rank as usize];
             #[cfg(not(has_drtio_routing))]
             let hop = 0;
 
             if hop == 0 {
-                // async messages
-                if let Some(status) = dmamgr.get_status() {
-                    info!("playback done, error: {}, channel: {}, timestamp: {}", status.error, status.channel, status.timestamp);
-                    drtioaux::send(0, &drtioaux::Packet::DmaPlaybackStatus { 
-                        destination: destination, id: status.id, error: status.error, channel: status.channel, timestamp: status.timestamp })?;
-                } else if let Some(subkernel_finished) = kernelmgr.get_last_finished() {
-                    info!("subkernel {} finished, with exception: {}", subkernel_finished.id, subkernel_finished.with_exception);
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelFinished {
-                        id: subkernel_finished.id, with_exception: subkernel_finished.with_exception
-                    })?;
-                } else if kernelmgr.message_is_ready() {
-                    let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
-                    let meta = kernelmgr.message_get_slice(&mut data_slice).unwrap();
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelMessage {
-                        destination: destination, id: kernelmgr.get_current_id().unwrap(),
-                        status: meta.status, length: meta.len as u16, data: data_slice
-                    })?;
-                } else {
-                    let errors;
+                *self_destination = destination;
+                let errors;
+                unsafe {
+                    errors = csr::drtiosat::rtio_error_read();
+                }
+                if errors & 1 != 0 {
+                    let channel;
                     unsafe {
-                        errors = csr::drtiosat::rtio_error_read();
+                        channel = csr::drtiosat::sequence_error_channel_read();
+                        csr::drtiosat::rtio_error_write(1);
                     }
-                    if errors & 1 != 0 {
-                        let channel;
-                        unsafe {
-                            channel = csr::drtiosat::sequence_error_channel_read();
-                            csr::drtiosat::rtio_error_write(1);
-                        }
-                        drtioaux::send(0,
-                            &drtioaux::Packet::DestinationSequenceErrorReply { channel })?;
-                    } else if errors & 2 != 0 {
-                        let channel;
-                        unsafe {
-                            channel = csr::drtiosat::collision_channel_read();
-                            csr::drtiosat::rtio_error_write(2);
-                        }
-                        drtioaux::send(0,
-                            &drtioaux::Packet::DestinationCollisionReply { channel })?;
-                    } else if errors & 4 != 0 {
-                        let channel;
-                        unsafe {
-                            channel = csr::drtiosat::busy_channel_read();
-                            csr::drtiosat::rtio_error_write(4);
-                        }
-                        drtioaux::send(0,
-                            &drtioaux::Packet::DestinationBusyReply { channel })?;
+                    drtioaux::send(0,
+                        &drtioaux::Packet::DestinationSequenceErrorReply { channel })?;
+                } else if errors & 2 != 0 {
+                    let channel;
+                    unsafe {
+                        channel = csr::drtiosat::collision_channel_read();
+                        csr::drtiosat::rtio_error_write(2);
                     }
-                    else {
-                        drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
+                    drtioaux::send(0,
+                        &drtioaux::Packet::DestinationCollisionReply { channel })?;
+                } else if errors & 4 != 0 {
+                    let channel;
+                    unsafe {
+                        channel = csr::drtiosat::busy_channel_read();
+                        csr::drtiosat::rtio_error_write(4);
                     }
+                    drtioaux::send(0,
+                        &drtioaux::Packet::DestinationBusyReply { channel })?;
+                }
+                else {
+                    drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
                 }
             }
 
@@ -204,7 +210,6 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                     }
                 }
             }
-
             Ok(())
         }
 
@@ -219,18 +224,18 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             drtioaux::send(0, &drtioaux::Packet::RoutingAck)
         }
         #[cfg(has_drtio_routing)]
-        drtioaux::Packet::RoutingSetRank { rank } => {
-            *_rank = rank;
-            drtio_routing::interconnect_enable_all(_routing_table, rank);
+        drtioaux::Packet::RoutingSetRank { rank: new_rank } => {
+            *rank = new_rank;
+            drtio_routing::interconnect_enable_all(_routing_table, new_rank);
 
-            let rep_rank = rank + 1;
+            let rep_rank = new_rank + 1;
             for rep in _repeaters.iter() {
                 if let Err(e) = rep.set_rank(rep_rank) {
                     error!("failed to set rank ({})", e);
                 }
             }
 
-            info!("rank: {}", rank);
+            info!("rank: {}", new_rank);
             info!("routing table: {}", _routing_table);
 
             drtioaux::send(0, &drtioaux::Packet::RoutingAck)
@@ -246,7 +251,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::MonitorRequest { destination: _destination, channel, probe } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let value;
             #[cfg(has_rtio_moninj)]
             unsafe {
@@ -263,7 +268,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             drtioaux::send(0, &reply)
         },
         drtioaux::Packet::InjectionRequest { destination: _destination, channel, overrd, value } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             #[cfg(has_rtio_moninj)]
             unsafe {
                 csr::rtio_moninj::inj_chan_sel_write(channel as _);
@@ -273,7 +278,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             Ok(())
         },
         drtioaux::Packet::InjectionStatusRequest { destination: _destination, channel, overrd } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let value;
             #[cfg(has_rtio_moninj)]
             unsafe {
@@ -289,22 +294,22 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         },
 
         drtioaux::Packet::I2cStartRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = i2c::start(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cRestartRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = i2c::restart(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cStopRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = i2c::stop(busno).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::I2cWriteRequest { destination: _destination, busno, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             match i2c::write(busno, data) {
                 Ok(ack) => drtioaux::send(0,
                     &drtioaux::Packet::I2cWriteReply { succeeded: true, ack: ack }),
@@ -313,7 +318,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             }
         }
         drtioaux::Packet::I2cReadRequest { destination: _destination, busno, ack } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             match i2c::read(busno, ack) {
                 Ok(data) => drtioaux::send(0,
                     &drtioaux::Packet::I2cReadReply { succeeded: true, data: data }),
@@ -322,25 +327,25 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             }
         }
         drtioaux::Packet::I2cSwitchSelectRequest { destination: _destination, busno, address, mask } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = i2c::switch_select(busno, address, mask).is_ok();
             drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
         }
 
         drtioaux::Packet::SpiSetConfigRequest { destination: _destination, busno, flags, length, div, cs } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = spi::set_config(busno, flags, length, div, cs).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SpiBasicReply { succeeded: succeeded })
         },
         drtioaux::Packet::SpiWriteRequest { destination: _destination, busno, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let succeeded = spi::write(busno, data).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SpiBasicReply { succeeded: succeeded })
         }
         drtioaux::Packet::SpiReadRequest { destination: _destination, busno } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             match spi::read(busno) {
                 Ok(data) => drtioaux::send(0,
                     &drtioaux::Packet::SpiReadReply { succeeded: true, data: data }),
@@ -350,7 +355,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::AnalyzerHeaderRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let header = analyzer.get_header();
             drtioaux::send(0, &drtioaux::Packet::AnalyzerHeader {
                 total_byte_count: header.total_byte_count,
@@ -360,7 +365,7 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
         }
 
         drtioaux::Packet::AnalyzerDataRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
             let meta = analyzer.get_data(&mut data_slice);
             drtioaux::send(0, &drtioaux::Packet::AnalyzerData {
@@ -370,34 +375,56 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
             })
         }
 
-        drtioaux::Packet::DmaAddTraceRequest { destination: _destination, id, status, length, trace } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
-            let succeeded = dmamgr.add(id, status, &trace, length as usize).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaAddTraceReply { succeeded: succeeded })
+        drtioaux::Packet::DmaAddTraceRequest { source, destination, id, status, length, trace } => {
+            forward!(_routing_table, destination, *rank, _repeaters, &packet);
+            *self_destination = destination;
+            let succeeded = dmamgr.add(source, id, status, &trace, length as usize).is_ok();
+            router.send(drtioaux::Packet::DmaAddTraceReply { 
+                source: *self_destination, destination: source, id: id, succeeded: succeeded 
+            }, _routing_table, *rank, *self_destination)
         }
-        drtioaux::Packet::DmaRemoveTraceRequest { destination: _destination, id } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
-            let succeeded = dmamgr.erase(id).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaRemoveTraceReply { succeeded: succeeded })
+        drtioaux::Packet::DmaAddTraceReply { source, destination: _destination, id, succeeded } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            dmamgr.ack_upload(kernelmgr, source, id, succeeded, router, *rank, *self_destination, _routing_table);
+            Ok(())
         }
-        drtioaux::Packet::DmaPlaybackRequest { destination: _destination, id, timestamp } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::DmaRemoveTraceRequest { source, destination: _destination, id } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            let succeeded = dmamgr.erase(source, id).is_ok();
+            router.send(drtioaux::Packet::DmaRemoveTraceReply { 
+                destination: source, succeeded: succeeded 
+            }, _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::DmaPlaybackRequest { source, destination: _destination, id, timestamp } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             // no DMA with a running kernel
-            let succeeded = !kernelmgr.is_running() && dmamgr.playback(id, timestamp).is_ok();
-            drtioaux::send(0,
-                &drtioaux::Packet::DmaPlaybackReply { succeeded: succeeded })
+            let succeeded = !kernelmgr.is_running() && dmamgr.playback(source, id, timestamp).is_ok();
+            router.send(drtioaux::Packet::DmaPlaybackReply { 
+                destination: source, succeeded: succeeded
+            }, _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::DmaPlaybackReply { destination: _destination, succeeded } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            if !succeeded {
+                kernelmgr.ddma_nack();
+            }
+            Ok(())
+        }
+        drtioaux::Packet::DmaPlaybackStatus { source: _, destination: _destination, id, error, channel, timestamp } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            dmamgr.remote_finished(kernelmgr, id, error, channel, timestamp);
+            Ok(())
         }
 
-        drtioaux::Packet::SubkernelAddDataRequest { destination: _destination, id, status, length, data } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::SubkernelAddDataRequest { destination, id, status, length, data } => {
+            forward!(_routing_table, destination, *rank, _repeaters, &packet);
+            *self_destination = destination;
             let succeeded = kernelmgr.add(id, status, &data, length as usize).is_ok();
             drtioaux::send(0,
                 &drtioaux::Packet::SubkernelAddDataReply { succeeded: succeeded })
         }
-        drtioaux::Packet::SubkernelLoadRunRequest { destination: _destination, id, run } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+        drtioaux::Packet::SubkernelLoadRunRequest { source, destination: _destination, id, run } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let mut succeeded = kernelmgr.load(id).is_ok();
             // allow preloading a kernel with delayed run
             if run {
@@ -405,14 +432,27 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                     // cannot run kernel while DDMA is running
                     succeeded = false;
                 } else {
-                    succeeded |= kernelmgr.run(id).is_ok();
+                    succeeded |= kernelmgr.run(source, id).is_ok();
                 }
             }
-            drtioaux::send(0,
-                &drtioaux::Packet::SubkernelLoadRunReply { succeeded: succeeded })
+            router.send(drtioaux::Packet::SubkernelLoadRunReply { 
+                    destination: source, succeeded: succeeded 
+                }, 
+            _routing_table, *rank, *self_destination)
+        }
+        drtioaux::Packet::SubkernelLoadRunReply { destination: _destination, succeeded } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            // received if local subkernel started another, remote subkernel
+            kernelmgr.subkernel_load_run_reply(succeeded, *self_destination);
+            Ok(())
+        }
+        drtioaux::Packet::SubkernelFinished { destination: _destination, id, with_exception, exception_src } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            kernelmgr.remote_subkernel_finished(id, with_exception, exception_src);
+            Ok(())
         }
         drtioaux::Packet::SubkernelExceptionRequest { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
             let meta = kernelmgr.exception_get_slice(&mut data_slice);
             drtioaux::send(0, &drtioaux::Packet::SubkernelException {
@@ -421,22 +461,23 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
                 data: data_slice,
             })
         }
-        drtioaux::Packet::SubkernelMessage { destination, id: _id, status, length, data } => {
-            forward!(_routing_table, destination, *_rank, _repeaters, &packet);
-            kernelmgr.message_handle_incoming(status, length as usize, &data);
-            drtioaux::send(0, &drtioaux::Packet::SubkernelMessageAck {
-                destination: destination
-            })
+        drtioaux::Packet::SubkernelMessage { source, destination: _destination, id, status, length, data } => {
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
+            kernelmgr.message_handle_incoming(status, length as usize, id, &data);
+            router.send(drtioaux::Packet::SubkernelMessageAck {
+                    destination: source
+                }, _routing_table, *rank, *self_destination)
         }
         drtioaux::Packet::SubkernelMessageAck { destination: _destination } => {
-            forward!(_routing_table, _destination, *_rank, _repeaters, &packet);
+            forward!(_routing_table, _destination, *rank, _repeaters, &packet);
             if kernelmgr.message_ack_slice() {
                 let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
                 if let Some(meta) = kernelmgr.message_get_slice(&mut data_slice) {
-                    drtioaux::send(0, &drtioaux::Packet::SubkernelMessage {
-                        destination: *_rank, id: kernelmgr.get_current_id().unwrap(),
+                    // route and not send immediately as ACKs are not a beginning of a transaction
+                    router.route(drtioaux::Packet::SubkernelMessage {
+                        source: *self_destination, destination: meta.destination, id: kernelmgr.get_current_id().unwrap(),
                         status: meta.status, length: meta.len as u16, data: data_slice
-                    })?
+                    }, _routing_table, *rank, *self_destination);
                 } else {
                     error!("Error receiving message slice");
                 }
@@ -453,18 +494,19 @@ fn process_aux_packet(dmamgr: &mut DmaManager, analyzer: &mut Analyzer, kernelmg
 
 fn process_aux_packets(dma_manager: &mut DmaManager, analyzer: &mut Analyzer,
         kernelmgr: &mut KernelManager, repeaters: &mut [repeater::Repeater],
-        routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8) {
+        routing_table: &mut drtio_routing::RoutingTable, rank: &mut u8, router: &mut routing::Router,
+        destination: &mut u8) {
     let result =
         drtioaux::recv(0).and_then(|packet| {
-            if let Some(packet) = packet {
-                process_aux_packet(dma_manager, analyzer, kernelmgr, repeaters, routing_table, rank, packet)
+            if let Some(packet) = packet.or_else(|| router.get_local_packet()) {
+                process_aux_packet(dma_manager, analyzer, kernelmgr, 
+                    repeaters, routing_table, rank, router, destination, packet)
             } else {
                 Ok(())
             }
         });
-    match result {
-        Ok(()) => (),
-        Err(e) => warn!("aux packet error ({})", e)
+    if let Err(e) = result {
+        warn!("aux packet error ({})", e);
     }
 }
 
@@ -557,6 +599,36 @@ const SI5324_SETTINGS: si5324::FrequencySettings
     crystal_as_ckin2: true
 };
 
+#[cfg(all(has_si549, rtio_frequency = "125.0"))]
+const SI549_SETTINGS: si549::FrequencySetting = si549::FrequencySetting {
+    main: si549::DividerConfig {
+        hsdiv: 0x058,
+        lsdiv: 0,
+        fbdiv: 0x04815791F25,
+    },
+    helper: si549::DividerConfig {
+        // 125MHz*32767/32768
+        hsdiv: 0x058,
+        lsdiv: 0,
+        fbdiv: 0x04814E8F442,
+    },
+};
+
+#[cfg(all(has_si549, rtio_frequency = "100.0"))]
+const SI549_SETTINGS: si549::FrequencySetting = si549::FrequencySetting {
+    main: si549::DividerConfig {
+        hsdiv: 0x06C,
+        lsdiv: 0,
+        fbdiv: 0x046C5F49797,
+    },
+    helper: si549::DividerConfig {
+        // 100MHz*32767/32768
+        hsdiv: 0x06C,
+        lsdiv: 0,
+        fbdiv: 0x046C5670BBD,
+    },
+};
+
 #[cfg(not(soc_platform = "efc"))]
 fn sysclk_setup() {
     let switched = unsafe {
@@ -569,6 +641,9 @@ fn sysclk_setup() {
     else {
         #[cfg(has_si5324)]
         si5324::setup(&SI5324_SETTINGS, si5324::Input::Ckin1).expect("cannot initialize Si5324");
+        #[cfg(has_si549)]
+        si549::main_setup(&SI549_SETTINGS).expect("cannot initialize main Si549");
+
         info!("Switching sys clock, rebooting...");
         // delay for clean UART log, wait until UART FIFO is empty
         clock::spin_us(3000);
@@ -592,6 +667,10 @@ pub extern fn main() -> i32 {
         ALLOC.add_range(&mut _fheap, &mut _eheap);
         pmp::init_stack_guard(&_sstack_guard as *const u8 as usize);
     }
+    #[cfg(soc_platform = "kasli")]
+    irq::enable_interrupts();
+    #[cfg(has_wrpll)]
+    irq::enable(csr::WRPLL_INTERRUPT);
 
     clock::init();
     uart_logger::ConsoleLogger::register();
@@ -627,10 +706,27 @@ pub extern fn main() -> i32 {
     #[cfg(not(soc_platform = "efc"))]
     sysclk_setup();
 
+    #[cfg(has_si549)]
+    si549::helper_setup(&SI549_SETTINGS).expect("cannot initialize helper Si549");    
+
     #[cfg(soc_platform = "efc")]
     let mut io_expander;
     #[cfg(soc_platform = "efc")]
     {
+        let p3v3_fmc_en_pin;
+        let vadj_fmc_en_pin;
+
+        #[cfg(hw_rev = "v1.0")]
+        {
+            p3v3_fmc_en_pin = 0;
+            vadj_fmc_en_pin = 1;
+        }
+        #[cfg(hw_rev = "v1.1")]
+        {
+            p3v3_fmc_en_pin = 1;
+            vadj_fmc_en_pin = 7;
+        }
+
         io_expander = board_misoc::io_expander::IoExpander::new().unwrap();
         io_expander.init().expect("I2C I/O expander initialization failed");
 
@@ -638,15 +734,15 @@ pub extern fn main() -> i32 {
         io_expander.set_oe(0, 1 << 5 | 1 << 6 | 1 << 7).unwrap();
         
         // Enable VADJ and P3V3_FMC
-        io_expander.set_oe(1, 1 << 0 | 1 << 1).unwrap();
+        io_expander.set_oe(1, 1 << p3v3_fmc_en_pin | 1 << vadj_fmc_en_pin).unwrap();
 
-        io_expander.set(1, 0, true);
-        io_expander.set(1, 1, true);
+        io_expander.set(1, p3v3_fmc_en_pin, true);
+        io_expander.set(1, vadj_fmc_en_pin, true);
 
         io_expander.service().unwrap();
     }
 
-    #[cfg(not(has_drtio_eem))]
+    #[cfg(not(soc_platform = "efc"))]
     unsafe {
         csr::gt_drtio::txenable_write(0xffffffffu32 as _);
     }
@@ -670,6 +766,7 @@ pub extern fn main() -> i32 {
     } 
     let mut routing_table = drtio_routing::RoutingTable::default_empty();
     let mut rank = 1;
+    let mut destination = 1;
 
     let mut hardware_tick_ts = 0;
 
@@ -677,10 +774,12 @@ pub extern fn main() -> i32 {
     ad9117::init().expect("AD9117 initialization failed");
     
     loop {
+        let mut router = routing::Router::new();
+
         while !drtiosat_link_rx_up() {
             drtiosat_process_errors();
             for rep in repeaters.iter_mut() {
-                rep.service(&routing_table, rank);
+                rep.service(&routing_table, rank, destination, &mut router);
             }
             #[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
             {
@@ -699,6 +798,9 @@ pub extern fn main() -> i32 {
             si5324::siphaser::calibrate_skew().expect("failed to calibrate skew");
         }
 
+        #[cfg(has_wrpll)]
+        si549::wrpll::select_recovered_clock(true);
+
         // various managers created here, so when link is dropped, DMA traces,
         // analyzer logs, kernels are cleared and/or stopped for a clean slate
         // on subsequent connections, without a manual intervention.
@@ -714,10 +816,10 @@ pub extern fn main() -> i32 {
         while drtiosat_link_rx_up() {
             drtiosat_process_errors();
             process_aux_packets(&mut dma_manager, &mut analyzer, 
-                &mut kernelmgr, &mut repeaters, 
-                &mut routing_table, &mut rank);
+                &mut kernelmgr, &mut repeaters, &mut routing_table,
+                &mut rank, &mut router, &mut destination);
             for rep in repeaters.iter_mut() {
-                rep.service(&routing_table, rank);
+                rep.service(&routing_table, rank, destination, &mut router);
             }
             #[cfg(all(soc_platform = "kasli", hw_rev = "v2.0"))]
             {
@@ -738,7 +840,26 @@ pub extern fn main() -> i32 {
                     error!("aux packet error: {}", e);
                 }
             }
-            kernelmgr.process_kern_requests(rank);
+            if let Some(status) = dma_manager.get_status() {
+                info!("playback done, error: {}, channel: {}, timestamp: {}", status.error, status.channel, status.timestamp);
+                router.route(drtioaux::Packet::DmaPlaybackStatus { 
+                    source: destination, destination: status.source, id: status.id,
+                    error: status.error, channel: status.channel, timestamp: status.timestamp 
+                }, &routing_table, rank, destination);
+            }
+
+            kernelmgr.process_kern_requests(&mut router, &routing_table, rank, destination, &mut dma_manager);
+            
+            #[cfg(has_drtio_routing)]
+            if let Some((repno, packet)) = router.get_downstream_packet() {
+                if let Err(e) = repeaters[repno].aux_send(&packet) {
+                    warn!("[REP#{}] Error when sending packet to satellite ({:?})", repno, e)
+                }
+            }
+
+            if let Some(packet) = router.get_upstream_packet() {
+                drtioaux::send(0, &packet).unwrap();
+            }
         }
 
         drtiosat_reset_phy(true);
@@ -747,11 +868,27 @@ pub extern fn main() -> i32 {
         info!("uplink is down, switching to local oscillator clock");
         #[cfg(has_si5324)]
         si5324::siphaser::select_recovered_clock(false).expect("failed to switch clocks");
+        #[cfg(has_wrpll)]
+        si549::wrpll::select_recovered_clock(false);
     }
 }
 
 #[cfg(soc_platform = "efc")]
 fn enable_error_led() {
+    let p3v3_fmc_en_pin;
+    let vadj_fmc_en_pin;
+
+    #[cfg(hw_rev = "v1.0")]
+    {
+        p3v3_fmc_en_pin = 0;
+        vadj_fmc_en_pin = 1;
+    }
+    #[cfg(hw_rev = "v1.1")]
+    {
+        p3v3_fmc_en_pin = 1;
+        vadj_fmc_en_pin = 7;
+    }
+
     let mut io_expander = board_misoc::io_expander::IoExpander::new().unwrap();
 
     // Keep LEDs enabled
@@ -760,10 +897,10 @@ fn enable_error_led() {
     io_expander.set(0, 7, true);
 
     // Keep VADJ and P3V3_FMC enabled
-    io_expander.set_oe(1, 1 << 0 | 1 << 1).unwrap();
+    io_expander.set_oe(1, 1 << p3v3_fmc_en_pin | 1 << vadj_fmc_en_pin).unwrap();
 
-    io_expander.set(1, 0, true);
-    io_expander.set(1, 1, true);
+    io_expander.set(1, p3v3_fmc_en_pin, true);
+    io_expander.set(1, vadj_fmc_en_pin, true);
 
     io_expander.service().unwrap();
 }
@@ -772,23 +909,33 @@ fn enable_error_led() {
 pub extern fn exception(_regs: *const u32) {
     let pc = mepc::read();
     let cause = mcause::read().cause();
-    
-    fn hexdump(addr: u32) {
-        let addr = (addr - addr % 4) as *const u32;
-        let mut ptr  = addr;
-        println!("@ {:08p}", ptr);
-        for _ in 0..4 {
-            print!("+{:04x}: ", ptr as usize - addr as usize);
-            print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
-            print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
-            print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
-            print!("{:08x}\n",  unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
+    match cause {
+        mcause::Trap::Interrupt(_source) => {
+            #[cfg(has_wrpll)]
+            if irq::is_pending(csr::WRPLL_INTERRUPT) {
+                si549::wrpll::interrupt_handler();
+            }
+        },
+
+        mcause::Trap::Exception(e) => {
+            fn hexdump(addr: u32) {
+                let addr = (addr - addr % 4) as *const u32;
+                let mut ptr  = addr;
+                println!("@ {:08p}", ptr);
+                for _ in 0..4 {
+                    print!("+{:04x}: ", ptr as usize - addr as usize);
+                    print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
+                    print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
+                    print!("{:08x} ",   unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
+                    print!("{:08x}\n",  unsafe { *ptr }); ptr = ptr.wrapping_offset(1);
+                }
+            }
+
+            hexdump(u32::try_from(pc).unwrap());
+            let mtval = mtval::read();
+            panic!("exception {:?} at PC 0x{:x}, trap value 0x{:x}", e, u32::try_from(pc).unwrap(), mtval)
         }
     }
-
-    hexdump(u32::try_from(pc).unwrap());
-    let mtval = mtval::read();
-    panic!("exception {:?} at PC 0x{:x}, trap value 0x{:x}", cause, u32::try_from(pc).unwrap(), mtval)
 }
 
 #[no_mangle]

artiq/firmware/satman/repeater.rs

@@ -1,6 +1,7 @@
 use board_artiq::{drtioaux, drtio_routing};
 #[cfg(has_drtio_routing)]
 use board_misoc::{csr, clock};
+use routing::Router;
 
 #[cfg(has_drtio_routing)]
 fn rep_link_rx_up(repno: u8) -> bool {
@@ -48,7 +49,7 @@ impl Repeater {
         self.state == RepeaterState::Up
     }
 
-    pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8) {
+    pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8, self_destination: u8, router: &mut Router) {
         self.process_local_errors();
 
         match self.state {
@@ -106,7 +107,7 @@ impl Repeater {
                 }
             }
             RepeaterState::Up => {
-                self.process_unsolicited_aux();
+                self.process_unsolicited_aux(routing_table, rank, self_destination, router);
                 if !rep_link_rx_up(self.repno) {
                     info!("[REP#{}] link is down", self.repno);
                     self.state = RepeaterState::Down;
@@ -121,9 +122,10 @@ impl Repeater {
         }
     }
 
-    fn process_unsolicited_aux(&self) {
+    fn process_unsolicited_aux(&self, routing_table: &drtio_routing::RoutingTable, 
+        rank: u8, self_destination: u8, router: &mut Router) {
         match drtioaux::recv(self.auxno) {
-            Ok(Some(packet)) => warn!("[REP#{}] unsolicited aux packet: {:?}", self.repno, packet),
+            Ok(Some(packet)) => router.route(packet, routing_table, rank, self_destination),
             Ok(None) => (),
             Err(_) => warn!("[REP#{}] aux packet error", self.repno)
         }
@@ -180,15 +182,19 @@ impl Repeater {
     }
 
     pub fn aux_forward(&self, request: &drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
-        if self.state != RepeaterState::Up {
-            return Err(drtioaux::Error::LinkDown);
-        }
-        drtioaux::send(self.auxno, request).unwrap();
+        self.aux_send(request)?;
         let reply = self.recv_aux_timeout(200)?;
         drtioaux::send(0, &reply).unwrap();
         Ok(())
     }
 
+    pub fn aux_send(&self, request: &drtioaux::Packet) -> Result<(), drtioaux::Error<!>> {
+        if self.state != RepeaterState::Up {
+            return Err(drtioaux::Error::LinkDown);
+        }
+        drtioaux::send(self.auxno, request)
+    }
+
     pub fn sync_tsc(&self) -> Result<(), drtioaux::Error<!>> {
         if self.state != RepeaterState::Up {
             return Ok(());
@@ -199,7 +205,6 @@ impl Repeater {
             (csr::DRTIOREP[repno].set_time_write)(1);
             while (csr::DRTIOREP[repno].set_time_read)() == 1 {}
         }
-
         // TSCAck is the only aux packet that is sent spontaneously
         // by the satellite, in response to a TSC set on the RT link.
         let reply = self.recv_aux_timeout(10000)?;
@@ -275,7 +280,7 @@ pub struct Repeater {
 impl Repeater {
     pub fn new(_repno: u8) -> Repeater { Repeater::default() }
 
-    pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8) { }
+    pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8, _destination: u8, _router: &mut Router) { }
 
     pub fn sync_tsc(&self) -> Result<(), drtioaux::Error<!>> { Ok(()) }
 

artiq/firmware/satman/routing.rs

@@ -0,0 +1,169 @@
+use alloc::{vec::Vec, collections::vec_deque::VecDeque};
+use board_artiq::{drtioaux, drtio_routing};
+#[cfg(has_drtio_routing)]
+use board_misoc::csr;
+use core::cmp::min;
+use proto_artiq::drtioaux_proto::PayloadStatus;
+use SAT_PAYLOAD_MAX_SIZE;
+use MASTER_PAYLOAD_MAX_SIZE;
+
+/* represents data that has to be sent with the aux protocol */
+#[derive(Debug)]
+pub struct Sliceable {
+    it: usize,
+    data: Vec<u8>,
+    destination: u8
+}
+
+pub struct SliceMeta {
+    pub destination: u8,
+    pub len: u16,
+    pub status: PayloadStatus
+}
+
+macro_rules! get_slice_fn {
+    ( $name:tt, $size:expr ) => {
+        pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
+            let first = self.it == 0;
+            let len = min($size, self.data.len() - self.it);
+            let last = self.it + len == self.data.len();
+            let status = PayloadStatus::from_status(first, last);
+            data_slice[..len].clone_from_slice(&self.data[self.it..self.it+len]);
+            self.it += len;
+    
+            SliceMeta {
+                destination: self.destination,
+                len: len as u16,
+                status: status
+            }
+        }
+    };
+}
+
+impl Sliceable {
+    pub fn new(destination: u8, data: Vec<u8>) -> Sliceable {
+        Sliceable {
+            it: 0,
+            data: data,
+            destination: destination
+        }
+    }
+
+    pub fn at_end(&self) -> bool {
+        self.it == self.data.len()
+    }
+
+    pub fn extend(&mut self, data: &[u8]) {
+        self.data.extend(data);
+    }
+
+    get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
+    get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
+}
+
+// Packets from downstream (further satellites) are received and routed appropriately.
+// they're passed as soon as possible downstream (within the subtree), or sent upstream,
+// which is notified about pending packets.
+// for rank 1 (connected to master) satellites, these packets are passed as an answer to DestinationStatusRequest;
+// for higher ranks, after getting a notification, it will transact with downstream to get the pending packets.
+
+// forward! macro is not deprecated, as routable packets are only these that can originate
+// from both master and satellite, e.g. DDMA and Subkernel.
+
+pub struct Router {
+    upstream_queue: VecDeque<drtioaux::Packet>, 
+    local_queue: VecDeque<drtioaux::Packet>,
+    #[cfg(has_drtio_routing)]
+    downstream_queue: VecDeque<(usize, drtioaux::Packet)>,
+}
+
+impl Router {
+    pub fn new() -> Router {
+        Router {
+            upstream_queue: VecDeque::new(),
+            local_queue: VecDeque::new(),
+            #[cfg(has_drtio_routing)]
+            downstream_queue: VecDeque::new(),
+        }
+    }
+
+    // called by local sources (DDMA, kernel) and by repeaters on receiving async data
+    // messages are always buffered for both upstream and downstream
+    pub fn route(&mut self, packet: drtioaux::Packet,
+        _routing_table: &drtio_routing::RoutingTable, _rank: u8,
+        self_destination: u8
+    ) {
+        let destination = packet.routable_destination();
+        #[cfg(has_drtio_routing)]
+        {
+            if let Some(destination) = destination {
+                let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
+                if destination == self_destination {
+                    self.local_queue.push_back(packet);
+                } else if hop > 0 && hop < csr::DRTIOREP.len() {
+                    let repno = (hop - 1) as usize;
+                    self.downstream_queue.push_back((repno, packet));
+                } else {
+                    self.upstream_queue.push_back(packet);
+                }
+            } else {
+                error!("Received an unroutable packet: {:?}", packet);
+            }
+        }
+        #[cfg(not(has_drtio_routing))]
+        {
+            if destination == Some(self_destination) {
+                self.local_queue.push_back(packet);
+            } else {
+                self.upstream_queue.push_back(packet);
+            }
+        }
+    }
+
+    // Sends a packet to a required destination, routing if it's necessary
+    pub fn send(&mut self, packet: drtioaux::Packet,
+        _routing_table: &drtio_routing::RoutingTable,
+        _rank: u8, _destination: u8
+    ) -> Result<(), drtioaux::Error<!>> {
+        #[cfg(has_drtio_routing)]
+        {
+            let destination = packet.routable_destination();
+            if let Some(destination) = destination {
+                let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
+                if destination == 0 {
+                    // response is needed immediately if master required it
+                    drtioaux::send(0, &packet)?;
+                } else if !(hop > 0 && hop < csr::DRTIOREP.len()) {
+                    // higher rank can wait
+                    self.upstream_queue.push_back(packet);
+                } else {
+                    let repno = (hop - 1) as usize;
+                    // transaction will occur at closest possible opportunity
+                    self.downstream_queue.push_back((repno, packet));
+                }
+                Ok(())
+            } else {
+                // packet not supported in routing, fallback - sent directly
+                drtioaux::send(0, &packet)
+            }
+        }
+        #[cfg(not(has_drtio_routing))]
+        {
+            drtioaux::send(0, &packet)
+        }
+    }
+
+    pub fn get_upstream_packet(&mut self) -> Option<drtioaux::Packet> {
+        let packet = self.upstream_queue.pop_front();
+        packet
+    }
+
+    #[cfg(has_drtio_routing)]
+    pub fn get_downstream_packet(&mut self) -> Option<(usize, drtioaux::Packet)> {
+        self.downstream_queue.pop_front()
+    }
+
+    pub fn get_local_packet(&mut self) -> Option<drtioaux::Packet> {
+        self.local_queue.pop_front()
+    }
+}

artiq/frontend/aqctl_coreanalyzer_proxy.py

@@ -9,7 +9,7 @@ from sipyco.asyncio_tools import AsyncioServer, SignalHandler, atexit_register_c
 from sipyco.pc_rpc import Server
 from sipyco import common_args
 
-from artiq.coredevice.comm_analyzer import get_analyzer_dump
+from artiq.coredevice.comm_analyzer import get_analyzer_dump, ANALYZER_MAGIC
 
 
 logger = logging.getLogger(__name__)
@@ -24,6 +24,7 @@ class ProxyServer(AsyncioServer):
 
     async def _handle_connection_cr(self, reader, writer):
         try:
+            writer.write(ANALYZER_MAGIC)
             queue = asyncio.Queue(self._queue_limit)
             self._recipients.add(queue)
             try:
@@ -60,9 +61,7 @@ class ProxyControl:
             self.distribute_cb(dump)
         except:
             logger.warning("Trigger failed:", exc_info=True)
-            return False
-        else:
-            return True
+            raise
 
 
 def get_argparser():

artiq/frontend/artiq_client.py

@@ -22,6 +22,7 @@ from sipyco.pc_rpc import Client
 from sipyco.sync_struct import Subscriber
 from sipyco.broadcast import Receiver
 from sipyco import common_args, pyon
+from sipyco.asyncio_tools import SignalHandler
 
 from artiq.tools import (scale_from_metadata, short_format, parse_arguments,
                          parse_devarg_override)
@@ -112,11 +113,25 @@ def get_argparser():
         "del-dataset", help="delete a dataset")
     parser_del_dataset.add_argument("name", help="name of the dataset")
 
+    parser_supply_interactive = subparsers.add_parser(
+        "supply-interactive", help="supply interactive arguments")
+    parser_supply_interactive.add_argument(
+        "rid", metavar="RID", type=int, help="RID of target experiment")
+    parser_supply_interactive.add_argument(
+        "arguments", metavar="ARGUMENTS", nargs="*",
+        help="interactive arguments")
+
+    parser_cancel_interactive = subparsers.add_parser(
+        "cancel-interactive", help="cancel interactive arguments")
+    parser_cancel_interactive.add_argument(
+        "rid", metavar="RID", type=int, help="RID of target experiment")
+
     parser_show = subparsers.add_parser(
         "show", help="show schedule, log, devices or datasets")
     parser_show.add_argument(
         "what", metavar="WHAT",
-        choices=["schedule", "log", "ccb", "devices", "datasets"],
+        choices=["schedule", "log", "ccb", "devices", "datasets",
+                 "interactive-args"],
         help="select object to show: %(choices)s")
 
     subparsers.add_parser(
@@ -135,8 +150,7 @@ def get_argparser():
         "ls", help="list a directory on the master")
     parser_ls.add_argument("directory", default="", nargs="?")
 
-    subparsers.add_parser(
-        "terminate", help="terminate the ARTIQ master")
+    subparsers.add_parser("terminate", help="terminate the ARTIQ master")
 
     common_args.verbosity_args(parser)
     return parser
@@ -208,6 +222,15 @@ def _action_scan_devices(remote, args):
     remote.scan()
 
 
+def _action_supply_interactive(remote, args):
+    arguments = parse_arguments(args.arguments)
+    remote.supply(args.rid, arguments)
+
+
+def _action_cancel_interactive(remote, args):
+    remote.cancel(args.rid)
+
+
 def _action_scan_repository(remote, args):
     if getattr(args, "async"):
         remote.scan_repository_async(args.revision)
@@ -274,17 +297,34 @@ def _show_datasets(datasets):
     print(table)
 
 
+def _show_interactive_args(interactive_args):
+    clear_screen()
+    table = PrettyTable(["RID", "Title", "Key", "Type", "Group", "Tooltip"])
+    for rid, input_request in sorted(interactive_args.items(), key=itemgetter(0)):
+        title = input_request["title"]
+        for key, procdesc, group, tooltip in input_request["arglist_desc"]:
+            table.add_row([rid, title, key, procdesc["ty"], group, tooltip])
+    print(table)
+
+
 def _run_subscriber(host, port, subscriber):
     loop = asyncio.new_event_loop()
     asyncio.set_event_loop(loop)
     try:
-        loop.run_until_complete(subscriber.connect(host, port))
+        signal_handler = SignalHandler()
+        signal_handler.setup()
         try:
-            loop.run_until_complete(asyncio.wait_for(subscriber.receive_task,
-                                                     None))
-            print("Connection to master lost")
+            loop.run_until_complete(subscriber.connect(host, port))
+            try:
+                _, pending = loop.run_until_complete(asyncio.wait(
+                    [loop.create_task(signal_handler.wait_terminate()), subscriber.receive_task],
+                    return_when=asyncio.FIRST_COMPLETED))
+                for task in pending:
+                    task.cancel()
+            finally:
+                loop.run_until_complete(subscriber.close())
         finally:
-            loop.run_until_complete(subscriber.close())
+            signal_handler.teardown()
     finally:
         loop.close()
 
@@ -338,18 +378,22 @@ def main():
             _show_dict(args, "devices", _show_devices)
         elif args.what == "datasets":
             _show_dict(args, "datasets", _show_datasets)
+        elif args.what == "interactive-args":
+            _show_dict(args, "interactive_args", _show_interactive_args)
         else:
             raise ValueError
     else:
         port = 3251 if args.port is None else args.port
         target_name = {
-            "submit": "master_schedule",
-            "delete": "master_schedule",
-            "set_dataset": "master_dataset_db",
-            "del_dataset": "master_dataset_db",
-            "scan_devices": "master_device_db",
-            "scan_repository": "master_experiment_db",
-            "ls": "master_experiment_db",
+            "submit": "schedule",
+            "delete": "schedule",
+            "set_dataset": "dataset_db",
+            "del_dataset": "dataset_db",
+            "scan_devices": "device_db",
+            "supply_interactive": "interactive_arg_db",
+            "cancel_interactive": "interactive_arg_db",
+            "scan_repository": "experiment_db",
+            "ls": "experiment_db",
             "terminate": "master_management",
         }[action]
         remote = Client(args.server, port, target_name)

artiq/frontend/artiq_compile.py

@@ -67,12 +67,21 @@ def main():
                 core.compile(exp.run, [exp_inst], {},
                              attribute_writeback=False, print_as_rpc=False)
 
-            subkernels = {}
-            for sid, subkernel_fn in object_map.subkernels().items():
-                destination, subkernel_library = core.compile_subkernel(
-                    sid, subkernel_fn, object_map, 
-                    [exp_inst], subkernel_arg_types)
-                subkernels[sid] = (destination, subkernel_library)
+            subkernels = object_map.subkernels()
+            compiled_subkernels = {}
+            while True:
+                new_subkernels = {}
+                for sid, subkernel_fn in subkernels.items():
+                    if sid in compiled_subkernels.keys():
+                        continue
+                    destination, subkernel_library, embedding_map = core.compile_subkernel(
+                        sid, subkernel_fn, object_map, 
+                        [exp_inst], subkernel_arg_types, subkernels)
+                    compiled_subkernels[sid] = (destination, subkernel_library)
+                    new_subkernels.update(embedding_map.subkernels())
+                if new_subkernels == subkernels:
+                    break
+                subkernels.update(new_subkernels)
         except CompileError as error:
             return
         finally:
@@ -107,7 +116,7 @@ def main():
             tar.addfile(main_kernel_info, fileobj=main_kernel_fileobj)
 
             # subkernels as "<sid> <destination>.elf"
-            for sid, (destination, subkernel_library) in subkernels.items():
+            for sid, (destination, subkernel_library) in compiled_subkernels.items():
                 subkernel_fileobj = io.BytesIO(subkernel_library)
                 subkernel_info = tarfile.TarInfo(name="{} {}.elf".format(sid, destination))
                 subkernel_info.size = len(subkernel_library)

artiq/frontend/artiq_dashboard.py

@@ -6,7 +6,6 @@ import atexit
 import importlib
 import os
 import logging
-import sys
 
 from PyQt5 import QtCore, QtGui, QtWidgets
 from qasync import QEventLoop
@@ -15,13 +14,15 @@ from sipyco.pc_rpc import AsyncioClient, Client
 from sipyco.broadcast import Receiver
 from sipyco import common_args
 from sipyco.asyncio_tools import atexit_register_coroutine
+from sipyco.sync_struct import Subscriber
 
 from artiq import __artiq_dir__ as artiq_dir, __version__ as artiq_version
 from artiq.tools import get_user_config_dir
 from artiq.gui.models import ModelSubscriber
 from artiq.gui import state, log
 from artiq.dashboard import (experiments, shortcuts, explorer,
-                             moninj, datasets, schedule, applets_ccb)
+                             moninj, datasets, schedule, applets_ccb,
+                             waveform, interactive_args)
 
 
 def get_argparser():
@@ -47,6 +48,15 @@ def get_argparser():
     parser.add_argument(
         "-p", "--load-plugin", dest="plugin_modules", action="append",
         help="Python module to load on startup")
+    parser.add_argument(
+        "--analyzer-proxy-timeout", default=5, type=float,
+        help="connection timeout to core analyzer proxy")
+    parser.add_argument(
+        "--analyzer-proxy-timer", default=5, type=float,
+        help="retry timer to core analyzer proxy")
+    parser.add_argument(
+        "--analyzer-proxy-timer-backoff", default=1.1, type=float,
+        help="retry timer backoff multiplier to core analyzer proxy")
     common_args.verbosity_args(parser)
     return parser
 
@@ -60,7 +70,7 @@ class MainWindow(QtWidgets.QMainWindow):
         self.setWindowTitle("ARTIQ Dashboard - {}".format(server))
 
         qfm = QtGui.QFontMetrics(self.font())
-        self.resize(140*qfm.averageCharWidth(), 38*qfm.lineSpacing())
+        self.resize(140 * qfm.averageCharWidth(), 38 * qfm.lineSpacing())
 
         self.exit_request = asyncio.Event()
 
@@ -100,8 +110,8 @@ class MdiArea(QtWidgets.QMdiArea):
     def paintEvent(self, event):
         QtWidgets.QMdiArea.paintEvent(self, event)
         painter = QtGui.QPainter(self.viewport())
-        x = (self.width() - self.pixmap.width())//2
-        y = (self.height() - self.pixmap.height())//2
+        x = (self.width() - self.pixmap.width()) // 2
+        y = (self.height() - self.pixmap.height()) // 2
         painter.setOpacity(0.5)
         painter.drawPixmap(x, y, self.pixmap)
 
@@ -118,9 +128,9 @@ def main():
 
     if args.db_file is None:
         args.db_file = os.path.join(get_user_config_dir(),
-                           "artiq_dashboard_{server}_{port}.pyon".format(
-                            server=args.server.replace(":","."),
-                            port=args.port_notify))
+                                    "artiq_dashboard_{server}_{port}.pyon".format(
+                                    server=args.server.replace(":", "."),
+                                    port=args.port_notify))
 
     app = QtWidgets.QApplication(["ARTIQ Dashboard"])
     loop = QEventLoop(app)
@@ -130,10 +140,10 @@ def main():
 
     # create connections to master
     rpc_clients = dict()
-    for target in "schedule", "experiment_db", "dataset_db", "device_db":
+    for target in "schedule", "experiment_db", "dataset_db", "device_db", "interactive_arg_db":
         client = AsyncioClient()
         loop.run_until_complete(client.connect_rpc(
-            args.server, args.port_control, "master_" + target))
+            args.server, args.port_control, target))
         atexit.register(client.close_rpc)
         rpc_clients[target] = client
 
@@ -144,6 +154,7 @@ def main():
         master_management.close_rpc()
 
     disconnect_reported = False
+
     def report_disconnect():
         nonlocal disconnect_reported
         if not disconnect_reported:
@@ -155,9 +166,9 @@ def main():
     for notifier_name, modelf in (("explist", explorer.Model),
                                   ("explist_status", explorer.StatusUpdater),
                                   ("datasets", datasets.Model),
-                                  ("schedule", schedule.Model)):
-        subscriber = ModelSubscriber(notifier_name, modelf,
-            report_disconnect)
+                                  ("schedule", schedule.Model),
+                                  ("interactive_args", interactive_args.Model)):
+        subscriber = ModelSubscriber(notifier_name, modelf, report_disconnect)
         loop.run_until_complete(subscriber.connect(
             args.server, args.port_notify))
         atexit_register_coroutine(subscriber.close, loop=loop)
@@ -215,10 +226,22 @@ def main():
     smgr.register(d_applets)
     broadcast_clients["ccb"].notify_cbs.append(d_applets.ccb_notify)
 
-    d_ttl_dds = moninj.MonInj(rpc_clients["schedule"])
-    loop.run_until_complete(d_ttl_dds.start(args.server, args.port_notify))
+    d_ttl_dds = moninj.MonInj(rpc_clients["schedule"], main_window)
+    smgr.register(d_ttl_dds)
     atexit_register_coroutine(d_ttl_dds.stop, loop=loop)
 
+    d_waveform = waveform.WaveformDock(
+        args.analyzer_proxy_timeout,
+        args.analyzer_proxy_timer,
+        args.analyzer_proxy_timer_backoff
+    )
+    atexit_register_coroutine(d_waveform.stop, loop=loop)
+
+    d_interactive_args = interactive_args.InteractiveArgsDock(
+        sub_clients["interactive_args"],
+        rpc_clients["interactive_arg_db"]
+    )
+
     d_schedule = schedule.ScheduleDock(
         rpc_clients["schedule"], sub_clients["schedule"])
     smgr.register(d_schedule)
@@ -231,8 +254,8 @@ def main():
     # lay out docks
     right_docks = [
         d_explorer, d_shortcuts,
-        d_ttl_dds.ttl_dock, d_ttl_dds.dds_dock, d_ttl_dds.dac_dock,
-        d_datasets, d_applets
+        d_datasets, d_applets,
+        d_waveform, d_interactive_args
     ]
     main_window.addDockWidget(QtCore.Qt.RightDockWidgetArea, right_docks[0])
     for d1, d2 in zip(right_docks, right_docks[1:]):
@@ -246,18 +269,25 @@ def main():
         # QDockWidgets fail to be embedded.
         main_window.show()
     smgr.load()
+
+    def init_cbs(ddb):
+        d_ttl_dds.dm.init_ddb(ddb)
+        d_waveform.init_ddb(ddb)
+        return ddb
+    devices_sub = Subscriber("devices", init_cbs, [d_ttl_dds.dm.notify_ddb, d_waveform.notify_ddb])
+    loop.run_until_complete(devices_sub.connect(args.server, args.port_notify))
+    atexit_register_coroutine(devices_sub.close, loop=loop)
+
     smgr.start(loop=loop)
     atexit_register_coroutine(smgr.stop, loop=loop)
 
-    # work around for https://github.com/m-labs/artiq/issues/1307
-    d_ttl_dds.ttl_dock.show()
-    d_ttl_dds.dds_dock.show()
-
     # create first log dock if not already in state
     d_log0 = logmgr.first_log_dock()
     if d_log0 is not None:
         main_window.tabifyDockWidget(d_schedule, d_log0)
-
+    d_moninj0 = d_ttl_dds.first_moninj_dock()
+    if d_moninj0 is not None:
+        main_window.tabifyDockWidget(right_docks[-1], d_moninj0)
 
     if server_name is not None:
         server_description = server_name + " ({})".format(args.server)
@@ -269,5 +299,6 @@ def main():
     main_window.show()
     loop.run_until_complete(main_window.exit_request.wait())
 
+
 if __name__ == "__main__":
     main()

artiq/frontend/artiq_ddb_template.py

@@ -211,6 +211,11 @@ class PeripheralManager:
         urukul_name = self.get_name("urukul")
         synchronization = peripheral["synchronization"]
         channel = count(0)
+        pll_en = peripheral["pll_en"]
+        clk_div = peripheral.get("clk_div")
+        if clk_div is None:
+            clk_div = 0 if pll_en else 1
+
         self.gen("""
             device_db["eeprom_{name}"] = {{
                 "type": "local",
@@ -277,7 +282,7 @@ class PeripheralManager:
             sync_device="\"ttl_{name}_sync\"".format(name=urukul_name) if synchronization else "None",
             refclk=peripheral.get("refclk", self.primary_description["rtio_frequency"]),
             clk_sel=peripheral["clk_sel"],
-            clk_div=peripheral["clk_div"])
+            clk_div=clk_div)
         dds = peripheral["dds"]
         pll_vco = peripheral.get("pll_vco")
         for i in range(4):
@@ -299,7 +304,7 @@ class PeripheralManager:
                     uchn=i,
                     sw=",\n        \"sw_device\": \"ttl_{name}_sw{uchn}\"".format(name=urukul_name, uchn=i) if len(peripheral["ports"]) > 1 else "",
                     pll_vco=",\n        \"pll_vco\": {}".format(pll_vco) if pll_vco is not None else "",
-                    pll_n=peripheral.get("pll_n", 32), pll_en=peripheral["pll_en"],
+                    pll_n=peripheral.get("pll_n", 32), pll_en=pll_en,
                     sync_delay_seed=",\n        \"sync_delay_seed\": \"eeprom_{}:{}\"".format(urukul_name, 64 + 4*i) if synchronization else "",
                     io_update_delay=",\n        \"io_update_delay\": \"eeprom_{}:{}\"".format(urukul_name, 64 + 4*i) if synchronization else "")
             elif dds == "ad9912":
@@ -320,7 +325,7 @@ class PeripheralManager:
                     uchn=i,
                     sw=",\n        \"sw_device\": \"ttl_{name}_sw{uchn}\"".format(name=urukul_name, uchn=i) if len(peripheral["ports"]) > 1 else "",
                     pll_vco=",\n        \"pll_vco\": {}".format(pll_vco) if pll_vco is not None else "",
-                    pll_n=peripheral.get("pll_n", 8), pll_en=peripheral["pll_en"])
+                    pll_n=peripheral.get("pll_n", 8), pll_en=pll_en)
             else:
                 raise ValueError
         return next(channel)

artiq/frontend/artiq_flash.py

@@ -261,12 +261,19 @@ def main():
             "storage":      ("spi0", 0x440000),
             "firmware":     ("spi0", 0x450000),
         },
-        "efc": {
+        "efc1v0": {
             "programmer":   partial(ProgrammerXC7, board="efc", proxy="bscan_spi_xc7a100t.bit"),
             "gateware":     ("spi0", 0x000000),
-            "bootloader":   ("spi0", 0x400000),
-            "storage":      ("spi0", 0x440000),
-            "firmware":     ("spi0", 0x450000),
+            "bootloader":   ("spi0", 0x600000),
+            "storage":      ("spi0", 0x640000),
+            "firmware":     ("spi0", 0x650000),
+        },
+        "efc1v1": {
+            "programmer":   partial(ProgrammerXC7, board="efc", proxy="bscan_spi_xc7a200t.bit"),
+            "gateware":     ("spi0", 0x000000),
+            "bootloader":   ("spi0", 0x600000),
+            "storage":      ("spi0", 0x640000),
+            "firmware":     ("spi0", 0x650000),
         },
         "kc705": {
             "programmer":   partial(ProgrammerXC7, board="kc705", proxy="bscan_spi_xc7k325t.bit"),

artiq/frontend/artiq_master.py

@@ -15,7 +15,8 @@ from sipyco.asyncio_tools import atexit_register_coroutine, SignalHandler
 
 from artiq import __version__ as artiq_version
 from artiq.master.log import log_args, init_log
-from artiq.master.databases import DeviceDB, DatasetDB
+from artiq.master.databases import (DeviceDB, DatasetDB,
+                                    InteractiveArgDB)
 from artiq.master.scheduler import Scheduler
 from artiq.master.rid_counter import RIDCounter
 from artiq.master.experiments import (FilesystemBackend, GitBackend,
@@ -57,10 +58,10 @@ def get_argparser():
     log_args(parser)
 
     parser.add_argument("--name",
-        help="friendly name, displayed in dashboards "
-             "to identify master instead of server address")
-    parser.add_argument("--log-submissions", default=None, 
-        help="set the filename to create the experiment subimission")
+                        help="friendly name, displayed in dashboards "
+                             "to identify master instead of server address")
+    parser.add_argument("--log-submissions", default=None,
+                        help="log experiment submissions to specified file")
 
     return parser
 
@@ -81,8 +82,7 @@ def main():
         bind, args.port_broadcast))
     atexit_register_coroutine(server_broadcast.stop, loop=loop)
 
-    log_forwarder.callback = (lambda msg:
-        server_broadcast.broadcast("log", msg))
+    log_forwarder.callback = lambda msg: server_broadcast.broadcast("log", msg)
     def ccb_issue(service, *args, **kwargs):
         msg = {
             "service": service,
@@ -96,6 +96,7 @@ def main():
     atexit.register(dataset_db.close_db)
     dataset_db.start(loop=loop)
     atexit_register_coroutine(dataset_db.stop, loop=loop)
+    interactive_arg_db = InteractiveArgDB()
     worker_handlers = dict()
 
     if args.git:
@@ -106,15 +107,22 @@ def main():
         repo_backend, worker_handlers, args.experiment_subdir)
     atexit.register(experiment_db.close)
 
-    scheduler = Scheduler(RIDCounter(), worker_handlers, experiment_db, args.log_submissions)
+    scheduler = Scheduler(RIDCounter(), worker_handlers, experiment_db,
+                          args.log_submissions)
     scheduler.start(loop=loop)
     atexit_register_coroutine(scheduler.stop, loop=loop)
 
+    # Python doesn't allow writing attributes to bound methods.
+    def get_interactive_arguments(*args, **kwargs):
+        return interactive_arg_db.get(*args, **kwargs)
+    get_interactive_arguments._worker_pass_rid = True
     worker_handlers.update({
         "get_device_db": device_db.get_device_db,
         "get_device": device_db.get,
         "get_dataset": dataset_db.get,
+        "get_dataset_metadata": dataset_db.get_metadata,
         "update_dataset": dataset_db.update,
+        "get_interactive_arguments": get_interactive_arguments,
         "scheduler_submit": scheduler.submit,
         "scheduler_delete": scheduler.delete,
         "scheduler_request_termination": scheduler.request_termination,
@@ -133,10 +141,11 @@ def main():
 
     server_control = RPCServer({
         "master_management": master_management,
-        "master_device_db": device_db,
-        "master_dataset_db": dataset_db,
-        "master_schedule": scheduler,
-        "master_experiment_db": experiment_db,
+        "device_db": device_db,
+        "dataset_db": dataset_db,
+        "interactive_arg_db": interactive_arg_db,
+        "schedule": scheduler,
+        "experiment_db": experiment_db,
     }, allow_parallel=True)
     loop.run_until_complete(server_control.start(
         bind, args.port_control))
@@ -146,8 +155,9 @@ def main():
         "schedule": scheduler.notifier,
         "devices": device_db.data,
         "datasets": dataset_db.data,
+        "interactive_args": interactive_arg_db.pending,
         "explist": experiment_db.explist,
-        "explist_status": experiment_db.status
+        "explist_status": experiment_db.status,
     })
     loop.run_until_complete(server_notify.start(
         bind, args.port_notify))

artiq/frontend/artiq_run.py

@@ -13,7 +13,7 @@ import h5py
 
 from llvmlite import binding as llvm
 
-from sipyco import common_args
+from sipyco import common_args, pyon
 
 from artiq import __version__ as artiq_version
 from artiq.language.environment import EnvExperiment, ProcessArgumentManager
@@ -166,9 +166,30 @@ def get_argparser(with_file=True):
     return parser
 
 
+class ArgumentManager(ProcessArgumentManager):
+    def get_interactive(self, interactive_arglist, title):
+        print(title)
+        result = dict()
+        for key, processor, group, tooltip in interactive_arglist:
+            success = False
+            while not success:
+                user_input = input("{}:{} (group={}, tooltip={}): ".format(
+                    key, type(processor).__name__, group, tooltip))
+                try:
+                    user_input_deser = pyon.decode(user_input)
+                    value = processor.process(user_input_deser)
+                except:
+                    logger.error("failed to process user input, retrying",
+                                 exc_info=True)
+                else:
+                    success = True
+            result[key] = value
+        return result
+
+
 def _build_experiment(device_mgr, dataset_mgr, args):
     arguments = parse_arguments(args.arguments)
-    argument_mgr = ProcessArgumentManager(arguments)
+    argument_mgr = ArgumentManager(arguments)
     managers = (device_mgr, dataset_mgr, argument_mgr, {})
     if hasattr(args, "file"):
         is_tar = tarfile.is_tarfile(args.file)

artiq/frontend/artiq_sinara_tester.py

@@ -9,6 +9,7 @@ import sys
 from artiq.experiment import *
 from artiq.coredevice.ad9910 import AD9910, SyncDataEeprom
 from artiq.coredevice.phaser import PHASER_GW_BASE, PHASER_GW_MIQRO
+from artiq.coredevice.shuttler import shuttler_volt_to_mu
 from artiq.master.databases import DeviceDB
 from artiq.master.worker_db import DeviceManager
 
@@ -60,7 +61,9 @@ class SinaraTester(EnvExperiment):
         self.mirnies = dict()
         self.suservos = dict()
         self.suschannels = dict()
+        self.legacy_almaznys = dict()
         self.almaznys = dict()
+        self.shuttler = dict()
 
         ddb = self.get_device_db()
         for name, desc in ddb.items():
@@ -99,7 +102,20 @@ class SinaraTester(EnvExperiment):
                 elif (module, cls) == ("artiq.coredevice.suservo", "Channel"):
                     self.suschannels[name] = self.get_device(name)
                 elif (module, cls) == ("artiq.coredevice.almazny", "AlmaznyLegacy"):
+                    self.legacy_almaznys[name] = self.get_device(name)
+                elif (module, cls) == ("artiq.coredevice.almazny", "AlmaznyChannel"):
                     self.almaznys[name] = self.get_device(name)
+                elif (module, cls) == ("artiq.coredevice.shuttler", "Config"):
+                    shuttler_name  = name.replace("_config", "")
+                    self.shuttler[shuttler_name] = ({
+                        "config": self.get_device(name),
+                        "trigger": self.get_device("{}_trigger".format(shuttler_name)),
+                        "leds": [self.get_device("{}_led{}".format(shuttler_name, i)) for i in range(2)],
+                        "dcbias": [self.get_device("{}_dcbias{}".format(shuttler_name, i)) for i in range(16)],
+                        "dds": [self.get_device("{}_dds{}".format(shuttler_name, i)) for i in range(16)],
+                        "relay": self.get_device("{}_relay".format(shuttler_name)),
+                        "adc": self.get_device("{}_adc".format(shuttler_name)),
+                    })
 
         # Remove Urukul, Sampler, Zotino and Mirny control signals
         # from TTL outs (tested separately) and remove Urukuls covered by
@@ -148,6 +164,7 @@ class SinaraTester(EnvExperiment):
         self.mirnies = sorted(self.mirnies.items(), key=lambda x: (x[1].cpld.bus.channel, x[1].channel))
         self.suservos = sorted(self.suservos.items(), key=lambda x: x[1].channel)
         self.suschannels = sorted(self.suschannels.items(), key=lambda x: x[1].channel)
+        self.shuttler = sorted(self.shuttler.items(), key=lambda x: x[1]["leds"][0].channel)
 
     @kernel
     def test_led(self, led):
@@ -356,67 +373,111 @@ class SinaraTester(EnvExperiment):
                 channel.pulse(100*ms)
                 delay(100*ms)
     @kernel
-    def init_almazny(self, almazny):
+    def init_legacy_almazny(self, almazny):
         self.core.break_realtime()
         almazny.init()
         almazny.output_toggle(True)
 
     @kernel
-    def almazny_set_attenuators_mu(self, almazny, ch, atts):
+    def legacy_almazny_set_attenuators_mu(self, almazny, ch, atts):
         self.core.break_realtime()
         almazny.set_att_mu(ch, atts)
 
     @kernel
-    def almazny_set_attenuators(self, almazny, ch, atts):
-        self.core.break_realtime()
-        almazny.set_att(ch, atts)
+    def legacy_almazny_att_test(self, almazny):
+        # change attenuation bit by bit over time for all channels
+        att_mu = 0
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            t = now_mu() - self.core.seconds_to_mu(0.5)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in range(4):
+                almazny.set_att_mu(ch, att_mu)
+            delay(250*ms)
+            if att_mu == 0:
+                att_mu = 1
+            else:
+                att_mu = (att_mu << 1) & 0x3F
     
     @kernel
-    def almazny_toggle_output(self, almazny, rf_on):
+    def legacy_almazny_toggle_output(self, almazny, rf_on):
         self.core.break_realtime()
         almazny.output_toggle(rf_on)
 
-    def test_almaznys(self):
-        print("*** Testing Almaznys.")
-        for name, almazny in sorted(self.almaznys.items(), key=lambda x: x[0]):
+    def test_legacy_almaznys(self):
+        print("*** Testing legacy Almaznys (v1.1 or older).")
+        for name, almazny in sorted(self.legacy_almaznys.items(), key=lambda x: x[0]):
             print(name + "...")
             print("Initializing Mirny CPLDs...")
             for name, cpld in sorted(self.mirny_cplds.items(), key=lambda x: x[0]):
                 print(name + "...")
                 self.init_mirny(cpld)
             print("...done")
-
             print("Testing attenuators. Frequencies:")
             for card_n, channels in enumerate(chunker(self.mirnies, 4)):
                 for channel_n, (channel_name, channel_dev) in enumerate(channels):
                     frequency = 2000 + card_n * 250 + channel_n * 50
                     print("{}\t{}MHz".format(channel_name, frequency*2))
                     self.setup_mirny(channel_dev, frequency)
-                    print("{} info: {}".format(channel_name, channel_dev.info()))
-            self.init_almazny(almazny)
-            print("RF ON, all attenuators ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators_mu(almazny, i, 63)
-            input()
-            print("RF ON, half power attenuators ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 15.5)
-            input()
-            print("RF ON, all attenuators OFF. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 0)
-            input()
+            self.init_legacy_almazny(almazny)
             print("SR outputs are OFF. Press ENTER when done.")
-            self.almazny_toggle_output(almazny, False)
-            input()
-            print("RF ON, all attenuators are ON. Press ENTER when done.")
-            for i in range(4):
-                self.almazny_set_attenuators(almazny, i, 31.5)
-            self.almazny_toggle_output(almazny, True)
-            input()
-            print("RF OFF. Press ENTER when done.")
-            self.almazny_toggle_output(almazny, False)
+            self.legacy_almazny_toggle_output(almazny, False)
             input()
+            print("RF ON, attenuators are tested. Press ENTER when done.")
+            self.legacy_almazny_toggle_output(almazny, True)
+            self.legacy_almazny_att_test(almazny)
+            self.legacy_almazny_toggle_output(almazny, False)
+
+    @kernel
+    def almazny_led_wave(self, almaznys):
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            # do not fill the FIFOs too much to avoid long response times
+            t = now_mu() - self.core.seconds_to_mu(0.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in almaznys:
+                ch.set(31.5, False, True)
+                delay(100*ms)
+                ch.set(31.5, False, False)
+    
+    @kernel
+    def almazny_att_test(self, almaznys):
+        rf_en = 1
+        led = 1
+        att_mu = 0
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            t = now_mu() - self.core.seconds_to_mu(0.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            setting = led << 7 | rf_en << 6 | (att_mu & 0x3F)
+            for ch in almaznys:
+                ch.set_mu(setting)
+            delay(250*ms)
+            if att_mu == 0:
+                att_mu = 1
+            else:
+                att_mu = (att_mu << 1) & 0x3F
+
+    def test_almaznys(self):
+        print("*** Testing Almaznys (v1.2+).")
+        print("Initializing Mirny CPLDs...")
+        for name, cpld in sorted(self.mirny_cplds.items(), key=lambda x: x[0]):
+            print(name + "...")
+            self.init_mirny(cpld)
+        print("...done")
+        print("Frequencies:")
+        for card_n, channels in enumerate(chunker(self.mirnies, 4)):
+            for channel_n, (channel_name, channel_dev) in enumerate(channels):
+                frequency = 2000 + card_n * 250 + channel_n * 50
+                print("{}\t{}MHz".format(channel_name, frequency*2))
+                self.setup_mirny(channel_dev, frequency)
+        print("RF ON, attenuators are tested. Press ENTER when done.")
+        self.almazny_att_test([ch for _, ch in self.almaznys.items()])
+        print("RF OFF, testing LEDs. Press ENTER when done.")
+        self.almazny_led_wave([ch for _, ch in self.almaznys.items()])
 
     def test_mirnies(self):
         print("*** Testing Mirny PLLs.")
@@ -747,6 +808,125 @@ class SinaraTester(EnvExperiment):
         print("Press ENTER when done.")
         input()
 
+    @kernel
+    def setup_shuttler_init(self, relay, adc, dcbias, dds, trigger, config):
+        self.core.break_realtime()
+        # Reset Shuttler Output Relay
+        relay.init()
+        delay_mu(int64(self.core.ref_multiplier))
+
+        relay.enable(0x0000)
+        delay_mu(int64(self.core.ref_multiplier))
+
+        # Setup ADC and and Calibration
+        delay_mu(int64(self.core.ref_multiplier))
+        adc.power_up()
+
+        delay_mu(int64(self.core.ref_multiplier))
+        if adc.read_id() >> 4 != 0x038d:
+            print("Remote AFE Board's ADC is not found. Check Remote AFE Board's Cables Connections")
+            assert adc.read_id() >> 4 == 0x038d
+
+        delay_mu(int64(self.core.ref_multiplier))
+        adc.calibrate(dcbias, trigger, config)
+
+        #Reset Shuttler DAC Output
+        for ch in range(16):
+            self.setup_shuttler_set_output(dcbias, dds, trigger, ch, 0.0)
+
+    @kernel 
+    def set_shuttler_relay(self, relay, val):
+        self.core.break_realtime()
+        relay.enable(val)
+
+    @kernel
+    def get_shuttler_output_voltage(self, adc, ch, cb):
+        self.core.break_realtime()
+        cb(adc.read_ch(ch))
+
+    @kernel
+    def setup_shuttler_set_output(self, dcbias, dds, trigger, ch, volt):
+        self.core.break_realtime()
+        dcbias[ch].set_waveform(
+            a0=shuttler_volt_to_mu(volt),
+            a1=0,
+            a2=0,
+            a3=0,
+        )
+        delay_mu(int64(self.core.ref_multiplier))
+
+        dds[ch].set_waveform(
+            b0=0,
+            b1=0,
+            b2=0,
+            b3=0,
+            c0=0,
+            c1=0,
+            c2=0,
+        )
+        delay_mu(int64(self.core.ref_multiplier))
+
+        trigger.trigger(1 << ch)
+        delay_mu(int64(self.core.ref_multiplier))
+
+    @kernel
+    def shuttler_relay_led_wave(self, relay):
+        while not is_enter_pressed():
+            self.core.break_realtime()
+            # do not fill the FIFOs too much to avoid long response times
+            t = now_mu() - self.core.seconds_to_mu(.2)
+            while self.core.get_rtio_counter_mu() < t:
+                pass
+            for ch in range(16):
+                relay.enable(1 << ch)
+                delay(100*ms)
+            relay.enable(0x0000)
+            delay(100*ms)
+
+    def test_shuttler(self):
+        print("*** Testing Shuttler.")
+
+        for card_n, (card_name, card_dev) in enumerate(self.shuttler):
+            print("Testing: ", card_name)
+
+            output_voltage = 0.0
+            def setv(x):
+                nonlocal output_voltage
+                output_voltage = x
+
+            self.setup_shuttler_init(card_dev["relay"], card_dev["adc"], card_dev["dcbias"], card_dev["dds"], card_dev["trigger"], card_dev["config"])
+            
+            print("Check Remote AFE Board Relay LED Indicators.")
+            print("Press Enter to Continue.")
+            self.shuttler_relay_led_wave(card_dev["relay"])
+
+            self.set_shuttler_relay(card_dev["relay"], 0xFFFF)
+            
+            passed = True
+            adc_readings = []
+            volt_set = [(-1)**i*(2.*card_n + .1*(i//2 + 1)) for i in range(16)]
+
+            print("Testing Shuttler DAC")
+            print("Voltages:", " ".join(["{:.1f}".format(x) for x in volt_set]))
+
+            for ch, volt in enumerate(volt_set):
+                self.setup_shuttler_set_output(card_dev["dcbias"], card_dev["dds"], card_dev["trigger"], ch, volt)
+                self.get_shuttler_output_voltage(card_dev["adc"], ch, setv)
+                if (abs(volt) - abs(output_voltage)) > 0.1:
+                    passed = False
+                adc_readings.append(output_voltage)
+
+            print("Press Enter to Continue.")
+            input()
+            self.set_shuttler_relay(card_dev["relay"], 0x0000)
+
+            if passed:
+                print("PASSED")
+            else:
+                print("FAILED")
+                print("Shuttler Remote AFE Board ADC has abnormal readings.")
+                print(f"ADC Readings:", " ".join(["{:.2f}".format(x) for x in adc_readings]))
+               
     def run(self, tests):
         print("****** Sinara system tester ******")
         print("")

artiq/gateware/drtio/aux_controller.py

@@ -10,6 +10,7 @@ from misoc.interconnect import wishbone
 
 
 max_packet = 1024
+aux_buffer_count = 8
 
 
 class Transmitter(Module, AutoCSR):
@@ -95,13 +96,13 @@ class Transmitter(Module, AutoCSR):
 
 class Receiver(Module, AutoCSR):
     def __init__(self, link_layer, min_mem_dw):
-        self.aux_rx_length = CSRStatus(bits_for(max_packet))
         self.aux_rx_present = CSR()
         self.aux_rx_error = CSR()
+        self.aux_read_pointer = CSR(log2_int(aux_buffer_count))
 
         ll_dw = len(link_layer.rx_aux_data)
         mem_dw = max(min_mem_dw, ll_dw)
-        self.specials.mem = Memory(mem_dw, max_packet//(mem_dw//8))
+        self.specials.mem = Memory(mem_dw, aux_buffer_count*max_packet//(mem_dw//8))
 
         converter = ClockDomainsRenamer("rtio_rx")(
             stream.Converter(ll_dw, mem_dw))
@@ -123,30 +124,45 @@ class Receiver(Module, AutoCSR):
         mem_port = self.mem.get_port(write_capable=True, clock_domain="rtio_rx")
         self.specials += mem_port
 
+        # write pointer represents where the gateware is
+        write_pointer = Signal(log2_int(aux_buffer_count))
+        write_pointer_sys = Signal.like(write_pointer)
+        # read pointer represents where CPU is
+        # write reaching read is an error, read reaching write is buffer clear
+        read_pointer = Signal.like(write_pointer)
+        read_pointer_rx = Signal.like(write_pointer)
+
+        read_pointer.attr.add("no_retiming")
+        write_pointer.attr.add("no_retiming")
+        signal_error = PulseSynchronizer("rtio_rx", "sys")
+
+        self.specials += [
+            MultiReg(read_pointer, read_pointer_rx, "rtio_rx"),
+            MultiReg(write_pointer, write_pointer_sys)
+        ]
+
         frame_counter_nbits = bits_for(max_packet) - log2_int(mem_dw//8)
         frame_counter = Signal(frame_counter_nbits)
+        # frame counter requires one more bit to represent overflow (bits_for)
+        # actual valid packet will be addressed with one bit less
+        packet_nbits = frame_counter_nbits - 1
         self.comb += [
-            mem_port.adr.eq(frame_counter),
+            mem_port.adr[:packet_nbits].eq(frame_counter),
+            # bits above the frame counter point to current frame
+            mem_port.adr[packet_nbits:].eq(write_pointer),
             mem_port.dat_w.eq(converter.source.data),
-            converter.source.ack.eq(1)
+            converter.source.ack.eq(1),
+            self.aux_read_pointer.w.eq(read_pointer)
         ]
 
-        frame_counter.attr.add("no_retiming")
-        frame_counter_sys = Signal(frame_counter_nbits)
-        self.specials += MultiReg(frame_counter, frame_counter_sys)
-        self.comb += self.aux_rx_length.status.eq(frame_counter_sys << log2_int(mem_dw//8))
-
-        signal_frame = PulseSynchronizer("rtio_rx", "sys")
-        frame_ack = PulseSynchronizer("sys", "rtio_rx")
-        signal_error = PulseSynchronizer("rtio_rx", "sys")
-        self.submodules += signal_frame, frame_ack, signal_error
+        self.submodules += signal_error
         self.sync += [
-            If(self.aux_rx_present.re, self.aux_rx_present.w.eq(0)),
-            If(signal_frame.o, self.aux_rx_present.w.eq(1)),
             If(self.aux_rx_error.re, self.aux_rx_error.w.eq(0)),
-            If(signal_error.o, self.aux_rx_error.w.eq(1))
+            If(signal_error.o, self.aux_rx_error.w.eq(1)),
+            self.aux_rx_present.w.eq(~(read_pointer == write_pointer_sys)),
+            If(self.aux_rx_present.re & self.aux_rx_present.w,
+                read_pointer.eq(read_pointer + 1)),
         ]
-        self.comb += frame_ack.i.eq(self.aux_rx_present.re)
 
         fsm = ClockDomainsRenamer("rtio_rx")(FSM(reset_state="IDLE"))
         self.submodules += fsm
@@ -171,7 +187,7 @@ class Receiver(Module, AutoCSR):
                     NextState("FRAME")
                 )
             ).Else(
-                NextValue(frame_counter, 0)
+                NextValue(frame_counter, 0),
             )
         )
         fsm.act("FRAME",
@@ -189,14 +205,12 @@ class Receiver(Module, AutoCSR):
             )
         )
         fsm.act("SIGNAL_FRAME",
-            signal_frame.i.eq(1),
-            NextState("WAIT_ACK"),
-            If(converter.source.stb, signal_error.i.eq(1))
-        )
-        fsm.act("WAIT_ACK",
-            If(frame_ack.o,
-                NextValue(frame_counter, 0), 
-                NextState("IDLE")
+            NextState("IDLE"),
+            If((write_pointer + 1) == read_pointer_rx,
+                # on full buffer, overwrite only current frame
+                signal_error.i.eq(1)
+            ).Else(
+                NextValue(write_pointer, write_pointer + 1)
             ),
             If(converter.source.stb, signal_error.i.eq(1))
         )
@@ -215,8 +229,8 @@ class DRTIOAuxController(Module):
         tx_sdram_if = wishbone.SRAM(self.transmitter.mem, read_only=False, data_width=dw)
         rx_sdram_if = wishbone.SRAM(self.receiver.mem, read_only=True, data_width=dw)
         decoder = wishbone.Decoder(self.bus,
-            [(lambda a: a[log2_int(max_packet)-wsb] == 0, tx_sdram_if.bus),
-             (lambda a: a[log2_int(max_packet)-wsb] == 1, rx_sdram_if.bus)],
+            [(lambda a: a[log2_int(max_packet*aux_buffer_count)-wsb] == 0, tx_sdram_if.bus),
+             (lambda a: a[log2_int(max_packet*aux_buffer_count)-wsb] == 1, rx_sdram_if.bus)],
             register=True)
         self.submodules += tx_sdram_if, rx_sdram_if, decoder
 

artiq/gateware/drtio/core.py

@@ -61,8 +61,8 @@ class SyncRTIO(Module):
         self.submodules.outputs = ClockDomainsRenamer("rio")(
             SED(channels, tsc.glbl_fine_ts_width,
                 lane_count=lane_count, fifo_depth=fifo_depth,
-                enable_spread=False, report_buffer_space=True,
-                interface=self.cri))
+                enable_spread=True, fifo_high_watermark=0.75,
+                report_buffer_space=True, interface=self.cri))
         self.comb += self.outputs.coarse_timestamp.eq(tsc.coarse_ts)
         self.sync += self.outputs.minimum_coarse_timestamp.eq(tsc.coarse_ts + 16)
 
@@ -136,7 +136,7 @@ class DRTIOSatellite(Module):
 
         self.sync += [
             If(self.tsc_loaded.re, self.tsc_loaded.w.eq(0)),
-            If(self.rt_packet.tsc_load, self.tsc_loaded.w.eq(1))
+            If(self.rt_packet.tsc_load, self.tsc_loaded.w.eq(1)),
         ]
 
         self.submodules.rt_errors = rt_errors_satellite.RTErrorsSatellite(

artiq/gateware/rtio/sed/core.py

@@ -12,10 +12,13 @@ __all__ = ["SED"]
 
 class SED(Module):
     def __init__(self, channels, glbl_fine_ts_width,
-                 lane_count=8, fifo_depth=128, enable_spread=True,
+                 lane_count=8, fifo_depth=128, fifo_high_watermark=1.0, enable_spread=True,
                  quash_channels=[], report_buffer_space=False, interface=None):
         seqn_width = layouts.seqn_width(lane_count, fifo_depth)
 
+        fifo_high_watermark = int(fifo_high_watermark * fifo_depth)
+        assert fifo_depth >= fifo_high_watermark
+
         self.submodules.lane_dist = LaneDistributor(lane_count, seqn_width,
             layouts.fifo_payload(channels),
             [channel.interface.o.delay for channel in channels],
@@ -23,7 +26,7 @@ class SED(Module):
             enable_spread=enable_spread,
             quash_channels=quash_channels,
             interface=interface)
-        self.submodules.fifos = FIFOs(lane_count, fifo_depth,
+        self.submodules.fifos = FIFOs(lane_count, fifo_depth, fifo_high_watermark,
             layouts.fifo_payload(channels), report_buffer_space)
         self.submodules.gates = Gates(lane_count, seqn_width,
             layouts.fifo_payload(channels),

artiq/gateware/rtio/sed/fifos.py

@@ -11,7 +11,7 @@ __all__ = ["FIFOs"]
 
 
 class FIFOs(Module):
-    def __init__(self, lane_count, fifo_depth, layout_payload, report_buffer_space=False):
+    def __init__(self, lane_count, fifo_depth, high_watermark, layout_payload, report_buffer_space=False):
         seqn_width = layouts.seqn_width(lane_count, fifo_depth)
         self.input = [Record(layouts.fifo_ingress(seqn_width, layout_payload))
                       for _ in range(lane_count)]
@@ -33,6 +33,7 @@ class FIFOs(Module):
                 fifo.din.eq(Cat(input.seqn, input.payload.raw_bits())),
                 fifo.we.eq(input.we),
                 input.writable.eq(fifo.writable),
+                input.high_watermark.eq(fifo.level >= high_watermark),
 
                 Cat(output.seqn, output.payload.raw_bits()).eq(fifo.dout),
                 output.readable.eq(fifo.readable),

artiq/gateware/rtio/sed/lane_distributor.py

@@ -154,8 +154,10 @@ class LaneDistributor(Module):
             self.comb += lio.payload.timestamp.eq(compensated_timestamp)
 
         # cycle #3, read status
+        current_lane_high_watermark = Signal()
         current_lane_writable = Signal()
         self.comb += [
+            current_lane_high_watermark.eq(Array(lio.high_watermark for lio in self.output)[current_lane]),
             current_lane_writable.eq(Array(lio.writable for lio in self.output)[current_lane]),
             o_status_wait.eq(~current_lane_writable)
         ]
@@ -170,12 +172,10 @@ class LaneDistributor(Module):
             self.sequence_error_channel.eq(self.cri.chan_sel[:16])
         ]
 
-        # current lane has been full, spread events by switching to the next.
+        # current lane has reached high watermark, spread events by switching to the next.
         if enable_spread:
-            current_lane_writable_r = Signal(reset=1)
             self.sync += [
-                current_lane_writable_r.eq(current_lane_writable),
-                If(~current_lane_writable_r & current_lane_writable,
+                If(current_lane_high_watermark | ~current_lane_writable,
                     force_laneB.eq(1)
                 ),
                 If(do_write,

artiq/gateware/rtio/sed/layouts.py

@@ -31,6 +31,7 @@ def fifo_ingress(seqn_width, layout_payload):
     return [
         ("we", 1, DIR_M_TO_S),
         ("writable", 1, DIR_S_TO_M),
+        ("high_watermark", 1, DIR_S_TO_M),
         ("seqn", seqn_width, DIR_M_TO_S),
         ("payload", [(a, b, DIR_M_TO_S) for a, b in layout_payload])
     ]

artiq/gateware/targets/efc.py

@@ -29,9 +29,10 @@ class Satellite(BaseSoC, AMPSoC):
     }
     mem_map.update(BaseSoC.mem_map)
 
-    def __init__(self, gateware_identifier_str=None, **kwargs):
+    def __init__(self, gateware_identifier_str=None, hw_rev="v1.1", **kwargs):
         BaseSoC.__init__(self,
                  cpu_type="vexriscv",
+                 hw_rev=hw_rev,
                  cpu_bus_width=64,
                  sdram_controller_type="minicon",
                  l2_size=128*1024,
@@ -82,9 +83,10 @@ class Satellite(BaseSoC, AMPSoC):
             core.link_layer, self.cpu_dw))
         self.csr_devices.append("drtioaux0")
 
+        drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
         memory_address = self.mem_map["drtioaux"]
-        self.add_wb_slave(memory_address, 0x800, self.drtioaux0.bus)
-        self.add_memory_region("drtioaux0_mem", memory_address | self.shadow_base, 0x800)
+        self.add_wb_slave(memory_address, drtio_aux_mem_size, self.drtioaux0.bus)
+        self.add_memory_region("drtioaux0_mem", memory_address | self.shadow_base, drtio_aux_mem_size)
 
         self.config["HAS_DRTIO"] = None
         self.add_csr_group("drtioaux", ["drtioaux0"])
@@ -242,12 +244,15 @@ def main():
     builder_args(parser)
     parser.set_defaults(output_dir="artiq_efc")
     parser.add_argument("-V", "--variant", default="shuttler")
+    parser.add_argument("--hw-rev", choices=["v1.0", "v1.1"], default="v1.1", 
+                        help="Hardware revision")
     parser.add_argument("--gateware-identifier-str", default=None,
                         help="Override ROM identifier")
     args = parser.parse_args()
 
     argdict = dict()
     argdict["gateware_identifier_str"] = args.gateware_identifier_str
+    argdict["hw_rev"] = args.hw_rev
 
     soc = Satellite(**argdict)
     build_artiq_soc(soc, builder_argdict(args))

artiq/gateware/targets/kasli.py

@@ -26,6 +26,7 @@ from artiq.gateware.drtio.transceiver import gtp_7series, eem_serdes
 from artiq.gateware.drtio.siphaser import SiPhaser7Series
 from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer
 from artiq.gateware.drtio import *
+from artiq.gateware.wrpll import wrpll
 from artiq.build_soc import *
 from artiq.coredevice import jsondesc
 
@@ -56,7 +57,7 @@ class StandaloneBase(MiniSoC, AMPSoC):
     }
     mem_map.update(MiniSoC.mem_map)
 
-    def __init__(self, gateware_identifier_str=None, hw_rev="v2.0", **kwargs):
+    def __init__(self, gateware_identifier_str=None, with_wrpll=False, hw_rev="v2.0", **kwargs):
         if hw_rev in ("v1.0", "v1.1"):
             cpu_bus_width = 32
         else:
@@ -82,7 +83,6 @@ class StandaloneBase(MiniSoC, AMPSoC):
             self.submodules.error_led = gpio.GPIOOut(Cat(
                 self.platform.request("error_led")))
             self.csr_devices.append("error_led")
-            self.submodules += SMAClkinForward(self.platform)
             cdr_clk_out = self.platform.request("cdr_clk_clean")
         else:
             cdr_clk_out = self.platform.request("si5324_clkout")
@@ -104,12 +104,31 @@ class StandaloneBase(MiniSoC, AMPSoC):
 
         self.crg.configure(cdr_clk_buf)
 
+        if with_wrpll:
+            clk_synth = self.platform.request("cdr_clk_clean_fabric")
+            clk_synth_se = Signal()
+            self.platform.add_period_constraint(clk_synth.p, 8.0)
+            self.specials += Instance("IBUFGDS", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
+            self.submodules.wrpll_refclk = wrpll.FrequencyMultiplier(self.platform.request("sma_clkin"))
+            self.submodules.wrpll = wrpll.WRPLL(
+                platform=self.platform,
+                cd_ref=self.wrpll_refclk.cd_ref,
+                main_clk_se=clk_synth_se)
+            self.csr_devices.append("wrpll_refclk")
+            self.csr_devices.append("wrpll")
+            self.interrupt_devices.append("wrpll")
+            self.config["HAS_SI549"] = None
+            self.config["WRPLL_REF_CLK"] = "SMA_CLKIN"
+        else:
+            if self.platform.hw_rev == "v2.0":
+                self.submodules += SMAClkinForward(self.platform)
+            self.config["HAS_SI5324"] = None
+            self.config["SI5324_SOFT_RESET"] = None
+
         i2c = self.platform.request("i2c")
         self.submodules.i2c = gpio.GPIOTristate([i2c.scl, i2c.sda])
         self.csr_devices.append("i2c")
         self.config["I2C_BUS_COUNT"] = 1
-        self.config["HAS_SI5324"] = None
-        self.config["SI5324_SOFT_RESET"] = None
 
     def add_rtio(self, rtio_channels, sed_lanes=8):
         fix_serdes_timing_path(self.platform)
@@ -146,7 +165,7 @@ class MasterBase(MiniSoC, AMPSoC):
     }
     mem_map.update(MiniSoC.mem_map)
 
-    def __init__(self, rtio_clk_freq=125e6, enable_sata=False, gateware_identifier_str=None, hw_rev="v2.0", **kwargs):
+    def __init__(self, rtio_clk_freq=125e6, enable_sata=False, with_wrpll=False, gateware_identifier_str=None, hw_rev="v2.0", **kwargs):
         if hw_rev in ("v1.0", "v1.1"):
             cpu_bus_width = 32
         else:
@@ -172,14 +191,33 @@ class MasterBase(MiniSoC, AMPSoC):
             self.submodules.error_led = gpio.GPIOOut(Cat(
                 self.platform.request("error_led")))
             self.csr_devices.append("error_led")
-            self.submodules += SMAClkinForward(platform)
 
         i2c = self.platform.request("i2c")
         self.submodules.i2c = gpio.GPIOTristate([i2c.scl, i2c.sda])
         self.csr_devices.append("i2c")
         self.config["I2C_BUS_COUNT"] = 1
-        self.config["HAS_SI5324"] = None
-        self.config["SI5324_SOFT_RESET"] = None
+
+        if with_wrpll:
+            clk_synth = platform.request("cdr_clk_clean_fabric")
+            clk_synth_se = Signal()
+            platform.add_period_constraint(clk_synth.p, 8.0)
+            self.specials += Instance("IBUFGDS", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
+            self.submodules.wrpll_refclk = wrpll.FrequencyMultiplier(platform.request("sma_clkin"))
+            self.submodules.wrpll = wrpll.WRPLL(
+                platform=self.platform,
+                cd_ref=self.wrpll_refclk.cd_ref,
+                main_clk_se=clk_synth_se)
+            self.csr_devices.append("wrpll_refclk")
+            self.csr_devices.append("wrpll")
+            self.interrupt_devices.append("wrpll")
+            self.config["HAS_SI549"] = None
+            self.config["WRPLL_REF_CLK"] = "SMA_CLKIN"
+        else:
+            if platform.hw_rev == "v2.0":
+                self.submodules += SMAClkinForward(self.platform)
+            self.config["HAS_SI5324"] = None
+            self.config["SI5324_SOFT_RESET"] = None
+
         self.config["RTIO_FREQUENCY"] = str(rtio_clk_freq/1e6)
 
         drtio_data_pads = []
@@ -236,10 +274,11 @@ class MasterBase(MiniSoC, AMPSoC):
             setattr(self.submodules, coreaux_name, coreaux)
             self.csr_devices.append(coreaux_name)
 
-            memory_address = self.mem_map["drtioaux"] + 0x800*i
-            self.add_wb_slave(memory_address, 0x800,
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size*i
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
                               coreaux.bus)
-            self.add_memory_region(memory_name, memory_address | self.shadow_base, 0x800)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
         self.config["HAS_DRTIO"] = None
         self.config["HAS_DRTIO_ROUTING"] = None
         self.config["DRTIO_ROLE"] = "master"
@@ -318,10 +357,11 @@ class MasterBase(MiniSoC, AMPSoC):
             setattr(self.submodules, coreaux_name, coreaux)
             self.csr_devices.append(coreaux_name)
 
-            memory_address = self.mem_map["drtioaux"] + 0x800*channel
-            self.add_wb_slave(memory_address, 0x800,
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size*channel
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
                             coreaux.bus)
-            self.add_memory_region(memory_name, memory_address | self.shadow_base, 0x800)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
 
     def add_drtio_cpuif_groups(self):
         self.add_csr_group("drtio", self.drtio_csr_group)
@@ -375,7 +415,7 @@ class SatelliteBase(BaseSoC, AMPSoC):
     }
     mem_map.update(BaseSoC.mem_map)
 
-    def __init__(self, rtio_clk_freq=125e6, enable_sata=False, *, gateware_identifier_str=None, hw_rev="v2.0", **kwargs):
+    def __init__(self, rtio_clk_freq=125e6, enable_sata=False, with_wrpll=False, *, gateware_identifier_str=None, hw_rev="v2.0", **kwargs):
         if hw_rev in ("v1.0", "v1.1"):
             cpu_bus_width = 32
         else:
@@ -453,15 +493,15 @@ class SatelliteBase(BaseSoC, AMPSoC):
 
         self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
 
-        drtioaux_csr_group = []
-        drtioaux_memory_group = []
-        drtiorep_csr_group = []
+        self.drtioaux_csr_group = []
+        self.drtioaux_memory_group = []
+        self.drtiorep_csr_group = []
         self.drtio_cri = []
         for i in range(len(self.gt_drtio.channels)):
             coreaux_name = "drtioaux" + str(i)
             memory_name = "drtioaux" + str(i) + "_mem"
-            drtioaux_csr_group.append(coreaux_name)
-            drtioaux_memory_group.append(memory_name)
+            self.drtioaux_csr_group.append(coreaux_name)
+            self.drtioaux_memory_group.append(memory_name)
 
             cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
 
@@ -474,7 +514,7 @@ class SatelliteBase(BaseSoC, AMPSoC):
                 self.csr_devices.append("drtiosat")
             else:
                 corerep_name = "drtiorep" + str(i-1)
-                drtiorep_csr_group.append(corerep_name)
+                self.drtiorep_csr_group.append(corerep_name)
 
                 core = cdr(DRTIORepeater(
                     self.rtio_tsc, self.gt_drtio.channels[i]))
@@ -486,16 +526,14 @@ class SatelliteBase(BaseSoC, AMPSoC):
             setattr(self.submodules, coreaux_name, coreaux)
             self.csr_devices.append(coreaux_name)
 
-            memory_address = self.mem_map["drtioaux"] + 0x800*i
-            self.add_wb_slave(memory_address, 0x800,
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size * i
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
                               coreaux.bus)
-            self.add_memory_region(memory_name, memory_address | self.shadow_base, 0x800)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
         self.config["HAS_DRTIO"] = None
         self.config["HAS_DRTIO_ROUTING"] = None
         self.config["DRTIO_ROLE"] = "satellite"
-        self.add_csr_group("drtioaux", drtioaux_csr_group)
-        self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
-        self.add_csr_group("drtiorep", drtiorep_csr_group)
 
         i2c = self.platform.request("i2c")
         self.submodules.i2c = gpio.GPIOTristate([i2c.scl, i2c.sda])
@@ -505,17 +543,33 @@ class SatelliteBase(BaseSoC, AMPSoC):
         rtio_clk_period = 1e9/rtio_clk_freq
         self.config["RTIO_FREQUENCY"] = str(rtio_clk_freq/1e6)
 
-        self.submodules.siphaser = SiPhaser7Series(
-            si5324_clkin=platform.request("cdr_clk") if platform.hw_rev == "v2.0"
-                else platform.request("si5324_clkin"),
-            rx_synchronizer=self.rx_synchronizer,
-            ref_clk=self.crg.clk125_div2, ref_div2=True,
-            rtio_clk_freq=rtio_clk_freq)
-        platform.add_false_path_constraints(
-            self.crg.cd_sys.clk, self.siphaser.mmcm_freerun_output)
-        self.csr_devices.append("siphaser")
-        self.config["HAS_SI5324"] = None
-        self.config["SI5324_SOFT_RESET"] = None
+        if with_wrpll:
+            clk_synth = platform.request("cdr_clk_clean_fabric")
+            clk_synth_se = Signal()
+            platform.add_period_constraint(clk_synth.p, 8.0)
+            self.specials += Instance("IBUFGDS", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
+            self.submodules.wrpll = wrpll.WRPLL(
+                platform=self.platform,
+                cd_ref=self.gt_drtio.cd_rtio_rx0,
+                main_clk_se=clk_synth_se)
+            self.submodules.wrpll_skewtester = wrpll.SkewTester(self.rx_synchronizer)
+            self.csr_devices.append("wrpll_skewtester")
+            self.csr_devices.append("wrpll")
+            self.interrupt_devices.append("wrpll")
+            self.config["HAS_SI549"] = None
+            self.config["WRPLL_REF_CLK"] = "GT_CDR"
+        else:
+            self.submodules.siphaser = SiPhaser7Series(
+                si5324_clkin=platform.request("cdr_clk") if platform.hw_rev == "v2.0"
+                    else platform.request("si5324_clkin"),
+                rx_synchronizer=self.rx_synchronizer,
+                ref_clk=self.crg.clk125_div2, ref_div2=True,
+                rtio_clk_freq=rtio_clk_freq)
+            platform.add_false_path_constraints(
+                self.crg.cd_sys.clk, self.siphaser.mmcm_freerun_output)
+            self.csr_devices.append("siphaser")
+            self.config["HAS_SI5324"] = None
+            self.config["SI5324_SOFT_RESET"] = None
 
         gtp = self.gt_drtio.gtps[0]
         txout_buf = Signal()
@@ -563,12 +617,54 @@ class SatelliteBase(BaseSoC, AMPSoC):
                                                 self.get_native_sdram_if(), cpu_dw=self.cpu_dw)
         self.csr_devices.append("rtio_analyzer")
 
+    def add_eem_drtio(self, eem_drtio_channels):
+        # Must be called before invoking add_rtio() to construct the CRI
+        # interconnect properly
+        self.submodules.eem_transceiver = eem_serdes.EEMSerdes(self.platform, eem_drtio_channels)
+        self.csr_devices.append("eem_transceiver")
+        self.config["HAS_DRTIO_EEM"] = None
+        self.config["EEM_DRTIO_COUNT"] = len(eem_drtio_channels)
+
+        cdr = ClockDomainsRenamer({"rtio_rx": "sys"})
+        for i in range(len(self.eem_transceiver.channels)):
+            channel = i + len(self.gt_drtio.channels)
+            corerep_name = "drtiorep" + str(channel-1)
+            coreaux_name = "drtioaux" + str(channel)
+            memory_name = "drtioaux" + str(channel) + "_mem"
+            self.drtiorep_csr_group.append(corerep_name)
+            self.drtioaux_csr_group.append(coreaux_name)
+            self.drtioaux_memory_group.append(memory_name)
+
+            core = cdr(DRTIORepeater(
+                self.rtio_tsc, self.eem_transceiver.channels[i]))
+            setattr(self.submodules, corerep_name, core)
+            self.drtio_cri.append(core.cri)
+            self.csr_devices.append(corerep_name)
+
+            coreaux = cdr(DRTIOAuxController(core.link_layer, self.cpu_dw))
+            setattr(self.submodules, coreaux_name, coreaux)
+            self.csr_devices.append(coreaux_name)
+
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size*channel
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
+                            coreaux.bus)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
+
+    def add_drtio_cpuif_groups(self):
+        self.add_csr_group("drtiorep", self.drtiorep_csr_group)
+        self.add_csr_group("drtioaux", self.drtioaux_csr_group)
+        self.add_memory_group("drtioaux_mem", self.drtioaux_memory_group)
+
 class GenericStandalone(StandaloneBase):
     def __init__(self, description, hw_rev=None,**kwargs):
         if hw_rev is None:
             hw_rev = description["hw_rev"]
         self.class_name_override = description["variant"]
-        StandaloneBase.__init__(self, hw_rev=hw_rev, **kwargs)
+        StandaloneBase.__init__(self,
+            hw_rev=hw_rev,
+            with_wrpll=description["enable_wrpll"],
+            **kwargs)
         self.config["RTIO_FREQUENCY"] = "{:.1f}".format(description["rtio_frequency"]/1e6)
         if "ext_ref_frequency" in description:
             self.config["SI5324_EXT_REF"] = None
@@ -623,6 +719,7 @@ class GenericMaster(MasterBase):
             rtio_clk_freq=description["rtio_frequency"],
             enable_sata=description["enable_sata_drtio"],
             enable_sys5x=has_drtio_over_eem,
+            with_wrpll=description["enable_wrpll"],
             **kwargs)
         if "ext_ref_frequency" in description:
             self.config["SI5324_EXT_REF"] = None
@@ -670,15 +767,19 @@ class GenericSatellite(SatelliteBase):
         if hw_rev is None:
             hw_rev = description["hw_rev"]
         self.class_name_override = description["variant"]
+        has_drtio_over_eem = any(peripheral["type"] == "shuttler" for peripheral in description["peripherals"])
         SatelliteBase.__init__(self,
                                hw_rev=hw_rev,
                                rtio_clk_freq=description["rtio_frequency"],
                                enable_sata=description["enable_sata_drtio"],
+                               enable_sys5x=has_drtio_over_eem,
+                               with_wrpll=description["enable_wrpll"],
                                **kwargs)
         if hw_rev == "v1.0":
             # EEM clock fan-out from Si5324, not MMCX
             self.comb += self.platform.request("clk_sel").eq(1)
-
+        if has_drtio_over_eem:
+            self.eem_drtio_channels = []
         has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
         if has_grabber:
             self.grabber_csr_group = []
@@ -696,6 +797,10 @@ class GenericSatellite(SatelliteBase):
         self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
         self.rtio_channels.append(rtio.LogChannel())
 
+        if has_drtio_over_eem:
+            self.add_eem_drtio(self.eem_drtio_channels)
+        self.add_drtio_cpuif_groups()
+
         self.add_rtio(self.rtio_channels, sed_lanes=description["sed_lanes"])
         if has_grabber:
             self.config["HAS_GRABBER"] = None
@@ -738,6 +843,8 @@ def main():
     has_shuttler = any(peripheral["type"] == "shuttler" for peripheral in description["peripherals"])
     if has_shuttler and (description["drtio_role"] == "standalone"):
         raise ValueError("Shuttler requires DRTIO, please switch role to master")
+    if description["enable_wrpll"] and description["hw_rev"] in ["v1.0", "v1.1"]:
+        raise ValueError("Kasli {} does not support WRPLL".format(description["hw_rev"])) 
 
     soc = cls(description, gateware_identifier_str=args.gateware_identifier_str, **soc_kasli_argdict(args))
     args.variant = description["variant"]

artiq/gateware/targets/kc705.py

@@ -247,10 +247,11 @@ class _MasterBase(MiniSoC, AMPSoC):
             setattr(self.submodules, coreaux_name, coreaux)
             self.csr_devices.append(coreaux_name)
 
-            memory_address = self.mem_map["drtioaux"] + 0x800*i
-            self.add_wb_slave(memory_address, 0x800,
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size*i
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
                               coreaux.bus)
-            self.add_memory_region(memory_name, memory_address | self.shadow_base, 0x800)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
         self.config["HAS_DRTIO"] = None
         self.config["HAS_DRTIO_ROUTING"] = None
         self.add_csr_group("drtio", drtio_csr_group)
@@ -405,10 +406,11 @@ class _SatelliteBase(BaseSoC, AMPSoC):
             setattr(self.submodules, coreaux_name, coreaux)
             self.csr_devices.append(coreaux_name)
 
-            memory_address = self.mem_map["drtioaux"] + 0x800*i
-            self.add_wb_slave(memory_address, 0x800,
+            drtio_aux_mem_size = 1024 * 16 # max_packet * 8 buffers * 2 (tx, rx halves)
+            memory_address = self.mem_map["drtioaux"] + drtio_aux_mem_size*i
+            self.add_wb_slave(memory_address, drtio_aux_mem_size,
                               coreaux.bus)
-            self.add_memory_region(memory_name, memory_address | self.shadow_base, 0x800)
+            self.add_memory_region(memory_name, memory_address | self.shadow_base, drtio_aux_mem_size)
         self.config["HAS_DRTIO"] = None
         self.config["HAS_DRTIO_ROUTING"] = None
         self.add_csr_group("drtioaux", drtioaux_csr_group)

artiq/gateware/test/drtio/test_aux_controller.py

@@ -60,28 +60,36 @@ class TestAuxController(unittest.TestCase):
             while (yield from dut[dw].aux_controller.transmitter.aux_tx.read()):
                 yield
 
-        def receive_packet(dw):
+        def receive_packet(pkt_no, len, dw):
             while not (yield from dut[dw].aux_controller.receiver.aux_rx_present.read()):
                 yield
-            length = yield from dut[dw].aux_controller.receiver.aux_rx_length.read()
+            p = yield from dut[dw].aux_controller.receiver.aux_read_pointer.read()
+            self.assertEqual(pkt_no, p)
             r = []
-            for i in range(length//(dw//8)):
+            # packet length is derived by software now, so we pass it for the test
+            for i in range(len):
+                if dw == 64:
+                    offset = 2048
+                if dw == 32:
+                    offset = 1024
+                print(dw)
+                max_packet = 1024
                 r.append((yield from dut[dw].aux_controller.bus.read(256+i)))
             yield from dut[dw].aux_controller.receiver.aux_rx_present.write(1)
             return r
 
         prng = random.Random(0)
 
-        def send_and_check_packet(dw):
+        def send_and_check_packet(i, dw):
             data = [prng.randrange(2**dw-1) for _ in range(prng.randrange(1, 16))]
             yield from send_packet(data, dw)
-            received = yield from receive_packet(dw)
+            received = yield from receive_packet(i, len(data), dw)
             self.assertEqual(data, received)
 
         def sim(dw):
             yield from link_init(dw)
             for i in range(8):
-                yield from send_and_check_packet(dw)
+                yield from send_and_check_packet(i, dw)
 
         @passive
         def rt_traffic(dw):
@@ -95,5 +103,5 @@ class TestAuxController(unittest.TestCase):
                 yield dut[dw].link_layer.tx_rt_frame.eq(0)
                 yield
 
-        run_simulation(dut[32], [sim(32), rt_traffic(32)])
-        run_simulation(dut[64], [sim(64), rt_traffic(64)])
+        run_simulation(dut[32], [sim(32), rt_traffic(32)], vcd_name="32bit.vcd")
+        run_simulation(dut[64], [sim(64), rt_traffic(64)], vcd_name="64bit.vcd")

artiq/gateware/wrpll/ddmtd.py

@@ -0,0 +1,119 @@
+from migen import *
+from migen.genlib.cdc import PulseSynchronizer, MultiReg
+from misoc.interconnect.csr import *
+
+
+class DDMTDSampler(Module):
+    def __init__(self, cd_ref, main_clk_se):
+        self.ref_beating = Signal()
+        self.main_beating = Signal()
+
+        # # #
+
+        ref_clk = Signal()
+        self.specials +=[
+            # ISERDESE2 can only be driven from fabric via IDELAYE2 (see UG471)
+            Instance("IDELAYE2",
+                    p_DELAY_SRC="DATAIN",
+                    p_HIGH_PERFORMANCE_MODE="TRUE",
+                    p_REFCLK_FREQUENCY=208.3,   # REFCLK frequency from IDELAYCTRL
+                    p_IDELAY_VALUE=0,
+
+                    i_DATAIN=cd_ref.clk,
+
+                    o_DATAOUT=ref_clk
+            ),
+            Instance("ISERDESE2",
+                    p_IOBDELAY="IFD",   # use DDLY as input
+                    p_DATA_RATE="SDR",
+                    p_DATA_WIDTH=2,     # min is 2
+                    p_NUM_CE=1,
+
+                    i_DDLY=ref_clk,
+                    i_CE1=1,
+                    i_CLK=ClockSignal("helper"),
+                    i_CLKDIV=ClockSignal("helper"),
+
+                    o_Q1=self.ref_beating
+            ),
+            Instance("ISERDESE2",
+                    p_DATA_RATE="SDR",
+                    p_DATA_WIDTH=2,     # min is 2
+                    p_NUM_CE=1,
+
+                    i_D=main_clk_se,
+                    i_CE1=1,
+                    i_CLK=ClockSignal("helper"),
+                    i_CLKDIV=ClockSignal("helper"),
+
+                    o_Q1=self.main_beating,
+            ),
+        ]
+
+
+class DDMTDDeglitcherMedianEdge(Module):
+    def __init__(self, counter, input_signal, stable_0_period=100, stable_1_period=100):
+        self.tag = Signal(len(counter))
+        self.detect = Signal()
+
+        stable_0_counter = Signal(reset=stable_0_period - 1, max=stable_0_period)
+        stable_1_counter = Signal(reset=stable_1_period - 1, max=stable_1_period)
+
+        # # #
+
+        # Based on CERN's median edge deglitcher FSM
+        # https://white-rabbit.web.cern.ch/documents/Precise_time_and_frequency_transfer_in_a_White_Rabbit_network.pdf (p.72)
+        fsm = ClockDomainsRenamer("helper")(FSM(reset_state="WAIT_STABLE_0"))
+        self.submodules += fsm
+
+        fsm.act("WAIT_STABLE_0",
+            If(stable_0_counter != 0,
+                NextValue(stable_0_counter, stable_0_counter - 1)
+            ).Else(
+                NextValue(stable_0_counter, stable_0_period - 1),
+                NextState("WAIT_EDGE")
+            ),
+            If(input_signal,
+                NextValue(stable_0_counter, stable_0_period - 1)
+            ),
+        )
+        fsm.act("WAIT_EDGE",
+            If(input_signal,
+                NextValue(self.tag, counter),
+                NextState("GOT_EDGE")
+            )
+        )
+        fsm.act("GOT_EDGE",
+            If(stable_1_counter != 0,
+                NextValue(stable_1_counter, stable_1_counter - 1)
+            ).Else(
+                NextValue(stable_1_counter, stable_1_period - 1),
+                self.detect.eq(1),
+                NextState("WAIT_STABLE_0")
+            ),
+            If(~input_signal,
+                NextValue(self.tag, self.tag + 1),
+                NextValue(stable_1_counter, stable_1_period - 1)
+            ),
+        )
+        
+
+class DDMTD(Module):
+    def __init__(self, counter, input_signal):
+
+        # in helper clock domain
+        self.h_tag = Signal(len(counter))
+        self.h_tag_update = Signal()
+
+        # # #
+
+        deglitcher = DDMTDDeglitcherMedianEdge(counter, input_signal)
+        self.submodules += deglitcher
+
+        self.sync.helper += [
+            self.h_tag_update.eq(0),
+            If(deglitcher.detect,
+                self.h_tag_update.eq(1),
+                self.h_tag.eq(deglitcher.tag)
+               )
+        ]