Merge branch 'master' into dataset-compression

CONTRIBUTING.rst

@@ -7,8 +7,7 @@ Reporting Issues/Bugs
 
 Thanks for `reporting issues to ARTIQ
 <https://github.com/m-labs/artiq/issues/new>`_! You can also discuss issues and
-ask questions on IRC (the `#m-labs channel on freenode
+ask questions on IRC (the #m-labs channel on OFTC), the `Mattermost chat
-<https://webchat.freenode.net/?channels=m-labs>`_), the `Mattermost chat
 <https://chat.m-labs.hk>`_, or on the `forum <https://forum.m-labs.hk>`_.
 
 The best bug reports are those which contain sufficient information. With

MANIFEST.in

@@ -4,3 +4,4 @@ include artiq/gui/logo*.svg
 include versioneer.py
 include artiq/_version.py
 include artiq/coredevice/coredevice_generic.schema.json
+include artiq/compiler/kernel.ld

README.rst

@@ -20,7 +20,7 @@ Like any open source software ARTIQ can equally be built and installed directly
 ARTIQ is supported by M-Labs and developed openly.
 Components, features, fixes, improvements, and extensions are often `funded <https://m-labs.hk/experiment-control/funding/>`_ by and developed for the partnering research groups.
 
-Core technologies employed include `Python <https://www.python.org/>`_, `Migen <https://github.com/m-labs/migen>`_, `Migen-AXI <https://github.com/peteut/migen-axi>`_, `Rust <https://www.rust-lang.org/>`_, `MiSoC <https://github.com/m-labs/misoc>`_/`mor1kx <https://github.com/openrisc/mor1kx>`_, `LLVM <https://llvm.org/>`_/`llvmlite <https://github.com/numba/llvmlite>`_, and `Qt5 <https://www.qt.io/>`_.
+Core technologies employed include `Python <https://www.python.org/>`_, `Migen <https://github.com/m-labs/migen>`_, `Migen-AXI <https://github.com/peteut/migen-axi>`_, `Rust <https://www.rust-lang.org/>`_, `MiSoC <https://github.com/m-labs/misoc>`_/`VexRiscv <https://github.com/SpinalHDL/VexRiscv>`_, `LLVM <https://llvm.org/>`_/`llvmlite <https://github.com/numba/llvmlite>`_, and `Qt5 <https://www.qt.io/>`_.
 
 Website: https://m-labs.hk/artiq
 

RELEASE_NOTES.rst

@@ -7,23 +7,48 @@ ARTIQ-7
 -------
 
 Highlights:
+
+* Support for Kasli-SoC, a new EEM carrier based on a Zynq SoC, enabling much faster kernel execution.
+* Softcore targets now use the RISC-V architecture (VexRiscv) instead of OR1K (mor1kx).
 * WRPLL
-* ``get()``, ``get_mu()``, ``get_att()``, and ``get_att_mu()`` functions added for AD9910 and AD9912
+* Compiler:
+   - Supports kernel decorator with paths.
+   - Faster compilation for large arrays/lists.
 * Phaser:
    - Improved documentation
-   - Expose the DAC coarse mixer and sif_sync
+   - Expose the DAC coarse mixer and ``sif_sync``
    - Exposes upconverter calibration and enabling/disabling of upconverter LO & RF outputs.
+   - Add helpers to align Phaser updates to the RTIO timeline (``get_next_frame_mu()``)
+* ``get()``, ``get_mu()``, ``get_att()``, and ``get_att_mu()`` functions added for AD9910 and AD9912
+* On Kasli, the number of FIFO lanes in the scalable events dispatcher (SED) can now be configured in
+  the JSON hardware description file.
+* New hardware support:
+   - HVAMP_8CH 8 channel HV amplifier for Fastino / Zotino
+* ``artiq_ddb_template`` generates edge-counter keys that start with the key of the corresponding
+  TTL device (e.g. ``"ttl_0_counter"`` for the edge counter on TTL device``"ttl_0"``)
+* ``artiq_master`` now has an ``--experiment-subdir`` option to scan only a subdirectory of the
+  repository when building the list of experiments.
+* The configuration entry ``rtio_clock`` supports multiple clocking settings, deprecating the usage
+  of compile-time options.
+* DRTIO: added support for 100MHz clock.
+* Previously detected RTIO async errors are reported to the host after each kernel terminates and a
+  warning is logged. The warning is additional to the one already printed in the core device log upon
+  detection of the error.
 * HDF5 options can now be passed when creating datasets with ``set_dataset``. This allows
   in particular to use transparent compression filters as follows:
   ``set_dataset(name, value, hdf5_options={"compression": "gzip"})``.
 
-
 Breaking changes:
+
 * Updated Phaser-Upconverter default frequency 2.875 GHz. The new default uses the target PFD
   frequency of the hardware design.
-* `Phaser.init()` now disables all Kasli-oscillators. This avoids full power RF output being
+* ``Phaser.init()`` now disables all Kasli-oscillators. This avoids full power RF output being
   generated for some configurations.
 * Phaser: fixed coarse mixer frequency configuration
+* Mirny: Added extra delays in ``ADF5356.sync()``. This avoids the need of an extra delay before
+  calling `ADF5356.init()`.
+* DRTIO: Changed message alignment from 32-bits to 64-bits.
+* The deprecated ``set_dataset(..., save=...)`` is no longer supported.
 * The internal dataset representation was changed to support tracking HDF5 options like e.g.
   a compression method. This requires changes to code reading the dataset persistence file
   (``dataset_db.pyon``) and to custom applets.
@@ -37,7 +62,7 @@ Highlights:
 * New hardware support:
    - Phaser, a quad channel 1GS/s RF generator card with dual IQ upconverter and dual 5MS/s
      ADC and FPGA.
-   - Zynq SoC core devices, enabling kernels to run on 1 GHz CPU core with a floating-point
+   - Zynq SoC core device (ZC706), enabling kernels to run on 1 GHz CPU core with a floating-point
      unit for faster computations. This currently requires an external
      repository (https://git.m-labs.hk/m-labs/artiq-zynq).
    - Mirny 4-channel wide-band PLL/VCO-based microwave frequency synthesiser
@@ -93,6 +118,9 @@ Breaking changes:
 * ``quamash`` has been replaced with ``qasync``.
 * Protocols are updated to use device endian.
 * Analyzer dump format includes a byte for device endianness.
+* To support variable numbers of Urukul cards in the future, the
+  ``artiq.coredevice.suservo.SUServo`` constructor now accepts two device name lists,
+  ``cpld_devices`` and ``dds_devices``, rather than four individual arguments.
 * Experiment classes with underscore-prefixed names are now ignored when ``artiq_client``
   determines which experiment to submit (consistent with ``artiq_run``).
 

artiq/applets/plot_hist.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env python3
 
-import PyQt5  # make sure pyqtgraph imports Qt5
+import PyQt5    # make sure pyqtgraph imports Qt5
+from PyQt5.QtCore import QTimer
 import pyqtgraph
 
 from artiq.applets.simple import TitleApplet
@@ -10,6 +11,9 @@ class HistogramPlot(pyqtgraph.PlotWidget):
     def __init__(self, args):
         pyqtgraph.PlotWidget.__init__(self)
         self.args = args
+        self.timer = QTimer()
+        self.timer.setSingleShot(True)
+        self.timer.timeout.connect(self.length_warning)
 
     def data_changed(self, data, mods, title):
         try:
@@ -24,10 +28,20 @@ class HistogramPlot(pyqtgraph.PlotWidget):
             x = list(range(len(y)+1))
 
         if len(y) and len(x) == len(y) + 1:
+            self.timer.stop()
             self.clear()
             self.plot(x, y, stepMode=True, fillLevel=0,
                       brush=(0, 0, 255, 150))
             self.setTitle(title)
+        else:
+            if not self.timer.isActive():
+                self.timer.start(1000)
+
+    def length_warning(self):
+        self.clear()
+        text = "⚠️ dataset lengths mismatch:\n"\
+            "There should be one more bin boundaries than there are Y values"
+        self.addItem(pyqtgraph.TextItem(text))
 
 
 def main():

artiq/applets/plot_xy.py

@@ -2,6 +2,7 @@
 
 import numpy as np
 import PyQt5  # make sure pyqtgraph imports Qt5
+from PyQt5.QtCore import QTimer
 import pyqtgraph
 
 from artiq.applets.simple import TitleApplet
@@ -12,6 +13,12 @@ class XYPlot(pyqtgraph.PlotWidget):
     def __init__(self, args):
         pyqtgraph.PlotWidget.__init__(self)
         self.args = args
+        self.timer = QTimer()
+        self.timer.setSingleShot(True)
+        self.timer.timeout.connect(self.length_warning)
+        self.mismatch = {'X values': False,
+                         'Error bars': False,
+                         'Fit values': False}
 
     def data_changed(self, data, mods, title):
         try:
@@ -25,17 +32,29 @@ class XYPlot(pyqtgraph.PlotWidget):
         fit = data.get(self.args.fit, empty_dataset())["value"]
 
         if not len(y) or len(y) != len(x):
-            return
+            self.mismatch['X values'] = True
+        else:
+            self.mismatch['X values'] = False
         if error is not None and hasattr(error, "__len__"):
             if not len(error):
                 error = None
             elif len(error) != len(y):
-                return
+                self.mismatch['Error bars'] = True
+            else:
+                self.mismatch['Error bars'] = False
         if fit is not None:
             if not len(fit):
                 fit = None
             elif len(fit) != len(y):
-                return
+                self.mismatch['Fit values'] = True
+            else:
+                self.mismatch['Fit values'] = False
+        if not any(self.mismatch.values()):
+            self.timer.stop()
+        else:
+            if not self.timer.isActive():
+                self.timer.start(1000)
+            return
 
         self.clear()
         self.plot(x, y, pen=None, symbol="x")
@@ -51,6 +70,13 @@ class XYPlot(pyqtgraph.PlotWidget):
             xi = np.argsort(x)
             self.plot(x[xi], fit[xi])
 
+    def length_warning(self):
+        self.clear()
+        text = "⚠️ dataset lengths mismatch:\n"
+        errors = ', '.join([k for k, v in self.mismatch.items() if v])
+        text = ' '.join([errors, "should have the same length as Y values"])
+        self.addItem(pyqtgraph.TextItem(text))
+
 
 def main():
     applet = TitleApplet(XYPlot)

artiq/applets/plot_xy_hist.py

@@ -2,6 +2,7 @@
 
 import numpy as np
 from PyQt5 import QtWidgets
+from PyQt5.QtCore import QTimer
 import pyqtgraph
 
 from artiq.applets.simple import SimpleApplet
@@ -37,6 +38,10 @@ class XYHistPlot(QtWidgets.QSplitter):
         self.hist_plot_data = None
 
         self.args = args
+        self.timer = QTimer()
+        self.timer.setSingleShot(True)
+        self.timer.timeout.connect(self.length_warning)
+        self.mismatch = {'bins': False, 'xs': False}
 
     def _set_full_data(self, xs, histogram_bins, histograms_counts):
         self.xy_plot.clear()
@@ -59,9 +64,9 @@ class XYHistPlot(QtWidgets.QSplitter):
             point.histogram_index = index
             point.histogram_counts = counts
 
-        self.hist_plot_data = self.hist_plot.plot(
+        text = "click on a data point at the left\n"\
-            stepMode=True, fillLevel=0,
+               "to see the corresponding histogram"
-            brush=(0, 0, 255, 150))
+        self.hist_plot.addItem(pyqtgraph.TextItem(text))
 
     def _set_partial_data(self, xs, histograms_counts):
         ys = _compute_ys(self.histogram_bins, histograms_counts)
@@ -87,8 +92,17 @@ class XYHistPlot(QtWidgets.QSplitter):
         else:
             self.arrow.setPos(position)
         self.selected_index = spot_item.histogram_index
-        self.hist_plot_data.setData(x=self.histogram_bins,
+
-                                    y=spot_item.histogram_counts)
+        if self.hist_plot_data is None:
+            self.hist_plot.clear()
+            self.hist_plot_data = self.hist_plot.plot(
+                x=self.histogram_bins,
+                y=spot_item.histogram_counts,
+                stepMode=True, fillLevel=0,
+                brush=(0, 0, 255, 150))
+        else:
+            self.hist_plot_data.setData(x=self.histogram_bins,
+                                        y=spot_item.histogram_counts)
 
     def _can_use_partial(self, mods):
         if self.hist_plot_data is None:
@@ -117,11 +131,41 @@ class XYHistPlot(QtWidgets.QSplitter):
             histograms_counts = data[self.args.histograms_counts]["value"]
         except KeyError:
             return
+        if len(xs) != histograms_counts.shape[0]:
+            self.mismatch['xs'] = True
+        else:
+            self.mismatch['xs'] = False
+        if histograms_counts.shape[1] != len(histogram_bins) - 1:
+            self.mismatch['bins'] = True
+        else:
+            self.mismatch['bins'] = False
+        if any(self.mismatch.values()):
+            if not self.timer.isActive():
+                self.timer.start(1000)
+            return
+        else:
+            self.timer.stop()
         if self._can_use_partial(mods):
             self._set_partial_data(xs, histograms_counts)
         else:
             self._set_full_data(xs, histogram_bins, histograms_counts)
 
+    def length_warning(self):
+        self.xy_plot.clear()
+        self.hist_plot.clear()
+        text = "⚠️ dataset lengths mismatch:\n\n"
+        if self.mismatch['bins']:
+            text = ''.join([text,
+                            "bin boundaries should have the same length\n"
+                            "as the first dimension of histogram counts."])
+        if self.mismatch['bins'] and self.mismatch['xs']:
+            text = ''.join([text, '\n\n'])
+        if self.mismatch['xs']:
+            text = ''.join([text,
+                            "point abscissas should have the same length\n"
+                            "as the second dimension of histogram counts."])
+        self.xy_plot.addItem(pyqtgraph.TextItem(text))
+
 
 def main():
     applet = SimpleApplet(XYHistPlot)

artiq/build_soc.py

@@ -2,6 +2,7 @@ import os
 import subprocess
 
 from migen import *
+from migen.build.platforms.sinara import kasli
 from misoc.interconnect.csr import *
 from misoc.integration.builder import *
 
@@ -57,11 +58,17 @@ def build_artiq_soc(soc, argdict):
     builder = Builder(soc, **argdict)
     builder.software_packages = []
     builder.add_software_package("bootloader", os.path.join(firmware_dir, "bootloader"))
+    is_kasli_v1 = isinstance(soc.platform, kasli.Platform) and soc.platform.hw_rev in ("v1.0", "v1.1")
     if isinstance(soc, AMPSoC):
-        builder.add_software_package("libm")
+        kernel_cpu_type = "vexriscv" if is_kasli_v1 else "vexriscv-g"
-        builder.add_software_package("libprintf")
+        builder.add_software_package("libm", cpu_type=kernel_cpu_type)
-        builder.add_software_package("libunwind")
+        builder.add_software_package("libprintf", cpu_type=kernel_cpu_type)
-        builder.add_software_package("ksupport", os.path.join(firmware_dir, "ksupport"))
+        builder.add_software_package("libunwind", cpu_type=kernel_cpu_type)
+        builder.add_software_package("ksupport", os.path.join(firmware_dir, "ksupport"), cpu_type=kernel_cpu_type)
+        # Generate unwinder for soft float target (ARTIQ runtime)
+        # If the kernel lacks FPU, then the runtime unwinder is already generated
+        if not is_kasli_v1:
+            builder.add_software_package("libunwind")
         builder.add_software_package("runtime", os.path.join(firmware_dir, "runtime"))
     else:
         # Assume DRTIO satellite.

artiq/compiler/embedding.py

@@ -5,7 +5,7 @@ the references to the host objects and translates the functions
 annotated as ``@kernel`` when they are referenced.
 """
 
-import sys, os, re, linecache, inspect, textwrap, types as pytypes, numpy
+import os, re, linecache, inspect, textwrap, types as pytypes, numpy
 from collections import OrderedDict, defaultdict
 
 from pythonparser import ast, algorithm, source, diagnostic, parse_buffer
@@ -156,6 +156,67 @@ class ASTSynthesizer:
         return source.Range(self.source_buffer, range_from, range_to,
                             expanded_from=self.expanded_from)
 
+    def fast_quote_list(self, value):
+        elts = [None] * len(value)
+        is_T = False
+        if len(value) > 0:
+            v = value[0]
+            is_T = True
+            if isinstance(v, int):
+                T = int
+            elif isinstance(v, float):
+                T = float
+            elif isinstance(v, numpy.int32):
+                T = numpy.int32
+            elif isinstance(v, numpy.int64):
+                T = numpy.int64
+            else:
+                is_T = False
+            if is_T:
+                for v in value:
+                    if not isinstance(v, T):
+                        is_T = False
+                        break
+        if is_T:
+            is_int = T != float
+            if T == int:
+                typ = builtins.TInt()
+            elif T == float:
+                typ = builtins.TFloat()
+            elif T == numpy.int32:
+                typ = builtins.TInt32()
+            elif T == numpy.int64:
+                typ = builtins.TInt64()
+            else:
+                assert False
+            text = [repr(elt) for elt in value]
+            start = len(self.source)
+            self.source += ", ".join(text)
+            if is_int:
+                for i, (v, t) in enumerate(zip(value, text)):
+                    l = len(t)
+                    elts[i] = asttyped.NumT(
+                        n=int(v), ctx=None, type=typ,
+                        loc=source.Range(
+                            self.source_buffer, start, start + l,
+                            expanded_from=self.expanded_from))
+                    start += l + 2
+            else:
+                for i, (v, t) in enumerate(zip(value, text)):
+                    l = len(t)
+                    elts[i] = asttyped.NumT(
+                        n=v, ctx=None, type=typ,
+                        loc=source.Range(
+                            self.source_buffer, start, start + l,
+                            expanded_from=self.expanded_from))
+                    start += l + 2
+        else:
+            for index, elt in enumerate(value):
+                elts[index] = self.quote(elt)
+                if index < len(value) - 1:
+                    self._add(", ")
+        return elts
+
     def quote(self, value):
         """Construct an AST fragment equal to `value`."""
         if value is None:
@@ -217,21 +278,14 @@ class ASTSynthesizer:
             return asttyped.QuoteT(value=value, type=builtins.TByteArray(), loc=loc)
         elif isinstance(value, list):
             begin_loc = self._add("[")
-            elts = []
+            elts = self.fast_quote_list(value)
-            for index, elt in enumerate(value):
-                elts.append(self.quote(elt))
-                if index < len(value) - 1:
-                    self._add(", ")
             end_loc   = self._add("]")
             return asttyped.ListT(elts=elts, ctx=None, type=builtins.TList(),
                                   begin_loc=begin_loc, end_loc=end_loc,
                                   loc=begin_loc.join(end_loc))
         elif isinstance(value, tuple):
             begin_loc = self._add("(")
-            elts = []
+            elts = self.fast_quote_list(value)
-            for index, elt in enumerate(value):
-                elts.append(self.quote(elt))
-                self._add(", ")
             end_loc   = self._add(")")
             return asttyped.TupleT(elts=elts, ctx=None,
                                    type=types.TTuple([e.type for e in elts]),
@@ -683,6 +737,7 @@ class Stitcher:
 
         self.embedding_map = EmbeddingMap()
         self.value_map = defaultdict(lambda: [])
+        self.definitely_changed = False
 
     def stitch_call(self, function, args, kwargs, callback=None):
         # We synthesize source code for the initial call so that
@@ -703,13 +758,19 @@ class Stitcher:
         old_attr_count = None
         while True:
             inferencer.visit(self.typedtree)
-            typedtree_hash = typedtree_hasher.visit(self.typedtree)
+            if self.definitely_changed:
-            attr_count = self.embedding_map.attribute_count()
+                changed = True
+                self.definitely_changed = False
+            else:
+                typedtree_hash = typedtree_hasher.visit(self.typedtree)
+                attr_count = self.embedding_map.attribute_count()
+                changed = old_attr_count != attr_count or \
+                          old_typedtree_hash != typedtree_hash
+                old_typedtree_hash = typedtree_hash
+                old_attr_count = attr_count
 
-            if old_typedtree_hash == typedtree_hash and old_attr_count == attr_count:
+            if not changed:
                 break
-            old_typedtree_hash = typedtree_hash
-            old_attr_count = attr_count
 
         # After we've discovered every referenced attribute, check if any kernel_invariant
         # specifications refers to ones we didn't encounter.
@@ -837,6 +898,9 @@ class Stitcher:
             return types.TVar()
 
     def _quote_embedded_function(self, function, flags):
+        # we are now parsing new functions... definitely changed the type
+        self.definitely_changed = True
+
         if isinstance(function, SpecializedFunction):
             host_function = function.host_function
         else:
@@ -903,13 +967,11 @@ class Stitcher:
 
         # Parse.
         source_buffer = source.Buffer(source_code, filename, first_line)
-        lexer = source_lexer.Lexer(source_buffer, version=sys.version_info[0:2],
+        lexer = source_lexer.Lexer(source_buffer, version=(3, 6), diagnostic_engine=self.engine)
-                                   diagnostic_engine=self.engine)
         lexer.indent = [(initial_indent,
                          source.Range(source_buffer, 0, len(initial_whitespace)),
                          initial_whitespace)]
-        parser = source_parser.Parser(lexer, version=sys.version_info[0:2],
+        parser = source_parser.Parser(lexer, version=(3, 6), diagnostic_engine=self.engine)
-                                      diagnostic_engine=self.engine)
         function_node = parser.file_input().body[0]
 
         # Mangle the name, since we put everything into a single module.
@@ -949,6 +1011,31 @@ class Stitcher:
         if annot is None:
             annot = builtins.TNone()
 
+        if isinstance(function, SpecializedFunction):
+            host_function = function.host_function
+        else:
+            host_function = function
+
+        if hasattr(host_function, 'artiq_embedded'):
+            embedded_function = host_function.artiq_embedded.function
+        else:
+            embedded_function = host_function
+
+        if isinstance(embedded_function, str):
+            embedded_function = host_function
+
+        if isinstance(annot, str):
+            try:
+                annot = eval(annot, embedded_function.__globals__)
+            except Exception:
+                diag = diagnostic.Diagnostic(
+                    "error",
+                    "type annotation for {kind}, {annot}, cannot be evaluated",
+                    {"kind": kind, "annot": repr(annot)},
+                    self._function_loc(function),
+                    notes=self._call_site_note(call_loc, fn_kind))
+                self.engine.process(diag)
+
         if not isinstance(annot, types.Type):
             diag = diagnostic.Diagnostic("error",
                 "type annotation for {kind}, '{annot}', is not an ARTIQ type",

artiq/compiler/kernel.ld

@@ -0,0 +1,56 @@
+/* Force ld to make the ELF header as loadable. */
+PHDRS
+{
+    headers     PT_LOAD FILEHDR PHDRS ;
+    text        PT_LOAD ;
+    data        PT_LOAD ;
+    dynamic     PT_DYNAMIC ;
+    eh_frame    PT_GNU_EH_FRAME ;
+}
+
+SECTIONS
+{
+    /* Push back .text section enough so that ld.lld not complain */
+    . = SIZEOF_HEADERS;
+
+    .text :
+    {
+        *(.text .text.*)
+    } : text
+
+    .rodata :
+    {
+        *(.rodata .rodata.*)
+    }
+
+    .eh_frame :
+    {
+        KEEP(*(.eh_frame))
+    } : text
+
+    .eh_frame_hdr :
+    {
+        KEEP(*(.eh_frame_hdr))
+    } : text : eh_frame
+
+    .data :
+    {
+        *(.data)
+    } : data
+
+    .dynamic :
+    {
+        *(.dynamic)
+    } : data : dynamic
+
+    .bss (NOLOAD) : ALIGN(4)
+    {
+        __bss_start = .;
+        *(.sbss .sbss.* .bss .bss.*);
+        . = ALIGN(4);
+        _end = .;
+    }
+
+    . = ALIGN(0x1000);
+    _sstack_guard = .;
+}

artiq/compiler/targets.py

@@ -1,6 +1,6 @@
 import os, sys, tempfile, subprocess, io
 from artiq.compiler import types, ir
-from llvmlite_artiq import ir as ll, binding as llvm
+from llvmlite import ir as ll, binding as llvm
 
 llvm.initialize()
 llvm.initialize_all_targets()
@@ -28,8 +28,10 @@ class RunTool:
         for argument in self._pattern:
             cmdline.append(argument.format(**self._tempnames))
 
+        # https://bugs.python.org/issue17023
+        windows = os.name == "nt"
         process = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                   universal_newlines=True)
+                                   universal_newlines=True, shell=windows)
         stdout, stderr = process.communicate()
         if process.returncode != 0:
             raise Exception("{} invocation failed: {}".
@@ -67,7 +69,7 @@ class Target:
     generated by the ARTIQ compiler will be deployed.
 
     :var triple: (string)
-        LLVM target triple, e.g. ``"or1k"``
+        LLVM target triple, e.g. ``"riscv32"``
     :var data_layout: (string)
         LLVM target data layout, e.g. ``"E-m:e-p:32:32-i64:32-f64:32-v64:32-v128:32-a:0:32-n32"``
     :var features: (list of string)
@@ -75,9 +77,6 @@ class Target:
     :var print_function: (string)
         Name of a formatted print functions (with the signature of ``printf``)
         provided by the target, e.g. ``"printf"``.
-    :var little_endian: (boolean)
-        Whether the code will be executed on a little-endian machine. This cannot be always
-        determined from data_layout due to JIT.
     :var now_pinning: (boolean)
         Whether the target implements the now-pinning RTIO optimization.
     """
@@ -85,7 +84,6 @@ class Target:
     data_layout = ""
     features = []
     print_function = "printf"
-    little_endian = False
     now_pinning = True
 
     tool_ld = "ld.lld"
@@ -100,7 +98,8 @@ class Target:
         lltarget = llvm.Target.from_triple(self.triple)
         llmachine = lltarget.create_target_machine(
                         features=",".join(["+{}".format(f) for f in self.features]),
-                        reloc="pic", codemodel="default")
+                        reloc="pic", codemodel="default",
+                        abiname="ilp32d" if isinstance(self, RV32GTarget) else "")
         llmachine.set_asm_verbosity(True)
         return llmachine
 
@@ -182,6 +181,7 @@ class Target:
     def link(self, objects):
         """Link the relocatable objects into a shared library for this target."""
         with RunTool([self.tool_ld, "-shared", "--eh-frame-hdr"] +
+                     ["-T" + os.path.join(os.path.dirname(__file__), "kernel.ld")] +
                      ["{{obj{}}}".format(index) for index in range(len(objects))] +
                      ["-x"] +
                      ["-o", "{output}"],
@@ -252,32 +252,39 @@ class NativeTarget(Target):
         super().__init__()
         self.triple = llvm.get_default_triple()
         host_data_layout = str(llvm.targets.Target.from_default_triple().create_target_machine().target_data)
-        assert host_data_layout[0] in "eE"
-        self.little_endian = host_data_layout[0] == "e"
 
-class OR1KTarget(Target):
+class RV32IMATarget(Target):
-    triple = "or1k-linux"
+    triple = "riscv32-unknown-linux"
-    data_layout = "E-m:e-p:32:32-i8:8:8-i16:16:16-i64:32:32-" \
+    data_layout = "e-m:e-p:32:32-i64:64-n32-S128"
-                  "f64:32:32-v64:32:32-v128:32:32-a0:0:32-n32"
+    features = ["m", "a"]
-    features = ["mul", "div", "ffl1", "cmov", "addc"]
     print_function = "core_log"
-    little_endian = False
     now_pinning = True
 
-    tool_ld = "or1k-linux-ld"
+    tool_ld = "ld.lld"
-    tool_strip = "or1k-linux-strip"
+    tool_strip = "llvm-strip"
-    tool_addr2line = "or1k-linux-addr2line"
+    tool_addr2line = "llvm-addr2line"
-    tool_cxxfilt = "or1k-linux-c++filt"
+    tool_cxxfilt = "llvm-cxxfilt"
+
+class RV32GTarget(Target):
+    triple = "riscv32-unknown-linux"
+    data_layout = "e-m:e-p:32:32-i64:64-n32-S128"
+    features = ["m", "a", "f", "d"]
+    print_function = "core_log"
+    now_pinning = True
+
+    tool_ld = "ld.lld"
+    tool_strip = "llvm-strip"
+    tool_addr2line = "llvm-addr2line"
+    tool_cxxfilt = "llvm-cxxfilt"
 
 class CortexA9Target(Target):
     triple = "armv7-unknown-linux-gnueabihf"
     data_layout = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
     features = ["dsp", "fp16", "neon", "vfp3"]
     print_function = "core_log"
-    little_endian = True
     now_pinning = False
 
-    tool_ld = "armv7-unknown-linux-gnueabihf-ld"
+    tool_ld = "ld.lld"
-    tool_strip = "armv7-unknown-linux-gnueabihf-strip"
+    tool_strip = "llvm-strip"
-    tool_addr2line = "armv7-unknown-linux-gnueabihf-addr2line"
+    tool_addr2line = "llvm-addr2line"
-    tool_cxxfilt = "armv7-unknown-linux-gnueabihf-c++filt"
+    tool_cxxfilt = "llvm-cxxfilt"

artiq/compiler/testbench/jit.py

@@ -1,6 +1,6 @@
 import os, sys, fileinput, ctypes
 from pythonparser import diagnostic
-from llvmlite_artiq import binding as llvm
+from llvmlite import binding as llvm
 from ..module import Module, Source
 from ..targets import NativeTarget
 

artiq/compiler/testbench/llvmgen.py

@@ -1,6 +1,6 @@
 import sys, fileinput
 from pythonparser import diagnostic
-from llvmlite_artiq import ir as ll
+from llvmlite import ir as ll
 from ..module import Module, Source
 from ..targets import NativeTarget
 

artiq/compiler/testbench/perf.py

@@ -1,7 +1,7 @@
 import sys, os
 from pythonparser import diagnostic
 from ..module import Module, Source
-from ..targets import OR1KTarget
+from ..targets import RV32GTarget
 from . import benchmark
 
 def main():
@@ -30,7 +30,7 @@ def main():
     benchmark(lambda: Module(source),
               "ARTIQ transforms and validators")
 
-    benchmark(lambda: OR1KTarget().compile_and_link([module]),
+    benchmark(lambda: RV32GTarget().compile_and_link([module]),
               "LLVM optimization and linking")
 
 if __name__ == "__main__":

artiq/compiler/testbench/perf_embedding.py

@@ -5,7 +5,7 @@ from ...master.databases import DeviceDB, DatasetDB
 from ...master.worker_db import DeviceManager, DatasetManager
 from ..module import Module
 from ..embedding import Stitcher
-from ..targets import OR1KTarget
+from ..targets import RV32GTarget
 from . import benchmark
 
 
@@ -45,7 +45,7 @@ def main():
 
     stitcher = embed()
     module = Module(stitcher)
-    target = OR1KTarget()
+    target = RV32GTarget()
     llvm_ir = target.compile(module)
     elf_obj = target.assemble(llvm_ir)
     elf_shlib = target.link([elf_obj])

artiq/compiler/testbench/shlib.py

@@ -1,7 +1,7 @@
 import sys, os
 from pythonparser import diagnostic
 from ..module import Module, Source
-from ..targets import OR1KTarget
+from ..targets import RV32GTarget
 
 def main():
     if not len(sys.argv) > 1:
@@ -20,7 +20,7 @@ def main():
     for filename in sys.argv[1:]:
         modules.append(Module(Source.from_filename(filename, engine=engine)))
 
-    llobj = OR1KTarget().compile_and_link(modules)
+    llobj = RV32GTarget().compile_and_link(modules)
 
     basename, ext = os.path.splitext(sys.argv[-1])
     with open(basename + ".so", "wb") as f:

artiq/compiler/transforms/inferencer.py

@@ -6,6 +6,7 @@ from collections import OrderedDict
 from pythonparser import algorithm, diagnostic, ast
 from .. import asttyped, types, builtins
 from .typedtree_printer import TypedtreePrinter
+from artiq.experiment import kernel
 
 
 def is_nested_empty_list(node):
@@ -1662,7 +1663,14 @@ class Inferencer(algorithm.Visitor):
 
     def visit_FunctionDefT(self, node):
         for index, decorator in enumerate(node.decorator_list):
-            if types.is_builtin(decorator.type, "kernel") or \
+            def eval_attr(attr):
+                if isinstance(attr.value, asttyped.QuoteT):
+                    return getattr(attr.value.value, attr.attr)
+                return getattr(eval_attr(attr.value), attr.attr)
+            if isinstance(decorator, asttyped.AttributeT):
+                decorator = eval_attr(decorator)
+            if id(decorator) == id(kernel) or \
+                    types.is_builtin(decorator.type, "kernel") or \
                     isinstance(decorator, asttyped.CallT) and \
                     types.is_builtin(decorator.func.type, "kernel"):
                 continue

artiq/compiler/transforms/llvm_ir_generator.py

@@ -6,10 +6,11 @@ into LLVM intermediate representation.
 import os, re, types as pytypes, numpy
 from collections import defaultdict
 from pythonparser import ast, diagnostic
-from llvmlite_artiq import ir as ll, binding as llvm
+from llvmlite import ir as ll, binding as llvm
 from ...language import core as language_core
 from .. import types, builtins, ir
 from ..embedding import SpecializedFunction
+from artiq.compiler.targets import RV32GTarget
 
 
 llvoid     = ll.VoidType()
@@ -37,93 +38,6 @@ def memoize(generator):
             return result
     return memoized
 
-class DebugInfoEmitter:
-    def __init__(self, llmodule):
-        self.llmodule = llmodule
-        self.llcompileunit = None
-        self.cache = {}
-
-        llident = self.llmodule.add_named_metadata('llvm.ident')
-        llident.add(self.emit_metadata(["ARTIQ"]))
-
-        llflags = self.llmodule.add_named_metadata('llvm.module.flags')
-        llflags.add(self.emit_metadata([2, "Debug Info Version", 3]))
-        llflags.add(self.emit_metadata([2, "Dwarf Version", 4]))
-
-    def emit_metadata(self, operands):
-        def map_operand(operand):
-            if operand is None:
-                return ll.Constant(llmetadata, None)
-            elif isinstance(operand, str):
-                return ll.MetaDataString(self.llmodule, operand)
-            elif isinstance(operand, int):
-                return ll.Constant(lli32, operand)
-            elif isinstance(operand, (list, tuple)):
-                return self.emit_metadata(operand)
-            else:
-                assert isinstance(operand, ll.NamedValue)
-                return operand
-        return self.llmodule.add_metadata(list(map(map_operand, operands)))
-
-    def emit_debug_info(self, kind, operands, is_distinct=False):
-        return self.llmodule.add_debug_info(kind, operands, is_distinct)
-
-    @memoize
-    def emit_file(self, source_buffer):
-        source_dir, source_file = os.path.split(source_buffer.name)
-        return self.emit_debug_info("DIFile", {
-            "filename":        source_file,
-            "directory":       source_dir,
-        })
-
-    @memoize
-    def emit_compile_unit(self, source_buffer):
-        return self.emit_debug_info("DICompileUnit", {
-            "language":        ll.DIToken("DW_LANG_Python"),
-            "file":            self.emit_file(source_buffer),
-            "producer":        "ARTIQ",
-            "runtimeVersion":  0,
-            "emissionKind":    2, # full=1, lines only=2
-        }, is_distinct=True)
-
-    @memoize
-    def emit_subroutine_type(self, typ):
-        return self.emit_debug_info("DISubroutineType", {
-            "types":           self.emit_metadata([None])
-        })
-
-    @memoize
-    def emit_subprogram(self, func, llfunc):
-        source_buffer = func.loc.source_buffer
-
-        if self.llcompileunit is None:
-            self.llcompileunit = self.emit_compile_unit(source_buffer)
-            llcompileunits = self.llmodule.add_named_metadata('llvm.dbg.cu')
-            llcompileunits.add(self.llcompileunit)
-
-        display_name = "{}{}".format(func.name, types.TypePrinter().name(func.type))
-        return self.emit_debug_info("DISubprogram", {
-            "name":            func.name,
-            "linkageName":     llfunc.name,
-            "type":            self.emit_subroutine_type(func.type),
-            "file":            self.emit_file(source_buffer),
-            "line":            func.loc.line(),
-            "unit":            self.llcompileunit,
-            "scope":           self.emit_file(source_buffer),
-            "scopeLine":       func.loc.line(),
-            "isLocal":         func.is_internal,
-            "isDefinition":    True,
-            "variables":       self.emit_metadata([])
-        }, is_distinct=True)
-
-    @memoize
-    def emit_loc(self, loc, scope):
-        return self.emit_debug_info("DILocation", {
-            "line":            loc.line(),
-            "column":          loc.column(),
-            "scope":           scope
-        })
-
 
 class ABILayoutInfo:
     """Caches DataLayout size/alignment lookup results.
@@ -170,16 +84,8 @@ class LLVMIRGenerator:
         self.llmap = {}
         self.llobject_map = {}
         self.phis = []
-        self.debug_info_emitter = DebugInfoEmitter(self.llmodule)
         self.empty_metadata = self.llmodule.add_metadata([])
-        self.tbaa_tree = self.llmodule.add_metadata([
+        self.quote_fail_msg = None
-            ll.MetaDataString(self.llmodule, "ARTIQ TBAA")
-        ])
-        self.tbaa_nowrite_call = self.llmodule.add_metadata([
-            ll.MetaDataString(self.llmodule, "ref-only function call"),
-            self.tbaa_tree,
-            ll.Constant(lli64, 1)
-        ])
 
     def needs_sret(self, lltyp, may_be_large=True):
         if isinstance(lltyp, ll.VoidType):
@@ -658,10 +564,6 @@ class LLVMIRGenerator:
             self.llbuilder = ll.IRBuilder()
             llblock_map = {}
 
-            if not func.is_generated:
-                lldisubprogram = self.debug_info_emitter.emit_subprogram(func, self.llfunction)
-                self.llfunction.set_metadata('dbg', lldisubprogram)
-
             # First, map arguments.
             if self.has_sret(func.type):
                 llactualargs = self.llfunction.args[1:]
@@ -681,10 +583,6 @@ class LLVMIRGenerator:
             for block in func.basic_blocks:
                 self.llbuilder.position_at_end(self.llmap[block])
                 for insn in block.instructions:
-                    if insn.loc is not None and not func.is_generated:
-                        self.llbuilder.debug_metadata = \
-                            self.debug_info_emitter.emit_loc(insn.loc, lldisubprogram)
-
                     llinsn = getattr(self, "process_" + type(insn).__name__)(insn)
                     assert llinsn is not None
                     self.llmap[insn] = llinsn
@@ -1198,19 +1096,30 @@ class LLVMIRGenerator:
             return self.map(insn.operands[0])
         elif insn.op == "now_mu":
             if self.target.now_pinning:
-                return self.llbuilder.load(self.llbuiltin("now"), name=insn.name)
+                # Word swap now.old as CPU is little endian
+                # Most significant word is stored in lower address (see generated csr.rs)
+                csr_offset = 2 if isinstance(self.target, RV32GTarget) else 1
+
+                llnow_hiptr = self.llbuilder.bitcast(self.llbuiltin("now"), lli32.as_pointer())
+                llnow_loptr = self.llbuilder.gep(llnow_hiptr, [self.llindex(csr_offset)])
+                llnow_hi = self.llbuilder.load(llnow_hiptr, name="now.hi")
+                llnow_lo = self.llbuilder.load(llnow_loptr, name="now.lo")
+                llzext_hi = self.llbuilder.zext(llnow_hi, lli64)
+                llshifted_hi = self.llbuilder.shl(llzext_hi, ll.Constant(lli64, 32))
+                llzext_lo = self.llbuilder.zext(llnow_lo, lli64)
+                return self.llbuilder.or_(llshifted_hi, llzext_lo)
             else:
                 return self.llbuilder.call(self.llbuiltin("now_mu"), [])
         elif insn.op == "at_mu":
             time, = insn.operands
             lltime = self.map(time)
             if self.target.now_pinning:
+                csr_offset = 2 if isinstance(self.target, RV32GTarget) else 1
+
                 lltime_hi = self.llbuilder.trunc(self.llbuilder.lshr(lltime, ll.Constant(lli64, 32)), lli32)
                 lltime_lo = self.llbuilder.trunc(lltime, lli32)
                 llnow_hiptr = self.llbuilder.bitcast(self.llbuiltin("now"), lli32.as_pointer())
-                llnow_loptr = self.llbuilder.gep(llnow_hiptr, [self.llindex(1)])
+                llnow_loptr = self.llbuilder.gep(llnow_hiptr, [self.llindex(csr_offset)])
-                if self.target.little_endian:
-                    lltime_hi, lltime_lo = lltime_lo, lltime_hi
                 llstore_hi = self.llbuilder.store_atomic(lltime_hi, llnow_hiptr, ordering="seq_cst", align=4)
                 llstore_lo = self.llbuilder.store_atomic(lltime_lo, llnow_loptr, ordering="seq_cst", align=4)
                 return llstore_lo
@@ -1220,15 +1129,22 @@ class LLVMIRGenerator:
             interval, = insn.operands
             llinterval = self.map(interval)
             if self.target.now_pinning:
-                llnowptr = self.llbuiltin("now")
+                # Word swap now.old as CPU is little endian
-                llnow = self.llbuilder.load(llnowptr, name="now.old")
+                # Most significant word is stored in lower address (see generated csr.rs)
+                csr_offset = 2 if isinstance(self.target, RV32GTarget) else 1
+
+                llnow_hiptr = self.llbuilder.bitcast(self.llbuiltin("now"), lli32.as_pointer())
+                llnow_loptr = self.llbuilder.gep(llnow_hiptr, [self.llindex(csr_offset)])
+                llnow_hi = self.llbuilder.load(llnow_hiptr, name="now.hi")
+                llnow_lo = self.llbuilder.load(llnow_loptr, name="now.lo")
+                llzext_hi = self.llbuilder.zext(llnow_hi, lli64)
+                llshifted_hi = self.llbuilder.shl(llzext_hi, ll.Constant(lli64, 32))
+                llzext_lo = self.llbuilder.zext(llnow_lo, lli64)
+                llnow = self.llbuilder.or_(llshifted_hi, llzext_lo)
+
                 lladjusted = self.llbuilder.add(llnow, llinterval, name="now.new")
                 lladjusted_hi = self.llbuilder.trunc(self.llbuilder.lshr(lladjusted, ll.Constant(lli64, 32)), lli32)
                 lladjusted_lo = self.llbuilder.trunc(lladjusted, lli32)
-                llnow_hiptr = self.llbuilder.bitcast(llnowptr, lli32.as_pointer())
-                llnow_loptr = self.llbuilder.gep(llnow_hiptr, [self.llindex(1)])
-                if self.target.little_endian:
-                    lladjusted_hi, lladjusted_lo = lladjusted_lo, lladjusted_hi
                 llstore_hi = self.llbuilder.store_atomic(lladjusted_hi, llnow_hiptr, ordering="seq_cst", align=4)
                 llstore_lo = self.llbuilder.store_atomic(lladjusted_lo, llnow_loptr, ordering="seq_cst", align=4)
                 return llstore_lo
@@ -1258,26 +1174,32 @@ class LLVMIRGenerator:
         else:
             llfun = self.map(insn.static_target_function)
         llenv     = self.llbuilder.extract_value(llclosure, 0, name="env.fun")
-        return llfun, [llenv] + list(llargs)
+        return llfun, [llenv] + list(llargs), {}
 
     def _prepare_ffi_call(self, insn):
         llargs = []
-        byvals = []
+        llarg_attrs = {}
         for i, arg in enumerate(insn.arguments()):
             llarg = self.map(arg)
             if isinstance(llarg.type, (ll.LiteralStructType, ll.IdentifiedStructType)):
                 llslot = self.llbuilder.alloca(llarg.type)
                 self.llbuilder.store(llarg, llslot)
                 llargs.append(llslot)
-                byvals.append(i)
+                llarg_attrs[i] = "byval"
             else:
                 llargs.append(llarg)
 
+        llretty = self.llty_of_type(insn.type, for_return=True)
+        is_sret = self.needs_sret(llretty)
+        if is_sret:
+            llarg_attrs = {i + 1: a for (i, a) in llarg_attrs.items()}
+            llarg_attrs[0] = "sret"
+
         llfunname = insn.target_function().type.name
         llfun     = self.llmodule.globals.get(llfunname)
         if llfun is None:
-            llretty = self.llty_of_type(insn.type, for_return=True)
+            # Function has not been declared in the current LLVM module, do it now.
-            if self.needs_sret(llretty):
+            if is_sret:
                 llfunty = ll.FunctionType(llvoid, [llretty.as_pointer()] +
                                           [llarg.type for llarg in llargs])
             else:
@@ -1285,15 +1207,14 @@ class LLVMIRGenerator:
 
             llfun = ll.Function(self.llmodule, llfunty,
                                 insn.target_function().type.name)
-            if self.needs_sret(llretty):
+            for idx, attr in llarg_attrs.items():
-                llfun.args[0].add_attribute('sret')
+                llfun.args[idx].add_attribute(attr)
-                byvals = [i + 1 for i in byvals]
-            for i in byvals:
-                llfun.args[i].add_attribute('byval')
             if 'nounwind' in insn.target_function().type.flags:
                 llfun.attributes.add('nounwind')
+            if 'nowrite' in insn.target_function().type.flags:
+                llfun.attributes.add('inaccessiblememonly')
 
-        return llfun, list(llargs)
+        return llfun, list(llargs), llarg_attrs
 
     def _build_rpc(self, fun_loc, fun_type, args, llnormalblock, llunwindblock):
         llservice = ll.Constant(lli32, fun_type.service)
@@ -1429,31 +1350,27 @@ class LLVMIRGenerator:
                                    insn.arguments(),
                                    llnormalblock=None, llunwindblock=None)
         elif types.is_external_function(functiontyp):
-            llfun, llargs = self._prepare_ffi_call(insn)
+            llfun, llargs, llarg_attrs = self._prepare_ffi_call(insn)
         else:
-            llfun, llargs = self._prepare_closure_call(insn)
+            llfun, llargs, llarg_attrs = self._prepare_closure_call(insn)
 
         if self.has_sret(functiontyp):
             llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [])
 
             llresultslot = self.llbuilder.alloca(llfun.type.pointee.args[0].pointee)
-            llcall = self.llbuilder.call(llfun, [llresultslot] + llargs)
+            self.llbuilder.call(llfun, [llresultslot] + llargs, arg_attrs=llarg_attrs)
             llresult = self.llbuilder.load(llresultslot)
 
             self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
         else:
-            llcall = llresult = self.llbuilder.call(llfun, llargs, name=insn.name)
+            llresult = self.llbuilder.call(llfun, llargs, name=insn.name,
+                                           arg_attrs=llarg_attrs)
 
             if isinstance(llresult.type, ll.VoidType):
                 # We have NoneType-returning functions return void, but None is
                 # {} elsewhere.
                 llresult = ll.Constant(llunit, [])
 
-            # Never add TBAA nowrite metadata to a functon with sret!
-            # This leads to miscompilations.
-            if types.is_external_function(functiontyp) and 'nowrite' in functiontyp.flags:
-                llcall.set_metadata('tbaa', self.tbaa_nowrite_call)
-
         return llresult
 
     def process_Invoke(self, insn):
@@ -1466,16 +1383,17 @@ class LLVMIRGenerator:
                                    insn.arguments(),
                                    llnormalblock, llunwindblock)
         elif types.is_external_function(functiontyp):
-            llfun, llargs = self._prepare_ffi_call(insn)
+            llfun, llargs, llarg_attrs = self._prepare_ffi_call(insn)
         else:
-            llfun, llargs = self._prepare_closure_call(insn)
+            llfun, llargs, llarg_attrs = self._prepare_closure_call(insn)
 
         if self.has_sret(functiontyp):
             llstackptr = self.llbuilder.call(self.llbuiltin("llvm.stacksave"), [])
 
             llresultslot = self.llbuilder.alloca(llfun.type.pointee.args[0].pointee)
             llcall = self.llbuilder.invoke(llfun, [llresultslot] + llargs,
-                                           llnormalblock, llunwindblock, name=insn.name)
+                                           llnormalblock, llunwindblock, name=insn.name,
+                                           arg_attrs=llarg_attrs)
 
             self.llbuilder.position_at_start(llnormalblock)
             llresult = self.llbuilder.load(llresultslot)
@@ -1483,7 +1401,7 @@ class LLVMIRGenerator:
             self.llbuilder.call(self.llbuiltin("llvm.stackrestore"), [llstackptr])
         else:
             llcall = self.llbuilder.invoke(llfun, llargs, llnormalblock, llunwindblock,
-                                           name=insn.name)
+                                           name=insn.name, arg_attrs=llarg_attrs)
             llresult = llcall
 
             # The !tbaa metadata is not legal to use with the invoke instruction,
@@ -1492,8 +1410,24 @@ class LLVMIRGenerator:
         return llresult
 
     def _quote_listish_to_llglobal(self, value, elt_type, path, kind_name):
-        llelts    = [self._quote(value[i], elt_type, lambda: path() + [str(i)])
+        fail_msg = "at " + ".".join(path())
-                     for i in range(len(value))]
+        if len(value) > 0:
+            if builtins.is_int(elt_type):
+                int_typ = (int, numpy.int32, numpy.int64)
+                for v in value:
+                    assert isinstance(v, int_typ), fail_msg
+                llty = self.llty_of_type(elt_type)
+                llelts = [ll.Constant(llty, int(v)) for v in value]
+            elif builtins.is_float(elt_type):
+                for v in value:
+                    assert isinstance(v, float), fail_msg
+                llty = self.llty_of_type(elt_type)
+                llelts = [ll.Constant(llty, v) for v in value]
+            else:
+                llelts = [self._quote(value[i], elt_type, lambda: path() + [str(i)])
+                          for i in range(len(value))]
+        else:
+            llelts = []
         lleltsary = ll.Constant(ll.ArrayType(self.llty_of_type(elt_type), len(llelts)),
                                 list(llelts))
         name = self.llmodule.scope.deduplicate("quoted.{}".format(kind_name))
@@ -1502,60 +1436,63 @@ class LLVMIRGenerator:
         llglobal.linkage = "private"
         return llglobal.bitcast(lleltsary.type.element.as_pointer())
 
+    def _quote_attributes(self, value, typ, path, value_id, llty):
+        llglobal = None
+        llfields = []
+        emit_as_constant = True
+        for attr in typ.attributes:
+            if attr == "__objectid__":
+                objectid = self.embedding_map.store_object(value)
+                llfields.append(ll.Constant(lli32, objectid))
+
+                assert llglobal is None
+                if types.is_constructor(typ):
+                    llglobal = self.get_class(typ)
+                else:
+                    llglobal = ll.GlobalVariable(self.llmodule, llty.pointee,
+                                                 name="O.{}".format(objectid))
+
+                self.llobject_map[value_id] = llglobal
+            else:
+                attrvalue = getattr(value, attr)
+                is_class_function = (types.is_constructor(typ) and
+                                     types.is_function(typ.attributes[attr]) and
+                                     not types.is_external_function(typ.attributes[attr]))
+                if is_class_function:
+                    attrvalue = self.embedding_map.specialize_function(typ.instance, attrvalue)
+                if not (types.is_instance(typ) and attr in typ.constant_attributes):
+                    emit_as_constant = False
+                llattrvalue = self._quote(attrvalue, typ.attributes[attr],
+                                          lambda: path() + [attr])
+                llfields.append(llattrvalue)
+                if is_class_function:
+                    llclosureptr = self.get_global_closure_ptr(typ, attr)
+                    llclosureptr.initializer = llattrvalue
+
+        llglobal.global_constant = emit_as_constant
+        llglobal.initializer = ll.Constant(llty.pointee, llfields)
+        llglobal.linkage = "private"
+        return llglobal
+
     def _quote(self, value, typ, path):
         value_id = id(value)
         if value_id in self.llobject_map:
             return self.llobject_map[value_id]
         llty = self.llty_of_type(typ)
 
-        def _quote_attributes():
+        fail_msg = self.quote_fail_msg
-            llglobal = None
+        if fail_msg == None:
-            llfields = []
+            self.quote_fail_msg = fail_msg = "at " + ".".join(path())
-            emit_as_constant = True
-            for attr in typ.attributes:
-                if attr == "__objectid__":
-                    objectid = self.embedding_map.store_object(value)
-                    llfields.append(ll.Constant(lli32, objectid))
 
-                    assert llglobal is None
-                    if types.is_constructor(typ):
-                        llglobal = self.get_class(typ)
-                    else:
-                        llglobal = ll.GlobalVariable(self.llmodule, llty.pointee,
-                                                     name="O.{}".format(objectid))
-
-                    self.llobject_map[value_id] = llglobal
-                else:
-                    attrvalue = getattr(value, attr)
-                    is_class_function = (types.is_constructor(typ) and
-                                         types.is_function(typ.attributes[attr]) and
-                                         not types.is_external_function(typ.attributes[attr]))
-                    if is_class_function:
-                        attrvalue = self.embedding_map.specialize_function(typ.instance, attrvalue)
-                    if not (types.is_instance(typ) and attr in typ.constant_attributes):
-                        emit_as_constant = False
-                    llattrvalue = self._quote(attrvalue, typ.attributes[attr],
-                                              lambda: path() + [attr])
-                    llfields.append(llattrvalue)
-                    if is_class_function:
-                        llclosureptr = self.get_global_closure_ptr(typ, attr)
-                        llclosureptr.initializer = llattrvalue
-
-            llglobal.global_constant = emit_as_constant
-            llglobal.initializer = ll.Constant(llty.pointee, llfields)
-            llglobal.linkage = "private"
-            return llglobal
-
-        fail_msg = "at " + ".".join(path())
         if types.is_constructor(typ) or types.is_instance(typ):
             if types.is_instance(typ):
                 # Make sure the class functions are quoted, as this has the side effect of
                 # initializing the global closures.
                 self._quote(type(value), typ.constructor,
                             lambda: path() + ['__class__'])
-            return _quote_attributes()
+            return self._quote_attributes(value, typ, path, value_id, llty)
         elif types.is_module(typ):
-            return _quote_attributes()
+            return self._quote_attributes(value, typ, path, value_id, llty)
         elif builtins.is_none(typ):
             assert value is None, fail_msg
             return ll.Constant.literal_struct([])

artiq/compiler/types.py

@@ -3,6 +3,7 @@ The :mod:`types` module contains the classes describing the types
 in :mod:`asttyped`.
 """
 
+import builtins
 import string
 from collections import OrderedDict
 from . import iodelay
@@ -55,40 +56,39 @@ class TVar(Type):
 
     def __init__(self):
         self.parent = self
+        self.rank = 0
 
     def find(self):
-        if self.parent is self:
+        parent = self.parent
+        if parent is self:
             return self
         else:
             # The recursive find() invocation is turned into a loop
             # because paths resulting from unification of large arrays
             # can easily cause a stack overflow.
             root = self
-            while root.__class__ == TVar:
+            while parent.__class__ == TVar and root is not parent:
-                if root is root.parent:
+                _, parent = root, root.parent = parent, parent.parent
-                    break
+            return root.parent
-                else:
-                    root = root.parent
-
-            # path compression
-            iter = self
-            while iter.__class__ == TVar:
-                if iter is root:
-                    break
-                else:
-                    iter, iter.parent = iter.parent, root
-
-            return root
 
     def unify(self, other):
         if other is self:
             return
-        other = other.find()
+        x = other.find()
-
+        y = self.find()
-        if self.parent is self:
+        if x is y:
-            self.parent = other
+            return
+        if y.__class__ == TVar:
+            if x.__class__ == TVar:
+                if x.rank < y.rank:
+                    x, y = y, x
+                y.parent = x
+                if x.rank == y.rank:
+                    x.rank += 1
+            else:
+                y.parent = x
         else:
-            self.find().unify(other)
+            y.unify(x)
 
     def fold(self, accum, fn):
         if self.parent is self:
@@ -97,6 +97,8 @@ class TVar(Type):
             return self.find().fold(accum, fn)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         if self.parent is self:
             return "<artiq.compiler.types.TVar %d>" % id(self)
         else:
@@ -143,6 +145,8 @@ class TMono(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+                return str(self)
         return "artiq.compiler.types.TMono(%s, %s)" % (repr(self.name), repr(self.params))
 
     def __getitem__(self, param):
@@ -191,6 +195,8 @@ class TTuple(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TTuple(%s)" % repr(self.elts)
 
     def __eq__(self, other):
@@ -269,6 +275,8 @@ class TFunction(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TFunction({}, {}, {})".format(
             repr(self.args), repr(self.optargs), repr(self.ret))
 
@@ -296,7 +304,7 @@ class TExternalFunction(TFunction):
         mangling rules).
     :ivar flags: (set of str) function flags.
         Flag ``nounwind`` means the function never raises an exception.
-        Flag ``nowrite`` means the function never writes any memory
+        Flag ``nowrite`` means the function never accesses any memory
         that the ARTIQ Python code can observe.
     :ivar broadcast_across_arrays: (bool)
         If True, the function is transparently applied element-wise when called
@@ -362,6 +370,8 @@ class TRPC(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TRPC({})".format(repr(self.ret))
 
     def __eq__(self, other):
@@ -399,6 +409,8 @@ class TBuiltin(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.{}({})".format(type(self).__name__, repr(self.name))
 
     def __eq__(self, other):
@@ -459,6 +471,8 @@ class TInstance(TMono):
         self.constant_attributes = set()
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TInstance({}, {})".format(
                     repr(self.name), repr(self.attributes))
 
@@ -474,6 +488,8 @@ class TModule(TMono):
         self.constant_attributes = set()
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TModule({}, {})".format(
                     repr(self.name), repr(self.attributes))
 
@@ -513,6 +529,8 @@ class TValue(Type):
         return fn(accum, self)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         return "artiq.compiler.types.TValue(%s)" % repr(self.value)
 
     def __eq__(self, other):
@@ -571,6 +589,8 @@ class TDelay(Type):
         return not (self == other)
 
     def __repr__(self):
+        if getattr(builtins, "__in_sphinx__", False):
+            return str(self)
         if self.duration is None:
             return "<{}.TIndeterminateDelay>".format(__name__)
         elif self.cause is None:

artiq/coredevice/ad53xx.py

@@ -127,9 +127,9 @@ class AD53xx:
       transactions (default: 1)
     :param div_write: SPI clock divider for write operations (default: 4,
       50MHz max SPI clock with {t_high, t_low} >=8ns)
-    :param div_read: SPI clock divider for read operations (default: 8, not
+    :param div_read: SPI clock divider for read operations (default: 16, not
-      optimized for speed, but cf data sheet t22: 25ns min SCLK edge to SDO
+      optimized for speed; datasheet says t22: 25ns min SCLK edge to SDO
-      valid)
+      valid, and suggests the SPI speed for reads should be <=20 MHz)
     :param vref: DAC reference voltage (default: 5.)
     :param offset_dacs: Initial register value for the two offset DACs, device
       dependent and must be set correctly for correct voltage to mu

artiq/coredevice/ad9910.py

@@ -324,7 +324,7 @@ class AD9910:
         self.bus.write(data_low)
 
     @kernel
-    def write_ram(self, data: TList(int32)):
+    def write_ram(self, data: TList(TInt32)):
         """Write data to RAM.
 
         The profile to write to and the step, start, and end address
@@ -345,7 +345,7 @@ class AD9910:
         self.bus.write(data[len(data) - 1])
 
     @kernel
-    def read_ram(self, data: TList(int32)):
+    def read_ram(self, data: TList(TInt32)):
         """Read data from RAM.
 
         The profile to read from and the step, start, and end address
@@ -374,18 +374,25 @@ class AD9910:
             data[(n - preload) + i] = self.bus.read()
 
     @kernel
-    def set_cfr1(self, power_down: TInt32 = 0b0000,
+    def set_cfr1(self,
+                 power_down: TInt32 = 0b0000,
                  phase_autoclear: TInt32 = 0,
-                 drg_load_lrr: TInt32 = 0, drg_autoclear: TInt32 = 0,
+                 drg_load_lrr: TInt32 = 0,
-                 internal_profile: TInt32 = 0, ram_destination: TInt32 = 0,
+                 drg_autoclear: TInt32 = 0,
-                 ram_enable: TInt32 = 0, manual_osk_external: TInt32 = 0,
+                 phase_clear: TInt32 = 0,
-                 osk_enable: TInt32 = 0, select_auto_osk: TInt32 = 0):
+                 internal_profile: TInt32 = 0,
+                 ram_destination: TInt32 = 0,
+                 ram_enable: TInt32 = 0,
+                 manual_osk_external: TInt32 = 0,
+                 osk_enable: TInt32 = 0,
+                 select_auto_osk: TInt32 = 0):
         """Set CFR1. See the AD9910 datasheet for parameter meanings.
 
         This method does not pulse IO_UPDATE.
 
         :param power_down: Power down bits.
         :param phase_autoclear: Autoclear phase accumulator.
+        :param phase_clear: Asynchronous, static reset of the phase accumulator.
         :param drg_load_lrr: Load digital ramp generator LRR.
         :param drg_autoclear: Autoclear digital ramp generator.
         :param internal_profile: Internal profile control.
@@ -405,11 +412,41 @@ class AD9910:
                      (drg_load_lrr << 15) |
                      (drg_autoclear << 14) |
                      (phase_autoclear << 13) |
+                     (phase_clear << 11) |
                      (osk_enable << 9) |
                      (select_auto_osk << 8) |
                      (power_down << 4) |
                      2)  # SDIO input only, MSB first
 
+    @kernel
+    def set_cfr2(self, 
+                 asf_profile_enable: TInt32 = 1, 
+                 drg_enable: TInt32 = 0, 
+                 effective_ftw: TInt32 = 1,
+                 sync_validation_disable: TInt32 = 0, 
+                 matched_latency_enable: TInt32 = 0):
+        """Set CFR2. See the AD9910 datasheet for parameter meanings.
+
+        This method does not pulse IO_UPDATE.
+
+        :param asf_profile_enable: Enable amplitude scale from single tone profiles.
+        :param drg_enable: Digital ramp enable.
+        :param effective_ftw: Read effective FTW.
+        :param sync_validation_disable: Disable the SYNC_SMP_ERR pin indicating
+            (active high) detection of a synchronization pulse sampling error.
+        :param matched_latency_enable: Simultaneous application of amplitude,
+            phase, and frequency changes to the DDS arrive at the output
+
+            * matched_latency_enable = 0: in the order listed
+            * matched_latency_enable = 1: simultaneously.
+        """
+        self.write32(_AD9910_REG_CFR2,
+                     (asf_profile_enable << 24) |
+                     (drg_enable << 19) |
+                     (effective_ftw << 16) |
+                     (matched_latency_enable << 7) |
+                     (sync_validation_disable << 5))
+
     @kernel
     def init(self, blind: TBool = False):
         """Initialize and configure the DDS.
@@ -442,7 +479,7 @@ class AD9910:
         # enable amplitude scale from profiles
         # read effective FTW
         # sync timing validation disable (enabled later)
-        self.write32(_AD9910_REG_CFR2, 0x01010020)
+        self.set_cfr2(sync_validation_disable=1)
         self.cpld.io_update.pulse(1 * us)
         cfr3 = (0x0807c000 | (self.pll_vco << 24) |
                 (self.pll_cp << 19) | (self.pll_en << 8) |
@@ -465,7 +502,7 @@ class AD9910:
                     if i >= 100 - 1:
                         raise ValueError("PLL lock timeout")
         delay(10 * us)  # slack
-        if self.sync_data.sync_delay_seed >= 0:
+        if self.sync_data.sync_delay_seed >= 0 and not blind:
             self.tune_sync_delay(self.sync_data.sync_delay_seed)
         delay(1 * ms)
 
@@ -481,7 +518,7 @@ class AD9910:
     @kernel
     def set_mu(self, ftw: TInt32, pow_: TInt32 = 0, asf: TInt32 = 0x3fff,
                phase_mode: TInt32 = _PHASE_MODE_DEFAULT,
-               ref_time_mu: TInt64 = int64(-1), profile: TInt32 = 0):
+               ref_time_mu: TInt64 = int64(-1), profile: TInt32 = 0) -> TInt32:
         """Set profile 0 data in machine units.
 
         This uses machine units (FTW, POW, ASF). The frequency tuning word
@@ -786,7 +823,7 @@ class AD9910:
     @kernel
     def set(self, frequency: TFloat, phase: TFloat = 0.0,
             amplitude: TFloat = 1.0, phase_mode: TInt32 = _PHASE_MODE_DEFAULT,
-            ref_time_mu: TInt64 = int64(-1), profile: TInt32 = 0):
+            ref_time_mu: TInt64 = int64(-1), profile: TInt32 = 0) -> TFloat:
         """Set profile 0 data in SI units.
 
         .. seealso:: :meth:`set_mu`
@@ -875,20 +912,26 @@ class AD9910:
         self.cpld.cfg_sw(self.chip_select - 4, state)
 
     @kernel
-    def set_sync(self, in_delay: TInt32, window: TInt32):
+    def set_sync(self, 
+                 in_delay: TInt32, 
+                 window: TInt32, 
+                 en_sync_gen: TInt32 = 0):
         """Set the relevant parameters in the multi device synchronization
         register. See the AD9910 datasheet for details. The SYNC clock
         generator preset value is set to zero, and the SYNC_OUT generator is
-        disabled.
+        disabled by default.
 
         :param in_delay: SYNC_IN delay tap (0-31) in steps of ~75ps
         :param window: Symmetric SYNC_IN validation window (0-15) in
             steps of ~75ps for both hold and setup margin.
+        :param en_sync_gen: Whether to enable the DDS-internal sync generator
+            (SYNC_OUT, cf. sync_sel == 1). Should be left off for the normal
+            use case, where the SYNC clock is supplied by the core device.
         """
         self.write32(_AD9910_REG_SYNC,
                      (window << 28) |  # SYNC S/H validation delay
                      (1 << 27) |  # SYNC receiver enable
-                     (0 << 26) |  # SYNC generator disable
+                     (en_sync_gen << 26) |  # SYNC generator enable
                      (0 << 25) |  # SYNC generator SYS rising edge
                      (0 << 18) |  # SYNC preset
                      (0 << 11) |  # SYNC output delay
@@ -904,9 +947,10 @@ class AD9910:
 
         Also modifies CFR2.
         """
-        self.write32(_AD9910_REG_CFR2, 0x01010020)  # clear SMP_ERR
+        self.set_cfr2(sync_validation_disable=1)  # clear SMP_ERR
         self.cpld.io_update.pulse(1 * us)
-        self.write32(_AD9910_REG_CFR2, 0x01010000)  # enable SMP_ERR
+        delay(10 * us)  # slack
+        self.set_cfr2(sync_validation_disable=0)  # enable SMP_ERR
         self.cpld.io_update.pulse(1 * us)
 
     @kernel
@@ -984,7 +1028,7 @@ class AD9910:
         # set up DRG
         self.set_cfr1(drg_load_lrr=1, drg_autoclear=1)
         # DRG -> FTW, DRG enable
-        self.write32(_AD9910_REG_CFR2, 0x01090000)
+        self.set_cfr2(drg_enable=1)
         # no limits
         self.write64(_AD9910_REG_RAMP_LIMIT, -1, 0)
         # DRCTL=0, dt=1 t_SYNC_CLK
@@ -1005,7 +1049,7 @@ class AD9910:
         ftw = self.read32(_AD9910_REG_FTW)  # read out effective FTW
         delay(100 * us)  # slack
         # disable DRG
-        self.write32(_AD9910_REG_CFR2, 0x01010000)
+        self.set_cfr2(drg_enable=0)
         self.cpld.io_update.pulse_mu(8)
         return ftw & 1
 

artiq/coredevice/ad9912.py

@@ -156,7 +156,7 @@ class AD9912:
         return self.cpld.get_channel_att(self.chip_select - 4)
 
     @kernel
-    def set_mu(self, ftw: TInt64, pow_: TInt32):
+    def set_mu(self, ftw: TInt64, pow_: TInt32 = 0):
         """Set profile 0 data in machine units.
 
         After the SPI transfer, the shared IO update pin is pulsed to

artiq/coredevice/adf5356.py

@@ -236,6 +236,7 @@ class ADF5356:
         Write all registers to the device. Attempts to lock the PLL.
         """
         f_pfd = self.f_pfd()
+        delay(200 * us)         # Slack
 
         if f_pfd <= 75.0 * MHz:
             for i in range(13, 0, -1):
@@ -249,6 +250,7 @@ class ADF5356:
             n, frac1, (frac2_msb, frac2_lsb), (mod2_msb, mod2_lsb) = calculate_pll(
                 self.f_vco(), f_pfd >> 1
             )
+            delay(200 * us)     # Slack
 
             self.write(
                 13

artiq/coredevice/cache.py

@@ -2,11 +2,11 @@ from artiq.language.core import *
 from artiq.language.types import *
 
 
-@syscall(flags={"nounwind", "nowrite"})
+@syscall(flags={"nounwind"})
 def cache_get(key: TStr) -> TList(TInt32):
     raise NotImplementedError("syscall not simulated")
 
-@syscall(flags={"nowrite"})
+@syscall
 def cache_put(key: TStr, value: TList(TInt32)) -> TNone:
     raise NotImplementedError("syscall not simulated")
 

artiq/coredevice/comm_kernel.py

@@ -621,6 +621,7 @@ class CommKernel:
         function = self._read_string()
 
         backtrace = [self._read_int32() for _ in range(self._read_int32())]
+        self._process_async_error()
 
         traceback = list(reversed(symbolizer(backtrace))) + \
             [(filename, line, column, *demangler([function]), None)]
@@ -635,6 +636,16 @@ class CommKernel:
         python_exn.artiq_core_exception = core_exn
         raise python_exn
 
+    def _process_async_error(self):
+        errors = self._read_int8()
+        if errors > 0:
+            map_name = lambda y, z: [f"{y}(s)"] if z else []
+            errors = map_name("collision",      errors & 2 ** 0) + \
+                     map_name("busy error",     errors & 2 ** 1) + \
+                     map_name("sequence error", errors & 2 ** 2)
+            logger.warning(f"{(', '.join(errors[:-1]) + ' and ') if len(errors) > 1 else ''}{errors[-1]} "
+                           f"reported during kernel execution")
+
     def serve(self, embedding_map, symbolizer, demangler):
         while True:
             self._read_header()
@@ -646,4 +657,5 @@ class CommKernel:
                 raise exceptions.ClockFailure
             else:
                 self._read_expect(Reply.KernelFinished)
+                self._process_async_error()
                 return

artiq/coredevice/comm_mgmt.py

@@ -20,11 +20,6 @@ class Request(Enum):
     ConfigRemove = 14
     ConfigErase = 15
 
-    StartProfiler = 9
-    StopProfiler = 10
-    GetProfile = 11
-
-    Hotswap = 4
     Reboot = 5
 
     DebugAllocator = 8
@@ -39,8 +34,6 @@ class Reply(Enum):
 
     ConfigData = 7
 
-    Profile = 5
-
     RebootImminent = 3
 
 
@@ -190,45 +183,6 @@ class CommMgmt:
         self._write_header(Request.ConfigErase)
         self._read_expect(Reply.Success)
 
-    def start_profiler(self, interval, edges_size, hits_size):
-        self._write_header(Request.StartProfiler)
-        self._write_int32(interval)
-        self._write_int32(edges_size)
-        self._write_int32(hits_size)
-        self._read_expect(Reply.Success)
-
-    def stop_profiler(self):
-        self._write_header(Request.StopProfiler)
-        self._read_expect(Reply.Success)
-
-    def stop_profiler(self):
-        self._write_header(Request.StopProfiler)
-        self._read_expect(Reply.Success)
-
-    def get_profile(self):
-        self._write_header(Request.GetProfile)
-        self._read_expect(Reply.Profile)
-
-        hits = {}
-        for _ in range(self._read_int32()):
-            addr = self._read_int32()
-            count = self._read_int32()
-            hits[addr] = count
-
-        edges = {}
-        for _ in range(self._read_int32()):
-            caller = self._read_int32()
-            callee = self._read_int32()
-            count = self._read_int32()
-            edges[(caller, callee)] = count
-
-        return hits, edges
-
-    def hotswap(self, firmware):
-        self._write_header(Request.Hotswap)
-        self._write_bytes(firmware)
-        self._read_expect(Reply.RebootImminent)
-
     def reboot(self):
         self._write_header(Request.Reboot)
         self._read_expect(Reply.RebootImminent)

artiq/coredevice/comm_moninj.py

@@ -82,12 +82,12 @@ class CommMonInj:
                 if not ty:
                     return
                 if ty == b"\x00":
-                    payload = await self._reader.read(9)
+                    payload = await self._reader.readexactly(9)
                     channel, probe, value = struct.unpack(
                         self.endian + "lbl", payload)
                     self.monitor_cb(channel, probe, value)
                 elif ty == b"\x01":
-                    payload = await self._reader.read(6)
+                    payload = await self._reader.readexactly(6)
                     channel, override, value = struct.unpack(
                         self.endian + "lbb", payload)
                     self.injection_status_cb(channel, override, value)

artiq/coredevice/core.py

@@ -11,7 +11,7 @@ from artiq.language.units import *
 
 from artiq.compiler.module import Module
 from artiq.compiler.embedding import Stitcher
-from artiq.compiler.targets import OR1KTarget, CortexA9Target
+from artiq.compiler.targets import RV32IMATarget, RV32GTarget, CortexA9Target
 
 from artiq.coredevice.comm_kernel import CommKernel, CommKernelDummy
 # Import for side effects (creating the exception classes).
@@ -71,11 +71,13 @@ class Core:
         "core", "ref_period", "coarse_ref_period", "ref_multiplier",
     }
 
-    def __init__(self, dmgr, host, ref_period, ref_multiplier=8, target="or1k"):
+    def __init__(self, dmgr, host, ref_period, ref_multiplier=8, target="rv32g"):
         self.ref_period = ref_period
         self.ref_multiplier = ref_multiplier
-        if target == "or1k":
+        if target == "rv32g":
-            self.target_cls = OR1KTarget
+            self.target_cls = RV32GTarget
+        elif target == "rv32ima":
+            self.target_cls = RV32IMATarget
         elif target == "cortexa9":
             self.target_cls = CortexA9Target
         else:

artiq/coredevice/coredevice_generic.schema.json

@@ -64,6 +64,13 @@
             "type": "boolean",
             "default": false
         },
+        "sed_lanes": {
+            "type": "number",
+            "minimum": 1,
+            "maximum": 32,
+            "default": 8,
+            "description": "Number of FIFOs in the SED, must be a power of 2"
+        },
         "peripherals": {
             "type": "array",
             "items": {
@@ -127,7 +134,7 @@
             "properties": {
                 "type": {
                     "type": "string",
-                    "enum": ["dio", "urukul", "novogorny", "sampler", "suservo", "zotino", "grabber", "mirny", "fastino", "phaser"]
+                    "enum": ["dio", "urukul", "novogorny", "sampler", "suservo", "zotino", "grabber", "mirny", "fastino", "phaser", "hvamp"]
                 },
                 "board": {
                     "type": "string"
@@ -455,6 +462,28 @@
                     },
                     "required": ["ports"]
                 }
+            }, {
+                "title": "HVAmp",
+                "if": {
+                    "properties": {
+                        "type": {
+                            "const": "hvamp"
+                        }
+                    }
+                },
+                "then": {
+                    "properties": {
+                        "ports": {
+                            "type": "array",
+                            "items": {
+                                "type": "integer"
+                            },
+                            "minItems": 1,
+                            "maxItems": 1
+                        }
+                    },
+                    "required": ["ports"]
+                }
             }]
         }
     }

artiq/coredevice/fastino.py

@@ -3,7 +3,7 @@ streaming DAC.
 """
 from numpy import int32
 
-from artiq.language.core import kernel, portable, delay
+from artiq.language.core import kernel, portable, delay, delay_mu
 from artiq.coredevice.rtio import (rtio_output, rtio_output_wide,
                                    rtio_input_data)
 from artiq.language.units import us
@@ -191,3 +191,82 @@ class Fastino:
             green LED.
         """
         self.write(0x23, leds)
+
+    @kernel
+    def set_continuous(self, channel_mask):
+        """Enable continuous DAC updates on channels regardless of new data
+        being submitted.
+        """
+        self.write(0x25, channel_mask)
+
+    @kernel
+    def stage_cic_mu(self, rate_mantissa, rate_exponent, gain_exponent):
+        """Stage machine unit CIC interpolator configuration.
+        """
+        if rate_mantissa < 0 or rate_mantissa >= 1 << 6:
+            raise ValueError("rate_mantissa out of bounds")
+        if rate_exponent < 0 or rate_exponent >= 1 << 4:
+            raise ValueError("rate_exponent out of bounds")
+        if gain_exponent < 0 or gain_exponent >= 1 << 6:
+            raise ValueError("gain_exponent out of bounds")
+        config = rate_mantissa | (rate_exponent << 6) | (gain_exponent << 10)
+        self.write(0x26, config)
+
+    @kernel
+    def stage_cic(self, rate) -> TInt32:
+        """Compute and stage interpolator configuration.
+
+        This method approximates the desired interpolation rate using a 10 bit
+        floating point representation (6 bit mantissa, 4 bit exponent) and
+        then determines an optimal interpolation gain compensation exponent
+        to avoid clipping. Gains for rates that are powers of two are accurately
+        compensated. Other rates lead to overall less than unity gain (but more
+        than 0.5 gain).
+
+        The overall gain including gain compensation is
+        `actual_rate**order/2**ceil(log2(actual_rate**order))`
+        where `order = 3`.
+
+        Returns the actual interpolation rate.
+        """
+        if rate <= 0 or rate > 1 << 16:
+            raise ValueError("rate out of bounds")
+        rate_mantissa = rate
+        rate_exponent = 0
+        while rate_mantissa > 1 << 6:
+            rate_exponent += 1
+            rate_mantissa >>= 1
+        order = 3
+        gain = 1
+        for i in range(order):
+            gain *= rate_mantissa
+        gain_exponent = 0
+        while gain > 1 << gain_exponent:
+            gain_exponent += 1
+        gain_exponent += order*rate_exponent
+        assert gain_exponent <= order*16
+        self.stage_cic_mu(rate_mantissa - 1, rate_exponent, gain_exponent)
+        return rate_mantissa << rate_exponent
+
+    @kernel
+    def apply_cic(self, channel_mask):
+        """Apply the staged interpolator configuration on the specified channels.
+
+        Each Fastino channel includes a fourth order (cubic) CIC interpolator with
+        variable rate change and variable output gain compensation (see
+        :meth:`stage_cic`).
+
+        Channels using non-unity interpolation rate should have
+        continous DAC updates enabled (see :meth:`set_continuous`) unless
+        their output is supposed to be constant.
+
+        This method resets and settles the affected interpolators. There will be
+        no output updates for the next `order = 3` input samples.
+        Affected channels will only accept one input sample per input sample
+        period. This method synchronizes the input sample period to the current
+        frame on the affected channels.
+
+        If application of new interpolator settings results in a change of the
+        overall gain, there will be a corresponding output step.
+        """
+        self.write(0x27, channel_mask)

artiq/coredevice/mirny.py

@@ -40,9 +40,8 @@ class Mirny:
     :param refclk: Reference clock (SMA, MMCX or on-board 100 MHz oscillator)
         frequency in Hz
     :param clk_sel: Reference clock selection.
-        valid options are: "XO" - onboard crystal oscillator
+        Valid options are: "XO" - onboard crystal oscillator;
-                           "SMA" - front-panel SMA connector
+        "SMA" - front-panel SMA connector; "MMCX" - internal MMCX connector.
-                           "MMCX" - internal MMCX connector
         Passing an integer writes it as ``clk_sel`` in the CPLD's register 1.
         The effect depends on the hardware revision.
     :param core_device: Core device name (default: "core")

artiq/coredevice/pcu.py

@@ -1,77 +0,0 @@
-from .spr import mtspr, mfspr
-from artiq.language.core import kernel
-
-
-_MAX_SPRS_PER_GRP_BITS = 11
-_SPRGROUP_PC = 7 << _MAX_SPRS_PER_GRP_BITS
-_SPR_PCMR_CP    = 0x00000001  # Counter present
-_SPR_PCMR_CISM  = 0x00000004  # Count in supervisor mode
-_SPR_PCMR_CIUM  = 0x00000008  # Count in user mode
-_SPR_PCMR_LA    = 0x00000010  # Load access event
-_SPR_PCMR_SA    = 0x00000020  # Store access event
-_SPR_PCMR_IF    = 0x00000040  # Instruction fetch event
-_SPR_PCMR_DCM   = 0x00000080  # Data cache miss event
-_SPR_PCMR_ICM   = 0x00000100  # Insn cache miss event
-_SPR_PCMR_IFS   = 0x00000200  # Insn fetch stall event
-_SPR_PCMR_LSUS  = 0x00000400  # LSU stall event
-_SPR_PCMR_BS    = 0x00000800  # Branch stall event
-_SPR_PCMR_DTLBM = 0x00001000  # DTLB miss event
-_SPR_PCMR_ITLBM = 0x00002000  # ITLB miss event
-_SPR_PCMR_DDS   = 0x00004000  # Data dependency stall event
-_SPR_PCMR_WPE   = 0x03ff8000  # Watchpoint events
-
-
-@kernel(flags={"nowrite", "nounwind"})
-def _PCCR(n):
-    return _SPRGROUP_PC + n
-
-
-@kernel(flags={"nowrite", "nounwind"})
-def _PCMR(n):
-    return _SPRGROUP_PC + 8 + n
-
-
-class CorePCU:
-    """Core device performance counter unit (PCU) access"""
-    def __init__(self, dmgr, core_device="core"):
-        self.core = dmgr.get(core_device)
-
-    @kernel
-    def start(self):
-        """
-        Configure and clear the kernel CPU performance counters.
-
-        The eight counters are configured to count the following events:
-            * Load or store
-            * Instruction fetch
-            * Data cache miss
-            * Instruction cache miss
-            * Instruction fetch stall
-            * Load-store-unit stall
-            * Branch stall
-            * Data dependency stall
-        """
-        for i in range(8):
-            if not mfspr(_PCMR(i)) & _SPR_PCMR_CP:
-                raise ValueError("counter not present")
-            mtspr(_PCMR(i), 0)
-            mtspr(_PCCR(i), 0)
-        mtspr(_PCMR(0), _SPR_PCMR_CISM | _SPR_PCMR_LA | _SPR_PCMR_SA)
-        mtspr(_PCMR(1), _SPR_PCMR_CISM | _SPR_PCMR_IF)
-        mtspr(_PCMR(2), _SPR_PCMR_CISM | _SPR_PCMR_DCM)
-        mtspr(_PCMR(3), _SPR_PCMR_CISM | _SPR_PCMR_ICM)
-        mtspr(_PCMR(4), _SPR_PCMR_CISM | _SPR_PCMR_IFS)
-        mtspr(_PCMR(5), _SPR_PCMR_CISM | _SPR_PCMR_LSUS)
-        mtspr(_PCMR(6), _SPR_PCMR_CISM | _SPR_PCMR_BS)
-        mtspr(_PCMR(7), _SPR_PCMR_CISM | _SPR_PCMR_DDS)
-
-    @kernel
-    def get(self, r):
-        """
-        Read the performance counters and store the counts in the
-        array provided.
-
-        :param list[int] r: array to store the counter values
-        """
-        for i in range(8):
-            r[i] = mfspr(_PCCR(i))

artiq/coredevice/phaser.py

@@ -1,7 +1,7 @@
 from numpy import int32, int64
 
 from artiq.language.core import kernel, delay_mu, delay
-from artiq.coredevice.rtio import rtio_output, rtio_input_data
+from artiq.coredevice.rtio import rtio_output, rtio_input_data, rtio_input_timestamp
 from artiq.language.units import us, ns, ms, MHz
 from artiq.language.types import TInt32
 from artiq.coredevice.dac34h84 import DAC34H84
@@ -92,7 +92,8 @@ class Phaser:
     The latency/group delay from the RTIO events setting
     :class:`PhaserOscillator` or :class:`PhaserChannel` DUC parameters all the
     way to the DAC outputs is deterministic. This enables deterministic
-    absolute phase with respect to other RTIO input and output events.
+    absolute phase with respect to other RTIO input and output events
+    (see `get_next_frame_mu()`).
 
     The four analog DAC outputs are passed through anti-aliasing filters.
 
@@ -160,6 +161,7 @@ class Phaser:
         # self.core.seconds_to_mu(10*8*4*ns)  # unfortunately this returns 319
         assert self.core.ref_period == 1*ns
         self.t_frame = 10*8*4
+        self.frame_tstamp = int64(0)
         self.clk_sel = clk_sel
         self.tune_fifo_offset = tune_fifo_offset
         self.sync_dly = sync_dly
@@ -188,7 +190,7 @@ class Phaser:
 
         gw_rev = self.read8(PHASER_ADDR_GW_REV)
         if debug:
-            print(gw_rev)
+            print("gw_rev:", gw_rev)
             self.core.break_realtime()
         delay(.1*ms)  # slack
 
@@ -197,6 +199,12 @@ class Phaser:
             raise ValueError("large number of frame CRC errors")
         delay(.1*ms)  # slack
 
+        # determine the origin for frame-aligned timestamps
+        self.measure_frame_timestamp()
+        if self.frame_tstamp < 0:
+            raise ValueError("frame timestamp measurement timed out")
+        delay(.1*ms)
+
         # reset
         self.set_cfg(dac_resetb=0, dac_sleep=1, dac_txena=0,
                      trf0_ps=1, trf1_ps=1,
@@ -262,7 +270,7 @@ class Phaser:
         if self.tune_fifo_offset:
             fifo_offset = self.dac_tune_fifo_offset()
             if debug:
-                print(fifo_offset)
+                print("fifo_offset:", fifo_offset)
                 self.core.break_realtime()
 
         # self.dac_write(0x20, 0x0000)  # stop fifo sync
@@ -274,7 +282,7 @@ class Phaser:
         delay(.1*ms)  # slack
         if alarms & ~0x0040:  # ignore PLL alarms (see DS)
             if debug:
-                print(alarms)
+                print("alarms:", alarms)
                 self.core.break_realtime()
                 # ignore alarms
             else:
@@ -468,6 +476,27 @@ class Phaser:
         """
         return self.read8(PHASER_ADDR_CRC_ERR)
 
+    @kernel
+    def measure_frame_timestamp(self):
+        """Measure the timestamp of an arbitrary frame and store it in `self.frame_tstamp`.
+
+        To be used as reference for aligning updates to the FastLink frames.
+        See `get_next_frame_mu()`.
+        """
+        rtio_output(self.channel_base << 8, 0)  # read any register
+        self.frame_tstamp = rtio_input_timestamp(now_mu() + 4 * self.t_frame, self.channel_base)
+        delay(100 * us)
+
+    @kernel
+    def get_next_frame_mu(self):
+        """Return the timestamp of the frame strictly after `now_mu()`.
+
+        Register updates (DUC, DAC, TRF, etc.) scheduled at this timestamp and multiples
+        of `self.t_frame` later will have deterministic latency to output.
+        """
+        n = int64((now_mu() - self.frame_tstamp) / self.t_frame)
+        return self.frame_tstamp + (n + 1) * self.t_frame
+
     @kernel
     def set_sync_dly(self, dly):
         """Set SYNC delay.
@@ -860,7 +889,7 @@ class PhaserChannel:
 
         By default, the new NCO phase applies on completion of the SPI
         transfer. This also causes a staged NCO frequency to be applied.
-        Different triggers for applying nco settings may be configured through
+        Different triggers for applying NCO settings may be configured through
         the `syncsel_mixerxx` fields in the `dac` configuration dictionary (see
         `__init__()`).
 
@@ -878,7 +907,7 @@ class PhaserChannel:
 
         By default, the new NCO phase applies on completion of the SPI
         transfer. This also causes a staged NCO frequency to be applied.
-        Different triggers for applying nco settings may be configured through
+        Different triggers for applying NCO settings may be configured through
         the `syncsel_mixerxx` fields in the `dac` configuration dictionary (see
         `__init__()`).
 
@@ -1015,7 +1044,7 @@ class PhaserOscillator:
     """Phaser IQ channel oscillator (NCO/DDS).
 
     .. note:: Latencies between oscillators within a channel and between
-        oscillator paramters (amplitude and phase/frequency) are deterministic
+        oscillator parameters (amplitude and phase/frequency) are deterministic
         (with respect to the 25 MS/s sample clock) but not matched.
     """
     kernel_invariants = {"channel", "base_addr"}

artiq/coredevice/profiler.py

@@ -1,92 +0,0 @@
-from collections import defaultdict
-import subprocess
-
-
-class Symbolizer:
-    def __init__(self, binary, triple, demangle=True):
-        cmdline = [
-            triple + "-addr2line", "--exe=" + binary,
-            "--addresses", "--functions", "--inlines"
-        ]
-        if demangle:
-            cmdline.append("--demangle=rust")
-        self._addr2line = subprocess.Popen(cmdline, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
-                                           universal_newlines=True)
-
-    def symbolize(self, addr):
-        self._addr2line.stdin.write("0x{:08x}\n0\n".format(addr))
-        self._addr2line.stdin.flush()
-        self._addr2line.stdout.readline() # 0x[addr]
-
-        result = []
-        while True:
-            function = self._addr2line.stdout.readline().rstrip()
-
-            # check for end marker
-            if function == "0x00000000":          # 0x00000000
-                self._addr2line.stdout.readline() # ??
-                self._addr2line.stdout.readline() # ??:0
-                return result
-
-            file, line = self._addr2line.stdout.readline().rstrip().split(":")
-
-            result.append((function, file, line, addr))
-
-
-class CallgrindWriter:
-    def __init__(self, output, binary, triple, compression=True, demangle=True):
-        self._output = output
-        self._binary = binary
-        self._current = defaultdict(lambda: None)
-        self._ids = defaultdict(lambda: {})
-        self._compression = compression
-        self._symbolizer = Symbolizer(binary, triple, demangle=demangle)
-
-    def _write(self, fmt, *args, **kwargs):
-        self._output.write(fmt.format(*args, **kwargs))
-        self._output.write("\n")
-
-    def _spec(self, spec, value):
-        if self._current[spec] == value:
-            return
-        self._current[spec] = value
-
-        if not self._compression or value == "??":
-            self._write("{}={}", spec, value)
-            return
-
-        spec_ids = self._ids[spec]
-        if value in spec_ids:
-            self._write("{}=({})", spec, spec_ids[value])
-        else:
-            spec_ids[value] = len(spec_ids) + 1
-            self._write("{}=({}) {}", spec, spec_ids[value], value)
-
-    def header(self):
-        self._write("# callgrind format")
-        self._write("version: 1")
-        self._write("creator: ARTIQ")
-        self._write("positions: instr line")
-        self._write("events: Hits")
-        self._write("")
-        self._spec("ob", self._binary)
-        self._spec("cob", self._binary)
-
-    def hit(self, addr, count):
-        for function, file, line, addr in self._symbolizer.symbolize(addr):
-            self._spec("fl", file)
-            self._spec("fn", function)
-            self._write("0x{:08x} {} {}", addr, line, count)
-
-    def edge(self, caller, callee, count):
-        edges = self._symbolizer.symbolize(callee) + self._symbolizer.symbolize(caller)
-        for (callee, caller) in zip(edges, edges[1:]):
-            function, file, line, addr = callee
-            self._spec("cfl", file)
-            self._spec("cfn", function)
-            self._write("calls={} 0x{:08x} {}", count, addr, line)
-
-            function, file, line, addr = caller
-            self._spec("fl", file)
-            self._spec("fn", function)
-            self._write("0x{:08x} {} {}", addr, line, count)

artiq/coredevice/spr.py

@@ -1,12 +0,0 @@
-from artiq.language.core import syscall
-from artiq.language.types import TInt32, TNone
-
-
-@syscall(flags={"nounwind", "nowrite"})
-def mfspr(spr: TInt32) -> TInt32:
-    raise NotImplementedError("syscall not simulated")
-
-
-@syscall(flags={"nowrite", "nowrite"})
-def mtspr(spr: TInt32, value: TInt32) -> TNone:
-    raise NotImplementedError("syscall not simulated")

artiq/coredevice/suservo.py

@@ -57,32 +57,26 @@ class SUServo:
 
     :param channel: RTIO channel number
     :param pgia_device: Name of the Sampler PGIA gain setting SPI bus
-    :param cpld0_device: Name of the first Urukul CPLD SPI bus
+    :param cpld_devices: Names of the Urukul CPLD SPI buses
-    :param cpld1_device: Name of the second Urukul CPLD SPI bus
+    :param dds_devices: Names of the AD9910 devices
-    :param dds0_device: Name of the AD9910 device for the DDS on the first
-        Urukul
-    :param dds1_device: Name of the AD9910 device for the DDS on the second
-        Urukul
     :param gains: Initial value for PGIA gains shift register
         (default: 0x0000). Knowledge of this state is not transferred
         between experiments.
     :param core_device: Core device name
     """
-    kernel_invariants = {"channel", "core", "pgia", "cpld0", "cpld1",
+    kernel_invariants = {"channel", "core", "pgia", "cplds", "ddses",
-                         "dds0", "dds1", "ref_period_mu"}
+                         "ref_period_mu"}
 
     def __init__(self, dmgr, channel, pgia_device,
-                 cpld0_device, cpld1_device,
+                 cpld_devices, dds_devices,
-                 dds0_device, dds1_device,
                  gains=0x0000, core_device="core"):
 
         self.core = dmgr.get(core_device)
         self.pgia = dmgr.get(pgia_device)
         self.pgia.update_xfer_duration_mu(div=4, length=16)
-        self.dds0 = dmgr.get(dds0_device)
+        assert len(dds_devices) == len(cpld_devices)
-        self.dds1 = dmgr.get(dds1_device)
+        self.ddses = [dmgr.get(dds) for dds in dds_devices]
-        self.cpld0 = dmgr.get(cpld0_device)
+        self.cplds = [dmgr.get(cpld) for cpld in cpld_devices]
-        self.cpld1 = dmgr.get(cpld1_device)
         self.channel = channel
         self.gains = gains
         self.ref_period_mu = self.core.seconds_to_mu(
@@ -109,17 +103,15 @@ class SUServo:
             sampler.SPI_CONFIG | spi.SPI_END,
             16, 4, sampler.SPI_CS_PGIA)
 
-        self.cpld0.init(blind=True)
+        for i in range(len(self.cplds)):
-        cfg0 = self.cpld0.cfg_reg
+            cpld = self.cplds[i]
-        self.cpld0.cfg_write(cfg0 | (0xf << urukul.CFG_MASK_NU))
+            dds = self.ddses[i]
-        self.dds0.init(blind=True)
-        self.cpld0.cfg_write(cfg0)
 
-        self.cpld1.init(blind=True)
+            cpld.init(blind=True)
-        cfg1 = self.cpld1.cfg_reg
+            prev_cpld_cfg = cpld.cfg_reg
-        self.cpld1.cfg_write(cfg1 | (0xf << urukul.CFG_MASK_NU))
+            cpld.cfg_write(prev_cpld_cfg | (0xf << urukul.CFG_MASK_NU))
-        self.dds1.init(blind=True)
+            dds.init(blind=True)
-        self.cpld1.cfg_write(cfg1)
+            cpld.cfg_write(prev_cpld_cfg)
 
     @kernel
     def write(self, addr, value):
@@ -257,9 +249,11 @@ class Channel:
         self.servo = dmgr.get(servo_device)
         self.core = self.servo.core
         self.channel = channel
-        # FIXME: this assumes the mem channel is right after the control
+        # This assumes the mem channel is right after the control channels
-        # channels
+        # Make sure this is always the case in eem.py
-        self.servo_channel = self.channel + 8 - self.servo.channel
+        self.servo_channel = (self.channel + 4 * len(self.servo.cplds) -
+                              self.servo.channel)
+        self.dds = self.servo.ddses[self.servo_channel // 4]
 
     @kernel
     def set(self, en_out, en_iir=0, profile=0):
@@ -311,12 +305,8 @@ class Channel:
             see :meth:`dds_offset_to_mu`
         :param phase: DDS phase in turns
         """
-        if self.servo_channel < 4:
+        ftw = self.dds.frequency_to_ftw(frequency)
-            dds = self.servo.dds0
+        pow_ = self.dds.turns_to_pow(phase)
-        else:
-            dds = self.servo.dds1
-        ftw = dds.frequency_to_ftw(frequency)
-        pow_ = dds.turns_to_pow(phase)
         offs = self.dds_offset_to_mu(offset)
         self.set_dds_mu(profile, ftw, offs, pow_)
 

artiq/coredevice/zotino.py

@@ -27,15 +27,15 @@ class Zotino(AD53xx):
     :param clr_device: CLR RTIO TTLOut channel name.
     :param div_write: SPI clock divider for write operations (default: 4,
       50MHz max SPI clock)
-    :param div_read: SPI clock divider for read operations (default: 8, not
+    :param div_read: SPI clock divider for read operations (default: 16, not
-      optimized for speed, but cf data sheet t22: 25ns min SCLK edge to SDO
+      optimized for speed; datasheet says t22: 25ns min SCLK edge to SDO
-      valid)
+      valid, and suggests the SPI speed for reads should be <=20 MHz)
     :param vref: DAC reference voltage (default: 5.)
     :param core_device: Core device name (default: "core")
     """
 
     def __init__(self, dmgr, spi_device, ldac_device=None, clr_device=None,
-                 div_write=4, div_read=8, vref=5., core="core"):
+                 div_write=4, div_read=16, vref=5., core="core"):
         AD53xx.__init__(self, dmgr=dmgr, spi_device=spi_device,
                         ldac_device=ldac_device, clr_device=clr_device,
                         chip_select=_SPI_CS_DAC, div_write=div_write,

artiq/dashboard/datasets.py

@@ -3,8 +3,9 @@ import logging
 
 import numpy as np
 from PyQt5 import QtCore, QtWidgets
+from sipyco import pyon
 
-from artiq.tools import short_format
+from artiq.tools import short_format, exc_to_warning
 from artiq.gui.tools import LayoutWidget, QRecursiveFilterProxyModel
 from artiq.gui.models import DictSyncTreeSepModel
 from artiq.gui.scientific_spinbox import ScientificSpinBox
@@ -82,6 +83,68 @@ class StringEditor(Editor):
         return self.edit_widget.text()
 
 
+class Creator(QtWidgets.QDialog):
+    def __init__(self, parent, dataset_ctl):
+        QtWidgets.QDialog.__init__(self, parent=parent)
+        self.dataset_ctl = dataset_ctl
+
+        self.setWindowTitle("Create dataset")
+        grid = QtWidgets.QGridLayout()
+        grid.setRowMinimumHeight(1, 40)
+        grid.setColumnMinimumWidth(2, 60)
+        self.setLayout(grid)
+
+        grid.addWidget(QtWidgets.QLabel("Name:"), 0, 0)
+        self.name_widget = QtWidgets.QLineEdit()
+        grid.addWidget(self.name_widget, 0, 1)
+
+        grid.addWidget(QtWidgets.QLabel("Value:"), 1, 0)
+        self.value_widget = QtWidgets.QLineEdit()
+        self.value_widget.setPlaceholderText('PYON (Python)')
+        grid.addWidget(self.value_widget, 1, 1)
+        self.data_type = QtWidgets.QLabel("data type")
+        grid.addWidget(self.data_type, 1, 2)
+        self.value_widget.textChanged.connect(self.dtype)
+
+        grid.addWidget(QtWidgets.QLabel("Persist:"), 2, 0)
+        self.box_widget = QtWidgets.QCheckBox()
+        grid.addWidget(self.box_widget, 2, 1)
+
+        self.ok = QtWidgets.QPushButton('&Ok')
+        self.ok.setEnabled(False)
+        self.cancel = QtWidgets.QPushButton('&Cancel')
+        self.buttons = QtWidgets.QDialogButtonBox(self)
+        self.buttons.addButton(
+            self.ok, QtWidgets.QDialogButtonBox.AcceptRole)
+        self.buttons.addButton(
+            self.cancel, QtWidgets.QDialogButtonBox.RejectRole)
+        grid.setRowStretch(3, 1)
+        grid.addWidget(self.buttons, 4, 0, 1, 3)
+        self.buttons.accepted.connect(self.accept)
+        self.buttons.rejected.connect(self.reject)
+
+    def accept(self):
+        key = self.name_widget.text()
+        value = self.value_widget.text()
+        persist = self.box_widget.isChecked()
+        asyncio.ensure_future(exc_to_warning(self.dataset_ctl.set(
+            key, pyon.decode(value), persist)))
+        QtWidgets.QDialog.accept(self)
+
+    def dtype(self):
+        txt = self.value_widget.text()
+        try:
+            result = pyon.decode(txt)
+        except:
+            pixmap = self.style().standardPixmap(
+                QtWidgets.QStyle.SP_MessageBoxWarning)
+            self.data_type.setPixmap(pixmap)
+            self.ok.setEnabled(False)
+        else:
+            self.data_type.setText(type(result).__name__)
+            self.ok.setEnabled(True)
+
+
 class Model(DictSyncTreeSepModel):
     def __init__(self, init):
         DictSyncTreeSepModel.__init__(
@@ -120,6 +183,11 @@ class DatasetsDock(QtWidgets.QDockWidget):
         grid.addWidget(self.table, 1, 0)
 
         self.table.setContextMenuPolicy(QtCore.Qt.ActionsContextMenu)
+        create_action = QtWidgets.QAction("New dataset", self.table)
+        create_action.triggered.connect(self.create_clicked)
+        create_action.setShortcut("CTRL+N")
+        create_action.setShortcutContext(QtCore.Qt.WidgetShortcut)
+        self.table.addAction(create_action)
         edit_action = QtWidgets.QAction("Edit dataset", self.table)
         edit_action.triggered.connect(self.edit_clicked)
         edit_action.setShortcut("RETURN")
@@ -146,6 +214,9 @@ class DatasetsDock(QtWidgets.QDockWidget):
         self.table_model_filter.setSourceModel(self.table_model)
         self.table.setModel(self.table_model_filter)
 
+    def create_clicked(self):
+        Creator(self, self.dataset_ctl).open()
+
     def edit_clicked(self):
         idx = self.table.selectedIndexes()
         if idx:

artiq/examples/kasli_suservo/device_db.py

@@ -191,10 +191,8 @@ device_db = {
         "arguments": {
             "channel": 24,
             "pgia_device": "spi_sampler0_pgia",
-            "cpld0_device": "urukul0_cpld",
+            "cpld_devices": ["urukul0_cpld", "urukul1_cpld"],
-            "cpld1_device": "urukul1_cpld",
+            "dds_devices": ["urukul0_dds", "urukul1_dds"],
-            "dds0_device": "urukul0_dds",
-            "dds1_device": "urukul1_dds"
         }
     },
 

artiq/firmware/Cargo.lock

@@ -1,56 +1,84 @@
 # This file is automatically @generated by Cargo.
 # It is not intended for manual editing.
+[[package]]
+name = "aho-corasick"
+version = "0.7.18"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1e37cfd5e7657ada45f742d6e99ca5788580b5c529dc78faf11ece6dc702656f"
+dependencies = [
+ "memchr",
+]
+
 [[package]]
 name = "alloc_list"
 version = "0.0.0"
 
 [[package]]
-name = "bitflags"
+name = "bare-metal"
-version = "1.0.3"
+version = "0.2.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "5deb64efa5bd81e31fcd1938615a6d98c82eafcbcd787162b6f63b91d6bac5b3"
+dependencies = [
+ "rustc_version",
+]
+
+[[package]]
+name = "bit_field"
+version = "0.10.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dcb6dd1c2376d2e096796e234a70e17e94cc2d5d54ff8ce42b28cef1d0d359a4"
+
+[[package]]
+name = "bitflags"
+version = "1.3.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a"
 
 [[package]]
 name = "board_artiq"
 version = "0.0.0"
 dependencies = [
- "bitflags 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "bitflags",
- "board_misoc 0.0.0",
+ "board_misoc",
- "build_misoc 0.0.0",
+ "build_misoc",
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "crc 1.8.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "crc",
- "failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure",
- "failure_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure_derive",
- "io 0.0.0",
+ "io",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
- "proto_artiq 0.0.0",
+ "proto_artiq",
 ]
 
 [[package]]
 name = "board_misoc"
 version = "0.0.0"
 dependencies = [
- "build_misoc 0.0.0",
+ "build_misoc",
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "cc 1.0.15 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cc",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
- "smoltcp 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "riscv",
+ "smoltcp",
 ]
 
 [[package]]
 name = "bootloader"
 version = "0.0.0"
 dependencies = [
- "board_misoc 0.0.0",
+ "board_misoc",
- "build_misoc 0.0.0",
+ "build_misoc",
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "crc 1.8.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "crc",
- "smoltcp 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "riscv",
+ "smoltcp",
 ]
 
 [[package]]
 name = "build_const"
-version = "0.2.1"
+version = "0.2.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b4ae4235e6dac0694637c763029ecea1a2ec9e4e06ec2729bd21ba4d9c863eb7"
 
 [[package]]
 name = "build_misoc"
@@ -58,31 +86,48 @@ version = "0.0.0"
 
 [[package]]
 name = "byteorder"
-version = "1.2.3"
+version = "1.4.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
 
 [[package]]
 name = "cc"
-version = "1.0.15"
+version = "1.0.70"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d26a6ce4b6a484fa3edb70f7efa6fc430fd2b87285fe8b84304fd0936faa0dc0"
 
 [[package]]
 name = "cfg-if"
-version = "0.1.3"
+version = "0.1.10"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
+
+[[package]]
+name = "cfg-if"
+version = "1.0.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
+
+[[package]]
+name = "compiler_builtins"
+version = "0.1.39"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3748f82c7d366a0b4950257d19db685d4958d2fa27c6d164a3f069fec42b748b"
 
 [[package]]
 name = "crc"
 version = "1.8.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d663548de7f5cca343f1e0a48d14dcfb0e9eb4e079ec58883b7251539fa10aeb"
 dependencies = [
- "build_const 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "build_const",
 ]
 
 [[package]]
 name = "cslice"
 version = "0.3.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0f8cb7306107e4b10e64994de6d3274bd08996a7c1322a27b86482392f96be0a"
 
 [[package]]
 name = "dyld"
@@ -92,202 +137,290 @@ version = "0.0.0"
 name = "eh"
 version = "0.0.0"
 dependencies = [
- "cslice 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "compiler_builtins",
+ "cslice",
+ "libc 0.1.0",
+ "unwind",
 ]
 
 [[package]]
 name = "failure"
 version = "0.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "934799b6c1de475a012a02dab0ace1ace43789ee4b99bcfbf1a2e3e8ced5de82"
 
 [[package]]
 name = "failure_derive"
 version = "0.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c7cdda555bb90c9bb67a3b670a0f42de8e73f5981524123ad8578aafec8ddb8b"
 dependencies = [
- "quote 0.3.15 (registry+https://github.com/rust-lang/crates.io-index)",
+ "quote",
- "syn 0.11.11 (registry+https://github.com/rust-lang/crates.io-index)",
+ "syn",
- "synstructure 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "synstructure",
 ]
 
 [[package]]
 name = "fringe"
-version = "1.1.0"
+version = "1.2.1"
-source = "git+https://github.com/m-labs/libfringe?rev=b8a6d8f#b8a6d8f68df0edaa3d67d9f3b7b62af9d3bb64a5"
+source = "git+https://git.m-labs.hk/M-Labs/libfringe.git?rev=3ecbe5#3ecbe53f7644b18ee46ebd5b2ca12c9cbceec43a"
 dependencies = [
- "libc 0.2.40 (registry+https://github.com/rust-lang/crates.io-index)",
+ "libc 0.2.101",
 ]
 
 [[package]]
 name = "io"
 version = "0.0.0"
 dependencies = [
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure",
- "failure_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure_derive",
 ]
 
 [[package]]
 name = "ksupport"
 version = "0.0.0"
 dependencies = [
- "board_artiq 0.0.0",
+ "board_artiq",
- "board_misoc 0.0.0",
+ "board_misoc",
- "build_misoc 0.0.0",
+ "build_misoc",
- "cslice 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cslice",
- "dyld 0.0.0",
+ "dyld",
- "eh 0.0.0",
+ "eh",
- "io 0.0.0",
+ "io",
- "proto_artiq 0.0.0",
+ "libc 0.1.0",
+ "proto_artiq",
+ "riscv",
+ "unwind",
 ]
 
 [[package]]
-name = "libc"
+name = "lazy_static"
-version = "0.2.40"
+version = "1.4.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
+
+[[package]]
+name = "libc"
+version = "0.1.0"
+
+[[package]]
+name = "libc"
+version = "0.2.101"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3cb00336871be5ed2c8ed44b60ae9959dc5b9f08539422ed43f09e34ecaeba21"
 
 [[package]]
 name = "log"
-version = "0.4.1"
+version = "0.4.14"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "51b9bbe6c47d51fc3e1a9b945965946b4c44142ab8792c50835a980d362c2710"
 dependencies = [
- "cfg-if 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cfg-if 1.0.0",
 ]
 
 [[package]]
 name = "log_buffer"
 version = "1.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f033173c9486b7fe97a79c895c0a3483ae395ab6744c985d10078950e2492419"
 
 [[package]]
 name = "logger_artiq"
 version = "0.0.0"
 dependencies = [
- "board_misoc 0.0.0",
+ "board_misoc",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
- "log_buffer 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log_buffer",
 ]
 
 [[package]]
 name = "managed"
-version = "0.7.0"
+version = "0.7.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c75de51135344a4f8ed3cfe2720dc27736f7711989703a0b43aadf3753c55577"
+
+[[package]]
+name = "memchr"
+version = "2.4.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "308cc39be01b73d0d18f82a0e7b2a3df85245f84af96fdddc5d202d27e47b86a"
 
 [[package]]
 name = "proto_artiq"
 version = "0.0.0"
 dependencies = [
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "cslice 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cslice",
- "dyld 0.0.0",
+ "dyld",
- "failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure",
- "failure_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure_derive",
- "io 0.0.0",
+ "io",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
 ]
 
 [[package]]
 name = "quote"
 version = "0.3.15"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "7a6e920b65c65f10b2ae65c831a81a073a89edd28c7cce89475bff467ab4167a"
+
+[[package]]
+name = "regex"
+version = "1.5.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d07a8629359eb56f1e2fb1652bb04212c072a87ba68546a04065d525673ac461"
+dependencies = [
+ "aho-corasick",
+ "memchr",
+ "regex-syntax",
+]
+
+[[package]]
+name = "regex-syntax"
+version = "0.6.25"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f497285884f3fcff424ffc933e56d7cbca511def0c9831a7f9b5f6153e3cc89b"
+
+[[package]]
+name = "riscv"
+version = "0.6.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a2f0b705d428e9d0f78e2bb73093887ee58a83c9688de3faedbb4c0631c4618e"
+dependencies = [
+ "bare-metal",
+ "bit_field",
+ "riscv-target",
+]
+
+[[package]]
+name = "riscv-target"
+version = "0.1.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "88aa938cda42a0cf62a20cfe8d139ff1af20c2e681212b5b34adb5a58333f222"
+dependencies = [
+ "lazy_static",
+ "regex",
+]
 
 [[package]]
 name = "runtime"
 version = "0.0.0"
 dependencies = [
- "alloc_list 0.0.0",
+ "alloc_list",
- "board_artiq 0.0.0",
+ "board_artiq",
- "board_misoc 0.0.0",
+ "board_misoc",
- "build_misoc 0.0.0",
+ "build_misoc",
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "cslice 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "cslice",
- "eh 0.0.0",
+ "eh",
- "failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure",
- "failure_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "failure_derive",
- "fringe 1.1.0 (git+https://github.com/m-labs/libfringe?rev=b8a6d8f)",
+ "fringe",
- "io 0.0.0",
+ "io",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
- "logger_artiq 0.0.0",
+ "logger_artiq",
- "managed 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "managed",
- "proto_artiq 0.0.0",
+ "proto_artiq",
- "smoltcp 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "riscv",
- "unwind_backtrace 0.0.0",
+ "smoltcp",
+ "unwind_backtrace",
+]
+
+[[package]]
+name = "rustc_version"
+version = "0.2.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a"
+dependencies = [
+ "semver",
 ]
 
 [[package]]
 name = "satman"
 version = "0.0.0"
 dependencies = [
- "board_artiq 0.0.0",
+ "board_artiq",
- "board_misoc 0.0.0",
+ "board_misoc",
- "build_misoc 0.0.0",
+ "build_misoc",
- "log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)",
+ "log",
+ "riscv",
 ]
 
+[[package]]
+name = "semver"
+version = "0.9.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1d7eb9ef2c18661902cc47e535f9bc51b78acd254da71d375c2f6720d9a40403"
+dependencies = [
+ "semver-parser",
+]
+
+[[package]]
+name = "semver-parser"
+version = "0.7.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3"
+
 [[package]]
 name = "smoltcp"
 version = "0.6.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fe46639fd2ec79eadf8fe719f237a7a0bd4dac5d957f1ca5bbdbc1c3c39e53a"
 dependencies = [
- "bitflags 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "bitflags",
- "byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "byteorder",
- "managed 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
+ "managed",
 ]
 
 [[package]]
 name = "syn"
 version = "0.11.11"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d3b891b9015c88c576343b9b3e41c2c11a51c219ef067b264bd9c8aa9b441dad"
 dependencies = [
- "quote 0.3.15 (registry+https://github.com/rust-lang/crates.io-index)",
+ "quote",
- "synom 0.11.3 (registry+https://github.com/rust-lang/crates.io-index)",
+ "synom",
- "unicode-xid 0.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
+ "unicode-xid",
 ]
 
 [[package]]
 name = "synom"
 version = "0.11.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a393066ed9010ebaed60b9eafa373d4b1baac186dd7e008555b0f702b51945b6"
 dependencies = [
- "unicode-xid 0.0.4 (registry+https://github.com/rust-lang/crates.io-index)",
+ "unicode-xid",
 ]
 
 [[package]]
 name = "synstructure"
 version = "0.6.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3a761d12e6d8dcb4dcf952a7a89b475e3a9d69e4a69307e01a470977642914bd"
 dependencies = [
- "quote 0.3.15 (registry+https://github.com/rust-lang/crates.io-index)",
+ "quote",
- "syn 0.11.11 (registry+https://github.com/rust-lang/crates.io-index)",
+ "syn",
 ]
 
 [[package]]
 name = "unicode-xid"
 version = "0.0.4"
 source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8c1f860d7d29cf02cb2f3f359fd35991af3d30bac52c57d265a3c461074cb4dc"
+
+[[package]]
+name = "unwind"
+version = "0.0.0"
+dependencies = [
+ "cfg-if 0.1.10",
+ "libc 0.1.0",
+]
 
 [[package]]
 name = "unwind_backtrace"
 version = "0.0.0"
-
+dependencies = [
-[metadata]
+ "libc 0.1.0",
-"checksum bitflags 1.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "d0c54bb8f454c567f21197eefcdbf5679d0bd99f2ddbe52e84c77061952e6789"
+ "unwind",
-"checksum build_const 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "39092a32794787acd8525ee150305ff051b0aa6cc2abaf193924f5ab05425f39"
+]
-"checksum byteorder 1.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "74c0b906e9446b0a2e4f760cdb3fa4b2c48cdc6db8766a845c54b6ff063fd2e9"
-"checksum cc 1.0.15 (registry+https://github.com/rust-lang/crates.io-index)" = "0ebb87d1116151416c0cf66a0e3fb6430cccd120fd6300794b4dfaa050ac40ba"
-"checksum cfg-if 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "405216fd8fe65f718daa7102ea808a946b6ce40c742998fbfd3463645552de18"
-"checksum crc 1.8.1 (registry+https://github.com/rust-lang/crates.io-index)" = "d663548de7f5cca343f1e0a48d14dcfb0e9eb4e079ec58883b7251539fa10aeb"
-"checksum cslice 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "0f8cb7306107e4b10e64994de6d3274bd08996a7c1322a27b86482392f96be0a"
-"checksum failure 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "934799b6c1de475a012a02dab0ace1ace43789ee4b99bcfbf1a2e3e8ced5de82"
-"checksum failure_derive 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "c7cdda555bb90c9bb67a3b670a0f42de8e73f5981524123ad8578aafec8ddb8b"
-"checksum fringe 1.1.0 (git+https://github.com/m-labs/libfringe?rev=b8a6d8f)" = "<none>"
-"checksum libc 0.2.40 (registry+https://github.com/rust-lang/crates.io-index)" = "6fd41f331ac7c5b8ac259b8bf82c75c0fb2e469bbf37d2becbba9a6a2221965b"
-"checksum log 0.4.1 (registry+https://github.com/rust-lang/crates.io-index)" = "89f010e843f2b1a31dbd316b3b8d443758bc634bed37aabade59c686d644e0a2"
-"checksum log_buffer 1.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "f033173c9486b7fe97a79c895c0a3483ae395ab6744c985d10078950e2492419"
-"checksum managed 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ba6713e624266d7600e9feae51b1926c6a6a6bebb18ec5a8e11a5f1d5661baba"
-"checksum quote 0.3.15 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6e920b65c65f10b2ae65c831a81a073a89edd28c7cce89475bff467ab4167a"
-"checksum smoltcp 0.6.0 (registry+https://github.com/rust-lang/crates.io-index)" = "0fe46639fd2ec79eadf8fe719f237a7a0bd4dac5d957f1ca5bbdbc1c3c39e53a"
-"checksum syn 0.11.11 (registry+https://github.com/rust-lang/crates.io-index)" = "d3b891b9015c88c576343b9b3e41c2c11a51c219ef067b264bd9c8aa9b441dad"
-"checksum synom 0.11.3 (registry+https://github.com/rust-lang/crates.io-index)" = "a393066ed9010ebaed60b9eafa373d4b1baac186dd7e008555b0f702b51945b6"
-"checksum synstructure 0.6.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3a761d12e6d8dcb4dcf952a7a89b475e3a9d69e4a69307e01a470977642914bd"
-"checksum unicode-xid 0.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "8c1f860d7d29cf02cb2f3f359fd35991af3d30bac52c57d265a3c461074cb4dc"

artiq/firmware/bootloader/Cargo.toml

@@ -17,3 +17,4 @@ byteorder = { version = "1.0", default-features = false }
 crc = { version = "1.7", default-features = false }
 board_misoc = { path = "../libboard_misoc", features = ["uart_console", "smoltcp"] }
 smoltcp = { version = "0.6.0", default-features = false, features = ["ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"] }
+riscv = { version = "0.6.0", features = ["inline-asm"] }

artiq/firmware/bootloader/Makefile

@@ -3,15 +3,19 @@ include $(MISOC_DIRECTORY)/software/common.mak
 
 RUSTFLAGS += -Cpanic=abort
 
+export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
+
 all:: bootloader.bin
 
 .PHONY: $(RUSTOUT)/libbootloader.a
 $(RUSTOUT)/libbootloader.a:
-	$(cargo) --manifest-path $(BOOTLOADER_DIRECTORY)/Cargo.toml
+	$(cargo) --target-dir ./cargo \
+	--manifest-path $(BOOTLOADER_DIRECTORY)/Cargo.toml \
+	--target $(BOOTLOADER_DIRECTORY)/../$(CARGO_TRIPLE).json
 
 bootloader.elf: $(RUSTOUT)/libbootloader.a
 	$(link) -T $(BOOTLOADER_DIRECTORY)/bootloader.ld
 
 %.bin: %.elf
 	$(objcopy) -O binary
-	$(MSCIMG) $@
+	$(MSCIMG) $@ --little

artiq/firmware/bootloader/bootloader.ld

@@ -15,12 +15,10 @@ SECTIONS
     *(.text .text.*)
   } > rom
 
-  /*
+  .eh_frame :
-   * The compiler_builtins crate includes some GOTPC relocations, which require a GOT symbol,
+  {
-   * but don't actually need a GOT. This really ought to be fixed on rustc level, but I'm afraid
+    *(.eh_frame.*)
-   * it will add further complications to our build system that aren't pulling their weight.
+  } > rom
-   */
-  _GLOBAL_OFFSET_TABLE_ = .;
 
   .rodata :
   {
@@ -29,25 +27,25 @@ SECTIONS
     _end = .;
   } > rom
 
-  .crc ALIGN(4) :
+  .crc (NOLOAD) : ALIGN(4)
   {
     _crc = .;
     . += 4;
-  }
+  } > rom
 
-  .bss :
+  .bss (NOLOAD) :
   {
     _fbss = .;
-    *(.bss .bss.*)
+    *(.sbss .sbss.* .bss .bss.*);
     . = ALIGN(4);
     _ebss = .;
   } > sram
 
-  .stack :
+  .stack (NOLOAD) : ALIGN(16)
   {
     /* Ensure we have a certain amount of space available for stack. */
     /*. = ORIGIN(sram) + LENGTH(sram) - 0x1a00; */
-    . = ORIGIN(sram) + LENGTH(sram) - 4;
+    . = ORIGIN(sram) + LENGTH(sram) - 16;
     _fstack = .;
   } > sram
 }

artiq/firmware/bootloader/main.rs

@@ -1,21 +1,23 @@
 #![no_std]
-#![feature(panic_implementation, panic_info_message)]
+#![feature(panic_info_message)]
 
 extern crate crc;
 extern crate byteorder;
 extern crate smoltcp;
 #[macro_use]
 extern crate board_misoc;
+extern crate riscv;
 
-use core::{ptr, slice};
+use core::{ptr, slice, convert::TryFrom};
 use crc::crc32;
-use byteorder::{ByteOrder, BigEndian};
+use byteorder::{ByteOrder, LittleEndian};
 use board_misoc::{ident, cache, sdram, config, boot, mem as board_mem};
 #[cfg(has_slave_fpga_cfg)]
 use board_misoc::slave_fpga;
 #[cfg(has_ethmac)]
 use board_misoc::{clock, ethmac, net_settings};
 use board_misoc::uart_console::Console;
+use riscv::register::{mcause, mepc, mtval};
 
 fn check_integrity() -> bool {
     extern {
@@ -45,7 +47,7 @@ fn memory_test(total: &mut usize, wrong: &mut usize) -> bool {
                 MEMORY[$index:expr] = $data:expr
             }
         ) => ({
-            $prepare;
+            $prepare
             for $i in $range {
                 unsafe { ptr::write_volatile(MEMORY.offset($index as isize), $data) };
                 *total += 1;
@@ -54,7 +56,7 @@ fn memory_test(total: &mut usize, wrong: &mut usize) -> bool {
             cache::flush_cpu_dcache();
             cache::flush_l2_cache();
 
-            $prepare;
+            $prepare
             for $i in $range {
                 if unsafe { ptr::read_volatile(MEMORY.offset($index as isize)) } != $data {
                     *wrong += 1;
@@ -119,8 +121,8 @@ fn load_slave_fpga() {
     const GATEWARE: *mut u8 = board_misoc::csr::CONFIG_SLAVE_FPGA_GATEWARE as *mut u8;
 
     let header = unsafe { slice::from_raw_parts(GATEWARE, 8) };
-    let magic = BigEndian::read_u32(&header[0..]);
+    let magic = LittleEndian::read_u32(&header[0..]);
-    let length = BigEndian::read_u32(&header[4..]) as usize;
+    let length = LittleEndian::read_u32(&header[4..]) as usize;
     println!("  magic: 0x{:08x}, length: 0x{:08x}", magic, length);
     if magic != 0x5352544d {
         println!("  ...Error: bad magic");
@@ -155,8 +157,8 @@ fn flash_boot() {
     println!("Booting from flash...");
 
     let header = unsafe { slice::from_raw_parts(FIRMWARE, 8) };
-    let length = BigEndian::read_u32(&header[0..]) as usize;
+    let length = LittleEndian::read_u32(&header[0..]) as usize;
-    let expected_crc = BigEndian::read_u32(&header[4..]);
+    let expected_crc = LittleEndian::read_u32(&header[4..]);
 
     if length == 0 || length == 0xffffffff {
         println!("No firmware present");
@@ -517,8 +519,11 @@ pub extern fn main() -> i32 {
 }
 
 #[no_mangle]
-pub extern fn exception(vect: u32, _regs: *const u32, pc: u32, ea: u32) {
+pub extern fn exception(_regs: *const u32) {
-    panic!("exception {} at PC {:#08x}, EA {:#08x}", vect, pc, ea)
+    let pc = mepc::read();
+    let cause = mcause::read().cause();
+    let mtval = mtval::read();
+    panic!("{:?} at PC {:#08x}, trap value {:#08x}", cause, u32::try_from(pc).unwrap(), mtval);
 }
 
 #[no_mangle]
@@ -528,7 +533,7 @@ pub extern fn abort() {
 }
 
 #[no_mangle] // https://github.com/rust-lang/rust/issues/{38281,51647}
-#[panic_implementation]
+#[panic_handler]
 pub fn panic_fmt(info: &core::panic::PanicInfo) -> ! {
     #[cfg(has_error_led)]
     unsafe {

artiq/firmware/cargosha256.nix

@@ -1 +0,0 @@
-"0ml6j4sxqrayqk25xkrikwg713mahfqa60nrx1jhrj8c2h3p07yk"

artiq/firmware/ksupport/Cargo.toml

@@ -20,3 +20,6 @@ dyld = { path = "../libdyld" }
 board_misoc = { path = "../libboard_misoc" }
 board_artiq = { path = "../libboard_artiq" }
 proto_artiq = { path = "../libproto_artiq" }
+riscv = { version = "0.6.0", features = ["inline-asm"] }
+libc = { path = "../libc" }
+unwind = { path = "../libunwind" }

artiq/firmware/ksupport/Makefile

@@ -7,21 +7,26 @@ CFLAGS += \
 	-I$(MISOC_DIRECTORY)/software/include/dyld
 
 LDFLAGS += --eh-frame-hdr \
+	--nmagic \
 	-L../libm \
 	-L../libprintf \
 	-L../libunwind
 
 RUSTFLAGS += -Cpanic=unwind
 
+export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
+
 all:: ksupport.elf
 
 .PHONY: $(RUSTOUT)/libksupport.a
 $(RUSTOUT)/libksupport.a:
-	$(cargo) --manifest-path $(KSUPPORT_DIRECTORY)/Cargo.toml
+	$(cargo) --target-dir ./cargo \
+	--manifest-path $(KSUPPORT_DIRECTORY)/Cargo.toml \
+	--target $(KSUPPORT_DIRECTORY)/../$(CARGO_TRIPLE).json
 
 ksupport.elf: $(RUSTOUT)/libksupport.a glue.o
 	$(link) -T $(KSUPPORT_DIRECTORY)/ksupport.ld \
-		-lunwind-elf -lprintf-float -lm
+		-lunwind-$(CPU)-elf -lprintf-float -lm
 
 %.o: $(KSUPPORT_DIRECTORY)/%.c
 	$(compile)

artiq/firmware/ksupport/api.rs

@@ -131,9 +131,6 @@ static mut API: &'static [(&'static str, *const ())] = &[
     api!(cache_get = ::cache_get),
     api!(cache_put = ::cache_put),
 
-    api!(mfspr = ::board_misoc::spr::mfspr),
-    api!(mtspr = ::board_misoc::spr::mtspr),
-
     /* direct syscalls */
     api!(rtio_init = ::rtio::init),
     api!(rtio_get_destination_status = ::rtio::get_destination_status),

artiq/firmware/ksupport/eh_artiq.rs

@@ -8,14 +8,14 @@
 // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
-#![allow(private_no_mangle_fns, non_camel_case_types)]
+#![allow(non_camel_case_types)]
 
 use core::{ptr, mem};
 use cslice::CSlice;
 use unwind as uw;
 use libc::{c_int, c_void};
 
-use eh::dwarf::{self, EHAction};
+use eh::dwarf::{self, EHAction, EHContext};
 
 type _Unwind_Stop_Fn = extern "C" fn(version: c_int,
                                      actions: uw::_Unwind_Action,
@@ -58,8 +58,8 @@ struct ExceptionInfo {
 #[cfg(target_arch = "x86_64")]
 const UNWIND_DATA_REG: (i32, i32) = (0, 1); // RAX, RDX
 
-#[cfg(any(target_arch = "or1k"))]
+#[cfg(any(target_arch = "riscv32"))]
-const UNWIND_DATA_REG: (i32, i32) = (3, 4); // R3, R4
+const UNWIND_DATA_REG: (i32, i32) = (10, 11); // X10, X11
 
 #[export_name="__artiq_personality"]
 pub extern fn personality(version: c_int,
@@ -74,13 +74,25 @@ pub extern fn personality(version: c_int,
         }
 
         let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8;
-        let ip = uw::_Unwind_GetIP(context) - 1;
+        let mut ip_before_instr: c_int = 0;
-        let func_start = uw::_Unwind_GetRegionStart(context);
+        let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr);
+        let eh_context = EHContext {
+            // The return address points 1 byte past the call instruction,
+            // which could be in the next IP range in LSDA range table.
+            ip: if ip_before_instr != 0 { ip } else { ip - 1 },
+            func_start: uw::_Unwind_GetRegionStart(context),
+            get_text_start: &|| uw::_Unwind_GetTextRelBase(context),
+            get_data_start: &|| uw::_Unwind_GetDataRelBase(context),
+        };
 
         let exception_info = &mut *(uw_exception as *mut ExceptionInfo);
         let exception = &exception_info.exception.unwrap();
 
-        let eh_action = dwarf::find_eh_action(lsda, func_start, ip, exception.name);
+        let eh_action = match dwarf::find_eh_action(lsda, &eh_context) {
+            Ok(action) => action,
+            Err(_) => return uw::_URC_FATAL_PHASE1_ERROR,
+        };
+
         if actions as u32 & uw::_UA_SEARCH_PHASE as u32 != 0 {
             match eh_action {
                 EHAction::None |

artiq/firmware/ksupport/ksupport.ld

@@ -15,7 +15,7 @@ MEMORY {
 }
 
 /* Kernel stack is at the end of main RAM. */
-_fstack = ORIGIN(main_ram) + LENGTH(main_ram) - 4;
+_fstack = ORIGIN(main_ram) + LENGTH(main_ram) - 16;
 
 /* Force ld to make the ELF header as loadable. */
 PHDRS
@@ -53,22 +53,22 @@ SECTIONS
     .eh_frame :
     {
         KEEP(*(.eh_frame))
-    } :text
+    } > ksupport :text
 
     .eh_frame_hdr :
     {
         KEEP(*(.eh_frame_hdr))
-    } :text :eh_frame
+    } > ksupport :text :eh_frame
 
     .data :
     {
         *(.data .data.*)
     }
 
-    .bss :
+    .bss (NOLOAD) : ALIGN(4)
     {
         _fbss = .;
-        *(.bss .bss.*)
+        *(.sbss .sbss.* .bss .bss.*);
         . = ALIGN(4);
         _ebss = .;
     }

artiq/firmware/ksupport/lib.rs

@@ -1,5 +1,5 @@
-#![feature(lang_items, asm, panic_unwind, libc, unwind_attributes,
+#![feature(lang_items, llvm_asm, panic_unwind, libc, unwind_attributes,
-           panic_implementation, panic_info_message, nll)]
+           panic_info_message, nll)]
 #![no_std]
 
 extern crate libc;
@@ -12,8 +12,9 @@ extern crate dyld;
 extern crate board_misoc;
 extern crate board_artiq;
 extern crate proto_artiq;
+extern crate riscv;
 
-use core::{mem, ptr, slice, str};
+use core::{mem, ptr, slice, str, convert::TryFrom};
 use cslice::{CSlice, AsCSlice};
 use io::Cursor;
 use dyld::Library;
@@ -22,6 +23,7 @@ use proto_artiq::{kernel_proto, rpc_proto};
 use kernel_proto::*;
 #[cfg(has_rtio_dma)]
 use board_misoc::csr;
+use riscv::register::{mcause, mepc, mtval};
 
 fn send(request: &Message) {
     unsafe { mailbox::send(request as *const _ as usize) }
@@ -49,7 +51,7 @@ macro_rules! recv {
 }
 
 #[no_mangle] // https://github.com/rust-lang/rust/issues/{38281,51647}
-#[panic_implementation]
+#[panic_handler]
 pub fn panic_fmt(info: &core::panic::PanicInfo) -> ! {
     if let Some(location) = info.location() {
         send(&Log(format_args!("panic at {}:{}:{}",
@@ -120,7 +122,7 @@ pub extern fn send_to_rtio_log(text: CSlice<u8>) {
 }
 
 #[unwind(aborts)]
-extern fn rpc_send(service: u32, tag: CSlice<u8>, data: *const *const ()) {
+extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while !rpc_queue::empty() {}
     send(&RpcSend {
         async:   false,
@@ -131,7 +133,7 @@ extern fn rpc_send(service: u32, tag: CSlice<u8>, data: *const *const ()) {
 }
 
 #[unwind(aborts)]
-extern fn rpc_send_async(service: u32, tag: CSlice<u8>, data: *const *const ()) {
+extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
     while rpc_queue::full() {}
     rpc_queue::enqueue(|mut slice| {
         let length = {
@@ -201,15 +203,18 @@ fn terminate(exception: &eh_artiq::Exception, backtrace: &mut [usize]) -> ! {
 }
 
 #[unwind(aborts)]
-extern fn cache_get(key: CSlice<u8>) -> CSlice<'static, i32> {
+extern fn cache_get<'a>(ret: &'a mut CSlice<i32>, key: &CSlice<u8>) -> &'a CSlice<'a, i32> {
     send(&CacheGetRequest {
         key:   str::from_utf8(key.as_ref()).unwrap()
     });
-    recv!(&CacheGetReply { value } => value.as_c_slice())
+    recv!(&CacheGetReply { value } => {
+        *ret = value.as_c_slice();
+        ret
+    })
 }
 
 #[unwind(allowed)]
-extern fn cache_put(key: CSlice<u8>, list: CSlice<i32>) {
+extern fn cache_put(key: &CSlice<u8>, list: &CSlice<i32>) {
     send(&CachePutRequest {
         key:   str::from_utf8(key.as_ref()).unwrap(),
         value: list.as_ref()
@@ -243,7 +248,7 @@ fn dma_record_flush() {
 }
 
 #[unwind(allowed)]
-extern fn dma_record_start(name: CSlice<u8>) {
+extern fn dma_record_start(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     unsafe {
@@ -324,7 +329,7 @@ unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
 #[unwind(aborts)]
 extern fn dma_record_output(target: i32, word: i32) {
     unsafe {
-        let timestamp = *(csr::rtio::NOW_HI_ADDR as *const i64);
+        let timestamp = ((csr::rtio::now_hi_read() as i64) << 32) | (csr::rtio::now_lo_read() as i64);
         let data = dma_record_output_prepare(timestamp, target, 1);
         data.copy_from_slice(&[
             (word >>  0) as u8,
@@ -340,7 +345,7 @@ extern fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
     assert!(words.len() <= 16); // enforce the hardware limit
 
     unsafe {
-        let timestamp = *(csr::rtio::NOW_HI_ADDR as *const i64);
+        let timestamp = ((csr::rtio::now_hi_read() as i64) << 32) | (csr::rtio::now_lo_read() as i64);
         let mut data = dma_record_output_prepare(timestamp, target, words.len());
         for word in words.as_ref().iter() {
             data[..4].copy_from_slice(&[
@@ -355,7 +360,7 @@ extern fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
 }
 
 #[unwind(aborts)]
-extern fn dma_erase(name: CSlice<u8>) {
+extern fn dma_erase(name: &CSlice<u8>) {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     send(&DmaEraseRequest { name: name });
@@ -368,7 +373,7 @@ struct DmaTrace {
 }
 
 #[unwind(allowed)]
-extern fn dma_retrieve(name: CSlice<u8>) -> DmaTrace {
+extern fn dma_retrieve(name: &CSlice<u8>) -> DmaTrace {
     let name = str::from_utf8(name.as_ref()).unwrap();
 
     send(&DmaRetrieveRequest { name: name });
@@ -454,7 +459,7 @@ unsafe fn attribute_writeback(typeinfo: *const ()) {
                 attributes = attributes.offset(1);
 
                 if (*attribute).tag.len() > 0 {
-                    rpc_send_async(0, (*attribute).tag, [
+                    rpc_send_async(0, &(*attribute).tag, [
                         &object as *const _ as *const (),
                         &(*attribute).name as *const _ as *const (),
                         (object as usize + (*attribute).offset) as *const ()
@@ -488,11 +493,16 @@ pub unsafe fn main() {
     let _end = library.lookup(b"_end").unwrap();
     let __modinit__ = library.lookup(b"__modinit__").unwrap();
     let typeinfo = library.lookup(b"typeinfo");
+    let _sstack_guard = library.lookup(b"_sstack_guard").unwrap();
 
     LIBRARY = Some(library);
 
     ptr::write_bytes(__bss_start as *mut u8, 0, (_end - __bss_start) as usize);
 
+    board_misoc::pmp::init_stack_guard(_sstack_guard as usize);
+    board_misoc::cache::flush_cpu_dcache();
+    board_misoc::cache::flush_cpu_icache();
+
     (mem::transmute::<u32, fn()>(__modinit__))();
 
     if let Some(typeinfo) = typeinfo {
@@ -519,8 +529,11 @@ pub unsafe fn main() {
 
 #[no_mangle]
 #[unwind(allowed)]
-pub extern fn exception(vect: u32, _regs: *const u32, pc: u32, ea: u32) {
+pub extern fn exception(_regs: *const u32) {
-    panic!("exception {:?} at PC 0x{:x}, EA 0x{:x}", vect, pc, ea)
+    let pc = mepc::read();
+    let cause = mcause::read().cause();
+    let mtval = mtval::read();
+    panic!("{:?} at PC {:#08x}, trap value {:#08x}", cause, u32::try_from(pc).unwrap(), mtval);
 }
 
 #[no_mangle]

artiq/firmware/ksupport/rtio.rs

@@ -23,6 +23,8 @@ mod imp {
     pub const RTIO_I_STATUS_WAIT_STATUS:               u8 = 4;
     pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE:   u8 = 8;
 
+    const OFFSET_MULTIPLE: isize = (csr::CONFIG_DATA_WIDTH_BYTES / 4) as isize;
+
     pub extern fn init() {
         send(&RtioInitRequest);
     }
@@ -47,19 +49,19 @@ mod imp {
     #[inline(always)]
     pub unsafe fn rtio_o_data_write(offset: usize, data: u32) {
         write_volatile(
-            csr::rtio::O_DATA_ADDR.offset((csr::rtio::O_DATA_SIZE - 1 - offset) as isize),
+            csr::rtio::O_DATA_ADDR.offset(OFFSET_MULTIPLE*(csr::rtio::O_DATA_SIZE - 1 - offset) as isize),
             data);
     }
 
     #[inline(always)]
     pub unsafe fn rtio_i_data_read(offset: usize) -> u32 {
         read_volatile(
-            csr::rtio::I_DATA_ADDR.offset((csr::rtio::I_DATA_SIZE - 1 - offset) as isize))
+            csr::rtio::I_DATA_ADDR.offset(OFFSET_MULTIPLE*(csr::rtio::I_DATA_SIZE - 1 - offset) as isize))
     }
 
     #[inline(never)]
     unsafe fn process_exceptional_status(channel: i32, status: u8) {
-        let timestamp = *(csr::rtio::NOW_HI_ADDR as *const i64);
+        let timestamp = ((csr::rtio::now_hi_read() as i64) << 32) | (csr::rtio::now_lo_read() as i64);
         if status & RTIO_O_STATUS_WAIT != 0 {
             while csr::rtio::o_status_read() & RTIO_O_STATUS_WAIT != 0 {}
         }

artiq/firmware/liballoc_list/lib.rs

@@ -3,9 +3,6 @@
 use core::{ptr, mem, fmt};
 use core::alloc::{GlobalAlloc, Layout};
 
-// The minimum alignment guaranteed by the architecture.
-const MIN_ALIGN: usize = 4;
-
 const MAGIC_FREE: usize = 0xDEADDEAD;
 const MAGIC_BUSY: usize = 0xFEEDFEED;
 
@@ -41,10 +38,6 @@ impl ListAlloc {
 
 unsafe impl GlobalAlloc for ListAlloc {
     unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
-        if layout.align() > MIN_ALIGN {
-            panic!("cannot allocate with alignment {}", layout.align())
-        }
-
         let header_size = mem::size_of::<Header>();
         let size;
         if layout.size() % header_size != 0 {
@@ -52,6 +45,7 @@ unsafe impl GlobalAlloc for ListAlloc {
         } else {
             size = layout.size()
         }
+        let align = layout.align();
 
         let mut curr = self.root;
         while !curr.is_null() {
@@ -67,20 +61,48 @@ unsafe impl GlobalAlloc for ListAlloc {
                         next = (*curr).next;
                     }
 
-                    if (*curr).size > size + header_size * 2 {
+                    unsafe fn split(header: *mut Header, split_size: usize) {
-                        // Split
+                        let offset = mem::size_of::<Header>() + split_size;
-                        let offset = header_size + size;
+                        let next = (header as *mut u8).offset(offset as isize) as *mut Header;
-                        let next = (curr as *mut u8).offset(offset as isize) as *mut Header;
                         (*next).magic = MAGIC_FREE;
-                        (*next).size  = (*curr).size - offset;
+                        (*next).size  = (*header).size - offset;
-                        (*next).next  = (*curr).next;
+                        (*next).next  = (*header).next;
-                        (*curr).next  = next;
+                        (*header).next  = next;
-                        (*curr).size  = size;
+                        (*header).size  = split_size;
                     }
 
-                    if (*curr).size >= size {
+                    // Case 1: Memory can be allocated straight from the current chunk
-                        (*curr).magic = MAGIC_BUSY;
+                    if (curr.offset(1) as usize) % align == 0 {
-                        return curr.offset(1) as *mut u8
+                        // Check available space
+                        if (*curr).size > size + header_size * 2 {
+                            split(curr, size);
+                        }
+
+                        if (*curr).size >= size {
+                            (*curr).magic = MAGIC_BUSY;
+                            return curr.offset(1) as *mut u8
+                        }
+                    }
+
+                    // Case 2: Padding is needed to satisfy the alignment
+                    else {
+                        let alloc_addr = curr.offset(2) as usize;
+                        let padding_size = align - (alloc_addr % align);
+
+                        if (*curr).size >= size + padding_size + header_size {
+                            // Create a padding region
+                            split(curr, padding_size);
+
+                            curr = (*curr).next;
+
+                            // Check if a padding is needed at the rear
+                            if (*curr).size > size + header_size * 2 {
+                                split(curr, size);
+                            }
+
+                            (*curr).magic = MAGIC_BUSY;
+                            return curr.offset(1) as *mut u8
+                        }
                     }
                 },
                 _ => panic!("heap corruption detected at {:p}", curr)

artiq/firmware/libboard_artiq/drtioaux.rs

@@ -137,11 +137,10 @@ pub fn send(linkno: u8, packet: &Packet) -> Result<(), Error<!>> {
 
         packet.write_to(&mut writer)?;
 
-        let padding = 4 - (writer.position() % 4);
+        // Pad till offset 4, insert checksum there
-        if padding != 4 {
+        let padding = (12 - (writer.position() % 8)) % 8;
-            for _ in 0..padding {
+        for _ in 0..padding {
-                writer.write_u8(0)?;
+            writer.write_u8(0)?;
-            }
         }
 
         let checksum = crc::crc32::checksum_ieee(&writer.get_ref()[0..writer.position()]);

artiq/firmware/libboard_artiq/lib.rs

@@ -1,12 +1,10 @@
-#![feature(asm, lang_items, never_type)]
+#![feature(lang_items, never_type)]
 #![no_std]
 
 extern crate failure;
 #[cfg(has_drtio)]
 #[macro_use]
 extern crate failure_derive;
-#[macro_use]
-extern crate bitflags;
 extern crate byteorder;
 extern crate crc;
 #[macro_use]
@@ -15,8 +13,6 @@ extern crate io;
 extern crate board_misoc;
 extern crate proto_artiq;
 
-pub mod pcr;
-
 pub mod spi;
 
 #[cfg(has_kernel_cpu)]

artiq/firmware/libboard_artiq/pcr.rs

@@ -1,44 +0,0 @@
-use board_misoc::spr::*;
-
-bitflags! {
-    pub struct Counters: u32 {
-        const LA     = SPR_PCMR_LA;
-        const SA     = SPR_PCMR_SA;
-        const IF     = SPR_PCMR_IF;
-        const DCM    = SPR_PCMR_DCM;
-        const ICM    = SPR_PCMR_ICM;
-        const IFS    = SPR_PCMR_IFS;
-        const LSUS   = SPR_PCMR_LSUS;
-        const BS     = SPR_PCMR_BS;
-        const DTLBM  = SPR_PCMR_DTLBM;
-        const ITLBM  = SPR_PCMR_ITLBM;
-        const DDS    = SPR_PCMR_DDS;
-
-        const INSTRN = Self::IF.bits;
-        const MEMORY = Self::LA.bits    | Self::SA.bits;
-        const STALL  = Self::DCM.bits   | Self::ICM.bits   | Self::IFS.bits |
-                       Self::LSUS.bits  | Self::BS.bits    | Self::DDS.bits ;
-        const MISS   = Self::DTLBM.bits | Self::ITLBM.bits ;
-    }
-}
-
-fn is_valid(index: u32) -> bool {
-    index < 8 && unsafe { mfspr(SPR_PCMR0 + index) } & SPR_PCMR_CP != 0
-}
-
-#[inline]
-pub fn setup(index: u32, counters: Counters) {
-    debug_assert!(is_valid(index));
-
-    unsafe {
-        mtspr(SPR_PCMR0 + index, SPR_PCMR_CISM | SPR_PCMR_CIUM | counters.bits);
-        mtspr(SPR_PCCR0 + index, 0);
-    }
-}
-
-#[inline]
-pub fn read(index: u32) -> u32 {
-    unsafe {
-        mfspr(SPR_PCCR0 + index)
-    }
-}

artiq/firmware/libboard_artiq/rpc_queue.rs

@@ -1,9 +1,9 @@
 use core::ptr::{read_volatile, write_volatile};
 use core::slice;
-use board_misoc::{mem, cache};
+use board_misoc::{mem, cache, csr::CONFIG_DATA_WIDTH_BYTES};
 
-const SEND_MAILBOX: *mut usize = (mem::MAILBOX_BASE + 4) as *mut usize;
+const SEND_MAILBOX: *mut usize = (mem::MAILBOX_BASE + CONFIG_DATA_WIDTH_BYTES as usize) as *mut usize;
-const RECV_MAILBOX: *mut usize = (mem::MAILBOX_BASE + 8) as *mut usize;
+const RECV_MAILBOX: *mut usize = (mem::MAILBOX_BASE + (CONFIG_DATA_WIDTH_BYTES * 2) as usize) as *mut usize;
 
 const QUEUE_BEGIN: usize = 0x44000000;
 const QUEUE_END:   usize = 0x44ffff80;

artiq/firmware/libboard_misoc/Cargo.toml

@@ -16,6 +16,7 @@ build_misoc = { path = "../libbuild_misoc" }
 byteorder = { version = "1.0", default-features = false }
 log = { version = "0.4", default-features = false, optional = true }
 smoltcp = { version = "0.6.0", default-features = false, optional = true }
+riscv = { version = "0.6.0", features = ["inline-asm"] }
 
 [features]
 uart_console = []

artiq/firmware/libboard_misoc/build.rs

@@ -1,18 +1,16 @@
 extern crate build_misoc;
 extern crate cc;
 
-use std::env;
 use std::path::Path;
 
 fn main() {
     build_misoc::cfg();
 
-    let triple = env::var("TARGET").unwrap();
+    let vectors_path = "riscv32/vectors.S";
-    let arch = triple.split("-").next().unwrap();
-    let vectors_path = Path::new(arch).join("vectors.S");
 
-    println!("cargo:rerun-if-changed={}", vectors_path.to_str().unwrap());
+    println!("cargo:rerun-if-changed={}", vectors_path);
     cc::Build::new()
-        .file(vectors_path)
+        .flag("--target=riscv32-unknown-elf")
+        .file(Path::new(vectors_path))
         .compile("vectors");
 }

artiq/firmware/libboard_misoc/config.rs

@@ -77,10 +77,10 @@ mod imp {
 
     mod lock {
         use core::slice;
-        use core::sync::atomic::{AtomicUsize, Ordering, ATOMIC_USIZE_INIT};
+        use core::sync::atomic::{AtomicUsize, Ordering};
         use super::Error;
 
-        static LOCKED: AtomicUsize = ATOMIC_USIZE_INIT;
+        static LOCKED: AtomicUsize = AtomicUsize::new(0);
 
         pub struct Lock;
 
@@ -216,7 +216,7 @@ mod imp {
         let mut offset = 0;
         let mut iter = Iter::new(old_data);
         'iter: while let Some(result) = iter.next() {
-            let (key, mut value) = result?;
+            let (key, value) = result?;
             if value.is_empty() {
                 // This is a removed entry, ignore it.
                 continue

artiq/firmware/libboard_misoc/lib.rs

@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(asm, try_from)]
+#![feature(llvm_asm)]
 
 extern crate byteorder;
 #[cfg(feature = "log")]
@@ -7,10 +7,13 @@ extern crate log;
 #[cfg(feature = "smoltcp")]
 extern crate smoltcp;
 
-#[cfg(target_arch = "or1k")]
+#[cfg(target_arch = "riscv32")]
-#[path = "or1k/mod.rs"]
+#[path = "riscv32/mod.rs"]
 mod arch;
 
+#[cfg(target_arch = "riscv32")]
+extern crate riscv;
+
 pub use arch::*;
 
 include!(concat!(env!("BUILDINC_DIRECTORY"), "/generated/mem.rs"));

artiq/firmware/libboard_misoc/or1k/boot.rs

@@ -1,49 +0,0 @@
-use super::{irq, cache};
-
-pub unsafe fn reset() -> ! {
-    irq::set_ie(false);
-    asm!(r#"
-        l.j     _reset_handler
-         l.nop
-    "# : : : : "volatile");
-    loop {}
-}
-
-pub unsafe fn jump(addr: usize) -> ! {
-    irq::set_ie(false);
-    cache::flush_cpu_icache();
-    asm!(r#"
-        l.jr    $0
-         l.nop
-    "# : : "r"(addr) : : "volatile");
-    loop {}
-}
-
-pub unsafe fn hotswap(firmware: &[u8]) -> ! {
-    irq::set_ie(false);
-    asm!(r#"
-        # This loop overwrites itself, but it's structured in such a way
-        # that before that happens, it loads itself into I$$ fully.
-        l.movhi    r4, hi(_reset_handler)
-        l.ori      r4, r4, lo(_reset_handler)
-        l.or       r7, r4, r0
-    0:  l.sfnei    r5, 0
-        l.bf       1f
-         l.nop
-        l.jr       r7
-         l.nop
-    1:  l.lwz      r6, 0(r3)
-        l.sw       0(r4), r6
-        l.addi     r3, r3, 4
-        l.addi     r4, r4, 4
-        l.addi     r5, r5, -4
-        l.bf       0b
-         l.nop
-    "#
-    :
-    : "{r3}"(firmware.as_ptr() as usize),
-      "{r5}"(firmware.len())
-    :
-    : "volatile");
-    loop {}
-}

artiq/firmware/libboard_misoc/or1k/cache.rs

@@ -1,49 +0,0 @@
-#[cfg(has_ddrphy)]
-use core::ptr;
-use super::spr::*;
-#[cfg(has_ddrphy)]
-use csr;
-#[cfg(has_ddrphy)]
-use mem;
-
-pub fn flush_cpu_icache() {
-    unsafe {
-        let iccfgr = mfspr(SPR_ICCFGR);
-        let ways = 1 << (iccfgr & SPR_ICCFGR_NCW);
-        let set_size = 1 << ((iccfgr & SPR_ICCFGR_NCS) >> 3);
-        let block_size = if iccfgr & SPR_ICCFGR_CBS != 0 { 32 } else { 16 };
-        let size = set_size * ways * block_size;
-
-        let mut i = 0;
-        while i < size {
-            mtspr(SPR_ICBIR, i);
-            i += block_size;
-        }
-    }
-}
-
-pub fn flush_cpu_dcache() {
-    unsafe {
-        let dccfgr = mfspr(SPR_DCCFGR);
-        let ways = 1 << (dccfgr & SPR_ICCFGR_NCW);
-        let set_size = 1 << ((dccfgr & SPR_DCCFGR_NCS) >> 3);
-        let block_size = if dccfgr & SPR_DCCFGR_CBS != 0 { 32 } else { 16 };
-        let size = set_size * ways * block_size;
-
-        let mut i = 0;
-        while i < size {
-            mtspr(SPR_DCBIR, i);
-            i += block_size;
-        }
-    }
-}
-
-#[cfg(has_ddrphy)]
-pub fn flush_l2_cache() {
-    unsafe {
-        for i in 0..2 * (csr::CONFIG_L2_SIZE as usize) / 4 {
-            let addr = mem::MAIN_RAM_BASE + i * 4;
-            ptr::read_volatile(addr as *const usize);
-        }
-    }
-}

artiq/firmware/libboard_misoc/or1k/irq.rs

@@ -1,107 +0,0 @@
-use core::{fmt, convert};
-
-use super::spr::*;
-
-#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
-pub enum Exception {
-    Reset = 0x1,
-    BusError = 0x2,
-    DataPageFault = 0x3,
-    InsnPageFault = 0x4,
-    Tick = 0x5,
-    Alignment = 0x6,
-    IllegalInsn = 0x7,
-    Interrupt = 0x8,
-    DtlbMiss = 0x9,
-    ItlbMiss = 0xa,
-    Range = 0xb,
-    Syscall = 0xc,
-    FloatingPoint = 0xd,
-    Trap = 0xe,
-}
-
-impl fmt::Display for Exception {
-    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
-        match *self {
-            Exception::Reset         => write!(f, "reset"),
-            Exception::BusError      => write!(f, "bus error"),
-            Exception::DataPageFault => write!(f, "data page fault"),
-            Exception::InsnPageFault => write!(f, "instruction page fault"),
-            Exception::Tick          => write!(f, "tick"),
-            Exception::Alignment     => write!(f, "alignment"),
-            Exception::IllegalInsn   => write!(f, "illegal instruction"),
-            Exception::Interrupt     => write!(f, "interrupt"),
-            Exception::DtlbMiss      => write!(f, "D-TLB miss"),
-            Exception::ItlbMiss      => write!(f, "I-TLB miss"),
-            Exception::Range         => write!(f, "range"),
-            Exception::Syscall       => write!(f, "system call"),
-            Exception::FloatingPoint => write!(f, "floating point"),
-            Exception::Trap          => write!(f, "trap"),
-        }
-    }
-}
-
-impl convert::TryFrom<u32> for Exception {
-    type Error = ();
-
-    fn try_from(num: u32) -> Result<Self, Self::Error> {
-        match num {
-            0x1 => Ok(Exception::Reset),
-            0x2 => Ok(Exception::BusError),
-            0x3 => Ok(Exception::DataPageFault),
-            0x4 => Ok(Exception::InsnPageFault),
-            0x5 => Ok(Exception::Tick),
-            0x6 => Ok(Exception::Alignment),
-            0x7 => Ok(Exception::IllegalInsn),
-            0x8 => Ok(Exception::Interrupt),
-            0x9 => Ok(Exception::DtlbMiss),
-            0xa => Ok(Exception::ItlbMiss),
-            0xb => Ok(Exception::Range),
-            0xc => Ok(Exception::Syscall),
-            0xd => Ok(Exception::FloatingPoint),
-            0xe => Ok(Exception::Trap),
-            _ => Err(())
-        }
-    }
-}
-
-#[inline]
-pub fn get_ie() -> bool {
-    unsafe { mfspr(SPR_SR) & SPR_SR_IEE != 0 }
-}
-
-#[inline]
-pub fn set_ie(ie: bool) {
-    if ie {
-        unsafe { mtspr(SPR_SR, mfspr(SPR_SR) | SPR_SR_IEE) }
-    } else {
-        unsafe { mtspr(SPR_SR, mfspr(SPR_SR) & !SPR_SR_IEE) }
-    }
-}
-
-#[inline]
-pub fn get_mask() -> u32 {
-    unsafe { mfspr(SPR_PICMR) }
-}
-
-#[inline]
-pub fn set_mask(mask: u32) {
-    unsafe { mtspr(SPR_PICMR, mask) }
-}
-
-#[inline]
-pub fn pending_mask() -> u32 {
-    unsafe { mfspr(SPR_PICSR) }
-}
-
-pub fn enable(irq: u32) {
-    set_mask(get_mask() | (1 << irq))
-}
-
-pub fn disable(irq: u32) {
-    set_mask(get_mask() & !(1 << irq))
-}
-
-pub fn is_pending(irq: u32) -> bool {
-    get_mask() & (1 << irq) != 0
-}

artiq/firmware/libboard_misoc/or1k/spr.rs

@@ -1,231 +0,0 @@
-#[inline(always)]
-pub unsafe fn mfspr(reg: u32) -> u32 {
-    let value: u32;
-    asm!("l.mfspr $0, $1, 0" : "=r"(value) : "r"(reg) : : "volatile");
-    value
-}
-
-#[inline(always)]
-pub unsafe fn mtspr(reg: u32, value: u32) {
-    asm!("l.mtspr $0, $1, 0" : : "r"(reg), "r"(value) : : "volatile")
-}
-
-/* Definition of special-purpose registers (SPRs). */
-
-pub const MAX_GRPS:         u32 = 32;
-pub const MAX_SPRS_PER_GRP_BITS: u32 = 11;
-pub const MAX_SPRS_PER_GRP: u32 = 1 << MAX_SPRS_PER_GRP_BITS;
-pub const MAX_SPRS:         u32 = 0x10000;
-
-/* Base addresses for the groups */
-pub const SPRGROUP_SYS:     u32 = 0 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_DMMU:    u32 = 1 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_IMMU:    u32 = 2 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_DC:      u32 = 3 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_IC:      u32 = 4 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_MAC:     u32 = 5 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_D:       u32 = 6 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_PC:      u32 = 7 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_PM:      u32 = 8 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_PIC:     u32 = 9 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_TT:      u32 = 10 << MAX_SPRS_PER_GRP_BITS;
-pub const SPRGROUP_FP:      u32 = 11 << MAX_SPRS_PER_GRP_BITS;
-
-/* System control and status group */
-pub const SPR_VR:           u32 = SPRGROUP_SYS + 0;
-pub const SPR_UPR:          u32 = SPRGROUP_SYS + 1;
-pub const SPR_CPUCFGR:      u32 = SPRGROUP_SYS + 2;
-pub const SPR_DMMUCFGR:     u32 = SPRGROUP_SYS + 3;
-pub const SPR_IMMUCFGR:     u32 = SPRGROUP_SYS + 4;
-pub const SPR_DCCFGR:       u32 = SPRGROUP_SYS + 5;
-pub const SPR_ICCFGR:       u32 = SPRGROUP_SYS + 6;
-pub const SPR_DCFGR:        u32 = SPRGROUP_SYS + 7;
-pub const SPR_PCCFGR:       u32 = SPRGROUP_SYS + 8;
-pub const SPR_VR2:          u32 = SPRGROUP_SYS + 9;
-pub const SPR_AVR:          u32 = SPRGROUP_SYS + 10;
-pub const SPR_EVBAR:        u32 = SPRGROUP_SYS + 11;
-pub const SPR_AECR:         u32 = SPRGROUP_SYS + 12;
-pub const SPR_AESR:         u32 = SPRGROUP_SYS + 13;
-pub const SPR_NPC:          u32 = SPRGROUP_SYS + 16;  /* CZ 21/06/01 */
-pub const SPR_SR:           u32 = SPRGROUP_SYS + 17;  /* CZ 21/06/01 */
-pub const SPR_PPC:          u32 = SPRGROUP_SYS + 18;  /* CZ 21/06/01 */
-pub const SPR_FPCSR:        u32 = SPRGROUP_SYS + 20;  /* CZ 21/06/01 */
-pub const SPR_ISR_BASE:     u32 = SPRGROUP_SYS + 21;
-pub const SPR_EPCR_BASE:    u32 = SPRGROUP_SYS + 32;  /* CZ 21/06/01 */
-pub const SPR_EPCR_LAST:    u32 = SPRGROUP_SYS + 47;  /* CZ 21/06/01 */
-pub const SPR_EEAR_BASE:    u32 = SPRGROUP_SYS + 48;
-pub const SPR_EEAR_LAST:    u32 = SPRGROUP_SYS + 63;
-pub const SPR_ESR_BASE:     u32 = SPRGROUP_SYS + 64;
-pub const SPR_ESR_LAST:     u32 = SPRGROUP_SYS + 79;
-pub const SPR_GPR_BASE:     u32 = SPRGROUP_SYS + 1024;
-
-/* Data MMU group */
-pub const SPR_DMMUCR:       u32 = SPRGROUP_DMMU + 0;
-pub const SPR_DTLBEIR:      u32 = SPRGROUP_DMMU + 2;
-
-/* Instruction MMU group */
-pub const SPR_IMMUCR:       u32 = SPRGROUP_IMMU + 0;
-pub const SPR_ITLBEIR:      u32 = SPRGROUP_IMMU + 2;
-
-/* Data cache group */
-pub const SPR_DCCR:         u32 = SPRGROUP_DC + 0;
-pub const SPR_DCBPR:        u32 = SPRGROUP_DC + 1;
-pub const SPR_DCBFR:        u32 = SPRGROUP_DC + 2;
-pub const SPR_DCBIR:        u32 = SPRGROUP_DC + 3;
-pub const SPR_DCBWR:        u32 = SPRGROUP_DC + 4;
-pub const SPR_DCBLR:        u32 = SPRGROUP_DC + 5;
-
-/* Instruction cache group */
-pub const SPR_ICCR:         u32 = SPRGROUP_IC + 0;
-pub const SPR_ICBPR:        u32 = SPRGROUP_IC + 1;
-pub const SPR_ICBIR:        u32 = SPRGROUP_IC + 2;
-pub const SPR_ICBLR:        u32 = SPRGROUP_IC + 3;
-
-// [snip]
-
-/* Performance counters group */
-pub const SPR_PCCR0:        u32 = SPRGROUP_PC + 0;
-pub const SPR_PCCR1:        u32 = SPRGROUP_PC + 1;
-pub const SPR_PCCR2:        u32 = SPRGROUP_PC + 2;
-pub const SPR_PCCR3:        u32 = SPRGROUP_PC + 3;
-pub const SPR_PCCR4:        u32 = SPRGROUP_PC + 4;
-pub const SPR_PCCR5:        u32 = SPRGROUP_PC + 5;
-pub const SPR_PCCR6:        u32 = SPRGROUP_PC + 6;
-pub const SPR_PCCR7:        u32 = SPRGROUP_PC + 7;
-pub const SPR_PCMR0:        u32 = SPRGROUP_PC + 8;
-pub const SPR_PCMR1:        u32 = SPRGROUP_PC + 9;
-pub const SPR_PCMR2:        u32 = SPRGROUP_PC + 10;
-pub const SPR_PCMR3:        u32 = SPRGROUP_PC + 11;
-pub const SPR_PCMR4:        u32 = SPRGROUP_PC + 12;
-pub const SPR_PCMR5:        u32 = SPRGROUP_PC + 13;
-pub const SPR_PCMR6:        u32 = SPRGROUP_PC + 14;
-pub const SPR_PCMR7:        u32 = SPRGROUP_PC + 15;
-
-/* PIC group */
-pub const SPR_PICMR:        u32 = SPRGROUP_PIC + 0;
-pub const SPR_PICPR:        u32 = SPRGROUP_PIC + 1;
-pub const SPR_PICSR:        u32 = SPRGROUP_PIC + 2;
-
-// [snip]
-
-/*
- * Bit definitions for the Supervision Register
- *
- */
-pub const SPR_SR_SM:        u32 = 0x00000001;  /* Supervisor Mode */
-pub const SPR_SR_TEE:       u32 = 0x00000002;  /* Tick timer Exception Enable */
-pub const SPR_SR_IEE:       u32 = 0x00000004;  /* Interrupt Exception Enable */
-pub const SPR_SR_DCE:       u32 = 0x00000008;  /* Data Cache Enable */
-pub const SPR_SR_ICE:       u32 = 0x00000010;  /* Instruction Cache Enable */
-pub const SPR_SR_DME:       u32 = 0x00000020;  /* Data MMU Enable */
-pub const SPR_SR_IME:       u32 = 0x00000040;  /* Instruction MMU Enable */
-pub const SPR_SR_LEE:       u32 = 0x00000080;  /* Little Endian Enable */
-pub const SPR_SR_CE:        u32 = 0x00000100;  /* CID Enable */
-pub const SPR_SR_F:         u32 = 0x00000200;  /* Condition Flag */
-pub const SPR_SR_CY:        u32 = 0x00000400;  /* Carry flag */
-pub const SPR_SR_OV:        u32 = 0x00000800;  /* Overflow flag */
-pub const SPR_SR_OVE:       u32 = 0x00001000;  /* Overflow flag Exception */
-pub const SPR_SR_DSX:       u32 = 0x00002000;  /* Delay Slot Exception */
-pub const SPR_SR_EPH:       u32 = 0x00004000;  /* Exception Prefix High */
-pub const SPR_SR_FO:        u32 = 0x00008000;  /* Fixed one */
-pub const SPR_SR_SUMRA:     u32 = 0x00010000;  /* Supervisor SPR read access */
-pub const SPR_SR_RES:       u32 = 0x0ffe0000;  /* Reserved */
-pub const SPR_SR_CID:       u32 = 0xf0000000;  /* Context ID */
-
-/*
- * Bit definitions for Data Cache Control register
- *
- */
-pub const SPR_DCCR_EW:      u32 = 0x000000ff;  /* Enable ways */
-
-/*
- * Bit definitions for Insn Cache Control register
- *
- */
-pub const SPR_ICCR_EW:      u32 = 0x000000ff;  /* Enable ways */
-
-/*
- * Bit definitions for Data Cache Configuration Register
- *
- */
-pub const SPR_DCCFGR_NCW:       u32 = 0x00000007;
-pub const SPR_DCCFGR_NCS:       u32 = 0x00000078;
-pub const SPR_DCCFGR_CBS:       u32 = 0x00000080;
-pub const SPR_DCCFGR_CWS:       u32 = 0x00000100;
-pub const SPR_DCCFGR_CCRI:      u32 = 0x00000200;
-pub const SPR_DCCFGR_CBIRI:     u32 = 0x00000400;
-pub const SPR_DCCFGR_CBPRI:     u32 = 0x00000800;
-pub const SPR_DCCFGR_CBLRI:     u32 = 0x00001000;
-pub const SPR_DCCFGR_CBFRI:     u32 = 0x00002000;
-pub const SPR_DCCFGR_CBWBRI:    u32 = 0x00004000;
-
-pub const SPR_DCCFGR_NCW_OFF:   u32 = 0;
-pub const SPR_DCCFGR_NCS_OFF:   u32 = 3;
-pub const SPR_DCCFGR_CBS_OFF:   u32 = 7;
-
-/*
- * Bit definitions for Instruction Cache Configuration Register
- *
- */
-pub const SPR_ICCFGR_NCW:       u32 = 0x00000007;
-pub const SPR_ICCFGR_NCS:       u32 = 0x00000078;
-pub const SPR_ICCFGR_CBS:       u32 = 0x00000080;
-pub const SPR_ICCFGR_CCRI:      u32 = 0x00000200;
-pub const SPR_ICCFGR_CBIRI:     u32 = 0x00000400;
-pub const SPR_ICCFGR_CBPRI:     u32 = 0x00000800;
-pub const SPR_ICCFGR_CBLRI:     u32 = 0x00001000;
-
-pub const SPR_ICCFGR_NCW_OFF:   u32 = 0;
-pub const SPR_ICCFGR_NCS_OFF:   u32 = 3;
-pub const SPR_ICCFGR_CBS_OFF:   u32 = 7;
-
-/*
- * Bit definitions for Data MMU Configuration Register
- *
- */
-pub const SPR_DMMUCFGR_NTW:     u32 = 0x00000003;
-pub const SPR_DMMUCFGR_NTS:     u32 = 0x0000001C;
-pub const SPR_DMMUCFGR_NAE:     u32 = 0x000000E0;
-pub const SPR_DMMUCFGR_CRI:     u32 = 0x00000100;
-pub const SPR_DMMUCFGR_PRI:     u32 = 0x00000200;
-pub const SPR_DMMUCFGR_TEIRI:   u32 = 0x00000400;
-pub const SPR_DMMUCFGR_HTR:     u32 = 0x00000800;
-
-pub const SPR_DMMUCFGR_NTW_OFF: u32 = 0;
-pub const SPR_DMMUCFGR_NTS_OFF: u32 = 2;
-
-/*
- * Bit definitions for Instruction MMU Configuration Register
- *
- */
-pub const SPR_IMMUCFGR_NTW:     u32 = 0x00000003;
-pub const SPR_IMMUCFGR_NTS:     u32 = 0x0000001C;
-pub const SPR_IMMUCFGR_NAE:     u32 = 0x000000E0;
-pub const SPR_IMMUCFGR_CRI:     u32 = 0x00000100;
-pub const SPR_IMMUCFGR_PRI:     u32 = 0x00000200;
-pub const SPR_IMMUCFGR_TEIRI:   u32 = 0x00000400;
-pub const SPR_IMMUCFGR_HTR:     u32 = 0x00000800;
-
-pub const SPR_IMMUCFGR_NTW_OFF: u32 = 0;
-pub const SPR_IMMUCFGR_NTS_OFF: u32 = 2;
-
-/*
- * Bit definitions for Performance counters mode registers
- *
- */
-pub const SPR_PCMR_CP:    u32 = 0x00000001;  /* Counter present */
-pub const SPR_PCMR_UMRA:  u32 = 0x00000002;  /* User mode read access */
-pub const SPR_PCMR_CISM:  u32 = 0x00000004;  /* Count in supervisor mode */
-pub const SPR_PCMR_CIUM:  u32 = 0x00000008;  /* Count in user mode */
-pub const SPR_PCMR_LA:    u32 = 0x00000010;  /* Load access event */
-pub const SPR_PCMR_SA:    u32 = 0x00000020;  /* Store access event */
-pub const SPR_PCMR_IF:    u32 = 0x00000040;  /* Instruction fetch event*/
-pub const SPR_PCMR_DCM:   u32 = 0x00000080;  /* Data cache miss event */
-pub const SPR_PCMR_ICM:   u32 = 0x00000100;  /* Insn cache miss event */
-pub const SPR_PCMR_IFS:   u32 = 0x00000200;  /* Insn fetch stall event */
-pub const SPR_PCMR_LSUS:  u32 = 0x00000400;  /* LSU stall event */
-pub const SPR_PCMR_BS:    u32 = 0x00000800;  /* Branch stall event */
-pub const SPR_PCMR_DTLBM: u32 = 0x00001000;  /* DTLB miss event */
-pub const SPR_PCMR_ITLBM: u32 = 0x00002000;  /* ITLB miss event */
-pub const SPR_PCMR_DDS:   u32 = 0x00004000;  /* Data dependency stall event */
-pub const SPR_PCMR_WPE:   u32 = 0x03ff8000;  /* Watchpoint events */

artiq/firmware/libboard_misoc/or1k/vectors.S

@@ -1,413 +0,0 @@
-/*
- * (C) Copyright 2012, Stefan Kristiansson <stefan.kristiansson@saunalahti.fi>
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation; either version 2 of
- * the License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston,
- * MA 02111-1307 USA
- */
-
-#include <spr-defs.h>
-
-/*
- * OR1K Architecture has a 128 byte "red zone" after the stack that can not be
- * touched by exception handlers. GCC uses this red zone for locals and
- * temps without needing to change the stack pointer.
- */
-#define OR1K_RED_ZONE_SIZE  128
-
-/*
- * We need 4 bytes (32 bits) * 32 registers space on the stack to save all the
- * registers.
- */
-#define EXCEPTION_STACK_SIZE    ((4*32) + OR1K_RED_ZONE_SIZE)
-
-#define HANDLE_EXCEPTION                    ; \
-    l.addi  r1, r1, -EXCEPTION_STACK_SIZE   ; \
-    l.sw    0x1c(r1), r9                    ; \
-    l.jal   _exception_handler              ; \
-     l.nop                                  ; \
-    l.lwz   r9, 0x1c(r1)                    ; \
-    l.addi  r1, r1, EXCEPTION_STACK_SIZE    ; \
-    l.rfe                                   ; \
-     l.nop
-
-
-.section    .vectors, "ax", @progbits
-.global     _reset_handler
-_reset_handler:
-    l.movhi r0, 0
-    l.movhi r1, 0
-    l.movhi r2, 0
-    l.movhi r3, 0
-    l.movhi r4, 0
-    l.movhi r5, 0
-    l.movhi r6, 0
-    l.movhi r7, 0
-    l.movhi r8, 0
-    l.movhi r9, 0
-    l.movhi r10, 0
-    l.movhi r11, 0
-    l.movhi r12, 0
-    l.movhi r13, 0
-    l.movhi r14, 0
-    l.movhi r15, 0
-    l.movhi r16, 0
-    l.movhi r17, 0
-    l.movhi r18, 0
-    l.movhi r19, 0
-    l.movhi r20, 0
-    l.movhi r21, 0
-    l.movhi r22, 0
-    l.movhi r23, 0
-    l.movhi r24, 0
-    l.movhi r25, 0
-    l.movhi r26, 0
-    l.movhi r27, 0
-    l.movhi r28, 0
-    l.movhi r29, 0
-    l.movhi r30, 0
-    l.movhi r31, 0
-
-    l.ori   r21, r0, SPR_SR_SM
-    l.mtspr r0, r21, SPR_SR
-    l.movhi r21, hi(_reset_handler)
-    l.ori   r21, r21, lo(_reset_handler)
-    l.mtspr r0, r21, SPR_EVBAR
-    /* enable caches */
-    l.jal   _cache_init
-     l.nop
-    l.j     _crt0
-     l.nop
-
-    /* bus error */
-    .org    0x200
-    HANDLE_EXCEPTION
-
-    /* data page fault */
-    .org    0x300
-    HANDLE_EXCEPTION
-
-    /* instruction page fault */
-    .org    0x400
-    HANDLE_EXCEPTION
-
-    /* tick timer */
-    .org    0x500
-    HANDLE_EXCEPTION
-
-    /* alignment */
-    .org    0x600
-    HANDLE_EXCEPTION
-
-    /* illegal instruction */
-    .org    0x700
-    HANDLE_EXCEPTION
-
-    /* external interrupt */
-    .org    0x800
-    HANDLE_EXCEPTION
-
-    /* D-TLB miss */
-    .org    0x900
-    HANDLE_EXCEPTION
-
-    /* I-TLB miss */
-    .org    0xa00
-    HANDLE_EXCEPTION
-
-    /* range */
-    .org    0xb00
-    HANDLE_EXCEPTION
-
-    /* system call */
-    .org    0xc00
-    HANDLE_EXCEPTION
-
-    /* floating point */
-    .org    0xd00
-    HANDLE_EXCEPTION
-
-    /* trap */
-    .org    0xe00
-    HANDLE_EXCEPTION
-
-    /* reserved */
-    .org    0xf00
-    HANDLE_EXCEPTION
-
-    .org 0x1000
-_crt0:
-    /* Setup stack and global pointer */
-    l.movhi    r1, hi(_fstack)
-    l.ori     r1, r1, lo(_fstack)
-
-    /* Clear BSS */
-    l.movhi    r21, hi(_fbss)
-    l.ori     r21, r21, lo(_fbss)
-    l.movhi    r3, hi(_ebss)
-    l.ori     r3, r3, lo(_ebss)
-.clearBSS:
-    l.sfeq  r21, r3
-    l.bf    .callMain
-     l.nop
-    l.sw      0(r21), r0
-    l.addi    r21, r21, 4
-    l.j      .clearBSS
-     l.nop
-
-.callMain:
-    l.j     main
-     l.nop
-
-_exception_handler:
-    .cfi_startproc
-    .cfi_return_column 32
-    .cfi_signal_frame
-    .cfi_def_cfa_offset EXCEPTION_STACK_SIZE
-    l.sw    0x00(r1), r2
-    .cfi_offset 2, 0x00-EXCEPTION_STACK_SIZE
-    l.sw    0x04(r1), r3
-    .cfi_offset 3, 0x04-EXCEPTION_STACK_SIZE
-    l.sw    0x08(r1), r4
-    .cfi_offset 4, 0x08-EXCEPTION_STACK_SIZE
-    l.sw    0x0c(r1), r5
-    .cfi_offset 5, 0x0c-EXCEPTION_STACK_SIZE
-    l.sw    0x10(r1), r6
-    .cfi_offset 6, 0x10-EXCEPTION_STACK_SIZE
-    l.sw    0x14(r1), r7
-    .cfi_offset 7, 0x14-EXCEPTION_STACK_SIZE
-    l.sw    0x18(r1), r8
-    .cfi_offset 8, 0x18-EXCEPTION_STACK_SIZE
-    /* r9 saved in HANDLE_EXCEPTION */
-    .cfi_offset 9, 0x1c-EXCEPTION_STACK_SIZE
-    l.sw    0x20(r1), r10
-    .cfi_offset 10, 0x20-EXCEPTION_STACK_SIZE
-    l.sw    0x24(r1), r11
-    .cfi_offset 11, 0x24-EXCEPTION_STACK_SIZE
-    l.sw    0x28(r1), r12
-    .cfi_offset 12, 0x28-EXCEPTION_STACK_SIZE
-    l.sw    0x2c(r1), r13
-    .cfi_offset 13, 0x2c-EXCEPTION_STACK_SIZE
-    l.sw    0x30(r1), r14
-    .cfi_offset 14, 0x30-EXCEPTION_STACK_SIZE
-    l.sw    0x34(r1), r15
-    .cfi_offset 15, 0x34-EXCEPTION_STACK_SIZE
-    l.sw    0x38(r1), r16
-    .cfi_offset 16, 0x38-EXCEPTION_STACK_SIZE
-    l.sw    0x3c(r1), r17
-    .cfi_offset 17, 0x3c-EXCEPTION_STACK_SIZE
-    l.sw    0x40(r1), r18
-    .cfi_offset 18, 0x40-EXCEPTION_STACK_SIZE
-    l.sw    0x44(r1), r19
-    .cfi_offset 19, 0x44-EXCEPTION_STACK_SIZE
-    l.sw    0x48(r1), r20
-    .cfi_offset 20, 0x48-EXCEPTION_STACK_SIZE
-    l.sw    0x4c(r1), r21
-    .cfi_offset 21, 0x4c-EXCEPTION_STACK_SIZE
-    l.sw    0x50(r1), r22
-    .cfi_offset 22, 0x50-EXCEPTION_STACK_SIZE
-    l.sw    0x54(r1), r23
-    .cfi_offset 23, 0x54-EXCEPTION_STACK_SIZE
-    l.sw    0x58(r1), r24
-    .cfi_offset 24, 0x58-EXCEPTION_STACK_SIZE
-    l.sw    0x5c(r1), r25
-    .cfi_offset 25, 0x5c-EXCEPTION_STACK_SIZE
-    l.sw    0x60(r1), r26
-    .cfi_offset 26, 0x60-EXCEPTION_STACK_SIZE
-    l.sw    0x64(r1), r27
-    .cfi_offset 27, 0x64-EXCEPTION_STACK_SIZE
-    l.sw    0x68(r1), r28
-    .cfi_offset 28, 0x68-EXCEPTION_STACK_SIZE
-    l.sw    0x6c(r1), r29
-    .cfi_offset 29, 0x6c-EXCEPTION_STACK_SIZE
-    l.sw    0x70(r1), r30
-    .cfi_offset 30, 0x70-EXCEPTION_STACK_SIZE
-    l.sw    0x74(r1), r31
-    .cfi_offset 31, 0x74-EXCEPTION_STACK_SIZE
-
-    /* Save return address */
-    l.or    r14, r0, r9
-    /* Calculate exception vector from handler address */
-    l.andi  r3, r9, 0xf00
-    l.srli  r3, r3, 8
-    /* Pass saved register state */
-    l.or    r4, r0, r1
-    /* Extract exception PC */
-    l.mfspr r5, r0, SPR_EPCR_BASE
-    /* Tell exception PC to the unwinder */
-    l.sw    0x78(r1), r5
-    .cfi_offset 32, 0x78-EXCEPTION_STACK_SIZE
-    /* Extract exception effective address */
-    l.mfspr r6, r0, SPR_EEAR_BASE
-    /* Extract exception SR */
-    l.mfspr r7, r0, SPR_ESR_BASE
-    /* Call exception handler with the link address as argument */
-    l.jal   exception
-     l.nop
-
-    /* Load return address */
-    l.or    r9, r0, r14
-    /* Restore state */
-    l.lwz   r2, 0x00(r1)
-    l.lwz   r3, 0x04(r1)
-    l.lwz   r4, 0x08(r1)
-    l.lwz   r5, 0x0c(r1)
-    l.lwz   r6, 0x10(r1)
-    l.lwz   r7, 0x14(r1)
-    l.lwz   r8, 0x18(r1)
-    l.lwz   r10, 0x20(r1)
-    l.lwz   r11, 0x24(r1)
-    l.lwz   r12, 0x28(r1)
-    l.lwz   r13, 0x2c(r1)
-    l.lwz   r14, 0x30(r1)
-    l.lwz   r15, 0x34(r1)
-    l.lwz   r16, 0x38(r1)
-    l.lwz   r17, 0x3c(r1)
-    l.lwz   r18, 0x40(r1)
-    l.lwz   r19, 0x44(r1)
-    l.lwz   r20, 0x48(r1)
-    l.lwz   r21, 0x4c(r1)
-    l.lwz   r22, 0x50(r1)
-    l.lwz   r23, 0x54(r1)
-    l.lwz   r24, 0x58(r1)
-    l.lwz   r25, 0x5c(r1)
-    l.lwz   r26, 0x60(r1)
-    l.lwz   r27, 0x64(r1)
-    l.lwz   r28, 0x68(r1)
-    l.lwz   r29, 0x6c(r1)
-    l.lwz   r30, 0x70(r1)
-    l.lwz   r31, 0x74(r1)
-    l.jr    r9
-     l.nop
-    .cfi_endproc
-
-.global _cache_init
-_cache_init:
-    /*
-    This function is to be used ONLY during reset, before main() is called.
-    TODO: Perhaps break into individual enable instruction/data cache
-          sections functions, and provide disable functions, also, all
-          callable from C
-    */
-
-    /* Instruction cache enable */
-    /* Check if IC present and skip enabling otherwise */
-#if 1
-.L6:
-    l.mfspr r3,r0,SPR_UPR
-    l.andi  r7,r3,SPR_UPR_ICP
-    l.sfeq  r7,r0
-    l.bf    .L8
-    l.nop
-
-    /* Disable IC */
-    l.mfspr r6,r0,SPR_SR
-    l.addi  r5,r0,-1
-    l.xori  r5,r5,SPR_SR_ICE
-    l.and   r5,r6,r5
-    l.mtspr r0,r5,SPR_SR
-
-    /* Establish cache block size
-    If BS=0, 16;
-    If BS=1, 32;
-    r14 contain block size
-    */
-    l.mfspr r3,r0,SPR_ICCFGR
-    l.andi  r7,r3,SPR_ICCFGR_CBS
-    l.srli  r8,r7,7
-    l.ori   r4,r0,16
-    l.sll   r14,r4,r8
-
-    /* Establish number of cache sets
-    r10 contains number of cache sets
-    r8 contains log(# of cache sets)
-    */
-    l.andi  r7,r3,SPR_ICCFGR_NCS
-    l.srli  r8,r7,3
-    l.ori   r4,r0,1
-    l.sll   r10,r4,r8
-
-    /* Invalidate IC */
-    l.addi  r6,r0,0
-    l.sll   r5,r14,r8
-
-.L7:    l.mtspr r0,r6,SPR_ICBIR
-    l.sfne  r6,r5
-    l.bf    .L7
-    l.add   r6,r6,r14
-
-    /* Enable IC */
-    l.mfspr r6,r0,SPR_SR
-    l.ori   r6,r6,SPR_SR_ICE
-    l.mtspr r0,r6,SPR_SR
-    l.nop
-    l.nop
-    l.nop
-    l.nop
-    l.nop
-    l.nop
-    l.nop
-    l.nop
-    /* Data cache enable */
-    /* Check if DC present and skip enabling otherwise */
-#endif
-.L8:
-#if 1
-    l.mfspr r3,r0,SPR_UPR
-    l.andi  r7,r3,SPR_UPR_DCP
-    l.sfeq  r7,r0
-    l.bf    .L10
-    l.nop
-    /* Disable DC */
-    l.mfspr r6,r0,SPR_SR
-    l.addi  r5,r0,-1
-    l.xori  r5,r5,SPR_SR_DCE
-    l.and   r5,r6,r5
-    l.mtspr r0,r5,SPR_SR
-    /* Establish cache block size
-       If BS=0, 16;
-       If BS=1, 32;
-       r14 contain block size
-    */
-    l.mfspr r3,r0,SPR_DCCFGR
-    l.andi  r7,r3,SPR_DCCFGR_CBS
-    l.srli  r8,r7,7
-    l.ori   r4,r0,16
-    l.sll   r14,r4,r8
-    /* Establish number of cache sets
-       r10 contains number of cache sets
-       r8 contains log(# of cache sets)
-    */
-    l.andi  r7,r3,SPR_DCCFGR_NCS
-    l.srli  r8,r7,3
-    l.ori   r4,r0,1
-    l.sll   r10,r4,r8
-    /* Invalidate DC */
-    l.addi  r6,r0,0
-    l.sll   r5,r14,r8
-
-.L9:
-    l.mtspr r0,r6,SPR_DCBIR
-    l.sfne  r6,r5
-    l.bf    .L9
-    l.add   r6,r6,r14
-    /* Enable DC */
-    l.mfspr r6,r0,SPR_SR
-    l.ori   r6,r6,SPR_SR_DCE
-    l.mtspr r0,r6,SPR_SR
-#endif
-.L10:
-    /* Return */
-    l.jr    r9
-    l.nop

artiq/firmware/libboard_misoc/riscv32/boot.rs

@@ -0,0 +1,30 @@
+use super::{cache, pmp};
+use riscv::register::*;
+
+pub unsafe fn reset() -> ! {
+    llvm_asm!(r#"
+        j          _reset_handler
+         nop
+    "# : : : : "volatile");
+    loop {}
+}
+
+pub unsafe fn jump(addr: usize) -> ! {
+    cache::flush_cpu_icache();
+    llvm_asm!(r#"
+        jalr       x0, 0($0)
+         nop
+    "# : : "r"(addr) : : "volatile");
+    loop {}
+}
+
+pub unsafe fn start_user(addr: usize) -> ! {
+    pmp::enable_user_memory();
+    mstatus::set_mpp(mstatus::MPP::User);
+    mepc::write(addr);
+    llvm_asm!(
+        "mret"
+        : : : : "volatile"
+    );
+    unreachable!()
+}

artiq/firmware/libboard_misoc/riscv32/cache.rs

@@ -0,0 +1,35 @@
+#[cfg(has_ddrphy)]
+use core::ptr;
+#[cfg(has_ddrphy)]
+use csr;
+#[cfg(has_ddrphy)]
+use mem;
+
+pub fn flush_cpu_icache() {
+    unsafe {
+        llvm_asm!(r#"
+            fence.i
+            nop
+            nop
+            nop
+            nop
+            nop
+        "# : : : : "volatile");
+    }
+}
+
+pub fn flush_cpu_dcache() {
+    unsafe {
+        llvm_asm!(".word(0x500F)" : : : : "volatile");
+    }
+}
+
+#[cfg(has_ddrphy)]
+pub fn flush_l2_cache() {
+    unsafe {
+        for i in 0..2 * (csr::CONFIG_L2_SIZE as usize) / 4 {
+            let addr = mem::MAIN_RAM_BASE + i * 4;
+            ptr::read_volatile(addr as *const usize);
+        }
+    }
+}

artiq/firmware/libboard_misoc/riscv32/mod.rs

@@ -1,4 +1,3 @@
-pub mod spr;
-pub mod irq;
 pub mod cache;
 pub mod boot;
+pub mod pmp;

artiq/firmware/libboard_misoc/riscv32/pmp.rs

@@ -0,0 +1,55 @@
+use riscv::register::{pmpaddr0, pmpaddr1, pmpaddr2, pmpaddr3, pmpcfg0};
+
+static mut THREAD_DEPTH: u8 = 0;
+
+const PMP_L    : usize = 0b10000000;
+const PMP_NAPOT: usize = 0b00011000;
+const PMP_X    : usize = 0b00000100;
+const PMP_W    : usize = 0b00000010;
+const PMP_R    : usize = 0b00000001;
+const PMP_OFF  : usize = 0b00000000;
+
+#[inline(always)]
+pub unsafe fn init_stack_guard(guard_base: usize) {
+    pmpaddr2::write((guard_base >> 2) | ((0x1000 - 1) >> 3));
+    pmpcfg0::write((PMP_L | PMP_NAPOT) << 16);
+}
+
+#[inline(always)]
+pub fn enable_user_memory() {
+    pmpaddr3::write((0x80000000 - 1) >> 3);
+    pmpcfg0::write((PMP_L | PMP_NAPOT | PMP_X | PMP_W | PMP_R) << 24);
+}
+
+#[inline(always)]
+pub unsafe fn push_pmp_region(addr: usize) {
+    let pmp_addr = (addr >> 2) | ((0x1000 - 1) >> 3);
+    match THREAD_DEPTH {
+        // Activate PMP0 when switching from main stack to thread
+        0 => {
+            pmpaddr0::write(pmp_addr);
+            pmpcfg0::write(PMP_NAPOT);
+        }
+
+        // Temporarily activate PMP1 when spawning a thread from a thread
+        // The thread should swap back to the main stack very soon after init
+        1 => {
+            pmpaddr1::write(pmp_addr);
+            pmpcfg0::write(PMP_NAPOT << 8 | PMP_NAPOT);
+        }
+
+        // Thread *running* another thread should not be possible
+        _ => unreachable!()
+    }
+    THREAD_DEPTH += 1;
+}
+
+#[inline(always)]
+pub unsafe fn pop_pmp_region() {
+    THREAD_DEPTH -= 1;
+    match THREAD_DEPTH {
+        0 => pmpcfg0::write(PMP_OFF),
+        1 => pmpcfg0::write(PMP_NAPOT),
+        _ => unreachable!()
+    }
+}

artiq/firmware/libboard_misoc/riscv32/vectors.S

@@ -0,0 +1,176 @@
+# Adapted from riscv-rt project, with the following license:
+#
+# Copyright 2018 RISC-V team
+#
+# Permission to use, copy, modify, and/or distribute this software for any purpose 
+# with or without fee is hereby granted, provided that the above copyright notice and
+# this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
+# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
+# FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT,
+# OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA
+# OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
+# ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+#if __riscv_xlen == 64
+# define STORE    sd
+# define LOAD     ld
+# define LOG_REGBYTES 3
+#else
+# define STORE    sw
+# define LOAD     lw
+# define LOG_REGBYTES 2
+#endif
+#define REGBYTES (1 << LOG_REGBYTES)
+
+/*
+    Entry point of all programs (_reset_handler).
+
+    It initializes DWARF call frame information, the stack pointer, the
+    frame pointer (needed for closures to work in start_rust) and the global
+    pointer. Then it calls _start_rust.
+*/
+
+.section .vectors, "ax", @progbits
+.global _reset_handler
+
+_reset_handler:
+    /* Jump to the absolute address defined by the linker script. */
+    // for 32bit
+    .if __riscv_xlen == 32
+    lui ra, %hi(_abs_start)
+    jr %lo(_abs_start)(ra)
+    .endif
+
+    // for 64bit
+    .if __riscv_xlen == 64
+.option push
+.option norelax // to prevent an unsupported R_RISCV_ALIGN relocation from being generated
+1:
+    auipc ra, %pcrel_hi(1f)
+    ld ra, %pcrel_lo(1b)(ra)
+    jr ra
+    .align  3
+1:
+    .dword _abs_start
+.option pop
+    .endif
+
+_abs_start:
+    .cfi_startproc
+    .cfi_undefined ra
+
+    csrw mie, 0
+    csrw mip, 0
+
+    li  x1, 0
+    li  x2, 0
+    li  x3, 0
+    li  x4, 0
+    li  x5, 0
+    li  x6, 0
+    li  x7, 0
+    li  x8, 0
+    li  x9, 0
+    li  x10,0
+    li  x11,0
+    li  x12,0
+    li  x13,0
+    li  x14,0
+    li  x15,0
+    li  x16,0
+    li  x17,0
+    li  x18,0
+    li  x19,0
+    li  x20,0
+    li  x21,0
+    li  x22,0
+    li  x23,0
+    li  x24,0
+    li  x25,0
+    li  x26,0
+    li  x27,0
+    li  x28,0
+    li  x29,0
+    li  x30,0
+    li  x31,0
+
+    // Check hart id
+    csrr a2, mhartid
+
+    // Allocate stacks
+    la sp, _fstack
+
+    // Set frame pointer
+    add s0, sp, zero
+
+    // Set trap vector
+    la t0, _start_trap
+    csrw mtvec, t0
+
+    // Zero initialize .bss
+    la t0, _fbss
+    la t1, _ebss
+    beq t0, t1, 2f
+1:  STORE zero, 0(t0)
+    addi t0, t0, REGBYTES
+    bne t0, t1, 1b
+2:
+    // Enter main firmware
+    jal zero, main
+
+    .cfi_endproc
+
+/*
+    Trap entry point (_start_trap)
+
+    Saves caller saved registers ra, t0..6, a0..7, calls exception,
+    restores caller saved registers and then returns.
+*/
+.global _start_trap
+/* Make it .weak so PAC/HAL can provide their own if needed. */
+.weak _start_trap
+
+_start_trap:
+    addi sp, sp, -16*REGBYTES
+
+    STORE ra, 0*REGBYTES(sp)
+    STORE t0, 1*REGBYTES(sp)
+    STORE t1, 2*REGBYTES(sp)
+    STORE t2, 3*REGBYTES(sp)
+    STORE t3, 4*REGBYTES(sp)
+    STORE t4, 5*REGBYTES(sp)
+    STORE t5, 6*REGBYTES(sp)
+    STORE t6, 7*REGBYTES(sp)
+    STORE a0, 8*REGBYTES(sp)
+    STORE a1, 9*REGBYTES(sp)
+    STORE a2, 10*REGBYTES(sp)
+    STORE a3, 11*REGBYTES(sp)
+    STORE a4, 12*REGBYTES(sp)
+    STORE a5, 13*REGBYTES(sp)
+    STORE a6, 14*REGBYTES(sp)
+    STORE a7, 15*REGBYTES(sp)
+
+    add a0, sp, zero
+    jal ra, exception
+
+    LOAD ra, 0*REGBYTES(sp)
+    LOAD t0, 1*REGBYTES(sp)
+    LOAD t1, 2*REGBYTES(sp)
+    LOAD t2, 3*REGBYTES(sp)
+    LOAD t3, 4*REGBYTES(sp)
+    LOAD t4, 5*REGBYTES(sp)
+    LOAD t5, 6*REGBYTES(sp)
+    LOAD t6, 7*REGBYTES(sp)
+    LOAD a0, 8*REGBYTES(sp)
+    LOAD a1, 9*REGBYTES(sp)
+    LOAD a2, 10*REGBYTES(sp)
+    LOAD a3, 11*REGBYTES(sp)
+    LOAD a4, 12*REGBYTES(sp)
+    LOAD a5, 13*REGBYTES(sp)
+    LOAD a6, 14*REGBYTES(sp)
+    LOAD a7, 15*REGBYTES(sp)
+
+    addi sp, sp, 16*REGBYTES
+    mret

artiq/firmware/libboard_misoc/sdram.rs

@@ -2,6 +2,7 @@
 mod ddr {
     use core::{ptr, fmt};
     use csr::{dfii, ddrphy};
+    use csr::CONFIG_DATA_WIDTH_BYTES;
     use sdram_phy::{self, spin_cycles};
     use sdram_phy::{DFII_COMMAND_CS, DFII_COMMAND_WE, DFII_COMMAND_CAS, DFII_COMMAND_RAS,
                     DFII_COMMAND_WRDATA, DFII_COMMAND_RDDATA};
@@ -14,6 +15,8 @@ mod ddr {
 
     const DQS_SIGNAL_COUNT: usize = DFII_PIX_DATA_SIZE / 2;
 
+    const CSR_SEPARATION: isize = CONFIG_DATA_WIDTH_BYTES as isize / 4;
+
     macro_rules! log {
         ($logger:expr, $( $arg:expr ),+) => (
             if let &mut Some(ref mut f) = $logger {
@@ -31,7 +34,7 @@ mod ddr {
     }
 
     #[cfg(ddrphy_wlevel)]
-    unsafe fn write_level_scan(logger: &mut Option<&mut fmt::Write>) {
+    unsafe fn write_level_scan(logger: &mut Option<&mut dyn fmt::Write>) {
         #[cfg(kusddrphy)]
         log!(logger, "DQS initial delay: {} taps\n", ddrphy::wdly_dqs_taps_read());
         log!(logger, "Write leveling scan:\n");
@@ -46,7 +49,7 @@ mod ddr {
 
         for n in 0..DQS_SIGNAL_COUNT {
             let dq_addr = dfii::PI0_RDDATA_ADDR
-                               .offset((DQS_SIGNAL_COUNT - 1 - n) as isize);
+                               .offset(CSR_SEPARATION * (DQS_SIGNAL_COUNT - 1 - n) as isize);
 
             log!(logger, "Module {}:\n", DQS_SIGNAL_COUNT - 1 - n);
 
@@ -82,7 +85,7 @@ mod ddr {
     }
 
     #[cfg(ddrphy_wlevel)]
-    unsafe fn write_level(logger: &mut Option<&mut fmt::Write>,
+    unsafe fn write_level(logger: &mut Option<&mut dyn fmt::Write>,
                           delay: &mut [u16; DQS_SIGNAL_COUNT],
                           high_skew: &mut [bool; DQS_SIGNAL_COUNT]) -> bool {
         #[cfg(kusddrphy)]
@@ -100,7 +103,7 @@ mod ddr {
         let mut failed = false;
         for n in 0..DQS_SIGNAL_COUNT {
             let dq_addr = dfii::PI0_RDDATA_ADDR
-                               .offset((DQS_SIGNAL_COUNT - 1 - n) as isize);
+                               .offset(CSR_SEPARATION * (DQS_SIGNAL_COUNT - 1 - n) as isize);
 
             delay[n] = 0;
             high_skew[n] = false;
@@ -172,7 +175,7 @@ mod ddr {
     }
 
     #[cfg(ddrphy_wlevel)]
-    unsafe fn read_bitslip(logger: &mut Option<&mut fmt::Write>,
+    unsafe fn read_bitslip(logger: &mut Option<&mut dyn fmt::Write>,
                            delay: &[u16; DQS_SIGNAL_COUNT],
                            high_skew: &[bool; DQS_SIGNAL_COUNT]) {
         let threshold_opt = delay.iter().zip(high_skew.iter())
@@ -203,7 +206,7 @@ mod ddr {
         }
     }
 
-    unsafe fn read_level_scan(logger: &mut Option<&mut fmt::Write>) {
+    unsafe fn read_level_scan(logger: &mut Option<&mut dyn fmt::Write>) {
         log!(logger, "Read leveling scan:\n");
 
         // Generate pseudo-random sequence
@@ -223,7 +226,7 @@ mod ddr {
         // Write test pattern
         for p in 0..DFII_NPHASES {
             for offset in 0..DFII_PIX_DATA_SIZE {
-                let addr = DFII_PIX_WRDATA_ADDR[p].offset(offset as isize);
+                let addr = DFII_PIX_WRDATA_ADDR[p].offset(CSR_SEPARATION * offset as isize);
                 let data = prs[DFII_PIX_DATA_SIZE * p + offset];
                 ptr::write_volatile(addr, data as u32);
             }
@@ -258,7 +261,7 @@ mod ddr {
 
                     for p in 0..DFII_NPHASES {
                         for &offset in [n, n + DQS_SIGNAL_COUNT].iter() {
-                            let addr = DFII_PIX_RDDATA_ADDR[p].offset(offset as isize);
+                            let addr = DFII_PIX_RDDATA_ADDR[p].offset(CSR_SEPARATION * offset as isize);
                             let data = prs[DFII_PIX_DATA_SIZE * p + offset];
                             if ptr::read_volatile(addr) as u8 != data {
                                 working = false;
@@ -286,7 +289,7 @@ mod ddr {
         spin_cycles(15);
     }
 
-    unsafe fn read_level(logger: &mut Option<&mut fmt::Write>) -> bool {
+    unsafe fn read_level(logger: &mut Option<&mut dyn fmt::Write>) -> bool {
         log!(logger, "Read leveling: ");
 
         // Generate pseudo-random sequence
@@ -306,7 +309,7 @@ mod ddr {
         // Write test pattern
         for p in 0..DFII_NPHASES {
             for offset in 0..DFII_PIX_DATA_SIZE {
-                let addr = DFII_PIX_WRDATA_ADDR[p].offset(offset as isize);
+                let addr = DFII_PIX_WRDATA_ADDR[p].offset(CSR_SEPARATION * offset as isize);
                 let data = prs[DFII_PIX_DATA_SIZE * p + offset];
                 ptr::write_volatile(addr, data as u32);
             }
@@ -349,7 +352,7 @@ mod ddr {
 
                     for p in 0..DFII_NPHASES {
                         for &offset in [n, n + DQS_SIGNAL_COUNT].iter() {
-                            let addr = DFII_PIX_RDDATA_ADDR[p].offset(offset as isize);
+                            let addr = DFII_PIX_RDDATA_ADDR[p].offset(CSR_SEPARATION * offset as isize);
                             let data = prs[DFII_PIX_DATA_SIZE * p + offset];
                             if ptr::read_volatile(addr) as u8 != data {
                                 valid = false;
@@ -417,7 +420,7 @@ mod ddr {
         true
     }
 
-    pub unsafe fn level(logger: &mut Option<&mut fmt::Write>) -> bool {
+    pub unsafe fn level(logger: &mut Option<&mut dyn fmt::Write>) -> bool {
         #[cfg(ddrphy_wlevel)]
         {
             let mut delay = [0; DQS_SIGNAL_COUNT];
@@ -442,7 +445,7 @@ use core::fmt;
 use csr;
 use sdram_phy;
 
-pub unsafe fn init(mut _logger: Option<&mut fmt::Write>) -> bool {
+pub unsafe fn init(mut _logger: Option<&mut dyn fmt::Write>) -> bool {
     sdram_phy::initialize();
 
     #[cfg(has_ddrphy)]

artiq/firmware/libboard_misoc/spiflash.rs

@@ -113,3 +113,9 @@ pub unsafe fn write(mut addr: usize, mut data: &[u8]) {
         data  = &data[size..];
     }
 }
+
+#[cfg(any(soc_platform = "kasli", soc_platform = "metlino", soc_platform = "kc705"))]
+pub unsafe fn reload () -> ! {
+    csr::icap::iprog_write(1);
+    loop {}
+}

artiq/firmware/libc/Cargo.toml

@@ -0,0 +1,5 @@
+[package]
+name = "libc"
+version = "0.1.0"
+authors = ["M-Labs"]
+edition = "2018"

artiq/firmware/libc/src/lib.rs

@@ -0,0 +1,8 @@
+// Helper crate for dealing with c ffi
+#![allow(non_camel_case_types)]
+#![no_std]
+pub type c_char = i8;
+pub type c_int = i32;
+pub type size_t = usize;
+pub type uintptr_t = usize;
+pub type c_void = core::ffi::c_void;

artiq/firmware/libdyld/elf.rs

@@ -131,6 +131,7 @@ pub const EM_TILEPRO: u16 = 188;
 pub const EM_MICROBLAZE: u16 = 189;
 pub const EM_TILEGX: u16 = 191;
 pub const EM_NUM: u16 = 192;
+pub const EM_RISCV: u16 = 243;
 pub const EM_ALPHA: u16 = 36902;
 pub const EV_NONE: u8 = 0;
 pub const EV_CURRENT: u8 = 1;
@@ -2229,6 +2230,67 @@ pub const R_OR1K_TLS_TPOFF: u8 = 32;
 pub const R_OR1K_TLS_DTPOFF: u8 = 33;
 pub const R_OR1K_TLS_DTPMOD: u8 = 34;
 pub const R_OR1K_NUM: u8 = 35;
+pub const EF_RISCV_RVC: u32 = 1;
+pub const EF_RISCV_FLOAT_ABI: u32 = 6;
+pub const EF_RISCV_FLOAT_ABI_SOFT: u32 = 0;
+pub const EF_RISCV_FLOAT_ABI_SINGLE: u32 = 2;
+pub const EF_RISCV_FLOAT_ABI_DOUBLE: u32 = 4;
+pub const EF_RISCV_FLOAT_ABI_QUAD: u32 = 6;
+pub const R_RISCV_NONE: u8 = 0;
+pub const R_RISCV_32: u8 = 1;
+pub const R_RISCV_64: u8 = 2;
+pub const R_RISCV_RELATIVE: u8 = 3;
+pub const R_RISCV_COPY: u8 = 4;
+pub const R_RISCV_JUMP_SLOT: u8 = 5;
+pub const R_RISCV_TLS_DTPMOD32: u8 = 6;
+pub const R_RISCV_TLS_DTPMOD64: u8 = 7;
+pub const R_RISCV_TLS_DTPREL32: u8 = 8;
+pub const R_RISCV_TLS_DTPREL64: u8 = 9;
+pub const R_RISCV_TLS_TPREL32: u8 = 10;
+pub const R_RISCV_TLS_TPREL64: u8 = 11;
+pub const R_RISCV_BRANCH: u8 = 16;
+pub const R_RISCV_JAL: u8 = 17;
+pub const R_RISCV_CALL: u8 = 18;
+pub const R_RISCV_CALL_PLT: u8 = 19;
+pub const R_RISCV_GOT_HI20: u8 = 20;
+pub const R_RISCV_TLS_GOT_HI20: u8 = 21;
+pub const R_RISCV_TLS_GD_HI20: u8 = 22;
+pub const R_RISCV_PCREL_HI20: u8 = 23;
+pub const R_RISCV_PCREL_LO12_I: u8 = 24;
+pub const R_RISCV_PCREL_LO12_S: u8 = 25;
+pub const R_RISCV_HI20: u8 = 26;
+pub const R_RISCV_LO12_I: u8 = 27;
+pub const R_RISCV_LO12_S: u8 = 28;
+pub const R_RISCV_TPREL_HI20: u8 = 29;
+pub const R_RISCV_TPREL_LO12_I: u8 = 30;
+pub const R_RISCV_TPREL_LO12_S: u8 = 31;
+pub const R_RISCV_TPREL_ADD: u8 = 32;
+pub const R_RISCV_ADD8: u8 = 33;
+pub const R_RISCV_ADD16: u8 = 34;
+pub const R_RISCV_ADD32: u8 = 35;
+pub const R_RISCV_ADD64: u8 = 36;
+pub const R_RISCV_SUB8: u8 = 37;
+pub const R_RISCV_SUB16: u8 = 38;
+pub const R_RISCV_SUB32: u8 = 39;
+pub const R_RISCV_SUB64: u8 = 40;
+pub const R_RISCV_GNU_VTINHERIT: u8 = 41;
+pub const R_RISCV_GNU_VTENTRY: u8 = 42;
+pub const R_RISCV_ALIGN: u8 = 43;
+pub const R_RISCV_RVC_BRANCH: u8 = 44;
+pub const R_RISCV_RVC_JUMP: u8 = 45;
+pub const R_RISCV_RVC_LUI: u8 = 46;
+pub const R_RISCV_GPREL_I: u8 = 47;
+pub const R_RISCV_GPREL_S: u8 = 48;
+pub const R_RISCV_TPREL_I: u8 = 49;
+pub const R_RISCV_TPREL_S: u8 = 50;
+pub const R_RISCV_RELAX: u8 = 51;
+pub const R_RISCV_SUB6: u8 = 52;
+pub const R_RISCV_SET6: u8 = 53;
+pub const R_RISCV_SET8: u8 = 54;
+pub const R_RISCV_SET16: u8 = 55;
+pub const R_RISCV_SET32: u8 = 56;
+pub const R_RISCV_32_PCREL: u8 = 57;
+pub const R_RISCV_NUM: u8 = 58;
 
 pub type Elf32_Half = u16;
 pub type Elf64_Half = u16;

artiq/firmware/libdyld/lib.rs

@@ -75,6 +75,12 @@ impl<'a> fmt::Display for Error<'a> {
     }
 }
 
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Arch {
+    RiscV,
+    OpenRisc,
+}
+
 pub struct Library<'a> {
     image_off:   Elf32_Addr,
     image_sz:    usize,
@@ -134,7 +140,7 @@ impl<'a> Library<'a> {
     pub unsafe fn rebind(&self, name: &[u8], addr: Elf32_Word) -> Result<(), Error<'a>> {
         for rela in self.pltrel.iter() {
             match ELF32_R_TYPE(rela.r_info) {
-                R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT => {
+                R_RISCV_32 | R_RISCV_JUMP_SLOT => {
                     let sym = self.symtab.get(ELF32_R_SYM(rela.r_info) as usize)
                                          .ok_or("symbol out of bounds of symbol table")?;
                     let sym_name = self.name_starting_at(sym.st_name as usize)?;
@@ -151,7 +157,7 @@ impl<'a> Library<'a> {
         Ok(())
     }
 
-    fn resolve_rela(&self, rela: &Elf32_Rela, resolve: &Fn(&[u8]) -> Option<Elf32_Word>)
+    fn resolve_rela(&self, rela: &Elf32_Rela, resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>)
             -> Result<(), Error<'a>> {
         let sym;
         if ELF32_R_SYM(rela.r_info) == 0 {
@@ -163,13 +169,13 @@ impl<'a> Library<'a> {
 
         let value;
         match ELF32_R_TYPE(rela.r_info) {
-            R_OR1K_NONE =>
+            R_RISCV_NONE =>
                 return Ok(()),
 
-            R_OR1K_RELATIVE =>
+            R_RISCV_RELATIVE =>
                 value = self.image_off + rela.r_addend as Elf32_Word,
 
-            R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT => {
+            R_RISCV_32 | R_RISCV_JUMP_SLOT => {
                 let sym = sym.ok_or("relocation requires an associated symbol")?;
                 let sym_name = self.name_starting_at(sym.st_name as usize)?;
 
@@ -195,7 +201,7 @@ impl<'a> Library<'a> {
         self.update_rela(rela, value)
     }
 
-    pub fn load(data: &[u8], image: &'a mut [u8], resolve: &Fn(&[u8]) -> Option<Elf32_Word>)
+    pub fn load(data: &[u8], image: &'a mut [u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>)
             -> Result<Library<'a>, Error<'a>> {
         #![allow(unused_assignments)]
 
@@ -204,16 +210,22 @@ impl<'a> Library<'a> {
 
         const IDENT: [u8; EI_NIDENT] = [
             ELFMAG0,    ELFMAG1,     ELFMAG2,    ELFMAG3,
-            ELFCLASS32, ELFDATA2MSB, EV_CURRENT, ELFOSABI_NONE,
+            ELFCLASS32, ELFDATA2LSB, EV_CURRENT, ELFOSABI_NONE,
             /* ABI version */ 0, /* padding */ 0, 0, 0, 0, 0, 0, 0
         ];
 
-        #[cfg(target_arch = "or1k")]
+        #[cfg(target_arch = "riscv32")]
-        const ARCH: u16 = EM_OPENRISC;
+        const ARCH: u16 = EM_RISCV;
-        #[cfg(not(target_arch = "or1k"))]
+        #[cfg(not(target_arch = "riscv32"))]
         const ARCH: u16 = EM_NONE;
 
-        if ehdr.e_ident != IDENT || ehdr.e_type != ET_DYN || ehdr.e_machine != ARCH {
+        #[cfg(all(target_feature = "f", target_feature = "d"))]
+        const FLAGS: u32 = EF_RISCV_FLOAT_ABI_DOUBLE;
+
+        #[cfg(not(all(target_feature = "f", target_feature = "d")))]
+        const FLAGS: u32 = EF_RISCV_FLOAT_ABI_SOFT;
+
+        if ehdr.e_ident != IDENT || ehdr.e_type != ET_DYN || ehdr.e_machine != ARCH || ehdr.e_flags != FLAGS {
             return Err("not a shared library for current architecture")?
         }
 

artiq/firmware/libeh/Cargo.toml

@@ -9,3 +9,6 @@ path = "lib.rs"
 
 [dependencies]
 cslice = { version = "0.3" }
+libc = { path = "../libc" }
+unwind = { path = "../libunwind" }
+compiler_builtins = "=0.1.39"    # A dependency of alloc, libeh doesn't need it

artiq/firmware/libeh/dwarf.rs

@@ -1,51 +1,71 @@
-#![allow(non_upper_case_globals, dead_code)]
+//! Parsing of GCC-style Language-Specific Data Area (LSDA)
+//! For details see:
+//!  * <https://refspecs.linuxfoundation.org/LSB_3.0.0/LSB-PDA/LSB-PDA/ehframechpt.html>
+//!  * <https://itanium-cxx-abi.github.io/cxx-abi/exceptions.pdf>
+//!  * <https://www.airs.com/blog/archives/460>
+//!  * <https://www.airs.com/blog/archives/464>
+//!
+//! A reference implementation may be found in the GCC source tree
+//! (`<root>/libgcc/unwind-c.c` as of this writing).
 
-use core::{ptr, mem};
+#![allow(non_upper_case_globals)]
-use cslice::CSlice;
+#![allow(unused)]
 
-const DW_EH_PE_omit: u8 = 0xFF;
+use core::mem;
-const DW_EH_PE_absptr: u8 = 0x00;
 
-const DW_EH_PE_uleb128: u8 = 0x01;
+pub const DW_EH_PE_omit: u8 = 0xFF;
-const DW_EH_PE_udata2: u8 = 0x02;
+pub const DW_EH_PE_absptr: u8 = 0x00;
-const DW_EH_PE_udata4: u8 = 0x03;
-const DW_EH_PE_udata8: u8 = 0x04;
-const DW_EH_PE_sleb128: u8 = 0x09;
-const DW_EH_PE_sdata2: u8 = 0x0A;
-const DW_EH_PE_sdata4: u8 = 0x0B;
-const DW_EH_PE_sdata8: u8 = 0x0C;
 
-const DW_EH_PE_pcrel: u8 = 0x10;
+pub const DW_EH_PE_uleb128: u8 = 0x01;
-const DW_EH_PE_textrel: u8 = 0x20;
+pub const DW_EH_PE_udata2: u8 = 0x02;
-const DW_EH_PE_datarel: u8 = 0x30;
+pub const DW_EH_PE_udata4: u8 = 0x03;
-const DW_EH_PE_funcrel: u8 = 0x40;
+pub const DW_EH_PE_udata8: u8 = 0x04;
-const DW_EH_PE_aligned: u8 = 0x50;
+pub const DW_EH_PE_sleb128: u8 = 0x09;
+pub const DW_EH_PE_sdata2: u8 = 0x0A;
+pub const DW_EH_PE_sdata4: u8 = 0x0B;
+pub const DW_EH_PE_sdata8: u8 = 0x0C;
 
-const DW_EH_PE_indirect: u8 = 0x80;
+pub const DW_EH_PE_pcrel: u8 = 0x10;
+pub const DW_EH_PE_textrel: u8 = 0x20;
+pub const DW_EH_PE_datarel: u8 = 0x30;
+pub const DW_EH_PE_funcrel: u8 = 0x40;
+pub const DW_EH_PE_aligned: u8 = 0x50;
 
-#[derive(Clone)]
+pub const DW_EH_PE_indirect: u8 = 0x80;
-struct DwarfReader {
+
+#[derive(Copy, Clone)]
+pub struct EHContext<'a> {
+    pub ip: usize,                             // Current instruction pointer
+    pub func_start: usize,                     // Address of the current function
+    pub get_text_start: &'a dyn Fn() -> usize, // Get address of the code section
+    pub get_data_start: &'a dyn Fn() -> usize, // Get address of the data section
+}
+
+pub struct DwarfReader {
     pub ptr: *const u8,
 }
 
+#[repr(C, packed)]
+struct Unaligned<T>(T);
+
 impl DwarfReader {
-    fn new(ptr: *const u8) -> DwarfReader {
+    pub fn new(ptr: *const u8) -> DwarfReader {
-        DwarfReader { ptr: ptr }
+        DwarfReader { ptr }
     }
 
-    // DWARF streams are packed, so e.g. a u32 would not necessarily be aligned
+    // DWARF streams are packed, so e.g., a u32 would not necessarily be aligned
     // on a 4-byte boundary. This may cause problems on platforms with strict
     // alignment requirements. By wrapping data in a "packed" struct, we are
     // telling the backend to generate "misalignment-safe" code.
-    unsafe fn read<T: Copy>(&mut self) -> T {
+    pub unsafe fn read<T: Copy>(&mut self) -> T {
-        let result = ptr::read_unaligned(self.ptr as *const T);
+        let Unaligned(result) = *(self.ptr as *const Unaligned<T>);
-        self.ptr = self.ptr.offset(mem::size_of::<T>() as isize);
+        self.ptr = self.ptr.add(mem::size_of::<T>());
         result
     }
 
     // ULEB128 and SLEB128 encodings are defined in Section 7.6 - "Variable
     // Length Data".
-    unsafe fn read_uleb128(&mut self) -> u64 {
+    pub unsafe fn read_uleb128(&mut self) -> u64 {
         let mut shift: usize = 0;
         let mut result: u64 = 0;
         let mut byte: u8;
@@ -60,8 +80,8 @@ impl DwarfReader {
         result
     }
 
-    unsafe fn read_sleb128(&mut self) -> i64 {
+    pub unsafe fn read_sleb128(&mut self) -> i64 {
-        let mut shift: usize = 0;
+        let mut shift: u32 = 0;
         let mut result: u64 = 0;
         let mut byte: u8;
         loop {
@@ -73,70 +93,61 @@ impl DwarfReader {
             }
         }
         // sign-extend
-        if shift < 8 * mem::size_of::<u64>() && (byte & 0x40) != 0 {
+        if shift < u64::BITS && (byte & 0x40) != 0 {
             result |= (!0 as u64) << shift;
         }
         result as i64
     }
-
-    unsafe fn read_encoded_pointer(&mut self, encoding: u8) -> usize {
-        fn round_up(unrounded: usize, align: usize) -> usize {
-            debug_assert!(align.is_power_of_two());
-            (unrounded + align - 1) & !(align - 1)
-        }
-
-        debug_assert!(encoding != DW_EH_PE_omit);
-
-        // DW_EH_PE_aligned implies it's an absolute pointer value
-        if encoding == DW_EH_PE_aligned {
-            self.ptr = round_up(self.ptr as usize, mem::size_of::<usize>()) as *const u8;
-            return self.read::<usize>()
-        }
-
-        let value_ptr = self.ptr;
-        let mut result = match encoding & 0x0F {
-            DW_EH_PE_absptr => self.read::<usize>(),
-            DW_EH_PE_uleb128 => self.read_uleb128() as usize,
-            DW_EH_PE_udata2 => self.read::<u16>() as usize,
-            DW_EH_PE_udata4 => self.read::<u32>() as usize,
-            DW_EH_PE_udata8 => self.read::<u64>() as usize,
-            DW_EH_PE_sleb128 => self.read_sleb128() as usize,
-            DW_EH_PE_sdata2 => self.read::<i16>() as usize,
-            DW_EH_PE_sdata4 => self.read::<i32>() as usize,
-            DW_EH_PE_sdata8 => self.read::<i64>() as usize,
-            _ => panic!(),
-        };
-
-        result += match encoding & 0x70 {
-            DW_EH_PE_absptr => 0,
-            // relative to address of the encoded value, despite the name
-            DW_EH_PE_pcrel => value_ptr as usize,
-            _ => panic!(),
-        };
-
-        if encoding & DW_EH_PE_indirect != 0 {
-            result = *(result as *const usize);
-        }
-
-        result
-    }
 }
 
-fn encoding_size(encoding: u8) -> usize {
+unsafe fn read_encoded_pointer(
+    reader: &mut DwarfReader,
+    context: &EHContext<'_>,
+    encoding: u8,
+) -> Result<usize, ()> {
     if encoding == DW_EH_PE_omit {
-        return 0
+        return Err(());
     }
 
-    match encoding & 0x0F {
+    // DW_EH_PE_aligned implies it's an absolute pointer value
-        DW_EH_PE_absptr => mem::size_of::<usize>(),
+    if encoding == DW_EH_PE_aligned {
-        DW_EH_PE_udata2 => 2,
+        reader.ptr = round_up(reader.ptr as usize, mem::size_of::<usize>())? as *const u8;
-        DW_EH_PE_udata4 => 4,
+        return Ok(reader.read::<usize>());
-        DW_EH_PE_udata8 => 8,
-        DW_EH_PE_sdata2 => 2,
-        DW_EH_PE_sdata4 => 4,
-        DW_EH_PE_sdata8 => 8,
-        _ => panic!()
     }
+
+    let mut result = match encoding & 0x0F {
+        DW_EH_PE_absptr => reader.read::<usize>(),
+        DW_EH_PE_uleb128 => reader.read_uleb128() as usize,
+        DW_EH_PE_udata2 => reader.read::<u16>() as usize,
+        DW_EH_PE_udata4 => reader.read::<u32>() as usize,
+        DW_EH_PE_udata8 => reader.read::<u64>() as usize,
+        DW_EH_PE_sleb128 => reader.read_sleb128() as usize,
+        DW_EH_PE_sdata2 => reader.read::<i16>() as usize,
+        DW_EH_PE_sdata4 => reader.read::<i32>() as usize,
+        DW_EH_PE_sdata8 => reader.read::<i64>() as usize,
+        _ => return Err(()),
+    };
+
+    result += match encoding & 0x70 {
+        DW_EH_PE_absptr => 0,
+        // relative to address of the encoded value, despite the name
+        DW_EH_PE_pcrel => reader.ptr as usize,
+        DW_EH_PE_funcrel => {
+            if context.func_start == 0 {
+                return Err(());
+            }
+            context.func_start
+        }
+        DW_EH_PE_textrel => (*context.get_text_start)(),
+        DW_EH_PE_datarel => (*context.get_data_start)(),
+        _ => return Err(()),
+    };
+
+    if encoding & DW_EH_PE_indirect != 0 {
+        result = *(result as *const usize);
+    }
+
+    Ok(result)
 }
 
 pub enum EHAction {
@@ -146,98 +157,94 @@ pub enum EHAction {
     Terminate,
 }
 
-pub unsafe fn find_eh_action(lsda: *const u8, func_start: usize, ip: usize,
+pub const USING_SJLJ_EXCEPTIONS: bool = cfg!(all(target_os = "ios", target_arch = "arm"));
-                             exn_name: CSlice<u8>) -> EHAction {
+
+pub unsafe fn find_eh_action(lsda: *const u8, context: &EHContext<'_>) -> Result<EHAction, ()> {
     if lsda.is_null() {
-        return EHAction::None
+        return Ok(EHAction::None);
     }
 
+    let func_start = context.func_start;
     let mut reader = DwarfReader::new(lsda);
 
     let start_encoding = reader.read::<u8>();
     // base address for landing pad offsets
     let lpad_base = if start_encoding != DW_EH_PE_omit {
-        reader.read_encoded_pointer(start_encoding)
+        read_encoded_pointer(&mut reader, context, start_encoding)?
     } else {
         func_start
     };
 
     let ttype_encoding = reader.read::<u8>();
-    let ttype_encoding_size = encoding_size(ttype_encoding) as isize;
-
-    let class_info;
     if ttype_encoding != DW_EH_PE_omit {
-        let class_info_offset = reader.read_uleb128();
+        // Rust doesn't analyze exception types, so we don't care about the type table
-        class_info = reader.ptr.offset(class_info_offset as isize);
+        reader.read_uleb128();
-    } else {
-        class_info = ptr::null();
     }
-    assert!(!class_info.is_null());
 
     let call_site_encoding = reader.read::<u8>();
     let call_site_table_length = reader.read_uleb128();
     let action_table = reader.ptr.offset(call_site_table_length as isize);
+    let ip = context.ip;
 
-    while reader.ptr < action_table {
+    if !USING_SJLJ_EXCEPTIONS {
-        let cs_start = reader.read_encoded_pointer(call_site_encoding);
+        while reader.ptr < action_table {
-        let cs_len = reader.read_encoded_pointer(call_site_encoding);
+            let cs_start = read_encoded_pointer(&mut reader, context, call_site_encoding)?;
-        let cs_lpad = reader.read_encoded_pointer(call_site_encoding);
+            let cs_len = read_encoded_pointer(&mut reader, context, call_site_encoding)?;
-        let cs_action = reader.read_uleb128();
+            let cs_lpad = read_encoded_pointer(&mut reader, context, call_site_encoding)?;
-
+            let cs_action = reader.read_uleb128();
-        if ip < func_start + cs_start {
             // Callsite table is sorted by cs_start, so if we've passed the ip, we
             // may stop searching.
-            break
+            if ip < func_start + cs_start {
+                break;
+            }
+            if ip < func_start + cs_start + cs_len {
+                if cs_lpad == 0 {
+                    return Ok(EHAction::None);
+                } else {
+                    let lpad = lpad_base + cs_lpad;
+                    return Ok(interpret_cs_action(cs_action, lpad));
+                }
+            }
         }
-        if ip > func_start + cs_start + cs_len {
+        // Ip is not present in the table.  This should not happen... but it does: issue #35011.
-            continue
+        // So rather than returning EHAction::Terminate, we do this.
+        Ok(EHAction::None)
+    } else {
+        // SjLj version:
+        // The "IP" is an index into the call-site table, with two exceptions:
+        // -1 means 'no-action', and 0 means 'terminate'.
+        match ip as isize {
+            -1 => return Ok(EHAction::None),
+            0 => return Ok(EHAction::Terminate),
+            _ => (),
         }
-
+        let mut idx = ip;
-        if cs_lpad == 0 {
-            return EHAction::None
-        }
-
-        let lpad = lpad_base + cs_lpad;
-        if cs_action == 0 {
-            return EHAction::Cleanup(lpad)
-        }
-
-        let action_entry = action_table.offset((cs_action - 1) as isize);
-        let mut action_reader = DwarfReader::new(action_entry);
         loop {
-            let type_info_offset = action_reader.read_sleb128() as isize;
+            let cs_lpad = reader.read_uleb128();
-            let action_offset = action_reader.clone().read_sleb128() as isize;
+            let cs_action = reader.read_uleb128();
-            assert!(type_info_offset >= 0);
+            idx -= 1;
-
+            if idx == 0 {
-            if type_info_offset > 0 {
+                // Can never have null landing pad for sjlj -- that would have
-                let type_info_ptr_ptr = class_info.offset(-type_info_offset * ttype_encoding_size);
+                // been indicated by a -1 call site index.
-                let type_info_ptr = DwarfReader::new(type_info_ptr_ptr)
+                let lpad = (cs_lpad + 1) as usize;
-                                                .read_encoded_pointer(ttype_encoding);
+                return Ok(interpret_cs_action(cs_action, lpad));
-                let type_info = *(type_info_ptr as *const CSlice<u8>);
-
-                if type_info.as_ref() == exn_name.as_ref() {
-                    return EHAction::Catch(lpad)
-                }
-
-                if type_info.len() == 0 {
-                    // This is a catch-all clause. We don't compare type_info_ptr with null here
-                    // because, in PIC mode, the OR1K LLVM backend emits a literal zero
-                    // encoded with DW_EH_PE_pcrel, which of course doesn't result in
-                    // a proper null pointer.
-                    return EHAction::Catch(lpad)
-                }
-            }
-
-            if action_offset == 0 {
-                break
-            } else {
-                action_reader.ptr = action_reader.ptr.offset(action_offset)
             }
         }
-
-        return EHAction::None
     }
-
+}
-    // the function has a personality but no landing pads; this is fine
+
-    EHAction::None
+fn interpret_cs_action(cs_action: u64, lpad: usize) -> EHAction {
+    if cs_action == 0 {
+        // If cs_action is 0 then this is a cleanup (Drop::drop). We run these
+        // for both Rust panics and foreign exceptions.
+        EHAction::Cleanup(lpad)
+    } else {
+        // Stop unwinding Rust panics at catch_unwind.
+        EHAction::Catch(lpad)
+    }
+}
+
+#[inline]
+fn round_up(unrounded: usize, align: usize) -> Result<usize, ()> {
+    if align.is_power_of_two() { Ok((unrounded + align - 1) & !(align - 1)) } else { Err(()) }
 }

artiq/firmware/libeh/eh_rust.rs

@@ -14,13 +14,10 @@
 //
 // By design, this personality function is only ever called in the search phase, although
 // to keep things simple and close to upstream, it is not modified
-#![allow(private_no_mangle_fns)]
-
 use unwind as uw;
 use libc::{c_int, uintptr_t};
-use cslice::AsCSlice;
 
-use dwarf::{self, EHAction};
+use dwarf::{self, EHAction, EHContext};
 
 // Register ids were lifted from LLVM's TargetLowering::getExceptionPointerRegister()
 // and TargetLowering::getExceptionSelectorRegister() for each architecture,
@@ -34,8 +31,8 @@ const UNWIND_DATA_REG: (i32, i32) = (0, 2); // EAX, EDX
 #[cfg(target_arch = "x86_64")]
 const UNWIND_DATA_REG: (i32, i32) = (0, 1); // RAX, RDX
 
-#[cfg(any(target_arch = "or1k"))]
+#[cfg(any(target_arch = "riscv32"))]
-const UNWIND_DATA_REG: (i32, i32) = (3, 4); // R3, R4
+const UNWIND_DATA_REG: (i32, i32) = (10, 11); // X10, X11
 
 // The following code is based on GCC's C and C++ personality routines.  For reference, see:
 // https://github.com/gcc-mirror/gcc/blob/master/libstdc++-v3/libsupc++/eh_personality.cc
@@ -78,11 +75,17 @@ unsafe extern "C" fn rust_eh_personality(version: c_int,
     }
 }
 
-unsafe fn find_eh_action(context: *mut uw::_Unwind_Context)
+unsafe fn find_eh_action(context: *mut uw::_Unwind_Context) -> Result<EHAction, ()> {
-    -> Result<EHAction, ()>
-{
     let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8;
-    let func = uw::_Unwind_GetRegionStart(context);
+    let mut ip_before_instr: c_int = 0;
-    let ip   = uw::_Unwind_GetIP(context);
+    let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr);
-    Ok(dwarf::find_eh_action(lsda, func, ip, [].as_c_slice()))
+    let eh_context = EHContext {
+        // The return address points 1 byte past the call instruction,
+        // which could be in the next IP range in LSDA range table.
+        ip: if ip_before_instr != 0 { ip } else { ip - 1 },
+        func_start: uw::_Unwind_GetRegionStart(context),
+        get_text_start: &|| uw::_Unwind_GetTextRelBase(context),
+        get_data_start: &|| uw::_Unwind_GetDataRelBase(context),
+    };
+    dwarf::find_eh_action(lsda, &eh_context)
 }

artiq/firmware/libeh/lib.rs

@@ -1,4 +1,4 @@
-#![feature(lang_items, panic_unwind, libc, unwind_attributes)]
+#![feature(lang_items, panic_unwind, libc, unwind_attributes, int_bits_const)]
 #![no_std]
 
 extern crate cslice;

artiq/firmware/libio/Cargo.toml

@@ -8,8 +8,8 @@ name = "io"
 path = "lib.rs"
 
 [dependencies]
-failure = { version = "0.1", default-features = false }
+failure = { version = "=0.1.1", default-features = false }
-failure_derive = { version = "0.1", default-features = false }
+failure_derive = { version = "=0.1.1", default-features = false }
 byteorder = { version = "1.0", default-features = false, optional = true }
 
 [features]

artiq/firmware/libio/cursor.rs

@@ -69,7 +69,7 @@ impl<'a> Write for Cursor<&'a mut [u8]> {
 }
 
 #[cfg(feature = "alloc")]
-impl Write for Cursor<::alloc::Vec<u8>> {
+impl Write for Cursor<::alloc::vec::Vec<u8>> {
     type WriteError = !;
     type FlushError = !;
 

artiq/firmware/libio/lib.rs

@@ -1,6 +1,5 @@
 #![no_std]
 #![feature(never_type)]
-#![cfg_attr(feature = "alloc", feature(alloc))]
 
 extern crate failure;
 #[macro_use]
@@ -131,7 +130,7 @@ impl<'a> Write for &'a mut [u8] {
 }
 
 #[cfg(feature = "alloc")]
-impl<'a> Write for alloc::Vec<u8> {
+impl<'a> Write for alloc::vec::Vec<u8> {
     type WriteError = !;
     type FlushError = !;
 

artiq/firmware/libio/proto.rs

@@ -1,6 +1,6 @@
 #[cfg(feature = "alloc")]
-use {core::str::Utf8Error, alloc::String};
+use {core::str::Utf8Error, alloc::string::String};
-use byteorder::{ByteOrder, NetworkEndian};
+use byteorder::{ByteOrder, NativeEndian};
 
 use ::{Read, Write, Error as IoError};
 
@@ -29,21 +29,21 @@ pub trait ProtoRead {
     fn read_u16(&mut self) -> Result<u16, Self::ReadError> {
         let mut bytes = [0; 2];
         self.read_exact(&mut bytes)?;
-        Ok(NetworkEndian::read_u16(&bytes))
+        Ok(NativeEndian::read_u16(&bytes))
     }
 
     #[inline]
     fn read_u32(&mut self) -> Result<u32, Self::ReadError> {
         let mut bytes = [0; 4];
         self.read_exact(&mut bytes)?;
-        Ok(NetworkEndian::read_u32(&bytes))
+        Ok(NativeEndian::read_u32(&bytes))
     }
 
     #[inline]
     fn read_u64(&mut self) -> Result<u64, Self::ReadError> {
         let mut bytes = [0; 8];
         self.read_exact(&mut bytes)?;
-        Ok(NetworkEndian::read_u64(&bytes))
+        Ok(NativeEndian::read_u64(&bytes))
     }
 
     #[inline]
@@ -53,7 +53,7 @@ pub trait ProtoRead {
 
     #[cfg(feature = "alloc")]
     #[inline]
-    fn read_bytes(&mut self) -> Result<::alloc::Vec<u8>, Self::ReadError> {
+    fn read_bytes(&mut self) -> Result<::alloc::vec::Vec<u8>, Self::ReadError> {
         let length = self.read_u32()?;
         let mut value = vec![0; length as usize];
         self.read_exact(&mut value)?;
@@ -62,7 +62,7 @@ pub trait ProtoRead {
 
     #[cfg(feature = "alloc")]
     #[inline]
-    fn read_string(&mut self) -> Result<::alloc::String, ReadStringError<Self::ReadError>> {
+    fn read_string(&mut self) -> Result<::alloc::string::String, ReadStringError<Self::ReadError>> {
         let bytes = self.read_bytes().map_err(ReadStringError::Other)?;
         String::from_utf8(bytes).map_err(|err| ReadStringError::Utf8(err.utf8_error()))
     }
@@ -88,42 +88,42 @@ pub trait ProtoWrite {
     #[inline]
     fn write_u16(&mut self, value: u16) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 2];
-        NetworkEndian::write_u16(&mut bytes, value);
+        NativeEndian::write_u16(&mut bytes, value);
         self.write_all(&bytes)
     }
 
     #[inline]
     fn write_i16(&mut self, value: i16) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 2];
-        NetworkEndian::write_i16(&mut bytes, value);
+        NativeEndian::write_i16(&mut bytes, value);
         self.write_all(&bytes)
     }
 
     #[inline]
     fn write_u32(&mut self, value: u32) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 4];
-        NetworkEndian::write_u32(&mut bytes, value);
+        NativeEndian::write_u32(&mut bytes, value);
         self.write_all(&bytes)
     }
 
     #[inline]
     fn write_i32(&mut self, value: i32) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 4];
-        NetworkEndian::write_i32(&mut bytes, value);
+        NativeEndian::write_i32(&mut bytes, value);
         self.write_all(&bytes)
     }
 
     #[inline]
     fn write_u64(&mut self, value: u64) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 8];
-        NetworkEndian::write_u64(&mut bytes, value);
+        NativeEndian::write_u64(&mut bytes, value);
         self.write_all(&bytes)
     }
 
     #[inline]
     fn write_i64(&mut self, value: i64) -> Result<(), Self::WriteError> {
         let mut bytes = [0; 8];
-        NetworkEndian::write_i64(&mut bytes, value);
+        NativeEndian::write_i64(&mut bytes, value);
         self.write_all(&bytes)
     }
 

artiq/firmware/libproto_artiq/lib.rs

@@ -1,5 +1,4 @@
 #![no_std]
-#![cfg_attr(feature = "alloc", feature(alloc))]
 
 extern crate failure;
 #[macro_use]

artiq/firmware/libproto_artiq/mgmt_proto.rs

@@ -1,5 +1,5 @@
 use core::str::Utf8Error;
-use alloc::{Vec, String};
+use alloc::{vec::Vec, string::String};
 #[cfg(feature = "log")]
 use log;
 
@@ -63,15 +63,6 @@ pub enum Request {
     ConfigRemove { key: String },
     ConfigErase,
 
-    StartProfiler {
-        interval_us: u32,
-        hits_size: u32,
-        edges_size: u32,
-    },
-    StopProfiler,
-    GetProfile,
-
-    Hotswap(Vec<u8>),
     Reboot,
 
     DebugAllocator,
@@ -86,8 +77,6 @@ pub enum Reply<'a> {
 
     ConfigData(&'a [u8]),
 
-    Profile,
-
     RebootImminent,
 }
 
@@ -130,15 +119,6 @@ impl Request {
             },
             15 => Request::ConfigErase,
 
-            9 => Request::StartProfiler {
-                interval_us: reader.read_u32()?,
-                hits_size: reader.read_u32()?,
-                edges_size: reader.read_u32()?,
-            },
-            10 => Request::StopProfiler,
-            11 => Request::GetProfile,
-
-            4 => Request::Hotswap(reader.read_bytes()?),
             5 => Request::Reboot,
 
             8 => Request::DebugAllocator,
@@ -174,11 +154,6 @@ impl<'a> Reply<'a> {
                 writer.write_bytes(bytes)?;
             },
 
-            Reply::Profile => {
-                writer.write_u8(5)?;
-                // profile data follows
-            }
-
             Reply::RebootImminent => {
                 writer.write_u8(3)?;
             }

artiq/firmware/libproto_artiq/rpc_proto.rs

@@ -1,19 +1,31 @@
 use core::str;
 use core::slice;
 use cslice::{CSlice, CMutSlice};
-use byteorder::{NetworkEndian, ByteOrder};
+use byteorder::{NativeEndian, ByteOrder};
 use io::{ProtoRead, Read, Write, ProtoWrite, Error};
 use self::tag::{Tag, TagIterator, split_tag};
 
+unsafe fn align_ptr<T>(ptr: *const ()) -> *const T {
+    let alignment = core::mem::align_of::<T>() as isize;
+    let fix = (alignment - (ptr as isize) % alignment) % alignment;
+    ((ptr as isize) + fix) as *const T
+}
+
+unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T {
+    let alignment = core::mem::align_of::<T>() as isize;
+    let fix = (alignment - (ptr as isize) % alignment) % alignment;
+    ((ptr as isize) + fix) as *mut T
+}
+
 unsafe fn recv_value<R, E>(reader: &mut R, tag: Tag, data: &mut *mut (),
-                           alloc: &Fn(usize) -> Result<*mut (), E>)
+                           alloc: &dyn Fn(usize) -> Result<*mut (), E>)
                           -> Result<(), E>
     where R: Read + ?Sized,
           E: From<Error<R::ReadError>>
 {
     macro_rules! consume_value {
         ($ty:ty, |$ptr:ident| $map:expr) => ({
-            let $ptr = (*data) as *mut $ty;
+            let $ptr = align_ptr_mut::<$ty>(*data) as *mut $ty;
             *data = $ptr.offset(1) as *mut ();
             $map
         })
@@ -51,7 +63,7 @@ unsafe fn recv_value<R, E>(reader: &mut R, tag: Tag, data: &mut *mut (),
         }
         Tag::List(it) => {
             #[repr(C)]
-            struct List { elements: *mut (), length: u32 };
+            struct List { elements: *mut (), length: u32 }
             consume_value!(List, |ptr| {
                 (*ptr).length = reader.read_u32()?;
                 let length = (*ptr).length as usize;
@@ -69,13 +81,13 @@ unsafe fn recv_value<R, E>(reader: &mut R, tag: Tag, data: &mut *mut (),
                         let dest = slice::from_raw_parts_mut(data as *mut u8, length * 4);
                         reader.read_exact(dest)?;
                         let dest = slice::from_raw_parts_mut(data as *mut i32, length);
-                        NetworkEndian::from_slice_i32(dest);
+                        NativeEndian::from_slice_i32(dest);
                     },
                     Tag::Int64 | Tag::Float64 => {
                         let dest = slice::from_raw_parts_mut(data as *mut u8, length * 8);
                         reader.read_exact(dest)?;
                         let dest = slice::from_raw_parts_mut(data as *mut i64, length);
-                        NetworkEndian::from_slice_i64(dest);
+                        NativeEndian::from_slice_i64(dest);
                     },
                     _ => {
                         for _ in 0..length {
@@ -109,13 +121,13 @@ unsafe fn recv_value<R, E>(reader: &mut R, tag: Tag, data: &mut *mut (),
                         let dest = slice::from_raw_parts_mut(data as *mut u8, length * 4);
                         reader.read_exact(dest)?;
                         let dest = slice::from_raw_parts_mut(data as *mut i32, length);
-                        NetworkEndian::from_slice_i32(dest);
+                        NativeEndian::from_slice_i32(dest);
                     },
                     Tag::Int64 | Tag::Float64 => {
                         let dest = slice::from_raw_parts_mut(data as *mut u8, length * 8);
                         reader.read_exact(dest)?;
                         let dest = slice::from_raw_parts_mut(data as *mut i64, length);
-                        NetworkEndian::from_slice_i64(dest);
+                        NativeEndian::from_slice_i64(dest);
                     },
                     _ => {
                         for _ in 0..length {
@@ -139,7 +151,7 @@ unsafe fn recv_value<R, E>(reader: &mut R, tag: Tag, data: &mut *mut (),
 }
 
 pub fn recv_return<R, E>(reader: &mut R, tag_bytes: &[u8], data: *mut (),
-                         alloc: &Fn(usize) -> Result<*mut (), E>)
+                         alloc: &dyn Fn(usize) -> Result<*mut (), E>)
                         -> Result<(), E>
     where R: Read + ?Sized,
           E: From<Error<R::ReadError>>
@@ -161,7 +173,7 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
 {
     macro_rules! consume_value {
         ($ty:ty, |$ptr:ident| $map:expr) => ({
-            let $ptr = (*data) as *const $ty;
+            let $ptr = align_ptr::<$ty>(*data);
             *data = $ptr.offset(1) as *const ();
             $map
         })
@@ -196,7 +208,7 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
         }
         Tag::List(it) => {
             #[repr(C)]
-            struct List { elements: *const (), length: u32 };
+            struct List { elements: *const (), length: u32 }
             consume_value!(List, |ptr| {
                 let length = (*ptr).length as usize;
                 writer.write_u32((*ptr).length)?;
@@ -204,8 +216,8 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
                 let mut data = (*ptr).elements;
                 writer.write_u8(tag.as_u8())?;
                 match tag {
-                    // we cannot use NetworkEndian::from_slice_i32 as the data is not mutable,
+                    // we cannot use NativeEndian::from_slice_i32 as the data is not mutable,
-                    // and that is not needed as the data is already in network endian
+                    // and that is not needed as the data is already in native endian
                     Tag::Bool => {
                         let slice = slice::from_raw_parts(data as *const u8, length);
                         writer.write_all(slice)?;
@@ -243,8 +255,8 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
                 let mut data = *buffer;
                 writer.write_u8(elt_tag.as_u8())?;
                 match elt_tag {
-                    // we cannot use NetworkEndian::from_slice_i32 as the data is not mutable,
+                    // we cannot use NativeEndian::from_slice_i32 as the data is not mutable,
-                    // and that is not needed as the data is already in network endian
+                    // and that is not needed as the data is already in native endian
                     Tag::Bool => {
                         let slice = slice::from_raw_parts(data as *const u8, length);
                         writer.write_all(slice)?;
@@ -275,7 +287,7 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
         }
         Tag::Keyword(it) => {
             #[repr(C)]
-            struct Keyword<'a> { name: CSlice<'a, u8> };
+            struct Keyword<'a> { name: CSlice<'a, u8> }
             consume_value!(Keyword, |ptr| {
                 writer.write_string(str::from_utf8((*ptr).name.as_ref()).unwrap())?;
                 let tag = it.clone().next().expect("truncated tag");
@@ -287,7 +299,7 @@ unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
         }
         Tag::Object => {
             #[repr(C)]
-            struct Object { id: u32 };
+            struct Object { id: u32 }
             consume_value!(*const Object, |ptr|
                 writer.write_u32((**ptr).id))
         }

artiq/firmware/libproto_artiq/session_proto.rs

@@ -1,5 +1,5 @@
 use core::str::Utf8Error;
-use alloc::{Vec, String};
+use alloc::{vec::Vec, string::String};
 
 use io::{Read, ProtoRead, Write, ProtoWrite, Error as IoError, ReadStringError};
 
@@ -90,7 +90,9 @@ pub enum Reply<'a> {
     LoadCompleted,
     LoadFailed(&'a str),
 
-    KernelFinished,
+    KernelFinished {
+        async_errors: u8
+    },
     KernelStartupFailed,
     KernelException {
         name:      &'a str,
@@ -100,7 +102,8 @@ pub enum Reply<'a> {
         line:      u32,
         column:    u32,
         function:  &'a str,
-        backtrace: &'a [usize]
+        backtrace: &'a [usize],
+        async_errors: u8
     },
 
     RpcRequest { async: bool },
@@ -160,14 +163,16 @@ impl<'a> Reply<'a> {
                 writer.write_string(reason)?;
             },
 
-            Reply::KernelFinished => {
+            Reply::KernelFinished { async_errors } => {
                 writer.write_u8(7)?;
+                writer.write_u8(async_errors)?;
             },
             Reply::KernelStartupFailed => {
                 writer.write_u8(8)?;
             },
             Reply::KernelException {
-                name, message, param, file, line, column, function, backtrace
+                name, message, param, file, line, column, function, backtrace,
+                async_errors
             } => {
                 writer.write_u8(9)?;
                 writer.write_string(name)?;
@@ -183,6 +188,7 @@ impl<'a> Reply<'a> {
                 for &addr in backtrace {
                     writer.write_u32(addr as u32)?
                 }
+                writer.write_u8(async_errors)?;
             },
 
             Reply::RpcRequest { async } => {

artiq/firmware/libunwind/Cargo.toml

@@ -0,0 +1,30 @@
+[package]
+authors = ["The Rust Project Developers"]
+name = "unwind"
+version = "0.0.0"
+license = "MIT OR Apache-2.0"
+repository = "https://github.com/rust-lang/rust.git"
+edition = "2018"
+include = [
+  '/libunwind/*',
+]
+
+[lib]
+test = false
+bench = false
+doc = false
+
+[dependencies]
+libc = { path = "../libc" }
+cfg-if = "0.1.8"
+
+[features]
+
+# Only applies for Linux and Fuchsia targets
+# Static link to the in-tree build of llvm libunwind
+llvm-libunwind = []
+
+# Only applies for Linux and Fuchsia targets
+# If crt-static is enabled, static link to `libunwind.a` provided by system
+# If crt-static is disabled, dynamic link to `libunwind.so` provided by system
+system-llvm-libunwind = []

artiq/firmware/libunwind/src/lib.rs

@@ -0,0 +1,11 @@
+#![no_std]
+#![unstable(feature = "panic_unwind", issue = "32837")]
+#![feature(link_cfg)]
+#![feature(nll)]
+#![feature(staged_api)]
+#![feature(unwind_attributes)]
+#![feature(static_nobundle)]
+#![cfg_attr(not(target_env = "msvc"), feature(libc))]
+
+mod libunwind;
+pub use libunwind::*;

artiq/firmware/libunwind/src/libunwind.rs

@@ -0,0 +1,282 @@
+#![allow(nonstandard_style)]
+
+use libc::{c_int, c_void, uintptr_t};
+
+#[repr(C)]
+#[derive(Debug, Copy, Clone, PartialEq)]
+pub enum _Unwind_Reason_Code {
+    _URC_NO_REASON = 0,
+    _URC_FOREIGN_EXCEPTION_CAUGHT = 1,
+    _URC_FATAL_PHASE2_ERROR = 2,
+    _URC_FATAL_PHASE1_ERROR = 3,
+    _URC_NORMAL_STOP = 4,
+    _URC_END_OF_STACK = 5,
+    _URC_HANDLER_FOUND = 6,
+    _URC_INSTALL_CONTEXT = 7,
+    _URC_CONTINUE_UNWIND = 8,
+    _URC_FAILURE = 9, // used only by ARM EHABI
+}
+pub use _Unwind_Reason_Code::*;
+
+pub type _Unwind_Exception_Class = u64;
+pub type _Unwind_Word = uintptr_t;
+pub type _Unwind_Ptr = uintptr_t;
+pub type _Unwind_Trace_Fn =
+    extern "C" fn(ctx: *mut _Unwind_Context, arg: *mut c_void) -> _Unwind_Reason_Code;
+
+#[cfg(target_arch = "x86")]
+pub const unwinder_private_data_size: usize = 5;
+
+#[cfg(target_arch = "x86_64")]
+pub const unwinder_private_data_size: usize = 6;
+
+#[cfg(all(target_arch = "arm", not(target_os = "ios")))]
+pub const unwinder_private_data_size: usize = 20;
+
+#[cfg(all(target_arch = "arm", target_os = "ios"))]
+pub const unwinder_private_data_size: usize = 5;
+
+#[cfg(all(target_arch = "aarch64", target_pointer_width = "64"))]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(all(target_arch = "aarch64", target_pointer_width = "32"))]
+pub const unwinder_private_data_size: usize = 5;
+
+#[cfg(target_arch = "mips")]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(target_arch = "mips64")]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(any(target_arch = "powerpc", target_arch = "powerpc64"))]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(target_arch = "s390x")]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(any(target_arch = "sparc", target_arch = "sparc64"))]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(any(target_arch = "riscv64", target_arch = "riscv32"))]
+pub const unwinder_private_data_size: usize = 2;
+
+#[cfg(target_os = "emscripten")]
+pub const unwinder_private_data_size: usize = 20;
+
+#[cfg(all(target_arch = "hexagon", target_os = "linux"))]
+pub const unwinder_private_data_size: usize = 35;
+
+#[repr(C)]
+pub struct _Unwind_Exception {
+    pub exception_class: _Unwind_Exception_Class,
+    pub exception_cleanup: _Unwind_Exception_Cleanup_Fn,
+    pub private: [_Unwind_Word; unwinder_private_data_size],
+}
+
+pub enum _Unwind_Context {}
+
+pub type _Unwind_Exception_Cleanup_Fn =
+    extern "C" fn(unwind_code: _Unwind_Reason_Code, exception: *mut _Unwind_Exception);
+#[cfg_attr(
+    all(feature = "llvm-libunwind", any(target_os = "fuchsia", target_os = "linux")),
+    link(name = "unwind", kind = "static")
+)]
+extern "C" {
+    #[unwind(allowed)]
+    pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !;
+    pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception);
+    pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void;
+    pub fn _Unwind_GetRegionStart(ctx: *mut _Unwind_Context) -> _Unwind_Ptr;
+    pub fn _Unwind_GetTextRelBase(ctx: *mut _Unwind_Context) -> _Unwind_Ptr;
+    pub fn _Unwind_GetDataRelBase(ctx: *mut _Unwind_Context) -> _Unwind_Ptr;
+}
+
+cfg_if::cfg_if! {
+if #[cfg(any(target_os = "ios", target_os = "netbsd", not(target_arch = "arm")))] {
+    // Not ARM EHABI
+    #[repr(C)]
+    #[derive(Copy, Clone, PartialEq)]
+    pub enum _Unwind_Action {
+        _UA_SEARCH_PHASE = 1,
+        _UA_CLEANUP_PHASE = 2,
+        _UA_HANDLER_FRAME = 4,
+        _UA_FORCE_UNWIND = 8,
+        _UA_END_OF_STACK = 16,
+    }
+    pub use _Unwind_Action::*;
+
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
+        pub fn _Unwind_GetGR(ctx: *mut _Unwind_Context, reg_index: c_int) -> _Unwind_Word;
+        pub fn _Unwind_SetGR(ctx: *mut _Unwind_Context, reg_index: c_int, value: _Unwind_Word);
+        pub fn _Unwind_GetIP(ctx: *mut _Unwind_Context) -> _Unwind_Word;
+        pub fn _Unwind_SetIP(ctx: *mut _Unwind_Context, value: _Unwind_Word);
+        pub fn _Unwind_GetIPInfo(ctx: *mut _Unwind_Context, ip_before_insn: *mut c_int)
+                                 -> _Unwind_Word;
+        pub fn _Unwind_FindEnclosingFunction(pc: *mut c_void) -> *mut c_void;
+    }
+
+} else {
+    // ARM EHABI
+    #[repr(C)]
+    #[derive(Copy, Clone, PartialEq)]
+    pub enum _Unwind_State {
+        _US_VIRTUAL_UNWIND_FRAME = 0,
+        _US_UNWIND_FRAME_STARTING = 1,
+        _US_UNWIND_FRAME_RESUME = 2,
+        _US_ACTION_MASK = 3,
+        _US_FORCE_UNWIND = 8,
+        _US_END_OF_STACK = 16,
+    }
+    pub use _Unwind_State::*;
+
+    #[repr(C)]
+    enum _Unwind_VRS_Result {
+        _UVRSR_OK = 0,
+        _UVRSR_NOT_IMPLEMENTED = 1,
+        _UVRSR_FAILED = 2,
+    }
+    #[repr(C)]
+    enum _Unwind_VRS_RegClass {
+        _UVRSC_CORE = 0,
+        _UVRSC_VFP = 1,
+        _UVRSC_FPA = 2,
+        _UVRSC_WMMXD = 3,
+        _UVRSC_WMMXC = 4,
+    }
+    use _Unwind_VRS_RegClass::*;
+    #[repr(C)]
+    enum _Unwind_VRS_DataRepresentation {
+        _UVRSD_UINT32 = 0,
+        _UVRSD_VFPX = 1,
+        _UVRSD_FPAX = 2,
+        _UVRSD_UINT64 = 3,
+        _UVRSD_FLOAT = 4,
+        _UVRSD_DOUBLE = 5,
+    }
+    use _Unwind_VRS_DataRepresentation::*;
+
+    pub const UNWIND_POINTER_REG: c_int = 12;
+    pub const UNWIND_SP_REG: c_int = 13;
+    pub const UNWIND_IP_REG: c_int = 15;
+
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
+        fn _Unwind_VRS_Get(ctx: *mut _Unwind_Context,
+                           regclass: _Unwind_VRS_RegClass,
+                           regno: _Unwind_Word,
+                           repr: _Unwind_VRS_DataRepresentation,
+                           data: *mut c_void)
+                           -> _Unwind_VRS_Result;
+
+        fn _Unwind_VRS_Set(ctx: *mut _Unwind_Context,
+                           regclass: _Unwind_VRS_RegClass,
+                           regno: _Unwind_Word,
+                           repr: _Unwind_VRS_DataRepresentation,
+                           data: *mut c_void)
+                           -> _Unwind_VRS_Result;
+    }
+
+    // On Android or ARM/Linux, these are implemented as macros:
+
+    pub unsafe fn _Unwind_GetGR(ctx: *mut _Unwind_Context, reg_index: c_int) -> _Unwind_Word {
+        let mut val: _Unwind_Word = 0;
+        _Unwind_VRS_Get(ctx, _UVRSC_CORE, reg_index as _Unwind_Word, _UVRSD_UINT32,
+                        &mut val as *mut _ as *mut c_void);
+        val
+    }
+
+    pub unsafe fn _Unwind_SetGR(ctx: *mut _Unwind_Context, reg_index: c_int, value: _Unwind_Word) {
+        let mut value = value;
+        _Unwind_VRS_Set(ctx, _UVRSC_CORE, reg_index as _Unwind_Word, _UVRSD_UINT32,
+                        &mut value as *mut _ as *mut c_void);
+    }
+
+    pub unsafe fn _Unwind_GetIP(ctx: *mut _Unwind_Context)
+                                -> _Unwind_Word {
+        let val = _Unwind_GetGR(ctx, UNWIND_IP_REG);
+        (val & !1) as _Unwind_Word
+    }
+
+    pub unsafe fn _Unwind_SetIP(ctx: *mut _Unwind_Context,
+                                value: _Unwind_Word) {
+        // Propagate thumb bit to instruction pointer
+        let thumb_state = _Unwind_GetGR(ctx, UNWIND_IP_REG) & 1;
+        let value = value | thumb_state;
+        _Unwind_SetGR(ctx, UNWIND_IP_REG, value);
+    }
+
+    pub unsafe fn _Unwind_GetIPInfo(ctx: *mut _Unwind_Context,
+                                    ip_before_insn: *mut c_int)
+                                    -> _Unwind_Word {
+        *ip_before_insn = 0;
+        _Unwind_GetIP(ctx)
+    }
+
+    // This function also doesn't exist on Android or ARM/Linux, so make it a no-op
+    pub unsafe fn _Unwind_FindEnclosingFunction(pc: *mut c_void) -> *mut c_void {
+        pc
+    }
+}
+} // cfg_if!
+
+cfg_if::cfg_if! {
+if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] {
+    // Not 32-bit iOS
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
+        #[unwind(allowed)]
+        pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
+        pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn,
+                                 trace_argument: *mut c_void)
+                                 -> _Unwind_Reason_Code;
+    }
+} else {
+    // 32-bit iOS uses SjLj and does not provide _Unwind_Backtrace()
+    #[cfg_attr(all(feature = "llvm-libunwind",
+                   any(target_os = "fuchsia", target_os = "linux")),
+               link(name = "unwind", kind = "static"))]
+    extern "C" {
+        #[unwind(allowed)]
+        pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code;
+    }
+
+    #[inline]
+    pub unsafe fn _Unwind_RaiseException(exc: *mut _Unwind_Exception) -> _Unwind_Reason_Code {
+        _Unwind_SjLj_RaiseException(exc)
+    }
+}
+} // cfg_if!
+
+cfg_if::cfg_if! {
+if #[cfg(all(windows, target_arch = "x86_64", target_env = "gnu"))] {
+    // We declare these as opaque types. This is fine since you just need to
+    // pass them to _GCC_specific_handler and forget about them.
+    pub enum EXCEPTION_RECORD {}
+    pub type LPVOID = *mut c_void;
+    pub enum CONTEXT {}
+    pub enum DISPATCHER_CONTEXT {}
+    pub type EXCEPTION_DISPOSITION = c_int;
+    type PersonalityFn = unsafe extern "C" fn(version: c_int,
+                                              actions: _Unwind_Action,
+                                              exception_class: _Unwind_Exception_Class,
+                                              exception_object: *mut _Unwind_Exception,
+                                              context: *mut _Unwind_Context)
+                                              -> _Unwind_Reason_Code;
+
+    extern "C" {
+        pub fn _GCC_specific_handler(exceptionRecord: *mut EXCEPTION_RECORD,
+                                establisherFrame: LPVOID,
+                                contextRecord: *mut CONTEXT,
+                                dispatcherContext: *mut DISPATCHER_CONTEXT,
+                                personality: PersonalityFn)
+                                -> EXCEPTION_DISPOSITION;
+    }
+}
+} // cfg_if!

artiq/firmware/libunwind_backtrace/Cargo.toml

@@ -6,3 +6,7 @@ version = "0.0.0"
 [lib]
 name = "unwind_backtrace"
 path = "lib.rs"
+
+[dependencies]
+unwind = { path = "../libunwind" }
+libc = { path = "../libc" }

artiq/firmware/riscv32g-unknown-none-elf.json

@@ -0,0 +1,40 @@
+{
+  "arch": "riscv32",
+  "code-model": "medium",
+  "cpu": "generic-rv32",
+  "crt-static-respected": true,
+  "data-layout": "e-m:e-p:32:32-i64:64-n32-S128",
+  "dynamic-linking": true,
+  "executables": true,
+  "features": "+m,+a,+f,+d",
+  "has-elf-tls": true,
+  "has-rpath": true,
+  "llvm-abiname": "ilp32d",
+  "llvm-target": "riscv32-unknown-linux",
+  "max-atomic-width": 32,
+  "position-independent-executables": true,
+  "pre-link-args": {
+    "gcc": [
+      "-Wl,--as-needed",
+      "-Wl,-z,noexecstack"
+    ]
+  },
+  "relro-level": "full",
+  "target-family": "unix",
+  "target-pointer-width": "32",
+  "unsupported-abis": [
+    "cdecl",
+    "stdcall",
+    "fastcall",
+    "vectorcall",
+    "thiscall",
+    "aapcs",
+    "win64",
+    "sysv64",
+    "ptx-kernel",
+    "msp430-interrupt",
+    "x86-interrupt",
+    "amdgpu-kernel"
+  ]
+}
+  

artiq/firmware/riscv32ima-unknown-none-elf.json

@@ -0,0 +1,31 @@
+{
+  "arch": "riscv32",
+  "cpu": "generic-rv32",
+  "data-layout": "e-m:e-p:32:32-i64:64-n32-S128",
+  "eh-frame-header": false,
+  "emit-debug-gdb-scripts": false,
+  "executables": true,
+  "features": "+m,+a,-c",
+  "is-builtin": false,
+  "linker": "rust-lld",
+  "linker-flavor": "ld.lld",
+  "llvm-target": "riscv32",
+  "max-atomic-width": 32,
+  "panic-strategy": "unwind",
+  "relocation-model": "static",
+  "target-pointer-width": "32",
+  "unsupported-abis": [
+    "cdecl",
+    "stdcall",
+    "fastcall",
+    "vectorcall",
+    "thiscall",
+    "aapcs",
+    "win64",
+    "sysv64",
+    "ptx-kernel",
+    "msp430-interrupt",
+    "x86-interrupt",
+    "amdgpu-kernel"
+  ]
+}

artiq/firmware/runtime/Cargo.toml

@@ -18,7 +18,7 @@ failure_derive = { version = "0.1", default-features = false }
 byteorder = { version = "1.0", default-features = false }
 cslice = { version = "0.3" }
 log = { version = "0.4", default-features = false }
-managed = { version = "= 0.7.0", default-features = false, features = ["alloc", "map"] }
+managed = { version = "^0.7.1", default-features = false, features = ["alloc", "map"] }
 eh = { path = "../libeh" }
 unwind_backtrace = { path = "../libunwind_backtrace" }
 io = { path = "../libio", features = ["byteorder"] }
@@ -27,10 +27,11 @@ board_misoc = { path = "../libboard_misoc", features = ["uart_console", "smoltcp
 logger_artiq = { path = "../liblogger_artiq" }
 board_artiq = { path = "../libboard_artiq" }
 proto_artiq = { path = "../libproto_artiq", features = ["log", "alloc"] }
-smoltcp = { version = "0.6.0", default-features = false, features = ["rust-1_28", "alloc", "ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"] }
+smoltcp = { version = "0.6.0", default-features = false, features = ["alloc", "ethernet", "proto-ipv4", "proto-ipv6", "socket-tcp"] }
+riscv = { version = "0.6.0", features = ["inline-asm"] }
 
 [dependencies.fringe]
-git = "https://github.com/m-labs/libfringe"
+git = "https://git.m-labs.hk/M-Labs/libfringe.git"
-rev = "b8a6d8f"
+rev = "3ecbe5"
 default-features = false
 features = ["alloc"]

artiq/firmware/runtime/Makefile

@@ -10,21 +10,25 @@ LDFLAGS += \
 
 RUSTFLAGS += -Cpanic=unwind
 
+export XBUILD_SYSROOT_PATH=$(BUILDINC_DIRECTORY)/../sysroot
+
 all:: runtime.bin runtime.fbi
 
 .PHONY: $(RUSTOUT)/libruntime.a
 $(RUSTOUT)/libruntime.a:
-	$(cargo) --manifest-path $(RUNTIME_DIRECTORY)/Cargo.toml
+	$(cargo) --target-dir ./cargo \
+	--manifest-path $(RUNTIME_DIRECTORY)/Cargo.toml \
+	--target $(RUNTIME_DIRECTORY)/../$(CARGO_TRIPLE).json
 
 runtime.elf: $(RUSTOUT)/libruntime.a ksupport_data.o
 	$(link) -T $(RUNTIME_DIRECTORY)/runtime.ld \
-		-lunwind-bare
+		-lunwind-bare -m elf32lriscv
 
 ksupport_data.o: ../ksupport/ksupport.elf
-	$(LD) -r -b binary -o $@ $<
+	$(LD) -r -m elf32lriscv -b binary -o $@ $<
 
 %.bin: %.elf
 	$(objcopy) -O binary
 
 %.fbi: %.bin
-	$(mscimg) -f
+	$(mscimg) -f --little

artiq/firmware/runtime/analyzer.rs

@@ -51,7 +51,7 @@ fn worker(stream: &mut TcpStream) -> Result<(), IoError<SchedError>> {
     };
     debug!("{:?}", header);
 
-    stream.write_all("E".as_bytes())?;
+    stream.write_all("e".as_bytes())?;
     header.write_to(stream)?;
     if wraparound {
         stream.write_all(&data[pointer..])?;

artiq/firmware/runtime/cache.rs

@@ -1,4 +1,4 @@
-use alloc::{Vec, String, BTreeMap};
+use alloc::{vec::Vec, string::String, collections::btree_map::BTreeMap};
 
 #[derive(Debug)]
 struct Entry {

artiq/firmware/runtime/main.rs

@@ -1,6 +1,4 @@
-#![feature(lang_items, alloc, try_from, nonzero, asm,
+#![feature(lang_items, panic_info_message)]
-           panic_implementation, panic_info_message,
-           const_slice_len)]
 #![no_std]
 
 extern crate eh;
@@ -25,12 +23,13 @@ extern crate board_misoc;
 extern crate board_artiq;
 extern crate logger_artiq;
 extern crate proto_artiq;
+extern crate riscv;
 
 use core::cell::RefCell;
 use core::convert::TryFrom;
 use smoltcp::wire::IpCidr;
 
-use board_misoc::{csr, irq, ident, clock, boot, config, net_settings};
+use board_misoc::{csr, ident, clock, spiflash, config, net_settings, pmp, boot};
 #[cfg(has_ethmac)]
 use board_misoc::ethmac;
 #[cfg(has_drtio)]
@@ -41,6 +40,8 @@ use proto_artiq::{mgmt_proto, moninj_proto, rpc_proto, session_proto, kernel_pro
 #[cfg(has_rtio_analyzer)]
 use proto_artiq::analyzer_proto;
 
+use riscv::register::{mcause, mepc, mtval};
+
 mod rtio_clocking;
 mod rtio_mgt;
 
@@ -50,7 +51,6 @@ mod cache;
 mod rtio_dma;
 
 mod mgmt;
-mod profiler;
 mod kernel;
 mod kern_hwreq;
 mod session;
@@ -88,8 +88,6 @@ fn setup_log_levels() {
 }
 
 fn startup() {
-    irq::set_mask(0);
-    irq::set_ie(true);
     clock::init();
     info!("ARTIQ runtime starting...");
     info!("software ident {}", csr::CONFIG_IDENTIFIER_STR);
@@ -145,7 +143,7 @@ fn startup() {
     };
 
     let neighbor_cache =
-        smoltcp::iface::NeighborCache::new(alloc::btree_map::BTreeMap::new());
+        smoltcp::iface::NeighborCache::new(alloc::collections::btree_map::BTreeMap::new());
     let net_addresses = net_settings::get_adresses();
     info!("network addresses: {}", net_addresses);
     let mut interface = match net_addresses.ipv6_addr {
@@ -249,29 +247,64 @@ pub extern fn main() -> i32 {
         extern {
             static mut _fheap: u8;
             static mut _eheap: u8;
+            static mut _sstack_guard: u8;
         }
         ALLOC.add_range(&mut _fheap, &mut _eheap);
 
-        logger_artiq::BufferLogger::new(&mut LOG_BUFFER[..]).register(startup);
+        pmp::init_stack_guard(&_sstack_guard as *const u8 as usize);
+
+        logger_artiq::BufferLogger::new(&mut LOG_BUFFER[..]).register(||
+            boot::start_user(startup as usize)
+        );
 
         0
     }
 }
 
+#[derive(Debug, Clone, Copy)]
+#[repr(C)]
+pub struct TrapFrame {
+    pub ra: usize,
+    pub t0: usize,
+    pub t1: usize,
+    pub t2: usize,
+    pub t3: usize,
+    pub t4: usize,
+    pub t5: usize,
+    pub t6: usize,
+    pub a0: usize,
+    pub a1: usize,
+    pub a2: usize,
+    pub a3: usize,
+    pub a4: usize,
+    pub a5: usize,
+    pub a6: usize,
+    pub a7: usize,
+}
+
 #[no_mangle]
-pub extern fn exception(vect: u32, _regs: *const u32, pc: u32, ea: u32) {
+pub extern fn exception(regs: *const TrapFrame) {
-    let vect = irq::Exception::try_from(vect).expect("unknown exception");
+    let pc = mepc::read();
-    match vect {
+    let cause = mcause::read().cause();
-        irq::Exception::Interrupt =>
+    match cause {
-            while irq::pending_mask() != 0 {
+        mcause::Trap::Interrupt(source) => {
-                match () {
+            info!("Called interrupt with {:?}", source);
-                    #[cfg(has_timer1)]
+        },
-                    () if irq::is_pending(csr::TIMER1_INTERRUPT) =>
+
-                        profiler::sample(pc as usize),
+        mcause::Trap::Exception(mcause::Exception::UserEnvCall) => {
-                    _ => panic!("spurious irq {}", irq::pending_mask().trailing_zeros())
+            unsafe {
+                if (*regs).a7 == 0 {
+                    pmp::pop_pmp_region()
+                } else {
+                    pmp::push_pmp_region((*regs).a7)
                 }
-            },
+            }
-        _ => {
+            mepc::write(pc + 4);
+        },
+
+        mcause::Trap::Exception(e) => {
+            println!("Trap frame: {:x?}", unsafe { *regs });
+
             fn hexdump(addr: u32) {
                 let addr = (addr - addr % 4) as *const u32;
                 let mut ptr  = addr;
@@ -285,9 +318,9 @@ pub extern fn exception(vect: u32, _regs: *const u32, pc: u32, ea: u32) {
                 }
             }
 
-            hexdump(pc);
+            hexdump(u32::try_from(pc).unwrap());
-            hexdump(ea);
+            let mtval = mtval::read();
-            panic!("exception {:?} at PC 0x{:x}, EA 0x{:x}", vect, pc, ea)
+            panic!("exception {:?} at PC 0x{:x}, trap value 0x{:x}", e, u32::try_from(pc).unwrap(), mtval)
         }
     }
 }
@@ -305,10 +338,8 @@ pub fn oom(layout: core::alloc::Layout) -> ! {
 }
 
 #[no_mangle] // https://github.com/rust-lang/rust/issues/{38281,51647}
-#[panic_implementation]
+#[panic_handler]
 pub fn panic_impl(info: &core::panic::PanicInfo) -> ! {
-    irq::set_ie(false);
-
     #[cfg(has_error_led)]
     unsafe {
         csr::error_led::out_write(1);
@@ -327,16 +358,17 @@ pub fn panic_impl(info: &core::panic::PanicInfo) -> ! {
 
     println!("backtrace for software version {}:", csr::CONFIG_IDENTIFIER_STR);
     let _ = unwind_backtrace::backtrace(|ip| {
-        // Backtrace gives us the return address, i.e. the address after the delay slot,
+        // Backtrace gives us the return address, i.e. the address after jal(r) insn,
         // but we're interested in the call instruction.
-        println!("{:#08x}", ip - 2 * 4);
+        println!("{:#08x}", ip - 4);
     });
 
-    if config::read_str("panic_reset", |r| r == Ok("1")) {
+    if config::read_str("panic_reset", |r| r == Ok("1")) && 
+        cfg!(any(soc_platform = "kasli", soc_platform = "metlino", soc_platform = "kc705")) {
         println!("restarting...");
         unsafe {
             kernel::stop();
-            boot::reset();
+            spiflash::reload();
         }
     } else {
         println!("halting.");