docs: RTIO manual page edit

doc/manual/getting_started_core.rst

@@ -200,11 +200,7 @@ Within a parallel block, some statements can be scheduled sequentially again usi
     
     In practice, the pulses to ``ttl0`` and ``ttl1`` will execute simultaneously, and the pulse to ``ttl2`` will execute after the pulse to ``ttl1``, bringing the total duration of the ``parallel`` block to 4 us. Internally, statements #3 and #4, contained within the top-level if statement, are considered an atomic sequence and executed within an implicit ``with sequential``. To execute #3 and #4 in parallel, it is necessary to place them inside a second, nested ``parallel`` block within the if statement.   
 
-Particular care needs to be taken when working with ``parallel`` blocks which generate large numbers of RTIO events, as it is possible to create sequence errors. A sequence error is caused when the scalable event dispatcher (SED) cannot queue an RTIO event due to its timestamp being the same as or earlier than another event in its queue. By default, the SED has 8 lanes, which suffice in most cases to avoid sequence errors; however, if many (>8) events are queued with interlaced timestamps the problem can still surface. See :ref:`sequence-errors`. 
+Particular care needs to be taken when working with ``parallel`` blocks which generate large numbers of RTIO events, as it is possible to cause sequencing issues in the gateware; see also :ref:`sequence-errors`.
-
-Note that for performance reasons sequence errors do not halt execution of the kernel. Instead, they are reported in the core log. If the ``aqctl_corelog`` process has been started with ``artiq_ctlmgr``, then these errors will be posted to the master log. If an experiment is executed through ``artiq_run``, the errors will only be visible in the core log. 
-
-Sequence errors can usually be overcome by reordering the generation of the events (again, different from and unrelated to reordering the events themselves). Alternatively, the number of SED lanes can be increased in the gateware.
 
 .. _rtio-analyzer-example:
 
@@ -229,14 +225,14 @@ The core device records the real-time I/O waveforms into a circular buffer. It i
                 rtio_log("ttl0", "i", i)
                 delay(...)
 
-When using ``artiq_run``, the recorded data can be extracted using ``artiq_coreanalyzer``(see :ref:`core-device-rtio-analyzer-tool`). To export it to VCD, which can be viewed using third-party tools such as GtkWave, use the command ``artiq_coreanalyzer -w rtio.vcd``. Recorded data can also be viewed directly with the ARTIQ dashboard, which will be presented later in :doc:`getting_started_mgmt`.   
+When using ``artiq_run``, the recorded data can be extracted using ``artiq_coreanalyzer`` (see :ref:`core-device-rtio-analyzer-tool`). To export it to VCD, which can be viewed using third-party tools such as GtkWave, use the command ``artiq_coreanalyzer -w rtio.vcd``. Recorded data can also be viewed directly with the ARTIQ dashboard, which will be presented later in :doc:`getting_started_mgmt`.   
 
 .. _getting-started-dma: 
 
 Direct Memory Access (DMA)
 --------------------------
 
-DMA allows for storing fixed sequences of RTIO events in system memory and having the DMA core in the FPGA play them back at high speed. Provided that the specifications of a desired event sequence are known far enough in advance, and no other RTIO issues (collisions, sequence errors) are provoked, even extremely fast and detailed event sequences are always possible to generate and execute. However, if they are time-consuming for the CPU to generate, they may require very large amounts of positive slack in order to allow the CPU enough time to complete the generation before the wall clock 'catches up' (that is, without running into RTIO underflows). A better option is to record these sequences to the DMA core. Once recorded, events sequences are fixed and cannot be modified, but can be safely replayed at any position in the timeline, potentially repeatedly. 
+DMA allows for storing fixed sequences of RTIO events in system memory and having the DMA core in the FPGA play them back at high speed. Provided that the specifications of a desired event sequence are known far enough in advance, and no other RTIO issues (collisions, sequence errors) are provoked, even extremely fast and detailed event sequences can always be generated and executed. RTIO underflows occur when events cannot be generated *as fast as* they need to be executed, resulting in an exception when the wall clock 'catches up'. The solution is to record these sequences to the DMA core. Once recorded, event sequences are fixed and cannot be modified, but can be safely replayed very quickly at any position in the timeline, potentially repeatedly. 
 
 Try this: ::
 

doc/manual/rtio.rst

@@ -7,8 +7,7 @@ The ARTIQ Real-Time I/O design employs several concepts to achieve its goals of
 
 In a typical environment two very different classes of hardware need to be controlled.
 One class is the vast arsenal of diverse laboratory hardware that interfaces with and is controlled from a typical PC.
-The other is specialized real-time hardware
+The other is specialized real-time hardware that requires tight coupling and a low-latency interface to a CPU.
-that requires tight coupling and a low-latency interface to a CPU.
 The ARTIQ code that describes a given experiment is composed of two types of "programs":
 regular Python code that is executed on the host and ARTIQ *kernels* that are executed on a *core device*.
 The CPU that executes the ARTIQ kernels has direct access to specialized programmable I/O timing logic (part of the *gateware*).
@@ -17,30 +16,41 @@ The two types of code can invoke each other and transitions between them are sea
 The ARTIQ kernels do not interface with the real-time gateware directly.
 That would lead to imprecise, indeterminate, and generally unpredictable timing.
 Instead the CPU operates at one end of a bank of FIFO (first-in-first-out) buffers while the real-time gateware at the other end guarantees the *all or nothing* level of excellent timing precision.
-A FIFO for an output channel hold timestamps and event data describing when and what is to be executed.
+
+A FIFO for an output channel holds timestamps and event data describing when and what is to be executed.
 The CPU feeds events into this FIFO.
-A FIFO for an input channel contains timestamps and event data for events that have been recorded by the real-time gateware and are waiting to be read out by
+A FIFO for an input channel contains timestamps and event data for events that have been recorded by the real-time gateware and are waiting to be read out by the CPU on the other end.
-the CPU on the other end.
 
 
-The timeline
+Timeline and terminology 
-------------
+------------------------
 
 The set of all input and output events on all channels constitutes the *timeline*.
-A high resolution wall clock (``rtio_counter``) counts clock cycles and causes output events to be executed when their timestamp matches the clock and input events to be recorded and stamped with the current clock value accordingly.
+A high-resolution wall clock (``rtio_counter``) counts clock cycles and manages the precise timing of the events. Output events are executed when their timestamp matches the current clock value. Input events are recorded when they reach the gateware and stamped with the current clock value accordingly. 
 
-The kernel runtime environment maintains a timeline cursor (called ``now``) used as the timestamp when output events are submitted to the FIFOs.
+The kernel runtime environment maintains a timeline cursor (called ``now``) used as the timestamp when output events are submitted to the FIFOs. Both ``now`` and ``rtio_counter`` are counted in integer *machine units,* or mu, rather than SI units. The machine unit represents the maximum resolution of RTIO timing in an ARTIQ system. The duration of a machine unit is the *reference period* of the system, and may be changed by the user, but normally corresponds to a duration of one nanosecond. 
-This timeline cursor can be moved forward or backward on the timeline relative to its current value using :func:`artiq.language.core.delay` and :func:`artiq.language.core.delay_mu`, the later being a delay given in *machine units* as opposed to SI units.
+
-The absolute value of ``now`` on the timeline can be retrieved using :func:`artiq.language.core.now_mu` and it can be set using :func:`artiq.language.core.at_mu`.
+The timeline cursor ``now`` can be moved forward or backward on the timeline using :func:`artiq.language.core.delay` and :func:`artiq.language.core.delay_mu` (for delays given in SI units or machine units respectively). The absolute value of ``now`` on the timeline can be retrieved using :func:`artiq.language.core.now_mu` and it can be set using :func:`artiq.language.core.at_mu`. The difference between the cursor and the wall clock is referred to as *slack.* A system is considered in a situation of *positive slack* when the cursor is ahead of the wall clock, i.e., in the future; respectively, it is in *negative slack* if the cursor is behind the wall clock, i.e. in the past. 
-RTIO timestamps, the timeline cursor, and the ``rtio_counter`` wall clock are all relative to the core device startup/boot time.
+
-The wall clock keeps running across experiments.
+RTIO timestamps, the timeline cursor, and the ``rtio_counter`` wall clock are all counted relative to the core device startup/boot time. The wall clock keeps running across experiments. 
 
 Absolute timestamps can be large numbers. 
-They are represented internally as 64-bit integers with a resolution of typically a nanosecond and a range of hundreds of years.
+They are represented internally as 64-bit integers. 
+With a typical one-nanosecond machine unit, this covers a range of hundreds of years. 
 Conversions between such a large integer number and a floating point representation can cause loss of precision through cancellation.
 When computing the difference of absolute timestamps, use ``self.core.mu_to_seconds(t2-t1)``, not ``self.core.mu_to_seconds(t2)-self.core.mu_to_seconds(t1)`` (see :meth:`artiq.coredevice.Core.mu_to_seconds`).
 When accumulating time, do it in machine units and not in SI units, so that rounding errors do not accumulate.
 
+.. note:: 
+  Absolute timestamps are also referred to as *RTIO fine timestamps,* because they run on a significantly finer resolution than the timestamps provided by the so-called *coarse RTIO clock,* the actual clocking signal provided to or generated by the core device.   
+  The frequency of the coarse RTIO clock is set by the core device :ref:`clocking settings <core-device-clocking>` but is most commonly 125MHz, which corresponds to eight one-nanosecond machine units per coarse RTIO cycle. 
+
+  The *coarse timestamp* of an event is its timestamp as according to the lower resolution of the coarse clock. 
+  It is in practice a truncated version of the fine timestamp. 
+  In general, ARTIQ offers *precision* on the fine level, but *operates* at the coarse level; this is rarely relevant to the user, but understanding it may clarify the behavior of some RTIO issues (e.g. sequence errors). 
+  
+  .. Related: https://github.com/m-labs/artiq/issues/1237
+
 The following basic example shows how to place output events on the timeline.
 It emits a precisely timed 2 µs pulse::
 
@@ -48,11 +58,10 @@ It emits a precisely timed 2 µs pulse::
   delay(2*us)
   ttl.off()
 
-The device ``ttl`` represents a single digital output channel
+The device ``ttl`` represents a single digital output channel (:class:`artiq.coredevice.ttl.TTLOut`).
-(:class:`artiq.coredevice.ttl.TTLOut`).
 The :meth:`artiq.coredevice.ttl.TTLOut.on` method places an rising edge on the timeline at the current cursor position (``now``).
-Then the cursor is moved forward 2 µs and a falling edge event is placed at the new cursor position.
+Then the cursor is moved forward 2 µs and a falling edge is placed at the new cursor position.
-Then later, when the wall clock reaches the respective timestamps the RTIO gateware executes the two events.
+Later, when the wall clock reaches the respective timestamps, the RTIO gateware executes the two events. 
 
 The following diagram shows what is going on at the different levels of the software and gateware stack (assuming one machine unit of time is 1 ns):
 
@@ -75,18 +84,21 @@ The following diagram shows what is going on at the different levels of the soft
     ]
   }
 
-The sequence is exactly equivalent to::
+This sequence is exactly equivalent to::
 
   ttl.pulse(2*us)
 
-The :meth:`artiq.coredevice.ttl.TTLOut.pulse` method advances the timeline cursor (using ``delay()``) while other methods such as :meth:`artiq.coredevice.ttl.TTLOut.on`, :meth:`artiq.coredevice.ttl.TTLOut.off`, :meth:`artiq.coredevice.ad9914.set`. The latter are called *zero-duration* methods.
+This method :meth:`artiq.coredevice.ttl.TTLOut.pulse` advances the timeline cursor (using ``delay()`` internally) by exactly the amount given. Other methods such as :meth:`artiq.coredevice.ttl.TTLOut.on`, :meth:`artiq.coredevice.ttl.TTLOut.off`, :meth:`artiq.coredevice.ad9914.set` do not modify the timeline cursor. The latter are called *zero-duration* methods.
 
-Underflow exceptions
+Output errors and exceptions 
---------------------
+----------------------------
 
-An RTIO event must always be programmed with a timestamp in the future.
+Underflows
-In other words, the timeline cursor ``now`` must be after the current wall clock ``rtio_counter``: the past can not be altered.
+^^^^^^^^^^
-The following example tries to place an rising edge event on the timeline.
+
+A RTIO ouput event must always be programmed with a timestamp in the future. 
+In other words, the timeline cursor ``now`` must be in advance of the current wall clock ``rtio_counter``: the past cannot be altered. 
+The following example tries to place a rising edge event on the timeline. 
 If the current cursor is in the past, an :class:`artiq.coredevice.exceptions.RTIOUnderflow` exception is thrown. 
 The experiment attempts to handle the exception by moving the cursor forward and repeating the programming of the rising edge:: 
 
@@ -97,6 +109,9 @@ The experiment attempts to handle the exception by moving the cursor forward and
       delay(16.6667*ms)
       ttl.on()
 
+Once the timeline cursor has overtaken the wall clock, the exception does not reoccur and the event can be scheduled successfully. 
+This can also be thought of as adding positive slack to the system.  
+
 .. wavedrom::
 
   {
@@ -117,48 +132,50 @@ The experiment attempts to handle the exception by moving the cursor forward and
 
 To track down ``RTIOUnderflows`` in an experiment there are a few approaches:
 
-  * Exception backtraces show where underflow has occurred while executing the
+  * Exception backtraces show where underflow has occurred while executing the code.
-    code.
+  * The :ref:`integrated logic analyzer <core-device-rtio-analyzer-tool>` shows the timeline context that lead to the exception. The analyzer is always active and supports plotting of RTIO slack. This may be useful to spot where and how an experiment has 'run out' of positive slack.
-  * The :any:`integrated logic analyzer <core-device-rtio-analyzer-tool>` shows the timeline context that lead to the exception. The analyzer is always active and supports plotting of RTIO slack. RTIO slack is the difference between timeline cursor and wall clock time (``now - rtio_counter``).
 
 .. _sequence-errors: 
 
 Sequence errors
----------------
+^^^^^^^^^^^^^^^
-A sequence error happens when the sequence of coarse timestamps cannot be supported by the gateware. For example, there may have been too many timeline rewinds.
 
-Internally, the gateware stores output events in an array of FIFO buffers (the "lanes") and the timestamps in each lane must be strictly increasing. If an event with a decreasing or equal timestamp is submitted, the gateware selects the next lane, wrapping around if the final lane is reached. If this lane also contains an event with a timestamp beyond the one being submitted then a sequence error occurs. See `this issue <https://github.com/m-labs/artiq/issues/1081>`_ for a real-life example of how this works. 
+A sequence error occurs when a sequence of coarse timestamps cannot be transferred to the gateware. Internally, the gateware stores output events in an array of FIFO buffers (the 'lanes'). Within each particular lane, the coarse timestamps of events must be strictly increasing. 
 
-Notes:
+If an event with a timestamp coarsely equal to or lesser than the previous timestamp is submitted, *or* if the current lane is nearly full, the scaleable event dispatcher (SED) selects the next lane, wrapping around once the final lane is reached. If this lane also contains an event with a timestamp equal to or beyond the one being submitted, the placement fails and a sequence error occurs.
 
-* Strictly increasing timestamps never cause sequence errors. 
+.. note:: 
-* Configuring the gateware with more lanes for the RTIO core reduces the frequency of sequence errors.
+  For performance reasons, unlike ``RTIOUnderflow``, most gateware errors do not halt execution of the kernel, because the kernel cannot wait for potential error reports before continuing. As a result, sequence errors are not raised as exceptions and cannot be caught. Instead, the offending event -- in this case, the event that could not be queued -- is discarded, the experiment continues, and the error is reported in the core log. To check the core log, use the command ``artiq_coremgmt log``.  
-* The number of lanes is a hard limit on the number of simultaneous RTIO output events.
-* Whether a particular sequence of timestamps causes a sequence error or not is fully deterministic (starting from a known RTIO state, e.g. after a reset). Adding a constant offset to the whole sequence does not affect the result.
-* Zero-duration methods (such as :meth:`artiq.coredevice.ttl.TTLOut.on()`) do not advance the timeline and so will consume additional lanes if they are scheduled simultaneously. Adding a tiny delay will prevent this (e.g. ``delay_mu(np.int64(self.core.ref_multiplier))``, at least one coarse rtio cycle).
 
-The offending event is discarded and the RTIO core keeps operating.
+By default, the ARTIQ SED has eight lanes, which normally suffices to avoid sequence errors, but problems may still occur if many (>8) events are issued to the gateware with interleaving timestamps. Due to the strict timing limitations imposed on RTIO gateware, it is not possible for the SED to rearrange events in a lane once submitted, nor to anticipate future events when making lane choices. This makes sequence errors fairly 'unintelligent', but also generally fairly easy to eliminate by manually rearranging the generation of events (*not* rearranging the timing of the events themselves, which is rarely necessary.) See `this issue <https://github.com/m-labs/artiq/issues/1081>`_ for a real life example. 
 
-This error is reported asynchronously via the core device log: for performance reasons with DRTIO, the CPU does not wait for an error report from the satellite after writing an event. Therefore, it is not possible to raise an exception precisely.
+It is also possible to simply increase the number of SED lanes in the gateware, which will reduce the frequency of sequencing issues accordingly. 
+
+Other notes: 
+
+* Strictly increasing (coarse) timestamps never cause sequence errors. 
+* Strictly increasing *fine* timestamps within the same coarse cycle may still cause sequence errors. 
+* The number of lanes is a hard limit on the number of RTIO output events that may be emitted within one coarse cycle.  
+* Whether a particular sequence of timestamps causes a sequence error or not is fully deterministic (starting from a known RTIO state, e.g. after a reset). Adding a constant offset to the sequence will not affect the result. 
+* Zero-duration methods (such as :meth:`artiq.coredevice.ttl.TTLOut.on()`) do not advance the timeline and so will always consume additional lanes if they are scheduled simultaneously. Adding a delay of at least one coarse RTIO cycle will prevent this (e.g. ``delay_mu(np.int64(self.core.ref_multiplier))``). 
+
+.. note:: 
+  To change the number of SED lanes, it is necessary to recompile the gateware and reflash your core device. Use the ``sed_lanes`` field in your system description file to set the value, then follow the instructions in :doc:`developing`. Alternatively, if you have an active firmware subscription with M-Labs, contact helpdesk@ for edited binaries.   
+
+.. _collisions-busy-errors: 
 
 Collisions
-----------
+^^^^^^^^^^
-A collision happens when more than one event is submitted on a given channel with the same coarse timestamp, and that channel does not implement replacement behavior or the fine timestamps are different.
+A collision occurs when events are submitted to a given RTIO output channel at a resolution the channel is not equipped to handle. By default, most channels implement 'replacement behavior', meaning that RTIO events submitted to the same timestamp will override each other (for example, if a ``ttl.off()`` and ``ttl.on()`` are scheduled to the same timestamp, the latter automatically overrides the former and only ``ttl.on()`` will be submitted to the channel). On the other hand, if replacement behavior is absent or disabled, or if the two events have the same coarse timestamp with differing fine timestamps, a collision error will be reported.
 
-Coarse timestamps correspond to the RTIO system clock (typically around 125MHz) whereas fine timestamps correspond to the RTIO SERDES clock (typically around 1GHz). Different channels may have different ratios between the coarse and fine timestamp clock frequencies.
+Like sequence errors, collisions originate in gateware and do not stop the execution of the kernel. The offending event is discarded and the problem is reported asynchronously via the core log. 
-
-The offending event is discarded and the RTIO core keeps operating.
-
-This error is reported asynchronously via the core device log: for performance reasons with DRTIO, the CPU does not wait for an error report from the satellite after writing an event. Therefore, it is not possible to raise an exception precisely.
 
 Busy errors
------------
+^^^^^^^^^^^
 
-A busy error happens when at least one output event could not be executed because the channel was already busy executing a previous event.
+A busy error occurs when at least one output event could not be executed because the output channel was already busy executing an event. This differs from a collision error in that a collision is triggered when a sequence of events overwhelms *communication* with a channel, and a busy error is triggered when *execution* is overwhelmed. Busy errors are only possible in the context of single events with execution times longer than a coarse RTIO clock cycle; the exact parameters will depend on the nature of the output channel (e.g. specific peripheral device).  
 
-The offending event was discarded.
+Offending event(s) are discarded and the problem is reported asynchronously via the core log. 
-
-This error is reported asynchronously via the core device log.
 
 Input channels and events
 -------------------------
@@ -171,8 +188,11 @@ If more than 20 rising edges are received, it outputs a pulse::
       delay(2*us)
       output.pulse(500*ns)
 
-The :meth:`artiq.coredevice.ttl.TTLInOut.count` method of an input channel will often lead to a situation of negative slack (timeline cursor ``now`` smaller than the current wall clock ``rtio_counter``):
+Note that many input methods will necessarily involve the wall clock catching up to the timeline cursor or advancing before it. 
-The :meth:`artiq.coredevice.ttl.TTLInOut.gate_rising` method leaves the timeline cursor at the closing time of the gate. ``count()`` must necessarily wait until the gate closing event has actually been executed, at which point ``rtio_counter > now``: ``count()`` synchronizes timeline cursor (``now``) and wall clock (``rtio_counter``). In these situations, a ``delay()`` is necessary to re-establish positive slack so that further output events can be placed.
+This is to be expected: managing output events means working to plan the future, but managing input events means working to react to the past. 
+For input channels, it is the past that is under discussion. 
+
+In this case, the :meth:`artiq.coredevice.ttl.TTLInOut.gate_rising` waits for the duration of the 500ns interval (or *gate window*) and records an event for each rising edge. At the end of the interval it exits, leaving the timeline cursor at the end of the interval (``now = rtio_counter``). :meth:`artiq.coredevice.ttl.TTLInOut.count` unloads these events from the input buffers and counts the number of events recorded, during which the wall clock necessarily advances (``rtio_counter > now``). Accordingly, before we place any further output events, a ``delay()`` is necessary to re-establish positive slack. 
 
 Similar situations arise with methods such as :meth:`artiq.coredevice.ttl.TTLInOut.sample_get` and :meth:`artiq.coredevice.ttl.TTLInOut.watch_done`.
 
@@ -195,11 +215,17 @@ Similar situations arise with methods such as :meth:`artiq.coredevice.ttl.TTLInO
 Overflow exceptions
 -------------------
 
-The RTIO input channels buffer input events received while an input gate is open, or at certain points in time when using the sampling API (:meth:`artiq.coredevice.ttl.TTLInOut.sample_input`).
+The RTIO input channels buffer input events received while an input gate is open, or when using the sampling API (:meth:`artiq.coredevice.ttl.TTLInOut.sample_input`) at certain points in time. 
 The events are kept in a FIFO until the CPU reads them out via e.g. :meth:`artiq.coredevice.ttl.TTLInOut.count`, :meth:`artiq.coredevice.ttl.TTLInOut.timestamp_mu` or :meth:`artiq.coredevice.ttl.TTLInOut.sample_get`. 
-If the FIFO is full and another event is coming in, this causes an overflow condition.
+The size of these FIFOs is finite and specified in gateware; in practice, it is limited by the resources available to the FPGA, and therefore differs depending on the specific core device being used. 
+If a FIFO is full and another event comes in, this causes an overflow condition. 
 The condition is converted into an :class:`artiq.coredevice.exceptions.RTIOOverflow` exception that is raised on a subsequent invocation of one of the readout methods (e.g. ``count()``, ``timestamp_mu()``, ``sample_get()``).
 
+Overflow exceptions are generally best dealt with simply by reading out from the input buffers more frequently. In odd or particular cases, users may consider modifying the length of individual buffers in gateware. 
+
+.. note:: 
+  It is not possible to provoke a ``RTIOOverflow`` on a RTIO output channel. While output buffers are also of finite size, and can be filled up, the CPU will simply stall the submission of further events until it is once again possible to buffer them. Among other things, this means that padding the timeline cursor with large amounts of positive slack is not usually a valid strategy to avoid ``RTIOUnderflows`` when generating fast event sequences. In practice only a fixed number of events can be generated in advance, and the rest of the processing will be carried out when the wall clock is much closer to ``now``. For situations requiring pre-calculation to avoid RTIO underflows, the correct method is to use :ref:`getting-started-dma`.  
+
 Seamless handover
 -----------------
 
@@ -238,17 +264,18 @@ Here, ``run()`` calls ``k1()`` which exits leaving the cursor one second after t
       "R-T", "S-U", "T<->U dt"
     ]
   }
-
-
 .. _rtio-handover-synchronization:
 
 Synchronization
 ---------------
 
-The seamless handover of the timeline (cursor and events) across kernels and experiments implies that a kernel can exit long before the events it has submitted have been executed.
+The seamless handover of the timeline (cursor and events) across kernels and experiments implies that a kernel can exit long before the events it has submitted have been executed. Generally, this is preferable: it frees up resources to the next kernel and allows work to be carried on from kernel to kernel without interruptions.
-If a previous kernel sets timeline cursor far in the future this effectively locks the system.
 
-When a kernel should wait until all the events have been executed, use the :meth:`artiq.coredevice.core.Core.wait_until_mu` with a timestamp after (or at) the last event:
+However, as a result, no guarantees are made about the state of the system when a new kernel enters. Slack may be positive, negative, or zero; input channels may be filled to overflowing, or empty; output channels may contain events currently being executed, contain events scheduled for the far future, or contain no events at all. Unexpected negative slack can cause RTIOUnderflows. Unexpected large positive slack may cause a system to appear to 'lock', as all its events are scheduled for a distant future and the CPU must wait for the output buffers to empty to continue. 
+
+As a result, when beginning a new experiment in an uncertain context, we often want to clear the RTIO FIFOs and initialize the timeline cursor to a reasonable point in the near future. The method :meth:`artiq.coredevice.core.Core.reset` is provided for this purpose. The example idle kernel implements this mechanism. 
+
+On the other hand, if a kernel exits while some of its events are still waiting to be executed, there is no guarantee made that the events in question ever *will* be executed (as opposed to being flushed out by a subsequent core reset). If a kernel should wait until all its events have been executed, use the method :meth:`artiq.coredevice.core.Core.wait_until_mu` with a timestamp after (or at) the last event:
 
 .. wavedrom::
 
@@ -269,14 +296,3 @@ When a kernel should wait until all the events have been executed, use the :meth
 In many cases, :meth:`~artiq.language.core.now_mu` will return an appropriate timestamp::
 
   self.core.wait_until_mu(now_mu())
-
-
-RTIO reset
------------
-
-The seamless handover also means that a kernel is not guaranteed to always be executed with positive slack.
-An experiment can face any of these circumstances (large positive slack, full FIFOs, or negative slack).
-Therefore, when switching experiments it can be adequate to clear the RTIO FIFOs and initialize the timeline cursor to "sometime in the near future" using :meth:`artiq.coredevice.core.Core.reset`.
-The example idle kernel implements this mechanism.
-Since it never waits for any input, it will rapidly fill the output FIFOs and would produce a large positive slack.
-To avoid large positive slack and to accommodate for seamless handover the idle kernel will only run when no other experiment is pending and the example will wait before submitting events until there is significant negative slack.