doc: Rewrite FAQ

doc/manual/configuring.rst

@@ -67,6 +67,8 @@ On Kasli or Kasli-SoC devices, specifying the MAC address is unnecessary, as the
 
 Regarding IPv6, note that the device also has a link-local address that corresponds to its EUI-64, which can be used simultaneously to the (potentially unrelated) IPv6 address defined by using the ``ip6`` configuration key.
 
+If problems persist, see the :ref:`network troubleshooting <faq-networking>` section of the FAQ.
+
 .. _core-device-config:
 
 Configuring the core device

doc/manual/faq.rst

@@ -1,26 +1,131 @@
-.. Copyright (C) 2014, 2015 Robert Jordens <jordens@gmail.com>
+FAQ (How do I...)
+=================
 
-FAQ
+use this documentation?
-===
+-----------------------
 
-How do I ...
+The content of this manual is arranged in rough reading order. If you start at the beginning and make your way through section by section, you should form a pretty good idea of how ARTIQ works and how to use it. Otherwise:
-------------
+
+**If you are just starting out,** and would like to get ARTIQ set up on your computer and your core device, start with :doc:`installing`, :doc:`flashing`, and :doc:`configuring`, in that order.
+
+**If you have a working ARTIQ setup** (or someone else has set it up for you), start with the tutorials: read :doc:`rtio`, then progress to :doc:`getting_started_core` and :doc:`getting_started_mgmt`. If your system is in a DRTIO configuration, :doc:`DRTIO and subkernels <using_drtio_subkernels>` will also be helpful.
+
+Pages like :doc:`management_system` and :doc:`core_device` describe **specific components of the ARTIQ ecosystem** in more detail. If you want to understand more about device and dataset databases, for example, read the :doc:`environment` page; if you want to understand the ARTIQ Python dialect and everything it does or does not support, read the :doc:`compiler` page.
+
+Reference pages, like :doc:`main_frontend_tools` and :doc:`core_drivers_reference`, contain the detailed documentation of the individual methods and command-line tools ARTIQ provides. They are heavily interlinked throughout the rest of the documentation: whenever a method, tool, or exception is mentioned by name, like :class:`~artiq.frontend.artiq_run`, :meth:`~artiq.language.core.now_mu`, or :exc:`~artiq.coredevice.exceptions.RTIOUnderflow`, it can normally be clicked on to directly access the reference material.
+
+.. _build-documentation:
+
+build this documentation?
+-------------------------
+
+To generate this manual from source, you can use ``nix build`` directives, for example: ::
+
+    $ nix build git+https://github.com/m-labs/artiq.git\?ref=release-[number]#artiq-manual-html
+
+Substitute ``artiq-manual-pdf`` to get the LaTeX PDF version. The results will be in ``result``.
+
+The manual is written in `reStructured Text <https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`_; you can find the source files in the ARTIQ repository under ``doc/manual``. If you spot a mistake, a typo, or something that's out of date or missing -- in particular, if you want to add something to this FAQ -- feel free to clone the repository, edit the RST files, and make a pull request with your version of an improvement. (If you're not a fan of or not familiar with command-line Git, both GitHub and Gitea support making edits and pull requests directly in the web interface; tutorial materials are easy to find online.) The second best thing is to open an issue to make M-Labs aware of the problem.
+
+.. _faq-find-examples:
 
 find ARTIQ examples?
 --------------------
 
-The examples are installed in the ``examples`` folder of the ARTIQ package. You can find where the ARTIQ package is installed on your machine with: ::
+Examples are stored in the ``examples`` folder of the ARTIQ package. You can find the location of the ARTIQ package on your machine with: ::
 
   python3 -c "import artiq; print(artiq.__path__[0])"
 
-Copy the ``examples`` folder from that path into your home/user directory, and start experimenting!
+Copy the ``examples`` folder from that path into your home or user directory, and start experimenting!
 
-prevent my first RTIO command from causing an underflow?
+fix 'no startup kernel found' / 'no idle kernel found' in the UART?
---------------------------------------------------------
+-------------------------------------------------------------------
 
-The first RTIO event is programmed with a small timestamp above the value of the timecounter when the core device is reset. If the kernel needs more time than this timestamp to produce the event, an underflow will occur. You can prevent it by calling ``break_realtime`` just before programming the first event, or by adding a sufficient delay.
+Don't. Note that these are ``INFO`` messages, and not ``ERROR`` or even ``WARN``. If you haven't flashed an idle or startup kernel yet, this is normal, and will not cause any problems; between experiments the core device will simply do nothing. The same applies to most other messages in the style of 'no configuration found' or 'falling back to default'. Your system will generally run just fine on its defaults until you get around to setting these configurations, though certain features may be limited until properly set up. See :doc:`configuring` and the list of keys in :ref:`core-device-flash-storage`.
 
-If you are not resetting the core device, the time cursor stays where the previous experiment left it.
+fix 'Mismatch between gateware and software versions'?
+------------------------------------------------------
+
+Either reflash your core device with a newer version of ARTIQ (see :doc:`flashing`) or update your software (see :ref:`installing-upgrading`), depending on which is out of date.
+
+.. note::
+    You can check the specific versions you are using at any time by comparing the gateware version given in the core startup log and the output given by adding ``--version`` to any of the standard ARTIQ front-end commands. This is especially useful when e.g. seeking help in the forum or helpdesk@, where your running ARTIQ version is often crucial information to diagnose a problem.
+
+    Minor version mismatches are common, even in stable ARTIQ versions, but should not cause any issues. The ARTIQ release system ensures breaking changes are strictly limited to new release versions, or to the beta branch (which explicitly makes no promises of stability.) Updates that *are* applied to the stable version are usually bug fixes, documentation improvements, or other quality-of-life changes. As long as gateware and software are using the same stable release version of ARTIQ, even if there is a minor mismatch, no warning will be displayed.
+
+change configuration settings of satellite devices?
+---------------------------------------------------
+
+Currently, it is not possible to reach satellites through ``artiq_coremgmt config``, although this is being worked on. On Kasli, use :class:`~artiq.frontend.artiq_mkfs` and :class:`~artiq.frontend.artiq_flash`; on Kasli-SoC, preload the SD card with a ``config.txt``, formatted as a list of ``key=value`` pairs, one per line.
+
+Don't worry about individually flashing idle or startup kernels. If your idle or startup kernel contains subkernels, it will automatically compile as a ``.tar``, which you only need to flash to the master.
+
+.. _faq-networking:
+
+troubleshoot networking problems?
+---------------------------------
+
+Diagnosis aids:
+
+    - Can you ``ping`` the device?
+    - Is the Ethernet LED on?
+    - Is the ERROR LED on?
+    - Is there anything unusual recorded in :ref:`the UART log <connecting-UART>`?
+
+Some things to consider:
+
+    - Is the ``core_addr`` field of your ``device_db.py`` set correctly?
+    - Are your core device's IP address and networking configurations definitely set correctly? Check the UART log for evidence of this, and talk to your network administrator about what the correct choices are.
+    - Is your core device configured for an external reference clock? If so, it cannot function correctly without one. Is the external reference clock plugged in?
+    - Are Ethernet and (on Kasli only) SFP0 plugged in all the way? Are they working? Try different cables and SFP adapters; M-Labs tests with CAT6 cables, but lower categories should be supported too.
+    - Are your PC and your crate in the same subnet?
+    - Is some other device in your network already using the configured IP address? Turn off the core device and try pinging the configured IP address; if it responds, you have a culprit. One of the two will need a different networking configuration.
+    - Are there restrictions or issues in your router or subnet that are preventing the core device from connecting? It may help to try connecting the core device to your PC directly.
+
+diagnose and fix sequence errors?
+---------------------------------
+
+Go through your code, keeping manual track of SED lanes. See the following example: ::
+
+    @kernel
+    def run(self):
+        self.core.reset()
+        with parallel:
+            self.ttl0.on()  # lane0
+            self.ttl_sma.pulse(800*us)  # lane1(rising) lane1(falling)
+            with sequential:
+                self.ttl1.on()  # lane2
+                self.ttl2.on()  # lane3
+                self.ttl3.on()  # lane4
+                self.ttl4.on()  # lane5
+                delay(800*us)
+                self.ttl1.off() # lane5
+                self.ttl2.off() # lane6
+                self.ttl3.off() # lane7
+                self.ttl4.off() # lane0
+        self.ttl0.off()  # lane1 -> clashes with the falling edge of ttl_sma,
+                         # which is already at +800us
+
+In most cases, as in this one, it's relatively easy to rearrange the generation of events so that they will be better spread out across SED lanes without sacrificing actual functionality. One possible solution for the above sequence looks like: ::
+
+    @kernel
+    def run(self):
+        self.core.reset()
+        self.ttl0.on()     # lane0
+        self.ttl_sma.on()  # lane1
+        self.ttl1.on()     # lane2
+        self.ttl2.on()     # lane3
+        self.ttl3.on()     # lane4
+        self.ttl4.on()     # lane5
+        delay(800*us)
+        self.ttl1.off()    # lane5
+        self.ttl2.off()    # lane6
+        self.ttl3.off()    # lane7
+        self.ttl4.off()    # lane0  (no clash: new timestamp is higher than last)
+        self.ttl_sma.off() # lane1
+        self.ttl0.off()    # lane2
+
+In this case, the :meth:`~artiq.coredevice.ttl.TTLInOut.pulse` is split up into its component :meth:`~artiq.coredevice.ttl.TTLInOut.on` and  :meth:`~artiq.coredevice.ttl.TTLInOut.off` so that events can be generated more linearly. It can also be worth keeping in mind that delaying by even a single coarse RTIO cycle between events avoids switching SED lanes at all; in contexts where perfect simultaneity is not a priority, this is an easy way to avoid sequencing issues. See again :ref:`sequence-errors`.
 
 organize datasets in folders?
 -----------------------------
@@ -37,74 +142,31 @@ Experiment windows can be organized by using the following hotkeys:
 
 The windows will be organized in the order they were last interacted with.
 
-write a generator feeding a kernel feeding an analyze function?
+create and use variable-length arrays in kernels?
----------------------------------------------------------------
+-------------------------------------------------
 
-  Like this::
+You can't, in general; see the corresponding notes under :ref:`compiler-types`. ARTIQ kernels do not support heap allocation, meaning in particular that lists, arrays, and strings must be of constant size. One option is to preallocate everything, as mentioned on the Compiler page; another option is to chunk it and e.g. read 100 events per function call, push them upstream and retry until the gate time closes.
-
-    def run(self):
-        self.parse(self.pipe(iter(range(10))))
-
-    def pipe(self, gen):
-        for i in gen:
-            r = self.do(i)
-            yield r
-
-    def parse(self, gen):
-        for i in gen:
-            pass
-
-    @kernel
-    def do(self, i):
-        return i
-
-create and use variable lengths arrays in kernels?
---------------------------------------------------
-
-Don't. Preallocate everything. Or chunk it and e.g. read 100 events per
-function call, push them upstream and retry until the gate time closes.
-
-execute multiple slow controller RPCs in parallel without losing time?
-----------------------------------------------------------------------
-
-Use ``threading.Thread``: portable, fast, simple for one-shot calls.
 
 write part of my experiment as a coroutine/asyncio task/generator?
 ------------------------------------------------------------------
 
-You can not change the API that your experiment exposes: ``build()``,
+You cannot change the API that your experiment exposes: :meth:`~artiq.language.environment.HasEnvironment.build`, :meth:`~artiq.language.environment.Experiment.prepare`, :meth:`~artiq.language.environment.Experiment.run` and :meth:`~artiq.language.environment.Experiment.analyze` need to be regular functions, not generators or asyncio coroutines. That would make reusing your own code in sub-experiments difficult and fragile. You can however wrap your own generators/coroutines/tasks in regular functions that you then expose as part of the API.
-``prepare()``, ``run()`` and ``analyze()`` need to be regular functions, not
-generators or asyncio coroutines. That would make reusing your own code in
-sub-experiments difficult and fragile. You can however wrap your own
-generators/coroutines/tasks in regular functions that you then expose as part
-of the API.
 
-determine the pyserial URL to attach to a device by its serial number?
+determine the pyserial URL to connect to a device by its serial number?
-----------------------------------------------------------------------
+-----------------------------------------------------------------------
 
-You can list your system's serial devices and print their vendor/product
+You can list your system's serial devices and print their vendor/product id and serial number by running::
-id and serial number by running::
 
     $ python3 -m serial.tools.list_ports -v
 
-It will give you the ``/dev/ttyUSBxx`` (or the ``COMxx`` for Windows) device
+This will give you the ``/dev/ttyUSBxx`` (or ``COMxx`` for Windows) device names. The ``hwid:`` field gives you the string you can pass via the ``hwgrep://`` feature of pyserial `serial_for_url() <https://pythonhosted.org/pyserial/pyserial_api.html#serial.serial_for_url>`_ in order to open a serial device.
-names.
-The ``hwid:`` field gives you the string you can pass via the ``hwgrep://``
-feature of pyserial
-`serial_for_url() <https://pythonhosted.org/pyserial/pyserial_api.html#serial.serial_for_url>`_
-in order to open a serial device.
 
-The preferred way to specify a serial device is to make use of the ``hwgrep://``
+The preferred way to specify a serial device is to make use of the ``hwgrep://`` URL: it allows for selecting the serial device by its USB vendor ID, product
-URL: it allows to select the serial device by its USB vendor ID, product
+ID and/or serial number. These never change, unlike the device file name.
-ID and/or serial number. Those never change, unlike the device file name.
 
-For instance, if you want to specify the Vendor/Product ID and the USB Serial Number, you can do:
+For instance, if you want to specify the Vendor/Product ID and the USB Serial Number, you can do: ::
-
-``-d "hwgrep://<VID>:<PID> SNR=<serial_number>"``.
-for example:
-
-``-d "hwgrep://0403:faf0 SNR=83852734"``
 
+    $ -d "hwgrep://<VID>:<PID> SNR=<serial_number>"``.
 
 run unit tests?
 ---------------
@@ -124,9 +186,33 @@ The core device tests require the following TTL devices and connections:
 
 If TTL devices are missing, the corresponding tests are skipped.
 
-find the dashboard and browser configuration files are stored?
+find the dashboard and browser configuration files?
---------------------------------------------------------------
+---------------------------------------------------
 
 ::
 
   python -c "from artiq.tools import get_user_config_dir; print(get_user_config_dir())"
+
+Additional Resources
+====================
+
+Other related documentation
+---------------------------
+
+- the `Sinara wiki <https://github.com/sinara-hw/meta/wiki>`_
+- the `SiPyCo manual <https://m-labs.hk/artiq/sipyco-manual/>`_
+- the `Migen manual <https://m-labs.hk/migen/manual/>`_
+- in a pinch, the `M-labs internal docs <https://git.m-labs.hk/sinara-hw/assembly>`_
+
+"Help, I've done my best and I can't get any further!"
+------------------------------------------------------
+
+- If you have an active M-Labs AFWS/support subscription, you can email helpdesk@ at any time for personalized assistance.
+- Compare your materials with the examples; see also :ref:`finding ARTIQ examples <faq-find-examples>` above.
+- Check the list of `active issues <https://github.com/m-labs/artiq/issues>`_ on the ARTIQ GitHub repository for possible known problems with ARTIQ. Search through the closed issues to see if your question or concern has been addressed before.
+- Search the `M-Labs forum <https://forum.m-labs.hk/>`_ for similar problems, or make a post asking for help yourself.
+- Look into the `Mattermost live chat <https://chat.m-labs.hk>`_ or the bridged IRC channel.
+- Read the open source code and its docstrings and figure it out.
+- If you're reasonably certain you've identified a bug, or if you'd like to suggest a feature that should be included in future ARTIQ releases, `file a GitHub issue <https://github.com/m-labs/artiq/issues/new/choose>`_ yourself, following one of the provided templates.
+
+In any of these cases, if you found the manual unclear or unhelpful, you might consider following the :ref:`directions for contribution <build-documentation>` and editing it to be more helpful for future readers.

doc/manual/installing.rst

@@ -197,6 +197,8 @@ After the installation, activate the newly created environment by name. ::
 
 This activation has to be performed in every new shell you open to make the ARTIQ tools from that environment available.
 
+.. _installing-upgrading:
+
 Upgrading ARTIQ
 ---------------