M-Labs
/
artiq
mirror of https://github.com/m-labs/artiq.git
2
0
Fork 2
Code Issues Projects Releases Wiki Activity

manual: revamp installation instructions

This commit is contained in:
Sebastien Bourdeauducq 2019-02-20 18:30:25 +08:00
parent 7c0353b4be
commit 84619adbab
1 changed files with 148 additions and 105 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

253
doc/manual/installing.rst
View File

@ -1,130 +1,172 @@
.. _install-from-conda:
Installing ARTIQ Installing ARTIQ
================ ================
The preferred way of installing ARTIQ is through the use of the conda package manager. ARTIQ can be installed using the Nix (on Linux) or Conda (on Windows or Linux) package managers.
The conda package contains pre-built binaries that you can directly flash to your board.
.. warning:: Nix is an innovative, robust, fast, and high-quality solution that comes with a larger collection of packages and features than Conda. However, Windows support is poor (using it with Windows Subsystem for Linux still has many problems) and Nix can be harder to learn.
NIST users on Linux need to pay close attention to their ``umask``.
The sledgehammer called ``secureconfig`` leaves you (and root) with umask 027 and files created by root (for example through ``sudo make install``) inaccessible to you.
The usual umask is 022.
Conda has a more traditional approach to package management, is much more limited, slow, and lower-quality than Nix, but it supports Windows and it is simpler to use when it functions correctly.
.. warning:: In the current state of affairs, we recommend that Linux users install ARTIQ via Nix and Windows users install it via Conda.
Conda packages are supported for Linux (64-bit) and Windows (64-bit).
Users of other operating systems (32-bit Linux or Windows, BSD, OSX ...) should and can :ref:`install from source <install-from-source>`.
.. _install-anaconda: Installing via Nix (Linux)
--------------------------
Installing Anaconda or Miniconda First, install the Nix package manager. Some distributions provide a package for the Nix package manager, otherwise, it can be installed via the script on the `Nix website <http://nixos.org/nix/>`_.
--------------------------------
You can either install Anaconda from https://www.anaconda.com/download/ or install the more minimalistic Miniconda from https://conda.io/miniconda.html Once Nix is installed, add the M-Labs package channels with: ::
nix-channel --add https://nixbld.m-labs.hk/channel/custom/artiq/main/channel m-labs
nix-channel --add https://nixbld.m-labs.hk/channel/custom/artiq/sinara-systems/channel sinara
nix-channel --update
Those channels track `nixpkgs 18.09 <https://github.com/NixOS/nixpkgs/tree/release-18.09>`_. You can check the latest status through the `Hydra interface <https://nixbld.m-labs.hk>`_.
Nix won't install packages without verifying their cryptographic signature. Add the M-Labs public key by creating the file ``~/.config/nix/nix.conf`` with the following contents:
::
substituters = https://cache.nixos.org https://nixbld.m-labs.hk
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= nixbld.m-labs.hk-1:5aSRVA5b320xbNvu30tqxVPXpld73bhtOeH6uAjRyHc=
The easiest way to obtain ARTIQ is then to install it into the user environment with ``nix-env -i python3.6-artiq``. This provides a minimal installation of ARTIQ where the usual commands (``artiq_master``, ``artiq_dashboard``, ``artiq_run``, etc.) are available.
This installation is however quite limited, as Nix creates a dedicated Python environment for the ARTIQ commands alone. This means that other useful Python packages that you may want (pandas, matplotlib, ...) are not available to them, and this restriction also applies to the M-Labs packages containing board binaries, which means that ``artiq_flash`` will not automatically find them.
Installing multiple packages and making them visible to the ARTIQ commands requires using the Nix language. Create a file ``my-artiq-env.nix`` with the following contents:
::
let
# Contains the NixOS package collection. ARTIQ depends on some of them, and
# you may also want certain packages from there.
pkgs = import <nixpkgs> {};
# Contains the main ARTIQ packages, their dependencies, and board packages
# for systems used in CI.
# List: https://nixbld.m-labs.hk/channel/custom/artiq/main/channel
m-labs = import <m-labs> {};
# Contains the board packages for the majority of systems.
# List: https://nixbld.m-labs.hk/channel/custom/artiq/sinara-systems/channel
sinara = import <sinara> {};
in
pkgs.mkShell {
buildInputs = [
(pkgs.python3.withPackages(ps: [
# List desired Python packages here.
# The board packages are also "Python" packages. You only need a board
# package if you intend to reflash that board.
m-labs.artiq
m-labs.artiq-board-kc705-nist_clock
sinara.artiq-board-kasli-wipm
# from the NixOS package collection:
ps.pandas
ps.numpy
ps.scipy
ps.numba
ps.matplotlib.override { enableQt = true; }
ps.bokeh
]))
# List desired non-Python packages here
m-labs.openocd # needed for flashing boards
pkgs.spyder
];
}
Then spawn a shell containing the packages with ``nix-shell my-artiq-env.nix``. The ARTIQ commands with all the additional packages should now be available.
You can exit the shell by typing Control-D. The next time ``nix-shell my-artiq-env.nix`` is invoked, Nix uses the cached packages so the shell startup is fast.
You can edit this file according to your needs, and also create multiple ``.nix`` files that correspond to different sets of packages. If you are familiar with Conda, using Nix in this way is similar to having multiple Conda environments.
If your favorite package is not available with Nix, contact us.
Installing via Conda (Windows, Linux)
-------------------------------------
First, install `Anaconda <https://www.anaconda.com/distribution/>`_ or the more minimalistic `Miniconda <https://conda.io/en/latest/miniconda.html>`_.
After installing either Anaconda or Miniconda, open a new terminal (also known as command line, console, or shell and denoted here as lines starting with ``$``) and verify the following command works:: After installing either Anaconda or Miniconda, open a new terminal (also known as command line, console, or shell and denoted here as lines starting with ``$``) and verify the following command works::
$ conda $ conda
Executing just ``conda`` should print the help of the ``conda`` command [1]_. Executing just ``conda`` should print the help of the ``conda`` command. If your shell does not find the ``conda`` command, make sure that the Conda binaries are in your ``$PATH``. If ``$ echo $PATH`` does not show the Conda directories, add them: execute ``$ export PATH=$HOME/miniconda3/bin:$PATH`` if you installed Conda into ``~/miniconda3``.
Installing the ARTIQ packages Download the `ARTIQ installer script <https://raw.githubusercontent.com/m-labs/artiq/master/conda/install-artiq.py>`_ and edit its beginning to define the Conda environment name (you can leave the default environment name if you are just getting started) and select the desired ARTIQ packages. Non-ARTIQ packages should be installed manually later. If you do not need to flash boards, the ``artiq`` package from the ``main`` Hydra build is sufficient.
-----------------------------
.. note:: Make sure the base Conda environment is activated and then run the installer script: ::
On a system with a pre-existing conda installation, it is recommended to update conda to the latest version prior to installing ARTIQ.
Add the M-Labs ``main`` Anaconda package repository containing stable releases and release candidates:: $ conda activate base
$ python install-artiq.py
$ conda config --prepend channels m-labs After the installation, activate the newly created environment by name. ::
.. note:: $ conda activate artiq
To use the development versions of ARTIQ, also add the ``dev`` label (m-labs/label/dev).
Development versions are built for every change and contain more features, but are not as well-tested and are more likely to contain more bugs or inconsistencies than the releases in the default ``main`` label.
Add the conda-forge repository containing ARTIQ dependencies to your conda configuration::
$ conda config --add channels conda-forge
Then prepare to create a new conda environment with the ARTIQ package and the matching binaries for your hardware:
choose a suitable name for the environment, for example ``artiq-main`` if you intend to track the main label, ``artiq-3`` for the 3.x release series, or ``artiq-2016-04-01`` if you consider the environment a snapshot of ARTIQ on 2016-04-01.
Choose the package containing the binaries for your hardware:
* ``artiq-kc705-nist_clock`` for the KC705 board with the NIST "clock" FMC backplane and AD9914 DDS chips.
* ``artiq-kc705-nist_qc2`` for the KC705 board with the NIST QC2 FMC backplane and AD9914 DDS chips.
Conda will create the environment, automatically resolve, download, and install the necessary dependencies and install the packages you select::
$ conda create -n artiq-main artiq-kc705-nist_clock
After the installation, activate the newly created environment by name.
On Unix::
$ source activate artiq-main
On Windows::
$ activate artiq-main
This activation has to be performed in every new shell you open to make the ARTIQ tools from that environment available. This activation has to be performed in every new shell you open to make the ARTIQ tools from that environment available.
.. note:: .. note::
Some ARTIQ examples also require matplotlib and numba, and they must be installed manually for running those examples. They are available in conda. Some ARTIQ examples also require matplotlib and numba, and they must be installed manually for running those examples. They are available in Conda.
Upgrading ARTIQ Upgrading ARTIQ
--------------- ---------------
When upgrading ARTIQ or when testing different versions it is recommended that new environments are created instead of upgrading the packages in existing environments. When upgrading ARTIQ or when testing different versions it is recommended that new Conda environments are created instead of upgrading the packages in existing environments.
Keep previous environments around until you are certain that they are not needed anymore and a new environment is known to work correctly. Keep previous environments around until you are certain that they are not needed anymore and a new environment is known to work correctly.
You can create a new conda environment specifically to test a certain version of ARTIQ::
$ conda create -n artiq-test-1.0rc2 artiq-kc705-nist_clock=1.0rc2 To install the latest version, just select a different environment name and run the installer script again.
Switching between Conda environments using commands such as ``$ conda deactivate artiq-6`` and ``$ conda activate artiq-5`` is the recommended way to roll back to previous versions of ARTIQ.
You may need to reflash the gateware and firmware of the core device to keep it synchronized with the software.
Switching between conda environments using ``$ source deactivate artiq-1.0rc2`` and ``$ source activate artiq-1.0rc1`` is the recommended way to roll back to previous versions of ARTIQ.
You can list the environments you have created using:: You can list the environments you have created using::
$ conda env list $ conda env list
See also the `conda documentation <http://conda.pydata.org/docs/using/envs.html>`_ for managing environments. Flashing gateware and firmware into the core device
---------------------------------------------------
Preparing the core device FPGA board .. note::
------------------------------------ If you have purchased a pre-assembled system from M-Labs or QUARTIQ, the gateware and firmware are already flashed and you can skip those steps, unless you want to replace them with a different version of ARTIQ.
You now need to write three binary images onto the FPGA board: You now need to write three binary images onto the FPGA board:
1. The FPGA gateware bitstream 1. The FPGA gateware bitstream
2. The BIOS 2. The bootloader
3. The ARTIQ runtime 3. The ARTIQ runtime or satellite manager
They are all shipped in the conda packages, along with the required flash proxy gateware bitstreams. They are all shipped in the Nix and Conda packages, along with the required flash proxy gateware bitstreams.
.. _install-openocd:
Installing OpenOCD Installing OpenOCD
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
OpenOCD can be used to write the binary images into the core device FPGA board's flash memory. OpenOCD can be used to write the binary images into the core device FPGA board's flash memory.
The ``artiq`` or ``artiq-dev`` conda packages install ``openocd`` automatically but it can also be installed explicitly using conda on both Linux and Windows::
With Nix, add ``m-labs.openocd`` to the shell packages. Be careful not to add ``pkgs.openocd`` instead - this would install OpenOCD from the NixOS package collection, which does not support ARTIQ boards.
With Conda, the ``artiq`` package installs ``openocd`` automatically but it can also be installed explicitly on both Linux and Windows::
$ conda install openocd $ conda install openocd
.. _setup-openocd:
Configuring OpenOCD Configuring OpenOCD
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
Some additional steps are necessary to ensure that OpenOCD can communicate with the FPGA board. Some additional steps are necessary to ensure that OpenOCD can communicate with the FPGA board.
On Linux, first ensure that the current user belongs to the ``plugdev`` group (i.e. `plugdev` shown when you run `$ groups`). If it does not, run ``sudo adduser $USER plugdev`` and relogin. If you installed OpenOCD using conda and are using the conda environment ``artiq-main``, then execute the statements below. If you are using a different environment, you will have to replace ``artiq-main`` with the name of your environment:: On Linux, first ensure that the current user belongs to the ``plugdev`` group (i.e. ``plugdev`` shown when you run ``$ groups``). If it does not, run ``sudo adduser $USER plugdev`` and re-login.
$ sudo cp ~/.conda/envs/artiq-main/share/openocd/contrib/60-openocd.rules /etc/udev/rules.d If you installed OpenOCD on Linux using Nix, use the ``which`` command to determine the path to OpenOCD, and then copy the udev rules: ::
$ sudo udevadm trigger
if you installed it from source:: Assuming you installed OpenOCD in ``/usr/local``, otherwise please substitute the install directory:: $ which openocd
/nix/store/2bmsssvk3d0y5hra06pv54s2324m4srs-openocd-mlabs-0.10.0/bin/openocd
$ sudo cp /nix/store/2bmsssvk3d0y5hra06pv54s2324m4srs-openocd-mlabs-0.10.0/share/openocd/contrib/60-openocd.rules /etc/udev/rules.d
$ sudo udevadm trigger
$ sudo cp /usr/local/share/openocd/contrib/60-openocd.rules /etc/udev/rules.d NixOS users should of course configure OpenOCD through ``/etc/nixos/configuration.nix`` instead.
If you installed OpenOCD on Linux using Conda and are using the Conda environment ``artiq``, then execute the statements below. If you are using a different environment, you will have to replace ``artiq`` with the name of your environment::
$ sudo cp ~/.conda/envs/artiq/share/openocd/contrib/60-openocd.rules /etc/udev/rules.d
$ sudo udevadm trigger $ sudo udevadm trigger
On Windows, a third-party tool, `Zadig <http://zadig.akeo.ie/>`_, is necessary. Use it as follows: On Windows, a third-party tool, `Zadig <http://zadig.akeo.ie/>`_, is necessary. Use it as follows:
@ -140,70 +182,71 @@ You may need to repeat these steps every time you plug the FPGA board into a por
.. _flashing-core-device: .. _flashing-core-device:
Flashing the core device Writing the flash
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
Then, you can flash the board: Then, you write the flash:
* For the KC705 board (selecting the appropriate hardware peripheral):: * For Kasli::
$ artiq_flash -V [your board variant]
* For the KC705 board::
$ artiq_flash -t kc705 -V [nist_clock/nist_qc2] $ artiq_flash -t kc705 -V [nist_clock/nist_qc2]
The SW13 switches also need to be set to 00001. The SW13 switches need to be set to 00001.
The next step is to flash the MAC and IP addresses to the board. See :ref:`those instructions <flash-mac-ip-addr>`. Setting up the core device IP networking
----------------------------------------
.. _configuring-core-device: For Kasli, insert a SFP/RJ45 transceiver (normally included with purchases from M-Labs and QUARTIQ) into the SFP0 port and connect it to a gigabit Ethernet port in your network. It is necessary that the port be gigabit - 10/100 ports cannot be used. If you need to interface Kasli with 10/100 network equipment, connect them through a gigabit switch.
Configuring the core device You can also insert other types of SFP transceivers into Kasli if you wish to use it directly in e.g. an optical fiber Ethernet network.
---------------------------
This should be done after either installation method (conda or source). If you purchased a device from M-Labs, it already comes with a valid MAC address and an IP address, usually ``192.168.1.75``. Once you can reach this IP, it can be changed with: ::
* (optional) If you are using DRTIO and the default routing table (for a star topology) is not suitable to your needs, prepare the routing table and add it to the ``flash_storage.img`` created in the next step. The routing table can be easily changed later, so you can skip this step if you are just getting started and only want to test local channels. See :ref:`Using DRTIO <using-drtio>`. artiq_coremgmt -D 192.168.1.75 config write -s ip [new IP]
.. _flash-mac-ip-addr: and then reboot the device (with ``artiq_flash start`` or a power cycle).
* Set the MAC and IP address in the :ref:`core device configuration flash storage <core-device-flash-storage>` (see above for the ``-t`` and ``-V`` options to ``artiq_flash`` that may be required): :: In other cases, install OpenOCD as before, and flash the IP and MAC addresses directly: ::
$ artiq_mkfs flash_storage.img -s mac xx:xx:xx:xx:xx:xx -s ip xx.xx.xx.xx $ artiq_mkfs flash_storage.img -s mac xx:xx:xx:xx:xx:xx -s ip xx.xx.xx.xx
$ artiq_flash -t [board] -V [adapter] -f flash_storage.img storage start $ artiq_flash -t [board] -V [adapter] -f flash_storage.img storage start
* (optional) Flash the idle kernel Check that you can ping the device. If ping fails, check that the Ethernet link LED is ON - on Kasli, it is the LED next to the SFP0 connector. As a next step, look at the messages emitted on the UART during boot. Use a program such as flterm or PuTTY to connect to the device's serial port at 115200bps 8-N-1 and reboot the device. On Kasli, the serial port is on FTDI channel 2 with v1.1 hardware (with channel 0 being JTAG) and on FTDI channel 1 with v1.0 hardware.
Miscellaneous configuration of the core device
----------------------------------------------
Those steps are optional. The core device usually needs to be restarted for changes to take effect.
* Load the idle kernel
The idle kernel is the kernel (some piece of code running on the core device) which the core device runs whenever it is not connected to a PC via Ethernet. The idle kernel is the kernel (some piece of code running on the core device) which the core device runs whenever it is not connected to a PC via Ethernet.
This kernel is therefore stored in the :ref:`core device configuration flash storage <core-device-flash-storage>`. This kernel is therefore stored in the :ref:`core device configuration flash storage <core-device-flash-storage>`.
To flash the idle kernel:
* Compile the idle experiment: To flash the idle kernel, first compile the idle experiment. The idle experiment's ``run()`` method must be a kernel: it must be decorated with the ``@kernel`` decorator (see :ref:`next topic <connecting-to-the-core-device>` for more information about kernels). Since the core device is not connected to the PC, RPCs (calling Python code running on the PC from the kernel) are forbidden in the idle experiment. Then write it into the core device configuration flash storage: ::
The idle experiment's ``run()`` method must be a kernel: it must be decorated with the ``@kernel`` decorator (see :ref:`next topic <connecting-to-the-core-device>` for more information about kernels).
Since the core device is not connected to the PC, RPCs (calling Python code running on the PC from the kernel) are forbidden in the idle experiment. $ artiq_compile idle.py
:: $ artiq_coremgmt config write -f idle_kernel idle.elf
$ artiq_compile idle.py
* Write it into the core device configuration flash storage: ::
$ artiq_coremgmt config -f idle_kernel idle.elf
.. note:: You can find more information about how to use the ``artiq_coremgmt`` utility on the :ref:`Utilities <core-device-management-tool>` page. .. note:: You can find more information about how to use the ``artiq_coremgmt`` utility on the :ref:`Utilities <core-device-management-tool>` page.
* (optional) Flash the startup kernel * Load the startup kernel
The startup kernel is executed once when the core device powers up. It should initialize DDSes, set up TTL directions, etc. Proceed as with the idle kernel, but using the ``startup_kernel`` key in the ``artiq_coremgmt`` command. The startup kernel is executed once when the core device powers up. It should initialize DDSes, set up TTL directions, etc. Proceed as with the idle kernel, but using the ``startup_kernel`` key in the ``artiq_coremgmt`` command.
For DRTIO systems, the startup kernel should wait until the desired destinations (including local RTIO) are up, using :meth:`artiq.coredevice.Core.get_rtio_destination_status`. For DRTIO systems, the startup kernel should wait until the desired destinations (including local RTIO) are up, using :meth:`artiq.coredevice.Core.get_rtio_destination_status`.
* (optional) Select the RTIO clock source * Load the DRTIO routing table
Some core devices may use either an external clock signal or their internal clock. The clock is selected at power-up. Use one of these commands: :: If you are using DRTIO and the default routing table (for a star topology) is not suitable to your needs, prepare and load a different routing table. See :ref:`Using DRTIO <using-drtio>`.
* Select the RTIO clock source (KC705 only)
The KC705 may use either an external clock signal or its internal clock. The clock is selected at power-up. Use one of these commands: ::
$ artiq_coremgmt config write -s rtio_clock i # internal clock (default) $ artiq_coremgmt config write -s rtio_clock i # internal clock (default)
$ artiq_coremgmt config write -s rtio_clock e # external clock $ artiq_coremgmt config write -s rtio_clock e # external clock
.. rubric:: Footnotes
.. [1] [Linux] If your shell does not find the ``conda`` command, make sure that the conda binaries are in your ``$PATH``:
If ``$ echo $PATH`` does not show the conda directories, add them: execute ``$ export PATH=$HOME/miniconda3/bin:$PATH`` if you installed conda into ``~/miniconda3``.