manual: management system tutorial, Git integration

doc/manual/getting_started_core.rst

@@ -1,5 +1,5 @@
-Getting started
-===============
+Getting started with the core language
+======================================
 
 .. _connecting-to-the-core-device:
 
@@ -23,7 +23,7 @@ As a very first step, we will turn on a LED on the core device. Create a file ``
 
 The central part of our code is our ``LED`` class, that derives from :class:`artiq.language.environment.EnvExperiment`. Among other features, ``EnvExperiment`` calls our ``build`` method and provides the ``attr_device`` method that interfaces to the device database to create the appropriate device drivers and make those drivers accessible as ``self.core`` and ``self.led``. The ``@kernel`` decorator tells the system that the ``run`` method must be executed on the core device (instead of the host). The decorator uses ``self.core`` internally, which is why we request the core device using ``attr_device`` like any other.
 
-Copy the files ``ddb.pyon`` and ``pdb.pyon`` (containing the device and parameter databases) from the ``examples`` folder of ARTIQ into the same directory as ``led.py`` (alternatively, you can use the ``-d`` and ``-p`` options of ``artiq_run.py``). You can open the database files using a text editor - their contents are in a human-readable format. You will probably want to set the IP address of the core device in ``ddb.pyon`` so that the computer can connect to it (it is the ``host`` parameter of the ``comm`` entry). See :ref:`ddb` for more information. The example device database is designed for the NIST QC1 hardware on the KC705; see :ref:`board-ports` for RTIO channel assignments if you need to adapt the device database to a different hardware platform.
+Copy the files ``ddb.pyon`` and ``pdb.pyon`` (containing the device and parameter databases) from the ``examples/master`` folder of ARTIQ into the same directory as ``led.py`` (alternatively, you can use the ``-d`` and ``-p`` options of ``artiq_run``). You can open the database files using a text editor - their contents are in a human-readable format. You will probably want to set the IP address of the core device in ``ddb.pyon`` so that the computer can connect to it (it is the ``host`` parameter of the ``comm`` entry). See :ref:`ddb` for more information. The example device database is designed for the NIST QC1 hardware on the KC705; see :ref:`board-ports` for RTIO channel assignments if you need to adapt the device database to a different hardware platform.
 
 Run your code using ``artiq_run``, which is part of the ARTIQ front-end tools: ::
 

doc/manual/getting_started_mgmt.rst

@@ -0,0 +1,138 @@
+Getting started with the management system
+==========================================
+
+The management system is the high-level part of ARTIQ that schedules the experiments, distributes and stores the results, and manages devices and parameters.
+
+The manipulations described in this tutorial can be carried out using a single computer, without any special hardware.
+
+Starting your first experiment with the master
+----------------------------------------------
+
+In the previous tutorial, we used the ``artiq_run`` utility to execute our experiments, which is a simple stand-alone tool that bypasses the ARTIQ management system. We will now see how to run an experiment using the master (the central program in the management system that schedules and executes experiments) and the GUI client (that connects to the master and controls it).
+
+First, create a folder ``~/artiq-master`` and copy the ``ddb.pyon`` and ``pdb.pyon`` files (device and parameter databases) found in the ``examples/master`` directory from the ARTIQ sources. The master uses those files in the same way as ``artiq_run``.
+
+Then create a ``~/artiq-master/repository`` sub-folder to contain experiments. The master scans this ``repository`` folder to determine what experiments are available (the name of the folder can be changed using ``-r``).
+
+Create a very simple experiment in ``~/artiq-master/repository`` and save it as ``mgmt_tutorial.py``: ::
+
+    from artiq import *
+
+
+    class MgmtTutorial(EnvExperiment):
+        """Management tutorial"""
+        def build(self):
+            pass  # no devices used
+
+        def run(self):
+            print("Hello World")
+
+
+Start the master with: ::
+    
+    $ cd ~/artiq-master
+    $ artiq_master
+
+This last command should not return, as the master keeps running.
+
+Now, start the GUI client with the following commands in another terminal: ::
+
+    $ cd ~
+    $ artiq_gui
+
+.. note:: The ``artiq_gui`` program uses a file called ``artiq_gui.pyon`` in the current directory to save and restore the GUI state (window/dock positions, last values entered by the user, etc.).
+
+The GUI should display the list of experiments from the repository folder in a dock called "Explorer". There should be only the experiment we created. Select it and click "Submit", lhen look at the "Log" dock for the output from this simple experiment.
+
+.. note:: Multiple clients may be connected at the same time, possibly on different machines, and will be synchronized. See the ``-s`` option of ``artiq_gui`` and the ``--bind`` option of ``artiq_master`` to use the network. Both IPv4 and IPv6 are supported.
+
+Adding an argument
+------------------
+
+Experiments may have arguments whose values can be set in the GUI and used in the experiment's code. Modify the experiment as follows: ::
+
+
+    def build(self):
+        self.attr_argument("count", NumberValue(ndecimals=0))
+
+    def run(self):
+        for i in range(int(self.count)):
+            print("Hello World", i)
+
+
+``NumberValue`` represents a floating point numeric argument. There are many other types, see :class:`artiq.language.environment` and :class:`artiq.language.scan`.
+
+Use the command-line client to trigger a repository rescan: ::
+
+    artiq_client scan-repository
+
+The GUI should now display a spin box that allows you to set the value of the ``count`` argument. Try submitting the experiment as before.
+
+Setting up Git integration
+--------------------------
+
+So far, we have used the bare filesystem for the experiment repository, without any version control. Using Git to host the experiment repository helps with the tracking of modifications to experiments and with the tracability of a result to a particular version of an experiment.
+
+.. note:: The workflow we will describe in this tutorial corresponds to a situation where the ARTIQ master machine is also used as a Git server where multiple users may push and pull code. The Git setup can be customized according to your needs; the main point to remember is that when scanning or submitting, the ARTIQ master uses the internal Git data (*not* any working directory that may be present) to fetch the latest *fully completed commit* at the repository's head.
+
+We will use the current ``repository`` folder as working directory for making local modifications to the experiments, move it away from the master data directory, and create a new ``repository`` folder that holds the Git data used by the master. Stop the master with Ctrl-C and enter the following commands: ::
+
+    $ cd ~/artiq-master
+    $ mv repository ~/artiq-work
+    $ mkdir repository
+    $ cd repository
+    $ git init --bare
+
+Start the master again with the ``-g`` flag, telling it to treat the contents of the ``repository`` folder as a bare Git repository: ::
+
+    $ cd ~/artiq-master
+    $ artiq_master -g
+
+There should be no errors displayed, and if you start the GUI again you should notice an empty experiment list. We will now add our previously written experiment to it.
+
+First, another small configuration step is needed. We must tell Git to make the master rescan the repository when new data is added to it. Create a file ``~/artiq-master/repository/hooks/post-receive`` with the following contents: ::
+
+   #!/bin/sh
+   artiq_client scan-repository
+
+Then set the execution permission on it: ::
+
+   $ chmod 755 ~/artiq-master/repository/hooks/post-receive
+
+The setup on the master side is now complete. All we need to do now is push data to into the bare repository. Initialize a regular (non-bare) Git repository into our working directory: ::
+
+    $ cd ~/artiq-work
+    $ git init    
+
+Then commit our experiment: ::
+
+    $ git add mgmt_tutorial.py
+    $ git commit -m "First version of the tutorial experiment"
+
+and finally, push the commit into the master's bare repository: ::
+
+    $ git remote add origin ~/artiq-master/repository
+    $ git push -u origin master
+
+The GUI should immediately list the experiment again, and you should be able to submit it as before.
+
+.. note:: Remote machines may also push and pull into the master's bare repository using e.g. Git over SSH.
+
+Let's now make a modification to the experiment. In the source present in the working directory, add an exclamation mark at the end of "Hello World". Before committing it, check that the experiment can still be executed correctly by running it directly from the filesystem using: ::
+
+    $ artiq_client submit ~/artiq-work/mgmt_tutorial.py
+
+.. note:: Submitting experiments outside the repository from the GUI is currently not supported. Submitting an experiment from the repository using the ``artiq_client`` command-line tool is done using the ``-R`` flag.
+
+Verify the log in the GUI. If you are happy with the result, commit the new version and push it into the master's repository: ::
+
+    $ cd ~/artiq-work
+    $ git commit -a -m "More enthusiasm"
+    $ git push
+
+.. note:: Notice that commands other than ``git push`` are not needed anymore.
+
+The master should now run the new version from its repository.
+
+As an exercise, add another argument to the experiment, commit and push the result, and verify that the new control is added in the GUI.
+

doc/manual/index.rst

@@ -8,10 +8,11 @@ Contents:
 
     introduction
     installing
-    getting_started
+    getting_started_core
+    getting_started_mgmt
+    core_device
     management_system
     environment
-    core_device
     core_language_reference
     core_drivers_reference
     protocols_reference