mwojcik
/
assembly
forked from sinara-hw/assembly
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Expand DRTIO related errors 

Signed-off-by: Egor Savkin <es@m-labs.hk>
This commit is contained in:
Egor Savkin 2024-10-17 13:46:12 +08:00
parent cf975f899e
commit b834726d92
2 changed files with 32 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
src/sw_sup/device_db.md
View File

@ -7,7 +7,7 @@ It is generated from JSON description file and tied with the configuration and t
In some cases you may need to regenerate `device_db.py`, like switching clock source or changing the configuration. In some cases you may need to regenerate `device_db.py`, like switching clock source or changing the configuration.
Also it is must-do in most cases once firmware/gateware is being updated (for example, when you add, move or remove EEM Also it is must-do in most cases once firmware/gateware is being updated (for example, when you add, move or remove EEM
cards). cards), and in case DRTIO layout changed.
Luckily, it is fairly easy to do. For standalone systems: Luckily, it is fairly easy to do. For standalone systems:
```shell ```shell
@ -19,3 +19,20 @@ For DRTIO systems:
```shell ```shell
artiq_ddb_template -o device_db.py -s 1 <satellite1>.json -s 2 <satellite2>.json <...> -s N <satelliteN>.json <master>.json artiq_ddb_template -o device_db.py -s 1 <satellite1>.json -s 2 <satellite2>.json <...> -s N <satelliteN>.json <master>.json
``` ```
Keep in mind, that for DRTIO systems the real SFP connections at master should match the numbers at
the `artiq_ddb_template` command, or routing table if specified.
Here is mapping for master Kasli 2.0 (without routing table):
* SFP0 - Ethernet
* SFP1 - Satellite 1
* SFP2 - Satellite 2
* SFP3 - Satellite 3
For master Kasli-SoC (without routing table):
* SFP0 - Satellite 1
* SFP1 - Satellite 2
* SFP2 - Satellite 3
* SFP3 - Satellite 4

17
src/sw_sup/drtio.md
View File

@ -11,9 +11,10 @@ or optical fibers SFP+ adapters (suitable for multiple crates that can be distri
[several kilometers](https://github.com/m-labs/artiq/issues/2022)). The DRTIO protocol is not compatible with Ethernet, [several kilometers](https://github.com/m-labs/artiq/issues/2022)). The DRTIO protocol is not compatible with Ethernet,
and moreover, satellites do not have any network access and can be controlled only by master. However, and moreover, satellites do not have any network access and can be controlled only by master. However,
both star (2 levels) and tree topologies are supported as well, with default one being the star (one master and up to both star (2 levels) and tree topologies are supported as well, with default one being the star (one master and up to
3-4 directly connected satellites), and if any chaining is needed, the routing table setup is needed. To switch between 3-4 directly connected satellites), and if any chaining is needed, the
satellite/master/standalone variants you just need to flash appropriate firmware, and set the respective `base` [routing table setup](https://m-labs.hk/artiq/manual/using_drtio_subkernels.html#configuring-the-routing-table)
field in the JSON description. is needed. To switch between satellite/master/standalone variants you just need to flash appropriate firmware,
and set the respective `base` field in the JSON description.
The master will attempt to connect the satellite whenever it sees that there are SFPs plugged in. For this purpose, The master will attempt to connect the satellite whenever it sees that there are SFPs plugged in. For this purpose,
it will _ping_ the satellite until it establishes the connection. This connection process can be observed from the logs: it will _ping_ the satellite until it establishes the connection. This connection process can be observed from the logs:
@ -38,6 +39,12 @@ crates to be synchronized.
### Master and satellite do not connect with each other ### Master and satellite do not connect with each other
During execution of experiments, may result in following error:
```pycon
artiq.coredevice.exceptions.RTIODestinationUnreachable: RTIO destination unreachable, output, at XXXXX mu, channel 0xXXX:DEV0
```
* Shady cables and SFP adapters are often the cause, use the adapters from reputable sources, or better, * Shady cables and SFP adapters are often the cause, use the adapters from reputable sources, or better,
use the one we ship. You may also contact our helpdesk to get help in choosing the right adapters if needed. use the one we ship. You may also contact our helpdesk to get help in choosing the right adapters if needed.
* The adapter is not pushed until the end. You shouldn't be able to pull out the adapters without * The adapter is not pushed until the end. You shouldn't be able to pull out the adapters without
@ -47,6 +54,10 @@ crates to be synchronized.
* Wrong setups - master to master, standalone to standalone. Messing up with SFP ports generally makes it unusable, * Wrong setups - master to master, standalone to standalone. Messing up with SFP ports generally makes it unusable,
but the connection should be established in most cases. but the connection should be established in most cases.
* The fiber adapters are not symmetrical - if one end has 1270/1330 label, another one should be 1330/1270. * The fiber adapters are not symmetrical - if one end has 1270/1330 label, another one should be 1330/1270.
* Connection race condition - rebooting one or both master and satellite may help.
* Mismatch with real SFP port and the one, specified during device_db generation: re-attach the SFP ports according to
device_db or regenerate device_db according to SFP port attachment.
[More info at the device_db article.](device_db.md)
### Master-satellite interrupted/unstable connection ### Master-satellite interrupted/unstable connection