Compare commits

..

1 Commits

Author SHA1 Message Date
Egor Savkin 5974dba3eb Small fix build
Signed-off-by: Egor Savkin <es@m-labs.hk>
2023-02-17 17:38:17 +08:00
94 changed files with 3025 additions and 13894 deletions

8
.gitignore vendored
View File

@ -3,11 +3,3 @@ examples/*.elf
__pycache__ __pycache__
build build
src/libboard_artiq/Cargo.toml
src/libc/Cargo.toml
src/libdyld/Cargo.toml
src/libio/Cargo.toml
src/libksupport/Cargo.toml
src/runtime/Cargo.toml
src/satman/Cargo.toml

View File

@ -4,107 +4,67 @@ ARTIQ on Zynq
How to use How to use
---------- ----------
1. [Install ARTIQ](https://m-labs.hk/artiq/manual/installing.html). Get the corresponding version to the ``artiq-zynq`` version you are targeting. 1. Install ARTIQ-7 or newer.
2. To obtain firmware binaries, use AFWS or build your own; see [the ARTIQ manual](https://m-labs.hk/artiq/manual/building_developing.html) for detailed instructions or skip to "Development" below. ZC706 variants only can also be downloaded from latest successful build on [Hydra](https://nixbld.m-labs.hk/). 2. Select the latest successful build on Hydra: https://nixbld.m-labs.hk/jobset/artiq/zynq
3. Place ``boot.bin`` file at the root ``/`` of a FAT-formatted SD card. 3. Search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``).
4. Optionally, create a ``config.txt`` configuration file containing ``key=value`` pairs on each line and place it at the root of the SD card. See below for valid keys. The ``ip``, ``ip6`` and ``mac`` keys can be used to set networking information. If these keys are not found, the firmware will use default values which may or may not be compatible with your network. 4. Download the ``boot.bin`` "binary distribution" and place it at the root of a FAT-formatted SD card.
5. Insert the SD card into the board and set the board to boot from the SD card. For ZC706, this is achieved by placing the large DIP switch SW11 into the 00110 position. On Kasli-SoC, place the BOOT MODE switches to SD. 5. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network.
6. Power up the board. After successful boot the firmware should respond to ping at its IP addresses. Boot output can be observed from UART at 115200bps 8-N-1. 6. Insert the SD card into the board and set up the board to boot from the SD card. For the ZC706, this is achieved by placing the large DIP switch SW11 in the 00110 position.
7. Create and use an ARTIQ device database as usual. 7. Power up the board. After the firmware starts successfully, it should respond to ping at its IP addresses, and boot messages can be observed from its UART at 115200bps.
8. Create and use an ARTIQ device database as usual, but set ``"target": "cortexa9"`` in the arguments of the core device.
Configuration Configuration
------------- -------------
Configuring the device is done using the ``config.txt`` text file at the root of the SD card plus optionally a ``config`` folder. When searching for a configuration key, the firmware first looks for a file named ``/config/[key].bin`` and, if it exists, returns the contents of that file. If not, it looks into ``/config.txt``, which should contain a list of ``key=value`` pairs, one per line. ``config.txt`` should be used for most keys but the ``config`` folder allows for setting configuration values which consist of binary data, such as the startup kernel. Configuring the device is done using the ``config.txt`` text file at the root of the SD card, plus the contents of the ``config`` folder. When searching for a configuration key, the firmware first looks for a file named ``/config/[key].bin`` and, if it exists, returns the contents of that file. If not, it looks into ``/config.txt``, which contains a list of ``key=value`` pairs, one per line. The ``config`` folder allows configuration values that consist in binary data, such as the startup kernel.
The following configuration keys are available among others: The following configuration keys are available:
- ``mac``: Ethernet MAC address. - ``mac``: Ethernet MAC address.
- ``ip``: IPv4 address. - ``ip``: IPv4 address.
- ``ip6``: IPv6 address. - ``ip6``: IPv6 address.
- ``idle_kernel``: idle kernel in ELF format (as produced by ``artiq_compile``). - ``startup``: startup kernel in ELF format (as produced by ``artiq_compile``).
- ``startup_kernel``: startup kernel in ELF format (as produced by ``artiq_compile``).
- ``rtio_clock``: source of RTIO clock; valid values are ``ext0_bypass`` and ``int_125``. - ``rtio_clock``: source of RTIO clock; valid values are ``ext0_bypass`` and ``int_125``.
- ``boot``: SD card "boot.bin" file, for replacing the boot firmware/gateware. Write only.
See [ARTIQ manual](https://m-labs.hk/artiq/manual-beta/core_device.html#configuration-storage) for full list. Configurations can be read/written/removed with ``artiq_coremgmt``. Config erase is not implemented, as it isn't particularly useful. Configurations can be read/written/removed via ``artiq_coremgmt``. Config erase is
not implemented as it seems not very useful.
For convenience, the ``boot`` key can be used with ``artiq_coremgmt`` and a ``boot.bin`` file to replace firmware/gateware in a running system. This key is read-only. When loading ``boot.bin`` onto the SD card directly, place it at the root and not in the ``config`` folder.
Development instructions Development instructions
------------------------ ------------------------
ARTIQ on Zynq is packaged using [Nix](https://nixos.org) Flakes. Install Nix 2.8+ and enable flakes by adding ``experimental-features = nix-command flakes`` to ``nix.conf`` (e.g. ``~/.config/nix/nix.conf``). ARTIQ on Zynq is packaged using the [Nix](https://nixos.org) Flakes system. Install Nix 2.8+ and enable flakes by adding ``experimental-features = nix-command flakes`` to ``nix.conf`` (e.g. ``~/.config/nix/nix.conf``).
**Pure build with Nix:** Pure build with Nix and execution on a remote JTAG server:
```shell ```shell
nix build .#zc706-nist_clock-jtag # or zc706-nist_qc2-jtag or zc706-nist_clock-sd or etc nix build .#zc706-nist_clock-jtag # or zc706-nist_qc2-jtag or zc706-nist_clock_satellite-jtag etc.
./remote_run.sh
``` ```
Run ``nix flake show`` to see all valid build targets. Targets suffixed with ``-jtag`` produce separate firmware and gateware files, intended for use in booting via JTAG server/Ethernet, e.g. ``./remote_run.sh -i`` with a remote JTAG server. Targets suffixed with ``-sd`` will produce ``boot.bin`` file suitable for SD card boot. ``-firmware`` and ``-gateware`` respectively build firmware and gateware only. Impure incremental build and execution on a remote JTAG server:
The Kasli-SoC target requires a system description file as input. See ARTIQ manual for exact instructions or use incremental build.
**Impure incremental build:**
For boards with fixed variants, i.e. ZC706, etc. :
```shell ```shell
nix develop nix develop
cd src cd src
gateware/<board>.py -g ../build/gateware -V <variant> # gateware gateware/zc706.py -g ../build/gateware -V <variant> # build gateware
make GWARGS="-V <variant>" <runtime/satman> # firmware make GWARGS="-V <variant>" <runtime/satman> # build firmware
``` cd ..
./remote_run.sh -i
For boards with system descriptions, i.e. Kasli-SoC, etc. :
```shell
nix develop
cd src
gateware/<board>.py -g ../build/gateware <description.json> # gateware
make TARGET=<board> GWARGS="path/to/description.json" <runtime/satman> # firmware
```
``szl.elf`` can be obtained with:
```shell
nix build git+https://git.m-labs.hk/m-labs/zynq-rs#<board>-szl
```
To generate ``boot.bin`` use ``mkbootimage``, e.g.:
```shell
echo "the_ROM_image:
{
[bootloader]result/szl.elf
gateware/top.bit
firmware/armv7-none-eabihf/release/<runtime/satman>
}
EOF" >> boot.bif
mkbootimage boot.bif boot.bin
``` ```
Notes: Notes:
- The impure build process is also compatible with non-Nix systems. - The impure build process is also compatible with non-Nix systems.
- When calling make, you need to specify both the variant and firmware type.
- Firmware type must be either ``runtime`` for DRTIO-less or DRTIO master variants, or ``satman`` for DRTIO satellite. - Firmware type must be either ``runtime`` for DRTIO-less or DRTIO master variants, or ``satman`` for DRTIO satellite.
- If the board is connected to the local machine by JTAG, use the ``local_run.sh`` script. - If the board is connected to the local machine, use the ``local_run.sh`` script.
- A known Xilinx hardware bug prevents repeatedly loading the bootloader over JTAG without a POR reset. If booting over JTAG, install a jumper on ``PS_POR_B`` and use the POR reset script [here](https://git.m-labs.hk/M-Labs/zynq-rs/src/branch/master/kasli_soc_por.py). - To update ``zynq-rs``, update the cargo files as per usual for Rust projects, but also keep ``flake.lock`` in sync.
Pre-Commit Hooks
----------------
You are strongly recommended to use the provided pre-commit hooks to automatically reformat files and check for non-optimal Rust/C/C++ practices. Run `pre-commit install` to install the hook and `pre-commit` will automatically run `cargo fmt`, `cargo clippy`, and `clang-format` for you.
Several things to note:
- If `cargo fmt`, `cargo clippy`, or `clang-format` returns an error, the pre-commit hook will fail. You should fix all errors before trying to commit again.
- If `cargo fmt` or `clang-format` reformats some files, the pre-commit hook will also fail. You should review the changes and, if satisfied, try to commit again.
License License
------- -------
Copyright (C) 2019-2024 M-Labs Limited. Copyright (C) 2019-2022 M-Labs Limited.
ARTIQ is free software: you can redistribute it and/or modify ARTIQ is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by it under the terms of the GNU Lesser General Public License as published by

View File

@ -59,14 +59,6 @@ device_db["ad9914dds1"] = {
"arguments": {"sysclk": 3e9, "bus_channel": 50, "channel": 1}, "arguments": {"sysclk": 3e9, "bus_channel": 50, "channel": 1},
} }
for i in range(4):
device_db["ttl"+str(i)+"_counter"] = {
"type": "local",
"module": "artiq.coredevice.edge_counter",
"class": "EdgeCounter",
"arguments": {"channel": 52+i}
}
# for ARTIQ test suite # for ARTIQ test suite
device_db.update( device_db.update(
loop_out="ttl0", loop_out="ttl0",

View File

@ -1,78 +0,0 @@
core_addr = "192.168.1.57"
device_db = {
"core": {
"type": "local",
"module": "artiq.coredevice.core",
"class": "Core",
"arguments": {
"host": core_addr,
"ref_period": 1e-9,
"target": "cortexa9",
},
},
"core_log": {
"type": "controller",
"host": "::1",
"port": 1068,
"command": "aqctl_corelog -p {port} --bind {bind} " + core_addr,
},
"core_moninj": {
"type": "controller",
"host": "::1",
"port_proxy": 1383,
"port": 1384,
"command": "aqctl_moninj_proxy --port-proxy {port_proxy} --port-control {port} --bind {bind} "
+ core_addr,
},
"core_analyzer": {
"type": "controller",
"host": "::1",
"port_proxy": 1385,
"port": 1386,
"command": "aqctl_coreanalyzer_proxy --port-proxy {port_proxy} --port-control {port} --bind {bind} "
+ core_addr,
},
"core_cache": {
"type": "local",
"module": "artiq.coredevice.cache",
"class": "CoreCache",
},
"core_dma": {"type": "local", "module": "artiq.coredevice.dma", "class": "CoreDMA"},
"led0": {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLOut",
"arguments": {"channel": 0},
},
"led1": {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLOut",
"arguments": {"channel": 1},
},
}
# TTLs starting at RTIO channel 2, ending at RTIO channel 15
for i in range(2, 16):
device_db["ttl" + str(i)] = {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLInOut",
"arguments": {"channel": i},
}
device_db.update(
spi0={
"type": "local",
"module": "artiq.coredevice.spi2",
"class": "SPIMaster",
"arguments": {"channel": 16},
},
dds0={
"type": "local",
"module": "artiq.coredevice.ad9834",
"class": "AD9834",
"arguments": {"spi_device": "spi0"},
},
)

View File

@ -3,19 +3,19 @@
"artiq": { "artiq": {
"inputs": { "inputs": {
"artiq-comtools": "artiq-comtools", "artiq-comtools": "artiq-comtools",
"mozilla-overlay": "mozilla-overlay",
"nixpkgs": "nixpkgs", "nixpkgs": "nixpkgs",
"rust-overlay": "rust-overlay",
"sipyco": "sipyco", "sipyco": "sipyco",
"src-migen": "src-migen", "src-migen": "src-migen",
"src-misoc": "src-misoc", "src-misoc": "src-misoc",
"src-pythonparser": "src-pythonparser" "src-pythonparser": "src-pythonparser"
}, },
"locked": { "locked": {
"lastModified": 1732066716, "lastModified": 1673758995,
"narHash": "sha256-krjvt9+RccnAxSEZcFhRpjA2S3CoqE4MSa1JUg421b4=", "narHash": "sha256-Nl00lPjySWyui12fGhU6/BiBZZVScI19ux3I+EGT4YM=",
"ref": "refs/heads/master", "ref": "master",
"rev": "270a417a28b516d36983779a1adb6d33a3c55a4a", "rev": "e9c65abebe8ce6912479b0a7334a813ae581458b",
"revCount": 9102, "revCount": 8305,
"type": "git", "type": "git",
"url": "https://github.com/m-labs/artiq.git" "url": "https://github.com/m-labs/artiq.git"
}, },
@ -37,11 +37,11 @@
] ]
}, },
"locked": { "locked": {
"lastModified": 1720768567, "lastModified": 1664405593,
"narHash": "sha256-3VoK7o5MtHtbHLrc6Pv+eQWFtaz5Gd/YWyV5TD3c5Ss=", "narHash": "sha256-yP441NerlLGig7n+9xHsx8yCtZ+Ggd0VqfBSzc20E04=",
"owner": "m-labs", "owner": "m-labs",
"repo": "artiq-comtools", "repo": "artiq-comtools",
"rev": "f93570d8f2ed5a3cfb3e1c16ab00f2540551e994", "rev": "15ddac62813ef623a076ccf982b3bc63d314e651",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -51,15 +51,12 @@
} }
}, },
"flake-utils": { "flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": { "locked": {
"lastModified": 1710146030, "lastModified": 1659877975,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=", "narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
"owner": "numtide", "owner": "numtide",
"repo": "flake-utils", "repo": "flake-utils",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a", "rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -68,18 +65,66 @@
"type": "github" "type": "github"
} }
}, },
"mozilla-overlay": {
"flake": false,
"locked": {
"lastModified": 1672878308,
"narHash": "sha256-0+fl6PHokhtSV+w58z2QD2rTf8QhcOGsT9o4LwHHZHE=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "d38863db88e100866b3e494a651ee4962b762fcc",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"mozilla-overlay_2": {
"flake": false,
"locked": {
"lastModified": 1672878308,
"narHash": "sha256-0+fl6PHokhtSV+w58z2QD2rTf8QhcOGsT9o4LwHHZHE=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "d38863db88e100866b3e494a651ee4962b762fcc",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"mozilla-overlay_3": {
"flake": false,
"locked": {
"lastModified": 1664789696,
"narHash": "sha256-UGWJHQShiwLCr4/DysMVFrYdYYHcOqAOVsWNUu+l6YU=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "80627b282705101e7b38e19ca6e8df105031b072",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"nixpkgs": { "nixpkgs": {
"locked": { "locked": {
"lastModified": 1731319897, "lastModified": 1673345971,
"narHash": "sha256-PbABj4tnbWFMfBp6OcUK5iGy1QY+/Z96ZcLpooIbuEI=", "narHash": "sha256-4DfFcKLRfVUTyuGrGNNmw37IeIZSoku9tgTVmu/iD98=",
"owner": "NixOS", "owner": "NixOS",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "dc460ec76cbff0e66e269457d7b728432263166c", "rev": "54644f409ab471e87014bb305eac8c50190bcf48",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "NixOS", "owner": "NixOS",
"ref": "nixos-unstable", "ref": "nixos-22.11",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }
@ -87,53 +132,10 @@
"root": { "root": {
"inputs": { "inputs": {
"artiq": "artiq", "artiq": "artiq",
"mozilla-overlay": "mozilla-overlay_2",
"zynq-rs": "zynq-rs" "zynq-rs": "zynq-rs"
} }
}, },
"rust-overlay": {
"inputs": {
"nixpkgs": [
"artiq",
"nixpkgs"
]
},
"locked": {
"lastModified": 1719454714,
"narHash": "sha256-MojqG0lyUINkEk0b3kM2drsU5vyaF8DFZe/FAlZVOGs=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "d1c527659cf076ecc4b96a91c702d080b213801e",
"type": "github"
},
"original": {
"owner": "oxalica",
"ref": "snapshot/2024-08-01",
"repo": "rust-overlay",
"type": "github"
}
},
"rust-overlay_2": {
"inputs": {
"nixpkgs": [
"zynq-rs",
"nixpkgs"
]
},
"locked": {
"lastModified": 1719454714,
"narHash": "sha256-MojqG0lyUINkEk0b3kM2drsU5vyaF8DFZe/FAlZVOGs=",
"owner": "oxalica",
"repo": "rust-overlay",
"rev": "d1c527659cf076ecc4b96a91c702d080b213801e",
"type": "github"
},
"original": {
"owner": "oxalica",
"ref": "snapshot/2024-08-01",
"repo": "rust-overlay",
"type": "github"
}
},
"sipyco": { "sipyco": {
"inputs": { "inputs": {
"nixpkgs": [ "nixpkgs": [
@ -142,11 +144,11 @@
] ]
}, },
"locked": { "locked": {
"lastModified": 1728371104, "lastModified": 1673433867,
"narHash": "sha256-PPnAyDedUQ7Og/Cby9x5OT9wMkNGTP8GS53V6N/dk4w=", "narHash": "sha256-a7Oq35YoDzPtISbqAsaT+2/v15HZ7G1q0ukXmKWdb7Q=",
"owner": "m-labs", "owner": "m-labs",
"repo": "sipyco", "repo": "sipyco",
"rev": "094a6cd63ffa980ef63698920170e50dc9ba77fd", "rev": "38f8f4185d7db6b68bd7f71546da9077b1e2561c",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -158,11 +160,11 @@
"src-migen": { "src-migen": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1727677091, "lastModified": 1673433200,
"narHash": "sha256-Zg3SQnTwMM/VkOGKogbPyuCC2NhLy8HB2SPEUWWNgCU=", "narHash": "sha256-ribBG06gsucz5oBS+O6aL8s2oJjx+qfl+vXmspts8gg=",
"owner": "m-labs", "owner": "m-labs",
"repo": "migen", "repo": "migen",
"rev": "c19ae9f8ae162ffe2d310a92bfce53ac2a821bc8", "rev": "f3e9145c9825514a1b4225378936569da4df8e12",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -174,11 +176,11 @@
"src-misoc": { "src-misoc": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1729234629, "lastModified": 1671158014,
"narHash": "sha256-TLsTCXV5AC2xh+bS7EhBVBKqdqIU3eKrnlWcFF9LtAM=", "narHash": "sha256-50w0K2E2ympYrG1Tte/HVbsp4FS2U+yohqZByXTOo4I=",
"ref": "refs/heads/master", "ref": "refs/heads/master",
"rev": "6085a312bca26adeca6584e37d08c8ba2e1d6e38", "rev": "26f039f9f6931a20a04ccd0f0a5402f67f553916",
"revCount": 2460, "revCount": 2436,
"submodules": true, "submodules": true,
"type": "git", "type": "git",
"url": "https://github.com/m-labs/misoc.git" "url": "https://github.com/m-labs/misoc.git"
@ -205,35 +207,20 @@
"type": "github" "type": "github"
} }
}, },
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"zynq-rs": { "zynq-rs": {
"inputs": { "inputs": {
"mozilla-overlay": "mozilla-overlay_3",
"nixpkgs": [ "nixpkgs": [
"artiq", "artiq",
"nixpkgs" "nixpkgs"
], ]
"rust-overlay": "rust-overlay_2"
}, },
"locked": { "locked": {
"lastModified": 1731749494, "lastModified": 1669819016,
"narHash": "sha256-WGigAhvVCGN5YZ1dHPyvoqAh47W1Gtph036O1aKFlLE=", "narHash": "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=",
"ref": "refs/heads/master", "ref": "master",
"rev": "12975de2e110d7948bf47b768559f727d0abc8fc", "rev": "67dbb5932fa8ff5f143983476f741f945871d286",
"revCount": 655, "revCount": 624,
"type": "git", "type": "git",
"url": "https://git.m-labs.hk/m-labs/zynq-rs" "url": "https://git.m-labs.hk/m-labs/zynq-rs"
}, },

124
flake.nix
View File

@ -2,26 +2,25 @@
description = "ARTIQ port to the Zynq-7000 platform"; description = "ARTIQ port to the Zynq-7000 platform";
inputs.artiq.url = git+https://github.com/m-labs/artiq.git; inputs.artiq.url = git+https://github.com/m-labs/artiq.git;
inputs.mozilla-overlay = { url = github:mozilla/nixpkgs-mozilla; flake = false; };
inputs.zynq-rs.url = git+https://git.m-labs.hk/m-labs/zynq-rs; inputs.zynq-rs.url = git+https://git.m-labs.hk/m-labs/zynq-rs;
inputs.zynq-rs.inputs.nixpkgs.follows = "artiq/nixpkgs"; inputs.zynq-rs.inputs.nixpkgs.follows = "artiq/nixpkgs";
outputs = { self, zynq-rs, artiq }: outputs = { self, mozilla-overlay, zynq-rs, artiq }:
let let
pkgs = import artiq.inputs.nixpkgs { system = "x86_64-linux"; overlays = [ (import zynq-rs.inputs.rust-overlay) ]; }; pkgs = import artiq.inputs.nixpkgs { system = "x86_64-linux"; overlays = [ (import mozilla-overlay) ]; };
zynqpkgs = zynq-rs.packages.x86_64-linux; zynqpkgs = zynq-rs.packages.x86_64-linux;
artiqpkgs = artiq.packages.x86_64-linux; artiqpkgs = artiq.packages.x86_64-linux;
llvmPackages_11 = zynq-rs.llvmPackages_11;
rust = zynq-rs.rust;
rustPlatform = zynq-rs.rustPlatform; rustPlatform = zynq-rs.rustPlatform;
fastnumbers = pkgs.python3Packages.buildPythonPackage rec { fastnumbers = pkgs.python3Packages.buildPythonPackage rec {
pname = "fastnumbers"; pname = "fastnumbers";
version = "5.1.0"; version = "2.2.1";
src = pkgs.python3Packages.fetchPypi { src = pkgs.python3Packages.fetchPypi {
inherit pname version; inherit pname version;
sha256 = "sha256-4JLTP4uVwxcaL7NOV57+DFSwKQ3X+W/6onYkN2AdkKc="; sha256 = "0j15i54p7nri6hkzn1wal9pxri4pgql01wgjccig6ar0v5jjbvsy";
}; };
}; };
@ -50,7 +49,7 @@
nativeBuildInputs = with pkgs.python3Packages; [ pbr ]; nativeBuildInputs = with pkgs.python3Packages; [ pbr ];
propagatedBuildInputs = with pkgs.python3Packages; [ future fastnumbers ]; propagatedBuildInputs = with pkgs.python3Packages; [ future fastnumbers ];
checkInputs = with pkgs.python3Packages; [ pytest ]; checkInputs = with pkgs.python3Packages; [ pytest pytest-flake8 ];
checkPhase = "pytest"; checkPhase = "pytest";
doCheck = false; doCheck = false;
@ -74,17 +73,15 @@
propagatedBuildInputs = with pkgs.python3Packages; [ setuptools click numpy toolz jinja2 ramda artiqpkgs.migen artiqpkgs.misoc ]; propagatedBuildInputs = with pkgs.python3Packages; [ setuptools click numpy toolz jinja2 ramda artiqpkgs.migen artiqpkgs.misoc ];
checkInputs = with pkgs.python3Packages; [ pytestCheckHook pytest-timeout ]; checkInputs = with pkgs.python3Packages; [ pytest pytest-timeout pytest-flake8 ];
checkPhase = "pytest";
# migen/misoc version checks are broken with pyproject for some reason # migen/misoc version checks are broken with pyproject for some reason
postPatch = '' postPatch = ''
sed -i "1,4d" pyproject.toml
substituteInPlace pyproject.toml \ substituteInPlace pyproject.toml \
--replace '"migen@git+https://github.com/m-labs/migen",' "" --replace '"migen@git+https://github.com/m-labs/migen",' ""
substituteInPlace pyproject.toml \ substituteInPlace pyproject.toml \
--replace '"misoc@git+https://github.com/m-labs/misoc.git",' "" --replace '"misoc@git+https://github.com/m-labs/misoc.git",' ""
# pytest-flake8 is broken with recent flake8. Re-enable after fix.
substituteInPlace setup.cfg --replace '--flake8' ""
''; '';
}; };
binutils = { platform, target, zlib }: pkgs.stdenv.mkDerivation rec { binutils = { platform, target, zlib }: pkgs.stdenv.mkDerivation rec {
@ -113,34 +110,34 @@
"nist_clock_satellite" "nist_qc2_satellite" "acpki_nist_clock_satellite" "acpki_nist_qc2_satellite" "nist_clock_satellite" "nist_qc2_satellite" "acpki_nist_clock_satellite" "acpki_nist_qc2_satellite"
"nist_clock_satellite_100mhz" "nist_qc2_satellite_100mhz" "acpki_nist_clock_satellite_100mhz" "acpki_nist_qc2_satellite_100mhz" "nist_clock_satellite_100mhz" "nist_qc2_satellite_100mhz" "acpki_nist_clock_satellite_100mhz" "acpki_nist_qc2_satellite_100mhz"
]; ];
board-package-set = { target, variant, json ? null }: let build = { target, variant, json ? null }: let
szl = zynqpkgs."${target}-szl"; szl = zynqpkgs."${target}-szl";
fsbl = zynqpkgs."${target}-fsbl"; fsbl = zynqpkgs."${target}-fsbl";
fwtype = if builtins.elem variant sat_variants then "satman" else "runtime"; fwtype = if builtins.elem variant sat_variants then "satman" else "runtime";
firmware = rustPlatform.buildRustPackage rec { firmware = rustPlatform.buildRustPackage rec {
name = "firmware"; name = "firmware";
src = ./src; src = ./src;
cargoLock = { cargoLock = {
lockFile = src/Cargo.lock; lockFile = src/Cargo.lock;
outputHashes = { outputHashes = {
"tar-no-std-0.1.8" = "sha256-xm17108v4smXOqxdLvHl9CxTCJslmeogjm4Y87IXFuM="; "libasync-0.0.0" = "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=";
"nalgebra-0.32.6" = "sha256-L/YudkVOtfGYoNQKBD7LMk/sMYgRDzPDdpGL5rO7G2I=";
}; };
}; };
nativeBuildInputs = [ nativeBuildInputs = [
pkgs.gnumake pkgs.gnumake
(pkgs.python3.withPackages(ps: [ ps.jsonschema artiqpkgs.migen migen-axi artiqpkgs.misoc artiqpkgs.artiq ])) (pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
zynqpkgs.cargo-xbuild zynqpkgs.cargo-xbuild
llvmPackages_11.llvm pkgs.llvmPackages_9.llvm
llvmPackages_11.clang-unwrapped pkgs.llvmPackages_9.clang-unwrapped
]; ];
buildPhase = '' buildPhase = ''
export XARGO_RUST_SRC="${rust}/lib/rustlib/src/rust/library" export XARGO_RUST_SRC="${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library"
export CLANG_EXTRA_INCLUDE_DIR="${llvmPackages_11.clang-unwrapped.lib}/lib/clang/11.1.0/include" export CLANG_EXTRA_INCLUDE_DIR="${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include"
export CARGO_HOME=$(mktemp -d cargo-home.XXX) export CARGO_HOME=$(mktemp -d cargo-home.XXX)
export ZYNQ_RS=${zynq-rs}
make TARGET=${target} GWARGS="${if json == null then "-V ${variant}" else json}" ${fwtype} make TARGET=${target} GWARGS="${if json == null then "-V ${variant}" else json}" ${fwtype}
''; '';
@ -154,12 +151,12 @@
doCheck = false; doCheck = false;
dontFixup = true; dontFixup = true;
auditable = false;
}; };
gateware = pkgs.runCommand "${target}-${variant}-gateware" gateware = pkgs.runCommand "${target}-${variant}-gateware"
{ {
nativeBuildInputs = [ nativeBuildInputs = [
(pkgs.python3.withPackages(ps: [ ps.jsonschema artiqpkgs.migen migen-axi artiqpkgs.misoc artiqpkgs.artiq ])) (pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
artiqpkgs.vivado artiqpkgs.vivado
]; ];
} }
@ -243,7 +240,8 @@
name = "gateware-sim"; name = "gateware-sim";
nativeBuildInputs = [ nativeBuildInputs = [
(pkgs.python3.withPackages(ps: [ artiqpkgs.migen migen-axi artiqpkgs.artiq ])) (pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi artiq ])))
artiqpkgs.artiq
]; ];
phases = [ "buildPhase" ]; phases = [ "buildPhase" ];
@ -255,26 +253,8 @@
''; '';
}; };
fmt-check = pkgs.stdenvNoCC.mkDerivation {
name = "fmt-check";
src = ./src;
nativeBuildInputs = [ rust pkgs.gnumake ];
phases = [ "unpackPhase" "buildPhase" ];
buildPhase =
''
export ZYNQ_RS=${zynq-rs}
make manifests
cargo fmt -- --check
touch $out
'';
};
# for hitl-tests # for hitl-tests
zc706-nist_qc2 = (board-package-set { target = "zc706"; variant = "nist_qc2"; }); zc706-nist_qc2 = (build { target = "zc706"; variant = "nist_qc2"; });
zc706-hitl-tests = pkgs.stdenv.mkDerivation { zc706-hitl-tests = pkgs.stdenv.mkDerivation {
name = "zc706-hitl-tests"; name = "zc706-hitl-tests";
@ -341,39 +321,35 @@
{ {
inherit fastnumbers artiq-netboot ramda migen-axi binutils-arm; inherit fastnumbers artiq-netboot ramda migen-axi binutils-arm;
} // } //
(board-package-set { target = "zc706"; variant = "nist_clock"; }) // (build { target = "zc706"; variant = "nist_clock"; }) //
(board-package-set { target = "zc706"; variant = "nist_clock_master"; }) // (build { target = "zc706"; variant = "nist_clock_master"; }) //
(board-package-set { target = "zc706"; variant = "nist_clock_master_100mhz"; }) // (build { target = "zc706"; variant = "nist_clock_satellite"; }) //
(board-package-set { target = "zc706"; variant = "nist_clock_satellite"; }) // (build { target = "zc706"; variant = "nist_clock_satellite_100mhz"; }) //
(board-package-set { target = "zc706"; variant = "nist_clock_satellite_100mhz"; }) // (build { target = "zc706"; variant = "nist_qc2"; }) //
(board-package-set { target = "zc706"; variant = "nist_qc2"; }) // (build { target = "zc706"; variant = "nist_qc2_master"; }) //
(board-package-set { target = "zc706"; variant = "nist_qc2_master"; }) // (build { target = "zc706"; variant = "nist_qc2_satellite"; }) //
(board-package-set { target = "zc706"; variant = "nist_qc2_master_100mhz"; }) // (build { target = "zc706"; variant = "nist_qc2_satellite_100mhz"; }) //
(board-package-set { target = "zc706"; variant = "nist_qc2_satellite"; }) // (build { target = "zc706"; variant = "acpki_nist_clock"; }) //
(board-package-set { target = "zc706"; variant = "nist_qc2_satellite_100mhz"; }) // (build { target = "zc706"; variant = "acpki_nist_clock_master"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_clock"; }) // (build { target = "zc706"; variant = "acpki_nist_clock_satellite"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_clock_master"; }) // (build { target = "zc706"; variant = "acpki_nist_clock_satellite_100mhz"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_clock_master_100mhz"; }) // (build { target = "zc706"; variant = "acpki_nist_qc2"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_clock_satellite"; }) // (build { target = "zc706"; variant = "acpki_nist_qc2_master"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_clock_satellite_100mhz"; }) // (build { target = "zc706"; variant = "acpki_nist_qc2_satellite"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_qc2"; }) // (build { target = "zc706"; variant = "acpki_nist_qc2_satellite_100mhz"; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_qc2_master"; }) // (build { target = "kasli_soc"; variant = "demo"; json = ./demo.json; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_qc2_master_100mhz"; }) // (build { target = "kasli_soc"; variant = "master"; json = ./kasli-soc-master.json; }) //
(board-package-set { target = "zc706"; variant = "acpki_nist_qc2_satellite"; }) // (build { target = "kasli_soc"; variant = "satellite"; json = ./kasli-soc-satellite.json; });
(board-package-set { target = "zc706"; variant = "acpki_nist_qc2_satellite_100mhz"; }) //
(board-package-set { target = "kasli_soc"; variant = "demo"; json = ./demo.json; }) //
(board-package-set { target = "kasli_soc"; variant = "master"; json = ./kasli-soc-master.json; }) //
(board-package-set { target = "kasli_soc"; variant = "satellite"; json = ./kasli-soc-satellite.json; }) //
(board-package-set { target = "ebaz4205"; variant = "base"; });
hydraJobs = packages.x86_64-linux // { inherit zc706-hitl-tests; inherit gateware-sim; inherit fmt-check; }; hydraJobs = packages.x86_64-linux // { inherit zc706-hitl-tests; inherit gateware-sim; };
devShell.x86_64-linux = pkgs.mkShell { devShell.x86_64-linux = pkgs.mkShell {
name = "artiq-zynq-dev-shell"; name = "artiq-zynq-dev-shell";
buildInputs = with pkgs; [ buildInputs = with pkgs; [
rust rustPlatform.rust.rustc
llvmPackages_11.llvm rustPlatform.rust.cargo
llvmPackages_11.clang-unwrapped llvmPackages_9.llvm
llvmPackages_9.clang-unwrapped
gnumake gnumake
cacert cacert
zynqpkgs.cargo-xbuild zynqpkgs.cargo-xbuild
@ -384,16 +360,14 @@
artiqpkgs.artiq artiqpkgs.artiq
artiqpkgs.vivado artiqpkgs.vivado
binutils-arm binutils-arm
pre-commit
]; ];
XARGO_RUST_SRC = "${rust}/lib/rustlib/src/rust/library"; XARGO_RUST_SRC = "${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library";
CLANG_EXTRA_INCLUDE_DIR = "${llvmPackages_11.clang-unwrapped.lib}/lib/clang/11.1.0/include"; CLANG_EXTRA_INCLUDE_DIR = "${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include";
ZYNQ_RS = "${zynq-rs}";
OPENOCD_ZYNQ = "${zynq-rs}/openocd"; OPENOCD_ZYNQ = "${zynq-rs}/openocd";
SZL = "${zynqpkgs.szl}"; SZL = "${zynqpkgs.szl}";
}; };
makeArtiqZynqPackage = board-package-set; makeArtiqZynqPackage = build;
}; };
} }

View File

@ -1,32 +0,0 @@
BasedOnStyle: LLVM
Language: Cpp
Standard: Cpp11
AccessModifierOffset: -1
AlignEscapedNewlines: Left
AlwaysBreakAfterReturnType: None
AlwaysBreakTemplateDeclarations: Yes
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortFunctionsOnASingleLine: Inline
BinPackParameters: false
BreakBeforeBinaryOperators: NonAssignment
BreakBeforeTernaryOperators: true
BreakConstructorInitializers: AfterColon
BreakInheritanceList: AfterColon
ColumnLimit: 120
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ContinuationIndentWidth: 4
DerivePointerAlignment: false
IndentCaseLabels: true
IndentPPDirectives: None
IndentWidth: 4
MaxEmptyLinesToKeep: 1
PointerAlignment: Left
ReflowComments: true
SortIncludes: false
SortUsingDeclarations: true
SpaceAfterTemplateKeyword: false
SpacesBeforeTrailingComments: 2
TabWidth: 4
UseTab: Never

View File

@ -1 +0,0 @@
doc-valid-idents = ["CPython", "NumPy", ".."]

View File

@ -1,32 +0,0 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
default_stages: [commit]
repos:
- repo: local
hooks:
- id: cargo-fmt
name: artiq-zynq cargo format
entry: nix
language: system
types: [file, rust]
pass_filenames: false
description: Runs cargo fmt on the codebase.
args: [develop, -c, cargo, fmt, --manifest-path, src/Cargo.toml, --all]
- id: cargo-clippy
name: artiq-zynq cargo clippy
entry: nix
language: system
types: [file, rust]
pass_filenames: false
description: Runs cargo clippy on the codebase.
args: [develop, -c, cargo, clippy, --manifest-path, src/Cargo.toml, --tests]
- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v19.1.0
hooks:
- id: clang-format
name: artiq-zynq clang-format
description: Runs clang-format on the codebase.
files: \.(cpp|h|hpp|c)$
args: [-style=file, -fallback-style=none, -assume-filename=src/.clang-format]

138
src/Cargo.lock generated
View File

@ -2,21 +2,6 @@
# It is not intended for manual editing. # It is not intended for manual editing.
version = 3 version = 3
[[package]]
name = "approx"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cab112f0a86d568ea0e627cc1d6be74a1e9cd55214684db5561995f6dad897c6"
dependencies = [
"num-traits",
]
[[package]]
name = "arrayvec"
version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96d30a06541fbafbc7f82ed10c06164cfbd2c401138f6addd8404629c4b16711"
[[package]] [[package]]
name = "async-recursion" name = "async-recursion"
version = "0.3.2" version = "0.3.2"
@ -233,38 +218,10 @@ dependencies = [
"libsupport_zynq", "libsupport_zynq",
] ]
[[package]]
name = "ksupport"
version = "0.1.0"
dependencies = [
"build_zynq",
"byteorder",
"core_io",
"cslice",
"dwarf",
"dyld",
"io",
"libasync",
"libboard_artiq",
"libboard_zynq",
"libc",
"libconfig",
"libcortex_a9",
"libm",
"libregister",
"libsupport_zynq",
"log",
"log_buffer",
"nalgebra",
"nb 0.1.3",
"unwind",
"vcell",
"void",
]
[[package]] [[package]]
name = "libasync" name = "libasync"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"embedded-hal", "embedded-hal",
"libcortex_a9", "libcortex_a9",
@ -287,7 +244,6 @@ dependencies = [
"libconfig", "libconfig",
"libcortex_a9", "libcortex_a9",
"libregister", "libregister",
"libsupport_zynq",
"log", "log",
"log_buffer", "log_buffer",
"nb 1.0.0", "nb 1.0.0",
@ -297,6 +253,7 @@ dependencies = [
[[package]] [[package]]
name = "libboard_zynq" name = "libboard_zynq"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"embedded-hal", "embedded-hal",
@ -321,6 +278,7 @@ dependencies = [
[[package]] [[package]]
name = "libconfig" name = "libconfig"
version = "0.1.0" version = "0.1.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"core_io", "core_io",
"fatfs", "fatfs",
@ -331,6 +289,7 @@ dependencies = [
[[package]] [[package]]
name = "libcortex_a9" name = "libcortex_a9"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"libregister", "libregister",
@ -346,6 +305,7 @@ checksum = "348108ab3fba42ec82ff6e9564fc4ca0247bdccdc68dd8af9764bbc79c3c8ffb"
[[package]] [[package]]
name = "libregister" name = "libregister"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"vcell", "vcell",
@ -355,6 +315,7 @@ dependencies = [
[[package]] [[package]]
name = "libsupport_zynq" name = "libsupport_zynq"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [ dependencies = [
"cc", "cc",
"compiler_builtins", "compiler_builtins",
@ -392,19 +353,6 @@ version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c75de51135344a4f8ed3cfe2720dc27736f7711989703a0b43aadf3753c55577" checksum = "c75de51135344a4f8ed3cfe2720dc27736f7711989703a0b43aadf3753c55577"
[[package]]
name = "nalgebra"
version = "0.32.6"
source = "git+https://git.m-labs.hk/M-Labs/nalgebra.git?rev=dd00f9b#dd00f9b46046e0b931d1b470166db02fd29591be"
dependencies = [
"approx",
"num-complex",
"num-rational",
"num-traits",
"simba",
"typenum",
]
[[package]] [[package]]
name = "nb" name = "nb"
version = "0.1.3" version = "0.1.3"
@ -420,15 +368,6 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "546c37ac5d9e56f55e73b677106873d9d9f5190605e41a856503623648488cae" checksum = "546c37ac5d9e56f55e73b677106873d9d9f5190605e41a856503623648488cae"
[[package]]
name = "num-complex"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "26873667bbbb7c5182d4a37c1add32cdf09f841af72da53318fdb81543c15085"
dependencies = [
"num-traits",
]
[[package]] [[package]]
name = "num-derive" name = "num-derive"
version = "0.3.3" version = "0.3.3"
@ -440,26 +379,6 @@ dependencies = [
"syn", "syn",
] ]
[[package]]
name = "num-integer"
version = "0.1.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7969661fd2958a5cb096e56c8e1ad0444ac2bbcd0061bd28660485a44879858f"
dependencies = [
"num-traits",
]
[[package]]
name = "num-rational"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d41702bd167c2df5520b384281bc111a4b5efcf7fbc4c9c222c815b07e0a6a6a"
dependencies = [
"autocfg",
"num-integer",
"num-traits",
]
[[package]] [[package]]
name = "num-traits" name = "num-traits"
version = "0.2.15" version = "0.2.15"
@ -467,15 +386,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "578ede34cf02f8924ab9447f50c28075b4d3e5b269972345e7e0372b38c6cdcd" checksum = "578ede34cf02f8924ab9447f50c28075b4d3e5b269972345e7e0372b38c6cdcd"
dependencies = [ dependencies = [
"autocfg", "autocfg",
"libm",
] ]
[[package]]
name = "paste"
version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]] [[package]]
name = "pin-project-lite" name = "pin-project-lite"
version = "0.2.9" version = "0.2.9"
@ -526,20 +438,20 @@ dependencies = [
"embedded-hal", "embedded-hal",
"futures", "futures",
"io", "io",
"ksupport",
"libasync", "libasync",
"libboard_artiq", "libboard_artiq",
"libboard_zynq", "libboard_zynq",
"libc", "libc",
"libconfig", "libconfig",
"libcortex_a9", "libcortex_a9",
"libm",
"libregister", "libregister",
"libsupport_zynq", "libsupport_zynq",
"log", "log",
"log_buffer", "log_buffer",
"nb 0.1.3",
"num-derive", "num-derive",
"num-traits", "num-traits",
"tar-no-std",
"unwind", "unwind",
"vcell", "vcell",
"void", "void",
@ -559,13 +471,7 @@ name = "satman"
version = "0.0.0" version = "0.0.0"
dependencies = [ dependencies = [
"build_zynq", "build_zynq",
"byteorder",
"core_io",
"crc",
"cslice",
"embedded-hal", "embedded-hal",
"io",
"ksupport",
"libasync", "libasync",
"libboard_artiq", "libboard_artiq",
"libboard_zynq", "libboard_zynq",
@ -584,18 +490,6 @@ version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d4f410fedcf71af0345d7607d246e7ad15faaadd49d240ee3b24e5dc21a820ac" checksum = "d4f410fedcf71af0345d7607d246e7ad15faaadd49d240ee3b24e5dc21a820ac"
[[package]]
name = "simba"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "50582927ed6f77e4ac020c057f37a268fc6aebc29225050365aacbb9deeeddc4"
dependencies = [
"approx",
"num-complex",
"num-traits",
"paste",
]
[[package]] [[package]]
name = "smoltcp" name = "smoltcp"
version = "0.7.5" version = "0.7.5"
@ -618,22 +512,6 @@ dependencies = [
"unicode-ident", "unicode-ident",
] ]
[[package]]
name = "tar-no-std"
version = "0.1.8"
source = "git+https://git.m-labs.hk/M-Labs/tar-no-std?rev=2ab6dc5#2ab6dc58e5249c59c4eb03eaf3a119bcdd678d32"
dependencies = [
"arrayvec",
"bitflags",
"log",
]
[[package]]
name = "typenum"
version = "1.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42ff0bf0c66b8238c6f3b578df37d0b7848e55df8577b3f74f92a69acceeb825"
[[package]] [[package]]
name = "unicode-ident" name = "unicode-ident"
version = "1.0.5" version = "1.0.5"

View File

@ -5,7 +5,6 @@ members = [
"libdwarf", "libdwarf",
"libio", "libio",
"libunwind", "libunwind",
"libksupport",
"runtime", "runtime",
"satman" "satman"
] ]

View File

@ -7,20 +7,13 @@ runtime: ../build/runtime.bin
satman: ../build/satman.bin satman: ../build/satman.bin
.PHONY: all manifests .PHONY: all runtime_target satman_target
manifests = libboard_artiq/Cargo.toml libc/Cargo.toml libdyld/Cargo.toml libio/Cargo.toml libksupport/Cargo.toml runtime/Cargo.toml satman/Cargo.toml
$(manifests): %.toml: %.toml.tpl
sed s+@@ZYNQ_RS@@+$(ZYNQ_RS)+g $< > $@
manifests: $(manifests)
../build/pl.rs ../build/rustc-cfg ../build/mem.rs: gateware/* ../build/pl.rs ../build/rustc-cfg ../build/mem.rs: gateware/*
mkdir -p ../build mkdir -p ../build
python gateware/$(TARGET).py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs $(GWARGS) python gateware/$(TARGET).py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs $(GWARGS)
../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(manifests) $(shell find . -type f -not -name Cargo.toml -print) ../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print)
cd runtime && \ cd runtime && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \ XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \ cargo xbuild --release \
@ -30,7 +23,7 @@ manifests: $(manifests)
../build/runtime.bin: ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin: ../build/firmware/armv7-none-eabihf/release/runtime
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin
../build/firmware/armv7-none-eabihf/release/satman: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(manifests) $(shell find . -type f -not -name Cargo.toml -print) ../build/firmware/armv7-none-eabihf/release/satman: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print)
cd satman && \ cd satman && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \ XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \ cargo xbuild --release \

View File

@ -1,16 +0,0 @@
from misoc.integration import cpu_interface
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_rust_cfg(
soc.get_csr_regions(), soc.get_constants()))

View File

@ -1,119 +0,0 @@
from migen import *
from migen.genlib.cdc import PulseSynchronizer, MultiReg
from misoc.interconnect.csr import *
class DDMTDSampler(Module):
def __init__(self, cd_ref, main_clk_se):
self.ref_beating = Signal()
self.main_beating = Signal()
# # #
ref_clk = Signal()
self.specials +=[
# ISERDESE2 can only be driven from fabric via IDELAYE2 (see UG471)
Instance("IDELAYE2",
p_DELAY_SRC="DATAIN",
p_HIGH_PERFORMANCE_MODE="TRUE",
p_REFCLK_FREQUENCY=208.3, # REFCLK frequency from IDELAYCTRL
p_IDELAY_VALUE=0,
i_DATAIN=cd_ref.clk,
o_DATAOUT=ref_clk
),
Instance("ISERDESE2",
p_IOBDELAY="IFD", # use DDLY as input
p_DATA_RATE="SDR",
p_DATA_WIDTH=2, # min is 2
p_NUM_CE=1,
i_DDLY=ref_clk,
i_CE1=1,
i_CLK=ClockSignal("helper"),
i_CLKDIV=ClockSignal("helper"),
o_Q1=self.ref_beating
),
Instance("ISERDESE2",
p_DATA_RATE="SDR",
p_DATA_WIDTH=2, # min is 2
p_NUM_CE=1,
i_D=main_clk_se,
i_CE1=1,
i_CLK=ClockSignal("helper"),
i_CLKDIV=ClockSignal("helper"),
o_Q1=self.main_beating,
),
]
class DDMTDDeglitcherMedianEdge(Module):
def __init__(self, counter, input_signal, stable_0_period=100, stable_1_period=100):
self.tag = Signal(len(counter))
self.detect = Signal()
stable_0_counter = Signal(reset=stable_0_period - 1, max=stable_0_period)
stable_1_counter = Signal(reset=stable_1_period - 1, max=stable_1_period)
# # #
# Based on CERN's median edge deglitcher FSM
# https://white-rabbit.web.cern.ch/documents/Precise_time_and_frequency_transfer_in_a_White_Rabbit_network.pdf (p.72)
fsm = ClockDomainsRenamer("helper")(FSM(reset_state="WAIT_STABLE_0"))
self.submodules += fsm
fsm.act("WAIT_STABLE_0",
If(stable_0_counter != 0,
NextValue(stable_0_counter, stable_0_counter - 1)
).Else(
NextValue(stable_0_counter, stable_0_period - 1),
NextState("WAIT_EDGE")
),
If(input_signal,
NextValue(stable_0_counter, stable_0_period - 1)
),
)
fsm.act("WAIT_EDGE",
If(input_signal,
NextValue(self.tag, counter),
NextState("GOT_EDGE")
)
)
fsm.act("GOT_EDGE",
If(stable_1_counter != 0,
NextValue(stable_1_counter, stable_1_counter - 1)
).Else(
NextValue(stable_1_counter, stable_1_period - 1),
self.detect.eq(1),
NextState("WAIT_STABLE_0")
),
If(~input_signal,
NextValue(self.tag, self.tag + 1),
NextValue(stable_1_counter, stable_1_period - 1)
),
)
class DDMTD(Module):
def __init__(self, counter, input_signal):
# in helper clock domain
self.h_tag = Signal(len(counter))
self.h_tag_update = Signal()
# # #
deglitcher = DDMTDDeglitcherMedianEdge(counter, input_signal)
self.submodules += deglitcher
self.sync.helper += [
self.h_tag_update.eq(0),
If(deglitcher.detect,
self.h_tag_update.eq(1),
self.h_tag.eq(deglitcher.tag)
)
]

View File

@ -1,12 +1,12 @@
"""Auxiliary controller, common to satellite and master""" """Auxiliary controller, common to satellite and master"""
from artiq.gateware.drtio.aux_controller import (max_packet, aux_buffer_count, from artiq.gateware.drtio.aux_controller import Transmitter, Receiver
Transmitter, Receiver)
from migen.fhdl.simplify import FullMemoryWE from migen.fhdl.simplify import FullMemoryWE
from misoc.interconnect.csr import * from misoc.interconnect.csr import *
from migen_axi.interconnect.sram import SRAM from migen_axi.interconnect.sram import SRAM
from migen_axi.interconnect import axi from migen_axi.interconnect import axi
max_packet = 1024
class _DRTIOAuxControllerBase(Module): class _DRTIOAuxControllerBase(Module):
def __init__(self, link_layer): def __init__(self, link_layer):
@ -27,12 +27,12 @@ class DRTIOAuxControllerAxi(_DRTIOAuxControllerBase):
tx_sdram_if = SRAM(self.transmitter.mem, read_only=False) tx_sdram_if = SRAM(self.transmitter.mem, read_only=False)
rx_sdram_if = SRAM(self.receiver.mem, read_only=True) rx_sdram_if = SRAM(self.receiver.mem, read_only=True)
aw_decoder = axi.AddressDecoder(self.bus.aw, aw_decoder = axi.AddressDecoder(self.bus.aw,
[(lambda a: a[log2_int(max_packet*aux_buffer_count)] == 0, tx_sdram_if.bus.aw), [(lambda a: a[log2_int(max_packet)] == 0, tx_sdram_if.bus.aw),
(lambda a: a[log2_int(max_packet*aux_buffer_count)] == 1, rx_sdram_if.bus.aw)], (lambda a: a[log2_int(max_packet)] == 1, rx_sdram_if.bus.aw)],
register=True) register=True)
ar_decoder = axi.AddressDecoder(self.bus.ar, ar_decoder = axi.AddressDecoder(self.bus.ar,
[(lambda a: a[log2_int(max_packet*aux_buffer_count)] == 0, tx_sdram_if.bus.ar), [(lambda a: a[log2_int(max_packet)] == 0, tx_sdram_if.bus.ar),
(lambda a: a[log2_int(max_packet*aux_buffer_count)] == 1, rx_sdram_if.bus.ar)], (lambda a: a[log2_int(max_packet)] == 1, rx_sdram_if.bus.ar)],
register=True) register=True)
# unlike wb, axi address decoder only connects ar/aw lanes, # unlike wb, axi address decoder only connects ar/aw lanes,
# the rest must also be connected! # the rest must also be connected!
@ -82,4 +82,4 @@ class DRTIOAuxControllerBare(_DRTIOAuxControllerBase):
return self.receiver.mem.get_port(write_capable=False) return self.receiver.mem.get_port(write_capable=False)
def get_mem_size(self): def get_mem_size(self):
return max_packet*aux_buffer_count return max_packet

View File

@ -1,307 +0,0 @@
#!/usr/bin/env python
import argparse
import analyzer
import dma
from artiq.gateware import rtio
from artiq.gateware.rtio.phy import spi2, ttl_simple
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from config import write_csr_file, write_mem_file, write_rustc_cfg_file
from migen import *
from migen.build.generic_platform import IOStandard, Misc, Pins, Subsignal
from migen.build.platforms import ebaz4205
from migen_axi.integration.soc_core import SoCCore
from misoc.interconnect.csr import *
_ps = [
(
"ps",
0,
Subsignal("clk", Pins("E7"), IOStandard("LVCMOS33"), Misc("SLEW=FAST")),
Subsignal("por_b", Pins("C7"), IOStandard("LVCMOS33"), Misc("SLEW=FAST")),
Subsignal("srst_b", Pins("B10"), IOStandard("LVCMOS18"), Misc("SLEW=FAST")),
)
]
_ddr = [
(
"ddr",
0,
Subsignal(
"a",
Pins("N2 K2 M3 K3 M4 L1 L4 K4 K1 J4 F5 G4 E4 D4 F4"),
IOStandard("SSTL15"),
),
Subsignal("ba", Pins("L5 R4 J5"), IOStandard("SSTL15")),
Subsignal("cas_n", Pins("P5"), IOStandard("SSTL15")),
Subsignal("cke", Pins("N3"), IOStandard("SSTL15")),
Subsignal("cs_n", Pins("N1"), IOStandard("SSTL15")),
Subsignal("ck_n", Pins("M2"), IOStandard("DIFF_SSTL15"), Misc("SLEW=FAST")),
Subsignal("ck_p", Pins("L2"), IOStandard("DIFF_SSTL15"), Misc("SLEW=FAST")),
# Pins "T1 Y1" not connected
Subsignal("dm", Pins("A1 F1"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
Subsignal(
"dq",
Pins("C3 B3 A2 A4 D3 D1 C1 E1 E2 E3 G3 H3 J3 H2 H1 J1"),
# Pins "P1 P3 R3 R1 T4 U4 U2 U3 V1 Y3 W1 Y4 Y2 W3 V2 V3" not connected
IOStandard("SSTL15_T_DCI"),
Misc("SLEW=FAST"),
),
Subsignal(
"dqs_n",
Pins("B2 F2"), # Pins "T2 W4" not connected
IOStandard("DIFF_SSTL15_T_DCI"),
Misc("SLEW=FAST"),
),
Subsignal(
"dqs_p",
Pins("C2 G2"), # Pins "R2 W5" not connected
IOStandard("DIFF_SSTL15_T_DCI"),
Misc("SLEW=FAST"),
),
Subsignal("vrn", Pins("G5"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
Subsignal("vrp", Pins("H5"), IOStandard("SSTL15_T_DCI"), Misc("SLEW=FAST")),
Subsignal("drst_n", Pins("B4"), IOStandard("SSTL15"), Misc("SLEW=FAST")),
Subsignal("odt", Pins("N5"), IOStandard("SSTL15")),
Subsignal("ras_n", Pins("P4"), IOStandard("SSTL15")),
Subsignal("we_n", Pins("M5"), IOStandard("SSTL15")),
)
]
# Connector J3
_i2c = [
(
"i2c",
0,
Subsignal("scl", Pins("U12"), IOStandard("LVCMOS33")),
Subsignal("sda", Pins("V13"), IOStandard("LVCMOS33")),
)
]
_spi = [
(
"spi",
0,
Subsignal("clk", Pins("V20")),
Subsignal("mosi", Pins("U20")),
Subsignal("cs_n", Pins("P19")),
IOStandard("LVCMOS33"),
)
]
# Connector DATA1
def _create_ttl():
_ttl = []
for idx, elem in enumerate([x for x in range(5, 21) if x not in (10, 12)]):
_ttl.append(
("ttl", idx, Pins("DATA1:DATA1-{}".format(elem)), IOStandard("LVCMOS33")),
)
return _ttl
class EBAZ4205(SoCCore):
def __init__(self, rtio_clk=125e6, acpki=False):
self.acpki = acpki
platform = ebaz4205.Platform()
platform.toolchain.bitstream_commands.extend(
[
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
]
)
platform.add_extension(_ps)
platform.add_extension(_ddr)
platform.add_extension(_i2c)
platform.add_extension(_spi)
platform.add_extension(_create_ttl())
gmii = platform.request("gmii")
platform.add_period_constraint(gmii.rx_clk, 10)
platform.add_period_constraint(gmii.tx_clk, 10)
platform.add_platform_command(
"set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets gmii_tx_clk_IBUF]"
)
ident = self.__class__.__name__
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident)
fix_serdes_timing_path(platform)
self.config["RTIO_FREQUENCY"] = str(rtio_clk / 1e6)
platform.add_period_constraint(self.ps7.cd_sys.clk, 10)
self.comb += [
self.ps7.enet0.enet.gmii.tx_clk.eq(gmii.tx_clk),
self.ps7.enet0.enet.gmii.rx_clk.eq(gmii.rx_clk),
]
self.clock_domains.cd_eth_rx = ClockDomain(reset_less=False)
self.clock_domains.cd_eth_tx = ClockDomain(reset_less=False)
self.comb += [
ClockSignal("eth_rx").eq(gmii.rx_clk),
ClockSignal("eth_tx").eq(gmii.tx_clk),
]
self.sync.eth_tx += [
gmii.txd.eq(self.ps7.enet0.enet.gmii.txd),
gmii.tx_en.eq(self.ps7.enet0.enet.gmii.tx_en),
]
self.sync.eth_rx += [
self.ps7.enet0.enet.gmii.rxd.eq(gmii.rxd),
self.ps7.enet0.enet.gmii.rx_dv.eq(gmii.rx_dv),
]
# MDIO
mdio = platform.request("mdio")
self.comb += mdio.mdc.eq(self.ps7.enet0.enet.mdio.mdc)
self.specials += Instance(
"IOBUF",
i_I=self.ps7.enet0.enet.mdio.o,
io_IO=mdio.mdio,
o_O=self.ps7.enet0.enet.mdio.i,
i_T=~self.ps7.enet0.enet.mdio.t_n,
)
# I2C
i2c = self.platform.request("i2c")
self.specials += [
# SCL
Instance(
"IOBUF",
i_I=self.ps7.i2c0.scl.o,
io_IO=i2c.scl,
o_O=self.ps7.i2c0.scl.i,
i_T=~self.ps7.i2c0.scl.t_n,
),
# SDA
Instance(
"IOBUF",
i_I=self.ps7.i2c0.sda.o,
io_IO=i2c.sda,
o_O=self.ps7.i2c0.sda.i,
i_T=~self.ps7.i2c0.sda.t_n,
),
]
self.rtio_channels = []
for i in (0, 1):
print("USER LED at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
user_led = self.platform.request("user_led", i)
phy = ttl_simple.Output(user_led)
self.submodules += phy
self.rtio_channels.append(rtio.Channel.from_phy(phy))
for i in range(14):
print("TTL at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
ttl = self.platform.request("ttl", i)
phy = ttl_simple.InOut(ttl)
self.submodules += phy
self.rtio_channels.append(rtio.Channel.from_phy(phy))
print("SPI at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
spi_phy = spi2.SPIMaster(platform.request("spi"))
self.submodules += spi_phy
self.rtio_channels.append(rtio.Channel.from_phy(spi_phy, ififo_depth=4))
self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
import acpki
self.config["KI_IMPL"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(
self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o,
)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri],
enable_routing=True,
)
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(
self.rtio_tsc, self.rtio_core.cri, self.ps7.s_axi_hp1
)
self.csr_devices.append("rtio_analyzer")
class BASE(EBAZ4205):
def __init__(self, rtio_clk, acpki):
EBAZ4205.__init__(self, rtio_clk, acpki)
VARIANTS = {cls.__name__.lower(): cls for cls in [BASE]}
def main():
parser = argparse.ArgumentParser(
description="ARTIQ port to the EBAZ4205 control card of Ebit E9+ BTC miner"
)
parser.add_argument(
"-r", default=None, help="build Rust interface into the specified file"
)
parser.add_argument(
"-m", default=None, help="build Rust memory interface into the specified file"
)
parser.add_argument(
"-c",
default=None,
help="build Rust compiler configuration into the specified file",
)
parser.add_argument(
"-g", default=None, help="build gateware into the specified directory"
)
parser.add_argument("--rtio-clk", default=125e6, help="RTIO Clock Frequency (Hz)")
parser.add_argument(
"-V",
"--variant",
default="base",
help="variant: " "[acpki_]base" "(default: %(default)s)",
)
args = parser.parse_args()
rtio_clk = int(args.rtio_clk)
variant = args.variant.lower()
acpki = variant.startswith("acpki_")
if acpki:
variant = variant[6:]
try:
cls = VARIANTS[variant]
except KeyError:
raise SystemExit("Invalid variant (-V/--variant)")
soc = cls(rtio_clk=rtio_clk, acpki=acpki)
soc.finalize()
if args.r is not None:
write_csr_file(soc, args.r)
if args.m is not None:
write_mem_file(soc, args.m)
if args.c is not None:
write_rustc_cfg_file(soc, args.c)
if args.g is not None:
soc.build(build_dir=args.g)
if __name__ == "__main__":
main()

View File

@ -10,24 +10,23 @@ from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import kasli_soc from migen_axi.platforms import kasli_soc
from misoc.interconnect.csr import * from misoc.interconnect.csr import *
from misoc.cores import virtual_leds from misoc.integration import cpu_interface
from artiq.coredevice import jsondesc from artiq.coredevice import jsondesc
from artiq.gateware import rtio, eem_7series from artiq.gateware import rtio, eem_7series
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from artiq.gateware.rtio.phy import ttl_simple from artiq.gateware.rtio.phy import ttl_simple
from artiq.gateware.drtio.transceiver import gtx_7series, eem_serdes from artiq.gateware.drtio.transceiver import gtx_7series
from artiq.gateware.drtio.siphaser import SiPhaser7Series from artiq.gateware.drtio.siphaser import SiPhaser7Series
from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer
from artiq.gateware.drtio import * from artiq.gateware.drtio import *
from artiq.gateware.wrpll import wrpll
import dma import dma
import analyzer import analyzer
import acpki import acpki
import drtio_aux_controller import drtio_aux_controller
import zynq_clocking import zynq_clocking
from config import write_csr_file, write_mem_file, write_rustc_cfg_file
eem_iostandard_dict = { eem_iostandard_dict = {
0: "LVDS_25", 0: "LVDS_25",
@ -62,54 +61,27 @@ class SMAClkinForward(Module):
] ]
class GTPBootstrapClock(Module): class GTP125BootstrapClock(Module):
def __init__(self, platform, freq=125e6): def __init__(self, platform):
self.clock_domains.cd_bootstrap = ClockDomain(reset_less=True) self.clock_domains.cd_bootstrap = ClockDomain(reset_less=True)
self.cd_bootstrap.clk.attr.add("keep") self.cd_bootstrap.clk.attr.add("keep")
bootstrap_125 = platform.request("clk125_gtp") bootstrap_125 = platform.request("clk125_gtp")
bootstrap_se = Signal() bootstrap_se = Signal()
clk_out = Signal()
platform.add_period_constraint(bootstrap_125.p, 8.0) platform.add_period_constraint(bootstrap_125.p, 8.0)
self.specials += [ self.specials += [
Instance("IBUFDS_GTE2", Instance("IBUFDS_GTE2",
p_CLKSWING_CFG="0b11",
i_CEB=0, i_CEB=0,
i_I=bootstrap_125.p, i_IB=bootstrap_125.n, i_I=bootstrap_125.p, i_IB=bootstrap_125.n, o_O=bootstrap_se),
o_O=bootstrap_se, Instance("BUFG", i_I=bootstrap_se, o_O=self.cd_bootstrap.clk)
p_CLKCM_CFG="TRUE",
p_CLKRCV_TRST="TRUE",
p_CLKSWING_CFG=3),
Instance("BUFG", i_I=bootstrap_se, o_O=clk_out)
] ]
if freq == 125e6:
self.comb += self.cd_bootstrap.clk.eq(clk_out)
elif freq == 100e6:
pll_fb = Signal()
pll_out = Signal()
self.specials += [
Instance("PLLE2_BASE",
p_CLKIN1_PERIOD=8.0,
i_CLKIN1=clk_out,
i_CLKFBIN=pll_fb,
o_CLKFBOUT=pll_fb,
# VCO @ 1GHz
p_CLKFBOUT_MULT=8, p_DIVCLK_DIVIDE=1,
# 100MHz for bootstrap
p_CLKOUT1_DIVIDE=10, p_CLKOUT1_PHASE=0.0, o_CLKOUT1=pll_out,
),
Instance("BUFG", i_I=pll_out, o_O=self.cd_bootstrap.clk)
]
else:
raise ValueError("Bootstrap frequency must be 100 or 125MHz")
class GenericStandalone(SoCCore): class GenericStandalone(SoCCore):
def __init__(self, description, acpki=False): def __init__(self, description, acpki=False):
self.acpki = acpki self.acpki = acpki
clk_freq = description["rtio_frequency"] self.rustc_cfg = dict()
with_wrpll = description["enable_wrpll"]
platform = kasli_soc.Platform() platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([ platform.toolchain.bitstream_commands.extend([
@ -120,48 +92,29 @@ class GenericStandalone(SoCCore):
ident = "acpki_" + ident ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False) SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"] self.submodules += SMAClkinForward(self.platform)
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_soft_reset"] = None
clk_synth = platform.request("cdr_clk_clean_fabric") clk_synth = platform.request("cdr_clk_clean_fabric")
clk_synth_se = Signal() clk_synth_se = Signal()
clk_synth_se_buf = Signal()
platform.add_period_constraint(clk_synth.p, 8.0) platform.add_period_constraint(clk_synth.p, 8.0)
self.specials += [ self.specials += Instance("IBUFGDS",
Instance("IBUFGDS",
p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE",
i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
),
Instance("BUFG", i_I=clk_synth_se, o_O=clk_synth_se_buf),
]
fix_serdes_timing_path(platform) fix_serdes_timing_path(platform)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, clk_freq) self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.config["RTIO_FREQUENCY"] = str(clk_freq/1e6)
self.config["CLOCK_FREQUENCY"] = int(clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, clk_synth_se_buf)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, clk_synth_se)
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk) self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg") self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys self.crg.cd_sys = self.sys_crg.cd_sys
if with_wrpll:
self.submodules.wrpll_refclk = wrpll.FrequencyMultiplier(platform.request("sma_clkin"))
self.submodules.wrpll = wrpll.WRPLL(
platform=self.platform,
cd_ref=self.wrpll_refclk.cd_ref,
main_clk_se=clk_synth_se)
self.csr_devices.append("wrpll_refclk")
self.csr_devices.append("wrpll")
self.comb += self.ps7.core.core0.nfiq.eq(self.wrpll.ev.irq)
self.config["HAS_SI549"] = None
self.config["WRPLL_REF_CLK"] = "SMA_CLKIN"
else:
self.submodules += SMAClkinForward(self.platform)
self.config["HAS_SI5324"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rtio_channels = [] self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"]) has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_grabber: if has_grabber:
@ -177,20 +130,18 @@ class GenericStandalone(SoCCore):
self.rtio_channels.append(rtio.LogChannel()) self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3) self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core( self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
)
self.csr_devices.append("rtio_core") self.csr_devices.append("rtio_core")
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
@ -210,7 +161,7 @@ class GenericStandalone(SoCCore):
self.csr_devices.append("rtio_analyzer") self.csr_devices.append("rtio_analyzer")
if has_grabber: if has_grabber:
self.config["HAS_GRABBER"] = None self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group) self.add_csr_group("grabber", self.grabber_csr_group)
for grabber in self.grabber_csr_group: for grabber in self.grabber_csr_group:
self.platform.add_false_path_constraints( self.platform.add_false_path_constraints(
@ -220,10 +171,9 @@ class GenericStandalone(SoCCore):
class GenericMaster(SoCCore): class GenericMaster(SoCCore):
def __init__(self, description, acpki=False): def __init__(self, description, acpki=False):
clk_freq = description["rtio_frequency"] clk_freq = description["rtio_frequency"]
with_wrpll = description["enable_wrpll"]
has_drtio_over_eem = any(peripheral["type"] == "shuttler" for peripheral in description["peripherals"])
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform() platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([ platform.toolchain.bitstream_commands.extend([
@ -234,32 +184,26 @@ class GenericMaster(SoCCore):
ident = "acpki_" + ident ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False) SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"] self.submodules += SMAClkinForward(self.platform)
data_pads = [platform.request("sfp", i) for i in range(4)] data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.gt_drtio = gtx_7series.GTX( self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"), clock_pads=platform.request("clk_gtp"),
pads=data_pads, pads=data_pads,
clk_freq=clk_freq) clk_freq=clk_freq)
self.csr_devices.append("gt_drtio") self.csr_devices.append("drtio_transceiver")
self.config["RTIO_FREQUENCY"] = str(clk_freq/1e6)
self.config["CLOCK_FREQUENCY"] = int(clk_freq)
txout_buf = Signal() txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0] gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf) self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
ext_async_rst = Signal() self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG( self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform, self.platform,
self.ps7, self.ps7,
txout_buf, txout_buf,
clk_sw=self.gt_drtio.stable_clkin.storage, clk_sw=gtx0.tx_init.done)
clk_sw_status=gtx0.tx_init.done,
ext_async_rst=ext_async_rst)
self.csr_devices.append("sys_crg") self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys self.crg.cd_sys = self.sys_crg.cd_sys
@ -267,33 +211,11 @@ class GenericMaster(SoCCore):
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk) self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
fix_serdes_timing_path(platform) fix_serdes_timing_path(platform)
self.comb += ext_async_rst.eq(self.sys_crg.clk_sw_fsm.o_clk_sw & ~gtx0.tx_init.done) self.rustc_cfg["has_si5324"] = None
self.specials += MultiReg(self.sys_crg.clk_sw_fsm.o_clk_sw & self.sys_crg.mmcm_locked, self.gt_drtio.clk_path_ready, odomain="bootstrap") self.rustc_cfg["si5324_soft_reset"] = None
if with_wrpll:
clk_synth = platform.request("cdr_clk_clean_fabric")
clk_synth_se = Signal()
platform.add_period_constraint(clk_synth.p, 8.0)
self.specials += Instance("IBUFGDS", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
self.submodules.wrpll_refclk = wrpll.FrequencyMultiplier(platform.request("sma_clkin"))
self.submodules.wrpll = wrpll.WRPLL(
platform=self.platform,
cd_ref=self.wrpll_refclk.cd_ref,
main_clk_se=clk_synth_se)
self.csr_devices.append("wrpll_refclk")
self.csr_devices.append("wrpll")
self.comb += self.ps7.core.core0.nfiq.eq(self.wrpll.ev.irq)
self.config["HAS_SI549"] = None
self.config["WRPLL_REF_CLK"] = "SMA_CLKIN"
else:
self.submodules += SMAClkinForward(self.platform)
self.config["HAS_SI5324"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rtio_channels = [] self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"]) has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_drtio_over_eem:
self.eem_drtio_channels = []
if has_grabber: if has_grabber:
self.grabber_csr_group = [] self.grabber_csr_group = []
eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard) eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard)
@ -308,21 +230,21 @@ class GenericMaster(SoCCore):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3) self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.drtio_csr_group = [] drtio_csr_group = []
self.drtioaux_csr_group = [] drtioaux_csr_group = []
self.drtioaux_memory_group = [] drtioaux_memory_group = []
self.drtio_cri = [] self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)): for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i) core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i) coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem" memory_name = "drtioaux" + str(i) + "_mem"
self.drtio_csr_group.append(core_name) drtio_csr_group.append(core_name)
self.drtioaux_csr_group.append(coreaux_name) drtioaux_csr_group.append(coreaux_name)
self.drtioaux_memory_group.append(memory_name) drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)}) cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster(self.rtio_tsc, self.gt_drtio.channels[i])) core = cdr(DRTIOMaster(self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core) setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri) self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name) self.csr_devices.append(core_name)
@ -335,27 +257,24 @@ class GenericMaster(SoCCore):
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size) memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size)
self.axi2csr.register_port(coreaux.get_rx_port(), size) self.axi2csr.register_port(coreaux.get_rx_port(), size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2) self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2)
self.config["HAS_DRTIO"] = None self.rustc_cfg["has_drtio"] = None
self.config["HAS_DRTIO_ROUTING"] = None self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
if has_drtio_over_eem: self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.add_eem_drtio(self.eem_drtio_channels)
self.add_drtio_cpuif_groups()
self.submodules.rtio_core = rtio.Core(
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
)
self.csr_devices.append("rtio_core") self.csr_devices.append("rtio_core")
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
@ -379,60 +298,16 @@ class GenericMaster(SoCCore):
self.csr_devices.append("rtio_analyzer") self.csr_devices.append("rtio_analyzer")
if has_grabber: if has_grabber:
self.config["HAS_GRABBER"] = None self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group) self.add_csr_group("grabber", self.grabber_csr_group)
self.submodules.virtual_leds = virtual_leds.VirtualLeds()
self.csr_devices.append("virtual_leds")
self.comb += [self.virtual_leds.get(i).eq(channel.rx_ready)
for i, channel in enumerate(self.gt_drtio.channels)]
def add_eem_drtio(self, eem_drtio_channels):
# Must be called before invoking add_rtio() to construct the CRI
# interconnect properly
self.submodules.eem_transceiver = eem_serdes.EEMSerdes(self.platform, eem_drtio_channels)
self.csr_devices.append("eem_transceiver")
self.config["HAS_DRTIO_EEM"] = None
self.config["EEM_DRTIO_COUNT"] = len(eem_drtio_channels)
cdr = ClockDomainsRenamer({"rtio_rx": "sys"})
for i in range(len(self.eem_transceiver.channels)):
channel = i + len(self.gt_drtio.channels)
core_name = "drtio" + str(channel)
coreaux_name = "drtioaux" + str(channel)
memory_name = "drtioaux" + str(channel) + "_mem"
self.drtio_csr_group.append(core_name)
self.drtioaux_csr_group.append(coreaux_name)
self.drtioaux_memory_group.append(memory_name)
core = cdr(DRTIOMaster(self.rtio_tsc, self.eem_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
size = coreaux.get_mem_size()
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size)
self.axi2csr.register_port(coreaux.get_rx_port(), size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2)
def add_drtio_cpuif_groups(self):
self.add_csr_group("drtio", self.drtio_csr_group)
self.add_csr_group("drtioaux", self.drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", self.drtioaux_memory_group)
class GenericSatellite(SoCCore): class GenericSatellite(SoCCore):
def __init__(self, description, acpki=False): def __init__(self, description, acpki=False):
clk_freq = description["rtio_frequency"] clk_freq = description["rtio_frequency"]
with_wrpll = description["enable_wrpll"]
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform() platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([ platform.toolchain.bitstream_commands.extend([
@ -443,41 +318,30 @@ class GenericSatellite(SoCCore):
ident = "acpki_" + ident ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False) SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"]
data_pads = [platform.request("sfp", i) for i in range(4)] data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.gt_drtio = gtx_7series.GTX( self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"), clock_pads=platform.request("clk_gtp"),
pads=data_pads, pads=data_pads,
clk_freq=clk_freq) clk_freq=clk_freq)
self.csr_devices.append("gt_drtio") self.csr_devices.append("drtio_transceiver")
txout_buf = Signal() txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0] gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf) self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
ext_async_rst = Signal() self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG( self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform, self.platform,
self.ps7, self.ps7,
txout_buf, txout_buf,
clk_sw=self.gt_drtio.stable_clkin.storage, clk_sw=gtx0.tx_init.done)
clk_sw_status=gtx0.tx_init.done,
ext_async_rst=ext_async_rst)
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk) self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg") self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys self.crg.cd_sys = self.sys_crg.cd_sys
fix_serdes_timing_path(platform)
self.comb += ext_async_rst.eq(self.sys_crg.clk_sw_fsm.o_clk_sw & ~gtx0.tx_init.done)
self.specials += MultiReg(self.sys_crg.clk_sw_fsm.o_clk_sw & self.sys_crg.mmcm_locked, self.gt_drtio.clk_path_ready, odomain="bootstrap")
self.rtio_channels = [] self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"]) has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_grabber: if has_grabber:
@ -498,7 +362,7 @@ class GenericSatellite(SoCCore):
drtioaux_memory_group = [] drtioaux_memory_group = []
drtiorep_csr_group = [] drtiorep_csr_group = []
self.drtio_cri = [] self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)): for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i) coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem" memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name) drtioaux_csr_group.append(coreaux_name)
@ -509,7 +373,7 @@ class GenericSatellite(SoCCore):
if i == 0: if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer()) self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite( core = cdr(DRTIOSatellite(
self.rtio_tsc, self.gt_drtio.channels[i], self.rtio_tsc, self.drtio_transceiver.channels[i],
self.rx_synchronizer)) self.rx_synchronizer))
self.submodules.drtiosat = core self.submodules.drtiosat = core
self.csr_devices.append("drtiosat") self.csr_devices.append("drtiosat")
@ -518,7 +382,7 @@ class GenericSatellite(SoCCore):
drtiorep_csr_group.append(corerep_name) drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater( core = cdr(DRTIORepeater(
self.rtio_tsc, self.gt_drtio.channels[i])) self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core) setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri) self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name) self.csr_devices.append(corerep_name)
@ -536,37 +400,32 @@ class GenericSatellite(SoCCore):
# and registered in PS interface # and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names # manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2) self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None self.rustc_cfg["has_drtio"] = None
self.config["HAS_DRTIO_ROUTING"] = None self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group) self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group) self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.add_csr_group("drtiorep", drtiorep_csr_group) self.add_csr_group("drtiorep", drtiorep_csr_group)
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0) self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma") self.csr_devices.append("rtio_dma")
self.submodules.local_io = SyncRTIO( self.submodules.local_io = SyncRTIO(self.rtio_tsc, self.rtio_channels)
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"] self.comb += self.drtiosat.async_errors.eq(self.local_io.async_errors)
)
self.comb += [
self.drtiosat.async_errors.eq(self.local_io.async_errors),
self.local_io.sed_spread_enable.eq(self.drtiosat.sed_spread_enable.storage)
]
self.submodules.cri_con = rtio.CRIInterconnectShared( self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri], [self.drtiosat.cri],
[self.local_io.cri] + self.drtio_cri, [self.local_io.cri] + self.drtio_cri,
enable_routing=True) enable_routing=True)
self.csr_devices.append("cri_con") self.csr_devices.append("cri_con")
@ -577,53 +436,49 @@ class GenericSatellite(SoCCore):
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels) self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj") self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.local_io.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
rtio_clk_period = 1e9/clk_freq rtio_clk_period = 1e9/clk_freq
self.config["RTIO_FREQUENCY"] = str(clk_freq/1e6) self.rustc_cfg["rtio_frequency"] = str(clk_freq/1e6)
self.config["CLOCK_FREQUENCY"] = int(clk_freq)
if with_wrpll:
clk_synth = platform.request("cdr_clk_clean_fabric")
clk_synth_se = Signal()
platform.add_period_constraint(clk_synth.p, 8.0)
self.specials += Instance("IBUFGDS", p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE", i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
self.submodules.wrpll = wrpll.WRPLL(
platform=self.platform,
cd_ref=self.gt_drtio.cd_rtio_rx0,
main_clk_se=clk_synth_se)
self.submodules.wrpll_skewtester = wrpll.SkewTester(self.rx_synchronizer)
self.csr_devices.append("wrpll_skewtester")
self.csr_devices.append("wrpll")
self.comb += self.ps7.core.core0.nfiq.eq(self.wrpll.ev.irq)
self.config["HAS_SI549"] = None
self.config["WRPLL_REF_CLK"] = "GT_CDR"
else:
self.submodules.siphaser = SiPhaser7Series( self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("cdr_clk"), si5324_clkin=platform.request("cdr_clk"),
rx_synchronizer=self.rx_synchronizer, rx_synchronizer=self.rx_synchronizer,
ultrascale=False, ultrascale=False,
rtio_clk_freq=self.gt_drtio.rtio_clk_freq) rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
self.csr_devices.append("siphaser") self.csr_devices.append("siphaser")
self.config["HAS_SI5324"] = None self.rustc_cfg["has_si5324"] = None
self.config["SI5324_SOFT_RESET"] = None self.rustc_cfg["has_siphaser"] = None
self.rustc_cfg["si5324_soft_reset"] = None
gtx0 = self.gt_drtio.gtxs[0] gtx0 = self.drtio_transceiver.gtxs[0]
platform.add_false_path_constraints( platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk) gtx0.txoutclk, gtx0.rxoutclk)
if has_grabber: if has_grabber:
self.config["HAS_GRABBER"] = None self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group) self.add_csr_group("grabber", self.grabber_csr_group)
# no RTIO CRG here # no RTIO CRG here
self.submodules.virtual_leds = virtual_leds.VirtualLeds()
self.csr_devices.append("virtual_leds")
self.comb += [self.virtual_leds.get(i).eq(channel.rx_ready) def write_mem_file(soc, filename):
for i, channel in enumerate(self.gt_drtio.channels)] with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
@ -646,14 +501,14 @@ def main():
if description["target"] != "kasli_soc": if description["target"] != "kasli_soc":
raise ValueError("Description is for a different target") raise ValueError("Description is for a different target")
if description["drtio_role"] == "standalone": if description["base"] == "standalone":
cls = GenericStandalone cls = GenericStandalone
elif description["drtio_role"] == "master": elif description["base"] == "master":
cls = GenericMaster cls = GenericMaster
elif description["drtio_role"] == "satellite": elif description["base"] == "satellite":
cls = GenericSatellite cls = GenericSatellite
else: else:
raise ValueError("Invalid DRTIO role") raise ValueError("Invalid base")
soc = cls(description, acpki=args.acpki) soc = cls(description, acpki=args.acpki)
soc.finalize() soc.finalize()

View File

@ -10,10 +10,11 @@ from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import zc706 from migen_axi.platforms import zc706
from misoc.interconnect.csr import * from misoc.interconnect.csr import *
from misoc.integration import cpu_interface
from misoc.cores import gpio from misoc.cores import gpio
from artiq.gateware import rtio, nist_clock, nist_qc2 from artiq.gateware import rtio, nist_clock, nist_qc2
from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2, edge_counter from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from artiq.gateware.drtio.transceiver import gtx_7series from artiq.gateware.drtio.transceiver import gtx_7series
from artiq.gateware.drtio.siphaser import SiPhaser7Series from artiq.gateware.drtio.siphaser import SiPhaser7Series
@ -25,7 +26,7 @@ import analyzer
import acpki import acpki
import drtio_aux_controller import drtio_aux_controller
import zynq_clocking import zynq_clocking
from config import write_csr_file, write_mem_file, write_rustc_cfg_file
class SMAClkinForward(Module): class SMAClkinForward(Module):
def __init__(self, platform): def __init__(self, platform):
@ -126,6 +127,7 @@ def prepare_zc706_platform(platform):
class ZC706(SoCCore): class ZC706(SoCCore):
def __init__(self, acpki=False): def __init__(self, acpki=False):
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict()
platform = zc706.Platform() platform = zc706.Platform()
prepare_zc706_platform(platform) prepare_zc706_platform(platform)
@ -147,14 +149,14 @@ class ZC706(SoCCore):
i_CEB=0, i_CEB=0,
i_I=si5324_out.p, i_IB=si5324_out.n, i_I=si5324_out.p, i_IB=si5324_out.n,
o_O=cdr_clk, o_O=cdr_clk,
p_CLKCM_CFG="TRUE", p_CLKCM_CFG="0b1",
p_CLKRCV_TRST="TRUE", p_CLKRCV_TRST="0b1",
p_CLKSWING_CFG=3), p_CLKSWING_CFG="0b11"),
Instance("BUFG", i_I=cdr_clk, o_O=cdr_clk_buf) Instance("BUFG", i_I=cdr_clk, o_O=cdr_clk_buf)
] ]
self.config["HAS_SI5324"] = None self.rustc_cfg["has_si5324"] = None
self.config["SI5324_AS_SYNTHESIZER"] = None self.rustc_cfg["si5324_as_synthesizer"] = None
self.config["SI5324_SOFT_RESET"] = None self.rustc_cfg["si5324_soft_reset"] = None
self.submodules.bootstrap = CLK200BootstrapClock(platform) self.submodules.bootstrap = CLK200BootstrapClock(platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, cdr_clk_buf) self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, cdr_clk_buf)
@ -168,14 +170,14 @@ class ZC706(SoCCore):
self.csr_devices.append("rtio_core") self.csr_devices.append("rtio_core")
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
@ -198,6 +200,7 @@ class ZC706(SoCCore):
class _MasterBase(SoCCore): class _MasterBase(SoCCore):
def __init__(self, acpki=False, drtio100mhz=False): def __init__(self, acpki=False, drtio100mhz=False):
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict()
clk_freq = 100e6 if drtio100mhz else 125e6 clk_freq = 100e6 if drtio100mhz else 125e6
@ -219,38 +222,32 @@ class _MasterBase(SoCCore):
self.submodules += SMAClkinForward(self.platform) self.submodules += SMAClkinForward(self.platform)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock # 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.gt_drtio = gtx_7series.GTX( self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"), clock_pads=platform.request("si5324_clkout"),
pads=data_pads, pads=data_pads,
clk_freq=clk_freq) clk_freq=clk_freq)
self.csr_devices.append("gt_drtio") self.csr_devices.append("drtio_transceiver")
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3) self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
ext_async_rst = Signal()
txout_buf = Signal() txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0] gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf) self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = CLK200BootstrapClock(platform, clk_freq) self.submodules.bootstrap = CLK200BootstrapClock(platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG( self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform, self.platform,
self.ps7, self.ps7,
txout_buf, txout_buf,
clk_sw=self.gt_drtio.stable_clkin.storage, clk_sw=gtx0.tx_init.done,
clk_sw_status=gtx0.tx_init.done,
ext_async_rst=ext_async_rst,
freq=clk_freq) freq=clk_freq)
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk) self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg") self.csr_devices.append("sys_crg")
self.comb += ext_async_rst.eq(self.sys_crg.clk_sw_fsm.o_clk_sw & ~gtx0.tx_init.done)
self.specials += MultiReg(self.sys_crg.clk_sw_fsm.o_clk_sw & self.sys_crg.mmcm_locked, self.gt_drtio.clk_path_ready, odomain="bootstrap")
drtio_csr_group = [] drtio_csr_group = []
drtioaux_csr_group = [] drtioaux_csr_group = []
drtioaux_memory_group = [] drtioaux_memory_group = []
self.drtio_cri = [] self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)): for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i) core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i) coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem" memory_name = "drtioaux" + str(i) + "_mem"
@ -261,7 +258,7 @@ class _MasterBase(SoCCore):
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)}) cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster( core = cdr(DRTIOMaster(
self.rtio_tsc, self.gt_drtio.channels[i])) self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core) setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri) self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name) self.csr_devices.append(core_name)
@ -274,18 +271,18 @@ class _MasterBase(SoCCore):
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), mem_size) memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), mem_size)
self.axi2csr.register_port(coreaux.get_rx_port(), mem_size) self.axi2csr.register_port(coreaux.get_rx_port(), mem_size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2) self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None self.rustc_cfg["has_drtio"] = None
self.config["HAS_DRTIO_ROUTING"] = None self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group) self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group) self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group) self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.config["RTIO_FREQUENCY"] = str(self.gt_drtio.rtio_clk_freq/1e6) self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n) self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n") self.csr_devices.append("si5324_rst_n")
self.config["HAS_SI5324"] = None self.rustc_cfg["has_si5324"] = None
self.config["SI5324_AS_SYNTHESIZER"] = None self.rustc_cfg["si5324_as_synthesizer"] = None
# Constrain TX & RX timing for the first transceiver channel # Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX) # (First channel acts as master for phase alignment for all channels' TX)
@ -293,7 +290,7 @@ class _MasterBase(SoCCore):
gtx0.txoutclk, gtx0.rxoutclk) gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel # Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX) # (Each channel performs single-lane phase alignment for RX)
for gtx in self.gt_drtio.gtxs[1:]: for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints( platform.add_false_path_constraints(
gtx0.txoutclk, gtx.rxoutclk) gtx0.txoutclk, gtx.rxoutclk)
@ -305,14 +302,14 @@ class _MasterBase(SoCCore):
self.csr_devices.append("rtio_core") self.csr_devices.append("rtio_core")
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
@ -339,6 +336,7 @@ class _MasterBase(SoCCore):
class _SatelliteBase(SoCCore): class _SatelliteBase(SoCCore):
def __init__(self, acpki=False, drtio100mhz=False): def __init__(self, acpki=False, drtio100mhz=False):
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict()
clk_freq = 100e6 if drtio100mhz else 125e6 clk_freq = 100e6 if drtio100mhz else 125e6
@ -361,16 +359,15 @@ class _SatelliteBase(SoCCore):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3) self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock # 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.gt_drtio = gtx_7series.GTX( self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"), clock_pads=platform.request("si5324_clkout"),
pads=data_pads, pads=data_pads,
clk_freq=clk_freq) clk_freq=clk_freq)
self.csr_devices.append("gt_drtio") self.csr_devices.append("drtio_transceiver")
ext_async_rst = Signal()
txout_buf = Signal() txout_buf = Signal()
txout_buf.attr.add("keep") txout_buf.attr.add("keep")
gtx0 = self.gt_drtio.gtxs[0] gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance( self.specials += Instance(
"BUFG", "BUFG",
i_I=gtx0.txoutclk, i_I=gtx0.txoutclk,
@ -380,22 +377,17 @@ class _SatelliteBase(SoCCore):
self.platform, self.platform,
self.ps7, self.ps7,
txout_buf, txout_buf,
clk_sw=self.gt_drtio.stable_clkin.storage, clk_sw=gtx0.tx_init.done,
clk_sw_status=gtx0.tx_init.done,
ext_async_rst=ext_async_rst,
freq=clk_freq) freq=clk_freq)
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk) self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg") self.csr_devices.append("sys_crg")
self.comb += ext_async_rst.eq(self.sys_crg.clk_sw_fsm.o_clk_sw & ~gtx0.tx_init.done)
self.specials += MultiReg(self.sys_crg.clk_sw_fsm.o_clk_sw & self.sys_crg.mmcm_locked, self.gt_drtio.clk_path_ready, odomain="bootstrap")
drtioaux_csr_group = [] drtioaux_csr_group = []
drtioaux_memory_group = [] drtioaux_memory_group = []
drtiorep_csr_group = [] drtiorep_csr_group = []
self.drtio_cri = [] self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)): for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i) coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem" memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name) drtioaux_csr_group.append(coreaux_name)
@ -407,7 +399,7 @@ class _SatelliteBase(SoCCore):
if i == 0: if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer()) self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite( core = cdr(DRTIOSatellite(
self.rtio_tsc, self.gt_drtio.channels[0], self.rx_synchronizer)) self.rtio_tsc, self.drtio_transceiver.channels[0], self.rx_synchronizer))
self.submodules.drtiosat = core self.submodules.drtiosat = core
self.csr_devices.append("drtiosat") self.csr_devices.append("drtiosat")
# Repeaters # Repeaters
@ -415,7 +407,7 @@ class _SatelliteBase(SoCCore):
corerep_name = "drtiorep" + str(i-1) corerep_name = "drtiorep" + str(i-1)
drtiorep_csr_group.append(corerep_name) drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater( core = cdr(DRTIORepeater(
self.rtio_tsc, self.gt_drtio.channels[i])) self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core) setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri) self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name) self.csr_devices.append(corerep_name)
@ -433,35 +425,36 @@ class _SatelliteBase(SoCCore):
# and registered in PS interface # and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names # manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2) self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None self.rustc_cfg["has_drtio"] = None
self.config["HAS_DRTIO_ROUTING"] = None self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group) self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_csr_group("drtiorep", drtiorep_csr_group) self.add_csr_group("drtiorep", drtiorep_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group) self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.config["RTIO_FREQUENCY"] = str(self.gt_drtio.rtio_clk_freq/1e6) self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
# Si5324 Phaser # Si5324 Phaser
self.submodules.siphaser = SiPhaser7Series( self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("si5324_clkin"), si5324_clkin=platform.request("si5324_clkin"),
rx_synchronizer=self.rx_synchronizer, rx_synchronizer=self.rx_synchronizer,
ultrascale=False, ultrascale=False,
rtio_clk_freq=self.gt_drtio.rtio_clk_freq) rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, self.siphaser.mmcm_freerun_output) self.sys_crg.cd_sys.clk, self.siphaser.mmcm_freerun_output)
self.csr_devices.append("siphaser") self.csr_devices.append("siphaser")
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n) self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n") self.csr_devices.append("si5324_rst_n")
self.config["HAS_SI5324"] = None self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["has_siphaser"] = None
rtio_clk_period = 1e9/self.gt_drtio.rtio_clk_freq rtio_clk_period = 1e9/self.drtio_transceiver.rtio_clk_freq
# Constrain TX & RX timing for the first transceiver channel # Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX) # (First channel acts as master for phase alignment for all channels' TX)
platform.add_false_path_constraints( platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk) gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel # Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX) # (Each channel performs single-lane phase alignment for RX)
for gtx in self.gt_drtio.gtxs[1:]: for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints( platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, gtx.rxoutclk) self.sys_crg.cd_sys.clk, gtx.rxoutclk)
@ -472,35 +465,24 @@ class _SatelliteBase(SoCCore):
self.csr_devices.append("rtio_moninj") self.csr_devices.append("rtio_moninj")
if self.acpki: if self.acpki:
self.config["KI_IMPL"] = "acp" self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc, self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp, bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user, user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o) evento=self.ps7.event.o)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
else: else:
self.config["KI_IMPL"] = "csr" self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True) self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio") self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels) self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels)
self.comb += [
self.drtiosat.async_errors.eq(self.local_io.async_errors),
self.local_io.sed_spread_enable.eq(self.drtiosat.sed_spread_enable.storage)
]
self.submodules.cri_con = rtio.CRIInterconnectShared( self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri], [self.drtiosat.cri],
[self.local_io.cri] + self.drtio_cri, [self.local_io.cri] + self.drtio_cri,
enable_routing=True) enable_routing=True)
self.csr_devices.append("cri_con") self.csr_devices.append("cri_con")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.local_io.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con) self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table") self.csr_devices.append("routing_table")
@ -592,16 +574,12 @@ class _NIST_QC2_RTIO:
platform.add_extension(pmod1_33) platform.add_extension(pmod1_33)
rtio_channels = [] rtio_channels = []
edge_counter_phy = []
# All TTL channels are In+Out capable # All TTL channels are In+Out capable
for i in range(40): for i in range(40):
phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i)) phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i))
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512)) rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512))
# first four TTLs will also have edge counters
if i < 4:
edge_counter_phy.append(phy)
# no SMA GPIO, replaced with PMOD1_0 # no SMA GPIO, replaced with PMOD1_0
phy = ttl_serdes_7series.InOut_8X(platform.request("pmod1_33", 0)) phy = ttl_serdes_7series.InOut_8X(platform.request("pmod1_33", 0))
@ -641,11 +619,6 @@ class _NIST_QC2_RTIO:
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=4)) rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=4))
for phy in edge_counter_phy:
counter = edge_counter.SimpleEdgeCounter(phy.input_state)
self.submodules += counter
rtio_channels.append(rtio.Channel.from_phy(counter))
self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels) self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels)
rtio_channels.append(rtio.LogChannel()) rtio_channels.append(rtio.LogChannel())
@ -687,6 +660,27 @@ class NIST_QC2_Satellite(_SatelliteBase, _NIST_QC2_RTIO):
VARIANTS = {cls.__name__.lower(): cls for cls in [NIST_CLOCK, NIST_CLOCK_Master, NIST_CLOCK_Satellite, VARIANTS = {cls.__name__.lower(): cls for cls in [NIST_CLOCK, NIST_CLOCK_Master, NIST_CLOCK_Satellite,
NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite]} NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite]}
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="ARTIQ port to the ZC706 Zynq development kit") description="ARTIQ port to the ZC706 Zynq development kit")

View File

@ -65,12 +65,10 @@ class ClockSwitchFSM(Module):
class SYSCRG(Module, AutoCSR): class SYSCRG(Module, AutoCSR):
def __init__(self, platform, ps7, main_clk, clk_sw=None, clk_sw_status=None, freq=125e6, ext_async_rst=None, ): def __init__(self, platform, ps7, main_clk, clk_sw=None, freq=125e6):
# assumes bootstrap clock is same freq as main and sys output # assumes bootstrap clock is same freq as main and sys output
self.clock_domains.cd_sys = ClockDomain() self.clock_domains.cd_sys = ClockDomain()
self.clock_domains.cd_sys4x = ClockDomain(reset_less=True) self.clock_domains.cd_sys4x = ClockDomain(reset_less=True)
self.clock_domains.cd_sys5x = ClockDomain(reset_less=True)
self.clock_domains.cd_clk200 = ClockDomain()
self.current_clock = CSRStatus() self.current_clock = CSRStatus()
@ -80,6 +78,11 @@ class SYSCRG(Module, AutoCSR):
period = 1e9/freq period = 1e9/freq
pll_locked = Signal()
pll_sys = Signal()
pll_sys4x = Signal()
fb_clk = Signal()
self.submodules.clk_sw_fsm = ClockSwitchFSM() self.submodules.clk_sw_fsm = ClockSwitchFSM()
if clk_sw is None: if clk_sw is None:
@ -88,67 +91,32 @@ class SYSCRG(Module, AutoCSR):
else: else:
self.comb += self.clk_sw_fsm.i_clk_sw.eq(clk_sw) self.comb += self.clk_sw_fsm.i_clk_sw.eq(clk_sw)
self.mmcm_locked = Signal()
mmcm_sys = Signal()
mmcm_sys4x = Signal()
mmcm_sys5x = Signal()
mmcm_clk208 = Signal()
mmcm_fb_clk = Signal()
self.specials += [ self.specials += [
Instance("MMCME2_ADV", Instance("PLLE2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=self.mmcm_locked, p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
p_BANDWIDTH="HIGH", p_BANDWIDTH="HIGH",
p_REF_JITTER1=0.001, p_REF_JITTER1=0.001,
p_CLKIN1_PERIOD=period, i_CLKIN1=main_clk, p_CLKIN1_PERIOD=period, i_CLKIN1=main_clk,
p_CLKIN2_PERIOD=period, i_CLKIN2=bootstrap_clk, p_CLKIN2_PERIOD=period, i_CLKIN2=bootstrap_clk,
i_CLKINSEL=self.clk_sw_fsm.o_clk_sw, i_CLKINSEL=self.clk_sw_fsm.o_clk_sw,
# VCO @ 1.25GHz # VCO @ 1.5GHz when using 125MHz input
p_CLKFBOUT_MULT_F=10, p_DIVCLK_DIVIDE=1, # 1.2GHz for 100MHz (zc706)
i_CLKFBIN=mmcm_fb_clk, p_CLKFBOUT_MULT=12, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=fb_clk,
i_RST=self.clk_sw_fsm.o_reset, i_RST=self.clk_sw_fsm.o_reset,
o_CLKFBOUT=mmcm_fb_clk, o_CLKFBOUT=fb_clk,
p_CLKOUT0_DIVIDE_F=2.5, p_CLKOUT0_PHASE=0.0, o_CLKOUT0=mmcm_sys4x, p_CLKOUT0_DIVIDE=3, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=pll_sys4x,
# 125MHz p_CLKOUT1_DIVIDE=12, p_CLKOUT1_PHASE=0.0,
p_CLKOUT1_DIVIDE=10, p_CLKOUT1_PHASE=0.0, o_CLKOUT1=mmcm_sys, o_CLKOUT1=pll_sys),
Instance("BUFG", i_I=pll_sys, o_O=self.cd_sys.clk),
Instance("BUFG", i_I=pll_sys4x, o_O=self.cd_sys4x.clk),
# 625MHz AsyncResetSynchronizer(self.cd_sys, ~pll_locked),
p_CLKOUT2_DIVIDE=2, p_CLKOUT2_PHASE=0.0, o_CLKOUT2=mmcm_sys5x,
# 208MHz
p_CLKOUT3_DIVIDE=6, p_CLKOUT3_PHASE=0.0, o_CLKOUT3=mmcm_clk208,
),
Instance("BUFG", i_I=mmcm_sys5x, o_O=self.cd_sys5x.clk),
Instance("BUFG", i_I=mmcm_sys, o_O=self.cd_sys.clk),
Instance("BUFG", i_I=mmcm_sys4x, o_O=self.cd_sys4x.clk),
Instance("BUFG", i_I=mmcm_clk208, o_O=self.cd_clk200.clk),
] ]
if ext_async_rst is not None:
self.specials += [
AsyncResetSynchronizer(self.cd_sys, ~self.mmcm_locked | ext_async_rst),
AsyncResetSynchronizer(self.cd_clk200, ~self.mmcm_locked | ext_async_rst),
]
else:
self.specials += [
AsyncResetSynchronizer(self.cd_sys, ~self.mmcm_locked),
AsyncResetSynchronizer(self.cd_clk200, ~self.mmcm_locked),
]
reset_counter = Signal(4, reset=15)
ic_reset = Signal(reset=1)
self.sync.clk200 += \
If(reset_counter != 0,
reset_counter.eq(reset_counter - 1)
).Else(
ic_reset.eq(0)
)
self.specials += Instance("IDELAYCTRL", i_REFCLK=ClockSignal("clk200"), i_RST=ic_reset)
if clk_sw_status is None:
self.comb += self.current_clock.status.eq(self.clk_sw_fsm.o_clk_sw) self.comb += self.current_clock.status.eq(self.clk_sw_fsm.o_clk_sw)
else:
self.comb += self.current_clock.status.eq(clk_sw_status)

View File

@ -10,8 +10,6 @@ name = "libboard_artiq"
[features] [features]
target_zc706 = ["libboard_zynq/target_zc706", "libconfig/target_zc706"] target_zc706 = ["libboard_zynq/target_zc706", "libconfig/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libconfig/target_kasli_soc"] target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libconfig/target_kasli_soc"]
target_ebaz4205 = ["libboard_zynq/target_ebaz4205", "libconfig/target_ebaz4205"]
calibrate_wrpll_skew = []
[build-dependencies] [build-dependencies]
build_zynq = { path = "../libbuild_zynq" } build_zynq = { path = "../libbuild_zynq" }
@ -26,9 +24,8 @@ nb = "1.0"
void = { version = "1", default-features = false } void = { version = "1", default-features = false }
io = { path = "../libio", features = ["byteorder"] } io = { path = "../libio", features = ["byteorder"] }
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq" } libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git"}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] } libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { path = "@@ZYNQ_RS@@/libregister" } libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn"] }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn"] } libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" } libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }

View File

@ -1,245 +0,0 @@
use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs;
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use libsupport_zynq::alloc::format;
use log::{debug, error, info};
use crate::pl;
struct SerdesConfig {
pub delay: [u8; 4],
}
impl SerdesConfig {
pub fn as_bytes(&self) -> &[u8] {
unsafe {
core::slice::from_raw_parts(
(self as *const SerdesConfig) as *const u8,
core::mem::size_of::<SerdesConfig>(),
)
}
}
}
fn select_lane(lane_no: u8) {
unsafe {
pl::csr::eem_transceiver::lane_sel_write(lane_no);
}
}
fn apply_delay(tap: u8, timer: &mut GlobalTimer) {
unsafe {
pl::csr::eem_transceiver::dly_cnt_in_write(tap);
pl::csr::eem_transceiver::dly_ld_write(1);
timer.delay_us(1);
assert!(tap as u8 == pl::csr::eem_transceiver::dly_cnt_out_read());
}
}
fn apply_config(config: &SerdesConfig, timer: &mut GlobalTimer) {
for lane_no in 0..4 {
select_lane(lane_no as u8);
apply_delay(config.delay[lane_no], timer);
}
}
unsafe fn assign_delay(timer: &mut GlobalTimer) -> SerdesConfig {
// Select an appropriate delay for lane 0
select_lane(0);
//
let mut best_dly = None;
loop {
let mut prev = None;
for curr_dly in 0..32 {
//let read_align = read_align_fn(curr_dly, timer);
let curr_low_rate = read_align(curr_dly, timer);
if let Some(prev_low_rate) = prev {
// This is potentially a crossover position
if prev_low_rate <= curr_low_rate && curr_low_rate >= 0.5 {
let prev_dev = 0.5 - prev_low_rate;
let curr_dev = curr_low_rate - 0.5;
let selected_idx = if prev_dev < curr_dev { curr_dly - 1 } else { curr_dly };
// The setup setup/hold calibration timing (even with
// tolerance) might be invalid in other lanes due to skew.
// 5 taps is very conservative, generally it is 1 or 2
if selected_idx < 5 {
prev = None;
continue;
} else {
best_dly = Some(selected_idx);
break;
}
}
}
// Only rising slope from <= 0.5 can result in a rising low rate
// crossover at 50%.
if curr_low_rate <= 0.5 {
prev = Some(curr_low_rate);
}
}
if best_dly.is_none() {
error!("setup/hold timing calibration failed, retry in 1s...");
timer.delay_us(1_000_000);
} else {
break;
}
}
let best_dly = best_dly.unwrap();
apply_delay(best_dly, timer);
let mut delay_list = [best_dly; 4];
// Assign delay for other lanes
for lane_no in 1..=3 {
select_lane(lane_no as u8);
let mut min_deviation = 0.5;
let mut min_idx = 0;
for dly_delta in -3..=3 {
let index = (best_dly as isize + dly_delta) as u8;
let low_rate = read_align(index, timer);
// abs() from f32 is not available in core library
let deviation = if low_rate < 0.5 { 0.5 - low_rate } else { low_rate - 0.5 };
if deviation < min_deviation {
min_deviation = deviation;
min_idx = index;
}
}
apply_delay(min_idx, timer);
delay_list[lane_no] = min_idx;
}
debug!("setup/hold timing calibration: {:?}", delay_list);
SerdesConfig { delay: delay_list }
}
fn read_align(dly: u8, timer: &mut GlobalTimer) -> f32 {
unsafe {
apply_delay(dly, timer);
pl::csr::eem_transceiver::counter_reset_write(1);
pl::csr::eem_transceiver::counter_enable_write(1);
timer.delay_us(2000);
pl::csr::eem_transceiver::counter_enable_write(0);
let (high, low) = (
pl::csr::eem_transceiver::counter_high_count_read(),
pl::csr::eem_transceiver::counter_low_count_read(),
);
if pl::csr::eem_transceiver::counter_overflow_read() == 1 {
panic!("Unexpected phase detector counter overflow");
}
low as f32 / (low + high) as f32
}
}
unsafe fn align_comma(timer: &mut GlobalTimer) {
loop {
for slip in 1..=10 {
// The soft transceiver has 2 8b10b decoders, which receives lane
// 0/1 and lane 2/3 respectively. The decoder are time-multiplexed
// to decode exactly 1 lane each sysclk cycle.
//
// The decoder decodes lane 0/2 data on odd sysclk cycles, buffer
// on even cycles, and vice versa for lane 1/3. Data/Clock latency
// could change timing. The extend bit flips the decoding timing,
// so lane 0/2 data are decoded on even cycles, and lane 1/3 data
// are decoded on odd cycles.
//
// This is needed because transmitting/receiving a 8b10b character
// takes 2 sysclk cycles. Adjusting bitslip only via ISERDES
// limits the range to 1 cycle. The wordslip bit extends the range
// to 2 sysclk cycles.
pl::csr::eem_transceiver::wordslip_write((slip > 5) as u8);
// Apply a double bitslip since the ISERDES is 2x oversampled.
// Bitslip is used for comma alignment purposes once setup/hold
// timing is met.
pl::csr::eem_transceiver::bitslip_write(1);
pl::csr::eem_transceiver::bitslip_write(1);
timer.delay_us(1);
pl::csr::eem_transceiver::comma_align_reset_write(1);
timer.delay_us(100);
if pl::csr::eem_transceiver::comma_read() == 1 {
debug!("comma alignment completed after {} bitslips", slip);
return;
}
}
error!("comma alignment failed, retrying in 1s...");
timer.delay_us(1_000_000);
}
}
pub unsafe fn align_wordslip(timer: &mut GlobalTimer, trx_no: u8) -> bool {
pl::csr::eem_transceiver::transceiver_sel_write(trx_no);
for slip in 0..=1 {
pl::csr::eem_transceiver::wordslip_write(slip as u8);
timer.delay_us(1);
pl::csr::eem_transceiver::comma_align_reset_write(1);
timer.delay_us(100);
if pl::csr::eem_transceiver::comma_read() == 1 {
debug!("comma alignment completed with {} wordslip", slip);
return true;
}
}
false
}
pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
for trx_no in 0..pl::csr::CONFIG_EEM_DRTIO_COUNT {
unsafe {
pl::csr::eem_transceiver::transceiver_sel_write(trx_no as u8);
}
let key = format!("eem_drtio_delay{}", trx_no);
let cfg_read = cfg.read(&key);
match cfg_read {
Ok(record) => {
info!("loading calibrated timing values from sd card");
unsafe {
apply_config(&*(record.as_ptr() as *const SerdesConfig), timer);
}
}
Err(_) => {
info!("calibrating...");
let config = unsafe { assign_delay(timer) };
match cfg.write(&key, config.as_bytes().to_vec()) {
Ok(()) => {
info!("storing calibration timing values into sd card");
}
Err(e) => {
error!(
"calibration successful but calibration timing values cannot be stored into sd card. \
Error:{}",
e
);
}
};
}
}
unsafe {
align_comma(timer);
}
}
}

View File

@ -1,10 +1,9 @@
use core::fmt;
use libconfig::Config; use libconfig::Config;
use log::{info, warn};
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
use crate::pl::csr; use crate::pl::csr;
use core::fmt;
use log::{warn, info};
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
pub const DEST_COUNT: usize = 256; pub const DEST_COUNT: usize = 256;
@ -59,10 +58,10 @@ impl fmt::Display for RoutingTable {
pub fn config_routing_table(default_n_links: usize, cfg: &Config) -> RoutingTable { pub fn config_routing_table(default_n_links: usize, cfg: &Config) -> RoutingTable {
let mut ret = RoutingTable::default_master(default_n_links); let mut ret = RoutingTable::default_master(default_n_links);
if let Ok(data) = cfg.read("routing_table") { if let Ok(data) = cfg.read("routing_table") {
if data.len() == DEST_COUNT * MAX_HOPS { if data.len() == DEST_COUNT*MAX_HOPS {
for i in 0..DEST_COUNT { for i in 0..DEST_COUNT {
for j in 0..MAX_HOPS { for j in 0..MAX_HOPS {
ret.0[i][j] = data[i * MAX_HOPS + j]; ret.0[i][j] = data[i*MAX_HOPS+j];
} }
} }
} else { } else {

View File

@ -1,13 +1,14 @@
use core::slice;
use core_io::{Error as IoError, ErrorKind as IoErrorKind};
use crc; use crc;
use io::{proto::{ProtoRead, ProtoWrite},
Cursor}; use core_io::{ErrorKind as IoErrorKind, Error as IoError};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer}; use io::{proto::ProtoRead, proto::ProtoWrite, Cursor};
use libboard_zynq::{timer::GlobalTimer, time::Milliseconds};
use libcortex_a9::asm::dmb;
use crate::mem::mem::DRTIOAUX_MEM;
use crate::pl::csr::DRTIOAUX;
use crate::drtioaux_proto::Error as ProtocolError;
pub use crate::drtioaux_proto::Packet; pub use crate::drtioaux_proto::Packet;
use crate::{drtioaux_proto::Error as ProtocolError, mem::mem::DRTIOAUX_MEM, pl::csr::DRTIOAUX};
#[derive(Debug)] #[derive(Debug)]
pub enum Error { pub enum Error {
@ -20,7 +21,7 @@ pub enum Error {
RoutingError, RoutingError,
Protocol(ProtocolError), Protocol(ProtocolError)
} }
impl From<ProtocolError> for Error { impl From<ProtocolError> for Error {
@ -57,14 +58,31 @@ pub fn has_rx_error(linkno: u8) -> bool {
} }
} }
pub fn copy_work_buffer(src: *mut u32, dst: *mut u32, len: isize) {
// AXI writes must be 4-byte aligned (drtio proto doesn't care for that),
// and AXI burst reads/writes are not implemented yet in gateware
// thus the need for a work buffer for transmitting and copying it over
unsafe {
for i in 0..(len/4) {
*dst.offset(i) = *src.offset(i);
//data memory barrier to prevent bursts
dmb();
}
}
}
fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error> fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error>
where F: FnOnce(&[u8]) -> Result<T, Error> { where F: FnOnce(&[u8]) -> Result<T, Error>
{
let linkidx = linkno as usize; let linkidx = linkno as usize;
unsafe { unsafe {
if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 { if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
let read_ptr = (DRTIOAUX[linkidx].aux_read_pointer_read)() as usize; let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2) as *mut u32;
let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2 + read_ptr * 0x400) as *mut u32; let len = (DRTIOAUX[linkidx].aux_rx_length_read)() as usize;
let result = f(slice::from_raw_parts(ptr as *mut u8, 0x400 as usize)); // work buffer to accomodate axi burst reads
let mut buf: [u8; 1024] = [0; 1024];
copy_work_buffer(ptr, buf.as_mut_ptr() as *mut u32, len as isize);
let result = f(&buf[0..len]);
(DRTIOAUX[linkidx].aux_rx_present_write)(1); (DRTIOAUX[linkidx].aux_rx_present_write)(1);
Ok(Some(result?)) Ok(Some(result?))
} else { } else {
@ -75,29 +93,31 @@ where F: FnOnce(&[u8]) -> Result<T, Error> {
pub fn recv(linkno: u8) -> Result<Option<Packet>, Error> { pub fn recv(linkno: u8) -> Result<Option<Packet>, Error> {
if has_rx_error(linkno) { if has_rx_error(linkno) {
return Err(Error::GatewareError); return Err(Error::GatewareError)
} }
receive(linkno, |buffer| { receive(linkno, |buffer| {
if buffer.len() < 8 { if buffer.len() < 8 {
return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into()); return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into())
} }
let mut reader = Cursor::new(buffer); let mut reader = Cursor::new(buffer);
let packet = Packet::read_from(&mut reader)?; let checksum_at = buffer.len() - 4;
let padding = (12 - (reader.position() % 8)) % 8;
let checksum_at = reader.position() + padding;
let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]); let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
reader.set_position(checksum_at); reader.set_position(checksum_at);
if reader.read_u32()? != checksum { if reader.read_u32()? != checksum {
return Err(Error::CorruptedPacket); return Err(Error::CorruptedPacket)
} }
Ok(packet) reader.set_position(0);
Ok(Packet::read_from(&mut reader)?)
}) })
} }
pub fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTimer) -> Result<Packet, Error> { pub fn recv_timeout(linkno: u8, timeout_ms: Option<u64>,
timer: GlobalTimer) -> Result<Packet, Error>
{
let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10)); let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10));
let limit = timer.get_time() + timeout_ms; let limit = timer.get_time() + timeout_ms;
while timer.get_time() < limit { while timer.get_time() < limit {
@ -110,12 +130,17 @@ pub fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTimer) ->
} }
fn transmit<F>(linkno: u8, f: F) -> Result<(), Error> fn transmit<F>(linkno: u8, f: F) -> Result<(), Error>
where F: FnOnce(&mut [u8]) -> Result<usize, Error> { where F: FnOnce(&mut [u8]) -> Result<usize, Error>
{
let linkno = linkno as usize; let linkno = linkno as usize;
unsafe { unsafe {
while (DRTIOAUX[linkno].aux_tx_read)() != 0 {} while (DRTIOAUX[linkno].aux_tx_read)() != 0 {}
let ptr = DRTIOAUX_MEM[linkno].base as *mut u32; let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
let len = f(slice::from_raw_parts_mut(ptr as *mut u8, 0x400 as usize))?; let len = DRTIOAUX_MEM[linkno].size / 2;
// work buffer, works with unaligned mem access
let mut buf: [u8; 1024] = [0; 1024];
let len = f(&mut buf[0..len])?;
copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
(DRTIOAUX[linkno].aux_tx_length_write)(len as u16); (DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
(DRTIOAUX[linkno].aux_tx_write)(1); (DRTIOAUX[linkno].aux_tx_write)(1);
Ok(()) Ok(())

View File

@ -1,18 +1,18 @@
use core::slice;
use core_io::{Error as IoError, ErrorKind as IoErrorKind};
use crc; use crc;
use io::{proto::{ProtoRead, ProtoWrite},
Cursor}; use core_io::{ErrorKind as IoErrorKind, Error as IoError};
use libasync::{block_async, task};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use nb;
use void::Void; use void::Void;
use nb;
use libboard_zynq::{timer::GlobalTimer, time::Milliseconds};
use libasync::{task, block_async};
use io::{proto::ProtoRead, proto::ProtoWrite, Cursor};
use crate::mem::mem::DRTIOAUX_MEM;
use crate::pl::csr::DRTIOAUX;
use crate::drtioaux::{Error, has_rx_error, copy_work_buffer};
pub use crate::drtioaux_proto::Packet; pub use crate::drtioaux_proto::Packet;
use crate::{drtioaux::{has_rx_error, Error},
mem::mem::DRTIOAUX_MEM,
pl::csr::DRTIOAUX};
pub async fn reset(linkno: u8) { pub async fn reset(linkno: u8) {
let linkno = linkno as usize; let linkno = linkno as usize;
@ -29,20 +29,25 @@ fn tx_ready(linkno: usize) -> nb::Result<(), Void> {
unsafe { unsafe {
if (DRTIOAUX[linkno].aux_tx_read)() != 0 { if (DRTIOAUX[linkno].aux_tx_read)() != 0 {
Err(nb::Error::WouldBlock) Err(nb::Error::WouldBlock)
} else { }
else {
Ok(()) Ok(())
} }
} }
} }
async fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error> async fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error>
where F: FnOnce(&[u8]) -> Result<T, Error> { where F: FnOnce(&[u8]) -> Result<T, Error>
{
let linkidx = linkno as usize; let linkidx = linkno as usize;
unsafe { unsafe {
if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 { if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
let read_ptr = (DRTIOAUX[linkidx].aux_read_pointer_read)() as usize; let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2) as *mut u32;
let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2 + read_ptr * 0x400) as *mut u32; let len = (DRTIOAUX[linkidx].aux_rx_length_read)() as usize;
let result = f(slice::from_raw_parts(ptr as *mut u8, 0x400 as usize)); // work buffer to accomodate axi burst reads
let mut buf: [u8; 1024] = [0; 1024];
copy_work_buffer(ptr, buf.as_mut_ptr() as *mut u32, len as isize);
let result = f(&buf[0..len]);
(DRTIOAUX[linkidx].aux_rx_present_write)(1); (DRTIOAUX[linkidx].aux_rx_present_write)(1);
Ok(Some(result?)) Ok(Some(result?))
} else { } else {
@ -53,30 +58,31 @@ where F: FnOnce(&[u8]) -> Result<T, Error> {
pub async fn recv(linkno: u8) -> Result<Option<Packet>, Error> { pub async fn recv(linkno: u8) -> Result<Option<Packet>, Error> {
if has_rx_error(linkno) { if has_rx_error(linkno) {
return Err(Error::GatewareError); return Err(Error::GatewareError)
} }
receive(linkno, |buffer| { receive(linkno, |buffer| {
if buffer.len() < 8 { if buffer.len() < 8 {
return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into()); return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into())
} }
let mut reader = Cursor::new(buffer); let mut reader = Cursor::new(buffer);
let packet = Packet::read_from(&mut reader)?; let checksum_at = buffer.len() - 4;
let padding = (12 - (reader.position() % 8)) % 8;
let checksum_at = reader.position() + padding;
let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]); let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
reader.set_position(checksum_at); reader.set_position(checksum_at);
if reader.read_u32()? != checksum { if reader.read_u32()? != checksum {
return Err(Error::CorruptedPacket); return Err(Error::CorruptedPacket)
} }
Ok(packet) reader.set_position(0);
})
.await Ok(Packet::read_from(&mut reader)?)
}).await
} }
pub async fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTimer) -> Result<Packet, Error> { pub async fn recv_timeout(linkno: u8, timeout_ms: Option<u64>,
timer: GlobalTimer) -> Result<Packet, Error>
{
let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10)); let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10));
let limit = timer.get_time() + timeout_ms; let limit = timer.get_time() + timeout_ms;
let mut would_block = false; let mut would_block = false;
@ -87,9 +93,7 @@ pub async fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTime
task::r#yield().await; task::r#yield().await;
} }
match recv(linkno).await? { match recv(linkno).await? {
None => { None => { would_block = true; },
would_block = true;
}
Some(packet) => return Ok(packet), Some(packet) => return Ok(packet),
} }
} }
@ -97,12 +101,17 @@ pub async fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTime
} }
async fn transmit<F>(linkno: u8, f: F) -> Result<(), Error> async fn transmit<F>(linkno: u8, f: F) -> Result<(), Error>
where F: FnOnce(&mut [u8]) -> Result<usize, Error> { where F: FnOnce(&mut [u8]) -> Result<usize, Error>
{
let linkno = linkno as usize; let linkno = linkno as usize;
unsafe { unsafe {
let _ = block_async!(tx_ready(linkno)).await; let _ = block_async!(tx_ready(linkno)).await;
let ptr = DRTIOAUX_MEM[linkno].base as *mut u32; let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
let len = f(slice::from_raw_parts_mut(ptr as *mut u8, 0x400 as usize))?; let len = DRTIOAUX_MEM[linkno].size / 2;
// work buffer, works with unaligned mem access
let mut buf: [u8; 1024] = [0; 1024];
let len = f(&mut buf[0..len])?;
copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
(DRTIOAUX[linkno].aux_tx_length_write)(len as u16); (DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
(DRTIOAUX[linkno].aux_tx_write)(1); (DRTIOAUX[linkno].aux_tx_write)(1);
Ok(()) Ok(())
@ -125,6 +134,5 @@ pub async fn send(linkno: u8, packet: &Packet) -> Result<(), Error> {
writer.write_u32(checksum)?; writer.write_u32(checksum)?;
Ok(writer.position()) Ok(writer.position())
}) }).await
.await
} }

File diff suppressed because it is too large Load Diff

View File

@ -1,22 +0,0 @@
use libboard_zynq::{println, stdio};
use libcortex_a9::{interrupt_handler, regs::MPIDR};
use libregister::RegisterR;
#[cfg(has_si549)]
use crate::si549;
interrupt_handler!(FIQ, fiq, __irq_stack0_start, __irq_stack1_start, {
match MPIDR.read().cpu_id() {
0 => {
// nFIQ is driven directly and bypass GIC
#[cfg(has_si549)]
si549::wrpll::interrupt_handler();
return;
}
_ => {}
};
stdio::drop_uart();
println!("FIQ");
loop {}
});

View File

@ -1,163 +0,0 @@
use log::info;
use crate::pl::csr;
#[derive(PartialEq, Clone, Copy)]
enum State {
Reset,
ExitReset,
Lock,
Align,
Watch,
}
#[derive(Clone, Copy)]
struct Info {
state: State,
frame_size: (u16, u16),
}
static mut INFO: [Info; csr::GRABBER_LEN] = [Info {
state: State::Reset,
frame_size: (0, 0),
}; csr::GRABBER_LEN];
fn get_pll_reset(g: usize) -> bool {
unsafe { (csr::GRABBER[g].pll_reset_read)() != 0 }
}
fn set_pll_reset(g: usize, reset: bool) {
let val = if reset { 1 } else { 0 };
unsafe { (csr::GRABBER[g].pll_reset_write)(val) }
}
fn pll_locked(g: usize) -> bool {
unsafe { (csr::GRABBER[g].pll_locked_read)() != 0 }
}
fn clock_pattern_ok(g: usize) -> bool {
unsafe { (csr::GRABBER[g].clk_sampled_read)() == 0b1100011 }
}
fn clock_pattern_ok_filter(g: usize) -> bool {
for _ in 0..128 {
if !clock_pattern_ok(g) {
return false;
}
}
true
}
fn phase_shift(g: usize, direction: u8) {
unsafe {
(csr::GRABBER[g].phase_shift_write)(direction);
while (csr::GRABBER[g].phase_shift_done_read)() == 0 {}
}
}
fn clock_align(g: usize) -> bool {
while clock_pattern_ok_filter(g) {
phase_shift(g, 1);
}
phase_shift(g, 1);
let mut count = 0;
while !clock_pattern_ok_filter(g) {
phase_shift(g, 1);
count += 1;
if count > 1024 {
return false;
}
}
let mut window = 1;
phase_shift(g, 1);
while clock_pattern_ok_filter(g) {
phase_shift(g, 1);
window += 1;
}
for _ in 0..window / 2 {
phase_shift(g, 0);
}
true
}
fn get_last_pixels(g: usize) -> (u16, u16) {
unsafe { ((csr::GRABBER[g].last_x_read)(), (csr::GRABBER[g].last_y_read)()) }
}
fn get_video_clock(g: usize) -> u32 {
let freq_count = unsafe { (csr::GRABBER[g].freq_count_read)() } as u32;
2 * freq_count * (csr::CONFIG_CLOCK_FREQUENCY / 1000) / (511 * 1000)
}
pub fn tick() {
for g in 0..csr::GRABBER.len() {
let next = match unsafe { INFO[g].state } {
State::Reset => {
set_pll_reset(g, true);
unsafe {
INFO[g].frame_size = (0, 0);
}
State::ExitReset
}
State::ExitReset => {
if get_pll_reset(g) {
set_pll_reset(g, false);
State::Lock
} else {
State::ExitReset
}
}
State::Lock => {
if pll_locked(g) {
info!("grabber{} locked: {}MHz", g, get_video_clock(g));
State::Align
} else {
State::Lock
}
}
State::Align => {
if pll_locked(g) {
if clock_align(g) {
info!("grabber{} alignment success", g);
State::Watch
} else {
info!("grabber{} alignment failure", g);
State::Reset
}
} else {
info!("grabber{} lock lost", g);
State::Reset
}
}
State::Watch => {
if pll_locked(g) {
if clock_pattern_ok(g) {
let last_xy = get_last_pixels(g);
if last_xy != unsafe { INFO[g].frame_size } {
// x capture is on ~LVAL which is after
// the last increment on DVAL
// y capture is on ~FVAL which coincides with the
// last increment on ~LVAL
info!("grabber{} frame size: {}x{}", g, last_xy.0, last_xy.1 + 1);
unsafe { INFO[g].frame_size = last_xy }
}
State::Watch
} else {
info!("grabber{} alignment lost", g);
State::Reset
}
} else {
info!("grabber{} lock lost", g);
State::Reset
}
}
};
unsafe {
INFO[g].state = next;
}
}
}

View File

@ -1,9 +1,6 @@
use libboard_zynq::i2c; use libboard_zynq::i2c;
use log::info; use log::info;
#[cfg(has_virtual_leds)]
use crate::pl::csr;
// Only the bare minimum registers. Bits/IO connections equivalent between IC types. // Only the bare minimum registers. Bits/IO connections equivalent between IC types.
struct Registers { struct Registers {
// PCA9539 equivalent register names in comments // PCA9539 equivalent register names in comments
@ -13,55 +10,25 @@ struct Registers {
gpiob: u8, // Output Port 1 gpiob: u8, // Output Port 1
} }
//IO expanders pins pub struct IoExpander<'a> {
const IODIR_OUT_SFP_TX_DISABLE: u8 = 0x02; i2c: &'a mut i2c::I2c,
const IODIR_OUT_SFP_LED: u8 = 0x40;
#[cfg(hw_rev = "v1.0")]
const IODIR_OUT_SFP0_LED: u8 = 0x40;
#[cfg(hw_rev = "v1.1")]
const IODIR_OUT_SFP0_LED: u8 = 0x80;
#[cfg(has_si549)]
const IODIR_CLK_SEL: u8 = 0x80; // out
#[cfg(has_si5324)]
const IODIR_CLK_SEL: u8 = 0x00; // in
//IO expander port direction
const IODIR0: [u8; 2] = [
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP0_LED,
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED & !IODIR_CLK_SEL,
];
const IODIR1: [u8; 2] = [
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED,
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED,
];
pub struct IoExpander {
address: u8, address: u8,
#[cfg(has_virtual_leds)]
virtual_led_mapping: &'static [(u8, u8, u8)],
iodir: [u8; 2], iodir: [u8; 2],
out_current: [u8; 2], out_current: [u8; 2],
out_target: [u8; 2], out_target: [u8; 2],
registers: Registers, registers: Registers,
} }
impl IoExpander {
pub fn new(i2c: &mut i2c::I2c, index: u8) -> Result<Self, &'static str> { impl<'a> IoExpander<'a> {
#[cfg(all(hw_rev = "v1.0", has_virtual_leds))] pub fn new(i2c: &'a mut i2c::I2c, index: u8) -> Result<Self, &'static str> {
const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 6), (1, 1, 6)];
#[cfg(all(hw_rev = "v1.1", has_virtual_leds))]
const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 7), (1, 1, 6)];
#[cfg(has_virtual_leds)]
const VIRTUAL_LED_MAPPING1: [(u8, u8, u8); 2] = [(2, 0, 6), (3, 1, 6)];
// Both expanders on SHARED I2C bus // Both expanders on SHARED I2C bus
let mut io_expander = match index { let mut io_expander = match index {
0 => IoExpander { 0 => IoExpander {
i2c,
address: 0x40, address: 0x40,
#[cfg(has_virtual_leds)] iodir: [0xff; 2],
virtual_led_mapping: &VIRTUAL_LED_MAPPING0,
iodir: IODIR0,
out_current: [0; 2], out_current: [0; 2],
out_target: [0; 2], out_target: [0; 2],
registers: Registers { registers: Registers {
@ -72,10 +39,9 @@ impl IoExpander {
}, },
}, },
1 => IoExpander { 1 => IoExpander {
i2c,
address: 0x42, address: 0x42,
#[cfg(has_virtual_leds)] iodir: [0xff; 2],
virtual_led_mapping: &VIRTUAL_LED_MAPPING1,
iodir: IODIR1,
out_current: [0; 2], out_current: [0; 2],
out_target: [0; 2], out_target: [0; 2],
registers: Registers { registers: Registers {
@ -87,8 +53,11 @@ impl IoExpander {
}, },
_ => return Err("incorrect I/O expander index"), _ => return Err("incorrect I/O expander index"),
}; };
if !io_expander.check_ack(i2c)? { if !io_expander.check_ack()? {
info!("MCP23017 io expander {} not found. Checking for PCA9539.", index); info!(
"MCP23017 io expander {} not found. Checking for PCA9539.",
index
);
io_expander.address += 0xa8; // translate to PCA9539 addresses (see schematic) io_expander.address += 0xa8; // translate to PCA9539 addresses (see schematic)
io_expander.registers = Registers { io_expander.registers = Registers {
iodira: 0x06, iodira: 0x06,
@ -96,58 +65,57 @@ impl IoExpander {
gpioa: 0x02, gpioa: 0x02,
gpiob: 0x03, gpiob: 0x03,
}; };
if !io_expander.check_ack(i2c)? { if !io_expander.check_ack()? {
return Err("Neither MCP23017 nor PCA9539 io expander found."); return Err("Neither MCP23017 nor PCA9539 io expander found.");
}; };
} }
Ok(io_expander) Ok(io_expander)
} }
fn select(&self, i2c: &mut i2c::I2c) -> Result<(), &'static str> { fn select(&mut self) -> Result<(), &'static str> {
i2c.pca954x_select(0x70, None)?; self.i2c.pca954x_select(0x70, None)?;
i2c.pca954x_select(0x71, Some(3))?; self.i2c.pca954x_select(0x71, Some(3))?;
Ok(()) Ok(())
} }
fn write(&self, i2c: &mut i2c::I2c, addr: u8, value: u8) -> Result<(), &'static str> { fn write(&mut self, addr: u8, value: u8) -> Result<(), &'static str> {
i2c.start()?; self.i2c.start()?;
i2c.write(self.address)?; self.i2c.write(self.address)?;
i2c.write(addr)?; self.i2c.write(addr)?;
i2c.write(value)?; self.i2c.write(value)?;
i2c.stop()?; self.i2c.stop()?;
Ok(()) Ok(())
} }
fn check_ack(&self, i2c: &mut i2c::I2c) -> Result<bool, &'static str> { fn check_ack(&mut self) -> Result<bool, &'static str> {
// Check for ack from io expander // Check for ack from io expander
self.select(i2c)?; self.select()?;
i2c.start()?; self.i2c.start()?;
let ack = i2c.write(self.address)?; let ack = self.i2c.write(self.address)?;
i2c.stop()?; self.i2c.stop()?;
Ok(ack) Ok(ack)
} }
fn update_iodir(&self, i2c: &mut i2c::I2c) -> Result<(), &'static str> { fn update_iodir(&mut self) -> Result<(), &'static str> {
self.write(i2c, self.registers.iodira, self.iodir[0])?; self.write(self.registers.iodira, self.iodir[0])?;
self.write(i2c, self.registers.iodirb, self.iodir[1])?; self.write(self.registers.iodirb, self.iodir[1])?;
Ok(()) Ok(())
} }
pub fn init(&mut self, i2c: &mut i2c::I2c) -> Result<(), &'static str> { pub fn init(&mut self) -> Result<(), &'static str> {
self.select(i2c)?; self.select()?;
self.update_iodir()?;
self.update_iodir(i2c)?;
self.out_current[0] = 0x00; self.out_current[0] = 0x00;
self.write(i2c, self.registers.gpioa, 0x00)?; self.write(self.registers.gpioa, 0x00)?;
self.out_current[1] = 0x00; self.out_current[1] = 0x00;
self.write(i2c, self.registers.gpiob, 0x00)?; self.write(self.registers.gpiob, 0x00)?;
Ok(()) Ok(())
} }
pub fn set_oe(&mut self, i2c: &mut i2c::I2c, port: u8, outputs: u8) -> Result<(), &'static str> { pub fn set_oe(&mut self, port: u8, outputs: u8) -> Result<(), &'static str> {
self.iodir[port as usize] &= !outputs; self.iodir[port as usize] &= !outputs;
self.update_iodir(i2c)?; self.update_iodir()?;
Ok(()) Ok(())
} }
@ -159,21 +127,15 @@ impl IoExpander {
} }
} }
pub fn service(&mut self, i2c: &mut i2c::I2c) -> Result<(), &'static str> { pub fn service(&mut self) -> Result<(), &'static str> {
#[cfg(has_virtual_leds)]
for (led, port, bit) in self.virtual_led_mapping.iter() {
let level = unsafe { csr::virtual_leds::status_read() >> led & 1 };
self.set(*port, *bit, level != 0);
}
if self.out_target != self.out_current { if self.out_target != self.out_current {
self.select(i2c)?; self.select()?;
if self.out_target[0] != self.out_current[0] { if self.out_target[0] != self.out_current[0] {
self.write(i2c, self.registers.gpioa, self.out_target[0])?; self.write(self.registers.gpioa, self.out_target[0])?;
self.out_current[0] = self.out_target[0]; self.out_current[0] = self.out_target[0];
} }
if self.out_target[1] != self.out_current[1] { if self.out_target[1] != self.out_current[1] {
self.write(i2c, self.registers.gpiob, self.out_target[1])?; self.write(self.registers.gpiob, self.out_target[1])?;
self.out_current[1] = self.out_target[1]; self.out_current[1] = self.out_target[1];
} }
} }

View File

@ -1,45 +1,35 @@
#![no_std] #![no_std]
#![feature(never_type)] #![feature(never_type)]
#![feature(naked_functions)]
#![feature(asm)]
extern crate core_io; extern crate log;
extern crate crc; extern crate crc;
extern crate embedded_hal; extern crate embedded_hal;
extern crate core_io;
extern crate io; extern crate io;
extern crate libasync;
extern crate libboard_zynq; extern crate libboard_zynq;
extern crate libregister;
extern crate libconfig; extern crate libconfig;
extern crate libcortex_a9; extern crate libcortex_a9;
extern crate libregister; extern crate libasync;
extern crate log;
extern crate log_buffer; extern crate log_buffer;
#[path = "../../../build/pl.rs"]
pub mod pl;
pub mod drtioaux_proto;
pub mod drtio_routing; pub mod drtio_routing;
pub mod logger;
#[cfg(has_si5324)]
pub mod si5324;
#[cfg(has_drtio)] #[cfg(has_drtio)]
pub mod drtioaux; pub mod drtioaux;
#[cfg(has_drtio)] #[cfg(has_drtio)]
pub mod drtioaux_async; pub mod drtioaux_async;
pub mod drtioaux_proto;
pub mod fiq;
#[cfg(feature = "target_kasli_soc")]
pub mod io_expander;
pub mod logger;
#[cfg(has_drtio)] #[cfg(has_drtio)]
#[rustfmt::skip]
#[path = "../../../build/mem.rs"] #[path = "../../../build/mem.rs"]
pub mod mem; pub mod mem;
#[rustfmt::skip] #[cfg(feature = "target_kasli_soc")]
#[path = "../../../build/pl.rs"] pub mod io_expander;
pub mod pl;
#[cfg(has_drtio_eem)]
pub mod drtio_eem;
#[cfg(has_grabber)]
pub mod grabber;
#[cfg(has_si5324)]
pub mod si5324;
#[cfg(has_si549)]
pub mod si549;
use core::{cmp, str}; use core::{cmp, str};
pub fn identifier_read(buf: &mut [u8]) -> &str { pub fn identifier_read(buf: &mut [u8]) -> &str {

View File

@ -1,13 +1,13 @@
use core::{cell::Cell, fmt::Write}; use core::cell::Cell;
use core::fmt::Write;
use libboard_zynq::{println, timer::GlobalTimer}; use log::{Log, LevelFilter};
use libcortex_a9::mutex::{Mutex, MutexGuard};
use log::{LevelFilter, Log};
use log_buffer::LogBuffer; use log_buffer::LogBuffer;
use libcortex_a9::mutex::{Mutex, MutexGuard};
use libboard_zynq::{println, timer::GlobalTimer};
pub struct LogBufferRef<'a> { pub struct LogBufferRef<'a> {
buffer: MutexGuard<'a, LogBuffer<&'static mut [u8]>>, buffer: MutexGuard<'a, LogBuffer<&'static mut [u8]>>,
old_log_level: LevelFilter, old_log_level: LevelFilter
} }
impl<'a> LogBufferRef<'a> { impl<'a> LogBufferRef<'a> {
@ -56,7 +56,8 @@ impl BufferLogger {
pub fn register(self) { pub fn register(self) {
unsafe { unsafe {
LOGGER = Some(self); LOGGER = Some(self);
log::set_logger(LOGGER.as_ref().unwrap()).expect("global logger can only be initialized once"); log::set_logger(LOGGER.as_ref().unwrap())
.expect("global logger can only be initialized once");
} }
} }
@ -65,7 +66,9 @@ impl BufferLogger {
} }
pub fn buffer<'a>(&'a self) -> Option<LogBufferRef<'a>> { pub fn buffer<'a>(&'a self) -> Option<LogBufferRef<'a>> {
self.buffer.try_lock().map(LogBufferRef::new) self.buffer
.try_lock()
.map(LogBufferRef::new)
} }
pub fn uart_log_level(&self) -> LevelFilter { pub fn uart_log_level(&self) -> LevelFilter {
@ -96,36 +99,25 @@ impl Log for BufferLogger {
fn log(&self, record: &log::Record) { fn log(&self, record: &log::Record) {
if self.enabled(record.metadata()) { if self.enabled(record.metadata()) {
let timestamp = unsafe { GlobalTimer::get() }.get_us().0; let timestamp = unsafe {
GlobalTimer::get()
}.get_us().0;
let seconds = timestamp / 1_000_000; let seconds = timestamp / 1_000_000;
let micros = timestamp % 1_000_000; let micros = timestamp % 1_000_000;
if record.level() <= self.buffer_log_level() { if record.level() <= self.buffer_log_level() {
let mut buffer = self.buffer.lock(); let mut buffer = self.buffer.lock();
writeln!( writeln!(buffer, "[{:6}.{:06}s] {:>5}({}): {}", seconds, micros,
buffer, record.level(), record.target(), record.args()).unwrap();
"[{:6}.{:06}s] {:>5}({}): {}",
seconds,
micros,
record.level(),
record.target(),
record.args()
)
.unwrap();
} }
if record.level() <= self.uart_log_level() { if record.level() <= self.uart_log_level() {
println!( println!("[{:6}.{:06}s] {:>5}({}): {}", seconds, micros,
"[{:6}.{:06}s] {:>5}({}): {}", record.level(), record.target(), record.args());
seconds,
micros,
record.level(),
record.target(),
record.args()
);
} }
} }
} }
fn flush(&self) {} fn flush(&self) {
}
} }

View File

@ -1,9 +1,7 @@
use core::result; use core::result;
use embedded_hal::blocking::delay::DelayUs;
use libboard_zynq::{i2c::I2c, time::Milliseconds, timer::GlobalTimer};
use log::info; use log::info;
use libboard_zynq::{i2c::I2c, timer::GlobalTimer, time::Milliseconds};
use embedded_hal::blocking::delay::DelayUs;
#[cfg(not(si5324_soft_reset))] #[cfg(not(si5324_soft_reset))]
use crate::pl::csr; use crate::pl::csr;
@ -13,13 +11,9 @@ const ADDRESS: u8 = 0x68;
#[cfg(not(si5324_soft_reset))] #[cfg(not(si5324_soft_reset))]
fn hard_reset(timer: &mut GlobalTimer) { fn hard_reset(timer: &mut GlobalTimer) {
unsafe { unsafe { csr::si5324_rst_n::out_write(0); }
csr::si5324_rst_n::out_write(0);
}
timer.delay_us(1_000); timer.delay_us(1_000);
unsafe { unsafe { csr::si5324_rst_n::out_write(1); }
csr::si5324_rst_n::out_write(1);
}
timer.delay_us(10_000); timer.delay_us(10_000);
} }
@ -35,7 +29,7 @@ pub struct FrequencySettings {
pub n31: u32, pub n31: u32,
pub n32: u32, pub n32: u32,
pub bwsel: u8, pub bwsel: u8,
pub crystal_as_ckin2: bool, pub crystal_as_ckin2: bool
} }
pub enum Input { pub enum Input {
@ -45,22 +39,22 @@ pub enum Input {
fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySettings> { fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySettings> {
if settings.nc1_ls != 0 && (settings.nc1_ls % 2) == 1 { if settings.nc1_ls != 0 && (settings.nc1_ls % 2) == 1 {
return Err("NC1_LS must be 0 or even"); return Err("NC1_LS must be 0 or even")
} }
if settings.nc1_ls > (1 << 20) { if settings.nc1_ls > (1 << 20) {
return Err("NC1_LS is too high"); return Err("NC1_LS is too high")
} }
if (settings.n2_ls % 2) == 1 { if (settings.n2_ls % 2) == 1 {
return Err("N2_LS must be even"); return Err("N2_LS must be even")
} }
if settings.n2_ls > (1 << 20) { if settings.n2_ls > (1 << 20) {
return Err("N2_LS is too high"); return Err("N2_LS is too high")
} }
if settings.n31 > (1 << 19) { if settings.n31 > (1 << 19) {
return Err("N31 is too high"); return Err("N31 is too high")
} }
if settings.n32 > (1 << 19) { if settings.n32 > (1 << 19) {
return Err("N32 is too high"); return Err("N32 is too high")
} }
let r = FrequencySettings { let r = FrequencySettings {
n1_hs: match settings.n1_hs { n1_hs: match settings.n1_hs {
@ -72,7 +66,7 @@ fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySetti
9 => 0b101, 9 => 0b101,
10 => 0b110, 10 => 0b110,
11 => 0b111, 11 => 0b111,
_ => return Err("N1_HS has an invalid value"), _ => return Err("N1_HS has an invalid value")
}, },
nc1_ls: settings.nc1_ls - 1, nc1_ls: settings.nc1_ls - 1,
n2_hs: match settings.n2_hs { n2_hs: match settings.n2_hs {
@ -84,13 +78,13 @@ fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySetti
9 => 0b101, 9 => 0b101,
10 => 0b110, 10 => 0b110,
11 => 0b111, 11 => 0b111,
_ => return Err("N2_HS has an invalid value"), _ => return Err("N2_HS has an invalid value")
}, },
n2_ls: settings.n2_ls - 1, n2_ls: settings.n2_ls - 1,
n31: settings.n31 - 1, n31: settings.n31 - 1,
n32: settings.n32 - 1, n32: settings.n32 - 1,
bwsel: settings.bwsel, bwsel: settings.bwsel,
crystal_as_ckin2: settings.crystal_as_ckin2, crystal_as_ckin2: settings.crystal_as_ckin2
}; };
Ok(r) Ok(r)
} }
@ -98,13 +92,13 @@ fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySetti
fn write(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> { fn write(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
i2c.start().unwrap(); i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() { if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address"); return Err("Si5324 failed to ack write address")
} }
if !i2c.write(reg).unwrap() { if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register"); return Err("Si5324 failed to ack register")
} }
if !i2c.write(val).unwrap() { if !i2c.write(val).unwrap() {
return Err("Si5324 failed to ack value"); return Err("Si5324 failed to ack value")
} }
i2c.stop().unwrap(); i2c.stop().unwrap();
Ok(()) Ok(())
@ -114,10 +108,10 @@ fn write(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
fn write_no_ack_value(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> { fn write_no_ack_value(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
i2c.start().unwrap(); i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() { if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address"); return Err("Si5324 failed to ack write address")
} }
if !i2c.write(reg).unwrap() { if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register"); return Err("Si5324 failed to ack register")
} }
i2c.write(val).unwrap(); i2c.write(val).unwrap();
i2c.stop().unwrap(); i2c.stop().unwrap();
@ -127,22 +121,22 @@ fn write_no_ack_value(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
fn read(i2c: &mut I2c, reg: u8) -> Result<u8> { fn read(i2c: &mut I2c, reg: u8) -> Result<u8> {
i2c.start().unwrap(); i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() { if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address"); return Err("Si5324 failed to ack write address")
} }
if !i2c.write(reg).unwrap() { if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register"); return Err("Si5324 failed to ack register")
} }
i2c.restart().unwrap(); i2c.restart().unwrap();
if !i2c.write((ADDRESS << 1) | 1).unwrap() { if !i2c.write((ADDRESS << 1) | 1).unwrap() {
return Err("Si5324 failed to ack read address"); return Err("Si5324 failed to ack read address")
} }
let val = i2c.read(false).unwrap(); let val = i2c.read(false).unwrap();
i2c.stop().unwrap(); i2c.stop().unwrap();
Ok(val) Ok(val)
} }
fn rmw<F>(i2c: &mut I2c, reg: u8, f: F) -> Result<()> fn rmw<F>(i2c: &mut I2c, reg: u8, f: F) -> Result<()> where
where F: Fn(u8) -> u8 { F: Fn(u8) -> u8 {
let value = read(i2c, reg)?; let value = read(i2c, reg)?;
write(i2c, reg, f(value))?; write(i2c, reg, f(value))?;
Ok(()) Ok(())
@ -241,15 +235,15 @@ pub fn setup(i2c: &mut I2c, settings: &FrequencySettings, input: Input, timer: &
rmw(i2c, 3, |v| (v & 0x2f) | (cksel_reg << 6) | 0x10)?; // CKSEL_REG, SQ_ICAL=1 rmw(i2c, 3, |v| (v & 0x2f) | (cksel_reg << 6) | 0x10)?; // CKSEL_REG, SQ_ICAL=1
rmw(i2c, 4, |v| (v & 0x3f) | (0b00 << 6))?; // AUTOSEL_REG=b00 rmw(i2c, 4, |v| (v & 0x3f) | (0b00 << 6))?; // AUTOSEL_REG=b00
rmw(i2c, 6, |v| (v & 0xc0) | 0b111111)?; // SFOUT2_REG=b111 SFOUT1_REG=b111 rmw(i2c, 6, |v| (v & 0xc0) | 0b111111)?; // SFOUT2_REG=b111 SFOUT1_REG=b111
write(i2c, 25, (s.n1_hs << 5) as u8)?; write(i2c, 25, (s.n1_hs << 5 ) as u8)?;
write(i2c, 31, (s.nc1_ls >> 16) as u8)?; write(i2c, 31, (s.nc1_ls >> 16) as u8)?;
write(i2c, 32, (s.nc1_ls >> 8) as u8)?; write(i2c, 32, (s.nc1_ls >> 8 ) as u8)?;
write(i2c, 33, (s.nc1_ls) as u8)?; write(i2c, 33, (s.nc1_ls) as u8)?;
write(i2c, 34, (s.nc1_ls >> 16) as u8)?; // write to NC2_LS as well write(i2c, 34, (s.nc1_ls >> 16) as u8)?; // write to NC2_LS as well
write(i2c, 35, (s.nc1_ls >> 8) as u8)?; write(i2c, 35, (s.nc1_ls >> 8 ) as u8)?;
write(i2c, 36, (s.nc1_ls) as u8)?; write(i2c, 36, (s.nc1_ls) as u8)?;
write(i2c, 40, (s.n2_hs << 5) as u8 | (s.n2_ls >> 16) as u8)?; write(i2c, 40, (s.n2_hs << 5 ) as u8 | (s.n2_ls >> 16) as u8)?;
write(i2c, 41, (s.n2_ls >> 8) as u8)?; write(i2c, 41, (s.n2_ls >> 8 ) as u8)?;
write(i2c, 42, (s.n2_ls) as u8)?; write(i2c, 42, (s.n2_ls) as u8)?;
write(i2c, 43, (s.n31 >> 16) as u8)?; write(i2c, 43, (s.n31 >> 16) as u8)?;
write(i2c, 44, (s.n31 >> 8) as u8)?; write(i2c, 44, (s.n31 >> 8) as u8)?;
@ -315,7 +309,9 @@ pub mod siphaser {
csr::siphaser::error_write(1); csr::siphaser::error_write(1);
} }
timer.delay_us(5_000); timer.delay_us(5_000);
unsafe { csr::siphaser::error_read() != 0 } unsafe {
csr::siphaser::error_read() != 0
}
} }
fn find_edge(target: bool, timer: &mut GlobalTimer) -> Result<u32> { fn find_edge(target: bool, timer: &mut GlobalTimer) -> Result<u32> {
@ -344,16 +340,11 @@ pub mod siphaser {
} }
let width = find_edge(true, timer)? + jitter_margin; let width = find_edge(true, timer)? + jitter_margin;
// width is 360 degrees (one full rotation of the phase between s/h limits) minus jitter // width is 360 degrees (one full rotation of the phase between s/h limits) minus jitter
info!( info!("calibration successful, lead: {}, width: {} ({}deg)", lead, width, width*360/(56*8));
"calibration successful, lead: {}, width: {} ({}deg)",
lead,
width,
width * 360 / (56 * 8)
);
// Apply reverse phase shift for half the width to get into the // Apply reverse phase shift for half the width to get into the
// middle of the working region. // middle of the working region.
for _ in 0..width / 2 { for _ in 0..width/2 {
phase_shift(0, timer); phase_shift(0, timer);
} }

View File

@ -1,854 +0,0 @@
use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs;
use libboard_zynq::timer::GlobalTimer;
use log::info;
use crate::pl::csr;
#[cfg(feature = "target_kasli_soc")]
const ADDRESS: u8 = 0x67;
const ADPLL_MAX: i32 = (950.0 / 0.0001164) as i32;
pub struct DividerConfig {
pub hsdiv: u16,
pub lsdiv: u8,
pub fbdiv: u64,
}
pub struct FrequencySetting {
pub main: DividerConfig,
pub helper: DividerConfig,
}
mod i2c {
use super::*;
#[derive(Clone, Copy)]
pub enum DCXO {
Main,
Helper,
}
fn half_period(timer: &mut GlobalTimer) {
timer.delay_us(1)
}
fn sda_i(dcxo: DCXO) -> bool {
match dcxo {
DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_in_read() == 1 },
DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_in_read() == 1 },
}
}
fn sda_oe(dcxo: DCXO, oe: bool) {
let val = if oe { 1 } else { 0 };
match dcxo {
DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_oe_write(val) },
DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_oe_write(val) },
};
}
fn sda_o(dcxo: DCXO, o: bool) {
let val = if o { 1 } else { 0 };
match dcxo {
DCXO::Main => unsafe { csr::wrpll::main_dcxo_sda_out_write(val) },
DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_sda_out_write(val) },
};
}
fn scl_oe(dcxo: DCXO, oe: bool) {
let val = if oe { 1 } else { 0 };
match dcxo {
DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_oe_write(val) },
DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_oe_write(val) },
};
}
fn scl_o(dcxo: DCXO, o: bool) {
let val = if o { 1 } else { 0 };
match dcxo {
DCXO::Main => unsafe { csr::wrpll::main_dcxo_scl_out_write(val) },
DCXO::Helper => unsafe { csr::wrpll::helper_dcxo_scl_out_write(val) },
};
}
pub fn init(dcxo: DCXO, timer: &mut GlobalTimer) -> Result<(), &'static str> {
// Set SCL as output, and high level
scl_o(dcxo, true);
scl_oe(dcxo, true);
// Prepare a zero level on SDA so that sda_oe pulls it down
sda_o(dcxo, false);
// Release SDA
sda_oe(dcxo, false);
// Check the I2C bus is ready
half_period(timer);
half_period(timer);
if !sda_i(dcxo) {
// Try toggling SCL a few times
for _bit in 0..8 {
scl_o(dcxo, false);
half_period(timer);
scl_o(dcxo, true);
half_period(timer);
}
}
if !sda_i(dcxo) {
return Err("SDA is stuck low and doesn't get unstuck");
}
Ok(())
}
pub fn start(dcxo: DCXO, timer: &mut GlobalTimer) {
// Set SCL high then SDA low
scl_o(dcxo, true);
half_period(timer);
sda_oe(dcxo, true);
half_period(timer);
}
pub fn stop(dcxo: DCXO, timer: &mut GlobalTimer) {
// First, make sure SCL is low, so that the target releases the SDA line
scl_o(dcxo, false);
half_period(timer);
// Set SCL high then SDA high
sda_oe(dcxo, true);
scl_o(dcxo, true);
half_period(timer);
sda_oe(dcxo, false);
half_period(timer);
}
pub fn write(dcxo: DCXO, data: u8, timer: &mut GlobalTimer) -> bool {
// MSB first
for bit in (0..8).rev() {
// Set SCL low and set our bit on SDA
scl_o(dcxo, false);
sda_oe(dcxo, data & (1 << bit) == 0);
half_period(timer);
// Set SCL high ; data is shifted on the rising edge of SCL
scl_o(dcxo, true);
half_period(timer);
}
// Check ack
// Set SCL low, then release SDA so that the I2C target can respond
scl_o(dcxo, false);
half_period(timer);
sda_oe(dcxo, false);
// Set SCL high and check for ack
scl_o(dcxo, true);
half_period(timer);
// returns true if acked (I2C target pulled SDA low)
!sda_i(dcxo)
}
pub fn read(dcxo: DCXO, ack: bool, timer: &mut GlobalTimer) -> u8 {
// Set SCL low first, otherwise setting SDA as input may cause a transition
// on SDA with SCL high which will be interpreted as START/STOP condition.
scl_o(dcxo, false);
half_period(timer); // make sure SCL has settled low
sda_oe(dcxo, false);
let mut data: u8 = 0;
// MSB first
for bit in (0..8).rev() {
scl_o(dcxo, false);
half_period(timer);
// Set SCL high and shift data
scl_o(dcxo, true);
half_period(timer);
if sda_i(dcxo) {
data |= 1 << bit
}
}
// Send ack
// Set SCL low and pull SDA low when acking
scl_o(dcxo, false);
if ack {
sda_oe(dcxo, true)
}
half_period(timer);
// then set SCL high
scl_o(dcxo, true);
half_period(timer);
data
}
}
fn write(dcxo: i2c::DCXO, reg: u8, val: u8, timer: &mut GlobalTimer) -> Result<(), &'static str> {
i2c::start(dcxo, timer);
if !i2c::write(dcxo, ADDRESS << 1, timer) {
return Err("Si549 failed to ack write address");
}
if !i2c::write(dcxo, reg, timer) {
return Err("Si549 failed to ack register");
}
if !i2c::write(dcxo, val, timer) {
return Err("Si549 failed to ack value");
}
i2c::stop(dcxo, timer);
Ok(())
}
fn read(dcxo: i2c::DCXO, reg: u8, timer: &mut GlobalTimer) -> Result<u8, &'static str> {
i2c::start(dcxo, timer);
if !i2c::write(dcxo, ADDRESS << 1, timer) {
return Err("Si549 failed to ack write address");
}
if !i2c::write(dcxo, reg, timer) {
return Err("Si549 failed to ack register");
}
i2c::stop(dcxo, timer);
i2c::start(dcxo, timer);
if !i2c::write(dcxo, (ADDRESS << 1) | 1, timer) {
return Err("Si549 failed to ack read address");
}
let val = i2c::read(dcxo, false, timer);
i2c::stop(dcxo, timer);
Ok(val)
}
fn setup(dcxo: i2c::DCXO, config: &DividerConfig, timer: &mut GlobalTimer) -> Result<(), &'static str> {
i2c::init(dcxo, timer)?;
write(dcxo, 255, 0x00, timer)?; // PAGE
write(dcxo, 69, 0x00, timer)?; // Disable FCAL override.
write(dcxo, 17, 0x00, timer)?; // Synchronously disable output
// The Si549 has no ID register, so we check that it responds correctly
// by writing values to a RAM-like register and reading them back.
for test_value in 0..255 {
write(dcxo, 23, test_value, timer)?;
let readback = read(dcxo, 23, timer)?;
if readback != test_value {
return Err("Si549 detection failed");
}
}
write(dcxo, 23, config.hsdiv as u8, timer)?;
write(dcxo, 24, (config.hsdiv >> 8) as u8 | (config.lsdiv << 4), timer)?;
write(dcxo, 26, config.fbdiv as u8, timer)?;
write(dcxo, 27, (config.fbdiv >> 8) as u8, timer)?;
write(dcxo, 28, (config.fbdiv >> 16) as u8, timer)?;
write(dcxo, 29, (config.fbdiv >> 24) as u8, timer)?;
write(dcxo, 30, (config.fbdiv >> 32) as u8, timer)?;
write(dcxo, 31, (config.fbdiv >> 40) as u8, timer)?;
write(dcxo, 7, 0x08, timer)?; // Start FCAL
timer.delay_us(30_000); // Internal FCAL VCO calibration
write(dcxo, 17, 0x01, timer)?; // Synchronously enable output
Ok(())
}
pub fn main_setup(timer: &mut GlobalTimer, settings: &FrequencySetting) -> Result<(), &'static str> {
unsafe {
csr::wrpll::main_dcxo_bitbang_enable_write(1);
csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
}
setup(i2c::DCXO::Main, &settings.main, timer)?;
// Si549 maximum settling time for large frequency change.
timer.delay_us(40_000);
unsafe {
csr::wrpll::main_dcxo_bitbang_enable_write(0);
}
info!("Main Si549 started");
Ok(())
}
pub fn helper_setup(timer: &mut GlobalTimer, settings: &FrequencySetting) -> Result<(), &'static str> {
unsafe {
csr::wrpll::helper_reset_write(1);
csr::wrpll::helper_dcxo_bitbang_enable_write(1);
csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
}
setup(i2c::DCXO::Helper, &settings.helper, timer)?;
// Si549 maximum settling time for large frequency change.
timer.delay_us(40_000);
unsafe {
csr::wrpll::helper_reset_write(0);
csr::wrpll::helper_dcxo_bitbang_enable_write(0);
}
info!("Helper Si549 started");
Ok(())
}
fn set_adpll(dcxo: i2c::DCXO, adpll: i32) -> Result<(), &'static str> {
if adpll.abs() > ADPLL_MAX {
return Err("adpll is too large");
}
match dcxo {
i2c::DCXO::Main => unsafe {
if csr::wrpll::main_dcxo_bitbang_enable_read() == 1 {
return Err("Main si549 bitbang mode is active when using gateware i2c");
}
while csr::wrpll::main_dcxo_adpll_busy_read() == 1 {}
if csr::wrpll::main_dcxo_nack_read() == 1 {
return Err("Main si549 failed to ack adpll write");
}
csr::wrpll::main_dcxo_i2c_address_write(ADDRESS);
csr::wrpll::main_dcxo_adpll_write(adpll as u32);
csr::wrpll::main_dcxo_adpll_stb_write(1);
},
i2c::DCXO::Helper => unsafe {
if csr::wrpll::helper_dcxo_bitbang_enable_read() == 1 {
return Err("Helper si549 bitbang mode is active when using gateware i2c");
}
while csr::wrpll::helper_dcxo_adpll_busy_read() == 1 {}
if csr::wrpll::helper_dcxo_nack_read() == 1 {
return Err("Helper si549 failed to ack adpll write");
}
csr::wrpll::helper_dcxo_i2c_address_write(ADDRESS);
csr::wrpll::helper_dcxo_adpll_write(adpll as u32);
csr::wrpll::helper_dcxo_adpll_stb_write(1);
},
};
Ok(())
}
#[cfg(has_wrpll)]
pub mod wrpll {
use super::*;
const BEATING_PERIOD: i32 = 0x8000;
const BEATING_HALFPERIOD: i32 = 0x4000;
const COUNTER_WIDTH: u32 = 24;
const DIV_WIDTH: u32 = 2;
// y[n] = b0*x[n] + b1*x[n-1] + b2*x[n-2] - a1*y[n-1] - a2*y[n-2]
struct FilterParameters {
pub b0: f64,
pub b1: f64,
pub b2: f64,
pub a1: f64,
pub a2: f64,
}
#[cfg(rtio_frequency = "100.0")]
const LPF: FilterParameters = FilterParameters {
b0: 0.03967479060647884,
b1: 0.07934958121295768,
b2: 0.03967479060647884,
a1: -1.3865593741228928,
a2: 0.5452585365488082,
};
#[cfg(rtio_frequency = "125.0")]
const LPF: FilterParameters = FilterParameters {
b0: 0.07209205036273991,
b1: 0.14418410072547982,
b2: 0.07209205036273991,
a1: -0.6114078511562919,
a2: -0.10022394739274834,
};
static mut H_ADPLL1: i32 = 0;
static mut H_ADPLL2: i32 = 0;
static mut PERIOD_ERR1: i32 = 0;
static mut PERIOD_ERR2: i32 = 0;
static mut M_ADPLL1: i32 = 0;
static mut M_ADPLL2: i32 = 0;
static mut PHASE_ERR1: i32 = 0;
static mut PHASE_ERR2: i32 = 0;
static mut BASE_ADPLL: i32 = 0;
#[derive(Clone, Copy)]
pub enum ISR {
RefTag,
MainTag,
}
mod tag_collector {
use super::*;
#[cfg(wrpll_ref_clk = "GT_CDR")]
static mut TAG_OFFSET: u32 = 8382;
#[cfg(wrpll_ref_clk = "SMA_CLKIN")]
static mut TAG_OFFSET: u32 = 0;
static mut REF_TAG: u32 = 0;
static mut REF_TAG_READY: bool = false;
static mut MAIN_TAG: u32 = 0;
static mut MAIN_TAG_READY: bool = false;
pub fn reset() {
clear_phase_diff_ready();
unsafe {
REF_TAG = 0;
MAIN_TAG = 0;
}
}
pub fn clear_phase_diff_ready() {
unsafe {
REF_TAG_READY = false;
MAIN_TAG_READY = false;
}
}
pub fn collect_tags(interrupt: ISR) {
match interrupt {
ISR::RefTag => unsafe {
REF_TAG = csr::wrpll::ref_tag_read();
REF_TAG_READY = true;
},
ISR::MainTag => unsafe {
MAIN_TAG = csr::wrpll::main_tag_read();
MAIN_TAG_READY = true;
},
}
}
pub fn phase_diff_ready() -> bool {
unsafe { REF_TAG_READY && MAIN_TAG_READY }
}
#[cfg(feature = "calibrate_wrpll_skew")]
pub fn set_tag_offset(offset: u32) {
unsafe {
TAG_OFFSET = offset;
}
}
#[cfg(feature = "calibrate_wrpll_skew")]
pub fn get_tag_offset() -> u32 {
unsafe { TAG_OFFSET }
}
pub fn get_period_error() -> i32 {
// n * BEATING_PERIOD - REF_TAG(n) mod BEATING_PERIOD
let mut period_error = unsafe { REF_TAG.overflowing_neg().0.rem_euclid(BEATING_PERIOD as u32) as i32 };
// mapping tags from [0, 2π] -> [-π, π]
if period_error > BEATING_HALFPERIOD {
period_error -= BEATING_PERIOD
}
period_error
}
pub fn get_phase_error() -> i32 {
// MAIN_TAG(n) - REF_TAG(n) - TAG_OFFSET mod BEATING_PERIOD
let mut phase_error = unsafe {
MAIN_TAG
.overflowing_sub(REF_TAG + TAG_OFFSET)
.0
.rem_euclid(BEATING_PERIOD as u32) as i32
};
// mapping tags from [0, 2π] -> [-π, π]
if phase_error > BEATING_HALFPERIOD {
phase_error -= BEATING_PERIOD
}
phase_error
}
}
fn set_isr(en: bool) {
let val = if en { 1 } else { 0 };
unsafe {
csr::wrpll::ref_tag_ev_enable_write(val);
csr::wrpll::main_tag_ev_enable_write(val);
}
}
fn set_base_adpll() -> Result<(), &'static str> {
let count2adpll =
|error: i32| ((error as f64 * 1e6) / (0.0001164 * (1 << (COUNTER_WIDTH - DIV_WIDTH)) as f64)) as i32;
let (ref_count, main_count) = get_freq_counts();
unsafe {
BASE_ADPLL = count2adpll(ref_count as i32 - main_count as i32);
set_adpll(i2c::DCXO::Main, BASE_ADPLL)?;
set_adpll(i2c::DCXO::Helper, BASE_ADPLL)?;
}
Ok(())
}
fn get_freq_counts() -> (u32, u32) {
unsafe {
csr::wrpll::frequency_counter_update_write(1);
while csr::wrpll::frequency_counter_busy_read() == 1 {}
#[cfg(wrpll_ref_clk = "GT_CDR")]
let ref_count = csr::wrpll::frequency_counter_counter_rtio_rx0_read();
#[cfg(wrpll_ref_clk = "SMA_CLKIN")]
let ref_count = csr::wrpll::frequency_counter_counter_ref_read();
let main_count = csr::wrpll::frequency_counter_counter_sys_read();
(ref_count, main_count)
}
}
fn reset_plls(timer: &mut GlobalTimer) -> Result<(), &'static str> {
unsafe {
H_ADPLL1 = 0;
H_ADPLL2 = 0;
PERIOD_ERR1 = 0;
PERIOD_ERR2 = 0;
M_ADPLL1 = 0;
M_ADPLL2 = 0;
PHASE_ERR1 = 0;
PHASE_ERR2 = 0;
}
set_adpll(i2c::DCXO::Main, 0)?;
set_adpll(i2c::DCXO::Helper, 0)?;
// wait for adpll to transfer and DCXO to settle
timer.delay_us(200);
Ok(())
}
fn clear_pending(interrupt: ISR) {
match interrupt {
ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_write(1) },
ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_write(1) },
};
}
fn is_pending(interrupt: ISR) -> bool {
match interrupt {
ISR::RefTag => unsafe { csr::wrpll::ref_tag_ev_pending_read() == 1 },
ISR::MainTag => unsafe { csr::wrpll::main_tag_ev_pending_read() == 1 },
}
}
pub fn interrupt_handler() {
if is_pending(ISR::RefTag) {
tag_collector::collect_tags(ISR::RefTag);
clear_pending(ISR::RefTag);
helper_pll().expect("failed to run helper DCXO PLL");
}
if is_pending(ISR::MainTag) {
tag_collector::collect_tags(ISR::MainTag);
clear_pending(ISR::MainTag);
}
if tag_collector::phase_diff_ready() {
main_pll().expect("failed to run main DCXO PLL");
tag_collector::clear_phase_diff_ready();
}
}
fn helper_pll() -> Result<(), &'static str> {
let period_err = tag_collector::get_period_error();
unsafe {
let adpll = ((LPF.b0 * period_err as f64) + (LPF.b1 * PERIOD_ERR1 as f64) + (LPF.b2 * PERIOD_ERR2 as f64)
- (LPF.a1 * H_ADPLL1 as f64)
- (LPF.a2 * H_ADPLL2 as f64)) as i32;
set_adpll(i2c::DCXO::Helper, BASE_ADPLL + adpll)?;
H_ADPLL2 = H_ADPLL1;
PERIOD_ERR2 = PERIOD_ERR1;
H_ADPLL1 = adpll;
PERIOD_ERR1 = period_err;
};
Ok(())
}
fn main_pll() -> Result<(), &'static str> {
let phase_err = tag_collector::get_phase_error();
unsafe {
let adpll = ((LPF.b0 * phase_err as f64) + (LPF.b1 * PHASE_ERR1 as f64) + (LPF.b2 * PHASE_ERR2 as f64)
- (LPF.a1 * M_ADPLL1 as f64)
- (LPF.a2 * M_ADPLL2 as f64)) as i32;
set_adpll(i2c::DCXO::Main, BASE_ADPLL + adpll)?;
M_ADPLL2 = M_ADPLL1;
PHASE_ERR2 = PHASE_ERR1;
M_ADPLL1 = adpll;
PHASE_ERR1 = phase_err;
};
Ok(())
}
#[cfg(wrpll_ref_clk = "GT_CDR")]
fn test_skew(timer: &mut GlobalTimer) -> Result<(), &'static str> {
// wait for PLL to stabilize
timer.delay_us(20_000);
info!("testing the skew of SYS CLK...");
if has_timing_error(timer) {
return Err("the skew cannot satisfy setup/hold time constraint of RX synchronizer");
}
info!("the skew of SYS CLK met the timing constraint");
Ok(())
}
#[cfg(wrpll_ref_clk = "GT_CDR")]
fn has_timing_error(timer: &mut GlobalTimer) -> bool {
unsafe {
csr::wrpll_skewtester::error_write(1);
}
timer.delay_us(5_000);
unsafe { csr::wrpll_skewtester::error_read() == 1 }
}
#[cfg(feature = "calibrate_wrpll_skew")]
fn find_edge(target: bool, timer: &mut GlobalTimer) -> Result<u32, &'static str> {
const STEP: u32 = 8;
const STABLE_THRESHOLD: u32 = 10;
enum FSM {
Init,
WaitEdge,
GotEdge,
}
let mut state: FSM = FSM::Init;
let mut offset: u32 = tag_collector::get_tag_offset();
let mut median_edge: u32 = 0;
let mut stable_counter: u32 = 0;
for _ in 0..(BEATING_PERIOD as u32 / STEP) as usize {
tag_collector::set_tag_offset(offset);
offset += STEP;
// wait for PLL to stabilize
timer.delay_us(20_000);
let error = has_timing_error(timer);
// A median edge deglitcher
match state {
FSM::Init => {
if error != target {
stable_counter += 1;
} else {
stable_counter = 0;
}
if stable_counter >= STABLE_THRESHOLD {
state = FSM::WaitEdge;
stable_counter = 0;
}
}
FSM::WaitEdge => {
if error == target {
state = FSM::GotEdge;
median_edge = offset;
}
}
FSM::GotEdge => {
if error != target {
median_edge += STEP;
stable_counter = 0;
} else {
stable_counter += 1;
}
if stable_counter >= STABLE_THRESHOLD {
return Ok(median_edge);
}
}
}
}
return Err("failed to find timing error edge");
}
#[cfg(feature = "calibrate_wrpll_skew")]
fn calibrate_skew(timer: &mut GlobalTimer) -> Result<(), &'static str> {
info!("calibrating skew to meet timing constraint...");
// clear calibrated value
tag_collector::set_tag_offset(0);
let rising = find_edge(true, timer)? as i32;
let falling = find_edge(false, timer)? as i32;
let width = BEATING_PERIOD - (falling - rising);
let result = falling + width / 2;
tag_collector::set_tag_offset(result as u32);
info!(
"calibration successful, error zone: {} -> {}, width: {} ({}deg), middle of working region: {}",
rising,
falling,
width,
360 * width / BEATING_PERIOD,
result,
);
Ok(())
}
pub fn select_recovered_clock(rc: bool, timer: &mut GlobalTimer) {
set_isr(false);
if rc {
tag_collector::reset();
reset_plls(timer).expect("failed to reset main and helper PLL");
// get within capture range
set_base_adpll().expect("failed to set base adpll");
// clear gateware pending flag
clear_pending(ISR::RefTag);
clear_pending(ISR::MainTag);
// use nFIQ to avoid IRQ being disabled by mutex lock and mess up PLL
set_isr(true);
info!("WRPLL interrupt enabled");
#[cfg(feature = "calibrate_wrpll_skew")]
calibrate_skew(timer).expect("failed to set the correct skew");
#[cfg(wrpll_ref_clk = "GT_CDR")]
test_skew(timer).expect("skew test failed");
}
}
}
#[cfg(has_wrpll_refclk)]
pub mod wrpll_refclk {
use super::*;
pub struct MmcmSetting {
pub clkout0_reg1: u16, //0x08
pub clkout0_reg2: u16, //0x09
pub clkfbout_reg1: u16, //0x14
pub clkfbout_reg2: u16, //0x15
pub div_reg: u16, //0x16
pub lock_reg1: u16, //0x18
pub lock_reg2: u16, //0x19
pub lock_reg3: u16, //0x1A
pub power_reg: u16, //0x28
pub filt_reg1: u16, //0x4E
pub filt_reg2: u16, //0x4F
}
fn one_clock_cycle() {
unsafe {
csr::wrpll_refclk::mmcm_dclk_write(1);
csr::wrpll_refclk::mmcm_dclk_write(0);
}
}
fn set_addr(address: u8) {
unsafe {
csr::wrpll_refclk::mmcm_daddr_write(address);
}
}
fn set_data(value: u16) {
unsafe {
csr::wrpll_refclk::mmcm_din_write(value);
}
}
fn set_enable(en: bool) {
unsafe {
let val = if en { 1 } else { 0 };
csr::wrpll_refclk::mmcm_den_write(val);
}
}
fn set_write_enable(en: bool) {
unsafe {
let val = if en { 1 } else { 0 };
csr::wrpll_refclk::mmcm_dwen_write(val);
}
}
fn get_data() -> u16 {
unsafe { csr::wrpll_refclk::mmcm_dout_read() }
}
fn drp_ready() -> bool {
unsafe { csr::wrpll_refclk::mmcm_dready_read() == 1 }
}
#[allow(dead_code)]
fn read(address: u8) -> u16 {
set_addr(address);
set_enable(true);
// Set DADDR on the mmcm and assert DEN for one clock cycle
one_clock_cycle();
set_enable(false);
while !drp_ready() {
// keep the clock signal until data is ready
one_clock_cycle();
}
get_data()
}
fn write(address: u8, value: u16) {
set_addr(address);
set_data(value);
set_write_enable(true);
set_enable(true);
// Set DADDR, DI on the mmcm and assert DWE, DEN for one clock cycle
one_clock_cycle();
set_write_enable(false);
set_enable(false);
while !drp_ready() {
// keep the clock signal until write is finished
one_clock_cycle();
}
}
fn reset(rst: bool) {
unsafe {
let val = if rst { 1 } else { 0 };
csr::wrpll_refclk::mmcm_reset_write(val)
}
}
pub fn setup(timer: &mut GlobalTimer, settings: MmcmSetting, mmcm_bypass: bool) -> Result<(), &'static str> {
unsafe {
csr::wrpll_refclk::refclk_reset_write(1);
}
if mmcm_bypass {
info!("Bypassing mmcm");
unsafe {
csr::wrpll_refclk::mmcm_bypass_write(1);
}
} else {
// Based on "DRP State Machine" from XAPP888
// hold reset HIGH during mmcm config
reset(true);
write(0x08, settings.clkout0_reg1);
write(0x09, settings.clkout0_reg2);
write(0x14, settings.clkfbout_reg1);
write(0x15, settings.clkfbout_reg2);
write(0x16, settings.div_reg);
write(0x18, settings.lock_reg1);
write(0x19, settings.lock_reg2);
write(0x1A, settings.lock_reg3);
write(0x28, settings.power_reg);
write(0x4E, settings.filt_reg1);
write(0x4F, settings.filt_reg2);
reset(false);
// wait for the mmcm to lock
timer.delay_us(100);
let locked = unsafe { csr::wrpll_refclk::mmcm_locked_read() == 1 };
if !locked {
return Err("mmcm failed to generate 125MHz ref clock from SMA CLKIN");
}
}
unsafe {
csr::wrpll_refclk::refclk_reset_write(0);
}
Ok(())
}
}

View File

@ -1,7 +1,8 @@
use std::{env, use std::env;
fs::File, use std::fs::File;
io::{BufRead, BufReader, Write}, use std::io::{BufRead, BufReader, Write};
path::PathBuf}; use std::path::PathBuf;
pub fn add_linker_script() { pub fn add_linker_script() {
// Put the linker script somewhere the linker can find it // Put the linker script somewhere the linker can find it

View File

@ -10,9 +10,7 @@ SECTIONS
__text_start = .; __text_start = .;
.text : .text :
{ {
__exceptions_start = .;
KEEP(*(.text.exceptions)); KEEP(*(.text.exceptions));
__exceptions_end = .;
*(.text.boot); *(.text.boot);
*(.text .text.*); *(.text .text.*);
} > SDRAM } > SDRAM

View File

@ -6,7 +6,7 @@ edition = "2018"
build = "build.rs" build = "build.rs"
[dependencies] [dependencies]
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq" } libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
[build-dependencies] [build-dependencies]
cc = { version = "1.0.1" } cc = { version = "1.0.1" }

View File

@ -4,7 +4,8 @@ fn main() {
} }
mod libc { mod libc {
use std::{env, path::Path}; use std::path::Path;
use std::env;
pub fn compile() { pub fn compile() {
let cfg = &mut cc::Build::new(); let cfg = &mut cc::Build::new();
@ -31,7 +32,9 @@ mod libc {
cfg.flag("-U_FORTIFY_SOURCE"); cfg.flag("-U_FORTIFY_SOURCE");
cfg.define("_FORTIFY_SOURCE", Some("0")); cfg.define("_FORTIFY_SOURCE", Some("0"));
let sources = vec!["printf.c"]; let sources = vec![
"printf.c"
];
let root = Path::new("./"); let root = Path::new("./");
for src in sources { for src in sources {

View File

@ -11,11 +11,9 @@
#![allow(non_upper_case_globals)] #![allow(non_upper_case_globals)]
#![allow(unused)] #![allow(unused)]
use core::mem;
use cslice::CSlice;
use crate::DwarfReader; use crate::DwarfReader;
use core::mem;
use cslice::CSlice;
pub const DW_EH_PE_omit: u8 = 0xFF; pub const DW_EH_PE_omit: u8 = 0xFF;
pub const DW_EH_PE_absptr: u8 = 0x00; pub const DW_EH_PE_absptr: u8 = 0x00;
@ -85,7 +83,10 @@ unsafe fn get_ttype_entry(
encoding | DW_EH_PE_pcrel, encoding | DW_EH_PE_pcrel,
ttype_base, ttype_base,
) )
.map(|v| (v != ttype_base).then(|| v as *const u8)) .map(|v| match v {
ttype_base => None,
ttype_entry => Some(ttype_entry as *const u8),
})
} }
pub unsafe fn find_eh_action( pub unsafe fn find_eh_action(
@ -159,11 +160,15 @@ pub unsafe fn find_eh_action(
if ar_filter == 0 { if ar_filter == 0 {
saw_cleanup = true; saw_cleanup = true;
} else if ar_filter > 0 { } else if ar_filter > 0 {
let catch_type = let catch_type = get_ttype_entry(
get_ttype_entry(ar_filter as usize, ttype_encoding, ttype_base, ttype_table)?; ar_filter as usize,
ttype_encoding,
ttype_base,
ttype_table,
)?;
match catch_type { match catch_type {
Some(clause_ptr) if *(clause_ptr as *const u32) == id => { Some(clause_ptr) if *(clause_ptr as *const u32) == id => {
return Ok(EHAction::Catch(lpad)); return Ok(EHAction::Catch(lpad))
} }
None => return Ok(EHAction::Catch(lpad)), None => return Ok(EHAction::Catch(lpad)),
_ => {} _ => {}
@ -246,11 +251,19 @@ fn get_base(encoding: u8, context: &EHContext<'_>) -> Result<usize, ()> {
} }
} }
unsafe fn read_encoded_pointer(reader: &mut DwarfReader, context: &EHContext<'_>, encoding: u8) -> Result<usize, ()> { unsafe fn read_encoded_pointer(
reader: &mut DwarfReader,
context: &EHContext<'_>,
encoding: u8,
) -> Result<usize, ()> {
read_encoded_pointer_with_base(reader, encoding, get_base(encoding, context)?) read_encoded_pointer_with_base(reader, encoding, get_base(encoding, context)?)
} }
unsafe fn read_encoded_pointer_with_base(reader: &mut DwarfReader, encoding: u8, base: usize) -> Result<usize, ()> { unsafe fn read_encoded_pointer_with_base(
reader: &mut DwarfReader,
encoding: u8,
base: usize,
) -> Result<usize, ()> {
if encoding == DW_EH_PE_omit { if encoding == DW_EH_PE_omit {
return Err(()); return Err(());
} }

View File

@ -8,4 +8,4 @@ name = "dyld"
[dependencies] [dependencies]
log = "0.4" log = "0.4"
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" } libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }

View File

@ -1451,7 +1451,8 @@ pub const R_AARCH64_TLSDESC_CALL: usize = 569;
pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12: usize = 570; pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12: usize = 570;
pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12_NC: usize = 571; pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12_NC: usize = 571;
pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12: usize = 572; pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12: usize = 572;
pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC: usize = 573; pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC: usize =
573;
pub const R_AARCH64_COPY: usize = 1024; pub const R_AARCH64_COPY: usize = 1024;
pub const R_AARCH64_GLOB_DAT: usize = 1025; pub const R_AARCH64_GLOB_DAT: usize = 1025;
pub const R_AARCH64_JUMP_SLOT: usize = 1026; pub const R_AARCH64_JUMP_SLOT: usize = 1026;
@ -2266,9 +2267,7 @@ pub struct Elf32_Ehdr {
pub e_shstrndx: Elf32_Half, pub e_shstrndx: Elf32_Half,
} }
impl Clone for Elf32_Ehdr { impl Clone for Elf32_Ehdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2289,9 +2288,7 @@ pub struct Elf64_Ehdr {
pub e_shstrndx: Elf64_Half, pub e_shstrndx: Elf64_Half,
} }
impl Clone for Elf64_Ehdr { impl Clone for Elf64_Ehdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2308,9 +2305,7 @@ pub struct Elf32_Shdr {
pub sh_entsize: Elf32_Word, pub sh_entsize: Elf32_Word,
} }
impl Clone for Elf32_Shdr { impl Clone for Elf32_Shdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2327,9 +2322,7 @@ pub struct Elf64_Shdr {
pub sh_entsize: Elf64_Xword, pub sh_entsize: Elf64_Xword,
} }
impl Clone for Elf64_Shdr { impl Clone for Elf64_Shdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2342,9 +2335,7 @@ pub struct Elf32_Sym {
pub st_shndx: Elf32_Section, pub st_shndx: Elf32_Section,
} }
impl Clone for Elf32_Sym { impl Clone for Elf32_Sym {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2357,9 +2348,7 @@ pub struct Elf64_Sym {
pub st_size: Elf64_Xword, pub st_size: Elf64_Xword,
} }
impl Clone for Elf64_Sym { impl Clone for Elf64_Sym {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2368,9 +2357,7 @@ pub struct Elf32_Syminfo {
pub si_flags: Elf32_Half, pub si_flags: Elf32_Half,
} }
impl Clone for Elf32_Syminfo { impl Clone for Elf32_Syminfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2379,9 +2366,7 @@ pub struct Elf64_Syminfo {
pub si_flags: Elf64_Half, pub si_flags: Elf64_Half,
} }
impl Clone for Elf64_Syminfo { impl Clone for Elf64_Syminfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2390,9 +2375,7 @@ pub struct Elf32_Rel {
pub r_info: Elf32_Word, pub r_info: Elf32_Word,
} }
impl Clone for Elf32_Rel { impl Clone for Elf32_Rel {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2401,9 +2384,7 @@ pub struct Elf64_Rel {
pub r_info: Elf64_Xword, pub r_info: Elf64_Xword,
} }
impl Clone for Elf64_Rel { impl Clone for Elf64_Rel {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2413,9 +2394,7 @@ pub struct Elf32_Rela {
pub r_addend: Elf32_Sword, pub r_addend: Elf32_Sword,
} }
impl Clone for Elf32_Rela { impl Clone for Elf32_Rela {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2425,9 +2404,7 @@ pub struct Elf64_Rela {
pub r_addend: Elf64_Sxword, pub r_addend: Elf64_Sxword,
} }
impl Clone for Elf64_Rela { impl Clone for Elf64_Rela {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2442,9 +2419,7 @@ pub struct Elf32_Phdr {
pub p_align: Elf32_Word, pub p_align: Elf32_Word,
} }
impl Clone for Elf32_Phdr { impl Clone for Elf32_Phdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2459,9 +2434,7 @@ pub struct Elf64_Phdr {
pub p_align: Elf64_Xword, pub p_align: Elf64_Xword,
} }
impl Clone for Elf64_Phdr { impl Clone for Elf64_Phdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2476,14 +2449,10 @@ pub union Elf32_Dyn__bindgen_ty_1 {
pub d_ptr: Elf32_Addr, pub d_ptr: Elf32_Addr,
} }
impl Clone for Elf32_Dyn__bindgen_ty_1 { impl Clone for Elf32_Dyn__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_Dyn { impl Clone for Elf32_Dyn {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2498,14 +2467,10 @@ pub union Elf64_Dyn__bindgen_ty_1 {
pub d_ptr: Elf64_Addr, pub d_ptr: Elf64_Addr,
} }
impl Clone for Elf64_Dyn__bindgen_ty_1 { impl Clone for Elf64_Dyn__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf64_Dyn { impl Clone for Elf64_Dyn {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2519,9 +2484,7 @@ pub struct Elf32_Verdef {
pub vd_next: Elf32_Word, pub vd_next: Elf32_Word,
} }
impl Clone for Elf32_Verdef { impl Clone for Elf32_Verdef {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2535,9 +2498,7 @@ pub struct Elf64_Verdef {
pub vd_next: Elf64_Word, pub vd_next: Elf64_Word,
} }
impl Clone for Elf64_Verdef { impl Clone for Elf64_Verdef {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2546,9 +2507,7 @@ pub struct Elf32_Verdaux {
pub vda_next: Elf32_Word, pub vda_next: Elf32_Word,
} }
impl Clone for Elf32_Verdaux { impl Clone for Elf32_Verdaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2557,9 +2516,7 @@ pub struct Elf64_Verdaux {
pub vda_next: Elf64_Word, pub vda_next: Elf64_Word,
} }
impl Clone for Elf64_Verdaux { impl Clone for Elf64_Verdaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2571,9 +2528,7 @@ pub struct Elf32_Verneed {
pub vn_next: Elf32_Word, pub vn_next: Elf32_Word,
} }
impl Clone for Elf32_Verneed { impl Clone for Elf32_Verneed {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2585,9 +2540,7 @@ pub struct Elf64_Verneed {
pub vn_next: Elf64_Word, pub vn_next: Elf64_Word,
} }
impl Clone for Elf64_Verneed { impl Clone for Elf64_Verneed {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2599,9 +2552,7 @@ pub struct Elf32_Vernaux {
pub vna_next: Elf32_Word, pub vna_next: Elf32_Word,
} }
impl Clone for Elf32_Vernaux { impl Clone for Elf32_Vernaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2613,9 +2564,7 @@ pub struct Elf64_Vernaux {
pub vna_next: Elf64_Word, pub vna_next: Elf64_Word,
} }
impl Clone for Elf64_Vernaux { impl Clone for Elf64_Vernaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2629,14 +2578,10 @@ pub union Elf32_auxv_t__bindgen_ty_1 {
pub a_val: u32, pub a_val: u32,
} }
impl Clone for Elf32_auxv_t__bindgen_ty_1 { impl Clone for Elf32_auxv_t__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_auxv_t { impl Clone for Elf32_auxv_t {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2650,14 +2595,10 @@ pub union Elf64_auxv_t__bindgen_ty_1 {
pub a_val: u64, pub a_val: u64,
} }
impl Clone for Elf64_auxv_t__bindgen_ty_1 { impl Clone for Elf64_auxv_t__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf64_auxv_t { impl Clone for Elf64_auxv_t {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2667,9 +2608,7 @@ pub struct Elf32_Nhdr {
pub n_type: Elf32_Word, pub n_type: Elf32_Word,
} }
impl Clone for Elf32_Nhdr { impl Clone for Elf32_Nhdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2679,9 +2618,7 @@ pub struct Elf64_Nhdr {
pub n_type: Elf64_Word, pub n_type: Elf64_Word,
} }
impl Clone for Elf64_Nhdr { impl Clone for Elf64_Nhdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2693,9 +2630,7 @@ pub struct Elf32_Move {
pub m_stride: Elf32_Half, pub m_stride: Elf32_Half,
} }
impl Clone for Elf32_Move { impl Clone for Elf32_Move {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2707,9 +2642,7 @@ pub struct Elf64_Move {
pub m_stride: Elf64_Half, pub m_stride: Elf64_Half,
} }
impl Clone for Elf64_Move { impl Clone for Elf64_Move {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2724,9 +2657,7 @@ pub struct Elf32_gptab__bindgen_ty_1 {
pub gt_unused: Elf32_Word, pub gt_unused: Elf32_Word,
} }
impl Clone for Elf32_gptab__bindgen_ty_1 { impl Clone for Elf32_gptab__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2735,14 +2666,10 @@ pub struct Elf32_gptab__bindgen_ty_2 {
pub gt_bytes: Elf32_Word, pub gt_bytes: Elf32_Word,
} }
impl Clone for Elf32_gptab__bindgen_ty_2 { impl Clone for Elf32_gptab__bindgen_ty_2 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_gptab { impl Clone for Elf32_gptab {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2752,9 +2679,7 @@ pub struct Elf32_RegInfo {
pub ri_gp_value: Elf32_Sword, pub ri_gp_value: Elf32_Sword,
} }
impl Clone for Elf32_RegInfo { impl Clone for Elf32_RegInfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2765,9 +2690,7 @@ pub struct Elf_Options {
pub info: Elf32_Word, pub info: Elf32_Word,
} }
impl Clone for Elf_Options { impl Clone for Elf_Options {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2776,9 +2699,7 @@ pub struct Elf_Options_Hw {
pub hwp_flags2: Elf32_Word, pub hwp_flags2: Elf32_Word,
} }
impl Clone for Elf_Options_Hw { impl Clone for Elf_Options_Hw {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2790,9 +2711,7 @@ pub struct Elf32_Lib {
pub l_flags: Elf32_Word, pub l_flags: Elf32_Word,
} }
impl Clone for Elf32_Lib { impl Clone for Elf32_Lib {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2804,31 +2723,17 @@ pub struct Elf64_Lib {
pub l_flags: Elf64_Word, pub l_flags: Elf64_Word,
} }
impl Clone for Elf64_Lib { impl Clone for Elf64_Lib {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
pub type Elf32_Conflict = Elf32_Addr; pub type Elf32_Conflict = Elf32_Addr;
#[repr(C)] #[repr(C)]
#[derive(Clone, Copy)] #[derive(Clone, Copy)]
pub struct EXIDX_Entry(u32, u32); pub struct EXIDX_Entry(u32, u32);
pub fn ELF32_R_SYM(info: Elf32_Word) -> Elf32_Word { pub fn ELF32_R_SYM(info: Elf32_Word) -> Elf32_Word { info >> 8 }
info >> 8 pub fn ELF32_R_TYPE(info: Elf32_Word) -> u8 { info as u8 }
} pub fn ELF32_R_INFO(sym: Elf32_Word, ty: u8) -> Elf32_Word { sym << 8 | ty as Elf32_Word }
pub fn ELF32_R_TYPE(info: Elf32_Word) -> u8 {
info as u8
}
pub fn ELF32_R_INFO(sym: Elf32_Word, ty: u8) -> Elf32_Word {
sym << 8 | ty as Elf32_Word
}
pub fn ELF32_ST_BIND(info: u8) -> u8 { pub fn ELF32_ST_BIND(info: u8) -> u8 { info >> 4 }
info >> 4 pub fn ELF32_ST_TYPE(info: u8) -> u8 { info & 0xf }
} pub fn ELF32_ST_INFO(bind: u8, ty: u8) -> u8 { (bind << 4) | (ty & 0xf) }
pub fn ELF32_ST_TYPE(info: u8) -> u8 {
info & 0xf
}
pub fn ELF32_ST_INFO(bind: u8, ty: u8) -> u8 {
(bind << 4) | (ty & 0xf)
}

View File

@ -1,8 +1,8 @@
use core::{mem, use core::{mem, ptr, ops::{Deref, Range}};
ops::{Deref, Range}, use super::{
ptr}; Arch,
elf::*,
use super::{elf::*, Arch}; };
fn read_unaligned<T: Copy>(data: &[u8], offset: usize) -> Option<T> { fn read_unaligned<T: Copy>(data: &[u8], offset: usize) -> Option<T> {
if data.len() < offset + mem::size_of::<T>() { if data.len() < offset + mem::size_of::<T>() {
@ -31,40 +31,14 @@ impl<'a> File<'a> {
pub fn arch(&self) -> Option<Arch> { pub fn arch(&self) -> Option<Arch> {
const IDENT_OPENRISC: [u8; EI_NIDENT] = [ const IDENT_OPENRISC: [u8; EI_NIDENT] = [
ELFMAG0, ELFMAG0, ELFMAG1, ELFMAG2, ELFMAG3,
ELFMAG1, ELFCLASS32, ELFDATA2MSB, EV_CURRENT, ELFOSABI_NONE,
ELFMAG2, /* ABI version */ 0, /* padding */ 0, 0, 0, 0, 0, 0, 0
ELFMAG3,
ELFCLASS32,
ELFDATA2MSB,
EV_CURRENT,
ELFOSABI_NONE,
/* ABI version */ 0,
/* padding */ 0,
0,
0,
0,
0,
0,
0,
]; ];
const IDENT_ARM: [u8; EI_NIDENT] = [ const IDENT_ARM: [u8; EI_NIDENT] = [
ELFMAG0, ELFMAG0, ELFMAG1, ELFMAG2, ELFMAG3,
ELFMAG1, ELFCLASS32, ELFDATA2LSB, EV_CURRENT, ELFOSABI_NONE,
ELFMAG2, /* ABI version */ 0, /* padding */ 0, 0, 0, 0, 0, 0, 0
ELFMAG3,
ELFCLASS32,
ELFDATA2LSB,
EV_CURRENT,
ELFOSABI_NONE,
/* ABI version */ 0,
/* padding */ 0,
0,
0,
0,
0,
0,
0,
]; ];
match (self.ehdr.e_ident, self.ehdr.e_machine) { match (self.ehdr.e_ident, self.ehdr.e_machine) {
@ -74,14 +48,16 @@ impl<'a> File<'a> {
} }
} }
pub fn program_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Phdr>> + 'b { pub fn program_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Phdr>> + 'b
{
(0..self.ehdr.e_phnum).map(move |i| { (0..self.ehdr.e_phnum).map(move |i| {
let phdr_off = self.ehdr.e_phoff as usize + mem::size_of::<Elf32_Phdr>() * i as usize; let phdr_off = self.ehdr.e_phoff as usize + mem::size_of::<Elf32_Phdr>() * i as usize;
self.read_unaligned::<Elf32_Phdr>(phdr_off) self.read_unaligned::<Elf32_Phdr>(phdr_off)
}) })
} }
pub fn section_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Shdr>> + 'b { pub fn section_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Shdr>> + 'b
{
(0..self.ehdr.e_shnum).map(move |i| { (0..self.ehdr.e_shnum).map(move |i| {
let shdr_off = self.ehdr.e_shoff as usize + mem::size_of::<Elf32_Shdr>() * i as usize; let shdr_off = self.ehdr.e_shoff as usize + mem::size_of::<Elf32_Shdr>() * i as usize;
self.read_unaligned::<Elf32_Shdr>(shdr_off) self.read_unaligned::<Elf32_Shdr>(shdr_off)

View File

@ -1,9 +1,13 @@
use alloc::alloc::{alloc_zeroed, dealloc, Layout, LayoutError}; use core::{
use core::{mem,
ops::{Deref, DerefMut, Range}, ops::{Deref, DerefMut, Range},
slice}; mem,
slice,
use super::{elf::*, Error}; };
use alloc::alloc::{alloc_zeroed, dealloc, Layout, LayoutError};
use super::{
elf::*,
Error,
};
pub struct DynamicSection { pub struct DynamicSection {
pub strtab: Range<usize>, pub strtab: Range<usize>,
@ -30,12 +34,17 @@ impl Image {
slice::from_raw_parts_mut(ptr, size) slice::from_raw_parts_mut(ptr, size)
}; };
Ok(Image { layout, data }) Ok(Image {
layout,
data,
})
} }
/// assumes that self.data is properly aligned /// assumes that self.data is properly aligned
pub(crate) fn get_ref<T>(&self, offset: usize) -> Option<&T> pub(crate) fn get_ref<T>(&self, offset: usize) -> Option<&T>
where T: Copy { where
T: Copy,
{
if self.data.len() < offset + mem::size_of::<T>() { if self.data.len() < offset + mem::size_of::<T>() {
None None
} else if (self.data.as_ptr() as usize + offset) & (mem::align_of::<T>() - 1) != 0 { } else if (self.data.as_ptr() as usize + offset) & (mem::align_of::<T>() - 1) != 0 {
@ -57,10 +66,14 @@ impl Image {
unsafe { slice::from_raw_parts(ptr, len) } unsafe { slice::from_raw_parts(ptr, len) }
} }
fn dyn_headers<'a>(&'a self, range: Range<usize>) -> impl Iterator<Item = &'a Elf32_Dyn> + 'a { fn dyn_headers<'a>(&'a self, range: Range<usize>) ->
impl Iterator<Item = &'a Elf32_Dyn> + 'a
{
range range
.step_by(mem::size_of::<Elf32_Dyn>()) .step_by(mem::size_of::<Elf32_Dyn>())
.filter_map(move |offset| self.get_ref::<Elf32_Dyn>(offset)) .filter_map(move |offset| {
self.get_ref::<Elf32_Dyn>(offset)
})
.take_while(|d| unsafe { d.d_un.d_val } as i32 != DT_NULL) .take_while(|d| unsafe { d.d_un.d_val } as i32 != DT_NULL)
} }
@ -94,16 +107,14 @@ impl Image {
DT_JMPREL => pltrel_off = val, DT_JMPREL => pltrel_off = val,
DT_PLTRELSZ => pltrel_sz = val, DT_PLTRELSZ => pltrel_sz = val,
DT_HASH => { DT_HASH => {
nbucket = *self nbucket = *self.get_ref::<Elf32_Word>(val + 0)
.get_ref::<Elf32_Word>(val + 0)
.ok_or("cannot read hash bucket count")? as usize; .ok_or("cannot read hash bucket count")? as usize;
nchain = *self nchain = *self.get_ref::<Elf32_Word>(val + 4)
.get_ref::<Elf32_Word>(val + 4)
.ok_or("cannot read hash chain count")? as usize; .ok_or("cannot read hash chain count")? as usize;
hash_off = val + 8; hash_off = val + 8;
hash_sz = (nbucket + nchain) * mem::size_of::<Elf32_Word>(); hash_sz = (nbucket + nchain) * mem::size_of::<Elf32_Word>();
} }
_ => (), _ => ()
} }
} }
@ -112,28 +123,28 @@ impl Image {
let symtab_sz = nchain * mem::size_of::<Elf32_Sym>(); let symtab_sz = nchain * mem::size_of::<Elf32_Sym>();
if strtab_off + strtab_sz > self.data.len() { if strtab_off + strtab_sz > self.data.len() {
return Err("invalid strtab offset/size")?; return Err("invalid strtab offset/size")?
} }
if symtab_off + symtab_sz > self.data.len() { if symtab_off + symtab_sz > self.data.len() {
return Err("invalid symtab offset/size")?; return Err("invalid symtab offset/size")?
} }
if sym_ent != mem::size_of::<Elf32_Sym>() { if sym_ent != mem::size_of::<Elf32_Sym>() {
return Err("incorrect symbol entry size")?; return Err("incorrect symbol entry size")?
} }
if rel_off + rel_sz > self.data.len() { if rel_off + rel_sz > self.data.len() {
return Err("invalid rel offset/size")?; return Err("invalid rel offset/size")?
} }
if rel_ent != 0 && rel_ent != mem::size_of::<Elf32_Rel>() { if rel_ent != 0 && rel_ent != mem::size_of::<Elf32_Rel>() {
return Err("incorrect relocation entry size")?; return Err("incorrect relocation entry size")?
} }
if rela_off + rela_sz > self.data.len() { if rela_off + rela_sz > self.data.len() {
return Err("invalid rela offset/size")?; return Err("invalid rela offset/size")?
} }
if rela_ent != 0 && rela_ent != mem::size_of::<Elf32_Rela>() { if rela_ent != 0 && rela_ent != mem::size_of::<Elf32_Rela>() {
return Err("incorrect relocation entry size")?; return Err("incorrect relocation entry size")?
} }
if pltrel_off + pltrel_sz > self.data.len() { if pltrel_off + pltrel_sz > self.data.len() {
return Err("invalid pltrel offset/size")?; return Err("invalid pltrel offset/size")?
} }
Ok(DynamicSection { Ok(DynamicSection {
@ -154,7 +165,7 @@ impl Image {
pub fn write(&self, offset: usize, value: Elf32_Word) -> Result<(), Error> { pub fn write(&self, offset: usize, value: Elf32_Word) -> Result<(), Error> {
if offset + mem::size_of::<Elf32_Addr>() > self.data.len() { if offset + mem::size_of::<Elf32_Addr>() > self.data.len() {
return Err("relocation out of image bounds")?; return Err("relocation out of image bounds")?
} }
let ptr = (self.data.as_ptr() as usize + offset) as *mut Elf32_Addr; let ptr = (self.data.as_ptr() as usize + offset) as *mut Elf32_Addr;

View File

@ -1,14 +1,13 @@
#![no_std] #![no_std]
extern crate alloc; extern crate alloc;
extern crate libcortex_a9;
extern crate log; extern crate log;
extern crate libcortex_a9;
use alloc::string::String;
use core::{convert, fmt, ops::Range, str}; use core::{convert, fmt, ops::Range, str};
use alloc::string::String;
use elf::*;
use log::{debug, trace}; use log::{debug, trace};
use elf::*;
pub mod elf; pub mod elf;
mod file; mod file;
@ -22,10 +21,11 @@ pub enum Arch {
OpenRisc, OpenRisc,
} }
#[derive(Debug)] #[derive(Debug)]
pub enum Error { pub enum Error {
Parsing(&'static str), Parsing(&'static str),
Lookup(String), Lookup(String)
} }
impl convert::From<&'static str> for Error { impl convert::From<&'static str> for Error {
@ -37,8 +37,10 @@ impl convert::From<&'static str> for Error {
impl fmt::Display for Error { impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {
&Error::Parsing(desc) => write!(f, "parse error: {}", desc), &Error::Parsing(desc) =>
&Error::Lookup(ref sym) => write!(f, "symbol lookup error: {}", sym), write!(f, "parse error: {}", desc),
&Error::Lookup(ref sym) =>
write!(f, "symbol lookup error: {}", sym),
} }
} }
} }
@ -101,22 +103,20 @@ impl Library {
let mut index = self.hash_bucket()[hash as usize % self.hash_bucket().len()] as usize; let mut index = self.hash_bucket()[hash as usize % self.hash_bucket().len()] as usize;
loop { loop {
if index == STN_UNDEF { if index == STN_UNDEF { return None }
return None;
}
let sym = &self.symtab()[index]; let sym = &self.symtab()[index];
let sym_name_off = sym.st_name as usize; let sym_name_off = sym.st_name as usize;
match self.strtab().get(sym_name_off..sym_name_off + name.len()) { match self.strtab().get(sym_name_off..sym_name_off + name.len()) {
Some(sym_name) if sym_name == name => { Some(sym_name) if sym_name == name => {
if ELF32_ST_BIND(sym.st_info) & STB_GLOBAL == 0 { if ELF32_ST_BIND(sym.st_info) & STB_GLOBAL == 0 {
return None; return None
} }
match sym.st_shndx { match sym.st_shndx {
SHN_UNDEF => return None, SHN_UNDEF => return None,
SHN_ABS => return Some(self.image.ptr() as u32 + sym.st_value), SHN_ABS => return Some(self.image.ptr() as u32 + sym.st_value),
_ => return Some(self.image.ptr() as u32 + sym.st_value), _ => return Some(self.image.ptr() as u32 + sym.st_value)
} }
} }
_ => (), _ => (),
@ -127,15 +127,9 @@ impl Library {
} }
pub fn name_starting_at(&self, offset: usize) -> Result<&[u8], Error> { pub fn name_starting_at(&self, offset: usize) -> Result<&[u8], Error> {
let size = self let size = self.strtab().iter().skip(offset).position(|&x| x == 0)
.strtab()
.iter()
.skip(offset)
.position(|&x| x == 0)
.ok_or("symbol in symbol table not null-terminated")?; .ok_or("symbol in symbol table not null-terminated")?;
Ok(self Ok(self.strtab().get(offset..offset + size)
.strtab()
.get(offset..offset + size)
.ok_or("cannot read symbol name")?) .ok_or("cannot read symbol name")?)
} }
@ -149,59 +143,53 @@ impl Library {
} }
} }
pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Result<Library, Error> { pub fn load(
data: &[u8],
resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>
) -> Result<Library, Error> {
// validate ELF file // validate ELF file
let file = file::File::new(data).ok_or("cannot read ELF header")?; let file = file::File::new(data)
.ok_or("cannot read ELF header")?;
if file.ehdr.e_type != ET_DYN { if file.ehdr.e_type != ET_DYN {
return Err("not a shared library")?; return Err("not a shared library")?
} }
let arch = file.arch().ok_or("not for a supported architecture")?; let arch = file.arch()
.ok_or("not for a supported architecture")?;
// prepare target memory // prepare target memory
let image_size = file let image_size = file.program_headers()
.program_headers()
.filter_map(|phdr| phdr.map(|phdr| phdr.p_vaddr + phdr.p_memsz)) .filter_map(|phdr| phdr.map(|phdr| phdr.p_vaddr + phdr.p_memsz))
.max() .max()
.unwrap_or(0) as usize; .unwrap_or(0) as usize;
let image_align = file let image_align = file.program_headers()
.program_headers() .filter_map(|phdr| phdr.and_then(|phdr| {
.filter_map(|phdr| {
phdr.and_then(|phdr| {
if phdr.p_type == PT_LOAD { if phdr.p_type == PT_LOAD {
Some(phdr.p_align) Some(phdr.p_align)
} else { } else {
None None
} }
}) }))
})
.max() .max()
.unwrap_or(4) as usize; .unwrap_or(4) as usize;
// 1 image for all segments // 1 image for all segments
let mut image = image::Image::new(image_size, image_align).map_err(|_| "cannot allocate target image")?; let mut image = image::Image::new(image_size, image_align)
debug!( .map_err(|_| "cannot allocate target image")?;
"ELF target: {} bytes, align to {:X}, allocated at {:08X}", debug!("ELF target: {} bytes, align to {:X}, allocated at {:08X}", image_size, image_align, image.ptr() as usize);
image_size,
image_align,
image.ptr() as usize
);
// LOAD // LOAD
for phdr in file.program_headers() { for phdr in file.program_headers() {
let phdr = phdr.ok_or("cannot read program header")?; let phdr = phdr.ok_or("cannot read program header")?;
trace!( trace!("Program header: {:08X}+{:08X} to {:08X}",
"Program header: {:08X}+{:08X} to {:08X}", phdr.p_offset, phdr.p_filesz,
phdr.p_offset,
phdr.p_filesz,
image.ptr() as u32 image.ptr() as u32
); );
let file_range = phdr.p_offset as usize..(phdr.p_offset + phdr.p_filesz) as usize; let file_range = phdr.p_offset as usize..(phdr.p_offset + phdr.p_filesz) as usize;
match phdr.p_type { match phdr.p_type {
PT_LOAD => { PT_LOAD => {
let src = file let src = file.get(file_range)
.get(file_range)
.ok_or("program header requests an out of bounds load (in file)")?; .ok_or("program header requests an out of bounds load (in file)")?;
let dst = image let dst = image.get_mut(phdr.p_vaddr as usize..
.get_mut(phdr.p_vaddr as usize..(phdr.p_vaddr + phdr.p_filesz) as usize) (phdr.p_vaddr + phdr.p_filesz) as usize)
.ok_or("program header requests an out of bounds load (in target)")?; .ok_or("program header requests an out of bounds load (in target)")?;
dst.copy_from_slice(src); dst.copy_from_slice(src);
} }
@ -215,9 +203,9 @@ pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Resul
let shdr = shdr.ok_or("cannot read section header")?; let shdr = shdr.ok_or("cannot read section header")?;
match shdr.sh_type as usize { match shdr.sh_type as usize {
SHT_ARM_EXIDX => { SHT_ARM_EXIDX => {
let range = shdr.sh_addr as usize..(shdr.sh_addr + shdr.sh_size) as usize; let range = shdr.sh_addr as usize..
let _ = image (shdr.sh_addr + shdr.sh_size) as usize;
.get(range.clone()) let _ = image.get(range.clone())
.ok_or("section header specifies EXIDX outside of image (in target)")?; .ok_or("section header specifies EXIDX outside of image (in target)")?;
exidx = Some(range); exidx = Some(range);
} }
@ -226,14 +214,11 @@ pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Resul
} }
// relocate DYNAMIC // relocate DYNAMIC
let dyn_range = file.dyn_header_vaddr().ok_or("cannot find a dynamic header")?; let dyn_range = file.dyn_header_vaddr()
.ok_or("cannot find a dynamic header")?;
let dyn_section = image.dyn_section(dyn_range.clone())?; let dyn_section = image.dyn_section(dyn_range.clone())?;
debug!( debug!("Relocating {} rela, {} rel, {} pltrel",
"Relocating {} rela, {} rel, {} pltrel", dyn_section.rela.len(), dyn_section.rel.len(), dyn_section.pltrel.len());
dyn_section.rela.len(),
dyn_section.rel.len(),
dyn_section.pltrel.len()
);
let lib = Library { let lib = Library {
arch, arch,
image, image,

View File

@ -1,10 +1,16 @@
use alloc::string::String; use alloc::string::String;
use libcortex_a9::{asm::{dsb, isb},
cache::{bpiall, dcci_slice, iciallu}};
use log::trace; use log::trace;
use super::{
use super::{elf::*, image::Image, Arch, Error, Library}; Arch,
elf::*,
Error,
image::Image,
Library,
};
use libcortex_a9::{
cache::{dcci_slice, iciallu, bpiall},
asm::{dsb, isb},
};
pub trait Relocatable { pub trait Relocatable {
fn offset(&self) -> usize; fn offset(&self) -> usize;
@ -60,18 +66,25 @@ enum RelType {
impl RelType { impl RelType {
pub fn new(arch: Arch, type_info: u8) -> Option<Self> { pub fn new(arch: Arch, type_info: u8) -> Option<Self> {
match type_info { match type_info {
R_OR1K_NONE if arch == Arch::OpenRisc => Some(RelType::None), R_OR1K_NONE if arch == Arch::OpenRisc =>
R_ARM_NONE if arch == Arch::Arm => Some(RelType::None), Some(RelType::None),
R_ARM_NONE if arch == Arch::Arm =>
Some(RelType::None),
R_OR1K_RELATIVE if arch == Arch::OpenRisc => Some(RelType::Relative), R_OR1K_RELATIVE if arch == Arch::OpenRisc =>
R_ARM_RELATIVE if arch == Arch::Arm => Some(RelType::Relative), Some(RelType::Relative),
R_ARM_RELATIVE if arch == Arch::Arm =>
Some(RelType::Relative),
R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT if arch == Arch::OpenRisc => Some(RelType::LookupAbs), R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT
R_ARM_GLOB_DAT | R_ARM_JUMP_SLOT | R_ARM_ABS32 if arch == Arch::Arm => Some(RelType::LookupAbs), if arch == Arch::OpenRisc => Some(RelType::LookupAbs),
R_ARM_GLOB_DAT | R_ARM_JUMP_SLOT | R_ARM_ABS32
if arch == Arch::Arm => Some(RelType::LookupAbs),
R_ARM_PREL31 if arch == Arch::Arm => Some(RelType::LookupRel), R_ARM_PREL31 if arch == Arch::Arm => Some(RelType::LookupRel),
_ => None, _ =>
None
} }
} }
} }
@ -83,25 +96,22 @@ fn format_sym_name(sym_name: &[u8]) -> String {
} }
pub fn relocate<R: Relocatable>( pub fn relocate<R: Relocatable>(
arch: Arch, arch: Arch, lib: &Library,
lib: &Library, rel: &R, resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>
rel: &R,
resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>,
) -> Result<(), Error> { ) -> Result<(), Error> {
let sym; let sym;
if rel.sym_info() == 0 { if rel.sym_info() == 0 {
sym = None; sym = None;
} else { } else {
sym = Some( sym = Some(lib.symtab().get(rel.sym_info() as usize)
lib.symtab() .ok_or("symbol out of bounds of symbol table")?)
.get(rel.sym_info() as usize)
.ok_or("symbol out of bounds of symbol table")?,
)
} }
let rel_type = RelType::new(arch, rel.type_info()).ok_or("unsupported relocation type")?; let rel_type = RelType::new(arch, rel.type_info())
.ok_or("unsupported relocation type")?;
let value = match rel_type { let value = match rel_type {
RelType::None => return Ok(()), RelType::None =>
return Ok(()),
RelType::Relative => { RelType::Relative => {
let addend = rel.addend(&lib.image); let addend = rel.addend(&lib.image);
@ -122,48 +132,42 @@ pub fn relocate<R: Relocatable>(
addr addr
} else { } else {
// We couldn't find it anywhere. // We couldn't find it anywhere.
return Err(Error::Lookup(format_sym_name(sym_name))); return Err(Error::Lookup(format_sym_name(sym_name)))
}; };
match rel_type { match rel_type {
RelType::LookupAbs => sym_addr, RelType::LookupAbs => sym_addr,
RelType::LookupRel => { RelType::LookupRel =>
sym_addr.wrapping_sub(lib.image.ptr().wrapping_offset(rel.offset() as isize) as Elf32_Addr) sym_addr.wrapping_sub(
} lib.image.ptr().wrapping_offset(rel.offset() as isize) as Elf32_Addr),
_ => unreachable!(), _ => unreachable!()
} }
} }
}; };
match rel.type_info() { match rel.type_info() {
R_ARM_PREL31 => { R_ARM_PREL31 => {
let reloc_word = lib let reloc_word = lib.image.get_ref::<Elf32_Word>(rel.offset())
.image
.get_ref::<Elf32_Word>(rel.offset())
.ok_or("relocation offset cannot be read")?; .ok_or("relocation offset cannot be read")?;
lib.image lib.image.write(rel.offset(), (reloc_word & 0x80000000) | (value & 0x7FFFFFFF))
.write(rel.offset(), (reloc_word & 0x80000000) | (value & 0x7FFFFFFF)) },
}
_ => lib.image.write(rel.offset(), value), _ => lib.image.write(rel.offset(), value),
} }
} }
pub fn rebind(arch: Arch, lib: &Library, name: &[u8], value: Elf32_Word) -> Result<(), Error> { pub fn rebind(
arch: Arch, lib: &Library, name: &[u8], value: Elf32_Word
) -> Result<(), Error> {
fn rebind_symbol_to_value<R: Relocatable>( fn rebind_symbol_to_value<R: Relocatable>(
arch: Arch, arch: Arch, lib: &Library,name: &[u8], value: Elf32_Word, relocs: &[R]
lib: &Library,
name: &[u8],
value: Elf32_Word,
relocs: &[R],
) -> Result<(), Error> { ) -> Result<(), Error> {
for reloc in relocs { for reloc in relocs {
let rel_type = RelType::new(arch, reloc.type_info()).ok_or("unsupported relocation type")?; let rel_type = RelType::new(arch, reloc.type_info())
.ok_or("unsupported relocation type")?;
match rel_type { match rel_type {
RelType::LookupAbs => { RelType::LookupAbs => {
let sym = lib let sym = lib.symtab().get(reloc.sym_info() as usize)
.symtab()
.get(reloc.sym_info() as usize)
.ok_or("symbol out of bounds of symbol table")?; .ok_or("symbol out of bounds of symbol table")?;
let sym_name = lib.name_starting_at(sym.st_name as usize)?; let sym_name = lib.name_starting_at(sym.st_name as usize)?;

View File

@ -11,7 +11,7 @@ path = "lib.rs"
core_io = { version = "0.1", features = ["collections"] } core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.0", default-features = false, optional = true } byteorder = { version = "1.0", default-features = false, optional = true }
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] } libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
[features] [features]
alloc = [] alloc = []

View File

@ -1,12 +1,9 @@
#[cfg(feature = "alloc")] use core_io::{Read, Write, Error as IoError};
use alloc::vec::Vec;
use core_io::{Error as IoError, Read, Write};
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct Cursor<T> { pub struct Cursor<T> {
inner: T, inner: T,
pos: usize, pos: usize
} }
impl<T> Cursor<T> { impl<T> Cursor<T> {
@ -42,25 +39,22 @@ impl<T> Cursor<T> {
} }
impl<T: AsRef<[u8]>> Read for Cursor<T> { impl<T: AsRef<[u8]>> Read for Cursor<T> {
fn read(&mut self, buf: &mut [u8]) -> Result<usize, IoError> { fn read(&mut self, buf: &mut [u8]) -> Result<usize, IoError> {
let data = &self.inner.as_ref()[self.pos..]; let data = &self.inner.as_ref()[self.pos..];
let len = buf.len().min(data.len()); let len = buf.len().min(data.len());
// ``copy_from_slice`` generates AXI bursts, use a regular loop instead buf[..len].copy_from_slice(&data[..len]);
for i in 0..len {
buf[i] = data[i];
}
self.pos += len; self.pos += len;
Ok(len) Ok(len)
} }
} }
impl Write for Cursor<&mut [u8]> { impl Write for Cursor<&mut [u8]> {
fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> { fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> {
let data = &mut self.inner[self.pos..]; let data = &mut self.inner[self.pos..];
let len = buf.len().min(data.len()); let len = buf.len().min(data.len());
for i in 0..len { data[..len].copy_from_slice(&buf[..len]);
data[i] = buf[i];
}
self.pos += len; self.pos += len;
Ok(len) Ok(len)
} }
@ -72,7 +66,9 @@ impl Write for Cursor<&mut [u8]> {
} }
#[cfg(feature = "alloc")] #[cfg(feature = "alloc")]
impl Write for Cursor<Vec<u8>> { impl Write for Cursor<::alloc::Vec<u8>> {
#[inline]
fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> { fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> {
self.inner.extend_from_slice(buf); self.inner.extend_from_slice(buf);
Ok(buf.len()) Ok(buf.len())

View File

@ -1,10 +1,13 @@
#![no_std] #![no_std]
#![feature(never_type)] #![feature(never_type)]
#![cfg_attr(feature = "alloc", feature(alloc))]
#[cfg(feature = "alloc")]
extern crate alloc; extern crate alloc;
extern crate core_io; extern crate core_io;
#[cfg(feature = "alloc")]
#[macro_use]
use alloc;
#[cfg(feature = "byteorder")] #[cfg(feature = "byteorder")]
extern crate byteorder; extern crate byteorder;
@ -13,7 +16,7 @@ pub mod cursor;
pub mod proto; pub mod proto;
pub use cursor::Cursor; pub use cursor::Cursor;
#[cfg(all(feature = "byteorder", feature = "alloc"))]
pub use proto::ReadStringError;
#[cfg(feature = "byteorder")] #[cfg(feature = "byteorder")]
pub use proto::{ProtoRead, ProtoWrite}; pub use proto::{ProtoRead, ProtoWrite};
#[cfg(all(feature = "byteorder", feature = "alloc"))]
pub use proto::ReadStringError;

View File

@ -1,15 +1,15 @@
#[cfg(feature = "alloc")]
use alloc::{string::String, vec};
use core::str::Utf8Error; use core::str::Utf8Error;
use byteorder::{ByteOrder, NativeEndian}; use byteorder::{ByteOrder, NativeEndian};
use core_io::{Error as IoError, Read, Write}; use alloc::vec;
use alloc::string::String;
use core_io::{Read, Write, Error as IoError};
#[allow(dead_code)] #[allow(dead_code)]
#[derive(Debug, Clone, PartialEq)] #[derive(Debug, Clone, PartialEq)]
pub enum ReadStringError<T> { pub enum ReadStringError<T> {
Utf8(Utf8Error), Utf8(Utf8Error),
Other(T), Other(T)
} }
pub trait ProtoRead { pub trait ProtoRead {
@ -51,8 +51,7 @@ pub trait ProtoRead {
} }
#[inline] #[inline]
#[cfg(feature = "alloc")] fn read_bytes(&mut self) -> Result<::alloc::vec::Vec<u8>, Self::ReadError> {
fn read_bytes(&mut self) -> Result<vec::Vec<u8>, Self::ReadError> {
let length = self.read_u32()?; let length = self.read_u32()?;
let mut value = vec![0; length as usize]; let mut value = vec![0; length as usize];
self.read_exact(&mut value)?; self.read_exact(&mut value)?;
@ -60,8 +59,7 @@ pub trait ProtoRead {
} }
#[inline] #[inline]
#[cfg(feature = "alloc")] fn read_string(&mut self) -> Result<::alloc::string::String, ReadStringError<Self::ReadError>> {
fn read_string(&mut self) -> Result<String, ReadStringError<Self::ReadError>> {
let bytes = self.read_bytes().map_err(ReadStringError::Other)?; let bytes = self.read_bytes().map_err(ReadStringError::Other)?;
String::from_utf8(bytes).map_err(|err| ReadStringError::Utf8(err.utf8_error())) String::from_utf8(bytes).map_err(|err| ReadStringError::Utf8(err.utf8_error()))
} }
@ -138,15 +136,12 @@ pub trait ProtoWrite {
} }
#[inline] #[inline]
#[cfg(feature = "alloc")]
fn write_string(&mut self, value: &str) -> Result<(), Self::WriteError> { fn write_string(&mut self, value: &str) -> Result<(), Self::WriteError> {
self.write_bytes(value.as_bytes()) self.write_bytes(value.as_bytes())
} }
} }
impl<T> ProtoRead for T impl<T> ProtoRead for T where T: Read + ?Sized {
where T: Read + ?Sized
{
type ReadError = IoError; type ReadError = IoError;
fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Self::ReadError> { fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Self::ReadError> {
@ -154,9 +149,7 @@ where T: Read + ?Sized
} }
} }
impl<T> ProtoWrite for T impl<T> ProtoWrite for T where T: Write + ?Sized {
where T: Write + ?Sized
{
type WriteError = IoError; type WriteError = IoError;
fn write_all(&mut self, buf: &[u8]) -> Result<(), Self::WriteError> { fn write_all(&mut self, buf: &[u8]) -> Result<(), Self::WriteError> {

View File

@ -1,40 +0,0 @@
[package]
name = "ksupport"
description = "Kernel support for Zynq-based platforms"
version = "0.1.0"
authors = ["M-Labs"]
edition = "2018"
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
cslice = "0.3"
log = "0.4"
nb = "0.1"
core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.3", default-features = false }
void = { version = "1", default-features = false }
log_buffer = { version = "1.2" }
libm = { version = "0.2", features = ["unstable"] }
vcell = "0.1"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] }
dyld = { path = "../libdyld" }
dwarf = { path = "../libdwarf" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }
io = { path = "../libio" }
libboard_artiq = { path = "../libboard_artiq" }
[dependencies.nalgebra]
git = "https://git.m-labs.hk/M-Labs/nalgebra.git"
rev = "dd00f9b"
default-features = false
features = ["libm", "alloc"]

View File

@ -1,5 +0,0 @@
extern crate build_zynq;
fn main() {
build_zynq::cfg();
}

View File

@ -1,255 +0,0 @@
use alloc::{string::String, vec::Vec};
use core::mem;
use cslice::CSlice;
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, KERNEL_IMAGE};
use crate::{artiq_raise, pl::csr, rtio};
#[repr(C)]
pub struct DmaTrace {
duration: i64,
address: i32,
uses_ddma: bool,
}
#[derive(Clone, Debug)]
pub struct DmaRecorder {
pub name: String,
pub buffer: Vec<u8>,
pub duration: i64,
pub enable_ddma: bool,
}
static mut RECORDER: Option<DmaRecorder> = None;
pub unsafe fn init_dma_recorder() {
// as static would remain after restart, we have to reset it,
// without running its destructor.
mem::forget(mem::replace(&mut RECORDER, None));
}
pub extern "C" fn dma_record_start(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaEraseRequest(name.clone()));
}
unsafe {
if RECORDER.is_some() {
artiq_raise!("DMAError", "DMA is already recording")
}
let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output", dma_record_output as *const ()).unwrap();
library
.rebind(b"rtio_output_wide", dma_record_output_wide as *const ())
.unwrap();
RECORDER = Some(DmaRecorder {
name,
buffer: Vec::new(),
duration: 0,
enable_ddma: false,
});
}
}
pub extern "C" fn dma_record_stop(duration: i64, enable_ddma: bool) {
unsafe {
if RECORDER.is_none() {
artiq_raise!("DMAError", "DMA is not recording")
}
let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output", rtio::output as *const ()).unwrap();
library
.rebind(b"rtio_output_wide", rtio::output_wide as *const ())
.unwrap();
let mut recorder = RECORDER.take().unwrap();
recorder.duration = duration;
recorder.enable_ddma = enable_ddma;
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaPutRequest(recorder));
}
}
#[inline(always)]
unsafe fn dma_record_output_prepare(timestamp: i64, target: i32, words: usize) {
// See gateware/rtio/dma.py.
const HEADER_LENGTH: usize = /*length*/1 + /*channel*/3 + /*timestamp*/8 + /*address*/1;
let length = HEADER_LENGTH + /*data*/words * 4;
let buffer = &mut RECORDER.as_mut().unwrap().buffer;
buffer.reserve(length);
buffer.extend_from_slice(&[
(length >> 0) as u8,
(target >> 8) as u8,
(target >> 16) as u8,
(target >> 24) as u8,
(timestamp >> 0) as u8,
(timestamp >> 8) as u8,
(timestamp >> 16) as u8,
(timestamp >> 24) as u8,
(timestamp >> 32) as u8,
(timestamp >> 40) as u8,
(timestamp >> 48) as u8,
(timestamp >> 56) as u8,
(target >> 0) as u8,
]);
}
pub extern "C" fn dma_record_output(target: i32, word: i32) {
unsafe {
let timestamp = rtio::now_mu();
dma_record_output_prepare(timestamp, target, 1);
RECORDER.as_mut().unwrap().buffer.extend_from_slice(&[
(word >> 0) as u8,
(word >> 8) as u8,
(word >> 16) as u8,
(word >> 24) as u8,
]);
}
}
pub extern "C" fn dma_record_output_wide(target: i32, words: &CSlice<i32>) {
assert!(words.len() <= 16); // enforce the hardware limit
unsafe {
let timestamp = rtio::now_mu();
dma_record_output_prepare(timestamp, target, words.len());
let buffer = &mut RECORDER.as_mut().unwrap().buffer;
for word in words.as_ref().iter() {
buffer.extend_from_slice(&[
(word >> 0) as u8,
(word >> 8) as u8,
(word >> 16) as u8,
(word >> 24) as u8,
]);
}
}
}
pub extern "C" fn dma_erase(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaEraseRequest(name));
}
}
pub extern "C" fn dma_retrieve(name: CSlice<u8>) -> DmaTrace {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaGetRequest(name));
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::DmaGetReply(None) => (),
Message::DmaGetReply(Some((address, duration, uses_ddma))) => {
return DmaTrace {
address,
duration,
uses_ddma,
};
}
_ => panic!("Expected DmaGetReply after DmaGetRequest!"),
}
// we have to defer raising error as we have to drop the message first...
artiq_raise!("DMAError", "DMA trace not found");
}
pub extern "C" fn dma_playback(timestamp: i64, ptr: i32, _uses_ddma: bool) {
unsafe {
csr::rtio_dma::base_address_write(ptr as u32);
csr::rtio_dma::time_offset_write(timestamp as u64);
let old_cri_master = csr::cri_con::selected_read();
csr::cri_con::selected_write(1);
csr::rtio_dma::enable_write(1);
#[cfg(has_drtio)]
if _uses_ddma {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaStartRemoteRequest {
id: ptr,
timestamp: timestamp,
});
}
while csr::rtio_dma::enable_read() != 0 {}
csr::cri_con::selected_write(old_cri_master);
let error = csr::rtio_dma::error_read();
if error != 0 {
let timestamp = csr::rtio_dma::error_timestamp_read();
let channel = csr::rtio_dma::error_channel_read();
csr::rtio_dma::error_write(1);
if error & 1 != 0 {
artiq_raise!(
"RTIOUnderflow",
"RTIO underflow at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
if error & 2 != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
"RTIO destination unreachable, output, at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
}
#[cfg(has_drtio)]
if _uses_ddma {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaAwaitRemoteRequest(ptr));
match KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv() {
Message::DmaAwaitRemoteReply {
timeout,
error,
channel,
timestamp,
} => {
if timeout {
artiq_raise!(
"DMAError",
"Error running DMA on satellite device, timed out waiting for results"
);
}
if error & 1 != 0 {
artiq_raise!(
"RTIOUnderflow",
"RTIO underflow at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
if error & 2 != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
"RTIO destination unreachable, output, at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
}
_ => panic!("Expected DmaAwaitRemoteReply after DmaAwaitRemoteRequest!"),
}
}
}
}

View File

@ -1,440 +0,0 @@
// Uses `nalgebra` crate to invoke `np_linalg` and `sp_linalg` functions
// When converting between `nalgebra::Matrix` and `NDArray` following considerations are necessary
//
// * Both `nalgebra::Matrix` and `NDArray` require their content to be stored in row-major order
// * `NDArray` data pointer can be directly read and converted to `nalgebra::Matrix` (row and column number must be known)
// * `nalgebra::Matrix::as_slice` returns the content of matrix in column-major order and initial data needs to be transposed before storing it in `NDArray` data pointer
use alloc::vec::Vec;
use core::slice;
use nalgebra::DMatrix;
use crate::artiq_raise;
pub struct InputMatrix {
pub ndims: usize,
pub dims: *const usize,
pub data: *mut f64,
}
impl InputMatrix {
fn get_dims(&mut self) -> Vec<usize> {
let dims = unsafe { slice::from_raw_parts(self.dims, self.ndims) };
dims.to_vec()
}
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_cholesky(mat1: *mut InputMatrix, out: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out = out.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
if dim1[0] != dim1[1] {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
let outdim = out.get_dims();
let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let result = matrix1.cholesky();
match result {
Some(res) => {
out_slice.copy_from_slice(res.unpack().transpose().as_slice());
}
None => {
artiq_raise!("LinAlgError", "Matrix is not positive definite");
}
};
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_qr(mat1: *mut InputMatrix, out_q: *mut InputMatrix, out_r: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out_q = out_q.as_mut().unwrap();
let out_r = out_r.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let outq_dim = (*out_q).get_dims();
let outr_dim = (*out_r).get_dims();
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let out_q_slice = slice::from_raw_parts_mut(out_q.data, outq_dim[0] * outq_dim[1]);
let out_r_slice = slice::from_raw_parts_mut(out_r.data, outr_dim[0] * outr_dim[1]);
// Refer to https://github.com/dimforge/nalgebra/issues/735
let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let res = matrix1.qr();
let (q, r) = res.unpack();
// Uses different algo need to match numpy
out_q_slice.copy_from_slice(q.transpose().as_slice());
out_r_slice.copy_from_slice(r.transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_svd(
mat1: *mut InputMatrix,
outu: *mut InputMatrix,
outs: *mut InputMatrix,
outvh: *mut InputMatrix,
) {
let mat1 = mat1.as_mut().unwrap();
let outu = outu.as_mut().unwrap();
let outs = outs.as_mut().unwrap();
let outvh = outvh.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let outu_dim = (*outu).get_dims();
let outs_dim = (*outs).get_dims();
let outvh_dim = (*outvh).get_dims();
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let out_u_slice = slice::from_raw_parts_mut(outu.data, outu_dim[0] * outu_dim[1]);
let out_s_slice = slice::from_raw_parts_mut(outs.data, outs_dim[0]);
let out_vh_slice = slice::from_raw_parts_mut(outvh.data, outvh_dim[0] * outvh_dim[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let result = matrix.svd(true, true);
out_u_slice.copy_from_slice(result.u.unwrap().transpose().as_slice());
out_s_slice.copy_from_slice(result.singular_values.as_slice());
out_vh_slice.copy_from_slice(result.v_t.unwrap().transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_inv(mat1: *mut InputMatrix, out: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out = out.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
if dim1[0] != dim1[1] {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
let outdim = out.get_dims();
let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
if !matrix.is_invertible() {
artiq_raise!("LinAlgError", "no inverse for Singular Matrix");
}
let inv = matrix.try_inverse().unwrap();
out_slice.copy_from_slice(inv.transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_pinv(mat1: *mut InputMatrix, out: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out = out.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let outdim = out.get_dims();
let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let svd = matrix.svd(true, true);
let inv = svd.pseudo_inverse(1e-15);
match inv {
Ok(m) => {
out_slice.copy_from_slice(m.transpose().as_slice());
}
Err(_) => {
artiq_raise!("LinAlgError", "SVD computation does not converge");
}
}
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_matrix_power(mat1: *mut InputMatrix, mat2: *mut InputMatrix, out: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let mat2 = mat2.as_mut().unwrap();
let out = out.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let power = slice::from_raw_parts_mut(mat2.data, 1);
let power = power[0];
let outdim = out.get_dims();
let out_slice = slice::from_raw_parts_mut(out.data, outdim[0] * outdim[1]);
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let mut abs_power = power;
if abs_power < 0.0 {
abs_power = abs_power * -1.0;
}
let matrix1 = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
if !matrix1.is_square() {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
let mut result = matrix1.pow(abs_power as u32);
if power < 0.0 {
if !matrix1.is_invertible() {
artiq_raise!("LinAlgError", "no inverse for Singular Matrix");
}
result = result.try_inverse().unwrap();
}
out_slice.copy_from_slice(result.transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn np_linalg_det(mat1: *mut InputMatrix, out: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out = out.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let out_slice = slice::from_raw_parts_mut(out.data, 1);
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
if !matrix.is_square() {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
out_slice[0] = matrix.determinant();
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn sp_linalg_lu(mat1: *mut InputMatrix, out_l: *mut InputMatrix, out_u: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out_l = out_l.as_mut().unwrap();
let out_u = out_u.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
let outl_dim = (*out_l).get_dims();
let outu_dim = (*out_u).get_dims();
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let out_l_slice = slice::from_raw_parts_mut(out_l.data, outl_dim[0] * outl_dim[1]);
let out_u_slice = slice::from_raw_parts_mut(out_u.data, outu_dim[0] * outu_dim[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let (_, l, u) = matrix.lu().unpack();
out_l_slice.copy_from_slice(l.transpose().as_slice());
out_u_slice.copy_from_slice(u.transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn sp_linalg_schur(mat1: *mut InputMatrix, out_t: *mut InputMatrix, out_z: *mut InputMatrix) {
let mat1 = mat1.as_mut().unwrap();
let out_t = out_t.as_mut().unwrap();
let out_z = out_z.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
if dim1[0] != dim1[1] {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
let out_t_dim = (*out_t).get_dims();
let out_z_dim = (*out_z).get_dims();
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let out_t_slice = slice::from_raw_parts_mut(out_t.data, out_t_dim[0] * out_t_dim[1]);
let out_z_slice = slice::from_raw_parts_mut(out_z.data, out_z_dim[0] * out_z_dim[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let (z, t) = matrix.schur().unpack();
out_t_slice.copy_from_slice(t.transpose().as_slice());
out_z_slice.copy_from_slice(z.transpose().as_slice());
}
/// # Safety
///
/// `mat1` should point to a valid 2DArray of `f64` floats in row-major order
#[no_mangle]
pub unsafe extern "C" fn sp_linalg_hessenberg(
mat1: *mut InputMatrix,
out_h: *mut InputMatrix,
out_q: *mut InputMatrix,
) {
let mat1 = mat1.as_mut().unwrap();
let out_h = out_h.as_mut().unwrap();
let out_q = out_q.as_mut().unwrap();
if mat1.ndims != 2 {
artiq_raise!(
"ValueError",
"expected 2D Vector Input, but received {1}D input)",
0,
mat1.ndims as i64,
0
);
}
let dim1 = (*mat1).get_dims();
if dim1[0] != dim1[1] {
artiq_raise!(
"ValueError",
"last 2 dimensions of the array must be square: {1} != {2}",
0,
dim1[0] as i64,
dim1[1] as i64
);
}
let out_h_dim = (*out_h).get_dims();
let out_q_dim = (*out_q).get_dims();
let data_slice1 = slice::from_raw_parts_mut(mat1.data, dim1[0] * dim1[1]);
let out_h_slice = slice::from_raw_parts_mut(out_h.data, out_h_dim[0] * out_h_dim[1]);
let out_q_slice = slice::from_raw_parts_mut(out_q.data, out_q_dim[0] * out_q_dim[1]);
let matrix = DMatrix::from_row_slice(dim1[0], dim1[1], data_slice1);
let (q, h) = matrix.hessenberg().unpack();
out_h_slice.copy_from_slice(h.transpose().as_slice());
out_q_slice.copy_from_slice(q.transpose().as_slice());
}

View File

@ -1,128 +0,0 @@
use alloc::{string::String, vec::Vec};
use core::ptr;
use libcortex_a9::{mutex::Mutex, semaphore::Semaphore, sync_channel};
use crate::{eh_artiq, RPCException};
mod control;
pub use control::Control;
mod api;
pub mod core1;
mod dma;
mod rpc;
pub use dma::DmaRecorder;
mod cache;
mod linalg;
#[cfg(has_drtio)]
mod subkernel;
#[cfg(has_drtio)]
#[derive(Debug, Clone)]
pub enum SubkernelStatus {
NoError,
Timeout,
IncorrectState,
CommLost,
Exception(Vec<u8>),
OtherError,
}
#[derive(Debug, Clone)]
pub enum Message {
LoadRequest(Vec<u8>),
LoadCompleted,
LoadFailed,
StartRequest,
KernelFinished(u8),
KernelException(
&'static [Option<eh_artiq::Exception<'static>>],
&'static [eh_artiq::StackPointerBacktrace],
&'static [(usize, usize)],
u8,
),
RpcSend {
is_async: bool,
data: Vec<u8>,
},
RpcRecvRequest(*mut ()),
RpcRecvReply(Result<usize, RPCException>),
CacheGetRequest(String),
CacheGetReply(Vec<i32>),
CachePutRequest(String, Vec<i32>),
DmaPutRequest(DmaRecorder),
DmaEraseRequest(String),
DmaGetRequest(String),
DmaGetReply(Option<(i32, i64, bool)>),
#[cfg(has_drtio)]
DmaStartRemoteRequest {
id: i32,
timestamp: i64,
},
#[cfg(has_drtio)]
DmaAwaitRemoteRequest(i32),
#[cfg(has_drtio)]
DmaAwaitRemoteReply {
timeout: bool,
error: u8,
channel: u32,
timestamp: u64,
},
#[cfg(has_drtio)]
UpDestinationsRequest(i32),
#[cfg(has_drtio)]
UpDestinationsReply(bool),
#[cfg(has_drtio)]
SubkernelLoadRunRequest {
id: u32,
destination: u8,
run: bool,
timestamp: u64,
},
#[cfg(has_drtio)]
SubkernelLoadRunReply {
succeeded: bool,
},
#[cfg(has_drtio)]
SubkernelAwaitFinishRequest {
id: u32,
timeout: i64,
},
#[cfg(has_drtio)]
SubkernelAwaitFinishReply,
#[cfg(has_drtio)]
SubkernelMsgSend {
id: u32,
destination: Option<u8>,
data: Vec<u8>,
},
#[cfg(has_drtio)]
SubkernelMsgSent,
#[cfg(has_drtio)]
SubkernelMsgRecvRequest {
id: i32,
timeout: i64,
tags: Vec<u8>,
},
#[cfg(has_drtio)]
SubkernelMsgRecvReply {
count: u8,
},
#[cfg(has_drtio)]
SubkernelError(SubkernelStatus),
}
static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None);
static CHANNEL_1TO0: Mutex<Option<sync_channel::Receiver<'static, Message>>> = Mutex::new(None);
static CHANNEL_SEM: Semaphore = Semaphore::new(0, 1);
static mut KERNEL_CHANNEL_0TO1: Option<sync_channel::Receiver<'static, Message>> = None;
static mut KERNEL_CHANNEL_1TO0: Option<sync_channel::Sender<'static, Message>> = None;
pub static mut KERNEL_IMAGE: *const core1::KernelImage = ptr::null();
static INIT_LOCK: Mutex<()> = Mutex::new(());

View File

@ -1,112 +0,0 @@
use alloc::vec::Vec;
use cslice::CSlice;
use super::{Message, SubkernelStatus, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0};
use crate::{artiq_raise, eh_artiq, rpc::send_args, rtio::now_mu};
pub extern "C" fn load_run(id: u32, destination: u8, run: bool) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelLoadRunRequest {
id: id,
destination: destination,
run: run,
timestamp: now_mu() as u64,
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelLoadRunReply { succeeded: true } => (),
Message::SubkernelLoadRunReply { succeeded: false } => {
artiq_raise!("SubkernelError", "Error loading or running the subkernel")
}
_ => panic!("Expected SubkernelLoadRunReply after SubkernelLoadRunRequest!"),
}
}
pub extern "C" fn await_finish(id: u32, timeout: i64) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelAwaitFinishRequest {
id: id,
timeout: timeout,
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelAwaitFinishReply => (),
Message::SubkernelError(SubkernelStatus::IncorrectState) => {
artiq_raise!("SubkernelError", "Subkernel not running")
}
Message::SubkernelError(SubkernelStatus::Timeout) => artiq_raise!("SubkernelError", "Subkernel timed out"),
Message::SubkernelError(SubkernelStatus::CommLost) => {
artiq_raise!("SubkernelError", "Lost communication with satellite")
}
Message::SubkernelError(SubkernelStatus::OtherError) => {
artiq_raise!("SubkernelError", "An error occurred during subkernel operation")
}
Message::SubkernelError(SubkernelStatus::Exception(raw_exception)) => eh_artiq::raise_raw(&raw_exception),
_ => panic!("expected SubkernelAwaitFinishReply after SubkernelAwaitFinishRequest"),
}
}
pub extern "C" fn send_message(
id: u32,
is_return: bool,
destination: u8,
count: u8,
tag: &CSlice<u8>,
data: *const *const (),
) {
let mut buffer = Vec::<u8>::new();
send_args(&mut buffer, 0, tag.as_ref(), data, false).expect("RPC encoding failed");
// overwrite service tag, include how many tags are in the message
buffer[3] = count;
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::SubkernelMsgSend {
id: id,
destination: if is_return { None } else { Some(destination) },
data: buffer[3..].to_vec(),
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelMsgSent => (),
_ => panic!("expected SubkernelMsgSent after SubkernelMsgSend"),
}
}
pub extern "C" fn await_message(id: i32, timeout: i64, tags: &CSlice<u8>, min: u8, max: u8) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelMsgRecvRequest {
id: id,
timeout: timeout,
tags: tags.as_ref().to_vec(),
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelMsgRecvReply { count } => {
if min > count || count > max {
artiq_raise!("SubkernelError", "Received more or less arguments than required")
}
}
Message::SubkernelError(SubkernelStatus::IncorrectState) => {
artiq_raise!("SubkernelError", "Subkernel not running")
}
Message::SubkernelError(SubkernelStatus::Timeout) => artiq_raise!("SubkernelError", "Subkernel timed out"),
Message::SubkernelError(SubkernelStatus::CommLost) => {
artiq_raise!("SubkernelError", "Lost communication with satellite")
}
Message::SubkernelError(SubkernelStatus::OtherError) => {
artiq_raise!("SubkernelError", "An error occurred during subkernel operation")
}
Message::SubkernelError(SubkernelStatus::Exception(raw_exception)) => eh_artiq::raise_raw(&raw_exception),
_ => panic!("expected SubkernelMsgRecvReply after SubkernelMsgRecvRequest"),
}
// RpcRecvRequest should be called after this to receive message data
}

View File

@ -1,163 +0,0 @@
#![no_std]
#![feature(c_variadic)]
#![feature(const_btree_new)]
#![feature(const_in_array_repeat_expressions)]
#![feature(naked_functions)]
#![feature(asm)]
#[macro_use]
extern crate alloc;
use alloc::{collections::BTreeMap, string::String};
use io::{Cursor, ProtoRead};
use libasync::block_async;
use libconfig::Config;
use log::{error, warn};
#[cfg(has_drtiosat)]
pub use pl::csr::drtiosat as rtio_core;
#[cfg(has_rtio_core)]
pub use pl::csr::rtio_core;
use void::Void;
pub mod eh_artiq;
pub mod i2c;
pub mod irq;
pub mod kernel;
pub mod rpc;
#[cfg(ki_impl = "csr")]
#[path = "rtio_csr.rs"]
pub mod rtio;
#[cfg(ki_impl = "acp")]
#[path = "rtio_acp.rs"]
pub mod rtio;
#[rustfmt::skip]
#[path = "../../../build/pl.rs"]
pub mod pl;
#[derive(Debug, Clone)]
pub struct RPCException {
pub id: u32,
pub message: u32,
pub param: [i64; 3],
pub file: u32,
pub line: i32,
pub column: i32,
pub function: u32,
}
pub static mut SEEN_ASYNC_ERRORS: u8 = 0;
pub const ASYNC_ERROR_COLLISION: u8 = 1 << 0;
pub const ASYNC_ERROR_BUSY: u8 = 1 << 1;
pub const ASYNC_ERROR_SEQUENCE_ERROR: u8 = 1 << 2;
pub unsafe fn get_async_errors() -> u8 {
let errors = SEEN_ASYNC_ERRORS;
SEEN_ASYNC_ERRORS = 0;
errors
}
fn wait_for_async_rtio_error() -> nb::Result<(), Void> {
unsafe {
#[cfg(has_rtio_core)]
let errors = rtio_core::async_error_read();
#[cfg(has_drtiosat)]
let errors = rtio_core::protocol_error_read();
if errors != 0 {
Ok(())
} else {
Err(nb::Error::WouldBlock)
}
}
}
pub async fn report_async_rtio_errors() {
loop {
let _ = block_async!(wait_for_async_rtio_error()).await;
unsafe {
#[cfg(has_rtio_core)]
let errors = rtio_core::async_error_read();
#[cfg(has_drtiosat)]
let errors = rtio_core::protocol_error_read();
if errors & ASYNC_ERROR_COLLISION != 0 {
let channel = rtio_core::collision_channel_read();
error!(
"RTIO collision involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_BUSY != 0 {
let channel = rtio_core::busy_channel_read();
error!(
"RTIO busy error involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_SEQUENCE_ERROR != 0 {
let channel = rtio_core::sequence_error_channel_read();
error!(
"RTIO sequence error involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
SEEN_ASYNC_ERRORS = errors;
#[cfg(has_rtio_core)]
rtio_core::async_error_write(errors);
#[cfg(has_drtiosat)]
rtio_core::protocol_error_write(errors);
}
}
}
static mut RTIO_DEVICE_MAP: BTreeMap<u32, String> = BTreeMap::new();
fn read_device_map(cfg: &Config) -> BTreeMap<u32, String> {
let mut device_map: BTreeMap<u32, String> = BTreeMap::new();
let _ = cfg
.read("device_map")
.and_then(|raw_bytes| {
let mut bytes_cr = Cursor::new(raw_bytes);
let size = bytes_cr.read_u32().unwrap();
for _ in 0..size {
let channel = bytes_cr.read_u32().unwrap();
let device_name = bytes_cr.read_string().unwrap();
if let Some(old_entry) = device_map.insert(channel, device_name.clone()) {
warn!(
"conflicting device map entries for RTIO channel {}: '{}' and '{}'",
channel, old_entry, device_name
);
}
}
Ok(())
})
.or_else(|err| {
warn!(
"error reading device map ({}), device names will not be available in RTIO error messages",
err
);
Err(err)
});
device_map
}
fn _resolve_channel_name(channel: u32, device_map: &BTreeMap<u32, String>) -> String {
match device_map.get(&channel) {
Some(val) => val.clone(),
None => String::from("unknown"),
}
}
pub fn resolve_channel_name(channel: u32) -> String {
_resolve_channel_name(channel, unsafe { &RTIO_DEVICE_MAP })
}
pub fn setup_device_map(cfg: &Config) {
unsafe {
RTIO_DEVICE_MAP = read_device_map(cfg);
}
}

View File

@ -1,274 +0,0 @@
use core::ptr::{read_volatile, write_volatile};
use cslice::CSlice;
use crate::{artiq_raise, pl::csr, resolve_channel_name, rtio_core};
pub const RTIO_O_STATUS_WAIT: u8 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: u8 = 2;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: u8 = 4;
pub const RTIO_I_STATUS_WAIT_EVENT: u8 = 1;
pub const RTIO_I_STATUS_OVERFLOW: u8 = 2;
pub const RTIO_I_STATUS_WAIT_STATUS: u8 = 4;
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: u8 = 8;
#[repr(C)]
pub struct TimestampedData {
timestamp: i64,
data: i32,
}
pub extern "C" fn init() {
unsafe {
rtio_core::reset_write(1);
}
}
pub extern "C" fn get_counter() -> i64 {
unsafe {
csr::rtio::counter_update_write(1);
csr::rtio::counter_read() as i64
}
}
pub extern "C" fn now_mu() -> i64 {
unsafe { csr::rtio::now_read() as i64 }
}
pub extern "C" fn at_mu(t: i64) {
unsafe {
csr::rtio::now_write(t as u64);
}
}
pub extern "C" fn delay_mu(dt: i64) {
unsafe {
csr::rtio::now_write(csr::rtio::now_read() + dt as u64);
}
}
// writing the LSB of o_data (offset=0) triggers the RTIO write
#[inline(always)]
pub unsafe fn rtio_o_data_write(offset: usize, data: u32) {
write_volatile(
csr::rtio::O_DATA_ADDR.offset((csr::rtio::O_DATA_SIZE - 1 - offset) as isize),
data,
);
}
#[inline(always)]
pub unsafe fn rtio_i_data_read(offset: usize) -> u32 {
read_volatile(csr::rtio::I_DATA_ADDR.offset((csr::rtio::I_DATA_SIZE - 1 - offset) as isize))
}
#[inline(never)]
unsafe fn process_exceptional_status(channel: i32, status: u8) {
let timestamp = csr::rtio::now_read() as i64;
if status & RTIO_O_STATUS_WAIT != 0 {
while csr::rtio::o_status_read() & RTIO_O_STATUS_WAIT != 0 {}
}
if status & RTIO_O_STATUS_UNDERFLOW != 0 {
artiq_raise!(
"RTIOUnderflow",
format!(
"RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
timestamp,
timestamp - get_counter()
);
}
if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
format!(
"RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
timestamp,
channel as i64,
0
);
}
}
pub extern "C" fn output(target: i32, data: i32) {
unsafe {
csr::rtio::target_write(target as u32);
// writing target clears o_data
rtio_o_data_write(0, data as _);
let status = csr::rtio::o_status_read();
if status != 0 {
process_exceptional_status(target >> 8, status);
}
}
}
pub extern "C" fn output_wide(target: i32, data: &CSlice<i32>) {
unsafe {
csr::rtio::target_write(target as u32);
// writing target clears o_data
for i in (0..data.len()).rev() {
rtio_o_data_write(i, data[i] as _)
}
let status = csr::rtio::o_status_read();
if status != 0 {
process_exceptional_status(target >> 8, status);
}
}
}
pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!(
"RTIOOverflow",
format!(
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return -1;
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
format!(
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
csr::rtio::i_timestamp_read() as i64
}
}
pub extern "C" fn input_data(channel: i32) -> i32 {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(0xffffffff_ffffffff);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!(
"RTIOOverflow",
format!(
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
format!(
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
rtio_i_data_read(0) as i32
}
}
pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!(
"RTIOOverflow",
format!(
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return TimestampedData { timestamp: -1, data: 0 };
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
format!(
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
}
TimestampedData {
timestamp: csr::rtio::i_timestamp_read() as i64,
data: rtio_i_data_read(0) as i32,
}
}
}
pub fn write_log(data: &[i8]) {
unsafe {
csr::rtio::target_write(csr::CONFIG_RTIO_LOG_CHANNEL << 8);
let mut word: u32 = 0;
for i in 0..data.len() {
word <<= 8;
word |= data[i] as u32;
if i % 4 == 3 {
rtio_o_data_write(0, word);
word = 0;
}
}
if word != 0 {
rtio_o_data_write(0, word);
}
}
}

View File

@ -1,25 +1,27 @@
use libc::{c_int, c_void}; use libc::{c_void, c_int};
use crate::libunwind as uw; use crate::libunwind as uw;
const UW_REG_SP: c_int = 13; const UW_REG_SP: c_int = 13;
pub fn backtrace<F>(f: F) -> Result<(), uw::_Unwind_Reason_Code> pub fn backtrace<F>(f: F) -> Result<(), uw::_Unwind_Reason_Code>
where F: FnMut(usize) -> () { where F: FnMut(usize) -> ()
{
struct TraceContext<F> { struct TraceContext<F> {
step_fn: F, step_fn: F,
prev_sp: uw::_Unwind_Word, prev_sp: uw::_Unwind_Word
} }
extern "C" fn trace<F>(context: *mut uw::_Unwind_Context, arg: *mut c_void) -> uw::_Unwind_Reason_Code extern fn trace<F>(context: *mut uw::_Unwind_Context, arg: *mut c_void)
where F: FnMut(usize) -> () { -> uw::_Unwind_Reason_Code
where F: FnMut(usize) -> ()
{
unsafe { unsafe {
let trace_context = &mut *(arg as *mut TraceContext<F>); let trace_context = &mut *(arg as *mut TraceContext<F>);
// Detect the root of a libfringe thread // Detect the root of a libfringe thread
let cur_sp = uw::_Unwind_GetGR(context, UW_REG_SP); let cur_sp = uw::_Unwind_GetGR(context, UW_REG_SP);
if cur_sp == trace_context.prev_sp { if cur_sp == trace_context.prev_sp {
return uw::_URC_END_OF_STACK; return uw::_URC_END_OF_STACK
} else { } else {
trace_context.prev_sp = cur_sp; trace_context.prev_sp = cur_sp;
} }
@ -33,7 +35,7 @@ where F: FnMut(usize) -> () {
let mut trace_context = TraceContext { step_fn: f, prev_sp: 0 }; let mut trace_context = TraceContext { step_fn: f, prev_sp: 0 };
match uw::_Unwind_Backtrace(trace::<F>, &mut trace_context as *mut _ as *mut c_void) { match uw::_Unwind_Backtrace(trace::<F>, &mut trace_context as *mut _ as *mut c_void) {
uw::_URC_NO_REASON => Ok(()), uw::_URC_NO_REASON => Ok(()),
err => Err(err), err => Err(err)
} }
} }
} }

View File

@ -5,7 +5,8 @@ fn main() {
} }
mod llvm_libunwind { mod llvm_libunwind {
use std::{env, path::Path}; use std::path::Path;
use std::env;
fn setup_options(cfg: &mut cc::Build) { fn setup_options(cfg: &mut cc::Build) {
cfg.no_default_flags(true); cfg.no_default_flags(true);
@ -81,7 +82,11 @@ mod llvm_libunwind {
cfg.flag("-fvisibility=hidden"); cfg.flag("-fvisibility=hidden");
cfg.flag_if_supported("-fvisibility-global-new-delete-hidden"); cfg.flag_if_supported("-fvisibility-global-new-delete-hidden");
let unwind_sources = vec!["Unwind-EHABI.cpp", "Unwind-seh.cpp", "libunwind.cpp"]; let unwind_sources = vec![
"Unwind-EHABI.cpp",
"Unwind-seh.cpp",
"libunwind.cpp"
];
let root = Path::new("../llvm_libunwind"); let root = Path::new("../llvm_libunwind");
cfg.include(root.join("include")); cfg.include(root.join("include"));

View File

@ -21,7 +21,8 @@ pub use _Unwind_Reason_Code::*;
pub type _Unwind_Exception_Class = u64; pub type _Unwind_Exception_Class = u64;
pub type _Unwind_Word = uintptr_t; pub type _Unwind_Word = uintptr_t;
pub type _Unwind_Ptr = uintptr_t; pub type _Unwind_Ptr = uintptr_t;
pub type _Unwind_Trace_Fn = extern "C" fn(ctx: *mut _Unwind_Context, arg: *mut c_void) -> _Unwind_Reason_Code; pub type _Unwind_Trace_Fn =
extern "C" fn(ctx: *mut _Unwind_Context, arg: *mut c_void) -> _Unwind_Reason_Code;
#[cfg(target_arch = "x86")] #[cfg(target_arch = "x86")]
pub const unwinder_private_data_size: usize = 5; pub const unwinder_private_data_size: usize = 5;
@ -278,6 +279,7 @@ if #[cfg(all(windows, target_arch = "x86_64", target_env = "gnu"))] {
} // cfg_if! } // cfg_if!
#[no_mangle] #[no_mangle]
extern "C" fn abort() { extern fn abort() {
panic!("Abort!"); panic!("Abort!");
} }

View File

@ -8,7 +8,6 @@ edition = "2018"
[features] [features]
target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"] target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"] target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"]
target_ebaz4205 = ["libboard_zynq/target_ebaz4205", "libsupport_zynq/target_ebaz4205", "libconfig/target_ebaz4205", "libboard_artiq/target_ebaz4205"]
default = ["target_zc706"] default = ["target_zc706"]
[build-dependencies] [build-dependencies]
@ -19,31 +18,27 @@ num-traits = { version = "0.2", default-features = false }
num-derive = "0.3" num-derive = "0.3"
cslice = "0.3" cslice = "0.3"
log = "0.4" log = "0.4"
nb = "0.1"
embedded-hal = "0.2" embedded-hal = "0.2"
core_io = { version = "0.1", features = ["collections"] } core_io = { version = "0.1", features = ["collections"] }
crc = { version = "1.7", default-features = false }
byteorder = { version = "1.3", default-features = false } byteorder = { version = "1.3", default-features = false }
void = { version = "1", default-features = false } void = { version = "1", default-features = false }
futures = { version = "0.3", default-features = false, features = ["async-await"] } futures = { version = "0.3", default-features = false, features = ["async-await"] }
async-recursion = "0.3" async-recursion = "0.3"
log_buffer = { version = "1.2" } log_buffer = { version = "1.2" }
libm = { version = "0.2", features = ["unstable"] }
vcell = "0.1" vcell = "0.1"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]} libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] } libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" } libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { path = "@@ZYNQ_RS@@/libasync" } libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { path = "@@ZYNQ_RS@@/libregister" } libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] } libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] }
dyld = { path = "../libdyld" } dyld = { path = "../libdyld" }
dwarf = { path = "../libdwarf" } dwarf = { path = "../libdwarf" }
unwind = { path = "../libunwind" } unwind = { path = "../libunwind" }
libc = { path = "../libc" } libc = { path = "../libc" }
io = { path = "../libio", features = ["alloc"] } io = { path = "../libio" }
ksupport = { path = "../libksupport" }
libboard_artiq = { path = "../libboard_artiq" } libboard_artiq = { path = "../libboard_artiq" }
[dependencies.tar-no-std]
git = "https://git.m-labs.hk/M-Labs/tar-no-std"
rev = "2ab6dc5"

View File

@ -1,15 +1,10 @@
use alloc::rc::Rc;
#[cfg(has_drtio)]
use alloc::vec::Vec;
use core::cell::RefCell;
use libasync::{smoltcp::TcpStream, task}; use libasync::{smoltcp::TcpStream, task};
use libboard_artiq::drtio_routing; use libboard_zynq::smoltcp::Error;
use libboard_zynq::{smoltcp::Error, timer::GlobalTimer}; use libcortex_a9::cache;
use libcortex_a9::{cache, mutex::Mutex};
use log::{debug, info, warn}; use log::{debug, info, warn};
use crate::{pl, proto_async::*}; use crate::proto_async::*;
use crate::pl;
const BUFFER_SIZE: usize = 512 * 1024; const BUFFER_SIZE: usize = 512 * 1024;
@ -18,7 +13,9 @@ struct Buffer {
data: [u8; BUFFER_SIZE], data: [u8; BUFFER_SIZE],
} }
static mut BUFFER: Buffer = Buffer { data: [0; BUFFER_SIZE] }; static mut BUFFER: Buffer = Buffer {
data: [0; BUFFER_SIZE]
};
fn arm() { fn arm() {
debug!("arming RTIO analyzer"); debug!("arming RTIO analyzer");
@ -43,57 +40,13 @@ fn disarm() {
debug!("RTIO analyzer disarmed"); debug!("RTIO analyzer disarmed");
} }
#[cfg(has_drtio)]
pub mod remote_analyzer {
use super::*;
use crate::rtio_mgt::drtio;
pub struct RemoteBuffer {
pub total_byte_count: u64,
pub sent_bytes: u32,
pub error: bool,
pub data: Vec<u8>,
}
pub async fn get_data(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) -> Result<RemoteBuffer, drtio::Error> {
// gets data from satellites and returns consolidated data
let mut remote_data: Vec<u8> = Vec::new();
let mut remote_error = false;
let mut remote_sent_bytes = 0;
let mut remote_total_bytes = 0;
let data_vec = match drtio::analyzer_query(aux_mutex, routing_table, up_destinations, timer).await {
Ok(data_vec) => data_vec,
Err(e) => return Err(e),
};
for data in data_vec {
remote_total_bytes += data.total_byte_count;
remote_sent_bytes += data.sent_bytes;
remote_error |= data.error;
remote_data.extend(data.data);
}
Ok(RemoteBuffer {
total_byte_count: remote_total_bytes,
sent_bytes: remote_sent_bytes,
error: remote_error,
data: remote_data,
})
}
}
#[derive(Debug)] #[derive(Debug)]
struct Header { struct Header {
sent_bytes: u32, sent_bytes: u32,
total_byte_count: u64, total_byte_count: u64,
error_occurred: bool, error_occurred: bool,
log_channel: u8, log_channel: u8,
dds_onehot_sel: bool, dds_onehot_sel: bool
} }
async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Error> { async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Error> {
@ -106,13 +59,7 @@ async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Err
Ok(()) Ok(())
} }
async fn handle_connection( async fn handle_connection(stream: &mut TcpStream) -> Result<(), Error> {
stream: &mut TcpStream,
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &drtio_routing::RoutingTable,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
_timer: GlobalTimer,
) -> Result<(), Error> {
info!("received connection"); info!("received connection");
let data = unsafe { &BUFFER.data[..] }; let data = unsafe { &BUFFER.data[..] };
@ -121,11 +68,6 @@ async fn handle_connection(
let total_byte_count = unsafe { pl::csr::rtio_analyzer::dma_byte_count_read() as u64 }; let total_byte_count = unsafe { pl::csr::rtio_analyzer::dma_byte_count_read() as u64 };
let pointer = (total_byte_count % BUFFER_SIZE as u64) as usize; let pointer = (total_byte_count % BUFFER_SIZE as u64) as usize;
let wraparound = total_byte_count >= BUFFER_SIZE as u64; let wraparound = total_byte_count >= BUFFER_SIZE as u64;
let sent_bytes = if wraparound {
BUFFER_SIZE as u32
} else {
total_byte_count as u32
};
if overflow_occurred { if overflow_occurred {
warn!("overflow occured"); warn!("overflow occured");
@ -134,42 +76,12 @@ async fn handle_connection(
warn!("bus error occured"); warn!("bus error occured");
} }
#[cfg(has_drtio)]
let remote = remote_analyzer::get_data(_aux_mutex, _routing_table, _up_destinations, _timer).await;
#[cfg(has_drtio)]
let (header, remote_data) = match remote {
Ok(remote) => (
Header {
total_byte_count: total_byte_count + remote.total_byte_count,
sent_bytes: sent_bytes + remote.sent_bytes,
error_occurred: overflow_occurred | bus_error_occurred | remote.error,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true,
},
remote.data,
),
Err(e) => {
warn!("Error getting remote analyzer data: {}", e);
(
Header {
total_byte_count: total_byte_count,
sent_bytes: sent_bytes,
error_occurred: true,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true,
},
Vec::new(),
)
}
};
#[cfg(not(has_drtio))]
let header = Header { let header = Header {
total_byte_count: total_byte_count, total_byte_count: total_byte_count,
sent_bytes: sent_bytes, sent_bytes: if wraparound { BUFFER_SIZE as u32 } else { total_byte_count as u32 },
error_occurred: overflow_occurred | bus_error_occurred, error_occurred: overflow_occurred | bus_error_occurred,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8, log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true, // kept for backward compatibility of analyzer dumps dds_onehot_sel: true // kept for backward compatibility of analyzer dumps
}; };
debug!("{:?}", header); debug!("{:?}", header);
@ -180,28 +92,17 @@ async fn handle_connection(
} else { } else {
stream.send(data[..pointer].iter().copied()).await?; stream.send(data[..pointer].iter().copied()).await?;
} }
#[cfg(has_drtio)]
stream.send(remote_data.iter().copied()).await?;
Ok(()) Ok(())
} }
pub fn start( pub fn start() {
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
let up_destinations = up_destinations.clone();
task::spawn(async move { task::spawn(async move {
loop { loop {
arm(); arm();
let mut stream = TcpStream::accept(1382, 2048, 2048).await.unwrap(); let mut stream = TcpStream::accept(1382, 2048, 2048).await.unwrap();
disarm(); disarm();
let routing_table = routing_table.borrow(); let _ = handle_connection(&mut stream)
let _ = handle_connection(&mut stream, &aux_mutex, &routing_table, &up_destinations, timer)
.await .await
.map_err(|e| warn!("connection terminated: {:?}", e)); .map_err(|e| warn!("connection terminated: {:?}", e));
let _ = stream.flush().await; let _ = stream.flush().await;

View File

@ -1,57 +1,46 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String, vec, vec::Vec}; use core::{fmt, slice, str};
use core::{cell::RefCell, fmt, slice, str}; use core::cell::RefCell;
use alloc::{vec, vec::Vec, string::String, collections::BTreeMap, rc::Rc};
use core_io::Error as IoError; use log::{info, warn, error};
use cslice::CSlice; use cslice::CSlice;
use dyld::elf;
use futures::{future::FutureExt, select_biased}; use num_derive::{FromPrimitive, ToPrimitive};
#[cfg(has_drtio)] use num_traits::{FromPrimitive, ToPrimitive};
use io::Cursor;
#[cfg(has_drtio)] use libboard_zynq::{
use ksupport::rpc; self as zynq,
use ksupport::{kernel, resolve_channel_name}; smoltcp::{
#[cfg(has_drtio)] self,
use libasync::delay; wire::IpCidr,
use libasync::{smoltcp::{Sockets, TcpStream}, iface::{NeighborCache, EthernetInterfaceBuilder},
task}; time::Instant,
},
timer::GlobalTimer,
};
use libcortex_a9::{semaphore::Semaphore, mutex::Mutex, sync_channel::{Sender, Receiver}};
use futures::{select_biased, future::FutureExt};
use libasync::{smoltcp::{Sockets, TcpStream}, task};
use libconfig::{Config, net_settings};
use libboard_artiq::drtio_routing; use libboard_artiq::drtio_routing;
#[cfg(feature = "target_kasli_soc")] #[cfg(feature = "target_kasli_soc")]
use libboard_zynq::error_led::ErrorLED; use libboard_zynq::error_led::ErrorLED;
#[cfg(has_drtio)]
use libboard_zynq::time::Milliseconds;
use libboard_zynq::{self as zynq,
smoltcp::{self,
iface::{EthernetInterfaceBuilder, NeighborCache},
time::Instant,
wire::IpCidr},
timer::GlobalTimer};
use libconfig::{net_settings, Config};
use libcortex_a9::{mutex::Mutex,
semaphore::Semaphore,
sync_channel::{Receiver, Sender}};
use log::{error, info, warn};
use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive};
#[cfg(has_drtio)]
use tar_no_std::TarArchiveRef;
use crate::proto_async::*;
use crate::kernel;
use crate::rpc;
use crate::moninj;
use crate::mgmt;
use crate::analyzer;
use crate::rtio_mgt::{self, resolve_channel_name};
#[cfg(has_drtio)] #[cfg(has_drtio)]
use crate::pl; use crate::pl;
use crate::{analyzer, mgmt, moninj, proto_async::*, rpc_async, rtio_dma, rtio_mgt};
#[cfg(has_drtio)]
use crate::{subkernel, subkernel::Error as SubkernelError};
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error { pub enum Error {
NetworkError(smoltcp::Error), NetworkError(smoltcp::Error),
IoError,
UnexpectedPattern, UnexpectedPattern,
UnrecognizedPacket, UnrecognizedPacket,
BufferExhausted, BufferExhausted,
#[cfg(has_drtio)]
SubkernelError(subkernel::Error),
#[cfg(has_drtio)]
DestinationDown,
} }
pub type Result<T> = core::result::Result<T, Error>; pub type Result<T> = core::result::Result<T, Error>;
@ -60,14 +49,9 @@ impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {
Error::NetworkError(error) => write!(f, "network error: {}", error), Error::NetworkError(error) => write!(f, "network error: {}", error),
Error::IoError => write!(f, "io error"),
Error::UnexpectedPattern => write!(f, "unexpected pattern"), Error::UnexpectedPattern => write!(f, "unexpected pattern"),
Error::UnrecognizedPacket => write!(f, "unrecognized packet"), Error::UnrecognizedPacket => write!(f, "unrecognized packet"),
Error::BufferExhausted => write!(f, "buffer exhausted"), Error::BufferExhausted => write!(f, "buffer exhausted"),
#[cfg(has_drtio)]
Error::SubkernelError(error) => write!(f, "subkernel error: {:?}", error),
#[cfg(has_drtio)]
Error::DestinationDown => write!(f, "subkernel destination down"),
} }
} }
} }
@ -78,19 +62,6 @@ impl From<smoltcp::Error> for Error {
} }
} }
impl From<IoError> for Error {
fn from(_error: IoError) -> Self {
Error::IoError
}
}
#[cfg(has_drtio)]
impl From<subkernel::Error> for Error {
fn from(error: subkernel::Error) -> Self {
Error::SubkernelError(error)
}
}
#[derive(Debug, FromPrimitive, ToPrimitive)] #[derive(Debug, FromPrimitive, ToPrimitive)]
enum Request { enum Request {
SystemInfo = 3, SystemInfo = 3,
@ -98,7 +69,6 @@ enum Request {
RunKernel = 6, RunKernel = 6,
RPCReply = 7, RPCReply = 7,
RPCException = 8, RPCException = 8,
UploadSubkernel = 9,
} }
#[derive(Debug, FromPrimitive, ToPrimitive)] #[derive(Debug, FromPrimitive, ToPrimitive)]
@ -115,18 +85,18 @@ enum Reply {
} }
static CACHE_STORE: Mutex<BTreeMap<String, Vec<i32>>> = Mutex::new(BTreeMap::new()); static CACHE_STORE: Mutex<BTreeMap<String, Vec<i32>>> = Mutex::new(BTreeMap::new());
static DMA_RECORD_STORE: Mutex<BTreeMap<String, (Vec<u8>, i64)>> = Mutex::new(BTreeMap::new());
async fn write_header(stream: &TcpStream, reply: Reply) -> Result<()> { async fn write_header(stream: &TcpStream, reply: Reply) -> Result<()> {
stream stream.send_slice(&[0x5a, 0x5a, 0x5a, 0x5a, reply.to_u8().unwrap()]).await?;
.send_slice(&[0x5a, 0x5a, 0x5a, 0x5a, reply.to_u8().unwrap()])
.await?;
Ok(()) Ok(())
} }
async fn read_request(stream: &TcpStream, allow_close: bool) -> Result<Option<Request>> { async fn read_request(stream: &TcpStream, allow_close: bool) -> Result<Option<Request>> {
match expect(stream, &[0x5a, 0x5a, 0x5a, 0x5a]).await { match expect(stream, &[0x5a, 0x5a, 0x5a, 0x5a]).await {
Ok(true) => {} Ok(true) => {}
Ok(false) => return Err(Error::UnexpectedPattern), Ok(false) =>
return Err(Error::UnexpectedPattern),
Err(smoltcp::Error::Finished) => { Err(smoltcp::Error::Finished) => {
if allow_close { if allow_close {
info!("peer closed connection"); info!("peer closed connection");
@ -135,12 +105,11 @@ async fn read_request(stream: &TcpStream, allow_close: bool) -> Result<Option<Re
error!("peer unexpectedly closed connection"); error!("peer unexpectedly closed connection");
return Err(smoltcp::Error::Finished)?; return Err(smoltcp::Error::Finished)?;
} }
},
Err(e) =>
return Err(e)?,
} }
Err(e) => return Err(e)?, Ok(Some(FromPrimitive::from_i8(read_i8(&stream).await?).ok_or(Error::UnrecognizedPacket)?))
}
Ok(Some(
FromPrimitive::from_i8(read_i8(&stream).await?).ok_or(Error::UnrecognizedPacket)?,
))
} }
async fn read_bytes(stream: &TcpStream, max_length: usize) -> Result<Vec<u8>> { async fn read_bytes(stream: &TcpStream, max_length: usize) -> Result<Vec<u8>> {
@ -159,7 +128,9 @@ async fn fast_send(sender: &mut Sender<'_, kernel::Message>, content: kernel::Me
let mut content = content; let mut content = content;
for _ in 0..RETRY_LIMIT { for _ in 0..RETRY_LIMIT {
match sender.try_send(content) { match sender.try_send(content) {
Ok(()) => return, Ok(()) => {
return
},
Err(v) => { Err(v) => {
content = v; content = v;
} }
@ -171,8 +142,10 @@ async fn fast_send(sender: &mut Sender<'_, kernel::Message>, content: kernel::Me
async fn fast_recv(receiver: &mut Receiver<'_, kernel::Message>) -> kernel::Message { async fn fast_recv(receiver: &mut Receiver<'_, kernel::Message>) -> kernel::Message {
for _ in 0..RETRY_LIMIT { for _ in 0..RETRY_LIMIT {
match receiver.try_recv() { match receiver.try_recv() {
Ok(v) => return v, Ok(v) => {
Err(()) => (), return v;
},
Err(()) => ()
} }
} }
receiver.async_recv().await receiver.async_recv().await
@ -188,14 +161,7 @@ async fn write_exception_string(stream: &TcpStream, s: CSlice<'static, u8>) -> R
Ok(()) Ok(())
} }
async fn handle_run_kernel( async fn handle_run_kernel(stream: Option<&TcpStream>, control: &Rc<RefCell<kernel::Control>>, _up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>) -> Result<()> {
stream: Option<&TcpStream>,
control: &Rc<RefCell<kernel::Control>>,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
) -> Result<()> {
control.borrow_mut().tx.async_send(kernel::Message::StartRequest).await; control.borrow_mut().tx.async_send(kernel::Message::StartRequest).await;
loop { loop {
let reply = control.borrow_mut().rx.async_recv().await; let reply = control.borrow_mut().rx.async_recv().await;
@ -203,7 +169,7 @@ async fn handle_run_kernel(
kernel::Message::RpcSend { is_async, data } => { kernel::Message::RpcSend { is_async, data } => {
if stream.is_none() { if stream.is_none() {
error!("Unexpected RPC from startup/idle kernel!"); error!("Unexpected RPC from startup/idle kernel!");
break; break
} }
let stream = stream.unwrap(); let stream = stream.unwrap();
write_header(stream, Reply::RPCRequest).await?; write_header(stream, Reply::RPCRequest).await?;
@ -218,7 +184,7 @@ async fn handle_run_kernel(
kernel::Message::RpcRecvRequest(slot) => slot, kernel::Message::RpcRecvRequest(slot) => slot,
other => panic!("expected root value slot from core1, not {:?}", other), other => panic!("expected root value slot from core1, not {:?}", other),
}; };
rpc_async::recv_return(stream, &tag, slot, &|size| { rpc::recv_return(stream, &tag, slot, &|size| {
let control = control.clone(); let control = control.clone();
async move { async move {
if size == 0 { if size == 0 {
@ -230,20 +196,13 @@ async fn handle_run_kernel(
fast_send(&mut control.tx, kernel::Message::RpcRecvReply(Ok(size))).await; fast_send(&mut control.tx, kernel::Message::RpcRecvReply(Ok(size))).await;
match fast_recv(&mut control.rx).await { match fast_recv(&mut control.rx).await {
kernel::Message::RpcRecvRequest(slot) => slot, kernel::Message::RpcRecvRequest(slot) => slot,
other => { other => panic!("expected nested value slot from kernel CPU, not {:?}", other),
panic!("expected nested value slot from kernel CPU, not {:?}", other)
} }
} }
} }
} }).await?;
}) control.borrow_mut().tx.async_send(kernel::Message::RpcRecvReply(Ok(0))).await;
.await?; },
control
.borrow_mut()
.tx
.async_send(kernel::Message::RpcRecvReply(Ok(0)))
.await;
}
Request::RPCException => { Request::RPCException => {
let mut control = control.borrow_mut(); let mut control = control.borrow_mut();
match control.rx.async_recv().await { match control.rx.async_recv().await {
@ -252,42 +211,31 @@ async fn handle_run_kernel(
} }
let id = read_i32(stream).await? as u32; let id = read_i32(stream).await? as u32;
let message = read_i32(stream).await? as u32; let message = read_i32(stream).await? as u32;
let param = [ let param = [read_i64(stream).await?,
read_i64(stream).await?, read_i64(stream).await?,
read_i64(stream).await?, read_i64(stream).await?];
read_i64(stream).await?,
];
let file = read_i32(stream).await? as u32; let file = read_i32(stream).await? as u32;
let line = read_i32(stream).await?; let line = read_i32(stream).await?;
let column = read_i32(stream).await?; let column = read_i32(stream).await?;
let function = read_i32(stream).await? as u32; let function = read_i32(stream).await? as u32;
control control.tx.async_send(kernel::Message::RpcRecvReply(Err(kernel::RPCException {
.tx id, message, param, file, line, column, function
.async_send(kernel::Message::RpcRecvReply(Err(ksupport::RPCException { }))).await;
id, },
message,
param,
file,
line,
column,
function,
})))
.await;
}
_ => { _ => {
error!("unexpected RPC request from host: {:?}", host_request); error!("unexpected RPC request from host: {:?}", host_request);
return Err(Error::UnrecognizedPacket); return Err(Error::UnrecognizedPacket)
}
} }
} }
} }
},
kernel::Message::KernelFinished(async_errors) => { kernel::Message::KernelFinished(async_errors) => {
if let Some(stream) = stream { if let Some(stream) = stream {
write_header(stream, Reply::KernelFinished).await?; write_header(stream, Reply::KernelFinished).await?;
write_i8(stream, async_errors as i8).await?; write_i8(stream, async_errors as i8).await?;
} }
break; break;
} },
kernel::Message::KernelException(exceptions, stack_pointers, backtrace, async_errors) => { kernel::Message::KernelException(exceptions, stack_pointers, backtrace, async_errors) => {
match stream { match stream {
Some(stream) => { Some(stream) => {
@ -299,22 +247,12 @@ async fn handle_run_kernel(
let exception = exception.as_ref().unwrap(); let exception = exception.as_ref().unwrap();
write_i32(stream, exception.id as i32).await?; write_i32(stream, exception.id as i32).await?;
if exception.message.len() == usize::MAX { if exception.message.len() == usize::MAX { // exception with host string
// exception with host string
write_exception_string(stream, exception.message).await?; write_exception_string(stream, exception.message).await?;
} else { } else {
let msg = str::from_utf8(unsafe { let msg = str::from_utf8(unsafe { slice::from_raw_parts(exception.message.as_ptr(), exception.message.len()) })
slice::from_raw_parts(exception.message.as_ptr(), exception.message.len())
})
.unwrap() .unwrap()
.replace( .replace("{rtio_channel_info:0}", &format!("0x{:04x}:{}", exception.param[0], resolve_channel_name(exception.param[0] as u32)));
"{rtio_channel_info:0}",
&format!(
"0x{:04x}:{}",
exception.param[0],
resolve_channel_name(exception.param[0] as u32)
),
);
write_exception_string(stream, unsafe { CSlice::new(msg.as_ptr(), msg.len()) }).await?; write_exception_string(stream, unsafe { CSlice::new(msg.as_ptr(), msg.len()) }).await?;
} }
@ -337,7 +275,7 @@ async fn handle_run_kernel(
write_i32(stream, sp as i32).await?; write_i32(stream, sp as i32).await?;
} }
write_i8(stream, async_errors as i8).await?; write_i8(stream, async_errors as i8).await?;
} },
None => { None => {
error!("Uncaught kernel exceptions: {:?}", exceptions); error!("Uncaught kernel exceptions: {:?}", exceptions);
} }
@ -346,184 +284,27 @@ async fn handle_run_kernel(
} }
kernel::Message::CachePutRequest(key, value) => { kernel::Message::CachePutRequest(key, value) => {
CACHE_STORE.lock().insert(key, value); CACHE_STORE.lock().insert(key, value);
} },
kernel::Message::CacheGetRequest(key) => { kernel::Message::CacheGetRequest(key) => {
const DEFAULT: Vec<i32> = Vec::new(); const DEFAULT: Vec<i32> = Vec::new();
let value = CACHE_STORE.lock().get(&key).unwrap_or(&DEFAULT).clone(); let value = CACHE_STORE.lock().get(&key).unwrap_or(&DEFAULT).clone();
control control.borrow_mut().tx.async_send(kernel::Message::CacheGetReply(value)).await;
.borrow_mut() },
.tx
.async_send(kernel::Message::CacheGetReply(value))
.await;
}
kernel::Message::DmaPutRequest(recorder) => { kernel::Message::DmaPutRequest(recorder) => {
let _id = rtio_dma::put_record(aux_mutex, routing_table, timer, recorder).await; DMA_RECORD_STORE.lock().insert(recorder.name, (recorder.buffer, recorder.duration));
#[cfg(has_drtio)] },
rtio_dma::remote_dma::upload_traces(aux_mutex, routing_table, timer, _id).await;
}
kernel::Message::DmaEraseRequest(name) => { kernel::Message::DmaEraseRequest(name) => {
// prevent possible OOM when we have large DMA record replacement. // prevent possible OOM when we have large DMA record replacement.
rtio_dma::erase(name, aux_mutex, routing_table, timer).await; DMA_RECORD_STORE.lock().remove(&name);
} },
kernel::Message::DmaGetRequest(name) => { kernel::Message::DmaGetRequest(name) => {
let result = rtio_dma::retrieve(name).await; let result = DMA_RECORD_STORE.lock().get(&name).map(|v| v.clone());
control control.borrow_mut().tx.async_send(kernel::Message::DmaGetReply(result)).await;
.borrow_mut()
.tx
.async_send(kernel::Message::DmaGetReply(result))
.await;
}
#[cfg(has_drtio)]
kernel::Message::DmaStartRemoteRequest { id, timestamp } => {
rtio_dma::remote_dma::playback(aux_mutex, routing_table, timer, id as u32, timestamp as u64).await;
}
#[cfg(has_drtio)]
kernel::Message::DmaAwaitRemoteRequest(id) => {
let result = rtio_dma::remote_dma::await_done(id as u32, Some(10_000), timer).await;
let reply = match result {
Ok(rtio_dma::remote_dma::RemoteState::PlaybackEnded {
error,
channel,
timestamp,
}) => kernel::Message::DmaAwaitRemoteReply {
timeout: false,
error: error,
channel: channel,
timestamp: timestamp,
}, },
_ => kernel::Message::DmaAwaitRemoteReply {
timeout: true,
error: 0,
channel: 0,
timestamp: 0,
},
};
control.borrow_mut().tx.async_send(reply).await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelLoadRunRequest {
id,
destination: _,
run,
timestamp,
} => {
let succeeded = match subkernel::load(aux_mutex, routing_table, timer, id, run, timestamp).await {
Ok(()) => true,
Err(e) => {
error!("Error loading subkernel: {:?}", e);
false
}
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelLoadRunReply { succeeded: succeeded })
.await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelAwaitFinishRequest { id, timeout } => {
let res = subkernel::await_finish(aux_mutex, routing_table, timer, id, timeout).await;
let response = match res {
Ok(res) => {
if res.status == subkernel::FinishStatus::CommLost {
kernel::Message::SubkernelError(kernel::SubkernelStatus::CommLost)
} else if let Some(exception) = res.exception {
kernel::Message::SubkernelError(kernel::SubkernelStatus::Exception(exception))
} else {
kernel::Message::SubkernelAwaitFinishReply
}
}
Err(SubkernelError::Timeout) => kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout),
Err(SubkernelError::IncorrectState) => {
kernel::Message::SubkernelError(kernel::SubkernelStatus::IncorrectState)
}
Err(_) => kernel::Message::SubkernelError(kernel::SubkernelStatus::OtherError),
};
control.borrow_mut().tx.async_send(response).await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelMsgSend { id, destination, data } => {
let res =
subkernel::message_send(aux_mutex, routing_table, timer, id, destination.unwrap(), data).await;
match res {
Ok(_) => (),
Err(e) => {
error!("error sending subkernel message: {:?}", e)
}
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelMsgSent)
.await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelMsgRecvRequest { id, timeout, tags } => {
let message_received = subkernel::message_await(id as u32, timeout, timer).await;
let response = match message_received {
Ok(ref message) => kernel::Message::SubkernelMsgRecvReply { count: message.count },
Err(SubkernelError::Timeout) => kernel::Message::SubkernelError(kernel::SubkernelStatus::Timeout),
Err(SubkernelError::IncorrectState) => {
kernel::Message::SubkernelError(kernel::SubkernelStatus::IncorrectState)
}
Err(SubkernelError::CommLost) => kernel::Message::SubkernelError(kernel::SubkernelStatus::CommLost),
Err(SubkernelError::SubkernelException) => {
// just retrieve the exception
let status = subkernel::await_finish(aux_mutex, routing_table, timer, id as u32, timeout)
.await
.unwrap();
kernel::Message::SubkernelError(kernel::SubkernelStatus::Exception(status.exception.unwrap()))
}
Err(_) => kernel::Message::SubkernelError(kernel::SubkernelStatus::OtherError),
};
control.borrow_mut().tx.async_send(response).await;
if let Ok(message) = message_received {
// receive code almost identical to RPC recv, except we are not reading from a stream
let mut reader = Cursor::new(message.data);
let mut current_tags: &[u8] = &tags;
let mut i = 0;
loop {
// kernel has to consume all arguments in the whole message
let slot = match fast_recv(&mut control.borrow_mut().rx).await {
kernel::Message::RpcRecvRequest(slot) => slot,
other => panic!("expected root value slot from core1, not {:?}", other),
};
let remaining_tags = rpc::recv_return(&mut reader, &current_tags, slot, &mut |size| {
if size == 0 {
0 as *mut ()
} else {
let mut control = control.borrow_mut();
control.tx.send(kernel::Message::RpcRecvReply(Ok(size)));
match control.rx.recv() {
kernel::Message::RpcRecvRequest(slot) => slot,
other => {
panic!("expected nested value slot from kernel CPU, not {:?}", other)
}
}
}
})?;
control
.borrow_mut()
.tx
.async_send(kernel::Message::RpcRecvReply(Ok(0)))
.await;
i += 1;
if i < message.count {
current_tags = remaining_tags;
} else {
break;
}
}
}
}
#[cfg(has_drtio)] #[cfg(has_drtio)]
kernel::Message::UpDestinationsRequest(destination) => { kernel::Message::UpDestinationsRequest(destination) => {
let result = _up_destinations.borrow()[destination as usize]; let result = _up_destinations.borrow()[destination as usize];
control control.borrow_mut().tx.async_send(kernel::Message::UpDestinationsReply(result)).await;
.borrow_mut()
.tx
.async_send(kernel::Message::UpDestinationsReply(result))
.await;
} }
_ => { _ => {
panic!("unexpected message from core1 while kernel was running: {:?}", reply); panic!("unexpected message from core1 while kernel was running: {:?}", reply);
@ -533,67 +314,11 @@ async fn handle_run_kernel(
Ok(()) Ok(())
} }
async fn handle_flash_kernel(
buffer: &Vec<u8>,
control: &Rc<RefCell<kernel::Control>>,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &drtio_routing::RoutingTable,
_timer: GlobalTimer,
) -> Result<()> {
if buffer[0] == elf::ELFMAG0 && buffer[1] == elf::ELFMAG1 && buffer[2] == elf::ELFMAG2 && buffer[3] == elf::ELFMAG3
{
// assume ELF file, proceed as before
load_kernel(buffer, control, None).await
} else {
#[cfg(has_drtio)]
{
let archive = TarArchiveRef::new(buffer.as_ref());
let entries = archive.entries();
let mut main_lib: Vec<u8> = Vec::new();
for entry in entries {
if entry.filename().as_str() == "main.elf" {
main_lib = entry.data().to_vec();
} else {
// subkernel filename must be in format:
// "<subkernel id> <destination>.elf"
let filename = entry.filename();
let mut iter = filename.as_str().split_whitespace();
let sid: u32 = iter.next().unwrap().parse().unwrap();
let dest: u8 = iter.next().unwrap().strip_suffix(".elf").unwrap().parse().unwrap();
let up = _up_destinations.borrow()[dest as usize];
if up {
let subkernel_lib = entry.data().to_vec();
subkernel::add_subkernel(sid, dest, subkernel_lib).await;
match subkernel::upload(_aux_mutex, _routing_table, _timer, sid).await {
Ok(_) => (),
Err(_) => return Err(Error::UnexpectedPattern),
}
} else {
return Err(Error::DestinationDown);
}
}
}
load_kernel(&main_lib, control, None).await
}
#[cfg(not(has_drtio))]
{
panic!("multi-kernel libraries are not supported in standalone systems");
}
}
}
async fn load_kernel( async fn load_kernel(buffer: &Vec<u8>, control: &Rc<RefCell<kernel::Control>>, stream: Option<&TcpStream>) -> Result<()> {
buffer: &Vec<u8>,
control: &Rc<RefCell<kernel::Control>>,
stream: Option<&TcpStream>,
) -> Result<()> {
let mut control = control.borrow_mut(); let mut control = control.borrow_mut();
control.restart(); control.restart();
control control.tx.async_send(kernel::Message::LoadRequest(buffer.to_vec())).await;
.tx
.async_send(kernel::Message::LoadRequest(buffer.to_vec()))
.await;
let reply = control.rx.async_recv().await; let reply = control.rx.async_recv().await;
match reply { match reply {
kernel::Message::LoadCompleted => { kernel::Message::LoadCompleted => {
@ -601,7 +326,7 @@ async fn load_kernel(
write_header(stream, Reply::LoadCompleted).await?; write_header(stream, Reply::LoadCompleted).await?;
} }
Ok(()) Ok(())
} },
kernel::Message::LoadFailed => { kernel::Message::LoadFailed => {
if let Some(stream) = stream { if let Some(stream) = stream {
write_header(stream, Reply::LoadFailed).await?; write_header(stream, Reply::LoadFailed).await?;
@ -610,7 +335,7 @@ async fn load_kernel(
error!("Kernel load failed"); error!("Kernel load failed");
} }
Err(Error::UnexpectedPattern) Err(Error::UnexpectedPattern)
} },
_ => { _ => {
error!("unexpected message from core1: {:?}", reply); error!("unexpected message from core1: {:?}", reply);
if let Some(stream) = stream { if let Some(stream) = stream {
@ -622,27 +347,16 @@ async fn load_kernel(
} }
} }
async fn handle_connection( async fn handle_connection(stream: &mut TcpStream, control: Rc<RefCell<kernel::Control>>, up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>) -> Result<()> {
stream: &mut TcpStream,
control: Rc<RefCell<kernel::Control>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
) -> Result<()> {
stream.set_ack_delay(None); stream.set_ack_delay(None);
if !expect(stream, b"ARTIQ coredev\n").await? { if !expect(stream, b"ARTIQ coredev\n").await? {
return Err(Error::UnexpectedPattern); return Err(Error::UnexpectedPattern);
} }
stream.send_slice("e".as_bytes()).await?; stream.send_slice("e".as_bytes()).await?;
#[cfg(has_drtio)]
subkernel::clear_subkernels().await;
loop { loop {
let request = read_request(stream, true).await?; let request = read_request(stream, true).await?;
if request.is_none() { if request.is_none() {
#[cfg(has_drtio)]
subkernel::clear_subkernels().await;
return Ok(()); return Ok(());
} }
let request = request.unwrap(); let request = request.unwrap();
@ -650,48 +364,17 @@ async fn handle_connection(
Request::SystemInfo => { Request::SystemInfo => {
write_header(stream, Reply::SystemInfo).await?; write_header(stream, Reply::SystemInfo).await?;
stream.send_slice("ARZQ".as_bytes()).await?; stream.send_slice("ARZQ".as_bytes()).await?;
} },
Request::LoadKernel => { Request::LoadKernel => {
let buffer = read_bytes(stream, 1024 * 1024).await?; let buffer = read_bytes(stream, 1024*1024).await?;
load_kernel(&buffer, &control, Some(stream)).await?; load_kernel(&buffer, &control, Some(stream)).await?;
} },
Request::RunKernel => { Request::RunKernel => {
handle_run_kernel( handle_run_kernel(Some(stream), &control, &up_destinations).await?;
Some(stream), },
&control,
&up_destinations,
aux_mutex,
routing_table,
timer,
)
.await?;
}
Request::UploadSubkernel => {
#[cfg(has_drtio)]
{
let id = read_i32(stream).await? as u32;
let destination = read_i8(stream).await? as u8;
let buffer = read_bytes(stream, 1024 * 1024).await?;
subkernel::add_subkernel(id, destination, buffer).await;
match subkernel::upload(aux_mutex, routing_table, timer, id).await {
Ok(_) => write_header(stream, Reply::LoadCompleted).await?,
Err(_) => {
write_header(stream, Reply::LoadFailed).await?;
write_chunk(stream, b"subkernel failed to load").await?;
return Err(Error::UnexpectedPattern);
}
}
}
#[cfg(not(has_drtio))]
{
write_header(stream, Reply::LoadFailed).await?;
write_chunk(stream, b"No DRTIO on this system, subkernels are not supported").await?;
return Err(Error::UnexpectedPattern);
}
}
_ => { _ => {
error!("unexpected request from host: {:?}", request); error!("unexpected request from host: {:?}", request);
return Err(Error::UnrecognizedPacket); return Err(Error::UnrecognizedPacket)
} }
} }
} }
@ -715,7 +398,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0),
IpCidr::new(addr, 0), IpCidr::new(addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
@ -726,7 +409,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
None => { None => {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
@ -738,79 +421,42 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
Sockets::init(32); Sockets::init(32);
// before, mutex was on io, but now that io isn't used...?
let aux_mutex: Rc<Mutex<bool>> = Rc::new(Mutex::new(false)); let aux_mutex: Rc<Mutex<bool>> = Rc::new(Mutex::new(false));
#[cfg(has_drtio)] #[cfg(has_drtio)]
let drtio_routing_table = Rc::new(RefCell::new(drtio_routing::config_routing_table( let drtio_routing_table = Rc::new(RefCell::new(
pl::csr::DRTIO.len(), drtio_routing::config_routing_table(pl::csr::DRTIO.len(), &cfg)));
&cfg,
)));
#[cfg(not(has_drtio))] #[cfg(not(has_drtio))]
let drtio_routing_table = Rc::new(RefCell::new(drtio_routing::RoutingTable::default_empty())); let drtio_routing_table = Rc::new(RefCell::new(drtio_routing::RoutingTable::default_empty()));
let up_destinations = Rc::new(RefCell::new([false; drtio_routing::DEST_COUNT])); let up_destinations = Rc::new(RefCell::new([false; drtio_routing::DEST_COUNT]));
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
drtio_routing::interconnect_disable_all(); drtio_routing::interconnect_disable_all();
rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, &cfg, timer); rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, timer, &cfg);
ksupport::setup_device_map(&cfg);
analyzer::start(&aux_mutex, &drtio_routing_table, &up_destinations, timer); analyzer::start();
moninj::start(timer, &aux_mutex, &drtio_routing_table); moninj::start(timer, aux_mutex, drtio_routing_table);
let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start())); let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start()));
let idle_kernel = Rc::new(cfg.read("idle_kernel").ok());
if let Ok(buffer) = cfg.read("startup_kernel") { if let Ok(buffer) = cfg.read("startup_kernel") {
info!("Loading startup kernel..."); info!("Loading startup kernel...");
let routing_table = drtio_routing_table.borrow(); if let Ok(()) = task::block_on(load_kernel(&buffer, &control, None)) {
if let Ok(()) = task::block_on(handle_flash_kernel(
&buffer,
&control,
&up_destinations,
&aux_mutex,
&routing_table,
timer,
)) {
info!("Starting startup kernel..."); info!("Starting startup kernel...");
let _ = task::block_on(handle_run_kernel( let _ = task::block_on(handle_run_kernel(None, &control, &up_destinations));
None,
&control,
&up_destinations,
&aux_mutex,
&routing_table,
timer,
));
info!("Startup kernel finished!"); info!("Startup kernel finished!");
} else { } else {
error!("Error loading startup kernel!"); error!("Error loading startup kernel!");
} }
} }
let cfg = Rc::new(cfg); mgmt::start(cfg);
let restart_idle = Rc::new(Semaphore::new(1, 1));
mgmt::start(
cfg.clone(),
restart_idle.clone(),
Some(mgmt::DrtioContext(
aux_mutex.clone(),
drtio_routing_table.clone(),
timer,
)),
);
task::spawn(async move { task::spawn(async move {
let connection = Rc::new(Semaphore::new(1, 1)); let connection = Rc::new(Semaphore::new(1, 1));
let terminate = Rc::new(Semaphore::new(0, 1)); let terminate = Rc::new(Semaphore::new(0, 1));
let can_restart_idle = Rc::new(Semaphore::new(1, 1));
let restart_idle = restart_idle.clone();
loop { loop {
let control = control.clone(); let mut stream = TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap();
let mut maybe_stream = select_biased! {
s = (async {
TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap()
}).fuse() => Some(s),
_ = (async {
restart_idle.async_wait().await;
can_restart_idle.async_wait().await;
}).fuse() => None
};
if connection.try_wait().is_none() { if connection.try_wait().is_none() {
// there is an existing connection // there is an existing connection
@ -818,66 +464,47 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
connection.async_wait().await; connection.async_wait().await;
} }
let maybe_idle_kernel = cfg.read("idle_kernel").ok();
if maybe_idle_kernel.is_none() && maybe_stream.is_none() {
control.borrow_mut().restart(); // terminate idle kernel if running
}
let control = control.clone(); let control = control.clone();
let idle_kernel = idle_kernel.clone();
let connection = connection.clone(); let connection = connection.clone();
let terminate = terminate.clone(); let terminate = terminate.clone();
let can_restart_idle = can_restart_idle.clone();
let up_destinations = up_destinations.clone(); let up_destinations = up_destinations.clone();
let aux_mutex = aux_mutex.clone();
let routing_table = drtio_routing_table.clone();
// we make sure the value of terminate is 0 before we start // we make sure the value of terminate is 0 before we start
let _ = terminate.try_wait(); let _ = terminate.try_wait();
let _ = can_restart_idle.try_wait();
task::spawn(async move { task::spawn(async move {
let routing_table = routing_table.borrow();
select_biased! { select_biased! {
_ = (async { _ = (async {
if let Some(stream) = &mut maybe_stream { let _ = handle_connection(&mut stream, control.clone(), &up_destinations)
let _ = handle_connection(stream, control.clone(), &up_destinations, &aux_mutex, &routing_table, timer)
.await .await
.map_err(|e| warn!("connection terminated: {}", e)); .map_err(|e| warn!("connection terminated: {}", e));
} if let Some(buffer) = &*idle_kernel {
can_restart_idle.signal(); info!("Loading idle kernel");
match maybe_idle_kernel { let _ = load_kernel(&buffer, &control, None)
Some(buffer) => { .await.map_err(|_| warn!("error loading idle kernel"));
loop { info!("Running idle kernel");
info!("loading idle kernel"); let _ = handle_run_kernel(None, &control, &up_destinations)
match handle_flash_kernel(&buffer, &control, &up_destinations, &aux_mutex, &routing_table, timer).await { .await.map_err(|_| warn!("error running idle kernel"));
Ok(_) => { info!("Idle kernel terminated");
info!("running idle kernel");
match handle_run_kernel(None, &control, &up_destinations, &aux_mutex, &routing_table, timer).await {
Ok(_) => info!("idle kernel finished"),
Err(_) => warn!("idle kernel running error")
}
},
Err(_) => warn!("idle kernel loading error")
}
}
},
None => info!("no idle kernel found")
} }
}).fuse() => (), }).fuse() => (),
_ = terminate.async_wait().fuse() => () _ = terminate.async_wait().fuse() => ()
} }
connection.signal(); connection.signal();
if let Some(stream) = maybe_stream {
let _ = stream.flush().await; let _ = stream.flush().await;
let _ = stream.abort().await; let _ = stream.abort().await;
}
}); });
} }
}); });
Sockets::run(&mut iface, || Instant::from_millis(timer.get_time().0 as i32)); Sockets::run(&mut iface, || {
Instant::from_millis(timer.get_time().0 as i32)
});
} }
pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! { pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
let net_addresses = net_settings::get_addresses(&cfg); let net_addresses = net_settings::get_addresses(&cfg);
info!("network addresses: {}", net_addresses); info!("network addresses: {}", net_addresses);
@ -895,7 +522,7 @@ pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0),
IpCidr::new(addr, 0), IpCidr::new(addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
@ -906,7 +533,7 @@ pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
None => { None => {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
@ -918,8 +545,7 @@ pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
Sockets::init(32); Sockets::init(32);
let dummy = Rc::new(Semaphore::new(0, 1)); mgmt::start(cfg);
mgmt::start(Rc::new(cfg), dummy, None);
// getting eth settings disables the LED as it resets GPIO // getting eth settings disables the LED as it resets GPIO
// need to re-enable it here // need to re-enable it here
@ -929,5 +555,7 @@ pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
err_led.toggle(true); err_led.toggle(true);
} }
Sockets::run(&mut iface, || Instant::from_millis(timer.get_time().0 as i32)); Sockets::run(&mut iface, || {
Instant::from_millis(timer.get_time().0 as i32)
});
} }

View File

@ -13,17 +13,15 @@
#![allow(non_camel_case_types)] #![allow(non_camel_case_types)]
use core::mem; use core::mem;
use cslice::CSlice;
use core_io::Error as ReadError;
use cslice::{AsCSlice, CSlice};
use dwarf::eh::{self, EHAction, EHContext};
use io::{Cursor, ProtoRead};
use libc::{c_int, c_void, uintptr_t};
use log::{error, trace};
use unwind as uw; use unwind as uw;
use libc::{c_int, c_void, uintptr_t};
use log::{trace, error};
use crate::kernel::KERNEL_IMAGE; use crate::kernel::KERNEL_IMAGE;
use dwarf::eh::{self, EHAction, EHContext};
const EXCEPTION_CLASS: uw::_Unwind_Exception_Class = 0x4d_4c_42_53_41_52_54_51; /* 'MLBSARTQ' */ const EXCEPTION_CLASS: uw::_Unwind_Exception_Class = 0x4d_4c_42_53_41_52_54_51; /* 'MLBSARTQ' */
#[cfg(target_arch = "arm")] #[cfg(target_arch = "arm")]
@ -40,7 +38,7 @@ pub struct Exception<'a> {
pub column: u32, pub column: u32,
pub function: CSlice<'a, u8>, pub function: CSlice<'a, u8>,
pub message: CSlice<'a, u8>, pub message: CSlice<'a, u8>,
pub param: [i64; 3], pub param: [i64; 3]
} }
fn str_err(_: core::str::Utf8Error) -> core::fmt::Error { fn str_err(_: core::str::Utf8Error) -> core::fmt::Error {
@ -57,16 +55,12 @@ fn exception_str<'a>(s: &'a CSlice<'a, u8>) -> Result<&'a str, core::str::Utf8Er
impl<'a> core::fmt::Debug for Exception<'a> { impl<'a> core::fmt::Debug for Exception<'a> {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!( write!(f, "Exception {} from {} in {}:{}:{}, message: {}",
f,
"Exception {} from {} in {}:{}:{}, message: {}",
self.id, self.id,
exception_str(&self.function).map_err(str_err)?, exception_str(&self.function).map_err(str_err)?,
exception_str(&self.file).map_err(str_err)?, exception_str(&self.file).map_err(str_err)?,
self.line, self.line, self.column,
self.column, exception_str(&self.message).map_err(str_err)?)
exception_str(&self.message).map_err(str_err)?
)
} }
} }
@ -108,12 +102,12 @@ static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
stack_pointers: [StackPointerBacktrace { stack_pointers: [StackPointerBacktrace {
stack_pointer: 0, stack_pointer: 0,
initial_backtrace_size: 0, initial_backtrace_size: 0,
current_backtrace_size: 0, current_backtrace_size: 0
}; MAX_INFLIGHT_EXCEPTIONS + 1], }; MAX_INFLIGHT_EXCEPTIONS + 1],
exception_count: 0, exception_count: 0
}; };
pub unsafe extern "C" fn reset_exception_buffer() { pub unsafe extern fn reset_exception_buffer() {
trace!("reset exception buffer"); trace!("reset exception buffer");
EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception { EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception {
exception_class: EXCEPTION_CLASS, exception_class: EXCEPTION_CLASS,
@ -126,25 +120,26 @@ pub unsafe extern "C" fn reset_exception_buffer() {
EXCEPTION_BUFFER.exception_count = 0; EXCEPTION_BUFFER.exception_count = 0;
} }
type _Unwind_Stop_Fn = extern "C" fn( type _Unwind_Stop_Fn = extern "C" fn(version: c_int,
version: c_int,
actions: i32, actions: i32,
exception_class: uw::_Unwind_Exception_Class, exception_class: uw::_Unwind_Exception_Class,
exception_object: *mut uw::_Unwind_Exception, exception_object: *mut uw::_Unwind_Exception,
context: *mut uw::_Unwind_Context, context: *mut uw::_Unwind_Context,
stop_parameter: *mut c_void, stop_parameter: *mut c_void)
) -> uw::_Unwind_Reason_Code; -> uw::_Unwind_Reason_Code;
extern "C" { extern {
// not defined in EHABI, but LLVM added it and is useful to us // not defined in EHABI, but LLVM added it and is useful to us
fn _Unwind_ForcedUnwind( fn _Unwind_ForcedUnwind(exception: *mut uw::_Unwind_Exception,
exception: *mut uw::_Unwind_Exception,
stop_fn: _Unwind_Stop_Fn, stop_fn: _Unwind_Stop_Fn,
stop_parameter: *mut c_void, stop_parameter: *mut c_void) -> uw::_Unwind_Reason_Code;
) -> uw::_Unwind_Reason_Code;
} }
unsafe fn find_eh_action(context: *mut uw::_Unwind_Context, foreign_exception: bool, id: u32) -> Result<EHAction, ()> { unsafe fn find_eh_action(
context: *mut uw::_Unwind_Context,
foreign_exception: bool,
id: u32,
) -> Result<EHAction, ()> {
let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8; let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8;
let mut ip_before_instr: c_int = 0; let mut ip_before_instr: c_int = 0;
let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr); let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr);
@ -159,11 +154,10 @@ unsafe fn find_eh_action(context: *mut uw::_Unwind_Context, foreign_exception: b
eh::find_eh_action(lsda, &eh_context, foreign_exception, id) eh::find_eh_action(lsda, &eh_context, foreign_exception, id)
} }
pub unsafe fn artiq_personality( pub unsafe fn artiq_personality(_state: uw::_Unwind_State,
_state: uw::_Unwind_State,
exception_object: *mut uw::_Unwind_Exception, exception_object: *mut uw::_Unwind_Exception,
context: *mut uw::_Unwind_Context, context: *mut uw::_Unwind_Context)
) -> uw::_Unwind_Reason_Code { -> uw::_Unwind_Reason_Code {
// we will only do phase 2 forced unwinding now // we will only do phase 2 forced unwinding now
// The DWARF unwinder assumes that _Unwind_Context holds things like the function // The DWARF unwinder assumes that _Unwind_Context holds things like the function
// and LSDA pointers, however ARM EHABI places them into the exception object. // and LSDA pointers, however ARM EHABI places them into the exception object.
@ -171,7 +165,9 @@ pub unsafe fn artiq_personality(
// take only the context pointer, GCC personality routines stash a pointer to // take only the context pointer, GCC personality routines stash a pointer to
// exception_object in the context, using location reserved for ARM's // exception_object in the context, using location reserved for ARM's
// "scratch register" (r12). // "scratch register" (r12).
uw::_Unwind_SetGR(context, uw::UNWIND_POINTER_REG, exception_object as uw::_Unwind_Ptr); uw::_Unwind_SetGR(context,
uw::UNWIND_POINTER_REG,
exception_object as uw::_Unwind_Ptr);
// ...A more principled approach would be to provide the full definition of ARM's // ...A more principled approach would be to provide the full definition of ARM's
// _Unwind_Context in our libunwind bindings and fetch the required data from there // _Unwind_Context in our libunwind bindings and fetch the required data from there
// directly, bypassing DWARF compatibility functions. // directly, bypassing DWARF compatibility functions.
@ -190,8 +186,10 @@ pub unsafe fn artiq_personality(
}; };
match eh_action { match eh_action {
EHAction::None => return continue_unwind(exception_object, context), EHAction::None => return continue_unwind(exception_object, context),
EHAction::Cleanup(lpad) | EHAction::Catch(lpad) => { EHAction::Cleanup(lpad) |
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.0, exception_object as uintptr_t); EHAction::Catch(lpad) => {
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.0,
exception_object as uintptr_t);
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.1, exception as *const _ as uw::_Unwind_Word); uw::_Unwind_SetGR(context, UNWIND_DATA_REG.1, exception as *const _ as uw::_Unwind_Word);
uw::_Unwind_SetIP(context, lpad); uw::_Unwind_SetIP(context, lpad);
return uw::_URC_INSTALL_CONTEXT; return uw::_URC_INSTALL_CONTEXT;
@ -201,10 +199,9 @@ pub unsafe fn artiq_personality(
// On ARM EHABI the personality routine is responsible for actually // On ARM EHABI the personality routine is responsible for actually
// unwinding a single stack frame before returning (ARM EHABI Sec. 6.1). // unwinding a single stack frame before returning (ARM EHABI Sec. 6.1).
unsafe fn continue_unwind( unsafe fn continue_unwind(exception_object: *mut uw::_Unwind_Exception,
exception_object: *mut uw::_Unwind_Exception, context: *mut uw::_Unwind_Context)
context: *mut uw::_Unwind_Context, -> uw::_Unwind_Reason_Code {
) -> uw::_Unwind_Reason_Code {
let reason = __gnu_unwind_frame(exception_object, context); let reason = __gnu_unwind_frame(exception_object, context);
if reason == uw::_URC_NO_REASON { if reason == uw::_URC_NO_REASON {
uw::_URC_CONTINUE_UNWIND uw::_URC_CONTINUE_UNWIND
@ -214,14 +211,15 @@ pub unsafe fn artiq_personality(
} }
// defined in libgcc // defined in libgcc
extern "C" { extern "C" {
fn __gnu_unwind_frame( fn __gnu_unwind_frame(exception_object: *mut uw::_Unwind_Exception,
exception_object: *mut uw::_Unwind_Exception, context: *mut uw::_Unwind_Context)
context: *mut uw::_Unwind_Context, -> uw::_Unwind_Reason_Code;
) -> uw::_Unwind_Reason_Code;
} }
} }
pub unsafe extern "C" fn raise(exception: *const Exception) -> ! { pub unsafe extern fn raise(exception: *const Exception) -> ! {
use cslice::AsCSlice;
let count = EXCEPTION_BUFFER.exception_count; let count = EXCEPTION_BUFFER.exception_count;
let stack = &mut EXCEPTION_BUFFER.exception_stack; let stack = &mut EXCEPTION_BUFFER.exception_stack;
let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize; let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize;
@ -246,11 +244,8 @@ pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
} }
} }
assert!(found); assert!(found);
let _result = _Unwind_ForcedUnwind( let _result = _Unwind_ForcedUnwind(&mut EXCEPTION_BUFFER.uw_exceptions[stack[count - 1] as usize],
&mut EXCEPTION_BUFFER.uw_exceptions[stack[count - 1] as usize], stop_fn, core::ptr::null_mut());
stop_fn,
core::ptr::null_mut(),
);
} else { } else {
if count < MAX_INFLIGHT_EXCEPTIONS { if count < MAX_INFLIGHT_EXCEPTIONS {
trace!("raising exception at level {}", count); trace!("raising exception at level {}", count);
@ -258,19 +253,20 @@ pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
for (i, slot) in EXCEPTION_BUFFER.exceptions.iter_mut().enumerate() { for (i, slot) in EXCEPTION_BUFFER.exceptions.iter_mut().enumerate() {
// we should always be able to find a slot // we should always be able to find a slot
if slot.is_none() { if slot.is_none() {
*slot = Some(*mem::transmute::<*const Exception, *const Exception<'static>>( *slot = Some(
exception, *mem::transmute::<*const Exception, *const Exception<'static>>
)); (exception));
EXCEPTION_BUFFER.exception_stack[count] = i as isize; EXCEPTION_BUFFER.exception_stack[count] = i as isize;
EXCEPTION_BUFFER.uw_exceptions[i].private = [0; uw::unwinder_private_data_size]; EXCEPTION_BUFFER.uw_exceptions[i].private =
[0; uw::unwinder_private_data_size];
EXCEPTION_BUFFER.stack_pointers[i] = StackPointerBacktrace { EXCEPTION_BUFFER.stack_pointers[i] = StackPointerBacktrace {
stack_pointer: 0, stack_pointer: 0,
initial_backtrace_size: EXCEPTION_BUFFER.backtrace_size, initial_backtrace_size: EXCEPTION_BUFFER.backtrace_size,
current_backtrace_size: 0, current_backtrace_size: 0,
}; };
EXCEPTION_BUFFER.exception_count += 1; EXCEPTION_BUFFER.exception_count += 1;
let _result = let _result = _Unwind_ForcedUnwind(&mut EXCEPTION_BUFFER.uw_exceptions[i],
_Unwind_ForcedUnwind(&mut EXCEPTION_BUFFER.uw_exceptions[i], stop_fn, core::ptr::null_mut()); stop_fn, core::ptr::null_mut());
} }
} }
} else { } else {
@ -284,7 +280,7 @@ pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
// https://github.com/rust-lang/rfcs/pull/1719 // https://github.com/rust-lang/rfcs/pull/1719
function: "__artiq_raise".as_c_slice(), function: "__artiq_raise".as_c_slice(),
message: "too many nested exceptions".as_c_slice(), message: "too many nested exceptions".as_c_slice(),
param: [0, 0, 0], param: [0, 0, 0]
}; };
EXCEPTION_BUFFER.exceptions[MAX_INFLIGHT_EXCEPTIONS] = Some(mem::transmute(exception)); EXCEPTION_BUFFER.exceptions[MAX_INFLIGHT_EXCEPTIONS] = Some(mem::transmute(exception));
EXCEPTION_BUFFER.stack_pointers[MAX_INFLIGHT_EXCEPTIONS] = Default::default(); EXCEPTION_BUFFER.stack_pointers[MAX_INFLIGHT_EXCEPTIONS] = Default::default();
@ -295,74 +291,17 @@ pub unsafe extern "C" fn raise(exception: *const Exception) -> ! {
unreachable!(); unreachable!();
} }
fn read_exception_string<'a>(reader: &mut Cursor<&[u8]>) -> Result<CSlice<'a, u8>, ReadError> { pub unsafe extern fn resume() -> ! {
let len = reader.read_u32()? as usize;
if len == usize::MAX {
let data = reader.read_u32()?;
Ok(unsafe { CSlice::new(data as *const u8, len) })
} else {
let pos = reader.position();
let slice = unsafe {
let ptr = reader.get_ref().as_ptr().offset(pos as isize);
CSlice::new(ptr, len)
};
reader.set_position(pos + len);
Ok(slice)
}
}
fn read_exception(raw_exception: &[u8]) -> Result<Exception, ReadError> {
let mut reader = Cursor::new(raw_exception);
let mut byte = reader.read_u8()?;
// to sync
while byte != 0x5a {
byte = reader.read_u8()?;
}
// skip sync bytes, 0x09 indicates exception
while byte != 0x09 {
byte = reader.read_u8()?;
}
let _len = reader.read_u32()?;
// ignore the remaining exceptions, stack traces etc. - unwinding from another device would be unwise anyway
Ok(Exception {
id: reader.read_u32()?,
message: read_exception_string(&mut reader)?,
param: [
reader.read_u64()? as i64,
reader.read_u64()? as i64,
reader.read_u64()? as i64,
],
file: read_exception_string(&mut reader)?,
line: reader.read_u32()?,
column: reader.read_u32()?,
function: read_exception_string(&mut reader)?,
})
}
pub fn raise_raw(raw_exception: &[u8]) -> ! {
use crate::artiq_raise;
if let Ok(exception) = read_exception(raw_exception) {
unsafe { raise(&exception) };
} else {
artiq_raise!("SubkernelError", "Error passing exception");
}
}
pub unsafe extern "C" fn resume() -> ! {
trace!("resume"); trace!("resume");
assert!(EXCEPTION_BUFFER.exception_count != 0); assert!(EXCEPTION_BUFFER.exception_count != 0);
let i = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1]; let i = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
assert!(i != -1); assert!(i != -1);
let _result = _Unwind_ForcedUnwind( let _result = _Unwind_ForcedUnwind(&mut EXCEPTION_BUFFER.uw_exceptions[i as usize],
&mut EXCEPTION_BUFFER.uw_exceptions[i as usize], stop_fn, core::ptr::null_mut());
stop_fn,
core::ptr::null_mut(),
);
unreachable!() unreachable!()
} }
pub unsafe extern "C" fn end_catch() { pub unsafe extern fn end_catch() {
let mut count = EXCEPTION_BUFFER.exception_count; let mut count = EXCEPTION_BUFFER.exception_count;
assert!(count != 0); assert!(count != 0);
// we remove all exceptions with SP <= current exception SP // we remove all exceptions with SP <= current exception SP
@ -370,7 +309,8 @@ pub unsafe extern "C" fn end_catch() {
let index = EXCEPTION_BUFFER.exception_stack[count - 1] as usize; let index = EXCEPTION_BUFFER.exception_stack[count - 1] as usize;
EXCEPTION_BUFFER.exception_stack[count - 1] = -1; EXCEPTION_BUFFER.exception_stack[count - 1] = -1;
EXCEPTION_BUFFER.exceptions[index] = None; EXCEPTION_BUFFER.exceptions[index] = None;
let outer_sp = EXCEPTION_BUFFER.stack_pointers[index].stack_pointer; let outer_sp = EXCEPTION_BUFFER.stack_pointers
[index].stack_pointer;
count -= 1; count -= 1;
for i in (0..count).rev() { for i in (0..count).rev() {
let index = EXCEPTION_BUFFER.exception_stack[i]; let index = EXCEPTION_BUFFER.exception_stack[i];
@ -394,7 +334,8 @@ pub unsafe extern "C" fn end_catch() {
}; };
} }
extern "C" fn cleanup(_unwind_code: uw::_Unwind_Reason_Code, _uw_exception: *mut uw::_Unwind_Exception) { extern fn cleanup(_unwind_code: uw::_Unwind_Reason_Code,
_uw_exception: *mut uw::_Unwind_Exception) {
unimplemented!() unimplemented!()
} }
@ -404,26 +345,18 @@ fn uncaught_exception() -> ! {
for i in 0..EXCEPTION_BUFFER.exception_count { for i in 0..EXCEPTION_BUFFER.exception_count {
if EXCEPTION_BUFFER.exception_stack[i] != i as isize { if EXCEPTION_BUFFER.exception_stack[i] != i as isize {
// find the correct index // find the correct index
let index = EXCEPTION_BUFFER let index = EXCEPTION_BUFFER.exception_stack
.exception_stack
.iter() .iter()
.position(|v| *v == i as isize) .position(|v| *v == i as isize).unwrap();
.unwrap();
let a = EXCEPTION_BUFFER.exception_stack[index]; let a = EXCEPTION_BUFFER.exception_stack[index];
let b = EXCEPTION_BUFFER.exception_stack[i]; let b = EXCEPTION_BUFFER.exception_stack[i];
assert!(a != -1 && b != -1); assert!(a != -1 && b != -1);
core::mem::swap( core::mem::swap(&mut EXCEPTION_BUFFER.exception_stack[index],
&mut EXCEPTION_BUFFER.exception_stack[index], &mut EXCEPTION_BUFFER.exception_stack[i]);
&mut EXCEPTION_BUFFER.exception_stack[i], core::mem::swap(&mut EXCEPTION_BUFFER.exceptions[a as usize],
); &mut EXCEPTION_BUFFER.exceptions[b as usize]);
core::mem::swap( core::mem::swap(&mut EXCEPTION_BUFFER.stack_pointers[a as usize],
&mut EXCEPTION_BUFFER.exceptions[a as usize], &mut EXCEPTION_BUFFER.stack_pointers[b as usize]);
&mut EXCEPTION_BUFFER.exceptions[b as usize],
);
core::mem::swap(
&mut EXCEPTION_BUFFER.stack_pointers[a as usize],
&mut EXCEPTION_BUFFER.stack_pointers[b as usize],
);
} }
} }
} }
@ -431,20 +364,17 @@ fn uncaught_exception() -> ! {
crate::kernel::core1::terminate( crate::kernel::core1::terminate(
EXCEPTION_BUFFER.exceptions[..EXCEPTION_BUFFER.exception_count].as_ref(), EXCEPTION_BUFFER.exceptions[..EXCEPTION_BUFFER.exception_count].as_ref(),
EXCEPTION_BUFFER.stack_pointers[..EXCEPTION_BUFFER.exception_count].as_ref(), EXCEPTION_BUFFER.stack_pointers[..EXCEPTION_BUFFER.exception_count].as_ref(),
EXCEPTION_BUFFER.backtrace[..EXCEPTION_BUFFER.backtrace_size].as_mut(), EXCEPTION_BUFFER.backtrace[..EXCEPTION_BUFFER.backtrace_size].as_mut())
)
} }
} }
// stop function which would be executed when we unwind each frame // stop function which would be executed when we unwind each frame
extern "C" fn stop_fn( extern fn stop_fn(_version: c_int,
_version: c_int,
actions: i32, actions: i32,
_uw_exception_class: uw::_Unwind_Exception_Class, _uw_exception_class: uw::_Unwind_Exception_Class,
_uw_exception: *mut uw::_Unwind_Exception, _uw_exception: *mut uw::_Unwind_Exception,
context: *mut uw::_Unwind_Context, context: *mut uw::_Unwind_Context,
_stop_parameter: *mut c_void, _stop_parameter: *mut c_void) -> uw::_Unwind_Reason_Code {
) -> uw::_Unwind_Reason_Code {
unsafe { unsafe {
let load_addr = KERNEL_IMAGE.as_ref().unwrap().get_load_addr(); let load_addr = KERNEL_IMAGE.as_ref().unwrap().get_load_addr();
let backtrace_size = EXCEPTION_BUFFER.backtrace_size; let backtrace_size = EXCEPTION_BUFFER.backtrace_size;
@ -475,36 +405,25 @@ extern "C" fn stop_fn(
} }
} }
// Must be kept in sync with preallocate_runtime_exception_names() in `artiq.compiler.embedding` // Must be kept in sync with preallocate_runtime_exception_names() in artiq/language/embedding_map.py
static EXCEPTION_ID_LOOKUP: [(&str, u32); 22] = [ static EXCEPTION_ID_LOOKUP: [(&str, u32); 11] = [
("RTIOUnderflow", 0), ("RuntimeError", 0),
("RTIOOverflow", 1), ("RTIOUnderflow", 1),
("RTIODestinationUnreachable", 2), ("RTIOOverflow", 2),
("DMAError", 3), ("RTIODestinationUnreachable", 3),
("I2CError", 4), ("DMAError", 4),
("CacheError", 5), ("I2CError", 5),
("SPIError", 6), ("CacheError", 6),
("SubkernelError", 7), ("SPIError", 7),
("AssertionError", 8), ("ZeroDivisionError", 8),
("AttributeError", 9), ("IndexError", 9),
("IndexError", 10), ("UnwrapNoneError", 10),
("IOError", 11),
("KeyError", 12),
("NotImplementedError", 13),
("OverflowError", 14),
("RuntimeError", 15),
("TimeoutError", 16),
("TypeError", 17),
("ValueError", 18),
("ZeroDivisionError", 19),
("LinAlgError", 20),
("UnwrapNoneError", 21),
]; ];
pub fn get_exception_id(name: &str) -> u32 { pub fn get_exception_id(name: &str) -> u32 {
for (n, id) in EXCEPTION_ID_LOOKUP.iter() { for (n, id) in EXCEPTION_ID_LOOKUP.iter() {
if *n == name { if *n == name {
return *id; return *id
} }
} }
unimplemented!("unallocated internal exception id") unimplemented!("unallocated internal exception id")
@ -512,7 +431,7 @@ pub fn get_exception_id(name: &str) -> u32 {
#[macro_export] #[macro_export]
macro_rules! artiq_raise { macro_rules! artiq_raise {
($name:expr, $message:expr, $param0:expr, $param1:expr, $param2:expr) => {{ ($name:expr, $message:expr, $param0:expr, $param1:expr, $param2:expr) => ({
use cslice::AsCSlice; use cslice::AsCSlice;
let name_id = $crate::eh_artiq::get_exception_id($name); let name_id = $crate::eh_artiq::get_exception_id($name);
let message_cl = $message.clone(); let message_cl = $message.clone();
@ -524,38 +443,14 @@ macro_rules! artiq_raise {
// https://github.com/rust-lang/rfcs/pull/1719 // https://github.com/rust-lang/rfcs/pull/1719
function: "(Rust function)".as_c_slice(), function: "(Rust function)".as_c_slice(),
message: message_cl.as_c_slice(), message: message_cl.as_c_slice(),
param: [$param0, $param1, $param2], param: [$param0, $param1, $param2]
}; };
#[allow(unused_unsafe)] #[allow(unused_unsafe)]
unsafe { unsafe {
$crate::eh_artiq::raise(&exn) $crate::eh_artiq::raise(&exn)
} }
}}; });
($name:expr, $message:expr) => {{ artiq_raise!($name, $message, 0, 0, 0) }}; ($name:expr, $message:expr) => ({
} artiq_raise!($name, $message, 0, 0, 0)
});
/// Takes as input exception id from host
/// Generates a new exception with:
/// * `id` set to `exn_id`
/// * `message` set to corresponding exception name from `EXCEPTION_ID_LOOKUP`
///
/// The message is matched on host to ensure correct exception is being referred
/// This test checks the synchronization of exception ids for runtime errors
#[no_mangle]
pub extern "C" fn test_exception_id_sync(exn_id: u32) {
let message = EXCEPTION_ID_LOOKUP
.iter()
.find_map(|&(name, id)| if id == exn_id { Some(name) } else { None })
.unwrap_or("unallocated internal exception id");
let exn = Exception {
id: exn_id,
file: file!().as_c_slice(),
line: 0,
column: 0,
function: "test_exception_id_sync".as_c_slice(),
message: message.as_c_slice(),
param: [0, 0, 0],
};
unsafe { raise(&exn) };
} }

View File

@ -1,10 +1,9 @@
use libboard_zynq; use libboard_zynq;
use crate::artiq_raise; use crate::artiq_raise;
pub static mut I2C_BUS: Option<libboard_zynq::i2c::I2c> = None; pub static mut I2C_BUS: Option<libboard_zynq::i2c::I2c> = None;
pub extern "C" fn start(busno: i32) { pub extern fn start(busno: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
@ -15,7 +14,7 @@ pub extern "C" fn start(busno: i32) {
} }
} }
pub extern "C" fn restart(busno: i32) { pub extern fn restart(busno: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
@ -26,7 +25,7 @@ pub extern "C" fn restart(busno: i32) {
} }
} }
pub extern "C" fn stop(busno: i32) { pub extern fn stop(busno: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
@ -37,7 +36,7 @@ pub extern "C" fn stop(busno: i32) {
} }
} }
pub extern "C" fn write(busno: i32, data: i32) -> bool { pub extern fn write(busno: i32, data: i32) -> bool {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
@ -49,7 +48,7 @@ pub extern "C" fn write(busno: i32, data: i32) -> bool {
} }
} }
pub extern "C" fn read(busno: i32, ack: bool) -> i32 { pub extern fn read(busno: i32, ack: bool) -> i32 {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
@ -61,12 +60,11 @@ pub extern "C" fn read(busno: i32, ack: bool) -> i32 {
} }
} }
pub extern "C" fn switch_select(busno: i32, address: i32, mask: i32) { pub extern fn switch_select(busno: i32, address: i32, mask: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
let ch = match mask { let ch = match mask { //decode from mainline, PCA9548-centric API
//decode from mainline, PCA9548-centric API
0x00 => None, 0x00 => None,
0x01 => Some(0), 0x01 => Some(0),
0x02 => Some(1), 0x02 => Some(1),
@ -76,15 +74,10 @@ pub extern "C" fn switch_select(busno: i32, address: i32, mask: i32) {
0x20 => Some(5), 0x20 => Some(5),
0x40 => Some(6), 0x40 => Some(6),
0x80 => Some(7), 0x80 => Some(7),
_ => artiq_raise!("I2CError", "switch select supports only one channel"), _ => artiq_raise!("I2CError", "switch select supports only one channel")
}; };
unsafe { unsafe {
if (&mut I2C_BUS) if (&mut I2C_BUS).as_mut().unwrap().pca954x_select(address as u8, ch).is_err() {
.as_mut()
.unwrap()
.pca954x_select(address as u8, ch)
.is_err()
{
artiq_raise!("I2CError", "switch select failed"); artiq_raise!("I2CError", "switch select failed");
} }
} }

View File

@ -1,8 +1,11 @@
use core::sync::atomic::{AtomicBool, Ordering};
use libboard_zynq::{gic, mpcore, println, stdio}; use libboard_zynq::{gic, mpcore, println, stdio};
use libcortex_a9::{asm, interrupt_handler, notify_spin_lock, regs::MPIDR, spin_lock_yield}; use libcortex_a9::{
asm, interrupt_handler,
regs::MPIDR,
spin_lock_yield, notify_spin_lock
};
use libregister::RegisterR; use libregister::RegisterR;
use core::sync::atomic::{AtomicBool, Ordering};
extern "C" { extern "C" {
static mut __stack1_start: u32; static mut __stack1_start: u32;

View File

@ -1,27 +1,29 @@
use alloc::vec; use core::ffi::VaList;
use core::{ffi::VaList, ptr, str}; use core::ptr;
use core::str;
use libc::{c_char, c_int, size_t}; use libc::{c_char, c_int, size_t};
use libm; use libm;
use log::{info, warn}; use log::{info, warn};
#[cfg(has_drtio)] use alloc::vec;
use super::subkernel;
use super::{cache, use crate::eh_artiq;
core1::rtio_get_destination_status, use crate::rtio;
dma, linalg, use crate::i2c;
rpc::{rpc_recv, rpc_send, rpc_send_async}}; use super::rpc::{rpc_send, rpc_send_async, rpc_recv};
use crate::{eh_artiq, i2c, rtio}; use super::dma;
use super::cache;
use super::core1::rtio_get_destination_status;
extern "C" { extern "C" {
fn vsnprintf_(buffer: *mut c_char, count: size_t, format: *const c_char, va: VaList) -> c_int; fn vsnprintf_(buffer: *mut c_char, count: size_t, format: *const c_char, va: VaList) -> c_int;
} }
unsafe extern "C" fn core_log(fmt: *const c_char, mut args: ...) { unsafe extern fn core_log(fmt: *const c_char, mut args: ...) {
let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize; let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize;
let mut buf = vec![0; size + 1]; let mut buf = vec![0; size + 1];
vsnprintf_(buf.as_mut_ptr() as *mut i8, size + 1, fmt, args.as_va_list()); vsnprintf_(buf.as_mut_ptr() as *mut i8, size + 1, fmt, args.as_va_list());
let buf: &[u8] = &buf.as_slice()[..size - 1]; // strip \n and NUL let buf: &[u8] = &buf.as_slice()[..size-1]; // strip \n and NUL
match str::from_utf8(buf) { match str::from_utf8(buf) {
Ok(s) => info!("kernel: {}", s), Ok(s) => info!("kernel: {}", s),
Err(e) => { Err(e) => {
@ -31,13 +33,14 @@ unsafe extern "C" fn core_log(fmt: *const c_char, mut args: ...) {
} }
} }
unsafe extern "C" fn rtio_log(fmt: *const c_char, mut args: ...) { unsafe extern fn rtio_log(fmt: *const c_char, mut args: ...) {
let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize; let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize;
let mut buf = vec![0; size + 1]; let mut buf = vec![0; size + 1];
vsnprintf_(buf.as_mut_ptr(), size + 1, fmt, args.as_va_list()); vsnprintf_(buf.as_mut_ptr(), size + 1, fmt, args.as_va_list());
rtio::write_log(buf.as_slice()); rtio::write_log(buf.as_slice());
} }
macro_rules! api { macro_rules! api {
($i:ident) => ({ ($i:ident) => ({
extern { static $i: u8; } extern { static $i: u8; }
@ -53,25 +56,24 @@ macro_rules! api {
} }
macro_rules! api_libm_f64f64 { macro_rules! api_libm_f64f64 {
($i:ident) => {{ ($i:ident) => ({
extern "C" fn $i(x: f64) -> f64 { extern fn $i(x: f64) -> f64 {
libm::$i(x) libm::$i(x)
} }
api!($i = $i) api!($i = $i)
}}; })
} }
macro_rules! api_libm_f64f64f64 { macro_rules! api_libm_f64f64f64 {
($i:ident) => {{ ($i:ident) => ({
extern "C" fn $i(x: f64, y: f64) -> f64 { extern fn $i(x: f64, y: f64) -> f64 {
libm::$i(x, y) libm::$i(x, y)
} }
api!($i = $i) api!($i = $i)
}}; })
} }
pub fn resolve(required: &[u8]) -> Option<u32> { pub fn resolve(required: &[u8]) -> Option<u32> {
#[rustfmt::skip]
let api = &[ let api = &[
// timing // timing
api!(now_mu = rtio::now_mu), api!(now_mu = rtio::now_mu),
@ -116,23 +118,12 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(i2c_read = i2c::read), api!(i2c_read = i2c::read),
api!(i2c_switch_select = i2c::switch_select), api!(i2c_switch_select = i2c::switch_select),
// subkernel
#[cfg(has_drtio)]
api!(subkernel_load_run = subkernel::load_run),
#[cfg(has_drtio)]
api!(subkernel_await_finish = subkernel::await_finish),
#[cfg(has_drtio)]
api!(subkernel_send_message = subkernel::send_message),
#[cfg(has_drtio)]
api!(subkernel_await_message = subkernel::await_message),
// Double-precision floating-point arithmetic helper functions // Double-precision floating-point arithmetic helper functions
// RTABI chapter 4.1.2, Table 2 // RTABI chapter 4.1.2, Table 2
api!(__aeabi_dadd), api!(__aeabi_dadd),
api!(__aeabi_ddiv), api!(__aeabi_ddiv),
api!(__aeabi_dmul), api!(__aeabi_dmul),
api!(__aeabi_dsub), api!(__aeabi_dsub),
// Double-precision floating-point comparison helper functions // Double-precision floating-point comparison helper functions
// RTABI chapter 4.1.2, Table 3 // RTABI chapter 4.1.2, Table 3
api!(__aeabi_dcmpeq), api!(__aeabi_dcmpeq),
@ -142,14 +133,12 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_dcmpge), api!(__aeabi_dcmpge),
api!(__aeabi_dcmpgt), api!(__aeabi_dcmpgt),
api!(__aeabi_dcmpun), api!(__aeabi_dcmpun),
// Single-precision floating-point arithmetic helper functions // Single-precision floating-point arithmetic helper functions
// RTABI chapter 4.1.2, Table 4 // RTABI chapter 4.1.2, Table 4
api!(__aeabi_fadd), api!(__aeabi_fadd),
api!(__aeabi_fdiv), api!(__aeabi_fdiv),
api!(__aeabi_fmul), api!(__aeabi_fmul),
api!(__aeabi_fsub), api!(__aeabi_fsub),
// Single-precision floating-point comparison helper functions // Single-precision floating-point comparison helper functions
// RTABI chapter 4.1.2, Table 5 // RTABI chapter 4.1.2, Table 5
api!(__aeabi_fcmpeq), api!(__aeabi_fcmpeq),
@ -159,7 +148,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_fcmpge), api!(__aeabi_fcmpge),
api!(__aeabi_fcmpgt), api!(__aeabi_fcmpgt),
api!(__aeabi_fcmpun), api!(__aeabi_fcmpun),
// Floating-point to integer conversions. // Floating-point to integer conversions.
// RTABI chapter 4.1.2, Table 6 // RTABI chapter 4.1.2, Table 6
api!(__aeabi_d2iz), api!(__aeabi_d2iz),
@ -170,11 +158,9 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_f2uiz), api!(__aeabi_f2uiz),
api!(__aeabi_f2lz), api!(__aeabi_f2lz),
api!(__aeabi_f2ulz), api!(__aeabi_f2ulz),
// Conversions between floating types. // Conversions between floating types.
// RTABI chapter 4.1.2, Table 7 // RTABI chapter 4.1.2, Table 7
api!(__aeabi_f2d), api!(__aeabi_f2d),
// Integer to floating-point conversions. // Integer to floating-point conversions.
// RTABI chapter 4.1.2, Table 8 // RTABI chapter 4.1.2, Table 8
api!(__aeabi_i2d), api!(__aeabi_i2d),
@ -185,14 +171,12 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_ui2f), api!(__aeabi_ui2f),
api!(__aeabi_l2f), api!(__aeabi_l2f),
api!(__aeabi_ul2f), api!(__aeabi_ul2f),
// Long long helper functions // Long long helper functions
// RTABI chapter 4.2, Table 9 // RTABI chapter 4.2, Table 9
api!(__aeabi_lmul), api!(__aeabi_lmul),
api!(__aeabi_llsl), api!(__aeabi_llsl),
api!(__aeabi_llsr), api!(__aeabi_llsr),
api!(__aeabi_lasr), api!(__aeabi_lasr),
// Integer division functions // Integer division functions
// RTABI chapter 4.3.1 // RTABI chapter 4.3.1
api!(__aeabi_idiv), api!(__aeabi_idiv),
@ -200,7 +184,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_idivmod), api!(__aeabi_idivmod),
api!(__aeabi_uidiv), api!(__aeabi_uidiv),
api!(__aeabi_uldivmod), api!(__aeabi_uldivmod),
// 4.3.4 Memory copying, clearing, and setting // 4.3.4 Memory copying, clearing, and setting
api!(__aeabi_memcpy8), api!(__aeabi_memcpy8),
api!(__aeabi_memcpy4), api!(__aeabi_memcpy4),
@ -216,30 +199,10 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_memclr), api!(__aeabi_memclr),
// libc // libc
api!( api!(memcpy, extern { fn memcpy(dest: *mut u8, src: *const u8, n: usize) -> *mut u8; }),
memcpy, api!(memmove, extern { fn memmove(dest: *mut u8, src: *const u8, n: usize) -> *mut u8; }),
extern "C" { api!(memset, extern { fn memset(s: *mut u8, c: i32, n: usize) -> *mut u8; }),
fn memcpy(dest: *mut u8, src: *const u8, n: usize) -> *mut u8; api!(memcmp, extern { fn memcmp(s1: *const u8, s2: *const u8, n: usize) -> i32; }),
}
),
api!(
memmove,
extern "C" {
fn memmove(dest: *mut u8, src: *const u8, n: usize) -> *mut u8;
}
),
api!(
memset,
extern "C" {
fn memset(s: *mut u8, c: i32, n: usize) -> *mut u8;
}
),
api!(
memcmp,
extern "C" {
fn memcmp(s1: *const u8, s2: *const u8, n: usize) -> i32;
}
),
// exceptions // exceptions
api!(_Unwind_Resume = unwind::_Unwind_Resume), api!(_Unwind_Resume = unwind::_Unwind_Resume),
@ -247,7 +210,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__nac3_raise = eh_artiq::raise), api!(__nac3_raise = eh_artiq::raise),
api!(__nac3_resume = eh_artiq::resume), api!(__nac3_resume = eh_artiq::resume),
api!(__nac3_end_catch = eh_artiq::end_catch), api!(__nac3_end_catch = eh_artiq::end_catch),
// legacy exception symbols // legacy exception symbols
api!(__artiq_personality = eh_artiq::artiq_personality), api!(__artiq_personality = eh_artiq::artiq_personality),
api!(__artiq_raise = eh_artiq::raise), api!(__artiq_raise = eh_artiq::raise),
@ -279,7 +241,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(fabs), api_libm_f64f64!(fabs),
api_libm_f64f64!(floor), api_libm_f64f64!(floor),
{ {
extern "C" fn fma(x: f64, y: f64, z: f64) -> f64 { extern fn fma(x: f64, y: f64, z: f64) -> f64 {
libm::fma(x, y, z) libm::fma(x, y, z)
} }
api!(fma = fma) api!(fma = fma)
@ -291,7 +253,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(j0), api_libm_f64f64!(j0),
api_libm_f64f64!(j1), api_libm_f64f64!(j1),
{ {
extern "C" fn jn(n: i32, x: f64) -> f64 { extern fn jn(n: i32, x: f64) -> f64 {
libm::jn(n, x) libm::jn(n, x)
} }
api!(jn = jn) api!(jn = jn)
@ -303,7 +265,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64f64!(nextafter), api_libm_f64f64f64!(nextafter),
api_libm_f64f64f64!(pow), api_libm_f64f64f64!(pow),
api_libm_f64f64!(round), api_libm_f64f64!(round),
api_libm_f64f64!(rint),
api_libm_f64f64!(sin), api_libm_f64f64!(sin),
api_libm_f64f64!(sinh), api_libm_f64f64!(sinh),
api_libm_f64f64!(sqrt), api_libm_f64f64!(sqrt),
@ -314,31 +275,11 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(y0), api_libm_f64f64!(y0),
api_libm_f64f64!(y1), api_libm_f64f64!(y1),
{ {
extern "C" fn yn(n: i32, x: f64) -> f64 { extern fn yn(n: i32, x: f64) -> f64 {
libm::yn(n, x) libm::yn(n, x)
} }
api!(yn = yn) api!(yn = yn)
}, },
// linalg
api!(np_linalg_cholesky = linalg::np_linalg_cholesky),
api!(np_linalg_qr = linalg::np_linalg_qr),
api!(np_linalg_svd = linalg::np_linalg_svd),
api!(np_linalg_inv = linalg::np_linalg_inv),
api!(np_linalg_pinv = linalg::np_linalg_pinv),
api!(np_linalg_matrix_power = linalg::np_linalg_matrix_power),
api!(np_linalg_det = linalg::np_linalg_det),
api!(sp_linalg_lu = linalg::sp_linalg_lu),
api!(sp_linalg_schur = linalg::sp_linalg_schur),
api!(sp_linalg_hessenberg = linalg::sp_linalg_hessenberg),
/*
* syscall for unit tests
* Used in `artiq.tests.coredevice.test_exceptions.ExceptionTest.test_raise_exceptions_kernel`
* This syscall checks that the exception IDs used in the Python `EmbeddingMap` (in `artiq.language.embedding`)
* match the `EXCEPTION_ID_LOOKUP` defined in the firmware (`libksupport::src::eh_artiq`)
*/
api!(test_exception_id_sync = eh_artiq::test_exception_id_sync)
]; ];
api.iter() api.iter()
.find(|&&(exported, _)| exported.as_bytes() == required) .find(|&&(exported, _)| exported.as_bytes() == required)

View File

@ -1,17 +1,12 @@
use alloc::{boxed::Box, string::String}; use alloc::{string::String, boxed::Box};
use core::mem::{forget, transmute}; use cslice::{CSlice, AsCSlice};
use core::mem::{transmute, forget};
use super::{KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, Message};
use cslice::{AsCSlice, CSlice}; pub extern fn get(key: CSlice<u8>) -> &CSlice<'static, i32> {
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0};
pub extern "C" fn get(key: CSlice<u8>) -> &CSlice<'static, i32> {
let key = String::from_utf8(key.as_ref().to_vec()).unwrap(); let key = String::from_utf8(key.as_ref().to_vec()).unwrap();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::CacheGetRequest(key));
.as_mut()
.unwrap()
.send(Message::CacheGetRequest(key));
let msg = KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv(); let msg = KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv();
if let Message::CacheGetReply(v) = msg { if let Message::CacheGetReply(v) = msg {
let leaked = Box::new(v.as_c_slice()); let leaked = Box::new(v.as_c_slice());
@ -25,13 +20,11 @@ pub extern "C" fn get(key: CSlice<u8>) -> &CSlice<'static, i32> {
} }
} }
pub extern "C" fn put(key: CSlice<u8>, list: &CSlice<i32>) { pub extern fn put(key: CSlice<u8>, list: &CSlice<i32>) {
let key = String::from_utf8(key.as_ref().to_vec()).unwrap(); let key = String::from_utf8(key.as_ref().to_vec()).unwrap();
let value = list.as_ref().to_vec(); let value = list.as_ref().to_vec();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::CachePutRequest(key, value));
.as_mut()
.unwrap()
.send(Message::CachePutRequest(key, value));
} }
} }

View File

@ -1,11 +1,11 @@
use core::mem::{forget, replace}; use libcortex_a9::sync_channel::{Sender, Receiver};
use libcortex_a9::sync_channel::{Receiver, Sender};
use libsupport_zynq::boot::Core1; use libsupport_zynq::boot::Core1;
use super::{Message, CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK}; use super::{CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK, Message};
use crate::irq::restart_core1; use crate::irq::restart_core1;
use core::mem::{forget, replace};
pub struct Control { pub struct Control {
pub tx: Sender<'static, Message>, pub tx: Sender<'static, Message>,
pub rx: Receiver<'static, Message>, pub rx: Receiver<'static, Message>,
@ -53,3 +53,4 @@ impl Control {
forget(replace(&mut self.rx, core0_rx)); forget(replace(&mut self.rx, core0_rx));
} }
} }

View File

@ -1,20 +1,31 @@
//! Kernel prologue/epilogue that runs on the 2nd CPU core //! Kernel prologue/epilogue that runs on the 2nd CPU core
use core::{mem, ptr, cell::UnsafeCell};
use alloc::borrow::ToOwned; use alloc::borrow::ToOwned;
use core::{cell::UnsafeCell, mem, ptr}; use log::{debug, info, error};
use cslice::CSlice; use cslice::CSlice;
use dyld::{self, elf::EXIDX_Entry, Library};
use libboard_zynq::{gic, mpcore};
use libcortex_a9::{asm::{dsb, isb},
cache::{bpiall, dcci_slice, iciallu},
enable_fpu, sync_channel};
use libsupport_zynq::ram;
use log::{debug, error, info};
use super::{api::resolve, dma, rpc::rpc_send_async, Message, CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK, use libcortex_a9::{
KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, KERNEL_IMAGE}; enable_fpu,
cache::{dcci_slice, iciallu, bpiall},
asm::{dsb, isb},
sync_channel,
};
use libboard_zynq::{mpcore, gic};
use libsupport_zynq::ram;
use dyld::{self, Library, elf::EXIDX_Entry};
use crate::{eh_artiq, get_async_errors, rtio}; use crate::{eh_artiq, get_async_errors, rtio};
use super::{
api::resolve,
rpc::rpc_send_async,
INIT_LOCK,
CHANNEL_0TO1, CHANNEL_1TO0,
CHANNEL_SEM,
KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0,
KERNEL_IMAGE,
Message,
dma,
};
// linker symbols // linker symbols
extern "C" { extern "C" {
@ -28,12 +39,12 @@ unsafe fn attribute_writeback(typeinfo: *const ()) {
struct Attr { struct Attr {
offset: usize, offset: usize,
tag: CSlice<'static, u8>, tag: CSlice<'static, u8>,
name: CSlice<'static, u8>, name: CSlice<'static, u8>
} }
struct Type { struct Type {
attributes: *const *const Attr, attributes: *const *const Attr,
objects: *const *const (), objects: *const *const ()
} }
let mut tys = typeinfo as *const *const Type; let mut tys = typeinfo as *const *const Type;
@ -52,16 +63,11 @@ unsafe fn attribute_writeback(typeinfo: *const ()) {
attributes = attributes.offset(1); attributes = attributes.offset(1);
if (*attribute).tag.len() > 0 { if (*attribute).tag.len() > 0 {
rpc_send_async( rpc_send_async(0, &(*attribute).tag, [
0,
&(*attribute).tag,
[
&object as *const _ as *const (), &object as *const _ as *const (),
&(*attribute).name as *const _ as *const (), &(*attribute).name as *const _ as *const (),
(object as usize + (*attribute).offset) as *const (), (object as usize + (*attribute).offset) as *const ()
] ].as_ptr());
.as_ptr(),
);
} }
} }
} }
@ -76,8 +82,7 @@ pub struct KernelImage {
impl KernelImage { impl KernelImage {
pub fn new(library: Library) -> Result<Self, dyld::Error> { pub fn new(library: Library) -> Result<Self, dyld::Error> {
let __modinit__ = library let __modinit__ = library.lookup(b"__modinit__")
.lookup(b"__modinit__")
.ok_or(dyld::Error::Lookup("__modinit__".to_owned()))?; .ok_or(dyld::Error::Lookup("__modinit__".to_owned()))?;
let typeinfo = library.lookup(b"typeinfo"); let typeinfo = library.lookup(b"typeinfo");
@ -85,7 +90,8 @@ impl KernelImage {
let bss_start = library.lookup(b"__bss_start"); let bss_start = library.lookup(b"__bss_start");
let end = library.lookup(b"_end"); let end = library.lookup(b"_end");
if let Some(bss_start) = bss_start { if let Some(bss_start) = bss_start {
let end = end.ok_or(dyld::Error::Lookup("_end".to_owned()))?; let end = end
.ok_or(dyld::Error::Lookup("_end".to_owned()))?;
unsafe { unsafe {
ptr::write_bytes(bss_start as *mut u8, 0, (end - bss_start) as usize); ptr::write_bytes(bss_start as *mut u8, 0, (end - bss_start) as usize);
} }
@ -120,7 +126,9 @@ impl KernelImage {
} }
pub fn get_load_addr(&self) -> usize { pub fn get_load_addr(&self) -> usize {
unsafe { self.library.get().as_ref().unwrap().image.as_ptr() as usize } unsafe {
self.library.get().as_ref().unwrap().image.as_ptr() as usize
}
} }
} }
@ -156,19 +164,20 @@ pub extern "C" fn main_core1() {
let message = core1_rx.recv(); let message = core1_rx.recv();
match message { match message {
Message::LoadRequest(data) => { Message::LoadRequest(data) => {
let result = dyld::load(&data, &resolve).and_then(KernelImage::new); let result = dyld::load(&data, &resolve)
.and_then(KernelImage::new);
match result { match result {
Ok(kernel) => { Ok(kernel) => {
loaded_kernel = Some(kernel); loaded_kernel = Some(kernel);
debug!("kernel loaded"); debug!("kernel loaded");
core1_tx.send(Message::LoadCompleted); core1_tx.send(Message::LoadCompleted);
} },
Err(error) => { Err(error) => {
error!("failed to load shared library: {}", error); error!("failed to load shared library: {}", error);
core1_tx.send(Message::LoadFailed); core1_tx.send(Message::LoadFailed);
} }
} }
} },
Message::StartRequest => { Message::StartRequest => {
info!("kernel starting"); info!("kernel starting");
if let Some(kernel) = loaded_kernel.take() { if let Some(kernel) = loaded_kernel.take() {
@ -193,11 +202,9 @@ pub extern "C" fn main_core1() {
} }
/// Called by eh_artiq /// Called by eh_artiq
pub fn terminate( pub fn terminate(exceptions: &'static [Option<eh_artiq::Exception<'static>>],
exceptions: &'static [Option<eh_artiq::Exception<'static>>],
stack_pointers: &'static [eh_artiq::StackPointerBacktrace], stack_pointers: &'static [eh_artiq::StackPointerBacktrace],
backtrace: &'static mut [(usize, usize)], backtrace: &'static mut [(usize, usize)]) -> ! {
) -> ! {
{ {
let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() }; let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() };
let errors = unsafe { get_async_errors() }; let errors = unsafe { get_async_errors() };
@ -208,7 +215,7 @@ pub fn terminate(
/// Called by llvm_libunwind /// Called by llvm_libunwind
#[no_mangle] #[no_mangle]
extern "C" fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const u32 { extern fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const u32 {
let length; let length;
let start: *const EXIDX_Entry; let start: *const EXIDX_Entry;
unsafe { unsafe {
@ -216,14 +223,9 @@ extern "C" fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const
length = (&__exidx_end as *const EXIDX_Entry).offset_from(&__exidx_start) as u32; length = (&__exidx_end as *const EXIDX_Entry).offset_from(&__exidx_start) as u32;
start = &__exidx_start; start = &__exidx_start;
} else if KERNEL_IMAGE != ptr::null() { } else if KERNEL_IMAGE != ptr::null() {
let exidx = KERNEL_IMAGE let exidx = KERNEL_IMAGE.as_ref()
.as_ref()
.expect("dl_unwind_find_exidx kernel image") .expect("dl_unwind_find_exidx kernel image")
.library .library.get().as_ref().unwrap().exidx();
.get()
.as_ref()
.unwrap()
.exidx();
length = exidx.len() as u32; length = exidx.len() as u32;
start = exidx.as_ptr(); start = exidx.as_ptr();
} else { } else {
@ -235,7 +237,7 @@ extern "C" fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const
start as *const u32 start as *const u32
} }
pub extern "C" fn rtio_get_destination_status(destination: i32) -> bool { pub extern fn rtio_get_destination_status(destination: i32) -> bool {
#[cfg(has_drtio)] #[cfg(has_drtio)]
if destination > 0 && destination < 255 { if destination > 0 && destination < 255 {
let reply = unsafe { let reply = unsafe {
@ -246,7 +248,7 @@ pub extern "C" fn rtio_get_destination_status(destination: i32) -> bool {
}; };
return match reply { return match reply {
Message::UpDestinationsReply(x) => x, Message::UpDestinationsReply(x) => x,
_ => panic!("received unexpected reply to UpDestinationsRequest: {:?}", reply), _ => panic!("received unexpected reply to UpDestinationsRequest: {:?}", reply)
}; };
} }

View File

@ -0,0 +1,212 @@
use crate::{
pl::csr,
artiq_raise,
rtio,
};
use alloc::{vec::Vec, string::String, boxed::Box};
use cslice::CSlice;
use super::{KERNEL_IMAGE, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, Message};
use core::mem;
use libcortex_a9::cache::dcci_slice;
const ALIGNMENT: usize = 16 * 8;
#[repr(C)]
pub struct DmaTrace {
duration: i64,
address: i32,
}
#[derive(Clone, Debug)]
pub struct DmaRecorder {
pub name: String,
pub buffer: Vec<u8>,
pub duration: i64,
}
static mut RECORDER: Option<DmaRecorder> = None;
pub unsafe fn init_dma_recorder() {
// as static would remain after restart, we have to reset it,
// without running its destructor.
mem::forget(mem::replace(&mut RECORDER, None));
}
pub extern fn dma_record_start(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaEraseRequest(name.clone()));
}
unsafe {
if RECORDER.is_some() {
artiq_raise!("DMAError", "DMA is already recording")
}
let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output",
dma_record_output as *const ()).unwrap();
library.rebind(b"rtio_output_wide",
dma_record_output_wide as *const ()).unwrap();
RECORDER = Some(DmaRecorder {
name,
buffer: Vec::new(),
duration: 0,
});
}
}
pub extern fn dma_record_stop(duration: i64) {
unsafe {
if RECORDER.is_none() {
artiq_raise!("DMAError", "DMA is not recording")
}
let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output",
rtio::output as *const ()).unwrap();
library.rebind(b"rtio_output_wide",
rtio::output_wide as *const ()).unwrap();
let mut recorder = RECORDER.take().unwrap();
recorder.duration = duration;
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(
Message::DmaPutRequest(recorder)
);
}
}
#[inline(always)]
unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
words: usize) {
// See gateware/rtio/dma.py.
const HEADER_LENGTH: usize = /*length*/1 + /*channel*/3 + /*timestamp*/8 + /*address*/1;
let length = HEADER_LENGTH + /*data*/words * 4;
let buffer = &mut RECORDER.as_mut().unwrap().buffer;
buffer.reserve(length);
buffer.extend_from_slice(&[
(length >> 0) as u8,
(target >> 8) as u8,
(target >> 16) as u8,
(target >> 24) as u8,
(timestamp >> 0) as u8,
(timestamp >> 8) as u8,
(timestamp >> 16) as u8,
(timestamp >> 24) as u8,
(timestamp >> 32) as u8,
(timestamp >> 40) as u8,
(timestamp >> 48) as u8,
(timestamp >> 56) as u8,
(target >> 0) as u8,
]);
}
pub extern fn dma_record_output(target: i32, word: i32) {
unsafe {
let timestamp = rtio::now_mu();
dma_record_output_prepare(timestamp, target, 1);
RECORDER.as_mut().unwrap().buffer.extend_from_slice(&[
(word >> 0) as u8,
(word >> 8) as u8,
(word >> 16) as u8,
(word >> 24) as u8,
]);
}
}
pub extern fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
assert!(words.len() <= 16); // enforce the hardware limit
unsafe {
let timestamp = rtio::now_mu();
dma_record_output_prepare(timestamp, target, words.len());
let buffer = &mut RECORDER.as_mut().unwrap().buffer;
for word in words.as_ref().iter() {
buffer.extend_from_slice(&[
(word >> 0) as u8,
(word >> 8) as u8,
(word >> 16) as u8,
(word >> 24) as u8,
]);
}
}
}
pub extern fn dma_erase(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaEraseRequest(name));
}
}
pub extern fn dma_retrieve(name: CSlice<u8>) -> DmaTrace {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaGetRequest(name));
}
match unsafe {KERNEL_CHANNEL_0TO1.as_mut().unwrap()}.recv() {
Message::DmaGetReply(None) => (),
Message::DmaGetReply(Some((mut v, duration))) => {
v.reserve(ALIGNMENT - 1);
let original_length = v.len();
let padding = ALIGNMENT - v.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
v.push(0);
}
// trailing zero to indicate end of buffer
v.push(0);
v.copy_within(0..original_length, padding);
dcci_slice(&v);
let v = Box::new(v);
let address = Box::into_raw(v) as *mut Vec<u8> as i32;
return DmaTrace {
address,
duration,
};
},
_ => panic!("Expected DmaGetReply after DmaGetRequest!"),
}
// we have to defer raising error as we have to drop the message first...
artiq_raise!("DMAError", "DMA trace not found");
}
pub extern fn dma_playback(timestamp: i64, ptr: i32) {
unsafe {
let v = Box::from_raw(ptr as *mut Vec<u8>);
let padding = ALIGNMENT - v.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
let ptr = v.as_ptr().add(padding) as i32;
csr::rtio_dma::base_address_write(ptr as u32);
csr::rtio_dma::time_offset_write(timestamp as u64);
csr::cri_con::selected_write(1);
csr::rtio_dma::enable_write(1);
while csr::rtio_dma::enable_read() != 0 {}
csr::cri_con::selected_write(0);
// leave the handle as we may try to do playback for another time.
mem::forget(v);
let error = csr::rtio_dma::error_read();
if error != 0 {
let timestamp = csr::rtio_dma::error_timestamp_read();
let channel = csr::rtio_dma::error_channel_read();
csr::rtio_dma::error_write(1);
if error & 1 != 0 {
artiq_raise!("RTIOUnderflow",
"RTIO underflow at {1} mu, channel {rtio_channel_info:0}",
channel as i64, timestamp as i64, 0);
}
if error & 2 != 0 {
artiq_raise!("RTIODestinationUnreachable",
"RTIO destination unreachable, output, at {1} mu, channel {rtio_channel_info:0}",
channel as i64, timestamp as i64, 0);
}
}
}
}

View File

@ -0,0 +1,67 @@
use core::ptr;
use alloc::{vec::Vec, string::String};
use libcortex_a9::{mutex::Mutex, sync_channel, semaphore::Semaphore};
use crate::eh_artiq;
mod control;
pub use control::Control;
pub mod core1;
mod api;
mod rpc;
mod dma;
pub use dma::DmaRecorder;
mod cache;
#[derive(Debug, Clone)]
pub struct RPCException {
pub id: u32,
pub message: u32,
pub param: [i64; 3],
pub file: u32,
pub line: i32,
pub column: i32,
pub function: u32
}
#[derive(Debug, Clone)]
pub enum Message {
LoadRequest(Vec<u8>),
LoadCompleted,
LoadFailed,
StartRequest,
KernelFinished(u8),
KernelException(&'static [Option<eh_artiq::Exception<'static>>],
&'static [eh_artiq::StackPointerBacktrace],
&'static [(usize, usize)],
u8),
RpcSend { is_async: bool, data: Vec<u8> },
RpcRecvRequest(*mut ()),
RpcRecvReply(Result<usize, RPCException>),
CacheGetRequest(String),
CacheGetReply(Vec<i32>),
CachePutRequest(String, Vec<i32>),
DmaPutRequest(DmaRecorder),
DmaEraseRequest(String),
DmaGetRequest(String),
DmaGetReply(Option<(Vec<u8>, i64)>),
#[cfg(has_drtio)]
UpDestinationsRequest(i32),
#[cfg(has_drtio)]
UpDestinationsReply(bool),
}
static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None);
static CHANNEL_1TO0: Mutex<Option<sync_channel::Receiver<'static, Message>>> = Mutex::new(None);
static CHANNEL_SEM: Semaphore = Semaphore::new(0, 1);
static mut KERNEL_CHANNEL_0TO1: Option<sync_channel::Receiver<'static, Message>> = None;
static mut KERNEL_CHANNEL_1TO0: Option<sync_channel::Sender<'static, Message>> = None;
pub static mut KERNEL_IMAGE: *const core1::KernelImage = ptr::null();
static INIT_LOCK: Mutex<()> = Mutex::new(());

View File

@ -1,28 +1,31 @@
//! Kernel-side RPC API //! Kernel-side RPC API
use alloc::vec::Vec; use alloc::vec::Vec;
use cslice::CSlice; use cslice::CSlice;
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0}; use crate::eh_artiq;
use crate::{eh_artiq, rpc::send_args}; use crate::rpc::send_args;
use super::{
KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0,
Message,
};
fn rpc_send_common(is_async: bool, service: u32, tag: &CSlice<u8>, data: *const *const ()) { fn rpc_send_common(is_async: bool, service: u32, tag: &CSlice<u8>, data: *const *const ()) {
let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() }; let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() };
let mut buffer = Vec::<u8>::new(); let mut buffer = Vec::<u8>::new();
send_args(&mut buffer, service, tag.as_ref(), data, true).expect("RPC encoding failed"); send_args(&mut buffer, service, tag.as_ref(), data).expect("RPC encoding failed");
core1_tx.send(Message::RpcSend { is_async, data: buffer }); core1_tx.send(Message::RpcSend { is_async, data: buffer });
} }
pub extern "C" fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) { pub extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
rpc_send_common(false, service, tag, data); rpc_send_common(false, service, tag, data);
} }
pub extern "C" fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) { pub extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
rpc_send_common(true, service, tag, data); rpc_send_common(true, service, tag, data);
} }
pub extern "C" fn rpc_recv(slot: *mut ()) -> usize { pub extern fn rpc_recv(slot: *mut ()) -> usize {
let reply = unsafe { let reply = unsafe {
let core1_rx = KERNEL_CHANNEL_0TO1.as_mut().unwrap(); let core1_rx = KERNEL_CHANNEL_0TO1.as_mut().unwrap();
let core1_tx = KERNEL_CHANNEL_1TO0.as_mut().unwrap(); let core1_tx = KERNEL_CHANNEL_1TO0.as_mut().unwrap();
@ -39,9 +42,9 @@ pub extern "C" fn rpc_recv(slot: *mut ()) -> usize {
column: exception.column as u32, column: exception.column as u32,
function: CSlice::new(exception.function as *const u8, usize::MAX), function: CSlice::new(exception.function as *const u8, usize::MAX),
message: CSlice::new(exception.message as *const u8, usize::MAX), message: CSlice::new(exception.message as *const u8, usize::MAX),
param: exception.param, param: exception.param
}) })
}, },
_ => panic!("received unexpected reply to RpcRecvRequest: {:?}", reply), _ => panic!("received unexpected reply to RpcRecvRequest: {:?}", reply)
} }
} }

View File

@ -1,94 +1,107 @@
#![no_std] #![no_std]
#![no_main] #![no_main]
#![recursion_limit = "1024"] // for futures_util::select! #![recursion_limit="1024"] // for futures_util::select!
#![feature(alloc_error_handler)] #![feature(alloc_error_handler)]
#![feature(const_btree_new)]
#![feature(panic_info_message)] #![feature(panic_info_message)]
#![feature(c_variadic)]
#![feature(const_btree_new)]
#![feature(const_in_array_repeat_expressions)]
#![feature(naked_functions)]
#![feature(asm)]
#[macro_use] #[macro_use]
extern crate alloc; extern crate alloc;
#[cfg(all(feature = "target_kasli_soc", has_virtual_leds))] use log::{info, warn, error};
use core::cell::RefCell;
use ksupport; use libboard_zynq::{timer::GlobalTimer, mpcore, gic};
use libasync::task; use libasync::{task, block_async};
#[cfg(has_drtio_eem)] use libsupport_zynq::ram;
use libboard_artiq::drtio_eem; use nb;
#[cfg(feature = "target_kasli_soc")] use void::Void;
use libboard_artiq::io_expander;
use libboard_artiq::{identifier_read, logger, pl};
use libboard_zynq::{gic, mpcore, timer::GlobalTimer};
use libconfig::Config; use libconfig::Config;
use libcortex_a9::l2c::enable_l2_cache; use libcortex_a9::l2c::enable_l2_cache;
use libsupport_zynq::{exception_vectors, ram}; use libboard_artiq::{logger, identifier_read, pl};
use log::{info, warn}; #[cfg(feature = "target_kasli_soc")]
use libboard_artiq::io_expander;
mod analyzer; const ASYNC_ERROR_COLLISION: u8 = 1 << 0;
mod comms; const ASYNC_ERROR_BUSY: u8 = 1 << 1;
const ASYNC_ERROR_SEQUENCE_ERROR: u8 = 1 << 2;
mod mgmt;
mod moninj;
mod panic;
mod proto_async; mod proto_async;
mod rpc_async; mod comms;
mod rtio_clocking; mod rpc;
mod rtio_dma; #[cfg(ki_impl = "csr")]
#[path = "rtio_csr.rs"]
mod rtio;
#[cfg(ki_impl = "acp")]
#[path = "rtio_acp.rs"]
mod rtio;
mod rtio_mgt; mod rtio_mgt;
#[cfg(has_drtio)] mod rtio_clocking;
mod subkernel; mod kernel;
mod moninj;
mod eh_artiq;
mod panic;
mod mgmt;
mod analyzer;
mod irq;
mod i2c;
// linker symbols static mut SEEN_ASYNC_ERRORS: u8 = 0;
extern "C" {
static __exceptions_start: u32; pub unsafe fn get_async_errors() -> u8 {
let errors = SEEN_ASYNC_ERRORS;
SEEN_ASYNC_ERRORS = 0;
errors
} }
#[cfg(all(feature = "target_kasli_soc", has_virtual_leds))] fn wait_for_async_rtio_error() -> nb::Result<(), Void> {
async fn io_expanders_service( unsafe {
i2c_bus: RefCell<&mut libboard_zynq::i2c::I2c>, if pl::csr::rtio_core::async_error_read() != 0 {
io_expander0: RefCell<io_expander::IoExpander>, Ok(())
io_expander1: RefCell<io_expander::IoExpander>, } else {
) { Err(nb::Error::WouldBlock)
}
}
}
async fn report_async_rtio_errors() {
loop { loop {
task::r#yield().await; let _ = block_async!(wait_for_async_rtio_error()).await;
io_expander0 unsafe {
.borrow_mut() let errors = pl::csr::rtio_core::async_error_read();
.service(&mut i2c_bus.borrow_mut()) if errors & ASYNC_ERROR_COLLISION != 0 {
.expect("I2C I/O expander #0 service failed"); let channel = pl::csr::rtio_core::collision_channel_read();
io_expander1 error!("RTIO collision involving channel 0x{:04x}:{}", channel, rtio_mgt::resolve_channel_name(channel as u32));
.borrow_mut()
.service(&mut i2c_bus.borrow_mut())
.expect("I2C I/O expander #1 service failed");
} }
} if errors & ASYNC_ERROR_BUSY != 0 {
let channel = pl::csr::rtio_core::busy_channel_read();
#[cfg(has_grabber)] error!("RTIO busy error involving channel 0x{:04x}:{}", channel, rtio_mgt::resolve_channel_name(channel as u32));
mod grabber { }
use libasync::delay; if errors & ASYNC_ERROR_SEQUENCE_ERROR != 0 {
use libboard_artiq::grabber; let channel = pl::csr::rtio_core::sequence_error_channel_read();
use libboard_zynq::time::Milliseconds; error!("RTIO sequence error involving channel 0x{:04x}:{}", channel, rtio_mgt::resolve_channel_name(channel as u32));
}
use crate::GlobalTimer; SEEN_ASYNC_ERRORS = errors;
pub async fn grabber_thread(timer: GlobalTimer) { pl::csr::rtio_core::async_error_write(errors);
let mut countdown = timer.countdown();
loop {
grabber::tick();
delay(&mut countdown, Milliseconds(200)).await;
} }
} }
} }
static mut LOG_BUFFER: [u8; 1 << 17] = [0; 1 << 17];
static mut LOG_BUFFER: [u8; 1<<17] = [0; 1<<17];
#[no_mangle] #[no_mangle]
pub fn main_core0() { pub fn main_core0() {
unsafe {
exception_vectors::set_vector_table(&__exceptions_start as *const u32 as u32);
}
enable_l2_cache(0x8); enable_l2_cache(0x8);
let mut timer = GlobalTimer::start(); let mut timer = GlobalTimer::start();
let buffer_logger = unsafe { logger::BufferLogger::new(&mut LOG_BUFFER[..]) }; let buffer_logger = unsafe {
logger::BufferLogger::new(&mut LOG_BUFFER[..])
};
buffer_logger.set_uart_log_level(log::LevelFilter::Info); buffer_logger.set_uart_log_level(log::LevelFilter::Info);
buffer_logger.register(); buffer_logger.register();
log::set_max_level(log::LevelFilter::Info); log::set_max_level(log::LevelFilter::Info);
@ -100,36 +113,21 @@ pub fn main_core0() {
info!("gateware ident: {}", identifier_read(&mut [0; 64])); info!("gateware ident: {}", identifier_read(&mut [0; 64]));
ksupport::i2c::init(); i2c::init();
#[cfg(feature = "target_kasli_soc")] #[cfg(feature = "target_kasli_soc")]
{ {
let i2c_bus = unsafe { (ksupport::i2c::I2C_BUS).as_mut().unwrap() }; let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
let mut io_expander0 = io_expander::IoExpander::new(i2c_bus, 0).unwrap(); for expander_i in 0..=1 {
let mut io_expander1 = io_expander::IoExpander::new(i2c_bus, 1).unwrap(); let mut io_expander = io_expander::IoExpander::new(i2c, expander_i).unwrap();
io_expander0 io_expander.init().expect("I2C I/O expander #0 initialization failed");
.init(i2c_bus) // Actively drive TX_DISABLE to false on SFP0..3
.expect("I2C I/O expander #0 initialization failed"); io_expander.set_oe(0, 1 << 1).unwrap();
io_expander1 io_expander.set_oe(1, 1 << 1).unwrap();
.init(i2c_bus) io_expander.set(0, 1, false);
.expect("I2C I/O expander #1 initialization failed"); io_expander.set(1, 1, false);
io_expander.service().unwrap();
// Drive CLK_SEL to true }
#[cfg(has_si549)]
io_expander0.set(1, 7, true);
// Drive TX_DISABLE to false on SFP0..3
io_expander0.set(0, 1, false);
io_expander1.set(0, 1, false);
io_expander0.set(1, 1, false);
io_expander1.set(1, 1, false);
io_expander0.service(i2c_bus).unwrap();
io_expander1.service(i2c_bus).unwrap();
#[cfg(has_virtual_leds)]
task::spawn(io_expanders_service(
RefCell::new(i2c_bus),
RefCell::new(io_expander0),
RefCell::new(io_expander1),
));
} }
let cfg = match Config::new() { let cfg = match Config::new() {
@ -142,13 +140,7 @@ pub fn main_core0() {
rtio_clocking::init(&mut timer, &cfg); rtio_clocking::init(&mut timer, &cfg);
#[cfg(has_drtio_eem)] task::spawn(report_async_rtio_errors());
drtio_eem::init(&mut timer, &cfg);
#[cfg(has_grabber)]
task::spawn(grabber::grabber_thread(timer));
task::spawn(ksupport::report_async_rtio_errors());
comms::main(timer, cfg); comms::main(timer, cfg);
} }

File diff suppressed because it is too large Load Diff

View File

@ -1,23 +1,26 @@
use core::{fmt, cell::RefCell};
use alloc::{collections::BTreeMap, rc::Rc}; use alloc::{collections::BTreeMap, rc::Rc};
use core::{cell::RefCell, fmt};
use futures::{pin_mut, select_biased, FutureExt};
use libasync::{block_async, nb, smoltcp::TcpStream, task};
use libboard_artiq::drtio_routing;
use libboard_zynq::{smoltcp, time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::mutex::Mutex;
use log::{debug, info, warn}; use log::{debug, info, warn};
use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive};
use void::Void; use void::Void;
use libboard_artiq::drtio_routing;
use libboard_zynq::{smoltcp, timer::GlobalTimer, time::Milliseconds};
use libasync::{task, smoltcp::TcpStream, block_async, nb};
use libcortex_a9::mutex::Mutex;
use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive};
use futures::{pin_mut, select_biased, FutureExt};
use crate::proto_async::*; use crate::proto_async::*;
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error { pub enum Error {
NetworkError(smoltcp::Error), NetworkError(smoltcp::Error),
UnexpectedPattern, UnexpectedPattern,
UnrecognizedPacket, UnrecognizedPacket
} }
pub type Result<T> = core::result::Result<T, Error>; pub type Result<T> = core::result::Result<T, Error>;
@ -43,107 +46,60 @@ enum HostMessage {
MonitorProbe = 0, MonitorProbe = 0,
MonitorInjection = 3, MonitorInjection = 3,
Inject = 1, Inject = 1,
GetInjectionStatus = 2, GetInjectionStatus = 2
} }
#[derive(Debug, FromPrimitive, ToPrimitive)] #[derive(Debug, FromPrimitive, ToPrimitive)]
enum DeviceMessage { enum DeviceMessage {
MonitorStatus = 0, MonitorStatus = 0,
InjectionStatus = 1, InjectionStatus = 1
} }
#[cfg(has_drtio)] #[cfg(has_drtio)]
mod remote_moninj { mod remote_moninj {
use super::*;
use libboard_artiq::drtioaux_async; use libboard_artiq::drtioaux_async;
use crate::rtio_mgt::drtio;
use log::error; use log::error;
use super::*; pub async fn read_probe(aux_mutex: &Rc<Mutex<bool>>, timer: GlobalTimer, linkno: u8, destination: u8, channel: i32, probe: i8) -> i64 {
use crate::rtio_mgt::{drtio, drtio::Error as DrtioError}; let reply = drtio::aux_transact(aux_mutex, linkno, &drtioaux_async::Packet::MonitorRequest {
pub async fn read_probe(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
probe: i8,
) -> i64 {
let reply = drtio::aux_transact(
aux_mutex,
linkno,
routing_table,
&drtioaux_async::Packet::MonitorRequest {
destination: destination, destination: destination,
channel: channel as _, channel: channel as _,
probe: probe as _, probe: probe as _},
}, timer).await;
timer,
)
.await;
match reply { match reply {
Ok(drtioaux_async::Packet::MonitorReply { value }) => return value as i64, Ok(drtioaux_async::Packet::MonitorReply { value }) => return value as i64,
Ok(packet) => error!("received unexpected aux packet: {:?}", packet), Ok(packet) => error!("received unexpected aux packet: {:?}", packet),
Err(DrtioError::LinkDown) => { Err("link went down") => { debug!("link is down"); },
warn!("link is down"); Err(e) => error!("aux packet error ({})", e)
}
Err(e) => error!("aux packet error ({})", e),
} }
0 0
} }
pub async fn inject( pub async fn inject(aux_mutex: &Rc<Mutex<bool>>, _timer: GlobalTimer, linkno: u8, destination: u8, channel: i32, overrd: i8, value: i8) {
aux_mutex: &Rc<Mutex<bool>>, let _lock = aux_mutex.lock();
_routing_table: &drtio_routing::RoutingTable, drtioaux_async::send(linkno, &drtioaux_async::Packet::InjectionRequest {
_timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
overrd: i8,
value: i8,
) {
let _lock = aux_mutex.async_lock().await;
drtioaux_async::send(
linkno,
&drtioaux_async::Packet::InjectionRequest {
destination: destination, destination: destination,
channel: channel as _, channel: channel as _,
overrd: overrd as _, overrd: overrd as _,
value: value as _, value: value as _
}, }).await.unwrap();
)
.await
.unwrap();
} }
pub async fn read_injection_status( pub async fn read_injection_status(aux_mutex: &Rc<Mutex<bool>>, timer: GlobalTimer, linkno: u8, destination: u8, channel: i32, overrd: i8) -> i8 {
aux_mutex: &Rc<Mutex<bool>>, let reply = drtio::aux_transact(aux_mutex,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
overrd: i8,
) -> i8 {
let reply = drtio::aux_transact(
aux_mutex,
linkno, linkno,
routing_table,
&drtioaux_async::Packet::InjectionStatusRequest { &drtioaux_async::Packet::InjectionStatusRequest {
destination: destination, destination: destination,
channel: channel as _, channel: channel as _,
overrd: overrd as _, overrd: overrd as _},
}, timer).await;
timer,
)
.await;
match reply { match reply {
Ok(drtioaux_async::Packet::InjectionStatusReply { value }) => return value as i8, Ok(drtioaux_async::Packet::InjectionStatusReply { value }) => return value as i8,
Ok(packet) => error!("received unexpected aux packet: {:?}", packet), Ok(packet) => error!("received unexpected aux packet: {:?}", packet),
Err(DrtioError::LinkDown) => { Err("link went down") => { debug!("link is down"); },
warn!("link is down"); Err(e) => error!("aux packet error ({})", e)
}
Err(e) => error!("aux packet error ({})", e),
} }
0 0
} }
@ -188,7 +144,7 @@ macro_rules! dispatch {
local_moninj::$func(channel.into(), $($param, )*) local_moninj::$func(channel.into(), $($param, )*)
} else { } else {
let linkno = hop - 1 as u8; let linkno = hop - 1 as u8;
remote_moninj::$func($aux_mutex, $routing_table, $timer, linkno, destination, channel, $($param, )*).await remote_moninj::$func($aux_mutex, $timer, linkno, destination, channel, $($param, )*).await
} }
}} }}
} }
@ -201,12 +157,8 @@ macro_rules! dispatch {
}} }}
} }
async fn handle_connection( async fn handle_connection(stream: &TcpStream, timer: GlobalTimer,
stream: &TcpStream, _aux_mutex: &Rc<Mutex<bool>>, _routing_table: &drtio_routing::RoutingTable) -> Result<()> {
timer: GlobalTimer,
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &drtio_routing::RoutingTable,
) -> Result<()> {
if !expect(&stream, b"ARTIQ moninj\n").await? { if !expect(&stream, b"ARTIQ moninj\n").await? {
return Err(Error::UnexpectedPattern); return Err(Error::UnexpectedPattern);
} }
@ -303,13 +255,7 @@ async fn handle_connection(
} }
} }
pub fn start( pub fn start(timer: GlobalTimer, aux_mutex: Rc<Mutex<bool>>, routing_table: Rc<RefCell<drtio_routing::RoutingTable>>) {
timer: GlobalTimer,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
) {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
task::spawn(async move { task::spawn(async move {
loop { loop {
let aux_mutex = aux_mutex.clone(); let aux_mutex = aux_mutex.clone();

View File

@ -1,13 +1,14 @@
#[cfg(feature = "target_kasli_soc")] use libboard_zynq::{print, println};
use libboard_zynq::error_led::ErrorLED;
use libboard_zynq::{print, println, timer::GlobalTimer};
use libconfig::Config;
use libcortex_a9::regs::MPIDR;
use libregister::RegisterR; use libregister::RegisterR;
use log::error; use libcortex_a9::regs::MPIDR;
use unwind::backtrace; use unwind::backtrace;
#[cfg(feature = "target_kasli_soc")]
use libboard_zynq::error_led::ErrorLED;
use crate::comms::soft_panic_main; use crate::comms::soft_panic_main;
use log::error;
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
static mut PANICKED: [bool; 2] = [false; 2]; static mut PANICKED: [bool; 2] = [false; 2];
static mut SOFT_PANICKED: bool = false; static mut SOFT_PANICKED: bool = false;
@ -67,7 +68,9 @@ fn soft_panic(info: &core::panic::PanicInfo) -> ! {
let timer = GlobalTimer::start(); let timer = GlobalTimer::start();
let cfg = match Config::new() { let cfg = match Config::new() {
Ok(cfg) => cfg, Ok(cfg) => cfg,
Err(_) => Config::new_dummy(), Err(_) => {
Config::new_dummy()
}
}; };
soft_panic_main(timer, cfg); soft_panic_main(timer, cfg);
} }

View File

@ -1,7 +1,8 @@
use core::{cell::RefCell, cmp::min}; use core::cmp::min;
use core::cell::RefCell;
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp; use libboard_zynq::smoltcp;
use libasync::smoltcp::TcpStream;
type Result<T> = core::result::Result<T, smoltcp::Error>; type Result<T> = core::result::Result<T, smoltcp::Error>;
@ -13,8 +14,7 @@ enum RecvState<T> {
pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> { pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> {
let mut state = RecvState::NeedsMore(0, true); let mut state = RecvState::NeedsMore(0, true);
loop { loop {
state = stream state = stream.recv(|buf| {
.recv(|buf| {
let mut consumed = 0; let mut consumed = 0;
if let RecvState::NeedsMore(mut cur_index, _) = state { if let RecvState::NeedsMore(mut cur_index, _) = state {
for b in buf.iter() { for b in buf.iter() {
@ -32,8 +32,7 @@ pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> {
} else { } else {
unreachable!(); unreachable!();
} }
}) }).await?;
.await?;
if let RecvState::Completed(result) = state { if let RecvState::Completed(result) = state {
return Ok(result); return Ok(result);
} }
@ -41,11 +40,15 @@ pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> {
} }
pub async fn read_bool(stream: &TcpStream) -> Result<bool> { pub async fn read_bool(stream: &TcpStream) -> Result<bool> {
Ok(stream.recv(|buf| (1, buf[0] != 0)).await?) Ok(stream.recv(|buf| {
(1, buf[0] != 0)
}).await?)
} }
pub async fn read_i8(stream: &TcpStream) -> Result<i8> { pub async fn read_i8(stream: &TcpStream) -> Result<i8> {
Ok(stream.recv(|buf| (1, buf[0] as i8)).await?) Ok(stream.recv(|buf| {
(1, buf[0] as i8)
}).await?)
} }
pub async fn read_i32(stream: &TcpStream) -> Result<i32> { pub async fn read_i32(stream: &TcpStream) -> Result<i32> {
@ -65,14 +68,12 @@ pub async fn read_chunk(stream: &TcpStream, destination: &mut [u8]) -> Result<()
let destination = RefCell::new(destination); let destination = RefCell::new(destination);
let mut done = 0; let mut done = 0;
while done < total { while done < total {
let count = stream let count = stream.recv(|buf| {
.recv(|buf| {
let mut destination = destination.borrow_mut(); let mut destination = destination.borrow_mut();
let count = min(total - done, buf.len()); let count = min(total - done, buf.len());
destination[done..done + count].copy_from_slice(&buf[..count]); destination[done..done + count].copy_from_slice(&buf[..count]);
(count, count) (count, count)
}) }).await?;
.await?;
done += count; done += count;
} }
Ok(()) Ok(())

View File

@ -1,116 +1,135 @@
use core::str; use core::str;
use core::future::Future;
use byteorder::{ByteOrder, NativeEndian}; use cslice::{CSlice, CMutSlice};
use core_io::{Error, Read, Write};
use cslice::{CMutSlice, CSlice};
use io::{ProtoRead, ProtoWrite};
use log::trace; use log::trace;
use byteorder::{NativeEndian, ByteOrder};
use self::tag::{split_tag, Tag, TagIterator}; use core_io::{Write, Error};
use libboard_zynq::smoltcp;
use libasync::smoltcp::TcpStream;
use alloc::boxed::Box;
use async_recursion::async_recursion;
use io::proto::ProtoWrite;
use crate::proto_async;
use self::tag::{Tag, TagIterator, split_tag};
#[inline] #[inline]
pub fn round_up(val: usize, power_of_two: usize) -> usize { fn round_up(val: usize, power_of_two: usize) -> usize {
assert!(power_of_two.is_power_of_two()); assert!(power_of_two.is_power_of_two());
let max_rem = power_of_two - 1; let max_rem = power_of_two - 1;
(val + max_rem) & (!max_rem) (val + max_rem) & (!max_rem)
} }
#[inline] #[inline]
pub unsafe fn round_up_mut<T>(ptr: *mut T, power_of_two: usize) -> *mut T { unsafe fn round_up_mut<T>(ptr: *mut T, power_of_two: usize) -> *mut T {
round_up(ptr as usize, power_of_two) as *mut T round_up(ptr as usize, power_of_two) as *mut T
} }
#[inline] #[inline]
pub unsafe fn round_up_const<T>(ptr: *const T, power_of_two: usize) -> *const T { unsafe fn round_up_const<T>(ptr: *const T, power_of_two: usize) -> *const T {
round_up(ptr as usize, power_of_two) as *const T round_up(ptr as usize, power_of_two) as *const T
} }
#[inline] #[inline]
pub unsafe fn align_ptr<T>(ptr: *const ()) -> *const T { unsafe fn align_ptr<T>(ptr: *const ()) -> *const T {
round_up_const(ptr, core::mem::align_of::<T>()) as *const T round_up_const(ptr, core::mem::align_of::<T>()) as *const T
} }
#[inline] #[inline]
pub unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T { unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T {
round_up_mut(ptr, core::mem::align_of::<T>()) as *mut T round_up_mut(ptr, core::mem::align_of::<T>()) as *mut T
} }
// versions for reader rather than TcpStream /// Reads (deserializes) `length` array or list elements of type `tag` from `stream`,
// they will be made into sync for satellite subkernels later /// writing them into the buffer given by `storage`.
unsafe fn recv_elements<F, R>( ///
reader: &mut R, /// `alloc` is used for nested allocations (if elements themselves contain
elt_tag: Tag, /// lists/arrays), see [recv_value].
#[async_recursion(?Send)]
async unsafe fn recv_elements<F>(
stream: &TcpStream,
elt_tag: Tag<'async_recursion>,
length: usize, length: usize,
storage: *mut (), storage: *mut (),
alloc: &mut F, alloc: &(impl Fn(usize) -> F + 'async_recursion)
) -> Result<(), Error> ) -> Result<(), smoltcp::Error>
where where
F: FnMut(usize) -> *mut (), F: Future<Output=*mut ()>,
R: Read + ?Sized,
{ {
// List of simple types are special-cased in the protocol for performance.
match elt_tag { match elt_tag {
Tag::Bool => { Tag::Bool => {
let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length); let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length);
reader.read_exact(dest)?; proto_async::read_chunk(stream, dest).await?;
} },
Tag::Int32 => { Tag::Int32 => {
let ptr = storage as *mut u32; let ptr = storage as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4); let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
reader.read_exact(dest)?; proto_async::read_chunk(stream, dest).await?;
drop(dest); drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length); let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest); NativeEndian::from_slice_u32(dest);
} },
Tag::Int64 | Tag::Float64 => { Tag::Int64 | Tag::Float64 => {
let ptr = storage as *mut u64; let ptr = storage as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8); let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
reader.read_exact(dest)?; proto_async::read_chunk(stream, dest).await?;
drop(dest); drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length); let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest); NativeEndian::from_slice_u64(dest);
} },
_ => { _ => {
let mut data = storage; let mut data = storage;
for _ in 0..length { for _ in 0..length {
recv_value(reader, elt_tag, &mut data, alloc)? recv_value(stream, elt_tag, &mut data, alloc).await?
} }
} }
} }
Ok(()) Ok(())
} }
unsafe fn recv_value<F, R>(reader: &mut R, tag: Tag, data: &mut *mut (), alloc: &mut F) -> Result<(), Error> /// Reads (deserializes) a value of type `tag` from `stream`, writing the results to
where /// the kernel-side buffer `data` (the passed pointer to which is incremented to point
F: FnMut(usize) -> *mut (), /// past the just-received data). For nested allocations (lists/arrays), `alloc` is
R: Read + ?Sized, /// invoked any number of times with the size of the required allocation as a parameter
/// (which is assumed to be correctly aligned for all payload types).
#[async_recursion(?Send)]
async unsafe fn recv_value<F>(stream: &TcpStream, tag: Tag<'async_recursion>, data: &mut *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion))
-> Result<(), smoltcp::Error>
where F: Future<Output=*mut ()>
{ {
macro_rules! consume_value { macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{ ($ty:ty, |$ptr:ident| $map:expr) => ({
let $ptr = align_ptr_mut::<$ty>(*data); let $ptr = align_ptr_mut::<$ty>(*data);
*data = $ptr.offset(1) as *mut (); *data = $ptr.offset(1) as *mut ();
$map $map
}}; })
} }
match tag { match tag {
Tag::None => Ok(()), Tag::None => Ok(()),
Tag::Bool => consume_value!(i8, |ptr| { Tag::Bool =>
*ptr = reader.read_u8()? as i8; consume_value!(i8, |ptr| {
*ptr = proto_async::read_i8(stream).await?;
Ok(()) Ok(())
}), }),
Tag::Int32 => consume_value!(i32, |ptr| { Tag::Int32 =>
*ptr = reader.read_u32()? as i32; consume_value!(i32, |ptr| {
*ptr = proto_async::read_i32(stream).await?;
Ok(()) Ok(())
}), }),
Tag::Int64 | Tag::Float64 => consume_value!(i64, |ptr| { Tag::Int64 | Tag::Float64 =>
*ptr = reader.read_u64()? as i64; consume_value!(i64, |ptr| {
*ptr = proto_async::read_i64(stream).await?;
Ok(()) Ok(())
}), }),
Tag::String | Tag::Bytes | Tag::ByteArray => { Tag::String | Tag::Bytes | Tag::ByteArray => {
consume_value!(CMutSlice<u8>, |ptr| { consume_value!(CMutSlice<u8>, |ptr| {
let length = reader.read_u32()? as usize; let length = proto_async::read_i32(stream).await? as usize;
*ptr = CMutSlice::new(alloc(length) as *mut u8, length); *ptr = CMutSlice::new(alloc(length).await as *mut u8, length);
reader.read_exact((*ptr).as_mut())?; proto_async::read_chunk(stream, (*ptr).as_mut()).await?;
Ok(()) Ok(())
}) })
} }
@ -120,173 +139,169 @@ where
let mut it = it.clone(); let mut it = it.clone();
for _ in 0..arity { for _ in 0..arity {
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
recv_value(reader, tag, data, alloc)? recv_value(stream, tag, data, alloc).await?
} }
// Take into account any tail padding (if element(s) with largest alignment
// are not at the end).
*data = round_up_mut(*data, alignment); *data = round_up_mut(*data, alignment);
Ok(()) Ok(())
} }
Tag::List(it) => { Tag::List(it) => {
#[repr(C)] #[repr(C)]
struct List { struct List { elements: *mut (), length: usize }
elements: *mut (),
length: usize,
}
consume_value!(*mut List, |ptr_to_list| { consume_value!(*mut List, |ptr_to_list| {
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
let length = reader.read_u32()? as usize; let length = proto_async::read_i32(stream).await? as usize;
// To avoid multiple kernel CPU roundtrips, use a single allocation for
// both the pointer/length List (slice) and the backing storage for the
// elements. We can assume that alloc() is aligned suitably, so just
// need to take into account any extra padding required.
// (Note: At the time of writing, there will never actually be any types
// with alignment larger than 8 bytes, so storage_offset == 0 always.)
let list_size = 4 + 4; let list_size = 4 + 4;
let storage_offset = round_up(list_size, tag.alignment()); let storage_offset = round_up(list_size, tag.alignment());
let storage_size = tag.size() * length; let storage_size = tag.size() * length;
let allocation = alloc(storage_offset + storage_size) as *mut u8; let allocation = alloc(storage_offset + storage_size).await as *mut u8;
*ptr_to_list = allocation as *mut List; *ptr_to_list = allocation as *mut List;
let storage = allocation.offset(storage_offset as isize) as *mut (); let storage = allocation.offset(storage_offset as isize) as *mut ();
(**ptr_to_list).length = length; (**ptr_to_list).length = length;
(**ptr_to_list).elements = storage; (**ptr_to_list).elements = storage;
recv_elements(reader, tag, length, storage, alloc) recv_elements(stream, tag, length, storage, alloc).await
}) })
} }
Tag::Array(it, num_dims) => { Tag::Array(it, num_dims) => {
consume_value!(*mut (), |buffer| { consume_value!(*mut (), |buffer| {
// Deserialize length along each dimension and compute total number of
// elements.
let mut total_len: usize = 1; let mut total_len: usize = 1;
for _ in 0..num_dims { for _ in 0..num_dims {
let len = reader.read_u32()? as usize; let len = proto_async::read_i32(stream).await? as usize;
total_len *= len; total_len *= len;
consume_value!(usize, |ptr| *ptr = len) consume_value!(usize, |ptr| *ptr = len )
} }
// Allocate backing storage for elements; deserialize them.
let elt_tag = it.clone().next().expect("truncated tag"); let elt_tag = it.clone().next().expect("truncated tag");
*buffer = alloc(elt_tag.size() * total_len); *buffer = alloc(elt_tag.size() * total_len).await;
recv_elements(reader, elt_tag, total_len, *buffer, alloc) recv_elements(stream, elt_tag, total_len, *buffer, alloc).await
}) })
} }
Tag::Range(it) => { Tag::Range(it) => {
*data = round_up_mut(*data, tag.alignment()); *data = round_up_mut(*data, tag.alignment());
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
recv_value(reader, tag, data, alloc)?; recv_value(stream, tag, data, alloc).await?;
recv_value(reader, tag, data, alloc)?; recv_value(stream, tag, data, alloc).await?;
recv_value(reader, tag, data, alloc)?; recv_value(stream, tag, data, alloc).await?;
Ok(()) Ok(())
} }
Tag::Keyword(_) => unreachable!(), Tag::Keyword(_) => unreachable!(),
Tag::Object => unreachable!(), Tag::Object => unreachable!()
} }
} }
pub fn recv_return<'a, F, R>( pub async fn recv_return<F>(stream: &TcpStream, tag_bytes: &[u8], data: *mut (),
reader: &mut R, alloc: &impl Fn(usize) -> F)
tag_bytes: &'a [u8], -> Result<(), smoltcp::Error>
data: *mut (), where F: Future<Output=*mut ()>
alloc: &mut F,
) -> Result<&'a [u8], Error>
where
F: FnMut(usize) -> *mut (),
R: Read + ?Sized,
{ {
let mut it = TagIterator::new(tag_bytes); let mut it = TagIterator::new(tag_bytes);
trace!("recv ...->{}", it); trace!("recv ...->{}", it);
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
let mut data = data; let mut data = data;
unsafe { recv_value(reader, tag, &mut data, alloc)? }; unsafe { recv_value(stream, tag, &mut data, alloc).await? };
Ok(it.data) Ok(())
} }
unsafe fn send_elements<W>( unsafe fn send_elements<W>(writer: &mut W, elt_tag: Tag, length: usize, data: *const ())
writer: &mut W, -> Result<(), Error>
elt_tag: Tag, where W: Write + ?Sized
length: usize,
data: *const (),
write_tags: bool,
) -> Result<(), Error>
where
W: Write + ?Sized,
{ {
if write_tags {
writer.write_u8(elt_tag.as_u8())?; writer.write_u8(elt_tag.as_u8())?;
}
match elt_tag { match elt_tag {
// we cannot use NativeEndian::from_slice_i32 as the data is not mutable, // we cannot use NativeEndian::from_slice_i32 as the data is not mutable,
// and that is not needed as the data is already in native endian // and that is not needed as the data is already in native endian
Tag::Bool => { Tag::Bool => {
let slice = core::slice::from_raw_parts(data as *const u8, length); let slice = core::slice::from_raw_parts(data as *const u8, length);
writer.write_all(slice)?; writer.write_all(slice)?;
} },
Tag::Int32 => { Tag::Int32 => {
let slice = core::slice::from_raw_parts(data as *const u8, length * 4); let slice = core::slice::from_raw_parts(data as *const u8, length * 4);
writer.write_all(slice)?; writer.write_all(slice)?;
} },
Tag::Int64 | Tag::Float64 => { Tag::Int64 | Tag::Float64 => {
let slice = core::slice::from_raw_parts(data as *const u8, length * 8); let slice = core::slice::from_raw_parts(data as *const u8, length * 8);
writer.write_all(slice)?; writer.write_all(slice)?;
} },
_ => { _ => {
let mut data = data; let mut data = data;
for _ in 0..length { for _ in 0..length {
send_value(writer, elt_tag, &mut data, write_tags)?; send_value(writer, elt_tag, &mut data)?;
} }
} }
} }
Ok(()) Ok(())
} }
unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const (), write_tags: bool) -> Result<(), Error> unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
where W: Write + ?Sized { -> Result<(), Error>
where W: Write + ?Sized
{
macro_rules! consume_value { macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{ ($ty:ty, |$ptr:ident| $map:expr) => ({
let $ptr = align_ptr::<$ty>(*data); let $ptr = align_ptr::<$ty>(*data);
*data = $ptr.offset(1) as *const (); *data = $ptr.offset(1) as *const ();
$map $map
}}; })
} }
if write_tags {
writer.write_u8(tag.as_u8())?; writer.write_u8(tag.as_u8())?;
}
match tag { match tag {
Tag::None => Ok(()), Tag::None => Ok(()),
Tag::Bool => consume_value!(u8, |ptr| writer.write_u8(*ptr)), Tag::Bool =>
Tag::Int32 => consume_value!(u32, |ptr| writer.write_u32(*ptr)), consume_value!(u8, |ptr|
Tag::Int64 | Tag::Float64 => consume_value!(u64, |ptr| writer.write_u64(*ptr)), writer.write_u8(*ptr)),
Tag::String => consume_value!(CSlice<u8>, |ptr| { Tag::Int32 =>
writer.write_string(str::from_utf8((*ptr).as_ref()).unwrap()) consume_value!(u32, |ptr|
}), writer.write_u32(*ptr)),
Tag::Bytes | Tag::ByteArray => consume_value!(CSlice<u8>, |ptr| writer.write_bytes((*ptr).as_ref())), Tag::Int64 | Tag::Float64 =>
consume_value!(u64, |ptr|
writer.write_u64(*ptr)),
Tag::String =>
consume_value!(CSlice<u8>, |ptr|
writer.write_string(str::from_utf8((*ptr).as_ref()).unwrap())),
Tag::Bytes | Tag::ByteArray =>
consume_value!(CSlice<u8>, |ptr|
writer.write_bytes((*ptr).as_ref())),
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let mut it = it.clone(); let mut it = it.clone();
if write_tags {
writer.write_u8(arity)?; writer.write_u8(arity)?;
}
let mut max_alignment = 0; let mut max_alignment = 0;
for _ in 0..arity { for _ in 0..arity {
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
max_alignment = core::cmp::max(max_alignment, tag.alignment()); max_alignment = core::cmp::max(max_alignment, tag.alignment());
send_value(writer, tag, data, write_tags)? send_value(writer, tag, data)?
} }
*data = round_up_const(*data, max_alignment); *data = round_up_const(*data, max_alignment);
Ok(()) Ok(())
} }
Tag::List(it) => { Tag::List(it) => {
#[repr(C)] #[repr(C)]
struct List { struct List { elements: *const (), length: u32 }
elements: *const (),
length: u32,
}
consume_value!(&List, |ptr| { consume_value!(&List, |ptr| {
let length = (**ptr).length as usize; let length = (**ptr).length as usize;
writer.write_u32((*ptr).length)?; writer.write_u32((*ptr).length)?;
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
send_elements(writer, tag, length, (**ptr).elements, write_tags) send_elements(writer, tag, length, (**ptr).elements)
}) })
} }
Tag::Array(it, num_dims) => { Tag::Array(it, num_dims) => {
if write_tags {
writer.write_u8(num_dims)?; writer.write_u8(num_dims)?;
} consume_value!(*const(), |buffer| {
consume_value!(*const (), |buffer| {
let elt_tag = it.clone().next().expect("truncated tag"); let elt_tag = it.clone().next().expect("truncated tag");
let mut total_len = 1; let mut total_len = 1;
@ -297,49 +312,40 @@ where W: Write + ?Sized {
}) })
} }
let length = total_len as usize; let length = total_len as usize;
send_elements(writer, elt_tag, length, *buffer, write_tags) send_elements(writer, elt_tag, length, *buffer)
}) })
} }
Tag::Range(it) => { Tag::Range(it) => {
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
send_value(writer, tag, data, write_tags)?; send_value(writer, tag, data)?;
send_value(writer, tag, data, write_tags)?; send_value(writer, tag, data)?;
send_value(writer, tag, data, write_tags)?; send_value(writer, tag, data)?;
Ok(()) Ok(())
} }
Tag::Keyword(it) => { Tag::Keyword(it) => {
#[repr(C)] #[repr(C)]
struct Keyword<'a> { struct Keyword<'a> { name: CSlice<'a, u8> }
name: CSlice<'a, u8>,
}
consume_value!(Keyword, |ptr| { consume_value!(Keyword, |ptr| {
writer.write_string(str::from_utf8((*ptr).name.as_ref()).unwrap())?; writer.write_string(str::from_utf8((*ptr).name.as_ref()).unwrap())?;
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
let mut data = ptr.offset(1) as *const (); let mut data = ptr.offset(1) as *const ();
send_value(writer, tag, &mut data, write_tags) send_value(writer, tag, &mut data)
}) })
// Tag::Keyword never appears in composite types, so we don't have // Tag::Keyword never appears in composite types, so we don't have
// to accurately advance data. // to accurately advance data.
} }
Tag::Object => { Tag::Object => {
#[repr(C)] #[repr(C)]
struct Object { struct Object { id: u32 }
id: u32, consume_value!(*const Object, |ptr|
} writer.write_u32((**ptr).id))
consume_value!(*const Object, |ptr| writer.write_u32((**ptr).id))
} }
} }
} }
pub fn send_args<W>( pub fn send_args<W>(writer: &mut W, service: u32, tag_bytes: &[u8], data: *const *const ())
writer: &mut W, -> Result<(), Error>
service: u32, where W: Write + ?Sized
tag_bytes: &[u8],
data: *const *const (),
write_tags: bool,
) -> Result<(), Error>
where
W: Write + ?Sized,
{ {
let (arg_tags_bytes, return_tag_bytes) = split_tag(tag_bytes); let (arg_tags_bytes, return_tag_bytes) = split_tag(tag_bytes);
@ -351,9 +357,9 @@ where
for index in 0.. { for index in 0.. {
if let Some(arg_tag) = args_it.next() { if let Some(arg_tag) = args_it.next() {
let mut data = unsafe { *data.offset(index) }; let mut data = unsafe { *data.offset(index) };
unsafe { send_value(writer, arg_tag, &mut data, write_tags)? }; unsafe { send_value(writer, arg_tag, &mut data)? };
} else { } else {
break; break
} }
} }
writer.write_u8(0)?; writer.write_u8(0)?;
@ -362,12 +368,12 @@ where
Ok(()) Ok(())
} }
pub mod tag { mod tag {
use core::fmt; use core::fmt;
pub fn split_tag(tag_bytes: &[u8]) -> (&[u8], &[u8]) { pub fn split_tag(tag_bytes: &[u8]) -> (&[u8], &[u8]) {
let tag_separator = tag_bytes let tag_separator =
.iter() tag_bytes.iter()
.position(|&b| b == b':') .position(|&b| b == b':')
.expect("tag without a return separator"); .expect("tag without a return separator");
let (arg_tags_bytes, rest) = tag_bytes.split_at(tag_separator); let (arg_tags_bytes, rest) = tag_bytes.split_at(tag_separator);
@ -390,7 +396,7 @@ pub mod tag {
Array(TagIterator<'a>, u8), Array(TagIterator<'a>, u8),
Range(TagIterator<'a>), Range(TagIterator<'a>),
Keyword(TagIterator<'a>), Keyword(TagIterator<'a>),
Object, Object
} }
impl<'a> Tag<'a> { impl<'a> Tag<'a> {
@ -425,15 +431,14 @@ pub mod tag {
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let it = it.clone(); let it = it.clone();
it.take(arity.into()).map(|t| t.alignment()).max().unwrap() it.take(arity.into()).map(|t| t.alignment()).max().unwrap()
} },
Tag::Range(it) => { Tag::Range(it) => {
let it = it.clone(); let it = it.clone();
it.take(3).map(|t| t.alignment()).max().unwrap() it.take(3).map(|t| t.alignment()).max().unwrap()
} }
// the ptr/length(s) pair is basically CSlice // the ptr/length(s) pair is basically CSlice
Tag::Bytes | Tag::String | Tag::ByteArray | Tag::List(_) | Tag::Array(_, _) => { Tag::Bytes | Tag::String | Tag::ByteArray | Tag::List(_) | Tag::Array(_, _) =>
core::mem::align_of::<CSlice<()>>() core::mem::align_of::<CSlice<()>>(),
}
Tag::Keyword(_) => unreachable!("Tag::Keyword should not appear in composite types"), Tag::Keyword(_) => unreachable!("Tag::Keyword should not appear in composite types"),
Tag::Object => core::mem::align_of::<u32>(), Tag::Object => core::mem::align_of::<u32>(),
} }
@ -479,7 +484,7 @@ pub mod tag {
#[derive(Debug, Clone, Copy)] #[derive(Debug, Clone, Copy)]
pub struct TagIterator<'a> { pub struct TagIterator<'a> {
pub data: &'a [u8], data: &'a [u8]
} }
impl<'a> TagIterator<'a> { impl<'a> TagIterator<'a> {
@ -487,14 +492,13 @@ pub mod tag {
TagIterator { data } TagIterator { data }
} }
fn sub(&mut self, count: u8) -> TagIterator<'a> { fn sub(&mut self, count: u8) -> TagIterator<'a> {
let data = self.data; let data = self.data;
for _ in 0..count { for _ in 0..count {
self.next().expect("truncated tag"); self.next().expect("truncated tag");
} }
TagIterator { TagIterator { data: &data[..(data.len() - self.data.len())] }
data: &data[..(data.len() - self.data.len())],
}
} }
} }
@ -503,7 +507,7 @@ pub mod tag {
fn next(&mut self) -> Option<Tag<'a>> { fn next(&mut self) -> Option<Tag<'a>> {
if self.data.len() == 0 { if self.data.len() == 0 {
return None; return None
} }
let tag_byte = self.data[0]; let tag_byte = self.data[0];
@ -531,7 +535,7 @@ pub mod tag {
b'r' => Tag::Range(self.sub(1)), b'r' => Tag::Range(self.sub(1)),
b'k' => Tag::Keyword(self.sub(1)), b'k' => Tag::Keyword(self.sub(1)),
b'O' => Tag::Object, b'O' => Tag::Object,
_ => unreachable!(), _ => unreachable!()
}) })
} }
} }
@ -548,14 +552,22 @@ pub mod tag {
} }
match tag { match tag {
Tag::None => write!(f, "None")?, Tag::None =>
Tag::Bool => write!(f, "Bool")?, write!(f, "None")?,
Tag::Int32 => write!(f, "Int32")?, Tag::Bool =>
Tag::Int64 => write!(f, "Int64")?, write!(f, "Bool")?,
Tag::Float64 => write!(f, "Float64")?, Tag::Int32 =>
Tag::String => write!(f, "String")?, write!(f, "Int32")?,
Tag::Bytes => write!(f, "Bytes")?, Tag::Int64 =>
Tag::ByteArray => write!(f, "ByteArray")?, write!(f, "Int64")?,
Tag::Float64 =>
write!(f, "Float64")?,
Tag::String =>
write!(f, "String")?,
Tag::Bytes =>
write!(f, "Bytes")?,
Tag::ByteArray =>
write!(f, "ByteArray")?,
Tag::Tuple(it, _) => { Tag::Tuple(it, _) => {
write!(f, "Tuple(")?; write!(f, "Tuple(")?;
it.fmt(f)?; it.fmt(f)?;
@ -581,7 +593,8 @@ pub mod tag {
it.fmt(f)?; it.fmt(f)?;
write!(f, ")")?; write!(f, ")")?;
} }
Tag::Object => write!(f, "Object")?, Tag::Object =>
write!(f, "Object")?,
} }
} }

View File

@ -1,197 +0,0 @@
use alloc::boxed::Box;
use core::future::Future;
use async_recursion::async_recursion;
use byteorder::{ByteOrder, NativeEndian};
use cslice::CMutSlice;
use ksupport::rpc::{tag::{Tag, TagIterator},
*};
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp;
use log::trace;
use crate::proto_async;
/// Reads (deserializes) `length` array or list elements of type `tag` from `stream`,
/// writing them into the buffer given by `storage`.
///
/// `alloc` is used for nested allocations (if elements themselves contain
/// lists/arrays), see [recv_value].
#[async_recursion(?Send)]
async unsafe fn recv_elements<F>(
stream: &TcpStream,
elt_tag: Tag<'async_recursion>,
length: usize,
storage: *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
// List of simple types are special-cased in the protocol for performance.
match elt_tag {
Tag::Bool => {
let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length);
proto_async::read_chunk(stream, dest).await?;
}
Tag::Int32 => {
let ptr = storage as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest);
}
Tag::Int64 | Tag::Float64 => {
let ptr = storage as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
}
_ => {
let mut data = storage;
for _ in 0..length {
recv_value(stream, elt_tag, &mut data, alloc).await?
}
}
}
Ok(())
}
/// Reads (deserializes) a value of type `tag` from `stream`, writing the results to
/// the kernel-side buffer `data` (the passed pointer to which is incremented to point
/// past the just-received data). For nested allocations (lists/arrays), `alloc` is
/// invoked any number of times with the size of the required allocation as a parameter
/// (which is assumed to be correctly aligned for all payload types).
#[async_recursion(?Send)]
async unsafe fn recv_value<F>(
stream: &TcpStream,
tag: Tag<'async_recursion>,
data: &mut *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{
let $ptr = align_ptr_mut::<$ty>(*data);
*data = $ptr.offset(1) as *mut ();
$map
}};
}
match tag {
Tag::None => Ok(()),
Tag::Bool => consume_value!(i8, |ptr| {
*ptr = proto_async::read_i8(stream).await?;
Ok(())
}),
Tag::Int32 => consume_value!(i32, |ptr| {
*ptr = proto_async::read_i32(stream).await?;
Ok(())
}),
Tag::Int64 | Tag::Float64 => consume_value!(i64, |ptr| {
*ptr = proto_async::read_i64(stream).await?;
Ok(())
}),
Tag::String | Tag::Bytes | Tag::ByteArray => {
consume_value!(CMutSlice<u8>, |ptr| {
let length = proto_async::read_i32(stream).await? as usize;
*ptr = CMutSlice::new(alloc(length).await as *mut u8, length);
proto_async::read_chunk(stream, (*ptr).as_mut()).await?;
Ok(())
})
}
Tag::Tuple(it, arity) => {
let alignment = tag.alignment();
*data = round_up_mut(*data, alignment);
let mut it = it.clone();
for _ in 0..arity {
let tag = it.next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await?
}
// Take into account any tail padding (if element(s) with largest alignment
// are not at the end).
*data = round_up_mut(*data, alignment);
Ok(())
}
Tag::List(it) => {
#[repr(C)]
struct List {
elements: *mut (),
length: usize,
}
consume_value!(*mut List, |ptr_to_list| {
let tag = it.clone().next().expect("truncated tag");
let length = proto_async::read_i32(stream).await? as usize;
// To avoid multiple kernel CPU roundtrips, use a single allocation for
// both the pointer/length List (slice) and the backing storage for the
// elements. We can assume that alloc() is aligned suitably, so just
// need to take into account any extra padding required.
// (Note: At the time of writing, there will never actually be any types
// with alignment larger than 8 bytes, so storage_offset == 0 always.)
let list_size = 4 + 4;
let storage_offset = round_up(list_size, tag.alignment());
let storage_size = tag.size() * length;
let allocation = alloc(storage_offset + storage_size).await as *mut u8;
*ptr_to_list = allocation as *mut List;
let storage = allocation.offset(storage_offset as isize) as *mut ();
(**ptr_to_list).length = length;
(**ptr_to_list).elements = storage;
recv_elements(stream, tag, length, storage, alloc).await
})
}
Tag::Array(it, num_dims) => {
consume_value!(*mut (), |buffer| {
// Deserialize length along each dimension and compute total number of
// elements.
let mut total_len: usize = 1;
for _ in 0..num_dims {
let len = proto_async::read_i32(stream).await? as usize;
total_len *= len;
consume_value!(usize, |ptr| *ptr = len)
}
// Allocate backing storage for elements; deserialize them.
let elt_tag = it.clone().next().expect("truncated tag");
*buffer = alloc(elt_tag.size() * total_len).await;
recv_elements(stream, elt_tag, total_len, *buffer, alloc).await
})
}
Tag::Range(it) => {
*data = round_up_mut(*data, tag.alignment());
let tag = it.clone().next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
Ok(())
}
Tag::Keyword(_) => unreachable!(),
Tag::Object => unreachable!(),
}
}
pub async fn recv_return<F>(
stream: &TcpStream,
tag_bytes: &[u8],
data: *mut (),
alloc: &impl Fn(usize) -> F,
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
let mut it = TagIterator::new(tag_bytes);
trace!("recv ...->{}", it);
let tag = it.next().expect("truncated tag");
let mut data = data;
unsafe { recv_value(stream, tag, &mut data, alloc).await? };
Ok(())
}

View File

@ -1,17 +1,17 @@
use cslice::CSlice;
use vcell::VolatileCell;
use libcortex_a9::asm;
use crate::artiq_raise;
use core::sync::atomic::{fence, Ordering}; use core::sync::atomic::{fence, Ordering};
use cslice::CSlice; use crate::pl::csr;
use libcortex_a9::asm; use crate::rtio_mgt::resolve_channel_name;
use vcell::VolatileCell;
use crate::{artiq_raise, pl::csr, resolve_channel_name, rtio_core};
pub const RTIO_O_STATUS_WAIT: i32 = 1; pub const RTIO_O_STATUS_WAIT: i32 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: i32 = 2; pub const RTIO_O_STATUS_UNDERFLOW: i32 = 2;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: i32 = 4; pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: i32 = 4;
pub const RTIO_I_STATUS_WAIT_EVENT: i32 = 1; pub const RTIO_I_STATUS_WAIT_EVENT: i32 = 1;
pub const RTIO_I_STATUS_OVERFLOW: i32 = 2; pub const RTIO_I_STATUS_OVERFLOW: i32 = 2;
#[allow(unused)]
pub const RTIO_I_STATUS_WAIT_STATUS: i32 = 4; // TODO pub const RTIO_I_STATUS_WAIT_STATUS: i32 = 4; // TODO
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: i32 = 8; pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: i32 = 8;
@ -47,18 +47,18 @@ static mut TRANSACTION_BUFFER: Transaction = Transaction {
reply_timestamp: VolatileCell::new(0), reply_timestamp: VolatileCell::new(0),
padding0: [0; 2], padding0: [0; 2],
padding1: [0; 2], padding1: [0; 2],
padding2: [0; 2], padding2: [0; 2]
}; };
pub extern "C" fn init() { pub extern fn init() {
unsafe { unsafe {
rtio_core::reset_write(1); csr::rtio_core::reset_write(1);
csr::rtio::engine_addr_base_write(&TRANSACTION_BUFFER as *const Transaction as u32); csr::rtio::engine_addr_base_write(&TRANSACTION_BUFFER as *const Transaction as u32);
csr::rtio::enable_write(1); csr::rtio::enable_write(1);
} }
} }
pub extern "C" fn get_counter() -> i64 { pub extern fn get_counter() -> i64 {
unsafe { unsafe {
csr::rtio::counter_update_write(1); csr::rtio::counter_update_write(1);
csr::rtio::counter_read() as i64 csr::rtio::counter_read() as i64
@ -67,15 +67,15 @@ pub extern "C" fn get_counter() -> i64 {
static mut NOW: i64 = 0; static mut NOW: i64 = 0;
pub extern "C" fn now_mu() -> i64 { pub extern fn now_mu() -> i64 {
unsafe { NOW } unsafe { NOW }
} }
pub extern "C" fn at_mu(t: i64) { pub extern fn at_mu(t: i64) {
unsafe { NOW = t } unsafe { NOW = t }
} }
pub extern "C" fn delay_mu(dt: i64) { pub extern fn delay_mu(dt: i64) {
unsafe { NOW += dt } unsafe { NOW += dt }
} }
@ -87,34 +87,18 @@ unsafe fn process_exceptional_status(channel: i32, status: i32) {
while csr::rtio::o_status_read() as i32 & RTIO_O_STATUS_WAIT != 0 {} while csr::rtio::o_status_read() as i32 & RTIO_O_STATUS_WAIT != 0 {}
} }
if status & RTIO_O_STATUS_UNDERFLOW != 0 { if status & RTIO_O_STATUS_UNDERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOUnderflow",
"RTIOUnderflow", format!("RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, timestamp, timestamp - get_counter());
"RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
timestamp,
timestamp - get_counter()
);
} }
if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", format!("RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( timestamp, channel as i64, 0);
"RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
timestamp,
channel as i64,
0
);
} }
} }
pub extern "C" fn output(target: i32, data: i32) { pub extern fn output(target: i32, data: i32) {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -142,7 +126,7 @@ pub extern "C" fn output(target: i32, data: i32) {
} }
} }
pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) { pub extern fn output_wide(target: i32, data: CSlice<i32>) {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -159,7 +143,7 @@ pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
@ -170,7 +154,7 @@ pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) {
} }
} }
pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 { pub extern fn input_timestamp(timeout: i64, channel: i32) -> i64 {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -187,45 +171,29 @@ pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_WAIT_EVENT != 0 { if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return -1; return -1
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TRANSACTION_BUFFER.reply_timestamp.get() TRANSACTION_BUFFER.reply_timestamp.get()
} }
} }
pub extern "C" fn input_data(channel: i32) -> i32 { pub extern fn input_data(channel: i32) -> i32 {
unsafe { unsafe {
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -241,42 +209,26 @@ pub extern "C" fn input_data(channel: i32) -> i32 {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TRANSACTION_BUFFER.reply_data.get() TRANSACTION_BUFFER.reply_data.get()
} }
} }
pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData { pub extern fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData {
unsafe { unsafe {
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -292,35 +244,19 @@ pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> Timestam
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TimestampedData { TimestampedData {

View File

@ -1,20 +1,14 @@
#[cfg(not(feature = "target_ebaz4205"))] use log::{info, warn};
use libboard_zynq::timer::GlobalTimer;
use embedded_hal::blocking::delay::DelayMs; use embedded_hal::blocking::delay::DelayMs;
#[cfg(has_si5324)] use libconfig::Config;
use ksupport::i2c;
#[cfg(not(feature = "target_ebaz4205"))]
use libboard_artiq::pl; use libboard_artiq::pl;
#[cfg(has_si5324)] #[cfg(has_si5324)]
use libboard_artiq::si5324;
#[cfg(has_si549)]
use libboard_artiq::si549;
#[cfg(has_si5324)]
use libboard_zynq::i2c::I2c; use libboard_zynq::i2c::I2c;
use libboard_zynq::timer::GlobalTimer; #[cfg(has_si5324)]
use libconfig::Config; use crate::i2c;
use log::{info, warn}; #[cfg(has_si5324)]
#[cfg(feature = "target_ebaz4205")] use libboard_artiq::si5324;
use {libboard_zynq::slcr, libregister::RegisterRW};
#[derive(Debug, PartialEq, Copy, Clone)] #[derive(Debug, PartialEq, Copy, Clone)]
#[allow(non_camel_case_types)] #[allow(non_camel_case_types)]
@ -25,7 +19,6 @@ pub enum RtioClock {
Int_150, Int_150,
Ext0_Bypass, Ext0_Bypass,
Ext0_Synth0_10to125, Ext0_Synth0_10to125,
Ext0_Synth0_80to125,
Ext0_Synth0_100to125, Ext0_Synth0_100to125,
Ext0_Synth0_125to125, Ext0_Synth0_125to125,
} }
@ -42,7 +35,6 @@ fn get_rtio_clock_cfg(cfg: &Config) -> RtioClock {
"ext0_bypass_125" => RtioClock::Ext0_Bypass, "ext0_bypass_125" => RtioClock::Ext0_Bypass,
"ext0_bypass_100" => RtioClock::Ext0_Bypass, "ext0_bypass_100" => RtioClock::Ext0_Bypass,
"ext0_synth0_10to125" => RtioClock::Ext0_Synth0_10to125, "ext0_synth0_10to125" => RtioClock::Ext0_Synth0_10to125,
"ext0_synth0_80to125" => RtioClock::Ext0_Synth0_80to125,
"ext0_synth0_100to125" => RtioClock::Ext0_Synth0_100to125, "ext0_synth0_100to125" => RtioClock::Ext0_Synth0_100to125,
"ext0_synth0_125to125" => RtioClock::Ext0_Synth0_125to125, "ext0_synth0_125to125" => RtioClock::Ext0_Synth0_125to125,
_ => { _ => {
@ -50,16 +42,17 @@ fn get_rtio_clock_cfg(cfg: &Config) -> RtioClock {
RtioClock::Default RtioClock::Default
} }
}; };
} else { }
else {
warn!("error reading configuration. Falling back to default."); warn!("error reading configuration. Falling back to default.");
} }
if res == RtioClock::Default { if res == RtioClock::Default {
#[cfg(rtio_frequency = "100.0")] #[cfg(rtio_frequency="100.0")]
{ {
warn!("Using default configuration - internal 100MHz RTIO clock."); warn!("Using default configuration - internal 100MHz RTIO clock.");
return RtioClock::Int_100; return RtioClock::Int_100;
} }
#[cfg(rtio_frequency = "125.0")] #[cfg(rtio_frequency="125.0")]
{ {
warn!("Using default configuration - internal 125MHz RTIO clock."); warn!("Using default configuration - internal 125MHz RTIO clock.");
return RtioClock::Int_125; return RtioClock::Int_125;
@ -73,7 +66,7 @@ fn get_rtio_clock_cfg(cfg: &Config) -> RtioClock {
res res
} }
#[cfg(not(any(has_drtio, feature = "target_ebaz4205")))] #[cfg(not(has_drtio))]
fn init_rtio(timer: &mut GlobalTimer) { fn init_rtio(timer: &mut GlobalTimer) {
info!("Switching SYS clocks..."); info!("Switching SYS clocks...");
unsafe { unsafe {
@ -81,7 +74,7 @@ fn init_rtio(timer: &mut GlobalTimer) {
} }
// if it's not locked, it will hang at the CSR. // if it's not locked, it will hang at the CSR.
timer.delay_ms(50); // wait for CPLL/QPLL/SYS PLL lock timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() }; let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 { if clk == 1 {
info!("SYS CLK switched successfully"); info!("SYS CLK switched successfully");
@ -92,15 +85,17 @@ fn init_rtio(timer: &mut GlobalTimer) {
pl::csr::rtio_core::reset_phy_write(1); pl::csr::rtio_core::reset_phy_write(1);
} }
info!("SYS PLL locked"); info!("SYS PLL locked");
} }
#[cfg(has_drtio)] #[cfg(has_drtio)]
fn init_drtio(timer: &mut GlobalTimer) { fn init_drtio(timer: &mut GlobalTimer)
{
unsafe { unsafe {
pl::csr::gt_drtio::stable_clkin_write(1); pl::csr::drtio_transceiver::stable_clkin_write(1);
} }
timer.delay_ms(50); // wait for CPLL/QPLL/SYS PLL lock timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() }; let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 { if clk == 1 {
info!("SYS CLK switched successfully"); info!("SYS CLK switched successfully");
@ -109,10 +104,7 @@ fn init_drtio(timer: &mut GlobalTimer) {
} }
unsafe { unsafe {
pl::csr::rtio_core::reset_phy_write(1); pl::csr::rtio_core::reset_phy_write(1);
pl::csr::gt_drtio::txenable_write(0xffffffffu32 as _); pl::csr::drtio_transceiver::txenable_write(0xffffffffu32 as _);
#[cfg(has_drtio_eem)]
pl::csr::eem_transceiver::txenable_write(0xffffffffu32 as _);
} }
} }
@ -123,372 +115,137 @@ const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin1;
#[cfg(has_si5324)] #[cfg(has_si5324)]
fn setup_si5324(i2c: &mut I2c, timer: &mut GlobalTimer, clk: RtioClock) { fn setup_si5324(i2c: &mut I2c, timer: &mut GlobalTimer, clk: RtioClock) {
let (si5324_settings, si5324_ref_input) = match clk { let (si5324_settings, si5324_ref_input) = match clk {
RtioClock::Ext0_Synth0_10to125 => { RtioClock::Ext0_Synth0_10to125 => { // 125 MHz output from 10 MHz CLKINx reference, 504 Hz BW
// 125 MHz output from 10 MHz CLKINx reference, 504 Hz BW
info!("using 10MHz reference to make 125MHz RTIO clock with PLL"); info!("using 10MHz reference to make 125MHz RTIO clock with PLL");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 10, n1_hs : 10,
nc1_ls: 4, nc1_ls : 4,
n2_hs: 10, n2_hs : 10,
n2_ls: 300, n2_ls : 300,
n31: 6, n31 : 6,
n32: 6, n32 : 6,
bwsel: 4, bwsel : 4,
crystal_as_ckin2: false, crystal_as_ckin2: false
}, },
SI5324_EXT_INPUT, SI5324_EXT_INPUT
) )
}
RtioClock::Ext0_Synth0_80to125 => {
// 125 MHz output from 80 MHz CLKINx reference, 611 Hz BW
info!("using 80MHz reference to make 125MHz RTIO clock with PLL");
(
si5324::FrequencySettings {
n1_hs: 4,
nc1_ls: 10,
n2_hs: 10,
n2_ls: 250,
n31: 40,
n32: 40,
bwsel: 4,
crystal_as_ckin2: false,
}, },
SI5324_EXT_INPUT, RtioClock::Ext0_Synth0_100to125 => { // 125MHz output, from 100MHz CLKINx reference, 586 Hz loop bandwidth
)
}
RtioClock::Ext0_Synth0_100to125 => {
// 125MHz output, from 100MHz CLKINx reference, 586 Hz loop bandwidth
info!("using 100MHz reference to make 125MHz RTIO clock with PLL"); info!("using 100MHz reference to make 125MHz RTIO clock with PLL");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 10, n1_hs : 10,
nc1_ls: 4, nc1_ls : 4,
n2_hs: 10, n2_hs : 10,
n2_ls: 260, n2_ls : 260,
n31: 52, n31 : 52,
n32: 52, n32 : 52,
bwsel: 4, bwsel : 4,
crystal_as_ckin2: false, crystal_as_ckin2: false
}, },
SI5324_EXT_INPUT, SI5324_EXT_INPUT
) )
} },
RtioClock::Ext0_Synth0_125to125 => { RtioClock::Ext0_Synth0_125to125 => { // 125MHz output, from 125MHz CLKINx reference, 606 Hz loop bandwidth
// 125MHz output, from 125MHz CLKINx reference, 606 Hz loop bandwidth
info!("using 125MHz reference to make 125MHz RTIO clock with PLL"); info!("using 125MHz reference to make 125MHz RTIO clock with PLL");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 5, n1_hs : 5,
nc1_ls: 8, nc1_ls : 8,
n2_hs: 7, n2_hs : 7,
n2_ls: 360, n2_ls : 360,
n31: 63, n31 : 63,
n32: 63, n32 : 63,
bwsel: 4, bwsel : 4,
crystal_as_ckin2: false, crystal_as_ckin2: false
}, },
SI5324_EXT_INPUT, SI5324_EXT_INPUT
) )
} },
RtioClock::Int_150 => { RtioClock::Int_150 => { // 150MHz output, from crystal
// 150MHz output, from crystal
info!("using internal 150MHz RTIO clock"); info!("using internal 150MHz RTIO clock");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 9, n1_hs : 9,
nc1_ls: 4, nc1_ls : 4,
n2_hs: 10, n2_hs : 10,
n2_ls: 33732, n2_ls : 33732,
n31: 7139, n31 : 7139,
n32: 7139, n32 : 7139,
bwsel: 3, bwsel : 3,
crystal_as_ckin2: true, crystal_as_ckin2: true
}, },
si5324::Input::Ckin2, si5324::Input::Ckin2
) )
} },
RtioClock::Int_100 => { RtioClock::Int_100 => { // 100MHz output, from crystal
// 100MHz output, from crystal
info!("using internal 100MHz RTIO clock"); info!("using internal 100MHz RTIO clock");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 9, n1_hs : 9,
nc1_ls: 6, nc1_ls : 6,
n2_hs: 10, n2_hs : 10,
n2_ls: 33732, n2_ls : 33732,
n31: 7139, n31 : 7139,
n32: 7139, n32 : 7139,
bwsel: 3, bwsel : 3,
crystal_as_ckin2: true, crystal_as_ckin2: true
}, },
si5324::Input::Ckin2, si5324::Input::Ckin2
) )
} },
RtioClock::Int_125 => { RtioClock::Int_125 => { // 125MHz output, from crystal, 7 Hz
// 125MHz output, from crystal, 7 Hz
info!("using internal 125MHz RTIO clock"); info!("using internal 125MHz RTIO clock");
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 10, n1_hs : 10,
nc1_ls: 4, nc1_ls : 4,
n2_hs: 10, n2_hs : 10,
n2_ls: 19972, n2_ls : 19972,
n31: 4565, n31 : 4565,
n32: 4565, n32 : 4565,
bwsel: 4, bwsel : 4,
crystal_as_ckin2: true, crystal_as_ckin2: true
}, },
si5324::Input::Ckin2, si5324::Input::Ckin2
) )
} },
_ => { _ => { // same setting as Int_125, but fallback to default
// same setting as Int_125, but fallback to default warn!("rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.", clk);
warn!(
"rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.",
clk
);
( (
si5324::FrequencySettings { si5324::FrequencySettings {
n1_hs: 10, n1_hs : 10,
nc1_ls: 4, nc1_ls : 4,
n2_hs: 10, n2_hs : 10,
n2_ls: 19972, n2_ls : 19972,
n31: 4565, n31 : 4565,
n32: 4565, n32 : 4565,
bwsel: 4, bwsel : 4,
crystal_as_ckin2: true, crystal_as_ckin2: true
}, },
si5324::Input::Ckin2, si5324::Input::Ckin2
) )
} }
}; };
si5324::setup(i2c, &si5324_settings, si5324_ref_input, timer).expect("cannot initialize Si5324"); si5324::setup(i2c, &si5324_settings, si5324_ref_input, timer).expect("cannot initialize Si5324");
} }
#[cfg(all(has_si549, has_wrpll))]
fn wrpll_setup(timer: &mut GlobalTimer, clk: RtioClock, si549_settings: &si549::FrequencySetting) {
// register values are directly copied from preconfigured mmcm
let (mmcm_setting, mmcm_bypass) = match clk {
RtioClock::Ext0_Synth0_10to125 => (
si549::wrpll_refclk::MmcmSetting {
// CLKFBOUT_MULT = 62.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 5
clkout0_reg1: 0x1083,
clkout0_reg2: 0x0080,
clkfbout_reg1: 0x179e,
clkfbout_reg2: 0x4c00,
div_reg: 0x1041,
lock_reg1: 0x00fa,
lock_reg2: 0x7c01,
lock_reg3: 0xffe9,
power_reg: 0x9900,
filt_reg1: 0x1008,
filt_reg2: 0x8800,
},
false,
),
RtioClock::Ext0_Synth0_80to125 => (
si549::wrpll_refclk::MmcmSetting {
// CLKFBOUT_MULT = 15.625, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
clkout0_reg1: 0x1145,
clkout0_reg2: 0x0000,
clkfbout_reg1: 0x11c7,
clkfbout_reg2: 0x5880,
div_reg: 0x1041,
lock_reg1: 0x028a,
lock_reg2: 0x7c01,
lock_reg3: 0xffe9,
power_reg: 0x9900,
filt_reg1: 0x9908,
filt_reg2: 0x8100,
},
false,
),
RtioClock::Ext0_Synth0_100to125 => (
si549::wrpll_refclk::MmcmSetting {
// CLKFBOUT_MULT = 12.5, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
clkout0_reg1: 0x1145,
clkout0_reg2: 0x0000,
clkfbout_reg1: 0x1145,
clkfbout_reg2: 0x4c00,
div_reg: 0x1041,
lock_reg1: 0x0339,
lock_reg2: 0x7c01,
lock_reg3: 0xffe9,
power_reg: 0x9900,
filt_reg1: 0x9108,
filt_reg2: 0x0100,
},
false,
),
RtioClock::Ext0_Synth0_125to125 => (
si549::wrpll_refclk::MmcmSetting {
// CLKFBOUT_MULT = 10, DIVCLK_DIVIDE = 1 , CLKOUT0_DIVIDE = 10
clkout0_reg1: 0x1145,
clkout0_reg2: 0x0000,
clkfbout_reg1: 0x1145,
clkfbout_reg2: 0x0000,
div_reg: 0x1041,
lock_reg1: 0x03e8,
lock_reg2: 0x7001,
lock_reg3: 0xf3e9,
power_reg: 0x0100,
filt_reg1: 0x9908,
filt_reg2: 0x1100,
},
true,
),
_ => unreachable!(),
};
si549::helper_setup(timer, &si549_settings).expect("cannot initialize helper Si549");
si549::wrpll_refclk::setup(timer, mmcm_setting, mmcm_bypass).expect("cannot initialize ref clk for wrpll");
si549::wrpll::select_recovered_clock(true, timer);
}
#[cfg(has_si549)]
fn get_si549_setting(clk: RtioClock) -> si549::FrequencySetting {
match clk {
RtioClock::Ext0_Synth0_10to125 => {
info!("using 10MHz reference to make 125MHz RTIO clock with WRPLL");
}
RtioClock::Ext0_Synth0_80to125 => {
info!("using 80MHz reference to make 125MHz RTIO clock with WRPLL");
}
RtioClock::Ext0_Synth0_100to125 => {
info!("using 100MHz reference to make 125MHz RTIO clock with WRPLL");
}
RtioClock::Ext0_Synth0_125to125 => {
info!("using 125MHz reference to make 125MHz RTIO clock with WRPLL");
}
RtioClock::Int_100 => {
info!("using internal 100MHz RTIO clock");
}
RtioClock::Int_125 => {
info!("using internal 125MHz RTIO clock");
}
_ => {
warn!(
"rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.",
clk
);
}
};
match clk {
RtioClock::Int_100 => {
si549::FrequencySetting {
main: si549::DividerConfig {
hsdiv: 0x06C,
lsdiv: 0,
fbdiv: 0x046C5F49797,
},
helper: si549::DividerConfig {
// 100MHz*32767/32768
hsdiv: 0x06C,
lsdiv: 0,
fbdiv: 0x046C5670BBD,
},
}
}
_ => {
// Everything else use 125MHz
si549::FrequencySetting {
main: si549::DividerConfig {
hsdiv: 0x058,
lsdiv: 0,
fbdiv: 0x04815791F25,
},
helper: si549::DividerConfig {
// 125MHz*32767/32768
hsdiv: 0x058,
lsdiv: 0,
fbdiv: 0x04814E8F442,
},
}
}
}
}
#[cfg(feature = "target_ebaz4205")]
fn set_fclk0_freq(clk: RtioClock, cfg: &Config) {
let io_pll_freq: u32 = 1_000_000_000; // Hardcoded in zynq-rs
let mut target_freq = 0;
let mut divisor0 = 1u8;
match clk {
RtioClock::Int_100 => {
target_freq = 100_000_000;
divisor0 = 10;
}
RtioClock::Int_125 => {
target_freq = 125_000_000;
divisor0 = 8;
}
_ => {
warn!("Unsupported RTIO Clock: '{:?}'", clk);
return;
}
}
slcr::RegisterBlock::unlocked(|slcr| {
slcr.fpga0_clk_ctrl.modify(|_, w| w.divisor0(divisor0));
});
info!(
"Set FCLK0 to {:.2} MHz (target: {} MHz).",
io_pll_freq as f64 / divisor0 as f64,
target_freq / 1_000_000
);
}
pub fn init(timer: &mut GlobalTimer, cfg: &Config) { pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
let clk = get_rtio_clock_cfg(cfg); let clk = get_rtio_clock_cfg(cfg);
#[cfg(has_si5324)] #[cfg(has_si5324)]
{ {
let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() }; let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
match clk { match clk {
RtioClock::Ext0_Bypass => { RtioClock::Ext0_Bypass => si5324::bypass(i2c, SI5324_EXT_INPUT, timer).expect("cannot bypass Si5324"),
info!("bypassing the PLL for RTIO clock");
si5324::bypass(i2c, SI5324_EXT_INPUT, timer).expect("cannot bypass Si5324")
}
_ => setup_si5324(i2c, timer, clk), _ => setup_si5324(i2c, timer, clk),
} }
} }
#[cfg(has_si549)]
let si549_settings = get_si549_setting(clk);
#[cfg(has_si549)]
si549::main_setup(timer, &si549_settings).expect("cannot initialize main Si549");
#[cfg(has_drtio)] #[cfg(has_drtio)]
init_drtio(timer); init_drtio(timer);
#[cfg(not(any(has_drtio, feature = "target_ebaz4205")))] #[cfg(not(has_drtio))]
init_rtio(timer); init_rtio(timer);
#[cfg(feature = "target_ebaz4205")]
{
match clk {
RtioClock::Int_100 | RtioClock::Int_125 => {
set_fclk0_freq(clk, cfg);
}
_ => {} // Not set for external clocks
}
}
#[cfg(all(has_si549, has_wrpll))]
{
// SYS CLK switch will reset CSRs that are used by WRPLL
match clk {
RtioClock::Ext0_Synth0_10to125
| RtioClock::Ext0_Synth0_80to125
| RtioClock::Ext0_Synth0_100to125
| RtioClock::Ext0_Synth0_125to125 => {
wrpll_setup(timer, clk, &si549_settings);
}
_ => {}
}
}
} }

214
src/runtime/src/rtio_csr.rs Normal file
View File

@ -0,0 +1,214 @@
use core::ptr::{read_volatile, write_volatile};
use cslice::CSlice;
use crate::artiq_raise;
use crate::pl::csr;
use crate::rtio_mgt::resolve_channel_name;
pub const RTIO_O_STATUS_WAIT: u8 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: u8 = 2;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: u8 = 4;
pub const RTIO_I_STATUS_WAIT_EVENT: u8 = 1;
pub const RTIO_I_STATUS_OVERFLOW: u8 = 2;
pub const RTIO_I_STATUS_WAIT_STATUS: u8 = 4;
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: u8 = 8;
#[repr(C)]
pub struct TimestampedData {
timestamp: i64,
data: i32,
}
pub extern fn init() {
unsafe {
csr::rtio_core::reset_write(1);
}
}
pub extern fn get_counter() -> i64 {
unsafe {
csr::rtio::counter_update_write(1);
csr::rtio::counter_read() as i64
}
}
pub extern fn now_mu() -> i64 {
unsafe {
csr::rtio::now_read() as i64
}
}
pub extern fn at_mu(t: i64) {
unsafe {
csr::rtio::now_write(t as u64);
}
}
pub extern fn delay_mu(dt: i64) {
unsafe {
csr::rtio::now_write(csr::rtio::now_read() + dt as u64);
}
}
// writing the LSB of o_data (offset=0) triggers the RTIO write
#[inline(always)]
pub unsafe fn rtio_o_data_write(offset: usize, data: u32) {
write_volatile(
csr::rtio::O_DATA_ADDR.offset((csr::rtio::O_DATA_SIZE - 1 - offset) as isize),
data);
}
#[inline(always)]
pub unsafe fn rtio_i_data_read(offset: usize) -> u32 {
read_volatile(
csr::rtio::I_DATA_ADDR.offset((csr::rtio::I_DATA_SIZE - 1 - offset) as isize))
}
#[inline(never)]
unsafe fn process_exceptional_status(channel: i32, status: u8) {
let timestamp = csr::rtio::now_read() as i64;
if status & RTIO_O_STATUS_WAIT != 0 {
while csr::rtio::o_status_read() & RTIO_O_STATUS_WAIT != 0 {}
}
if status & RTIO_O_STATUS_UNDERFLOW != 0 {
artiq_raise!("RTIOUnderflow",
format!("RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu", channel, resolve_channel_name(channel as u32)),
channel as i64, timestamp, timestamp - get_counter());
}
if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!("RTIODestinationUnreachable",
format!("RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
timestamp, channel as i64, 0);
}
}
pub extern fn output(target: i32, data: i32) {
unsafe {
csr::rtio::target_write(target as u32);
// writing target clears o_data
rtio_o_data_write(0, data as _);
let status = csr::rtio::o_status_read();
if status != 0 {
process_exceptional_status(target >> 8, status);
}
}
}
pub extern fn output_wide(target: i32, data: CSlice<i32>) {
unsafe {
csr::rtio::target_write(target as u32);
// writing target clears o_data
for i in (0..data.len()).rev() {
rtio_o_data_write(i, data[i] as _)
}
let status = csr::rtio::o_status_read();
if status != 0 {
process_exceptional_status(target >> 8, status);
}
}
}
pub extern fn input_timestamp(timeout: i64, channel: i32) -> i64 {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!("RTIOOverflow",
format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return -1
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!("RTIODestinationUnreachable",
format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
csr::rtio::i_timestamp_read() as i64
}
}
pub extern fn input_data(channel: i32) -> i32 {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(0xffffffff_ffffffff);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!("RTIOOverflow",
format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!("RTIODestinationUnreachable",
format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
rtio_i_data_read(0) as i32
}
}
pub extern fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData {
unsafe {
csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64);
let mut status = RTIO_I_STATUS_WAIT_STATUS;
while status & RTIO_I_STATUS_WAIT_STATUS != 0 {
status = csr::rtio::i_status_read();
}
if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!("RTIOOverflow",
format!("RTIO input overflow on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return TimestampedData { timestamp: -1, data: 0 }
}
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!("RTIODestinationUnreachable",
format!("RTIO destination unreachable, input, on channel 0x{:04x}:{}", channel, resolve_channel_name(channel as u32)),
channel as i64, 0, 0);
}
TimestampedData {
timestamp: csr::rtio::i_timestamp_read() as i64,
data: rtio_i_data_read(0) as i32
}
}
}
pub fn write_log(data: &[i8]) {
unsafe {
csr::rtio::target_write(csr::CONFIG_RTIO_LOG_CHANNEL << 8);
let mut word: u32 = 0;
for i in 0..data.len() {
word <<= 8;
word |= data[i] as u32;
if i % 4 == 3 {
rtio_o_data_write(0, word);
word = 0;
}
}
if word != 0 {
rtio_o_data_write(0, word);
}
}
}

View File

@ -1,361 +0,0 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String, vec::Vec};
#[cfg(has_drtio)]
use core::mem;
use ksupport::kernel::DmaRecorder;
#[cfg(has_drtio)]
use libasync::task;
use libboard_artiq::drtio_routing::RoutingTable;
use libboard_zynq::timer::GlobalTimer;
use libcortex_a9::{cache::dcci_slice, mutex::Mutex};
const ALIGNMENT: usize = 16 * 8;
static DMA_RECORD_STORE: Mutex<BTreeMap<String, (u32, Vec<u8>, i64)>> = Mutex::new(BTreeMap::new());
#[cfg(has_drtio)]
pub mod remote_dma {
use libboard_zynq::time::Milliseconds;
use log::error;
use super::*;
use crate::rtio_mgt::drtio;
#[derive(Debug, PartialEq, Clone)]
pub enum RemoteState {
NotLoaded,
Loaded,
PlaybackEnded { error: u8, channel: u32, timestamp: u64 },
}
#[derive(Debug, Clone)]
struct RemoteTrace {
trace: Vec<u8>,
pub state: RemoteState,
}
impl From<Vec<u8>> for RemoteTrace {
fn from(trace: Vec<u8>) -> Self {
RemoteTrace {
trace: trace,
state: RemoteState::NotLoaded,
}
}
}
impl RemoteTrace {
pub fn get_trace(&self) -> &Vec<u8> {
&self.trace
}
}
// represents all traces for a given ID
struct TraceSet {
id: u32,
done_count: Mutex<usize>,
traces: Mutex<BTreeMap<u8, RemoteTrace>>,
}
impl TraceSet {
pub fn new(id: u32, traces: BTreeMap<u8, Vec<u8>>) -> TraceSet {
let mut trace_map: BTreeMap<u8, RemoteTrace> = BTreeMap::new();
for (destination, trace) in traces {
trace_map.insert(destination, trace.into());
}
TraceSet {
id: id,
done_count: Mutex::new(0),
traces: Mutex::new(trace_map),
}
}
pub async fn await_done(&self, timeout: Option<u64>, timer: GlobalTimer) -> Result<RemoteState, &'static str> {
let timeout_ms = Milliseconds(timeout.unwrap_or(10_000));
let limit = timer.get_time() + timeout_ms;
while (timer.get_time() < limit)
& (*(self.done_count.async_lock().await) < self.traces.async_lock().await.len())
{
task::r#yield().await;
}
if timer.get_time() >= limit {
error!("Remote DMA await done timed out");
return Err("Timed out waiting for results.");
}
let mut playback_state: RemoteState = RemoteState::PlaybackEnded {
error: 0,
channel: 0,
timestamp: 0,
};
let mut lock = self.traces.async_lock().await;
let trace_iter = lock.iter_mut();
for (_dest, trace) in trace_iter {
match trace.state {
RemoteState::PlaybackEnded {
error: e,
channel: _c,
timestamp: _ts,
} => {
if e != 0 {
playback_state = trace.state.clone();
}
}
_ => (),
}
trace.state = RemoteState::Loaded;
}
Ok(playback_state)
}
pub async fn upload_traces(
&mut self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
) {
let mut lock = self.traces.async_lock().await;
let trace_iter = lock.iter_mut();
for (destination, trace) in trace_iter {
match drtio::ddma_upload_trace(
aux_mutex,
routing_table,
timer,
self.id,
*destination,
trace.get_trace(),
)
.await
{
Ok(_) => trace.state = RemoteState::Loaded,
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
}
*(self.done_count.async_lock().await) = 0;
}
pub async fn erase(&mut self, aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer) {
let lock = self.traces.async_lock().await;
let trace_iter = lock.keys();
for destination in trace_iter {
match drtio::ddma_send_erase(aux_mutex, routing_table, timer, self.id, *destination).await {
Ok(_) => (),
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
}
}
pub async fn playback_done(&mut self, source: u8, error: u8, channel: u32, timestamp: u64) {
let mut traces_locked = self.traces.async_lock().await;
let mut trace = traces_locked.get_mut(&source).unwrap();
trace.state = RemoteState::PlaybackEnded {
error: error,
channel: channel,
timestamp: timestamp,
};
*(self.done_count.async_lock().await) += 1;
}
pub async fn playback(
&self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
timestamp: u64,
) {
let mut dest_list: Vec<u8> = Vec::new();
{
let lock = self.traces.async_lock().await;
let trace_iter = lock.iter();
for (dest, trace) in trace_iter {
if trace.state != RemoteState::Loaded {
error!("Destination {} not ready for DMA, state: {:?}", dest, trace.state);
continue;
}
dest_list.push(dest.clone());
}
}
// mutex lock must be dropped before sending a playback request to avoid a deadlock,
// if PlaybackStatus is sent from another satellite and the state must be updated.
for destination in dest_list {
match drtio::ddma_send_playback(aux_mutex, routing_table, timer, self.id, destination, timestamp).await
{
Ok(_) => (),
Err(e) => error!("Error during remote DMA playback: {}", e),
}
}
}
pub async fn destination_changed(
&mut self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
// update state of the destination, resend traces if it's up
if let Some(trace) = self.traces.async_lock().await.get_mut(&destination) {
if up {
match drtio::ddma_upload_trace(
aux_mutex,
routing_table,
timer,
self.id,
destination,
trace.get_trace(),
)
.await
{
Ok(_) => trace.state = RemoteState::Loaded,
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
} else {
trace.state = RemoteState::NotLoaded;
}
}
}
pub async fn is_empty(&self) -> bool {
self.traces.async_lock().await.is_empty()
}
}
static mut TRACES: BTreeMap<u32, TraceSet> = BTreeMap::new();
pub fn add_traces(id: u32, traces: BTreeMap<u8, Vec<u8>>) {
unsafe { TRACES.insert(id, TraceSet::new(id, traces)) };
}
pub async fn await_done(id: u32, timeout: Option<u64>, timer: GlobalTimer) -> Result<RemoteState, &'static str> {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.await_done(timeout, timer).await
}
pub async fn erase(aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer, id: u32) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.erase(aux_mutex, routing_table, timer).await;
unsafe {
TRACES.remove(&id);
}
}
pub async fn upload_traces(aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer, id: u32) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.upload_traces(aux_mutex, routing_table, timer).await;
}
pub async fn playback(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
timestamp: u64,
) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.playback(aux_mutex, routing_table, timer, timestamp).await;
}
pub async fn playback_done(id: u32, destination: u8, error: u8, channel: u32, timestamp: u64) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.playback_done(destination, error, channel, timestamp).await;
}
pub async fn destination_changed(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
let trace_iter = unsafe { TRACES.values_mut() };
for trace_set in trace_iter {
trace_set
.destination_changed(aux_mutex, routing_table, timer, destination, up)
.await;
}
}
pub async fn has_remote_traces(id: u32) -> bool {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
!(trace_set.is_empty().await)
}
}
pub async fn put_record(
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &RoutingTable,
_timer: GlobalTimer,
mut recorder: DmaRecorder,
) -> u32 {
#[cfg(has_drtio)]
let mut remote_traces: BTreeMap<u8, Vec<u8>> = BTreeMap::new();
#[cfg(has_drtio)]
if recorder.enable_ddma {
let mut local_trace: Vec<u8> = Vec::new();
// analyze each entry and put in proper buckets, as the kernel core
// sends whole chunks, to limit comms/kernel CPU communication,
// and as only comms core has access to varios DMA buffers.
let mut ptr = 0;
recorder.buffer.push(0);
while recorder.buffer[ptr] != 0 {
// ptr + 3 = tgt >> 24 (destination)
let len = recorder.buffer[ptr] as usize;
let destination = recorder.buffer[ptr + 3];
if destination == 0 {
local_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
if let Some(remote_trace) = remote_traces.get_mut(&destination) {
remote_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
remote_traces.insert(destination, recorder.buffer[ptr..ptr + len].to_vec());
}
}
// and jump to the next event
ptr += len;
}
mem::swap(&mut recorder.buffer, &mut local_trace);
}
// trailing zero to indicate end of buffer
recorder.buffer.push(0);
recorder.buffer.reserve(ALIGNMENT - 1);
let original_length = recorder.buffer.len();
let padding = ALIGNMENT - recorder.buffer.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
recorder.buffer.push(0);
}
recorder.buffer.copy_within(0..original_length, padding);
dcci_slice(&recorder.buffer);
let ptr = recorder.buffer[padding..].as_ptr() as u32;
let _old_record = DMA_RECORD_STORE
.lock()
.insert(recorder.name, (ptr, recorder.buffer, recorder.duration));
#[cfg(has_drtio)]
{
if let Some((old_id, _v, _d)) = _old_record {
remote_dma::erase(_aux_mutex, _routing_table, _timer, old_id).await;
}
remote_dma::add_traces(ptr, remote_traces);
}
ptr
}
pub async fn erase(name: String, _aux_mutex: &Rc<Mutex<bool>>, _routing_table: &RoutingTable, _timer: GlobalTimer) {
let _entry = DMA_RECORD_STORE.lock().remove(&name);
#[cfg(has_drtio)]
if let Some((id, _v, _d)) = _entry {
remote_dma::erase(_aux_mutex, _routing_table, _timer, id).await;
}
}
pub async fn retrieve(name: String) -> Option<(i32, i64, bool)> {
let (ptr, _v, duration) = DMA_RECORD_STORE.lock().get(&name)?.clone();
#[cfg(has_drtio)]
let uses_ddma = remote_dma::has_remote_traces(ptr).await;
#[cfg(not(has_drtio))]
let uses_ddma = false;
Some((ptr as i32, duration, uses_ddma))
}

File diff suppressed because it is too large Load Diff

View File

@ -1,335 +0,0 @@
use alloc::{collections::BTreeMap, rc::Rc, vec::Vec};
use libasync::task;
use libboard_artiq::{drtio_routing::RoutingTable,
drtioaux_proto::{PayloadStatus, MASTER_PAYLOAD_MAX_SIZE}};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::mutex::Mutex;
use log::{error, warn};
use crate::rtio_mgt::{drtio, drtio::Error as DrtioError};
#[derive(Debug, PartialEq, Clone, Copy)]
pub enum FinishStatus {
Ok,
CommLost,
Exception(u8), // exception source
}
#[derive(Debug, PartialEq, Clone, Copy)]
pub enum SubkernelState {
NotLoaded,
Uploaded,
Running,
Finished { status: FinishStatus },
}
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum Error {
Timeout,
IncorrectState,
SubkernelNotFound,
SubkernelException,
CommLost,
DrtioError(DrtioError),
}
impl From<DrtioError> for Error {
fn from(value: DrtioError) -> Error {
Error::DrtioError(value)
}
}
pub struct SubkernelFinished {
pub id: u32,
pub status: FinishStatus,
pub exception: Option<Vec<u8>>,
}
struct Subkernel {
pub destination: u8,
pub data: Vec<u8>,
pub state: SubkernelState,
}
impl Subkernel {
pub fn new(destination: u8, data: Vec<u8>) -> Self {
Subkernel {
destination: destination,
data: data,
state: SubkernelState::NotLoaded,
}
}
}
static SUBKERNELS: Mutex<BTreeMap<u32, Subkernel>> = Mutex::new(BTreeMap::new());
pub async fn add_subkernel(id: u32, destination: u8, kernel: Vec<u8>) {
SUBKERNELS
.async_lock()
.await
.insert(id, Subkernel::new(destination, kernel));
}
pub async fn upload(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
) -> Result<(), Error> {
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
drtio::subkernel_upload(
aux_mutex,
routing_table,
timer,
id,
subkernel.destination,
&subkernel.data,
)
.await?;
subkernel.state = SubkernelState::Uploaded;
Ok(())
} else {
Err(Error::SubkernelNotFound)
}
}
pub async fn load(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
run: bool,
timestamp: u64,
) -> Result<(), Error> {
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
if subkernel.state != SubkernelState::Uploaded {
return Err(Error::IncorrectState);
}
drtio::subkernel_load(
aux_mutex,
routing_table,
timer,
id,
subkernel.destination,
run,
timestamp,
)
.await?;
if run {
subkernel.state = SubkernelState::Running;
}
Ok(())
} else {
Err(Error::SubkernelNotFound)
}
}
pub async fn clear_subkernels() {
SUBKERNELS.async_lock().await.clear();
MESSAGE_QUEUE.async_lock().await.clear();
CURRENT_MESSAGES.async_lock().await.clear();
}
pub async fn subkernel_finished(id: u32, with_exception: bool, exception_src: u8) {
// called upon receiving DRTIO SubkernelRunDone
// may be None if session ends and is cleared
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
if subkernel.state == SubkernelState::Running {
subkernel.state = SubkernelState::Finished {
status: match with_exception {
true => FinishStatus::Exception(exception_src),
false => FinishStatus::Ok,
},
}
}
}
}
pub async fn destination_changed(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
let mut locked_subkernels = SUBKERNELS.async_lock().await;
for (id, subkernel) in locked_subkernels.iter_mut() {
if subkernel.destination == destination {
if up {
match drtio::subkernel_upload(aux_mutex, routing_table, timer, *id, destination, &subkernel.data).await
{
Ok(_) => subkernel.state = SubkernelState::Uploaded,
Err(e) => error!("Error adding subkernel on destination {}: {}", destination, e),
}
} else {
subkernel.state = match subkernel.state {
SubkernelState::Running => SubkernelState::Finished {
status: FinishStatus::CommLost,
},
_ => SubkernelState::NotLoaded,
}
}
}
}
}
pub async fn await_finish(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
timeout: i64,
) -> Result<SubkernelFinished, Error> {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Running | SubkernelState::Finished { .. } => (),
_ => return Err(Error::IncorrectState),
}
if timeout > 0 {
let max_time = timer.get_time() + Milliseconds(timeout as u64);
while timer.get_time() < max_time {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished { .. } => break,
_ => (),
};
task::r#yield().await;
}
if timer.get_time() >= max_time {
error!("Remote subkernel finish await timed out");
return Err(Error::Timeout);
}
} else {
// no timeout, wait forever
loop {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished { .. } => break,
_ => (),
};
task::r#yield().await;
}
}
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
match subkernel.state {
SubkernelState::Finished { status } => {
subkernel.state = SubkernelState::Uploaded;
Ok(SubkernelFinished {
id: id,
status: status,
exception: if let FinishStatus::Exception(dest) = status {
Some(drtio::subkernel_retrieve_exception(aux_mutex, routing_table, timer, dest).await?)
} else {
None
},
})
}
_ => Err(Error::IncorrectState),
}
} else {
Err(Error::SubkernelNotFound)
}
}
pub struct Message {
from_id: u32,
pub count: u8,
pub data: Vec<u8>,
}
// FIFO queue of messages
static MESSAGE_QUEUE: Mutex<Vec<Message>> = Mutex::new(Vec::new());
// currently under construction message(s) (can be from multiple sources)
static CURRENT_MESSAGES: Mutex<BTreeMap<u32, Message>> = Mutex::new(BTreeMap::new());
pub async fn message_handle_incoming(
id: u32,
status: PayloadStatus,
length: usize,
data: &[u8; MASTER_PAYLOAD_MAX_SIZE],
) {
// called when receiving a message from satellite
{
let subkernel_lock = SUBKERNELS.async_lock().await;
let subkernel = subkernel_lock.get(&id);
if subkernel.is_some() && subkernel.unwrap().state != SubkernelState::Running {
// do not add messages for non-running or deleted subkernels
warn!("received a message for a non-running subkernel #{}", id);
return;
}
}
let mut current_messages = CURRENT_MESSAGES.async_lock().await;
if status.is_first() {
current_messages.remove(&id);
}
match current_messages.get_mut(&id) {
Some(message) => message.data.extend(&data[..length]),
None => {
current_messages.insert(
id,
Message {
from_id: id,
count: data[0],
data: data[1..length].to_vec(),
},
);
}
};
if status.is_last() {
// when done, remove from working queue
MESSAGE_QUEUE
.async_lock()
.await
.push(current_messages.remove(&id).unwrap());
}
}
pub async fn message_await(id: u32, timeout: i64, timer: GlobalTimer) -> Result<Message, Error> {
let is_subkernel = SUBKERNELS.async_lock().await.get(&id).is_some();
if is_subkernel {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished {
status: FinishStatus::CommLost,
} => return Err(Error::CommLost),
SubkernelState::Running | SubkernelState::Finished { .. } => (),
_ => return Err(Error::IncorrectState),
}
}
let max_time = timer.get_time() + Milliseconds(timeout as u64);
while timeout < 0 || (timeout > 0 && timer.get_time() < max_time) {
{
let mut message_queue = MESSAGE_QUEUE.async_lock().await;
for i in 0..message_queue.len() {
let msg = &message_queue[i];
if msg.from_id == id {
let message = message_queue.remove(i);
return Ok(message);
}
}
}
if is_subkernel {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished {
status: FinishStatus::CommLost,
} => return Err(Error::CommLost),
SubkernelState::Finished {
status: FinishStatus::Exception(_),
} => return Err(Error::SubkernelException),
_ => (),
}
}
task::r#yield().await;
}
Err(Error::Timeout)
}
pub async fn message_send<'a>(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
message: Vec<u8>,
) -> Result<(), Error> {
Ok(drtio::subkernel_send_message(aux_mutex, routing_table, timer, id, destination, &message).await?)
}

View File

@ -1,68 +0,0 @@
max_width = 120
hard_tabs = false
tab_spaces = 4
newline_style = "Auto"
use_small_heuristics = "Default"
indent_style = "Block"
wrap_comments = false
format_code_in_doc_comments = false
comment_width = 100
normalize_comments = false
normalize_doc_attributes = false
license_template_path = ""
format_strings = true
format_macro_matchers = true
format_macro_bodies = true
empty_item_single_line = true
struct_lit_single_line = true
fn_single_line = false
where_single_line = true
imports_indent = "Visual"
imports_layout = "Mixed"
merge_imports = true
group_imports = "StdExternalCrate"
reorder_imports = true
reorder_modules = true
reorder_impl_items = false
type_punctuation_density = "Wide"
space_before_colon = false
space_after_colon = true
spaces_around_ranges = false
binop_separator = "Front"
remove_nested_parens = true
combine_control_expr = true
overflow_delimited_expr = false
struct_field_align_threshold = 0
enum_discrim_align_threshold = 0
match_arm_blocks = true
match_arm_leading_pipes = "Never"
force_multiline_blocks = false
fn_args_layout = "Tall"
brace_style = "SameLineWhere"
control_brace_style = "AlwaysSameLine"
trailing_semicolon = true
trailing_comma = "Vertical"
match_block_trailing_comma = false
blank_lines_upper_bound = 1
blank_lines_lower_bound = 0
edition = "2018"
version = "Two"
inline_attribute_width = 0
merge_derives = true
use_try_shorthand = false
use_field_init_shorthand = false
force_explicit_abi = true
condense_wildcard_suffixes = false
color = "Auto"
required_version = "1.4.32"
unstable_features = false
disable_all_formatting = false
skip_children = false
hide_parse_errors = false
error_on_line_overflow = false
error_on_unformatted = false
report_todo = "Never"
report_fixme = "Never"
ignore = []
emit_mode = "Files"
make_backup = false

28
src/satman/Cargo.toml Normal file
View File

@ -0,0 +1,28 @@
[package]
authors = ["M-Labs"]
name = "satman"
version = "0.0.0"
build = "build.rs"
[features]
target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"]
default = ["target_zc706", ]
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
log = { version = "0.4", default-features = false }
embedded-hal = "0.2"
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"]}
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] }
libboard_artiq = { path = "../libboard_artiq" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }

View File

@ -1,35 +0,0 @@
[package]
authors = ["M-Labs"]
name = "satman"
version = "0.0.0"
build = "build.rs"
[features]
target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"]
calibrate_wrpll_skew = ["libboard_artiq/calibrate_wrpll_skew"]
default = ["target_zc706", ]
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
log = { version = "0.4", default-features = false }
byteorder = { version = "1.3", default-features = false }
core_io = { version = "0.1", features = ["collections"] }
crc = { version = "1.7", default-features = false }
cslice = "0.3"
embedded-hal = "0.2"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] }
libboard_artiq = { path = "../libboard_artiq" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }
io = { path = "../libio", features = ["alloc"] }
ksupport = { path = "../libksupport" }

View File

@ -1,126 +0,0 @@
use core::cmp::min;
use libboard_artiq::{drtioaux_proto::SAT_PAYLOAD_MAX_SIZE, pl::csr};
use libcortex_a9::cache;
const BUFFER_SIZE: usize = 512 * 1024;
#[repr(align(64))]
struct Buffer {
data: [u8; BUFFER_SIZE],
}
static mut BUFFER: Buffer = Buffer { data: [0; BUFFER_SIZE] };
fn arm() {
unsafe {
let base_addr = &mut BUFFER.data[0] as *mut _ as usize;
let last_addr = &mut BUFFER.data[BUFFER_SIZE - 1] as *mut _ as usize;
csr::rtio_analyzer::dma_base_address_write(base_addr as u32);
csr::rtio_analyzer::message_encoder_overflow_reset_write(1);
csr::rtio_analyzer::dma_last_address_write(last_addr as u32);
csr::rtio_analyzer::dma_reset_write(1);
csr::rtio_analyzer::enable_write(1);
}
}
fn disarm() {
unsafe {
csr::rtio_analyzer::enable_write(0);
while csr::rtio_analyzer::busy_read() != 0 {}
cache::dcci_slice(&BUFFER.data);
}
}
pub struct Analyzer {
// necessary for keeping track of sent data
data_len: usize,
sent_bytes: usize,
data_pointer: usize,
}
pub struct Header {
pub total_byte_count: u64,
pub sent_bytes: u32,
pub error: bool,
}
pub struct AnalyzerSliceMeta {
pub len: u16,
pub last: bool,
}
impl Drop for Analyzer {
fn drop(&mut self) {
disarm();
}
}
impl Analyzer {
pub fn new() -> Analyzer {
// create and arm new Analyzer
arm();
Analyzer {
data_len: 0,
sent_bytes: 0,
data_pointer: 0,
}
}
pub fn get_header(&mut self) -> Header {
disarm();
let overflow = unsafe { csr::rtio_analyzer::message_encoder_overflow_read() != 0 };
let bus_err = unsafe { csr::rtio_analyzer::dma_bus_error_read() != 0 };
let total_byte_count = unsafe { csr::rtio_analyzer::dma_byte_count_read() as u64 };
let wraparound = total_byte_count >= BUFFER_SIZE as u64;
self.data_len = if wraparound {
BUFFER_SIZE
} else {
total_byte_count as usize
};
self.data_pointer = if wraparound {
(total_byte_count % BUFFER_SIZE as u64) as usize
} else {
0
};
self.sent_bytes = 0;
if overflow {
warn!("overflow occured");
}
if bus_err {
warn!("bus error occured");
}
Header {
total_byte_count: total_byte_count,
sent_bytes: self.data_len as u32,
error: overflow | bus_err,
}
}
pub fn get_data(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> AnalyzerSliceMeta {
let data = unsafe { &BUFFER.data[..] };
let i = (self.data_pointer + self.sent_bytes) % BUFFER_SIZE;
let len = min(SAT_PAYLOAD_MAX_SIZE, self.data_len - self.sent_bytes);
let last = self.sent_bytes + len == self.data_len;
if i + len >= BUFFER_SIZE {
data_slice[..(BUFFER_SIZE - i)].clone_from_slice(&data[i..BUFFER_SIZE]);
data_slice[(BUFFER_SIZE - i)..len].clone_from_slice(&data[..(i + len) % BUFFER_SIZE]);
} else {
data_slice[..len].clone_from_slice(&data[i..i + len]);
}
self.sent_bytes += len;
if last {
arm();
}
AnalyzerSliceMeta {
len: len as u16,
last: last,
}
}
}

View File

@ -1,563 +0,0 @@
use alloc::{collections::btree_map::BTreeMap, string::String, vec::Vec};
use core::mem;
use ksupport::kernel::DmaRecorder;
use libboard_artiq::{drtio_routing::RoutingTable,
drtioaux_proto::{Packet, PayloadStatus, MASTER_PAYLOAD_MAX_SIZE},
pl::csr};
use libcortex_a9::cache::dcci_slice;
use routing::{Router, Sliceable};
use subkernel::Manager as KernelManager;
const ALIGNMENT: usize = 64;
#[derive(Debug, PartialEq)]
enum ManagerState {
Idle,
Playback,
}
pub struct RtioStatus {
pub source: u8,
pub id: u32,
pub error: u8,
pub channel: u32,
pub timestamp: u64,
}
#[derive(Debug)]
pub enum Error {
IdNotFound,
PlaybackInProgress,
EntryNotComplete,
MasterDmaFound,
UploadFail,
}
#[derive(Debug)]
struct Entry {
trace: Vec<u8>,
padding_len: usize,
complete: bool,
duration: i64, // relevant for local DMA
}
impl Entry {
pub fn from_vec(data: Vec<u8>, duration: i64) -> Entry {
let mut entry = Entry {
trace: data,
padding_len: 0,
complete: true,
duration: duration,
};
entry.realign();
entry
}
pub fn id(&self) -> u32 {
self.trace[self.padding_len..].as_ptr() as u32
}
pub fn realign(&mut self) {
self.trace.push(0);
let data_len = self.trace.len();
self.trace.reserve(ALIGNMENT - 1);
let padding = ALIGNMENT - self.trace.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
// Vec guarantees that this will not reallocate
self.trace.push(0)
}
for i in 1..data_len + 1 {
self.trace[data_len + padding - i] = self.trace[data_len - i]
}
self.complete = true;
self.padding_len = padding;
dcci_slice(&self.trace);
}
}
#[derive(Debug)]
enum RemoteTraceState {
Unsent,
Sending(usize),
Ready,
Running(usize),
}
#[derive(Debug)]
struct RemoteTraces {
remote_traces: BTreeMap<u8, Sliceable>,
state: RemoteTraceState,
}
impl RemoteTraces {
pub fn new(traces: BTreeMap<u8, Sliceable>) -> RemoteTraces {
RemoteTraces {
remote_traces: traces,
state: RemoteTraceState::Unsent,
}
}
// on subkernel request
pub fn upload_traces(
&mut self,
id: u32,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) -> usize {
let len = self.remote_traces.len();
if len > 0 {
self.state = RemoteTraceState::Sending(self.remote_traces.len());
for (dest, trace) in self.remote_traces.iter_mut() {
// queue up the first packet for all destinations, rest will be sent after first ACK
let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
let meta = trace.get_slice_master(&mut data_slice);
router.route(
Packet::DmaAddTraceRequest {
source: self_destination,
destination: *dest,
id: id,
status: meta.status,
length: meta.len,
trace: data_slice,
},
routing_table,
rank,
self_destination,
);
}
}
len
}
// on incoming Packet::DmaAddTraceReply
pub fn ack_upload(
&mut self,
kernel_manager: &mut KernelManager,
source: u8,
id: u32,
succeeded: bool,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) {
if let RemoteTraceState::Sending(count) = self.state {
if let Some(trace) = self.remote_traces.get_mut(&source) {
if trace.at_end() {
if count - 1 == 0 {
self.state = RemoteTraceState::Ready;
if let Some((id, timestamp)) = kernel_manager.ddma_remote_uploaded(succeeded) {
self.playback(id, timestamp, router, rank, self_destination, routing_table);
}
} else {
self.state = RemoteTraceState::Sending(count - 1);
}
} else {
// send next slice
let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
let meta = trace.get_slice_master(&mut data_slice);
router.route(
Packet::DmaAddTraceRequest {
source: self_destination,
destination: meta.destination,
id: id,
status: meta.status,
length: meta.len,
trace: data_slice,
},
routing_table,
rank,
self_destination,
);
}
}
}
}
// on subkernel request
pub fn playback(
&mut self,
id: u32,
timestamp: u64,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) {
// route all the playback requests
// remote traces (local trace runs on core1 unlike mainline firmware)
self.state = RemoteTraceState::Running(self.remote_traces.len());
for (dest, _) in self.remote_traces.iter() {
router.route(
Packet::DmaPlaybackRequest {
source: self_destination,
destination: *dest,
id: id,
timestamp: timestamp,
},
routing_table,
rank,
self_destination,
);
// response will be ignored (succeeded = false handled by the main thread)
}
}
// on incoming Packet::DmaPlaybackDone
pub fn remote_finished(&mut self, kernel_manager: &mut KernelManager, error: u8, channel: u32, timestamp: u64) {
if let RemoteTraceState::Running(count) = self.state {
if error != 0 || count - 1 == 0 {
// notify the kernel about a DDMA error or finish
kernel_manager.ddma_finished(error, channel, timestamp);
self.state = RemoteTraceState::Ready;
// further messages will be ignored (if there was an error)
} else {
// no error and not the last one awaited
self.state = RemoteTraceState::Running(count - 1);
}
}
}
pub fn erase(
&mut self,
id: u32,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) {
for (dest, _) in self.remote_traces.iter() {
router.route(
Packet::DmaRemoveTraceRequest {
source: self_destination,
destination: *dest,
id: id,
},
routing_table,
rank,
self_destination,
);
// response will be ignored as this object will stop existing too
}
}
pub fn has_remote_traces(&self) -> bool {
self.remote_traces.len() > 0
}
}
#[derive(Debug)]
pub struct Manager {
entries: BTreeMap<(u8, u32), Entry>,
state: ManagerState,
current_id: u32,
current_source: u8,
remote_entries: BTreeMap<u32, RemoteTraces>,
name_map: BTreeMap<String, u32>,
}
impl Manager {
pub fn new() -> Manager {
// in case Manager is created during a DMA in progress
// wait for it to end
unsafe { while csr::rtio_dma::enable_read() != 0 {} }
Manager {
entries: BTreeMap::new(),
current_id: 0,
current_source: 0,
state: ManagerState::Idle,
remote_entries: BTreeMap::new(),
name_map: BTreeMap::new(),
}
}
pub fn add(
&mut self,
source: u8,
id: u32,
status: PayloadStatus,
trace: &[u8],
trace_len: usize,
) -> Result<(), Error> {
let entry = match self.entries.get_mut(&(source, id)) {
Some(entry) => {
if entry.complete || status.is_first() {
// replace entry
self.entries.remove(&(source, id));
self.entries.insert(
(source, id),
Entry {
trace: Vec::new(),
padding_len: 0,
complete: false,
duration: 0,
},
);
self.entries.get_mut(&(source, id)).unwrap()
} else {
entry
}
}
None => {
self.entries.insert(
(source, id),
Entry {
trace: Vec::new(),
padding_len: 0,
complete: false,
duration: 0,
},
);
self.entries.get_mut(&(source, id)).unwrap()
}
};
entry.trace.extend(&trace[0..trace_len]);
if status.is_last() {
entry.realign();
}
Ok(())
}
// api for DRTIO
pub fn erase(&mut self, source: u8, id: u32) -> Result<(), Error> {
match self.entries.remove(&(source, id)) {
Some(_) => Ok(()),
None => Err(Error::IdNotFound),
}
}
// API for subkernel
pub fn erase_name(
&mut self,
name: &str,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) {
if let Some(id) = self.name_map.get(name) {
if let Some(traces) = self.remote_entries.get_mut(&id) {
traces.erase(*id, router, rank, self_destination, routing_table);
self.remote_entries.remove(&id);
}
self.entries.remove(&(self_destination, *id));
self.name_map.remove(name);
}
}
pub fn remote_finished(
&mut self,
kernel_manager: &mut KernelManager,
id: u32,
error: u8,
channel: u32,
timestamp: u64,
) {
if let Some(entry) = self.remote_entries.get_mut(&id) {
entry.remote_finished(kernel_manager, error, channel, timestamp);
}
}
pub fn ack_upload(
&mut self,
kernel_manager: &mut KernelManager,
source: u8,
id: u32,
succeeded: bool,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) {
if let Some(entry) = self.remote_entries.get_mut(&id) {
entry.ack_upload(
kernel_manager,
source,
id,
succeeded,
router,
rank,
self_destination,
routing_table,
);
}
}
// API for subkernel
pub fn upload_traces(
&mut self,
id: u32,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) -> Result<usize, Error> {
let remote_traces = self.remote_entries.get_mut(&id);
let mut len = 0;
if let Some(traces) = remote_traces {
len = traces.upload_traces(id, router, rank, self_destination, routing_table);
}
Ok(len)
}
// API for subkernel
pub fn playback_remote(
&mut self,
id: u32,
timestamp: u64,
router: &mut Router,
rank: u8,
self_destination: u8,
routing_table: &RoutingTable,
) -> Result<(), Error> {
if let Some(traces) = self.remote_entries.get_mut(&id) {
traces.playback(id, timestamp, router, rank, self_destination, routing_table);
Ok(())
} else {
Err(Error::IdNotFound)
}
}
// API for subkernel
pub fn cleanup(&mut self, router: &mut Router, rank: u8, self_destination: u8, routing_table: &RoutingTable) {
// after subkernel ends, remove all self-generated traces
for (_, id) in self.name_map.iter_mut() {
if let Some(traces) = self.remote_entries.get_mut(&id) {
traces.erase(*id, router, rank, self_destination, routing_table);
self.remote_entries.remove(&id);
}
self.entries.remove(&(self_destination, *id));
}
self.name_map.clear();
}
// API for subkernel
pub fn retrieve(&self, self_destination: u8, name: &String) -> Option<(i32, i64, bool)> {
let id = self.name_map.get(name)?;
let duration = self.entries.get(&(self_destination, *id))?.duration;
let uses_ddma = self.has_remote_traces(*id);
Some((*id as i32, duration, uses_ddma))
}
pub fn has_remote_traces(&self, id: u32) -> bool {
match self.remote_entries.get(&id) {
Some(traces) => traces.has_remote_traces(),
_ => false,
}
}
pub fn put_record(&mut self, mut recorder: DmaRecorder, self_destination: u8) -> Result<u32, Error> {
let mut remote_traces: BTreeMap<u8, Sliceable> = BTreeMap::new();
let mut local_trace: Vec<u8> = Vec::new();
// analyze each entry and put in proper buckets, as the kernel core
// sends whole chunks, to limit comms/kernel CPU communication,
// and as only comms core has access to varios DMA buffers.
let mut ptr = 0;
recorder.buffer.push(0);
while recorder.buffer[ptr] != 0 {
// ptr + 3 = tgt >> 24 (destination)
let len = recorder.buffer[ptr] as usize;
let destination = recorder.buffer[ptr + 3];
if destination == 0 {
return Err(Error::MasterDmaFound);
} else if destination == self_destination {
local_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
if let Some(remote_trace) = remote_traces.get_mut(&destination) {
remote_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
remote_traces.insert(
destination,
Sliceable::new(destination, recorder.buffer[ptr..ptr + len].to_vec()),
);
}
}
// and jump to the next event
ptr += len;
}
let local_entry = Entry::from_vec(local_trace, recorder.duration);
let id = local_entry.id();
self.entries.insert((self_destination, id), local_entry);
self.remote_entries.insert(id, RemoteTraces::new(remote_traces));
let mut name = String::new();
mem::swap(&mut recorder.name, &mut name);
self.name_map.insert(name, id);
Ok(id)
}
pub fn playback(&mut self, source: u8, id: u32, timestamp: u64) -> Result<(), Error> {
if self.state != ManagerState::Idle {
return Err(Error::PlaybackInProgress);
}
let entry = match self.entries.get(&(source, id)) {
Some(entry) => entry,
None => {
return Err(Error::IdNotFound);
}
};
if !entry.complete {
return Err(Error::EntryNotComplete);
}
let ptr = entry.trace[entry.padding_len..].as_ptr();
assert!(ptr as u32 % 64 == 0);
self.state = ManagerState::Playback;
self.current_id = id;
self.current_source = source;
unsafe {
csr::rtio_dma::base_address_write(ptr as u32);
csr::rtio_dma::time_offset_write(timestamp as u64);
csr::cri_con::selected_write(1);
csr::rtio_dma::enable_write(1);
// playback has begun here, for status call check_state
}
Ok(())
}
pub fn check_state(&mut self) -> Option<RtioStatus> {
if self.state != ManagerState::Playback {
// nothing to report
return None;
}
let dma_enable = unsafe { csr::rtio_dma::enable_read() };
if dma_enable != 0 {
return None;
} else {
self.state = ManagerState::Idle;
unsafe {
csr::cri_con::selected_write(0);
let error = csr::rtio_dma::error_read();
let channel = csr::rtio_dma::error_channel_read();
let timestamp = csr::rtio_dma::error_timestamp_read();
if error != 0 {
csr::rtio_dma::error_write(1);
}
return Some(RtioStatus {
source: self.current_source,
id: self.current_id,
error: error,
channel: channel,
timestamp: timestamp,
});
}
}
}
pub fn running(&self) -> bool {
self.state == ManagerState::Playback
}
}

File diff suppressed because it is too large Load Diff

View File

@ -1,149 +0,0 @@
use alloc::vec::Vec;
use byteorder::{ByteOrder, NativeEndian};
use crc::crc32;
use io::{ProtoRead, ProtoWrite};
use libboard_artiq::{drtioaux_proto::SAT_PAYLOAD_MAX_SIZE,
logger::{BufferLogger, LogBufferRef}};
use libconfig::Config;
use log::{debug, error, info, warn, LevelFilter};
use crate::routing::{SliceMeta, Sliceable};
type Result<T> = core::result::Result<T, ()>;
pub fn byte_to_level_filter(level_byte: u8) -> Result<LevelFilter> {
Ok(match level_byte {
0 => LevelFilter::Off,
1 => LevelFilter::Error,
2 => LevelFilter::Warn,
3 => LevelFilter::Info,
4 => LevelFilter::Debug,
5 => LevelFilter::Trace,
lv => {
error!("unknown log level: {}", lv);
return Err(());
}
})
}
fn get_logger_buffer() -> LogBufferRef<'static> {
let logger = unsafe { BufferLogger::get_logger().as_mut().unwrap() };
loop {
if let Some(buffer_ref) = logger.buffer() {
return buffer_ref;
}
}
}
pub fn clear_log() {
let mut buffer = get_logger_buffer();
buffer.clear();
}
pub struct Manager<'a> {
cfg: &'a mut Config,
last_log: Sliceable,
config_payload: Vec<u8>,
last_value: Sliceable,
image_payload: Vec<u8>,
}
impl<'a> Manager<'_> {
pub fn new(cfg: &mut Config) -> Manager {
Manager {
cfg: cfg,
last_log: Sliceable::new(0, Vec::new()),
config_payload: Vec::new(),
last_value: Sliceable::new(0, Vec::new()),
image_payload: Vec::new(),
}
}
pub fn log_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE], consume: bool) -> SliceMeta {
// Populate buffer if depleted
if self.last_log.at_end() {
let mut buffer = get_logger_buffer();
self.last_log.extend(buffer.extract().as_bytes());
if consume {
buffer.clear();
}
}
self.last_log.get_slice_satellite(data_slice)
}
pub fn fetch_config_value(&mut self, key: &str) -> Result<()> {
self.cfg
.read(&key)
.map(|value| {
debug!("got value");
self.last_value = Sliceable::new(0, value)
})
.map_err(|_| warn!("read error: no such key"))
}
pub fn get_config_value_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
self.last_value.get_slice_satellite(data_slice)
}
pub fn add_config_data(&mut self, data: &[u8], data_len: usize) {
self.config_payload.write_all(&data[..data_len]).unwrap();
}
pub fn clear_config_data(&mut self) {
self.config_payload.clear();
}
pub fn write_config(&mut self) -> Result<()> {
let mut payload = &self.config_payload[..];
let key = payload.read_string().map_err(|_err| error!("error on reading key"))?;
debug!("write key: {}", key);
let value = payload.read_bytes().unwrap();
self.cfg
.write(&key, value)
.map(|()| debug!("write success"))
.map_err(|err| error!("failed to write: {:?}", err))
}
pub fn remove_config(&mut self, key: &str) -> Result<()> {
debug!("erase key: {}", key);
self.cfg
.remove(&key)
.map(|()| debug!("erase success"))
.map_err(|err| warn!("failed to erase: {:?}", err))
}
pub fn allocate_image_buffer(&mut self, image_size: usize) {
self.image_payload = Vec::with_capacity(image_size);
}
pub fn add_image_data(&mut self, data: &[u8], data_len: usize) {
self.image_payload.extend(&data[..data_len]);
}
pub fn write_image(&self) {
let mut image = self.image_payload.clone();
let image_ref = &image[..];
let bin_len = image.len() - 4;
let (image_ref, expected_crc) = {
let (image_ref, crc_slice) = image_ref.split_at(bin_len);
(image_ref, NativeEndian::read_u32(crc_slice))
};
let actual_crc = crc32::checksum_ieee(image_ref);
if actual_crc == expected_crc {
info!("CRC passed. Writing boot image to SD card...");
image.truncate(bin_len);
self.cfg.write("boot", image).expect("failed to write boot image");
} else {
panic!(
"CRC failed, images have not been written to flash.\n(actual {:08x}, expected {:08x})",
actual_crc, expected_crc
);
}
}
}

View File

@ -1,17 +1,19 @@
use libboard_artiq::{drtioaux, drtio_routing};
use libboard_zynq::timer::GlobalTimer;
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs; use libboard_artiq::{pl::csr};
#[cfg(has_drtio_routing)]
use libboard_artiq::pl::csr;
use libboard_artiq::{drtio_routing, drtioaux};
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
use libboard_zynq::time::Milliseconds; use libboard_zynq::time::Milliseconds;
use libboard_zynq::timer::GlobalTimer; #[cfg(has_drtio_routing)]
use routing::Router; use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs;
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
fn rep_link_rx_up(repno: u8) -> bool { fn rep_link_rx_up(repno: u8) -> bool {
let repno = repno as usize; let repno = repno as usize;
unsafe { (csr::DRTIOREP[repno].rx_up_read)() == 1 } unsafe {
(csr::DRTIOREP[repno].rx_up_read)() == 1
}
} }
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
@ -21,14 +23,12 @@ enum RepeaterState {
SendPing { ping_count: u16 }, SendPing { ping_count: u16 },
WaitPingReply { ping_count: u16, timeout: Milliseconds }, WaitPingReply { ping_count: u16, timeout: Milliseconds },
Up, Up,
Failed, Failed
} }
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
impl Default for RepeaterState { impl Default for RepeaterState {
fn default() -> RepeaterState { fn default() -> RepeaterState { RepeaterState::Down }
RepeaterState::Down
}
} }
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
@ -36,7 +36,7 @@ impl Default for RepeaterState {
pub struct Repeater { pub struct Repeater {
repno: u8, repno: u8,
auxno: u8, auxno: u8,
state: RepeaterState, state: RepeaterState
} }
#[cfg(has_drtio_routing)] #[cfg(has_drtio_routing)]
@ -45,7 +45,7 @@ impl Repeater {
Repeater { Repeater {
repno: repno, repno: repno,
auxno: repno + 1, auxno: repno + 1,
state: RepeaterState::Down, state: RepeaterState::Down
} }
} }
@ -54,14 +54,8 @@ impl Repeater {
self.state == RepeaterState::Up self.state == RepeaterState::Up
} }
pub fn service( pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8,
&mut self, timer: &mut GlobalTimer) {
routing_table: &drtio_routing::RoutingTable,
rank: u8,
destination: u8,
router: &mut Router,
timer: &mut GlobalTimer,
) {
self.process_local_errors(); self.process_local_errors();
match self.state { match self.state {
@ -76,7 +70,7 @@ impl Repeater {
drtioaux::send(self.auxno, &drtioaux::Packet::EchoRequest).unwrap(); drtioaux::send(self.auxno, &drtioaux::Packet::EchoRequest).unwrap();
self.state = RepeaterState::WaitPingReply { self.state = RepeaterState::WaitPingReply {
ping_count: ping_count + 1, ping_count: ping_count + 1,
timeout: timer.get_time() + Milliseconds(100), timeout: timer.get_time() + Milliseconds(100)
} }
} else { } else {
error!("[REP#{}] link RX went down during ping", self.repno); error!("[REP#{}] link RX went down during ping", self.repno);
@ -87,10 +81,6 @@ impl Repeater {
if rep_link_rx_up(self.repno) { if rep_link_rx_up(self.repno) {
if let Ok(Some(drtioaux::Packet::EchoReply)) = drtioaux::recv(self.auxno) { if let Ok(Some(drtioaux::Packet::EchoReply)) = drtioaux::recv(self.auxno) {
info!("[REP#{}] remote replied after {} packets", self.repno, ping_count); info!("[REP#{}] remote replied after {} packets", self.repno, ping_count);
let max_time = timer.get_time() + Milliseconds(200);
while timer.get_time() < max_time {
let _ = drtioaux::recv(self.auxno);
}
self.state = RepeaterState::Up; self.state = RepeaterState::Up;
if let Err(e) = self.sync_tsc(timer) { if let Err(e) = self.sync_tsc(timer) {
error!("[REP#{}] failed to sync TSC ({:?})", self.repno, e); error!("[REP#{}] failed to sync TSC ({:?})", self.repno, e);
@ -123,7 +113,7 @@ impl Repeater {
} }
} }
RepeaterState::Up => { RepeaterState::Up => {
self.process_unsolicited_aux(routing_table, rank, destination, router); self.process_unsolicited_aux();
if !rep_link_rx_up(self.repno) { if !rep_link_rx_up(self.repno) {
info!("[REP#{}] link is down", self.repno); info!("[REP#{}] link is down", self.repno);
self.state = RepeaterState::Down; self.state = RepeaterState::Down;
@ -138,17 +128,11 @@ impl Repeater {
} }
} }
fn process_unsolicited_aux( fn process_unsolicited_aux(&self) {
&self,
routing_table: &drtio_routing::RoutingTable,
rank: u8,
destination: u8,
router: &mut Router,
) {
match drtioaux::recv(self.auxno) { match drtioaux::recv(self.auxno) {
Ok(Some(packet)) => router.route(packet, routing_table, rank, destination), Ok(Some(packet)) => warn!("[REP#{}] unsolicited aux packet: {:?}", self.repno, packet),
Ok(None) => (), Ok(None) => (),
Err(_) => warn!("[REP#{}] aux packet error", self.repno), Err(_) => warn!("[REP#{}] aux packet error", self.repno)
} }
} }
@ -171,20 +155,14 @@ impl Repeater {
cmd = (csr::DRTIOREP[repno].command_missed_cmd_read)(); cmd = (csr::DRTIOREP[repno].command_missed_cmd_read)();
chan_sel = (csr::DRTIOREP[repno].command_missed_chan_sel_read)(); chan_sel = (csr::DRTIOREP[repno].command_missed_chan_sel_read)();
} }
error!( error!("[REP#{}] CRI command missed, cmd={}, chan_sel=0x{:06x}", repno, cmd, chan_sel)
"[REP#{}] CRI command missed, cmd={}, chan_sel=0x{:06x}",
repno, cmd, chan_sel
)
} }
if errors & 8 != 0 { if errors & 8 != 0 {
let destination; let destination;
unsafe { unsafe {
destination = (csr::DRTIOREP[repno].buffer_space_timeout_dest_read)(); destination = (csr::DRTIOREP[repno].buffer_space_timeout_dest_read)();
} }
error!( error!("[REP#{}] timeout attempting to get remote buffer space, destination=0x{:02x}", repno, destination);
"[REP#{}] timeout attempting to get remote buffer space, destination=0x{:02x}",
repno, destination
);
} }
unsafe { unsafe {
(csr::DRTIOREP[repno].protocol_error_write)(errors); (csr::DRTIOREP[repno].protocol_error_write)(errors);
@ -203,50 +181,19 @@ impl Repeater {
match drtioaux::recv(self.auxno) { match drtioaux::recv(self.auxno) {
Ok(Some(packet)) => return Ok(packet), Ok(Some(packet)) => return Ok(packet),
Ok(None) => (), Ok(None) => (),
Err(e) => return Err(e), Err(e) => return Err(e)
} }
} }
} }
pub fn aux_forward( pub fn aux_forward(&self, request: &drtioaux::Packet, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
&self,
request: &drtioaux::Packet,
router: &mut Router,
routing_table: &drtio_routing::RoutingTable,
rank: u8,
self_destination: u8,
timer: &mut GlobalTimer,
) -> Result<(), drtioaux::Error> {
self.aux_send(request)?;
loop {
let reply = self.recv_aux_timeout(200, timer)?;
match reply {
// async/locally requested packets to be consumed or routed
// these may come while a packet would be forwarded
drtioaux::Packet::DmaPlaybackStatus { .. }
| drtioaux::Packet::SubkernelFinished { .. }
| drtioaux::Packet::SubkernelMessage { .. }
| drtioaux::Packet::SubkernelMessageAck { .. }
| drtioaux::Packet::SubkernelLoadRunReply { .. }
| drtioaux::Packet::SubkernelException { .. }
| drtioaux::Packet::DmaAddTraceReply { .. }
| drtioaux::Packet::DmaPlaybackReply { .. } => {
router.route(reply, routing_table, rank, self_destination);
}
_ => {
drtioaux::send(0, &reply).unwrap();
break;
}
}
}
Ok(())
}
pub fn aux_send(&self, request: &drtioaux::Packet) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up { if self.state != RepeaterState::Up {
return Err(drtioaux::Error::LinkDown); return Err(drtioaux::Error::LinkDown);
} }
drtioaux::send(self.auxno, request) drtioaux::send(self.auxno, request).unwrap();
let reply = self.recv_aux_timeout(200, timer)?;
drtioaux::send(0, &reply).unwrap();
Ok(())
} }
pub fn sync_tsc(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { pub fn sync_tsc(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
@ -270,24 +217,15 @@ impl Repeater {
} }
} }
pub fn set_path( pub fn set_path(&self, destination: u8, hops: &[u8; drtio_routing::MAX_HOPS], timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
&self,
destination: u8,
hops: &[u8; drtio_routing::MAX_HOPS],
timer: &mut GlobalTimer,
) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up { if self.state != RepeaterState::Up {
return Ok(()); return Ok(());
} }
drtioaux::send( drtioaux::send(self.auxno, &drtioaux::Packet::RoutingSetPath {
self.auxno,
&drtioaux::Packet::RoutingSetPath {
destination: destination, destination: destination,
hops: *hops, hops: *hops
}, }).unwrap();
)
.unwrap();
let reply = self.recv_aux_timeout(200, timer)?; let reply = self.recv_aux_timeout(200, timer)?;
if reply != drtioaux::Packet::RoutingAck { if reply != drtioaux::Packet::RoutingAck {
return Err(drtioaux::Error::UnexpectedReply); return Err(drtioaux::Error::UnexpectedReply);
@ -295,11 +233,7 @@ impl Repeater {
Ok(()) Ok(())
} }
pub fn load_routing_table( pub fn load_routing_table(&self, routing_table: &drtio_routing::RoutingTable, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
&self,
routing_table: &drtio_routing::RoutingTable,
timer: &mut GlobalTimer,
) -> Result<(), drtioaux::Error> {
for i in 0..drtio_routing::DEST_COUNT { for i in 0..drtio_routing::DEST_COUNT {
self.set_path(i as u8, &routing_table.0[i], timer)?; self.set_path(i as u8, &routing_table.0[i], timer)?;
} }
@ -310,7 +244,9 @@ impl Repeater {
if self.state != RepeaterState::Up { if self.state != RepeaterState::Up {
return Ok(()); return Ok(());
} }
drtioaux::send(self.auxno, &drtioaux::Packet::RoutingSetRank { rank: rank }).unwrap(); drtioaux::send(self.auxno, &drtioaux::Packet::RoutingSetRank {
rank: rank
}).unwrap();
let reply = self.recv_aux_timeout(200, timer)?; let reply = self.recv_aux_timeout(200, timer)?;
if reply != drtioaux::Packet::RoutingAck { if reply != drtioaux::Packet::RoutingAck {
return Err(drtioaux::Error::UnexpectedReply); return Err(drtioaux::Error::UnexpectedReply);
@ -320,13 +256,9 @@ impl Repeater {
pub fn rtio_reset(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { pub fn rtio_reset(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
let repno = self.repno as usize; let repno = self.repno as usize;
unsafe { unsafe { (csr::DRTIOREP[repno].reset_write)(1); }
(csr::DRTIOREP[repno].reset_write)(1);
}
timer.delay_us(100); timer.delay_us(100);
unsafe { unsafe { (csr::DRTIOREP[repno].reset_write)(0); }
(csr::DRTIOREP[repno].reset_write)(0);
}
if self.state != RepeaterState::Up { if self.state != RepeaterState::Up {
return Ok(()); return Ok(());
@ -343,29 +275,16 @@ impl Repeater {
#[cfg(not(has_drtio_routing))] #[cfg(not(has_drtio_routing))]
#[derive(Clone, Copy, Default)] #[derive(Clone, Copy, Default)]
pub struct Repeater {} pub struct Repeater {
}
#[cfg(not(has_drtio_routing))] #[cfg(not(has_drtio_routing))]
impl Repeater { impl Repeater {
pub fn new(_repno: u8) -> Repeater { pub fn new(_repno: u8) -> Repeater { Repeater::default() }
Repeater::default()
}
pub fn service( pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8, _timer: &mut GlobalTimer) { }
&self,
_routing_table: &drtio_routing::RoutingTable,
_rank: u8,
_destination: u8,
_router: &mut Router,
_timer: &mut GlobalTimer,
) {
}
pub fn sync_tsc(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { pub fn sync_tsc(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { Ok(()) }
Ok(())
}
pub fn rtio_reset(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { pub fn rtio_reset(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> { Ok(()) }
Ok(())
}
} }

View File

@ -1,174 +0,0 @@
use alloc::{collections::vec_deque::VecDeque, vec::Vec};
use core::cmp::min;
#[cfg(has_drtio_routing)]
use libboard_artiq::pl::csr;
use libboard_artiq::{drtio_routing, drtioaux,
drtioaux_proto::{PayloadStatus, MASTER_PAYLOAD_MAX_SIZE, SAT_PAYLOAD_MAX_SIZE}};
pub struct SliceMeta {
pub destination: u8,
pub len: u16,
pub status: PayloadStatus,
}
/* represents data that has to be sent to Master */
#[derive(Debug)]
pub struct Sliceable {
it: usize,
data: Vec<u8>,
destination: u8,
}
macro_rules! get_slice_fn {
($name:tt, $size:expr) => {
pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
let first = self.it == 0;
let len = min($size, self.data.len() - self.it);
let last = self.it + len == self.data.len();
let status = PayloadStatus::from_status(first, last);
data_slice[..len].clone_from_slice(&self.data[self.it..self.it + len]);
self.it += len;
SliceMeta {
destination: self.destination,
len: len as u16,
status: status,
}
}
};
}
impl Sliceable {
pub fn new(destination: u8, data: Vec<u8>) -> Sliceable {
Sliceable {
it: 0,
data: data,
destination: destination,
}
}
pub fn at_end(&self) -> bool {
self.it == self.data.len()
}
pub fn extend(&mut self, data: &[u8]) {
self.data.extend(data);
}
get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
get_slice_fn!(get_slice_satellite, SAT_PAYLOAD_MAX_SIZE);
}
// Packets from downstream (further satellites) are received and routed appropriately.
// they're passed as soon as possible downstream (within the subtree), or sent upstream,
// which is notified about pending packets.
// for rank 1 (connected to master) satellites, these packets are passed as an answer to DestinationStatusRequest;
// for higher ranks, after getting a notification, it will transact with downstream to get the pending packets.
// forward! macro is not deprecated, as routable packets are only these that can originate
// from both master and satellite, e.g. DDMA and Subkernel.
pub struct Router {
upstream_queue: VecDeque<drtioaux::Packet>,
local_queue: VecDeque<drtioaux::Packet>,
#[cfg(has_drtio_routing)]
downstream_queue: VecDeque<(usize, drtioaux::Packet)>,
}
impl Router {
pub fn new() -> Router {
Router {
upstream_queue: VecDeque::new(),
local_queue: VecDeque::new(),
#[cfg(has_drtio_routing)]
downstream_queue: VecDeque::new(),
}
}
// Called by local sources (DDMA, kernel) and by repeaters on receiving async data;
// messages are always buffered for both upstream and downstream
pub fn route(
&mut self,
packet: drtioaux::Packet,
_routing_table: &drtio_routing::RoutingTable,
_rank: u8,
self_destination: u8,
) {
let destination = packet.routable_destination();
#[cfg(has_drtio_routing)]
{
if let Some(destination) = destination {
let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
if destination == self_destination {
self.local_queue.push_back(packet);
} else if hop > 0 && hop < csr::DRTIOREP.len() {
let repno = (hop - 1) as usize;
self.downstream_queue.push_back((repno, packet));
} else {
self.upstream_queue.push_back(packet);
}
} else {
error!("Received an unroutable packet: {:?}", packet);
}
}
#[cfg(not(has_drtio_routing))]
{
if destination == Some(self_destination) {
self.local_queue.push_back(packet);
} else {
self.upstream_queue.push_back(packet);
}
}
}
// Sends a packet to a required destination, routing if necessary
pub fn send(
&mut self,
packet: drtioaux::Packet,
_routing_table: &drtio_routing::RoutingTable,
_rank: u8,
_destination: u8,
) -> Result<(), drtioaux::Error> {
#[cfg(has_drtio_routing)]
{
let destination = packet.routable_destination();
if let Some(destination) = destination {
let hop = _routing_table.0[destination as usize][_rank as usize] as usize;
if destination == 0 {
// response is needed immediately if master required it
drtioaux::send(0, &packet)?;
} else if !(hop > 0 && hop < csr::DRTIOREP.len()) {
// higher rank can wait
self.upstream_queue.push_back(packet);
} else {
let repno = (hop - 1) as usize;
// transaction will occur at closest possible opportunity
self.downstream_queue.push_back((repno, packet));
}
Ok(())
} else {
// packet not supported in routing, fallback - sent directly
drtioaux::send(0, &packet)
}
}
#[cfg(not(has_drtio_routing))]
{
drtioaux::send(0, &packet)
}
}
pub fn get_upstream_packet(&mut self) -> Option<drtioaux::Packet> {
self.upstream_queue.pop_front()
}
#[cfg(has_drtio_routing)]
pub fn get_downstream_packet(&mut self) -> Option<(usize, drtioaux::Packet)> {
self.downstream_queue.pop_front()
}
pub fn get_local_packet(&mut self) -> Option<drtioaux::Packet> {
self.local_queue.pop_front()
}
}

File diff suppressed because it is too large Load Diff