esavkin
/
artiq-zynq
forked from M-Labs/artiq-zynq
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

base: esavkin:812aea33b3cae41f70d35d5ca8a6eb4a507fdbed
Branches Tags
esavkin:master
esavkin:264-grabber-backport
esavkin:264-fix-grabber
esavkin:release-7
esavkin:add_80_to_125_clock_setting
esavkin:rustfmt-policy
esavkin:213-small-fix
esavkin:213-satman-sfp-fiber-support
esavkin:213-backport-sfp-fiber
esavkin:213-sfp-fiber-support
esavkin:102-update-rust
esavkin:1697-exception-format
esavkin:1697-rtiomap
esavkin:204-idle-kernel-typo
esavkin:reboot_support_mngmt
esavkin:release-6
M-Labs:master
M-Labs:release-7
M-Labs:release-6
..
compare: esavkin:731b6a89dda61efca2ffb0a79526b8aa7251ef08
Branches Tags
esavkin:264-grabber-backport
esavkin:264-fix-grabber
esavkin:master
esavkin:release-7
esavkin:add_80_to_125_clock_setting
esavkin:rustfmt-policy
esavkin:213-small-fix
esavkin:213-satman-sfp-fiber-support
esavkin:213-backport-sfp-fiber
esavkin:213-sfp-fiber-support
esavkin:102-update-rust
esavkin:1697-exception-format
esavkin:1697-rtiomap
esavkin:204-idle-kernel-typo
esavkin:reboot_support_mngmt
esavkin:release-6
M-Labs:master
M-Labs:release-7
M-Labs:release-6

No commits in common. "812aea33b3cae41f70d35d5ca8a6eb4a507fdbed" and "731b6a89dda61efca2ffb0a79526b8aa7251ef08" have entirely different histories.
812aea33b3 ... 731b6a89dd

53 changed files with 835 additions and 3716 deletions
Show Stats Expand all files Collapse all files

8
.gitignore vendored
View File

@ -3,11 +3,3 @@ examples/*.elf
__pycache__
build
src/libboard_artiq/Cargo.toml
src/libc/Cargo.toml
src/libdyld/Cargo.toml
src/libio/Cargo.toml
src/libksupport/Cargo.toml
src/runtime/Cargo.toml
src/satman/Cargo.toml

18
README.md
View File

@ -4,13 +4,14 @@ ARTIQ on Zynq
How to use
----------
1. Install the ARTIQ version that corresponds to the artiq-zynq version you are targeting.
2. To obtain firmware binaries, select the latest successful build on [Hydra](https://nixbld.m-labs.hk/) for the targeted artiq-zynq version, or use AFWS. If using Hydra, search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``).
3. Place the ``boot.bin`` file, obtained from Hydra's "binary distribution" download link or from AFWS, at the root of a FAT-formatted SD card.
4. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network.
5. Insert the SD card into the board and set up the board to boot from the SD card. For the ZC706, this is achieved by placing the large DIP switch SW11 in the 00110 position.
6. Power up the board. After the firmware starts successfully, it should respond to ping at its IP addresses, and boot messages can be observed from its UART at 115200bps.
7. Create and use an ARTIQ device database as usual, but set ``"target": "cortexa9"`` in the arguments of the core device.
1. Install the ARTIQ version that corresponds to the artiq-zynq version are targeting.
2. Select the latest successful build on [Hydra](https://nixbld.m-labs.hk/) for the targeted artiq-zynq version, or use AFWS to obtain firmware binaries.
3. Search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``).
4. Download the ``boot.bin`` "binary distribution" and place it at the root of a FAT-formatted SD card.
5. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network.
6. Insert the SD card into the board and set up the board to boot from the SD card. For the ZC706, this is achieved by placing the large DIP switch SW11 in the 00110 position.
7. Power up the board. After the firmware starts successfully, it should respond to ping at its IP addresses, and boot messages can be observed from its UART at 115200bps.
8. Create and use an ARTIQ device database as usual, but set ``"target": "cortexa9"`` in the arguments of the core device.
Configuration
-------------
@ -58,11 +59,12 @@ Notes:
- When calling make, you need to specify both the variant and firmware type.
- Firmware type must be either ``runtime`` for DRTIO-less or DRTIO master variants, or ``satman`` for DRTIO satellite.
- If the board is connected to the local machine, use the ``local_run.sh`` script.
- To update ``zynq-rs``, update the cargo files as per usual for Rust projects, but also keep ``flake.lock`` in sync.
License
-------
Copyright (C) 2019-2023 M-Labs Limited.
Copyright (C) 2019-2022 M-Labs Limited.
ARTIQ is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by

92
flake.lock
View File

@ -11,11 +11,11 @@
"src-pythonparser": "src-pythonparser"
},
"locked": {
"lastModified": 1697013694,
"narHash": "sha256-jmr+pFeyV0+nUbFY8/g2Aw4kU06i5CG0XhLobI8vUR8=",
"lastModified": 1682846196,
"narHash": "sha256-LXuksP08gTKw3TB6z8tH2FMYiNpghewcMtLgvhj69vM=",
"ref": "refs/heads/master",
"rev": "f7abc156cbc0a371452148d11436e5f241798092",
"revCount": 8569,
"rev": "0e7e30d46e91b7a4000f83413e7aae3d4a267aca",
"revCount": 8344,
"type": "git",
"url": "https://github.com/m-labs/artiq.git"
},
@ -37,11 +37,11 @@
]
},
"locked": {
"lastModified": 1693473687,
"narHash": "sha256-BdLddCWbvoEyakcGwhph9b5dIU1iA0hCQV7KYgU8nos=",
"lastModified": 1664405593,
"narHash": "sha256-yP441NerlLGig7n+9xHsx8yCtZ+Ggd0VqfBSzc20E04=",
"owner": "m-labs",
"repo": "artiq-comtools",
"rev": "f522ef3dbc65961f17b2d3d41e927409d970fd79",
"rev": "15ddac62813ef623a076ccf982b3bc63d314e651",
"type": "github"
},
"original": {
@ -51,15 +51,12 @@
}
},
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1692799911,
"narHash": "sha256-3eihraek4qL744EvQXsK1Ha6C3CR7nnT8X2qWap4RNk=",
"lastModified": 1659877975,
"narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "f9e7cf818399d17d347f847525c5a5a8032e4e44",
"rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
"type": "github"
},
"original": {
@ -71,11 +68,11 @@
"mozilla-overlay": {
"flake": false,
"locked": {
"lastModified": 1695805681,
"narHash": "sha256-1ElPLD8eFfnuIk0G52HGGpRtQZ4QPCjChRlEOfkZ5ro=",
"lastModified": 1677493379,
"narHash": "sha256-A1gO8zlWLv3+tZ3cGVB1WYvvoN9pbFyv0xIJHcTsckw=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "6eabade97bc28d707a8b9d82ad13ef143836736e",
"rev": "78e723925daf5c9e8d0a1837ec27059e61649cb6",
"type": "github"
},
"original": {
@ -87,11 +84,11 @@
"mozilla-overlay_2": {
"flake": false,
"locked": {
"lastModified": 1695805681,
"narHash": "sha256-1ElPLD8eFfnuIk0G52HGGpRtQZ4QPCjChRlEOfkZ5ro=",
"lastModified": 1677493379,
"narHash": "sha256-A1gO8zlWLv3+tZ3cGVB1WYvvoN9pbFyv0xIJHcTsckw=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "6eabade97bc28d707a8b9d82ad13ef143836736e",
"rev": "78e723925daf5c9e8d0a1837ec27059e61649cb6",
"type": "github"
},
"original": {
@ -103,11 +100,11 @@
"mozilla-overlay_3": {
"flake": false,
"locked": {
"lastModified": 1690536331,
"narHash": "sha256-aRIf2FB2GTdfF7gl13WyETmiV/J7EhBGkSWXfZvlxcA=",
"lastModified": 1664789696,
"narHash": "sha256-UGWJHQShiwLCr4/DysMVFrYdYYHcOqAOVsWNUu+l6YU=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "db89c8707edcffefcd8e738459d511543a339ff5",
"rev": "80627b282705101e7b38e19ca6e8df105031b072",
"type": "github"
},
"original": {
@ -118,16 +115,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1696697597,
"narHash": "sha256-q26Qv4DQ+h6IeozF2o1secyQG0jt2VUT3V0K58jr3pg=",
"lastModified": 1682173319,
"narHash": "sha256-tPhOpJJ+wrWIusvGgIB2+x6ILfDkEgQMX0BTtM5vd/4=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "5a237aecb57296f67276ac9ab296a41c23981f56",
"rev": "ee7ec1c71adc47d2e3c2d5eb0d6b8fbbd42a8d1c",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05",
"ref": "nixos-22.11",
"repo": "nixpkgs",
"type": "github"
}
@ -147,11 +144,11 @@
]
},
"locked": {
"lastModified": 1693473454,
"narHash": "sha256-kr8Ur6JNW/xVRHdPn3ou980IAxg/n+f3ZQBHuJ1uaC4=",
"lastModified": 1681290481,
"narHash": "sha256-VEZcGhbtJGonRrrWi31evNDVSerlLjEPL0MZGm9VlB8=",
"owner": "m-labs",
"repo": "sipyco",
"rev": "5467dcf9738673ab9a49e6f2377bda7c551b5f90",
"rev": "727631ada6e59dc6ef0ad50bfcc376d2ffe805aa",
"type": "github"
},
"original": {
@ -163,11 +160,11 @@
"src-migen": {
"flake": false,
"locked": {
"lastModified": 1697013661,
"narHash": "sha256-qNCqgWyE4vTDmyjE2XMJqW1djuBxT25A36AzQfZqluU=",
"lastModified": 1674045327,
"narHash": "sha256-oYdeY0MbTReKbAwmSznnqw0wNawdInJoFJVWW3tesFA=",
"owner": "m-labs",
"repo": "migen",
"rev": "aadc19df93b7aa9ca761aaebbb98a11e0cf2d7c7",
"rev": "ccaee68e14d3636e1d8fb2e0864dd89b1b1f7384",
"type": "github"
},
"original": {
@ -179,11 +176,11 @@
"src-misoc": {
"flake": false,
"locked": {
"lastModified": 1693709836,
"narHash": "sha256-YiCk05RYLzZu1CYkQ2r7XtjwVEqkUGTQn388uOls9tI=",
"lastModified": 1679903508,
"narHash": "sha256-TI0agjSSMJtH4mgAMpSO128zxcwSo/AjY1B6AW7zBQQ=",
"ref": "refs/heads/master",
"rev": "58dc4ee60d165ce9145cf3d904241fc154b6407f",
"revCount": 2448,
"rev": "0cf0ebb7d4f56cc6d44a3dea3e386efab9d82419",
"revCount": 2437,
"submodules": true,
"type": "git",
"url": "https://github.com/m-labs/misoc.git"
@ -210,21 +207,6 @@
"type": "github"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"zynq-rs": {
"inputs": {
"mozilla-overlay": "mozilla-overlay_3",
@ -234,11 +216,11 @@
]
},
"locked": {
"lastModified": 1693479539,
"narHash": "sha256-MppR7yxs3cjG7tQc82vX0MhyN71CJL2QWkM65F5hrFU=",
"lastModified": 1669819016,
"narHash": "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=",
"ref": "refs/heads/master",
"rev": "c15b54f92b3d4e125ae47a0dce7abe4b2bc9e054",
"revCount": 628,
"rev": "67dbb5932fa8ff5f143983476f741f945871d286",
"revCount": 624,
"type": "git",
"url": "https://git.m-labs.hk/m-labs/zynq-rs"
},

47
flake.nix
View File

@ -12,7 +12,6 @@
zynqpkgs = zynq-rs.packages.x86_64-linux;
artiqpkgs = artiq.packages.x86_64-linux;
rust = zynq-rs.rust;
rustPlatform = zynq-rs.rustPlatform;
fastnumbers = pkgs.python3Packages.buildPythonPackage rec {
@ -50,7 +49,7 @@
nativeBuildInputs = with pkgs.python3Packages; [ pbr ];
propagatedBuildInputs = with pkgs.python3Packages; [ future fastnumbers ];
checkInputs = with pkgs.python3Packages; [ pytest ];
checkInputs = with pkgs.python3Packages; [ pytest pytest-flake8 ];
checkPhase = "pytest";
doCheck = false;
@ -74,7 +73,8 @@
propagatedBuildInputs = with pkgs.python3Packages; [ setuptools click numpy toolz jinja2 ramda artiqpkgs.migen artiqpkgs.misoc ];
checkInputs = with pkgs.python3Packages; [ pytest-runner pytestCheckHook pytest-timeout ];
checkInputs = with pkgs.python3Packages; [ pytest pytest-timeout pytest-flake8 ];
checkPhase = "pytest";
# migen/misoc version checks are broken with pyproject for some reason
postPatch = ''
@ -82,8 +82,6 @@
--replace '"migen@git+https://github.com/m-labs/migen",' ""
substituteInPlace pyproject.toml \
--replace '"misoc@git+https://github.com/m-labs/misoc.git",' ""
# pytest-flake8 is broken with recent flake8. Re-enable after fix.
substituteInPlace setup.cfg --replace '--flake8' ""
'';
};
binutils = { platform, target, zlib }: pkgs.stdenv.mkDerivation rec {
@ -118,24 +116,28 @@
fwtype = if builtins.elem variant sat_variants then "satman" else "runtime";
firmware = rustPlatform.buildRustPackage rec {
name = "firmware";
src = ./src;
cargoLock = {
lockFile = src/Cargo.lock;
outputHashes = {
"libasync-0.0.0" = "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=";
};
};
nativeBuildInputs = [
pkgs.gnumake
(pkgs.python3.withPackages(ps: [ ps.jsonschema artiqpkgs.migen migen-axi artiqpkgs.misoc artiqpkgs.artiq ]))
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
zynqpkgs.cargo-xbuild
pkgs.llvmPackages_9.llvm
pkgs.llvmPackages_9.clang-unwrapped
];
buildPhase = ''
export XARGO_RUST_SRC="${rust}/lib/rustlib/src/rust/library"
export XARGO_RUST_SRC="${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library"
export CLANG_EXTRA_INCLUDE_DIR="${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include"
export CARGO_HOME=$(mktemp -d cargo-home.XXX)
export ZYNQ_RS=${zynq-rs}
make TARGET=${target} GWARGS="${if json == null then "-V ${variant}" else json}" ${fwtype}
'';
@ -153,7 +155,8 @@
gateware = pkgs.runCommand "${target}-${variant}-gateware"
{
nativeBuildInputs = [
(pkgs.python3.withPackages(ps: [ ps.jsonschema artiqpkgs.migen migen-axi artiqpkgs.misoc artiqpkgs.artiq ]))
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
artiqpkgs.vivado
];
}
@ -237,7 +240,8 @@
name = "gateware-sim";
nativeBuildInputs = [
(pkgs.python3.withPackages(ps: [ artiqpkgs.migen migen-axi artiqpkgs.artiq ]))
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi artiq ])))
artiqpkgs.artiq
];
phases = [ "buildPhase" ];
@ -249,19 +253,18 @@
'';
};
fmt-check = pkgs.stdenvNoCC.mkDerivation {
fmt-check = pkgs.stdenv.mkDerivation {
name = "fmt-check";
src = ./src;
nativeBuildInputs = [
rustPlatform.rust.cargo
];
nativeBuildInputs = [ rust pkgs.gnumake ];
phases = [ "unpackPhase" "buildPhase" ];
phases = [ "buildPhase" ];
buildPhase =
''
export ZYNQ_RS=${zynq-rs}
make manifests
cd ${self}/src
cargo fmt -- --check
touch $out
'';
@ -337,22 +340,18 @@
} //
(build { target = "zc706"; variant = "nist_clock"; }) //
(build { target = "zc706"; variant = "nist_clock_master"; }) //
(build { target = "zc706"; variant = "nist_clock_master_100mhz"; }) //
(build { target = "zc706"; variant = "nist_clock_satellite"; }) //
(build { target = "zc706"; variant = "nist_clock_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "nist_qc2"; }) //
(build { target = "zc706"; variant = "nist_qc2_master"; }) //
(build { target = "zc706"; variant = "nist_qc2_master_100mhz"; }) //
(build { target = "zc706"; variant = "nist_qc2_satellite"; }) //
(build { target = "zc706"; variant = "nist_qc2_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_master"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_master_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_satellite"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_master"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_master_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_satellite"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_satellite_100mhz"; }) //
(build { target = "kasli_soc"; variant = "demo"; json = ./demo.json; }) //
@ -364,7 +363,8 @@
devShell.x86_64-linux = pkgs.mkShell {
name = "artiq-zynq-dev-shell";
buildInputs = with pkgs; [
rust
rustPlatform.rust.rustc
rustPlatform.rust.cargo
llvmPackages_9.llvm
llvmPackages_9.clang-unwrapped
gnumake
@ -378,9 +378,8 @@
artiqpkgs.vivado
binutils-arm
];
XARGO_RUST_SRC = "${rust}/lib/rustlib/src/rust/library";
XARGO_RUST_SRC = "${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library";
CLANG_EXTRA_INCLUDE_DIR = "${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include";
ZYNQ_RS = "${zynq-rs}";
OPENOCD_ZYNQ = "${zynq-rs}/openocd";
SZL = "${zynqpkgs.szl}";
};

42
src/Cargo.lock generated
View File

@ -218,37 +218,10 @@ dependencies = [
"libsupport_zynq",
]
[[package]]
name = "ksupport"
version = "0.1.0"
dependencies = [
"build_zynq",
"byteorder",
"core_io",
"cslice",
"dwarf",
"dyld",
"io",
"libasync",
"libboard_artiq",
"libboard_zynq",
"libc",
"libconfig",
"libcortex_a9",
"libm",
"libregister",
"libsupport_zynq",
"log",
"log_buffer",
"nb 0.1.3",
"unwind",
"vcell",
"void",
]
[[package]]
name = "libasync"
version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"embedded-hal",
"libcortex_a9",
@ -271,7 +244,6 @@ dependencies = [
"libconfig",
"libcortex_a9",
"libregister",
"libsupport_zynq",
"log",
"log_buffer",
"nb 1.0.0",
@ -281,6 +253,7 @@ dependencies = [
[[package]]
name = "libboard_zynq"
version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"bit_field",
"embedded-hal",
@ -305,6 +278,7 @@ dependencies = [
[[package]]
name = "libconfig"
version = "0.1.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"core_io",
"fatfs",
@ -315,6 +289,7 @@ dependencies = [
[[package]]
name = "libcortex_a9"
version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"bit_field",
"libregister",
@ -330,6 +305,7 @@ checksum = "348108ab3fba42ec82ff6e9564fc4ca0247bdccdc68dd8af9764bbc79c3c8ffb"
[[package]]
name = "libregister"
version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"bit_field",
"vcell",
@ -339,6 +315,7 @@ dependencies = [
[[package]]
name = "libsupport_zynq"
version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286"
dependencies = [
"cc",
"compiler_builtins",
@ -461,17 +438,18 @@ dependencies = [
"embedded-hal",
"futures",
"io",
"ksupport",
"libasync",
"libboard_artiq",
"libboard_zynq",
"libc",
"libconfig",
"libcortex_a9",
"libm",
"libregister",
"libsupport_zynq",
"log",
"log_buffer",
"nb 0.1.3",
"num-derive",
"num-traits",
"unwind",
@ -493,11 +471,7 @@ name = "satman"
version = "0.0.0"
dependencies = [
"build_zynq",
"core_io",
"cslice",
"embedded-hal",
"io",
"ksupport",
"libasync",
"libboard_artiq",
"libboard_zynq",

1
src/Cargo.toml
View File

@ -5,7 +5,6 @@ members = [
"libdwarf",
"libio",
"libunwind",
"libksupport",
"runtime",
"satman"
]

15
src/Makefile
View File

@ -7,20 +7,13 @@ runtime: ../build/runtime.bin
satman: ../build/satman.bin
.PHONY: all manifests
manifests = libboard_artiq/Cargo.toml libc/Cargo.toml libdyld/Cargo.toml libio/Cargo.toml libksupport/Cargo.toml runtime/Cargo.toml satman/Cargo.toml
$(manifests): %.toml: %.toml.tpl
sed s+@@ZYNQ_RS@@+$(ZYNQ_RS)+g $< > $@
manifests: $(manifests)
.PHONY: all runtime_target satman_target
../build/pl.rs ../build/rustc-cfg ../build/mem.rs: gateware/*
mkdir -p ../build
python gateware/$(TARGET).py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs $(GWARGS)
../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(manifests) $(shell find . -type f -not -name Cargo.toml -print)
../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print)
cd runtime && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \
@ -30,7 +23,7 @@ manifests: $(manifests)
../build/runtime.bin: ../build/firmware/armv7-none-eabihf/release/runtime
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin
../build/firmware/armv7-none-eabihf/release/satman: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(manifests) $(shell find . -type f -not -name Cargo.toml -print)
../build/firmware/armv7-none-eabihf/release/satman: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print)
cd satman && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \
@ -38,4 +31,4 @@ manifests: $(manifests)
--no-default-features --features=target_$(TARGET)
../build/satman.bin: ../build/firmware/armv7-none-eabihf/release/satman
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/satman ../build/satman.bin
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/satman ../build/satman.bin

16
src/gateware/config.py
View File

@ -1,16 +0,0 @@
from misoc.integration import cpu_interface
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_rust_cfg(
soc.get_csr_regions(), soc.get_constants()))

255
src/gateware/kasli_soc.py
View File

@ -10,13 +10,13 @@ from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import kasli_soc
from misoc.interconnect.csr import *
from misoc.cores import virtual_leds
from misoc.integration import cpu_interface
from artiq.coredevice import jsondesc
from artiq.gateware import rtio, eem_7series
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from artiq.gateware.rtio.phy import ttl_simple
from artiq.gateware.drtio.transceiver import gtx_7series, eem_serdes
from artiq.gateware.drtio.transceiver import gtx_7series
from artiq.gateware.drtio.siphaser import SiPhaser7Series
from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer
from artiq.gateware.drtio import *
@ -26,7 +26,7 @@ import analyzer
import acpki
import drtio_aux_controller
import zynq_clocking
from config import write_csr_file, write_mem_file, write_rustc_cfg_file
eem_iostandard_dict = {
0: "LVDS_25",
@ -61,52 +61,27 @@ class SMAClkinForward(Module):
]
class GTPBootstrapClock(Module):
def __init__(self, platform, freq=125e6):
class GTP125BootstrapClock(Module):
def __init__(self, platform):
self.clock_domains.cd_bootstrap = ClockDomain(reset_less=True)
self.cd_bootstrap.clk.attr.add("keep")
bootstrap_125 = platform.request("clk125_gtp")
bootstrap_se = Signal()
clk_out = Signal()
platform.add_period_constraint(bootstrap_125.p, 8.0)
self.specials += [
Instance("IBUFDS_GTE2",
p_CLKSWING_CFG="0b11",
i_CEB=0,
i_I=bootstrap_125.p, i_IB=bootstrap_125.n,
o_O=bootstrap_se,
p_CLKCM_CFG="TRUE",
p_CLKRCV_TRST="TRUE",
p_CLKSWING_CFG=3),
Instance("BUFG", i_I=bootstrap_se, o_O=clk_out)
i_I=bootstrap_125.p, i_IB=bootstrap_125.n, o_O=bootstrap_se),
Instance("BUFG", i_I=bootstrap_se, o_O=self.cd_bootstrap.clk)
]
if freq == 125e6:
self.comb += self.cd_bootstrap.clk.eq(clk_out)
elif freq == 100e6:
pll_fb = Signal()
pll_out = Signal()
self.specials += [
Instance("PLLE2_BASE",
p_CLKIN1_PERIOD=8.0,
i_CLKIN1=clk_out,
i_CLKFBIN=pll_fb,
o_CLKFBOUT=pll_fb,
# VCO @ 1GHz
p_CLKFBOUT_MULT=8, p_DIVCLK_DIVIDE=1,
# 100MHz for bootstrap
p_CLKOUT1_DIVIDE=10, p_CLKOUT1_PHASE=0.0, o_CLKOUT1=pll_out,
),
Instance("BUFG", i_I=pll_out, o_O=self.cd_bootstrap.clk)
]
else:
raise ValueError("Bootstrap frequency must be 100 or 125MHz")
class GenericStandalone(SoCCore):
def __init__(self, description, acpki=False):
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
@ -117,30 +92,23 @@ class GenericStandalone(SoCCore):
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"]
self.submodules += SMAClkinForward(self.platform)
self.config["HAS_SI5324"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_soft_reset"] = None
clk_synth = platform.request("cdr_clk_clean_fabric")
clk_synth_se = Signal()
clk_synth_se_buf = Signal()
platform.add_period_constraint(clk_synth.p, 8.0)
self.specials += [
Instance("IBUFGDS",
self.specials += Instance("IBUFGDS",
p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE",
i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se
),
Instance("BUFG", i_I=clk_synth_se, o_O=clk_synth_se_buf),
]
i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
fix_serdes_timing_path(platform)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, description["rtio_frequency"])
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, clk_synth_se_buf)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, clk_synth_se)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
@ -162,20 +130,18 @@ class GenericStandalone(SoCCore):
self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
@ -195,7 +161,7 @@ class GenericStandalone(SoCCore):
self.csr_devices.append("rtio_analyzer")
if has_grabber:
self.config["HAS_GRABBER"] = None
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
for grabber in self.grabber_csr_group:
self.platform.add_false_path_constraints(
@ -206,8 +172,8 @@ class GenericMaster(SoCCore):
def __init__(self, description, acpki=False):
clk_freq = description["rtio_frequency"]
has_drtio_over_eem = any(peripheral["type"] == "shuttler" for peripheral in description["peripherals"])
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
@ -218,23 +184,21 @@ class GenericMaster(SoCCore):
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"]
self.submodules += SMAClkinForward(self.platform)
data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.gt_drtio = gtx_7series.GTX(
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("gt_drtio")
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0]
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, clk_freq)
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
@ -247,13 +211,11 @@ class GenericMaster(SoCCore):
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
fix_serdes_timing_path(platform)
self.config["HAS_SI5324"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_soft_reset"] = None
self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_drtio_over_eem:
self.eem_drtio_channels = []
if has_grabber:
self.grabber_csr_group = []
eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard)
@ -268,21 +230,21 @@ class GenericMaster(SoCCore):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.drtio_csr_group = []
self.drtioaux_csr_group = []
self.drtioaux_memory_group = []
drtio_csr_group = []
drtioaux_csr_group = []
drtioaux_memory_group = []
self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)):
for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
self.drtio_csr_group.append(core_name)
self.drtioaux_csr_group.append(coreaux_name)
self.drtioaux_memory_group.append(memory_name)
drtio_csr_group.append(core_name)
drtioaux_csr_group.append(coreaux_name)
drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster(self.rtio_tsc, self.gt_drtio.channels[i]))
core = cdr(DRTIOMaster(self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
@ -295,27 +257,24 @@ class GenericMaster(SoCCore):
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size)
self.axi2csr.register_port(coreaux.get_rx_port(), size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2)
self.config["HAS_DRTIO"] = None
self.config["HAS_DRTIO_ROUTING"] = None
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
if has_drtio_over_eem:
self.add_eem_drtio(self.eem_drtio_channels)
self.add_drtio_cpuif_groups()
self.submodules.rtio_core = rtio.Core(
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
@ -339,52 +298,8 @@ class GenericMaster(SoCCore):
self.csr_devices.append("rtio_analyzer")
if has_grabber:
self.config["HAS_GRABBER"] = None
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
self.submodules.virtual_leds = virtual_leds.VirtualLeds()
self.csr_devices.append("virtual_leds")
self.comb += [self.virtual_leds.get(i).eq(channel.rx_ready)
for i, channel in enumerate(self.gt_drtio.channels)]
def add_eem_drtio(self, eem_drtio_channels):
# Must be called before invoking add_rtio() to construct the CRI
# interconnect properly
self.submodules.eem_transceiver = eem_serdes.EEMSerdes(self.platform, eem_drtio_channels)
self.csr_devices.append("eem_transceiver")
self.config["HAS_DRTIO_EEM"] = None
self.config["EEM_DRTIO_COUNT"] = len(eem_drtio_channels)
cdr = ClockDomainsRenamer({"rtio_rx": "sys"})
for i in range(len(self.eem_transceiver.channels)):
channel = i + len(self.gt_drtio.channels)
core_name = "drtio" + str(channel)
coreaux_name = "drtioaux" + str(channel)
memory_name = "drtioaux" + str(channel) + "_mem"
self.drtio_csr_group.append(core_name)
self.drtioaux_csr_group.append(coreaux_name)
self.drtioaux_memory_group.append(memory_name)
core = cdr(DRTIOMaster(self.rtio_tsc, self.eem_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
size = coreaux.get_mem_size()
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size)
self.axi2csr.register_port(coreaux.get_rx_port(), size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2)
def add_drtio_cpuif_groups(self):
self.add_csr_group("drtio", self.drtio_csr_group)
self.add_csr_group("drtioaux", self.drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", self.drtioaux_memory_group)
class GenericSatellite(SoCCore):
@ -392,6 +307,7 @@ class GenericSatellite(SoCCore):
clk_freq = description["rtio_frequency"]
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
@ -402,21 +318,19 @@ class GenericSatellite(SoCCore):
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.config["HW_REV"] = description["hw_rev"]
data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.gt_drtio = gtx_7series.GTX(
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("gt_drtio")
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0]
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = GTPBootstrapClock(self.platform, clk_freq)
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
@ -450,7 +364,7 @@ class GenericSatellite(SoCCore):
drtioaux_memory_group = []
drtiorep_csr_group = []
self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)):
for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name)
@ -461,7 +375,7 @@ class GenericSatellite(SoCCore):
if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite(
self.rtio_tsc, self.gt_drtio.channels[i],
self.rtio_tsc, self.drtio_transceiver.channels[i],
self.rx_synchronizer))
self.submodules.drtiosat = core
self.csr_devices.append("drtiosat")
@ -470,7 +384,7 @@ class GenericSatellite(SoCCore):
drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater(
self.rtio_tsc, self.gt_drtio.channels[i]))
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name)
@ -488,34 +402,32 @@ class GenericSatellite(SoCCore):
# and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None
self.config["HAS_DRTIO_ROUTING"] = None
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.add_csr_group("drtiorep", drtiorep_csr_group)
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.local_io = SyncRTIO(
self.rtio_tsc, self.rtio_channels, lane_count=description["sed_lanes"]
)
self.submodules.local_io = SyncRTIO(self.rtio_tsc, self.rtio_channels)
self.comb += self.drtiosat.async_errors.eq(self.local_io.async_errors)
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri],
[self.drtiosat.cri, self.rtio_dma.cri],
[self.local_io.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
@ -526,36 +438,49 @@ class GenericSatellite(SoCCore):
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.local_io.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
rtio_clk_period = 1e9/clk_freq
self.config["RTIO_FREQUENCY"] = str(clk_freq/1e6)
self.rustc_cfg["rtio_frequency"] = str(clk_freq/1e6)
self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("cdr_clk"),
rx_synchronizer=self.rx_synchronizer,
ultrascale=False,
rtio_clk_freq=self.gt_drtio.rtio_clk_freq)
rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
self.csr_devices.append("siphaser")
self.config["HAS_SI5324"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["has_siphaser"] = None
self.rustc_cfg["si5324_soft_reset"] = None
gtx0 = self.gt_drtio.gtxs[0]
gtx0 = self.drtio_transceiver.gtxs[0]
platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk)
if has_grabber:
self.config["HAS_GRABBER"] = None
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
# no RTIO CRG here
self.submodules.virtual_leds = virtual_leds.VirtualLeds()
self.csr_devices.append("virtual_leds")
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
self.comb += [self.virtual_leds.get(i).eq(channel.rx_ready)
for i, channel in enumerate(self.gt_drtio.channels)]
def main():
parser = argparse.ArgumentParser(
@ -578,14 +503,14 @@ def main():
if description["target"] != "kasli_soc":
raise ValueError("Description is for a different target")
if description["drtio_role"] == "standalone":
if description["base"] == "standalone":
cls = GenericStandalone
elif description["drtio_role"] == "master":
elif description["base"] == "master":
cls = GenericMaster
elif description["drtio_role"] == "satellite":
elif description["base"] == "satellite":
cls = GenericSatellite
else:
raise ValueError("Invalid DRTIO role")
raise ValueError("Invalid base")
soc = cls(description, acpki=args.acpki)
soc.finalize()

106
src/gateware/zc706.py
View File

@ -10,6 +10,7 @@ from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import zc706
from misoc.interconnect.csr import *
from misoc.integration import cpu_interface
from misoc.cores import gpio
from artiq.gateware import rtio, nist_clock, nist_qc2
@ -25,7 +26,7 @@ import analyzer
import acpki
import drtio_aux_controller
import zynq_clocking
from config import write_csr_file, write_mem_file, write_rustc_cfg_file
class SMAClkinForward(Module):
def __init__(self, platform):
@ -126,6 +127,7 @@ def prepare_zc706_platform(platform):
class ZC706(SoCCore):
def __init__(self, acpki=False):
self.acpki = acpki
self.rustc_cfg = dict()
platform = zc706.Platform()
prepare_zc706_platform(platform)
@ -147,14 +149,14 @@ class ZC706(SoCCore):
i_CEB=0,
i_I=si5324_out.p, i_IB=si5324_out.n,
o_O=cdr_clk,
p_CLKCM_CFG="TRUE",
p_CLKRCV_TRST="TRUE",
p_CLKSWING_CFG=3),
p_CLKCM_CFG="0b1",
p_CLKRCV_TRST="0b1",
p_CLKSWING_CFG="0b11"),
Instance("BUFG", i_I=cdr_clk, o_O=cdr_clk_buf)
]
self.config["HAS_SI5324"] = None
self.config["SI5324_AS_SYNTHESIZER"] = None
self.config["SI5324_SOFT_RESET"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_as_synthesizer"] = None
self.rustc_cfg["si5324_soft_reset"] = None
self.submodules.bootstrap = CLK200BootstrapClock(platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, cdr_clk_buf)
@ -168,14 +170,14 @@ class ZC706(SoCCore):
self.csr_devices.append("rtio_core")
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
@ -198,6 +200,7 @@ class ZC706(SoCCore):
class _MasterBase(SoCCore):
def __init__(self, acpki=False, drtio100mhz=False):
self.acpki = acpki
self.rustc_cfg = dict()
clk_freq = 100e6 if drtio100mhz else 125e6
@ -219,15 +222,15 @@ class _MasterBase(SoCCore):
self.submodules += SMAClkinForward(self.platform)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.gt_drtio = gtx_7series.GTX(
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("gt_drtio")
self.csr_devices.append("drtio_transceiver")
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
txout_buf = Signal()
gtx0 = self.gt_drtio.gtxs[0]
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = CLK200BootstrapClock(platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
@ -244,7 +247,7 @@ class _MasterBase(SoCCore):
drtioaux_csr_group = []
drtioaux_memory_group = []
self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)):
for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
@ -255,7 +258,7 @@ class _MasterBase(SoCCore):
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster(
self.rtio_tsc, self.gt_drtio.channels[i]))
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
@ -268,18 +271,18 @@ class _MasterBase(SoCCore):
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), mem_size)
self.axi2csr.register_port(coreaux.get_rx_port(), mem_size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None
self.config["HAS_DRTIO_ROUTING"] = None
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.config["RTIO_FREQUENCY"] = str(self.gt_drtio.rtio_clk_freq/1e6)
self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n")
self.config["HAS_SI5324"] = None
self.config["SI5324_AS_SYNTHESIZER"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_as_synthesizer"] = None
# Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX)
@ -287,7 +290,7 @@ class _MasterBase(SoCCore):
gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX)
for gtx in self.gt_drtio.gtxs[1:]:
for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints(
gtx0.txoutclk, gtx.rxoutclk)
@ -299,14 +302,14 @@ class _MasterBase(SoCCore):
self.csr_devices.append("rtio_core")
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
@ -333,6 +336,7 @@ class _MasterBase(SoCCore):
class _SatelliteBase(SoCCore):
def __init__(self, acpki=False, drtio100mhz=False):
self.acpki = acpki
self.rustc_cfg = dict()
clk_freq = 100e6 if drtio100mhz else 125e6
@ -355,15 +359,15 @@ class _SatelliteBase(SoCCore):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.gt_drtio = gtx_7series.GTX(
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("gt_drtio")
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
txout_buf.attr.add("keep")
gtx0 = self.gt_drtio.gtxs[0]
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance(
"BUFG",
i_I=gtx0.txoutclk,
@ -383,7 +387,7 @@ class _SatelliteBase(SoCCore):
drtioaux_memory_group = []
drtiorep_csr_group = []
self.drtio_cri = []
for i in range(len(self.gt_drtio.channels)):
for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name)
@ -395,7 +399,7 @@ class _SatelliteBase(SoCCore):
if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite(
self.rtio_tsc, self.gt_drtio.channels[0], self.rx_synchronizer))
self.rtio_tsc, self.drtio_transceiver.channels[0], self.rx_synchronizer))
self.submodules.drtiosat = core
self.csr_devices.append("drtiosat")
# Repeaters
@ -403,7 +407,7 @@ class _SatelliteBase(SoCCore):
corerep_name = "drtiorep" + str(i-1)
drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater(
self.rtio_tsc, self.gt_drtio.channels[i]))
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name)
@ -421,35 +425,36 @@ class _SatelliteBase(SoCCore):
# and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.config["HAS_DRTIO"] = None
self.config["HAS_DRTIO_ROUTING"] = None
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_csr_group("drtiorep", drtiorep_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.config["RTIO_FREQUENCY"] = str(self.gt_drtio.rtio_clk_freq/1e6)
self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
# Si5324 Phaser
self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("si5324_clkin"),
rx_synchronizer=self.rx_synchronizer,
ultrascale=False,
rtio_clk_freq=self.gt_drtio.rtio_clk_freq)
rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, self.siphaser.mmcm_freerun_output)
self.csr_devices.append("siphaser")
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n")
self.config["HAS_SI5324"] = None
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["has_siphaser"] = None
rtio_clk_period = 1e9/self.gt_drtio.rtio_clk_freq
rtio_clk_period = 1e9/self.drtio_transceiver.rtio_clk_freq
# Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX)
platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX)
for gtx in self.gt_drtio.gtxs[1:]:
for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, gtx.rxoutclk)
@ -460,14 +465,14 @@ class _SatelliteBase(SoCCore):
self.csr_devices.append("rtio_moninj")
if self.acpki:
self.config["KI_IMPL"] = "acp"
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.config["KI_IMPL"] = "csr"
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
@ -476,15 +481,11 @@ class _SatelliteBase(SoCCore):
self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels)
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri, self.rtio.cri],
[self.drtiosat.cri, self.rtio_dma.cri],
[self.local_io.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.local_io.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table")
@ -671,6 +672,27 @@ class NIST_QC2_Satellite(_SatelliteBase, _NIST_QC2_RTIO):
VARIANTS = {cls.__name__.lower(): cls for cls in [NIST_CLOCK, NIST_CLOCK_Master, NIST_CLOCK_Satellite,
NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite]}
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main():
parser = argparse.ArgumentParser(
description="ARTIQ port to the ZC706 Zynq development kit")

70
src/gateware/zynq_clocking.py
View File

@ -69,8 +69,6 @@ class SYSCRG(Module, AutoCSR):
# assumes bootstrap clock is same freq as main and sys output
self.clock_domains.cd_sys = ClockDomain()
self.clock_domains.cd_sys4x = ClockDomain(reset_less=True)
self.clock_domains.cd_sys5x = ClockDomain(reset_less=True)
self.clock_domains.cd_clk200 = ClockDomain()
self.current_clock = CSRStatus()
@ -80,6 +78,11 @@ class SYSCRG(Module, AutoCSR):
period = 1e9/freq
pll_locked = Signal()
pll_sys = Signal()
pll_sys4x = Signal()
fb_clk = Signal()
self.submodules.clk_sw_fsm = ClockSwitchFSM()
if clk_sw is None:
@ -88,55 +91,32 @@ class SYSCRG(Module, AutoCSR):
else:
self.comb += self.clk_sw_fsm.i_clk_sw.eq(clk_sw)
mmcm_locked = Signal()
mmcm_sys = Signal()
mmcm_sys4x = Signal()
mmcm_sys5x = Signal()
mmcm_clk208 = Signal()
mmcm_fb_clk = Signal()
self.specials += [
Instance("MMCME2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=mmcm_locked,
p_BANDWIDTH="HIGH",
p_REF_JITTER1=0.001,
p_CLKIN1_PERIOD=period, i_CLKIN1=main_clk,
p_CLKIN2_PERIOD=period, i_CLKIN2=bootstrap_clk,
i_CLKINSEL=self.clk_sw_fsm.o_clk_sw,
Instance("PLLE2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
p_BANDWIDTH="HIGH",
p_REF_JITTER1=0.001,
p_CLKIN1_PERIOD=period, i_CLKIN1=main_clk,
p_CLKIN2_PERIOD=period, i_CLKIN2=bootstrap_clk,
i_CLKINSEL=self.clk_sw_fsm.o_clk_sw,
# VCO @ 1.25GHz
p_CLKFBOUT_MULT_F=10, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=mmcm_fb_clk,
i_RST=self.clk_sw_fsm.o_reset,
# VCO @ 1.5GHz when using 125MHz input
# 1.2GHz for 100MHz (zc706)
p_CLKFBOUT_MULT=12, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=fb_clk,
i_RST=self.clk_sw_fsm.o_reset,
o_CLKFBOUT=mmcm_fb_clk,
o_CLKFBOUT=fb_clk,
p_CLKOUT0_DIVIDE_F=2.5, p_CLKOUT0_PHASE=0.0, o_CLKOUT0=mmcm_sys4x,
p_CLKOUT0_DIVIDE=3, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=pll_sys4x,
# 125MHz
p_CLKOUT1_DIVIDE=10, p_CLKOUT1_PHASE=0.0, o_CLKOUT1=mmcm_sys,
p_CLKOUT1_DIVIDE=12, p_CLKOUT1_PHASE=0.0,
o_CLKOUT1=pll_sys),
Instance("BUFG", i_I=pll_sys, o_O=self.cd_sys.clk),
Instance("BUFG", i_I=pll_sys4x, o_O=self.cd_sys4x.clk),
# 625MHz
p_CLKOUT2_DIVIDE=2, p_CLKOUT2_PHASE=0.0, o_CLKOUT2=mmcm_sys5x,
# 208MHz
p_CLKOUT3_DIVIDE=6, p_CLKOUT3_PHASE=0.0, o_CLKOUT3=mmcm_clk208,
),
Instance("BUFG", i_I=mmcm_sys5x, o_O=self.cd_sys5x.clk),
Instance("BUFG", i_I=mmcm_sys, o_O=self.cd_sys.clk),
Instance("BUFG", i_I=mmcm_sys4x, o_O=self.cd_sys4x.clk),
Instance("BUFG", i_I=mmcm_clk208, o_O=self.cd_clk200.clk),
AsyncResetSynchronizer(self.cd_sys, ~mmcm_locked),
AsyncResetSynchronizer(self.cd_clk200, ~mmcm_locked),
AsyncResetSynchronizer(self.cd_sys, ~pll_locked),
]
reset_counter = Signal(4, reset=15)
ic_reset = Signal(reset=1)
self.sync.clk200 += \
If(reset_counter != 0,
reset_counter.eq(reset_counter - 1)
).Else(
ic_reset.eq(0)
)
self.specials += Instance("IDELAYCTRL", i_REFCLK=ClockSignal("clk200"), i_RST=ic_reset)
self.comb += self.current_clock.status.eq(self.clk_sw_fsm.o_clk_sw)

11
src/libboard_artiq/Cargo.toml.tpl → src/libboard_artiq/Cargo.toml
View File

@ -24,9 +24,8 @@ nb = "1.0"
void = { version = "1", default-features = false }
io = { path = "../libio", features = ["byteorder"] }
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq" }
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git"}
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn"] }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }

228
src/libboard_artiq/src/drtio_eem.rs
View File

@ -1,228 +0,0 @@
use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs;
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use libsupport_zynq::alloc::format;
use log::{debug, error, info};
use crate::pl;
struct SerdesConfig {
pub delay: [u8; 4],
}
impl SerdesConfig {
pub fn as_bytes(&self) -> &[u8] {
unsafe {
core::slice::from_raw_parts(
(self as *const SerdesConfig) as *const u8,
core::mem::size_of::<SerdesConfig>(),
)
}
}
}
fn select_lane(lane_no: u8) {
unsafe {
pl::csr::eem_transceiver::lane_sel_write(lane_no);
}
}
fn apply_delay(tap: u8, timer: &mut GlobalTimer) {
unsafe {
pl::csr::eem_transceiver::dly_cnt_in_write(tap);
pl::csr::eem_transceiver::dly_ld_write(1);
timer.delay_us(1);
assert!(tap as u8 == pl::csr::eem_transceiver::dly_cnt_out_read());
}
}
fn apply_config(config: &SerdesConfig, timer: &mut GlobalTimer) {
for lane_no in 0..4 {
select_lane(lane_no as u8);
apply_delay(config.delay[lane_no], timer);
}
}
unsafe fn assign_delay(timer: &mut GlobalTimer) -> SerdesConfig {
// Select an appropriate delay for lane 0
select_lane(0);
//
let mut best_dly = None;
loop {
let mut prev = None;
for curr_dly in 0..32 {
//let read_align = read_align_fn(curr_dly, timer);
let curr_low_rate = read_align(curr_dly, timer);
if let Some(prev_low_rate) = prev {
// This is potentially a crossover position
if prev_low_rate <= curr_low_rate && curr_low_rate >= 0.5 {
let prev_dev = 0.5 - prev_low_rate;
let curr_dev = curr_low_rate - 0.5;
let selected_idx = if prev_dev < curr_dev { curr_dly - 1 } else { curr_dly };
// The setup setup/hold calibration timing (even with
// tolerance) might be invalid in other lanes due to skew.
// 5 taps is very conservative, generally it is 1 or 2
if selected_idx < 5 {
prev = None;
continue;
} else {
best_dly = Some(selected_idx);
break;
}
}
}
// Only rising slope from <= 0.5 can result in a rising low rate
// crossover at 50%.
if curr_low_rate <= 0.5 {
prev = Some(curr_low_rate);
}
}
if best_dly.is_none() {
error!("setup/hold timing calibration failed, retry in 1s...");
timer.delay_us(1_000_000);
} else {
break;
}
}
let best_dly = best_dly.unwrap();
apply_delay(best_dly, timer);
let mut delay_list = [best_dly; 4];
// Assign delay for other lanes
for lane_no in 1..=3 {
select_lane(lane_no as u8);
let mut min_deviation = 0.5;
let mut min_idx = 0;
for dly_delta in -3..=3 {
let index = (best_dly as isize + dly_delta) as u8;
let low_rate = read_align(index, timer);
// abs() from f32 is not available in core library
let deviation = if low_rate < 0.5 { 0.5 - low_rate } else { low_rate - 0.5 };
if deviation < min_deviation {
min_deviation = deviation;
min_idx = index;
}
}
apply_delay(min_idx, timer);
delay_list[lane_no] = min_idx;
}
debug!("setup/hold timing calibration: {:?}", delay_list);
SerdesConfig { delay: delay_list }
}
fn read_align(dly: u8, timer: &mut GlobalTimer) -> f32 {
unsafe {
apply_delay(dly, timer);
pl::csr::eem_transceiver::counter_reset_write(1);
pl::csr::eem_transceiver::counter_enable_write(1);
timer.delay_us(2000);
pl::csr::eem_transceiver::counter_enable_write(0);
let (high, low) = (
pl::csr::eem_transceiver::counter_high_count_read(),
pl::csr::eem_transceiver::counter_low_count_read(),
);
if pl::csr::eem_transceiver::counter_overflow_read() == 1 {
panic!("Unexpected phase detector counter overflow");
}
low as f32 / (low + high) as f32
}
}
unsafe fn align_comma(timer: &mut GlobalTimer) {
loop {
for slip in 1..=10 {
// The soft transceiver has 2 8b10b decoders, which receives lane
// 0/1 and lane 2/3 respectively. The decoder are time-multiplexed
// to decode exactly 1 lane each sysclk cycle.
//
// The decoder decodes lane 0/2 data on odd sysclk cycles, buffer
// on even cycles, and vice versa for lane 1/3. Data/Clock latency
// could change timing. The extend bit flips the decoding timing,
// so lane 0/2 data are decoded on even cycles, and lane 1/3 data
// are decoded on odd cycles.
//
// This is needed because transmitting/receiving a 8b10b character
// takes 2 sysclk cycles. Adjusting bitslip only via ISERDES
// limits the range to 1 cycle. The wordslip bit extends the range
// to 2 sysclk cycles.
pl::csr::eem_transceiver::wordslip_write((slip > 5) as u8);
// Apply a double bitslip since the ISERDES is 2x oversampled.
// Bitslip is used for comma alignment purposes once setup/hold
// timing is met.
pl::csr::eem_transceiver::bitslip_write(1);
pl::csr::eem_transceiver::bitslip_write(1);
timer.delay_us(1);
pl::csr::eem_transceiver::comma_align_reset_write(1);
timer.delay_us(100);
if pl::csr::eem_transceiver::comma_read() == 1 {
debug!("comma alignment completed after {} bitslips", slip);
return;
}
}
error!("comma alignment failed, retrying in 1s...");
timer.delay_us(1_000_000);
}
}
pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
for trx_no in 0..pl::csr::CONFIG_EEM_DRTIO_COUNT {
unsafe {
pl::csr::eem_transceiver::transceiver_sel_write(trx_no as u8);
}
let key = format!("eem_drtio_delay{}", trx_no);
let cfg_read = cfg.read(&key);
match cfg_read {
Ok(record) => {
info!("loading calibrated timing values from sd card");
unsafe {
apply_config(&*(record.as_ptr() as *const SerdesConfig), timer);
}
}
Err(_) => {
info!("calibrating...");
let config = unsafe { assign_delay(timer) };
match cfg.write(&key, config.as_bytes().to_vec()) {
Ok(()) => {
info!("storing calibration timing values into sd card");
}
Err(e) => {
error!(
"calibration successful but calibration timing values cannot be stored into sd card. \
Error:{}",
e
);
}
};
}
}
unsafe {
align_comma(timer);
pl::csr::eem_transceiver::rx_ready_write(1);
}
}
}

262
src/libboard_artiq/src/drtioaux_proto.rs
View File

@ -1,11 +1,8 @@
use core_io::{Error as IoError, Read, Write};
use io::proto::{ProtoRead, ProtoWrite};
// maximum size of arbitrary payloads
// used by satellite -> master analyzer, subkernel exceptions
pub const SAT_PAYLOAD_MAX_SIZE: usize = /*max size*/512 - /*CRC*/4 - /*packet ID*/1 - /*last*/1 - /*length*/2;
// used by DDMA, subkernel program data (need to provide extra ID and destination)
pub const MASTER_PAYLOAD_MAX_SIZE: usize = SAT_PAYLOAD_MAX_SIZE - /*destination*/1 - /*ID*/4;
/* 512 (max size) - 4 (CRC) - 1 (packet ID) - 1 (destination) - 4 (trace ID) - 1 (last) - 2 (length) */
pub const DMA_TRACE_MAX_SIZE: usize = 499;
#[derive(Debug)]
pub enum Error {
@ -139,29 +136,12 @@ pub enum Packet {
succeeded: bool,
},
AnalyzerHeaderRequest {
destination: u8,
},
AnalyzerHeader {
sent_bytes: u32,
total_byte_count: u64,
overflow_occurred: bool,
},
AnalyzerDataRequest {
destination: u8,
},
AnalyzerData {
last: bool,
length: u16,
data: [u8; SAT_PAYLOAD_MAX_SIZE],
},
DmaAddTraceRequest {
destination: u8,
id: u32,
last: bool,
length: u16,
trace: [u8; MASTER_PAYLOAD_MAX_SIZE],
trace: [u8; DMA_TRACE_MAX_SIZE],
},
DmaAddTraceReply {
succeeded: bool,
@ -188,53 +168,6 @@ pub enum Packet {
channel: u32,
timestamp: u64,
},
SubkernelAddDataRequest {
destination: u8,
id: u32,
last: bool,
length: u16,
data: [u8; MASTER_PAYLOAD_MAX_SIZE],
},
SubkernelAddDataReply {
succeeded: bool,
},
SubkernelLoadRunRequest {
destination: u8,
id: u32,
run: bool,
},
SubkernelLoadRunReply {
succeeded: bool,
},
SubkernelStopRequest {
destination: u8,
},
SubkernelStopReply {
succeeded: bool,
},
SubkernelFinished {
id: u32,
with_exception: bool,
},
SubkernelExceptionRequest {
destination: u8,
},
SubkernelException {
last: bool,
length: u16,
data: [u8; SAT_PAYLOAD_MAX_SIZE],
},
SubkernelMessage {
destination: u8,
id: u32,
last: bool,
length: u16,
data: [u8; MASTER_PAYLOAD_MAX_SIZE],
},
SubkernelMessageAck {
destination: u8,
},
}
impl Packet {
@ -365,35 +298,12 @@ impl Packet {
succeeded: reader.read_bool()?,
},
0xa0 => Packet::AnalyzerHeaderRequest {
destination: reader.read_u8()?,
},
0xa1 => Packet::AnalyzerHeader {
sent_bytes: reader.read_u32()?,
total_byte_count: reader.read_u64()?,
overflow_occurred: reader.read_bool()?,
},
0xa2 => Packet::AnalyzerDataRequest {
destination: reader.read_u8()?,
},
0xa3 => {
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut data: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
reader.read_exact(&mut data[0..length as usize])?;
Packet::AnalyzerData {
last: last,
length: length,
data: data,
}
}
0xb0 => {
let destination = reader.read_u8()?;
let id = reader.read_u32()?;
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut trace: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
let mut trace: [u8; DMA_TRACE_MAX_SIZE] = [0; DMA_TRACE_MAX_SIZE];
reader.read_exact(&mut trace[0..length as usize])?;
Packet::DmaAddTraceRequest {
destination: destination,
@ -429,75 +339,6 @@ impl Packet {
timestamp: reader.read_u64()?,
},
0xc0 => {
let destination = reader.read_u8()?;
let id = reader.read_u32()?;
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
reader.read_exact(&mut data[0..length as usize])?;
Packet::SubkernelAddDataRequest {
destination: destination,
id: id,
last: last,
length: length as u16,
data: data,
}
}
0xc1 => Packet::SubkernelAddDataReply {
succeeded: reader.read_bool()?,
},
0xc4 => Packet::SubkernelLoadRunRequest {
destination: reader.read_u8()?,
id: reader.read_u32()?,
run: reader.read_bool()?,
},
0xc5 => Packet::SubkernelLoadRunReply {
succeeded: reader.read_bool()?,
},
0xc6 => Packet::SubkernelStopRequest {
destination: reader.read_u8()?,
},
0xc7 => Packet::SubkernelStopReply {
succeeded: reader.read_bool()?,
},
0xc8 => Packet::SubkernelFinished {
id: reader.read_u32()?,
with_exception: reader.read_bool()?,
},
0xc9 => Packet::SubkernelExceptionRequest {
destination: reader.read_u8()?,
},
0xca => {
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut data: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
reader.read_exact(&mut data[0..length as usize])?;
Packet::SubkernelException {
last: last,
length: length,
data: data,
}
}
0xcb => {
let destination = reader.read_u8()?;
let id = reader.read_u32()?;
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut data: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
reader.read_exact(&mut data[0..length as usize])?;
Packet::SubkernelMessage {
destination: destination,
id: id,
last: last,
length: length as u16,
data: data,
}
}
0xcc => Packet::SubkernelMessageAck {
destination: reader.read_u8()?,
},
ty => return Err(Error::UnknownPacket(ty)),
})
}
@ -685,31 +526,6 @@ impl Packet {
writer.write_bool(succeeded)?;
}
Packet::AnalyzerHeaderRequest { destination } => {
writer.write_u8(0xa0)?;
writer.write_u8(destination)?;
}
Packet::AnalyzerHeader {
sent_bytes,
total_byte_count,
overflow_occurred,
} => {
writer.write_u8(0xa1)?;
writer.write_u32(sent_bytes)?;
writer.write_u64(total_byte_count)?;
writer.write_bool(overflow_occurred)?;
}
Packet::AnalyzerDataRequest { destination } => {
writer.write_u8(0xa2)?;
writer.write_u8(destination)?;
}
Packet::AnalyzerData { last, length, data } => {
writer.write_u8(0xa3)?;
writer.write_bool(last)?;
writer.write_u16(length)?;
writer.write_all(&data[0..length as usize])?;
}
Packet::DmaAddTraceRequest {
destination,
id,
@ -767,76 +583,6 @@ impl Packet {
writer.write_u32(channel)?;
writer.write_u64(timestamp)?;
}
Packet::SubkernelAddDataRequest {
destination,
id,
last,
data,
length,
} => {
writer.write_u8(0xc0)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_bool(last)?;
writer.write_u16(length)?;
writer.write_all(&data[0..length as usize])?;
}
Packet::SubkernelAddDataReply { succeeded } => {
writer.write_u8(0xc1)?;
writer.write_bool(succeeded)?;
}
Packet::SubkernelLoadRunRequest { destination, id, run } => {
writer.write_u8(0xc4)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_bool(run)?;
}
Packet::SubkernelLoadRunReply { succeeded } => {
writer.write_u8(0xc5)?;
writer.write_bool(succeeded)?;
}
Packet::SubkernelStopRequest { destination } => {
writer.write_u8(0xc6)?;
writer.write_u8(destination)?;
}
Packet::SubkernelStopReply { succeeded } => {
writer.write_u8(0xc7)?;
writer.write_bool(succeeded)?;
}
Packet::SubkernelFinished { id, with_exception } => {
writer.write_u8(0xc8)?;
writer.write_u32(id)?;
writer.write_bool(with_exception)?;
}
Packet::SubkernelExceptionRequest { destination } => {
writer.write_u8(0xc9)?;
writer.write_u8(destination)?;
}
Packet::SubkernelException { last, length, data } => {
writer.write_u8(0xca)?;
writer.write_bool(last)?;
writer.write_u16(length)?;
writer.write_all(&data[0..length as usize])?;
}
Packet::SubkernelMessage {
destination,
id,
last,
data,
length,
} => {
writer.write_u8(0xcb)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_bool(last)?;
writer.write_u16(length)?;
writer.write_all(&data[0..length as usize])?;
}
Packet::SubkernelMessageAck { destination } => {
writer.write_u8(0xcc)?;
writer.write_u8(destination)?;
}
}
Ok(())
}

111
src/libboard_artiq/src/io_expander.rs
View File

@ -1,8 +1,6 @@
use libboard_zynq::i2c;
use log::info;
use crate::pl::csr;
// Only the bare minimum registers. Bits/IO connections equivalent between IC types.
struct Registers {
// PCA9539 equivalent register names in comments
@ -12,49 +10,23 @@ struct Registers {
gpiob: u8, // Output Port 1
}
//IO expanders pins
const IODIR_OUT_SFP_TX_DISABLE: u8 = 0x02;
const IODIR_OUT_SFP_LED: u8 = 0x40;
#[cfg(hw_rev = "v1.0")]
const IODIR_OUT_SFP0_LED: u8 = 0x40;
#[cfg(hw_rev = "v1.1")]
const IODIR_OUT_SFP0_LED: u8 = 0x80;
//IO expander port direction
const IODIR0: [u8; 2] = [
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP0_LED,
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED,
];
const IODIR1: [u8; 2] = [
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED,
0xFF & !IODIR_OUT_SFP_TX_DISABLE & !IODIR_OUT_SFP_LED,
];
pub struct IoExpander {
pub struct IoExpander<'a> {
i2c: &'a mut i2c::I2c,
address: u8,
virtual_led_mapping: &'static [(u8, u8, u8)],
iodir: [u8; 2],
out_current: [u8; 2],
out_target: [u8; 2],
registers: Registers,
}
impl IoExpander {
pub fn new(i2c: &mut i2c::I2c, index: u8) -> Result<Self, &'static str> {
#[cfg(hw_rev = "v1.0")]
const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 6), (1, 1, 6)];
#[cfg(hw_rev = "v1.1")]
const VIRTUAL_LED_MAPPING0: [(u8, u8, u8); 2] = [(0, 0, 7), (1, 1, 6)];
const VIRTUAL_LED_MAPPING1: [(u8, u8, u8); 2] = [(2, 0, 6), (3, 1, 6)];
impl<'a> IoExpander<'a> {
pub fn new(i2c: &'a mut i2c::I2c, index: u8) -> Result<Self, &'static str> {
// Both expanders on SHARED I2C bus
let mut io_expander = match index {
0 => IoExpander {
i2c,
address: 0x40,
virtual_led_mapping: &VIRTUAL_LED_MAPPING0,
iodir: IODIR0,
iodir: [0xff; 2],
out_current: [0; 2],
out_target: [0; 2],
registers: Registers {
@ -65,9 +37,9 @@ impl IoExpander {
},
},
1 => IoExpander {
i2c,
address: 0x42,
virtual_led_mapping: &VIRTUAL_LED_MAPPING1,
iodir: IODIR1,
iodir: [0xff; 2],
out_current: [0; 2],
out_target: [0; 2],
registers: Registers {
@ -79,7 +51,7 @@ impl IoExpander {
},
_ => return Err("incorrect I/O expander index"),
};
if !io_expander.check_ack(i2c)? {
if !io_expander.check_ack()? {
info!("MCP23017 io expander {} not found. Checking for PCA9539.", index);
io_expander.address += 0xa8; // translate to PCA9539 addresses (see schematic)
io_expander.registers = Registers {
@ -88,58 +60,57 @@ impl IoExpander {
gpioa: 0x02,
gpiob: 0x03,
};
if !io_expander.check_ack(i2c)? {
if !io_expander.check_ack()? {
return Err("Neither MCP23017 nor PCA9539 io expander found.");
};
}
Ok(io_expander)
}
fn select(&self, i2c: &mut i2c::I2c) -> Result<(), &'static str> {
i2c.pca954x_select(0x70, None)?;
i2c.pca954x_select(0x71, Some(3))?;
fn select(&mut self) -> Result<(), &'static str> {
self.i2c.pca954x_select(0x70, None)?;
self.i2c.pca954x_select(0x71, Some(3))?;
Ok(())
}
fn write(&self, i2c: &mut i2c::I2c, addr: u8, value: u8) -> Result<(), &'static str> {
i2c.start()?;
i2c.write(self.address)?;
i2c.write(addr)?;
i2c.write(value)?;
i2c.stop()?;
fn write(&mut self, addr: u8, value: u8) -> Result<(), &'static str> {
self.i2c.start()?;
self.i2c.write(self.address)?;
self.i2c.write(addr)?;
self.i2c.write(value)?;
self.i2c.stop()?;
Ok(())
}
fn check_ack(&self, i2c: &mut i2c::I2c) -> Result<bool, &'static str> {
fn check_ack(&mut self) -> Result<bool, &'static str> {
// Check for ack from io expander
self.select(i2c)?;
i2c.start()?;
let ack = i2c.write(self.address)?;
i2c.stop()?;
self.select()?;
self.i2c.start()?;
let ack = self.i2c.write(self.address)?;
self.i2c.stop()?;
Ok(ack)
}
fn update_iodir(&self, i2c: &mut i2c::I2c) -> Result<(), &'static str> {
self.write(i2c, self.registers.iodira, self.iodir[0])?;
self.write(i2c, self.registers.iodirb, self.iodir[1])?;
fn update_iodir(&mut self) -> Result<(), &'static str> {
self.write(self.registers.iodira, self.iodir[0])?;
self.write(self.registers.iodirb, self.iodir[1])?;
Ok(())
}
pub fn init(&mut self, i2c: &mut i2c::I2c) -> Result<(), &'static str> {
self.select(i2c)?;
self.update_iodir(i2c)?;
pub fn init(&mut self) -> Result<(), &'static str> {
self.select()?;
self.update_iodir()?;
self.out_current[0] = 0x00;
self.write(i2c, self.registers.gpioa, 0x00)?;
self.write(self.registers.gpioa, 0x00)?;
self.out_current[1] = 0x00;
self.write(i2c, self.registers.gpiob, 0x00)?;
self.write(self.registers.gpiob, 0x00)?;
Ok(())
}
pub fn set_oe(&mut self, i2c: &mut i2c::I2c, port: u8, outputs: u8) -> Result<(), &'static str> {
pub fn set_oe(&mut self, port: u8, outputs: u8) -> Result<(), &'static str> {
self.iodir[port as usize] &= !outputs;
self.update_iodir(i2c)?;
self.update_iodir()?;
Ok(())
}
@ -151,21 +122,15 @@ impl IoExpander {
}
}
pub fn service(&mut self, i2c: &mut i2c::I2c) -> Result<(), &'static str> {
#[cfg(has_virtual_leds)]
for (led, port, bit) in self.virtual_led_mapping.iter() {
let level = unsafe { csr::virtual_leds::status_read() >> led & 1 };
self.set(*port, *bit, level != 0);
}
pub fn service(&mut self) -> Result<(), &'static str> {
if self.out_target != self.out_current {
self.select(i2c)?;
self.select()?;
if self.out_target[0] != self.out_current[0] {
self.write(i2c, self.registers.gpioa, self.out_target[0])?;
self.write(self.registers.gpioa, self.out_target[0])?;
self.out_current[0] = self.out_target[0];
}
if self.out_target[1] != self.out_current[1] {
self.write(i2c, self.registers.gpiob, self.out_target[1])?;
self.write(self.registers.gpiob, self.out_target[1])?;
self.out_current[1] = self.out_target[1];
}
}

2
src/libboard_artiq/src/lib.rs
View File

@ -29,8 +29,6 @@ pub mod mem;
#[rustfmt::skip]
#[path = "../../../build/pl.rs"]
pub mod pl;
#[cfg(has_drtio_eem)]
pub mod drtio_eem;
#[cfg(has_si5324)]
pub mod si5324;

2
src/libc/Cargo.toml.tpl → src/libc/Cargo.toml
View File

@ -6,7 +6,7 @@ edition = "2018"
build = "build.rs"
[dependencies]
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq" }
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
[build-dependencies]
cc = { version = "1.0.1" }

2
src/libdyld/Cargo.toml.tpl → src/libdyld/Cargo.toml
View File

@ -8,4 +8,4 @@ name = "dyld"
[dependencies]
log = "0.4"
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }

2
src/libio/Cargo.toml.tpl → src/libio/Cargo.toml
View File

@ -11,7 +11,7 @@ path = "lib.rs"
core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.0", default-features = false, optional = true }
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
[features]
alloc = []

5
src/libio/cursor.rs
View File

@ -1,6 +1,3 @@
#[cfg(feature = "alloc")]
use alloc::vec::Vec;
use core_io::{Error as IoError, Read, Write};
#[derive(Debug, Clone)]
@ -67,7 +64,7 @@ impl Write for Cursor<&mut [u8]> {
}
#[cfg(feature = "alloc")]
impl Write for Cursor<Vec<u8>> {
impl Write for Cursor<::alloc::Vec<u8>> {
#[inline]
fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> {
self.inner.extend_from_slice(buf);

5
src/libio/lib.rs
View File

@ -1,10 +1,13 @@
#![no_std]
#![feature(never_type)]
#![cfg_attr(feature = "alloc", feature(alloc))]
#[cfg(feature = "alloc")]
extern crate alloc;
extern crate core_io;
#[cfg(feature = "alloc")]
#[macro_use]
use alloc;
#[cfg(feature = "byteorder")]
extern crate byteorder;

8
src/libio/proto.rs
View File

@ -1,4 +1,3 @@
#[cfg(feature = "alloc")]
use alloc::{string::String, vec};
use core::str::Utf8Error;
@ -51,8 +50,7 @@ pub trait ProtoRead {
}
#[inline]
#[cfg(feature = "alloc")]
fn read_bytes(&mut self) -> Result<vec::Vec<u8>, Self::ReadError> {
fn read_bytes(&mut self) -> Result<::alloc::vec::Vec<u8>, Self::ReadError> {
let length = self.read_u32()?;
let mut value = vec![0; length as usize];
self.read_exact(&mut value)?;
@ -60,8 +58,7 @@ pub trait ProtoRead {
}
#[inline]
#[cfg(feature = "alloc")]
fn read_string(&mut self) -> Result<String, ReadStringError<Self::ReadError>> {
fn read_string(&mut self) -> Result<::alloc::string::String, ReadStringError<Self::ReadError>> {
let bytes = self.read_bytes().map_err(ReadStringError::Other)?;
String::from_utf8(bytes).map_err(|err| ReadStringError::Utf8(err.utf8_error()))
}
@ -138,7 +135,6 @@ pub trait ProtoWrite {
}
#[inline]
#[cfg(feature = "alloc")]
fn write_string(&mut self, value: &str) -> Result<(), Self::WriteError> {
self.write_bytes(value.as_bytes())
}

34
src/libksupport/Cargo.toml.tpl
View File

@ -1,34 +0,0 @@
[package]
name = "ksupport"
description = "Kernel support for Zynq-based platforms"
version = "0.1.0"
authors = ["M-Labs"]
edition = "2018"
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
cslice = "0.3"
log = "0.4"
nb = "0.1"
core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.3", default-features = false }
void = { version = "1", default-features = false }
log_buffer = { version = "1.2" }
libm = { version = "0.2", features = ["unstable"] }
vcell = "0.1"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] }
dyld = { path = "../libdyld" }
dwarf = { path = "../libdwarf" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }
io = { path = "../libio" }
libboard_artiq = { path = "../libboard_artiq" }

5
src/libksupport/build.rs
View File

@ -1,5 +0,0 @@
extern crate build_zynq;
fn main() {
build_zynq::cfg();
}

109
src/libksupport/src/kernel/subkernel.rs
View File

@ -1,109 +0,0 @@
use alloc::vec::Vec;
use cslice::CSlice;
use super::{Message, SubkernelStatus, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0};
use crate::{artiq_raise, rpc::send_args};
pub extern "C" fn load_run(id: u32, run: bool) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelLoadRunRequest { id: id, run: run });
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelLoadRunReply { succeeded: true } => (),
Message::SubkernelLoadRunReply { succeeded: false } => {
artiq_raise!("SubkernelError", "Error loading or running the subkernel")
}
_ => panic!("Expected SubkernelLoadRunReply after SubkernelLoadRunRequest!"),
}
}
pub extern "C" fn await_finish(id: u32, timeout: u64) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelAwaitFinishRequest {
id: id,
timeout: timeout,
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelAwaitFinishReply {
status: SubkernelStatus::NoError,
} => (),
Message::SubkernelAwaitFinishReply {
status: SubkernelStatus::IncorrectState,
} => artiq_raise!("SubkernelError", "Subkernel not running"),
Message::SubkernelAwaitFinishReply {
status: SubkernelStatus::Timeout,
} => artiq_raise!("SubkernelError", "Subkernel timed out"),
Message::SubkernelAwaitFinishReply {
status: SubkernelStatus::CommLost,
} => artiq_raise!("SubkernelError", "Lost communication with satellite"),
Message::SubkernelAwaitFinishReply {
status: SubkernelStatus::OtherError,
} => artiq_raise!("SubkernelError", "An error occurred during subkernel operation"),
_ => panic!("expected SubkernelAwaitFinishReply after SubkernelAwaitFinishRequest"),
}
}
pub extern "C" fn send_message(id: u32, count: u8, tag: &CSlice<u8>, data: *const *const ()) {
let mut buffer = Vec::<u8>::new();
send_args(&mut buffer, 0, tag.as_ref(), data).expect("RPC encoding failed");
// overwrite service tag, include how many tags are in the message
buffer[3] = count;
unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::SubkernelMsgSend {
id: id,
data: buffer[3..].to_vec(),
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelMsgSent => (),
_ => panic!("expected SubkernelMsgSent after SubkernelMsgSend"),
}
}
pub extern "C" fn await_message(id: u32, timeout: u64, min: u8, max: u8) {
unsafe {
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::SubkernelMsgRecvRequest {
id: id,
timeout: timeout,
});
}
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() {
Message::SubkernelMsgRecvReply {
status: SubkernelStatus::NoError,
count,
} => {
if min > count || count > max {
artiq_raise!("SubkernelError", "Received more or less arguments than required")
}
}
Message::SubkernelMsgRecvReply {
status: SubkernelStatus::IncorrectState,
..
} => artiq_raise!("SubkernelError", "Subkernel not running"),
Message::SubkernelMsgRecvReply {
status: SubkernelStatus::Timeout,
..
} => artiq_raise!("SubkernelError", "Subkernel timed out"),
Message::SubkernelMsgRecvReply {
status: SubkernelStatus::CommLost,
..
} => artiq_raise!("SubkernelError", "Lost communication with satellite"),
Message::SubkernelMsgRecvReply {
status: SubkernelStatus::OtherError,
..
} => artiq_raise!("SubkernelError", "An error occurred during subkernel operation"),
_ => panic!("expected SubkernelMsgRecvReply after SubkernelMsgRecvRequest"),
}
// RpcRecvRequest should be called after this to receive message data
}

163
src/libksupport/src/lib.rs
View File

@ -1,163 +0,0 @@
#![no_std]
#![feature(c_variadic)]
#![feature(const_btree_new)]
#![feature(const_in_array_repeat_expressions)]
#![feature(naked_functions)]
#![feature(asm)]
#[macro_use]
extern crate alloc;
use alloc::{collections::BTreeMap, string::String};
use io::{Cursor, ProtoRead};
use libasync::block_async;
use libconfig::Config;
use log::{error, warn};
#[cfg(has_drtiosat)]
pub use pl::csr::drtiosat as rtio_core;
#[cfg(has_rtio_core)]
pub use pl::csr::rtio_core;
use void::Void;
pub mod eh_artiq;
pub mod i2c;
pub mod irq;
pub mod kernel;
pub mod rpc;
#[cfg(ki_impl = "csr")]
#[path = "rtio_csr.rs"]
pub mod rtio;
#[cfg(ki_impl = "acp")]
#[path = "rtio_acp.rs"]
pub mod rtio;
#[rustfmt::skip]
#[path = "../../../build/pl.rs"]
pub mod pl;
#[derive(Debug, Clone)]
pub struct RPCException {
pub id: u32,
pub message: u32,
pub param: [i64; 3],
pub file: u32,
pub line: i32,
pub column: i32,
pub function: u32,
}
pub static mut SEEN_ASYNC_ERRORS: u8 = 0;
pub const ASYNC_ERROR_COLLISION: u8 = 1 << 0;
pub const ASYNC_ERROR_BUSY: u8 = 1 << 1;
pub const ASYNC_ERROR_SEQUENCE_ERROR: u8 = 1 << 2;
pub unsafe fn get_async_errors() -> u8 {
let errors = SEEN_ASYNC_ERRORS;
SEEN_ASYNC_ERRORS = 0;
errors
}
fn wait_for_async_rtio_error() -> nb::Result<(), Void> {
unsafe {
#[cfg(has_rtio_core)]
let errors = rtio_core::async_error_read();
#[cfg(has_drtiosat)]
let errors = rtio_core::protocol_error_read();
if errors != 0 {
Ok(())
} else {
Err(nb::Error::WouldBlock)
}
}
}
pub async fn report_async_rtio_errors() {
loop {
let _ = block_async!(wait_for_async_rtio_error()).await;
unsafe {
#[cfg(has_rtio_core)]
let errors = rtio_core::async_error_read();
#[cfg(has_drtiosat)]
let errors = rtio_core::protocol_error_read();
if errors & ASYNC_ERROR_COLLISION != 0 {
let channel = rtio_core::collision_channel_read();
error!(
"RTIO collision involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_BUSY != 0 {
let channel = rtio_core::busy_channel_read();
error!(
"RTIO busy error involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_SEQUENCE_ERROR != 0 {
let channel = rtio_core::sequence_error_channel_read();
error!(
"RTIO sequence error involving channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
);
}
SEEN_ASYNC_ERRORS = errors;
#[cfg(has_rtio_core)]
rtio_core::async_error_write(errors);
#[cfg(has_drtiosat)]
rtio_core::protocol_error_write(errors);
}
}
}
static mut RTIO_DEVICE_MAP: BTreeMap<u32, String> = BTreeMap::new();
fn read_device_map(cfg: &Config) -> BTreeMap<u32, String> {
let mut device_map: BTreeMap<u32, String> = BTreeMap::new();
let _ = cfg
.read("device_map")
.and_then(|raw_bytes| {
let mut bytes_cr = Cursor::new(raw_bytes);
let size = bytes_cr.read_u32().unwrap();
for _ in 0..size {
let channel = bytes_cr.read_u32().unwrap();
let device_name = bytes_cr.read_string().unwrap();
if let Some(old_entry) = device_map.insert(channel, device_name.clone()) {
warn!(
"conflicting device map entries for RTIO channel {}: '{}' and '{}'",
channel, old_entry, device_name
);
}
}
Ok(())
})
.or_else(|err| {
warn!(
"error reading device map ({}), device names will not be available in RTIO error messages",
err
);
Err(err)
});
device_map
}
fn _resolve_channel_name(channel: u32, device_map: &BTreeMap<u32, String>) -> String {
match device_map.get(&channel) {
Some(val) => val.clone(),
None => String::from("unknown"),
}
}
pub fn resolve_channel_name(channel: u32) -> String {
_resolve_channel_name(channel, unsafe { &RTIO_DEVICE_MAP })
}
pub fn setup_device_map(cfg: &Config) {
unsafe {
RTIO_DEVICE_MAP = read_device_map(cfg);
}
}

17
src/runtime/Cargo.toml.tpl → src/runtime/Cargo.toml
View File

@ -18,6 +18,7 @@ num-traits = { version = "0.2", default-features = false }
num-derive = "0.3"
cslice = "0.3"
log = "0.4"
nb = "0.1"
embedded-hal = "0.2"
core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.3", default-features = false }
@ -25,19 +26,19 @@ void = { version = "1", default-features = false }
futures = { version = "0.3", default-features = false, features = ["async-await"] }
async-recursion = "0.3"
log_buffer = { version = "1.2" }
libm = { version = "0.2", features = ["unstable"] }
vcell = "0.1"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] }
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"]}
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] }
dyld = { path = "../libdyld" }
dwarf = { path = "../libdwarf" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }
io = { path = "../libio", features = ["alloc"] }
ksupport = { path = "../libksupport" }
io = { path = "../libio" }
libboard_artiq = { path = "../libboard_artiq" }

118
src/runtime/src/analyzer.rs
View File

@ -1,12 +1,6 @@
use alloc::rc::Rc;
#[cfg(has_drtio)]
use alloc::vec::Vec;
use core::cell::RefCell;
use libasync::{smoltcp::TcpStream, task};
use libboard_artiq::drtio_routing;
use libboard_zynq::{smoltcp::Error, timer::GlobalTimer};
use libcortex_a9::{cache, mutex::Mutex};
use libboard_zynq::smoltcp::Error;
use libcortex_a9::cache;
use log::{debug, info, warn};
use crate::{pl, proto_async::*};
@ -43,50 +37,6 @@ fn disarm() {
debug!("RTIO analyzer disarmed");
}
#[cfg(has_drtio)]
pub mod remote_analyzer {
use super::*;
use crate::rtio_mgt::drtio;
pub struct RemoteBuffer {
pub total_byte_count: u64,
pub sent_bytes: u32,
pub error: bool,
pub data: Vec<u8>,
}
pub async fn get_data(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) -> Result<RemoteBuffer, &'static str> {
// gets data from satellites and returns consolidated data
let mut remote_data: Vec<u8> = Vec::new();
let mut remote_error = false;
let mut remote_sent_bytes = 0;
let mut remote_total_bytes = 0;
let data_vec = match drtio::analyzer_query(aux_mutex, routing_table, up_destinations, timer).await {
Ok(data_vec) => data_vec,
Err(e) => return Err(e),
};
for data in data_vec {
remote_total_bytes += data.total_byte_count;
remote_sent_bytes += data.sent_bytes;
remote_error |= data.error;
remote_data.extend(data.data);
}
Ok(RemoteBuffer {
total_byte_count: remote_total_bytes,
sent_bytes: remote_sent_bytes,
error: remote_error,
data: remote_data,
})
}
}
#[derive(Debug)]
struct Header {
sent_bytes: u32,
@ -106,13 +56,7 @@ async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Err
Ok(())
}
async fn handle_connection(
stream: &mut TcpStream,
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &drtio_routing::RoutingTable,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
_timer: GlobalTimer,
) -> Result<(), Error> {
async fn handle_connection(stream: &mut TcpStream) -> Result<(), Error> {
info!("received connection");
let data = unsafe { &BUFFER.data[..] };
@ -121,11 +65,6 @@ async fn handle_connection(
let total_byte_count = unsafe { pl::csr::rtio_analyzer::dma_byte_count_read() as u64 };
let pointer = (total_byte_count % BUFFER_SIZE as u64) as usize;
let wraparound = total_byte_count >= BUFFER_SIZE as u64;
let sent_bytes = if wraparound {
BUFFER_SIZE as u32
} else {
total_byte_count as u32
};
if overflow_occurred {
warn!("overflow occured");
@ -134,39 +73,13 @@ async fn handle_connection(
warn!("bus error occured");
}
#[cfg(has_drtio)]
let remote = remote_analyzer::get_data(_aux_mutex, _routing_table, _up_destinations, _timer).await;
#[cfg(has_drtio)]
let (header, remote_data) = match remote {
Ok(remote) => (
Header {
total_byte_count: total_byte_count + remote.total_byte_count,
sent_bytes: sent_bytes + remote.sent_bytes,
error_occurred: overflow_occurred | bus_error_occurred | remote.error,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true,
},
remote.data,
),
Err(e) => {
warn!("Error getting remote analyzer data: {}", e);
(
Header {
total_byte_count: total_byte_count,
sent_bytes: sent_bytes,
error_occurred: true,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true,
},
Vec::new(),
)
}
};
#[cfg(not(has_drtio))]
let header = Header {
total_byte_count: total_byte_count,
sent_bytes: sent_bytes,
sent_bytes: if wraparound {
BUFFER_SIZE as u32
} else {
total_byte_count as u32
},
error_occurred: overflow_occurred | bus_error_occurred,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true, // kept for backward compatibility of analyzer dumps
@ -180,28 +93,17 @@ async fn handle_connection(
} else {
stream.send(data[..pointer].iter().copied()).await?;
}
#[cfg(has_drtio)]
stream.send(remote_data.iter().copied()).await?;
Ok(())
}
pub fn start(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
let up_destinations = up_destinations.clone();
pub fn start() {
task::spawn(async move {
loop {
arm();
let mut stream = TcpStream::accept(1382, 2048, 2048).await.unwrap();
disarm();
let routing_table = routing_table.borrow();
let _ = handle_connection(&mut stream, &aux_mutex, &routing_table, &up_destinations, timer)
let _ = handle_connection(&mut stream)
.await
.map_err(|e| warn!("connection terminated: {:?}", e));
let _ = stream.flush().await;

187
src/runtime/src/comms.rs
View File

@ -1,14 +1,8 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String, vec, vec::Vec};
use core::{cell::RefCell, fmt, slice, str};
use core_io::Error as IoError;
use cslice::CSlice;
use futures::{future::FutureExt, select_biased};
#[cfg(has_drtio)]
use io::{Cursor, ProtoRead};
#[cfg(has_drtio)]
use ksupport::rpc;
use ksupport::{kernel, resolve_channel_name};
use libasync::{smoltcp::{Sockets, TcpStream},
task};
use libboard_artiq::drtio_routing;
@ -30,19 +24,17 @@ use num_traits::{FromPrimitive, ToPrimitive};
#[cfg(has_drtio)]
use crate::pl;
use crate::{analyzer, mgmt, moninj, proto_async::*, rpc_async, rtio_dma, rtio_mgt};
#[cfg(has_drtio)]
use crate::{subkernel, subkernel::Error as SubkernelError};
use crate::{analyzer, kernel, mgmt, moninj,
proto_async::*,
rpc, rtio_dma,
rtio_mgt::{self, resolve_channel_name}};
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error {
NetworkError(smoltcp::Error),
IoError,
UnexpectedPattern,
UnrecognizedPacket,
BufferExhausted,
#[cfg(has_drtio)]
SubkernelError(subkernel::Error),
}
pub type Result<T> = core::result::Result<T, Error>;
@ -51,12 +43,9 @@ impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self {
Error::NetworkError(error) => write!(f, "network error: {}", error),
Error::IoError => write!(f, "io error"),
Error::UnexpectedPattern => write!(f, "unexpected pattern"),
Error::UnrecognizedPacket => write!(f, "unrecognized packet"),
Error::BufferExhausted => write!(f, "buffer exhausted"),
#[cfg(has_drtio)]
Error::SubkernelError(error) => write!(f, "subkernel error: {:?}", error),
}
}
}
@ -67,19 +56,6 @@ impl From<smoltcp::Error> for Error {
}
}
impl From<IoError> for Error {
fn from(_error: IoError) -> Self {
Error::IoError
}
}
#[cfg(has_drtio)]
impl From<subkernel::Error> for Error {
fn from(error: subkernel::Error) -> Self {
Error::SubkernelError(error)
}
}
#[derive(Debug, FromPrimitive, ToPrimitive)]
enum Request {
SystemInfo = 3,
@ -87,7 +63,6 @@ enum Request {
RunKernel = 6,
RPCReply = 7,
RPCException = 8,
UploadSubkernel = 9,
}
#[derive(Debug, FromPrimitive, ToPrimitive)]
@ -207,7 +182,7 @@ async fn handle_run_kernel(
kernel::Message::RpcRecvRequest(slot) => slot,
other => panic!("expected root value slot from core1, not {:?}", other),
};
rpc_async::recv_return(stream, &tag, slot, &|size| {
rpc::recv_return(stream, &tag, slot, &|size| {
let control = control.clone();
async move {
if size == 0 {
@ -252,7 +227,7 @@ async fn handle_run_kernel(
let function = read_i32(stream).await? as u32;
control
.tx
.async_send(kernel::Message::RpcRecvReply(Err(ksupport::RPCException {
.async_send(kernel::Message::RpcRecvReply(Err(kernel::RPCException {
id,
message,
param,
@ -390,124 +365,6 @@ async fn handle_run_kernel(
control.borrow_mut().tx.async_send(reply).await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelLoadRunRequest { id, run } => {
let succeeded = match subkernel::load(aux_mutex, routing_table, timer, id, run).await {
Ok(()) => true,
Err(e) => {
error!("Error loading subkernel: {:?}", e);
false
}
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelLoadRunReply { succeeded: succeeded })
.await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelAwaitFinishRequest { id, timeout } => {
let res = subkernel::await_finish(aux_mutex, routing_table, timer, id, timeout).await;
let status = match res {
Ok(ref res) => {
if res.status == subkernel::FinishStatus::CommLost {
kernel::SubkernelStatus::CommLost
} else if let Some(exception) = &res.exception {
error!("Exception in subkernel");
match stream {
None => (),
Some(stream) => {
write_chunk(stream, exception).await?;
}
}
// will not be called after exception is served
kernel::SubkernelStatus::OtherError
} else {
kernel::SubkernelStatus::NoError
}
}
Err(SubkernelError::Timeout) => kernel::SubkernelStatus::Timeout,
Err(SubkernelError::IncorrectState) => kernel::SubkernelStatus::IncorrectState,
Err(_) => kernel::SubkernelStatus::OtherError,
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelAwaitFinishReply { status: status })
.await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelMsgSend { id, data } => {
let res = subkernel::message_send(aux_mutex, routing_table, timer, id, data).await;
match res {
Ok(_) => (),
Err(e) => {
error!("error sending subkernel message: {:?}", e)
}
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelMsgSent)
.await;
}
#[cfg(has_drtio)]
kernel::Message::SubkernelMsgRecvRequest { id, timeout } => {
let message_received = subkernel::message_await(id, timeout, timer).await;
let (status, count) = match message_received {
Ok(ref message) => (kernel::SubkernelStatus::NoError, message.count),
Err(SubkernelError::Timeout) => (kernel::SubkernelStatus::Timeout, 0),
Err(SubkernelError::IncorrectState) => (kernel::SubkernelStatus::IncorrectState, 0),
Err(SubkernelError::CommLost) => (kernel::SubkernelStatus::CommLost, 0),
Err(_) => (kernel::SubkernelStatus::OtherError, 0),
};
control
.borrow_mut()
.tx
.async_send(kernel::Message::SubkernelMsgRecvReply {
status: status,
count: count,
})
.await;
if let Ok(message) = message_received {
// receive code almost identical to RPC recv, except we are not reading from a stream
let mut reader = Cursor::new(message.data);
let mut tag: [u8; 1] = [message.tag];
let mut i = 0;
loop {
// kernel has to consume all arguments in the whole message
let slot = match fast_recv(&mut control.borrow_mut().rx).await {
kernel::Message::RpcRecvRequest(slot) => slot,
other => panic!("expected root value slot from core1, not {:?}", other),
};
rpc::recv_return(&mut reader, &tag, slot, &mut |size| {
if size == 0 {
0 as *mut ()
} else {
let mut control = control.borrow_mut();
control.tx.send(kernel::Message::RpcRecvReply(Ok(size)));
match control.rx.recv() {
kernel::Message::RpcRecvRequest(slot) => slot,
other => {
panic!("expected nested value slot from kernel CPU, not {:?}", other)
}
}
}
})?;
control
.borrow_mut()
.tx
.async_send(kernel::Message::RpcRecvReply(Ok(0)))
.await;
i += 1;
if i < count {
tag[0] = reader.read_u8()?;
} else {
break;
}
}
}
}
#[cfg(has_drtio)]
kernel::Message::UpDestinationsRequest(destination) => {
let result = _up_destinations.borrow()[destination as usize];
control
@ -577,13 +434,9 @@ async fn handle_connection(
return Err(Error::UnexpectedPattern);
}
stream.send_slice("e".as_bytes()).await?;
#[cfg(has_drtio)]
subkernel::clear_subkernels().await;
loop {
let request = read_request(stream, true).await?;
if request.is_none() {
#[cfg(has_drtio)]
subkernel::clear_subkernels().await;
return Ok(());
}
let request = request.unwrap();
@ -607,29 +460,6 @@ async fn handle_connection(
)
.await?;
}
Request::UploadSubkernel => {
#[cfg(has_drtio)]
{
let id = read_i32(stream).await? as u32;
let destination = read_i8(stream).await? as u8;
let buffer = read_bytes(stream, 1024 * 1024).await?;
subkernel::add_subkernel(id, destination, buffer).await;
match subkernel::upload(aux_mutex, routing_table, timer, id).await {
Ok(_) => write_header(stream, Reply::LoadCompleted).await?,
Err(_) => {
write_header(stream, Reply::LoadFailed).await?;
write_chunk(stream, b"subkernel failed to load").await?;
return Err(Error::UnexpectedPattern);
}
}
}
#[cfg(not(has_drtio))]
{
write_header(stream, Reply::LoadFailed).await?;
write_chunk(stream, b"No DRTIO on this system, subkernels are not supported").await?;
return Err(Error::UnexpectedPattern);
}
}
_ => {
error!("unexpected request from host: {:?}", request);
return Err(Error::UnrecognizedPacket);
@ -692,10 +522,9 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
#[cfg(has_drtio_routing)]
drtio_routing::interconnect_disable_all();
rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, timer);
ksupport::setup_device_map(&cfg);
rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, timer, &cfg);
analyzer::start(&aux_mutex, &drtio_routing_table, &up_destinations, timer);
analyzer::start();
moninj::start(timer, &aux_mutex, &drtio_routing_table);
let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start()));

3
src/libksupport/src/eh_artiq.rs → src/runtime/src/eh_artiq.rs
View File

@ -422,7 +422,7 @@ extern "C" fn stop_fn(
}
// Must be kept in sync with preallocate_runtime_exception_names() in artiq/language/embedding_map.py
static EXCEPTION_ID_LOOKUP: [(&str, u32); 12] = [
static EXCEPTION_ID_LOOKUP: [(&str, u32); 11] = [
("RuntimeError", 0),
("RTIOUnderflow", 1),
("RTIOOverflow", 2),
@ -434,7 +434,6 @@ static EXCEPTION_ID_LOOKUP: [(&str, u32); 12] = [
("ZeroDivisionError", 8),
("IndexError", 9),
("UnwrapNoneError", 10),
("SubkernelError", 11),
];
pub fn get_exception_id(name: &str) -> u32 {

0
src/libksupport/src/i2c.rs → src/runtime/src/i2c.rs
View File

0
src/libksupport/src/irq.rs → src/runtime/src/irq.rs
View File

12
src/libksupport/src/kernel/api.rs → src/runtime/src/kernel/api.rs
View File

@ -5,8 +5,6 @@ use libc::{c_char, c_int, size_t};
use libm;
use log::{info, warn};
#[cfg(has_drtio)]
use super::subkernel;
use super::{cache,
core1::rtio_get_destination_status,
dma,
@ -116,16 +114,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(i2c_read = i2c::read),
api!(i2c_switch_select = i2c::switch_select),
// subkernel
#[cfg(has_drtio)]
api!(subkernel_load_run = subkernel::load_run),
#[cfg(has_drtio)]
api!(subkernel_await_finish = subkernel::await_finish),
#[cfg(has_drtio)]
api!(subkernel_send_message = subkernel::send_message),
#[cfg(has_drtio)]
api!(subkernel_await_message = subkernel::await_message),
// Double-precision floating-point arithmetic helper functions
// RTABI chapter 4.1.2, Table 2
api!(__aeabi_dadd),

0
src/libksupport/src/kernel/cache.rs → src/runtime/src/kernel/cache.rs
View File

0
src/libksupport/src/kernel/control.rs → src/runtime/src/kernel/control.rs
View File

0
src/libksupport/src/kernel/core1.rs → src/runtime/src/kernel/core1.rs
View File

0
src/libksupport/src/kernel/dma.rs → src/runtime/src/kernel/dma.rs
View File

55
src/libksupport/src/kernel/mod.rs → src/runtime/src/kernel/mod.rs
View File

@ -3,7 +3,7 @@ use core::ptr;
use libcortex_a9::{mutex::Mutex, semaphore::Semaphore, sync_channel};
use crate::{eh_artiq, RPCException};
use crate::eh_artiq;
mod control;
pub use control::Control;
@ -13,17 +13,16 @@ mod dma;
mod rpc;
pub use dma::DmaRecorder;
mod cache;
#[cfg(has_drtio)]
mod subkernel;
#[cfg(has_drtio)]
#[derive(Debug, Clone)]
pub enum SubkernelStatus {
NoError,
Timeout,
IncorrectState,
CommLost,
OtherError,
pub struct RPCException {
pub id: u32,
pub message: u32,
pub param: [i64; 3],
pub file: u32,
pub line: i32,
pub column: i32,
pub function: u32,
}
#[derive(Debug, Clone)]
@ -73,42 +72,6 @@ pub enum Message {
UpDestinationsRequest(i32),
#[cfg(has_drtio)]
UpDestinationsReply(bool),
#[cfg(has_drtio)]
SubkernelLoadRunRequest {
id: u32,
run: bool,
},
#[cfg(has_drtio)]
SubkernelLoadRunReply {
succeeded: bool,
},
#[cfg(has_drtio)]
SubkernelAwaitFinishRequest {
id: u32,
timeout: u64,
},
#[cfg(has_drtio)]
SubkernelAwaitFinishReply {
status: SubkernelStatus,
},
#[cfg(has_drtio)]
SubkernelMsgSend {
id: u32,
data: Vec<u8>,
},
#[cfg(has_drtio)]
SubkernelMsgSent,
#[cfg(has_drtio)]
SubkernelMsgRecvRequest {
id: u32,
timeout: u64,
},
#[cfg(has_drtio)]
SubkernelMsgRecvReply {
status: SubkernelStatus,
count: u8,
},
}
static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None);

0
src/libksupport/src/kernel/rpc.rs → src/runtime/src/kernel/rpc.rs
View File

144
src/runtime/src/main.rs
View File

@ -2,19 +2,17 @@
#![no_main]
#![recursion_limit = "1024"] // for futures_util::select!
#![feature(alloc_error_handler)]
#![feature(const_btree_new)]
#![feature(panic_info_message)]
#![feature(c_variadic)]
#![feature(const_btree_new)]
#![feature(const_in_array_repeat_expressions)]
#![feature(naked_functions)]
#![feature(asm)]
#[macro_use]
extern crate alloc;
#[cfg(feature = "target_kasli_soc")]
use core::cell::RefCell;
use ksupport;
use libasync::task;
#[cfg(has_drtio_eem)]
use libboard_artiq::drtio_eem;
use libasync::{block_async, task};
#[cfg(feature = "target_kasli_soc")]
use libboard_artiq::io_expander;
use libboard_artiq::{identifier_read, logger, pl};
@ -22,38 +20,85 @@ use libboard_zynq::{gic, mpcore, timer::GlobalTimer};
use libconfig::Config;
use libcortex_a9::l2c::enable_l2_cache;
use libsupport_zynq::ram;
use log::{info, warn};
use log::{error, info, warn};
use nb;
use void::Void;
const ASYNC_ERROR_COLLISION: u8 = 1 << 0;
const ASYNC_ERROR_BUSY: u8 = 1 << 1;
const ASYNC_ERROR_SEQUENCE_ERROR: u8 = 1 << 2;
mod analyzer;
mod comms;
mod eh_artiq;
mod i2c;
mod irq;
mod kernel;
mod mgmt;
mod moninj;
mod panic;
mod proto_async;
mod rpc_async;
mod rpc;
#[cfg(ki_impl = "csr")]
#[path = "rtio_csr.rs"]
mod rtio;
#[cfg(ki_impl = "acp")]
#[path = "rtio_acp.rs"]
mod rtio;
mod rtio_clocking;
mod rtio_dma;
mod rtio_mgt;
#[cfg(has_drtio)]
mod subkernel;
#[cfg(feature = "target_kasli_soc")]
async fn io_expanders_service(
i2c_bus: RefCell<&mut libboard_zynq::i2c::I2c>,
io_expander0: RefCell<io_expander::IoExpander>,
io_expander1: RefCell<io_expander::IoExpander>,
) {
static mut SEEN_ASYNC_ERRORS: u8 = 0;
pub unsafe fn get_async_errors() -> u8 {
let errors = SEEN_ASYNC_ERRORS;
SEEN_ASYNC_ERRORS = 0;
errors
}
fn wait_for_async_rtio_error() -> nb::Result<(), Void> {
unsafe {
if pl::csr::rtio_core::async_error_read() != 0 {
Ok(())
} else {
Err(nb::Error::WouldBlock)
}
}
}
async fn report_async_rtio_errors() {
loop {
task::r#yield().await;
io_expander0
.borrow_mut()
.service(&mut i2c_bus.borrow_mut())
.expect("I2C I/O expander #0 service failed");
io_expander1
.borrow_mut()
.service(&mut i2c_bus.borrow_mut())
.expect("I2C I/O expander #1 service failed");
let _ = block_async!(wait_for_async_rtio_error()).await;
unsafe {
let errors = pl::csr::rtio_core::async_error_read();
if errors & ASYNC_ERROR_COLLISION != 0 {
let channel = pl::csr::rtio_core::collision_channel_read();
error!(
"RTIO collision involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_BUSY != 0 {
let channel = pl::csr::rtio_core::busy_channel_read();
error!(
"RTIO busy error involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
}
if errors & ASYNC_ERROR_SEQUENCE_ERROR != 0 {
let channel = pl::csr::rtio_core::sequence_error_channel_read();
error!(
"RTIO sequence error involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
}
SEEN_ASYNC_ERRORS = errors;
pl::csr::rtio_core::async_error_write(errors);
}
}
}
@ -76,29 +121,21 @@ pub fn main_core0() {
info!("gateware ident: {}", identifier_read(&mut [0; 64]));
ksupport::i2c::init();
#[cfg(feature = "target_kasli_soc")]
let i2c_bus = unsafe { (ksupport::i2c::I2C_BUS).as_mut().unwrap() };
i2c::init();
#[cfg(feature = "target_kasli_soc")]
let (mut io_expander0, mut io_expander1);
#[cfg(feature = "target_kasli_soc")]
{
io_expander0 = io_expander::IoExpander::new(i2c_bus, 0).unwrap();
io_expander1 = io_expander::IoExpander::new(i2c_bus, 1).unwrap();
io_expander0
.init(i2c_bus)
.expect("I2C I/O expander #0 initialization failed");
io_expander1
.init(i2c_bus)
.expect("I2C I/O expander #1 initialization failed");
// Drive TX_DISABLE to false on SFP0..3
io_expander0.set(0, 1, false);
io_expander1.set(0, 1, false);
io_expander0.set(1, 1, false);
io_expander1.set(1, 1, false);
io_expander0.service(i2c_bus).unwrap();
io_expander1.service(i2c_bus).unwrap();
let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
for expander_i in 0..=1 {
let mut io_expander = io_expander::IoExpander::new(i2c, expander_i).unwrap();
io_expander.init().expect("I2C I/O expander #0 initialization failed");
// Actively drive TX_DISABLE to false on SFP0..3
io_expander.set_oe(0, 1 << 1).unwrap();
io_expander.set_oe(1, 1 << 1).unwrap();
io_expander.set(0, 1, false);
io_expander.set(1, 1, false);
io_expander.service().unwrap();
}
}
let cfg = match Config::new() {
@ -111,16 +148,7 @@ pub fn main_core0() {
rtio_clocking::init(&mut timer, &cfg);
#[cfg(has_drtio_eem)]
drtio_eem::init(&mut timer, &cfg);
task::spawn(report_async_rtio_errors());
task::spawn(ksupport::report_async_rtio_errors());
#[cfg(feature = "target_kasli_soc")]
task::spawn(io_expanders_service(
RefCell::new(i2c_bus),
RefCell::new(io_expander0),
RefCell::new(io_expander1),
));
comms::main(timer, cfg);
}

124
src/libksupport/src/rpc.rs → src/runtime/src/rpc.rs
View File

@ -1,62 +1,71 @@
use core::str;
use alloc::boxed::Box;
use core::{future::Future, str};
use async_recursion::async_recursion;
use byteorder::{ByteOrder, NativeEndian};
use core_io::{Error, Read, Write};
use core_io::{Error, Write};
use cslice::{CMutSlice, CSlice};
use io::{ProtoRead, ProtoWrite};
use io::proto::ProtoWrite;
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp;
use log::trace;
use self::tag::{split_tag, Tag, TagIterator};
use crate::proto_async;
#[inline]
pub fn round_up(val: usize, power_of_two: usize) -> usize {
fn round_up(val: usize, power_of_two: usize) -> usize {
assert!(power_of_two.is_power_of_two());
let max_rem = power_of_two - 1;
(val + max_rem) & (!max_rem)
}
#[inline]
pub unsafe fn round_up_mut<T>(ptr: *mut T, power_of_two: usize) -> *mut T {
unsafe fn round_up_mut<T>(ptr: *mut T, power_of_two: usize) -> *mut T {
round_up(ptr as usize, power_of_two) as *mut T
}
#[inline]
pub unsafe fn round_up_const<T>(ptr: *const T, power_of_two: usize) -> *const T {
unsafe fn round_up_const<T>(ptr: *const T, power_of_two: usize) -> *const T {
round_up(ptr as usize, power_of_two) as *const T
}
#[inline]
pub unsafe fn align_ptr<T>(ptr: *const ()) -> *const T {
unsafe fn align_ptr<T>(ptr: *const ()) -> *const T {
round_up_const(ptr, core::mem::align_of::<T>()) as *const T
}
#[inline]
pub unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T {
unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T {
round_up_mut(ptr, core::mem::align_of::<T>()) as *mut T
}
// versions for reader rather than TcpStream
// they will be made into sync for satellite subkernels later
unsafe fn recv_elements<F, R>(
reader: &mut R,
elt_tag: Tag,
/// Reads (deserializes) `length` array or list elements of type `tag` from `stream`,
/// writing them into the buffer given by `storage`.
///
/// `alloc` is used for nested allocations (if elements themselves contain
/// lists/arrays), see [recv_value].
#[async_recursion(?Send)]
async unsafe fn recv_elements<F>(
stream: &TcpStream,
elt_tag: Tag<'async_recursion>,
length: usize,
storage: *mut (),
alloc: &mut F,
) -> Result<(), Error>
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: FnMut(usize) -> *mut (),
R: Read + ?Sized,
F: Future<Output = *mut ()>,
{
// List of simple types are special-cased in the protocol for performance.
match elt_tag {
Tag::Bool => {
let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length);
reader.read_exact(dest)?;
proto_async::read_chunk(stream, dest).await?;
}
Tag::Int32 => {
let ptr = storage as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
reader.read_exact(dest)?;
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest);
@ -64,7 +73,7 @@ where
Tag::Int64 | Tag::Float64 => {
let ptr = storage as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
reader.read_exact(dest)?;
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
@ -72,17 +81,27 @@ where
_ => {
let mut data = storage;
for _ in 0..length {
recv_value(reader, elt_tag, &mut data, alloc)?
recv_value(stream, elt_tag, &mut data, alloc).await?
}
}
}
Ok(())
}
unsafe fn recv_value<F, R>(reader: &mut R, tag: Tag, data: &mut *mut (), alloc: &mut F) -> Result<(), Error>
/// Reads (deserializes) a value of type `tag` from `stream`, writing the results to
/// the kernel-side buffer `data` (the passed pointer to which is incremented to point
/// past the just-received data). For nested allocations (lists/arrays), `alloc` is
/// invoked any number of times with the size of the required allocation as a parameter
/// (which is assumed to be correctly aligned for all payload types).
#[async_recursion(?Send)]
async unsafe fn recv_value<F>(
stream: &TcpStream,
tag: Tag<'async_recursion>,
data: &mut *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: FnMut(usize) -> *mut (),
R: Read + ?Sized,
F: Future<Output = *mut ()>,
{
macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{
@ -95,22 +114,22 @@ where
match tag {
Tag::None => Ok(()),
Tag::Bool => consume_value!(i8, |ptr| {
*ptr = reader.read_u8()? as i8;
*ptr = proto_async::read_i8(stream).await?;
Ok(())
}),
Tag::Int32 => consume_value!(i32, |ptr| {
*ptr = reader.read_u32()? as i32;
*ptr = proto_async::read_i32(stream).await?;
Ok(())
}),
Tag::Int64 | Tag::Float64 => consume_value!(i64, |ptr| {
*ptr = reader.read_u64()? as i64;
*ptr = proto_async::read_i64(stream).await?;
Ok(())
}),
Tag::String | Tag::Bytes | Tag::ByteArray => {
consume_value!(CMutSlice<u8>, |ptr| {
let length = reader.read_u32()? as usize;
*ptr = CMutSlice::new(alloc(length) as *mut u8, length);
reader.read_exact((*ptr).as_mut())?;
let length = proto_async::read_i32(stream).await? as usize;
*ptr = CMutSlice::new(alloc(length).await as *mut u8, length);
proto_async::read_chunk(stream, (*ptr).as_mut()).await?;
Ok(())
})
}
@ -120,8 +139,10 @@ where
let mut it = it.clone();
for _ in 0..arity {
let tag = it.next().expect("truncated tag");
recv_value(reader, tag, data, alloc)?
recv_value(stream, tag, data, alloc).await?
}
// Take into account any tail padding (if element(s) with largest alignment
// are not at the end).
*data = round_up_mut(*data, alignment);
Ok(())
}
@ -133,41 +154,50 @@ where
}
consume_value!(*mut List, |ptr_to_list| {
let tag = it.clone().next().expect("truncated tag");
let length = reader.read_u32()? as usize;
let length = proto_async::read_i32(stream).await? as usize;
// To avoid multiple kernel CPU roundtrips, use a single allocation for
// both the pointer/length List (slice) and the backing storage for the
// elements. We can assume that alloc() is aligned suitably, so just
// need to take into account any extra padding required.
// (Note: At the time of writing, there will never actually be any types
// with alignment larger than 8 bytes, so storage_offset == 0 always.)
let list_size = 4 + 4;
let storage_offset = round_up(list_size, tag.alignment());
let storage_size = tag.size() * length;
let allocation = alloc(storage_offset + storage_size) as *mut u8;
let allocation = alloc(storage_offset + storage_size).await as *mut u8;
*ptr_to_list = allocation as *mut List;
let storage = allocation.offset(storage_offset as isize) as *mut ();
(**ptr_to_list).length = length;
(**ptr_to_list).elements = storage;
recv_elements(reader, tag, length, storage, alloc)
recv_elements(stream, tag, length, storage, alloc).await
})
}
Tag::Array(it, num_dims) => {
consume_value!(*mut (), |buffer| {
// Deserialize length along each dimension and compute total number of
// elements.
let mut total_len: usize = 1;
for _ in 0..num_dims {
let len = reader.read_u32()? as usize;
let len = proto_async::read_i32(stream).await? as usize;
total_len *= len;
consume_value!(usize, |ptr| *ptr = len)
}
// Allocate backing storage for elements; deserialize them.
let elt_tag = it.clone().next().expect("truncated tag");
*buffer = alloc(elt_tag.size() * total_len);
recv_elements(reader, elt_tag, total_len, *buffer, alloc)
*buffer = alloc(elt_tag.size() * total_len).await;
recv_elements(stream, elt_tag, total_len, *buffer, alloc).await
})
}
Tag::Range(it) => {
*data = round_up_mut(*data, tag.alignment());
let tag = it.clone().next().expect("truncated tag");
recv_value(reader, tag, data, alloc)?;
recv_value(reader, tag, data, alloc)?;
recv_value(reader, tag, data, alloc)?;
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
Ok(())
}
Tag::Keyword(_) => unreachable!(),
@ -175,17 +205,21 @@ where
}
}
pub fn recv_return<F, R>(reader: &mut R, tag_bytes: &[u8], data: *mut (), alloc: &mut F) -> Result<(), Error>
pub async fn recv_return<F>(
stream: &TcpStream,
tag_bytes: &[u8],
data: *mut (),
alloc: &impl Fn(usize) -> F,
) -> Result<(), smoltcp::Error>
where
F: FnMut(usize) -> *mut (),
R: Read + ?Sized,
F: Future<Output = *mut ()>,
{
let mut it = TagIterator::new(tag_bytes);
trace!("recv ...->{}", it);
let tag = it.next().expect("truncated tag");
let mut data = data;
unsafe { recv_value(reader, tag, &mut data, alloc)? };
unsafe { recv_value(stream, tag, &mut data, alloc).await? };
Ok(())
}
@ -333,7 +367,7 @@ where W: Write + ?Sized {
Ok(())
}
pub mod tag {
mod tag {
use core::fmt;
pub fn split_tag(tag_bytes: &[u8]) -> (&[u8], &[u8]) {

197
src/runtime/src/rpc_async.rs
View File

@ -1,197 +0,0 @@
use alloc::boxed::Box;
use core::future::Future;
use async_recursion::async_recursion;
use byteorder::{ByteOrder, NativeEndian};
use cslice::CMutSlice;
use ksupport::rpc::{tag::{Tag, TagIterator},
*};
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp;
use log::trace;
use crate::proto_async;
/// Reads (deserializes) `length` array or list elements of type `tag` from `stream`,
/// writing them into the buffer given by `storage`.
///
/// `alloc` is used for nested allocations (if elements themselves contain
/// lists/arrays), see [recv_value].
#[async_recursion(?Send)]
async unsafe fn recv_elements<F>(
stream: &TcpStream,
elt_tag: Tag<'async_recursion>,
length: usize,
storage: *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
// List of simple types are special-cased in the protocol for performance.
match elt_tag {
Tag::Bool => {
let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length);
proto_async::read_chunk(stream, dest).await?;
}
Tag::Int32 => {
let ptr = storage as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest);
}
Tag::Int64 | Tag::Float64 => {
let ptr = storage as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
}
_ => {
let mut data = storage;
for _ in 0..length {
recv_value(stream, elt_tag, &mut data, alloc).await?
}
}
}
Ok(())
}
/// Reads (deserializes) a value of type `tag` from `stream`, writing the results to
/// the kernel-side buffer `data` (the passed pointer to which is incremented to point
/// past the just-received data). For nested allocations (lists/arrays), `alloc` is
/// invoked any number of times with the size of the required allocation as a parameter
/// (which is assumed to be correctly aligned for all payload types).
#[async_recursion(?Send)]
async unsafe fn recv_value<F>(
stream: &TcpStream,
tag: Tag<'async_recursion>,
data: &mut *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{
let $ptr = align_ptr_mut::<$ty>(*data);
*data = $ptr.offset(1) as *mut ();
$map
}};
}
match tag {
Tag::None => Ok(()),
Tag::Bool => consume_value!(i8, |ptr| {
*ptr = proto_async::read_i8(stream).await?;
Ok(())
}),
Tag::Int32 => consume_value!(i32, |ptr| {
*ptr = proto_async::read_i32(stream).await?;
Ok(())
}),
Tag::Int64 | Tag::Float64 => consume_value!(i64, |ptr| {
*ptr = proto_async::read_i64(stream).await?;
Ok(())
}),
Tag::String | Tag::Bytes | Tag::ByteArray => {
consume_value!(CMutSlice<u8>, |ptr| {
let length = proto_async::read_i32(stream).await? as usize;
*ptr = CMutSlice::new(alloc(length).await as *mut u8, length);
proto_async::read_chunk(stream, (*ptr).as_mut()).await?;
Ok(())
})
}
Tag::Tuple(it, arity) => {
let alignment = tag.alignment();
*data = round_up_mut(*data, alignment);
let mut it = it.clone();
for _ in 0..arity {
let tag = it.next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await?
}
// Take into account any tail padding (if element(s) with largest alignment
// are not at the end).
*data = round_up_mut(*data, alignment);
Ok(())
}
Tag::List(it) => {
#[repr(C)]
struct List {
elements: *mut (),
length: usize,
}
consume_value!(*mut List, |ptr_to_list| {
let tag = it.clone().next().expect("truncated tag");
let length = proto_async::read_i32(stream).await? as usize;
// To avoid multiple kernel CPU roundtrips, use a single allocation for
// both the pointer/length List (slice) and the backing storage for the
// elements. We can assume that alloc() is aligned suitably, so just
// need to take into account any extra padding required.
// (Note: At the time of writing, there will never actually be any types
// with alignment larger than 8 bytes, so storage_offset == 0 always.)
let list_size = 4 + 4;
let storage_offset = round_up(list_size, tag.alignment());
let storage_size = tag.size() * length;
let allocation = alloc(storage_offset + storage_size).await as *mut u8;
*ptr_to_list = allocation as *mut List;
let storage = allocation.offset(storage_offset as isize) as *mut ();
(**ptr_to_list).length = length;
(**ptr_to_list).elements = storage;
recv_elements(stream, tag, length, storage, alloc).await
})
}
Tag::Array(it, num_dims) => {
consume_value!(*mut (), |buffer| {
// Deserialize length along each dimension and compute total number of
// elements.
let mut total_len: usize = 1;
for _ in 0..num_dims {
let len = proto_async::read_i32(stream).await? as usize;
total_len *= len;
consume_value!(usize, |ptr| *ptr = len)
}
// Allocate backing storage for elements; deserialize them.
let elt_tag = it.clone().next().expect("truncated tag");
*buffer = alloc(elt_tag.size() * total_len).await;
recv_elements(stream, elt_tag, total_len, *buffer, alloc).await
})
}
Tag::Range(it) => {
*data = round_up_mut(*data, tag.alignment());
let tag = it.clone().next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?;
Ok(())
}
Tag::Keyword(_) => unreachable!(),
Tag::Object => unreachable!(),
}
}
pub async fn recv_return<F>(
stream: &TcpStream,
tag_bytes: &[u8],
data: *mut (),
alloc: &impl Fn(usize) -> F,
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
let mut it = TagIterator::new(tag_bytes);
trace!("recv ...->{}", it);
let tag = it.next().expect("truncated tag");
let mut data = data;
unsafe { recv_value(stream, tag, &mut data, alloc).await? };
Ok(())
}

5
src/libksupport/src/rtio_acp.rs → src/runtime/src/rtio_acp.rs
View File

@ -4,14 +4,13 @@ use cslice::CSlice;
use libcortex_a9::asm;
use vcell::VolatileCell;
use crate::{artiq_raise, pl::csr, resolve_channel_name, rtio_core};
use crate::{artiq_raise, pl::csr, rtio_mgt::resolve_channel_name};
pub const RTIO_O_STATUS_WAIT: i32 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: i32 = 2;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: i32 = 4;
pub const RTIO_I_STATUS_WAIT_EVENT: i32 = 1;
pub const RTIO_I_STATUS_OVERFLOW: i32 = 2;
#[allow(unused)]
pub const RTIO_I_STATUS_WAIT_STATUS: i32 = 4; // TODO
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: i32 = 8;
@ -52,7 +51,7 @@ static mut TRANSACTION_BUFFER: Transaction = Transaction {
pub extern "C" fn init() {
unsafe {
rtio_core::reset_write(1);
csr::rtio_core::reset_write(1);
csr::rtio::engine_addr_base_write(&TRANSACTION_BUFFER as *const Transaction as u32);
csr::rtio::enable_write(1);
}

21
src/runtime/src/rtio_clocking.rs
View File

@ -1,6 +1,4 @@
use embedded_hal::blocking::delay::DelayMs;
#[cfg(has_si5324)]
use ksupport::i2c;
use libboard_artiq::pl;
#[cfg(has_si5324)]
use libboard_artiq::si5324;
@ -10,6 +8,9 @@ use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use log::{info, warn};
#[cfg(has_si5324)]
use crate::i2c;
#[derive(Debug, PartialEq, Copy, Clone)]
#[allow(non_camel_case_types)]
pub enum RtioClock {
@ -75,7 +76,7 @@ fn init_rtio(timer: &mut GlobalTimer) {
}
// if it's not locked, it will hang at the CSR.
timer.delay_ms(50); // wait for CPLL/QPLL/SYS PLL lock
timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
@ -91,10 +92,10 @@ fn init_rtio(timer: &mut GlobalTimer) {
#[cfg(has_drtio)]
fn init_drtio(timer: &mut GlobalTimer) {
unsafe {
pl::csr::gt_drtio::stable_clkin_write(1);
pl::csr::drtio_transceiver::stable_clkin_write(1);
}
timer.delay_ms(50); // wait for CPLL/QPLL/SYS PLL lock
timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
@ -103,10 +104,7 @@ fn init_drtio(timer: &mut GlobalTimer) {
}
unsafe {
pl::csr::rtio_core::reset_phy_write(1);
pl::csr::gt_drtio::txenable_write(0xffffffffu32 as _);
#[cfg(has_drtio_eem)]
pl::csr::eem_transceiver::txenable_write(0xffffffffu32 as _);
pl::csr::drtio_transceiver::txenable_write(0xffffffffu32 as _);
}
}
@ -266,10 +264,7 @@ pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
{
let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
match clk {
RtioClock::Ext0_Bypass => {
info!("bypassing the PLL for RTIO clock");
si5324::bypass(i2c, SI5324_EXT_INPUT, timer).expect("cannot bypass Si5324")
}
RtioClock::Ext0_Bypass => si5324::bypass(i2c, SI5324_EXT_INPUT, timer).expect("cannot bypass Si5324"),
_ => setup_si5324(i2c, timer, clk),
}
}

4
src/libksupport/src/rtio_csr.rs → src/runtime/src/rtio_csr.rs
View File

@ -2,7 +2,7 @@ use core::ptr::{read_volatile, write_volatile};
use cslice::CSlice;
use crate::{artiq_raise, pl::csr, resolve_channel_name, rtio_core};
use crate::{artiq_raise, pl::csr, rtio_mgt::resolve_channel_name};
pub const RTIO_O_STATUS_WAIT: u8 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: u8 = 2;
@ -20,7 +20,7 @@ pub struct TimestampedData {
pub extern "C" fn init() {
unsafe {
rtio_core::reset_write(1);
csr::rtio_core::reset_write(1);
}
}

3
src/runtime/src/rtio_dma.rs
View File

@ -2,13 +2,14 @@ use alloc::{collections::BTreeMap, rc::Rc, string::String, vec::Vec};
#[cfg(has_drtio)]
use core::mem;
use ksupport::kernel::DmaRecorder;
#[cfg(has_drtio)]
use libasync::task;
use libboard_artiq::drtio_routing::RoutingTable;
use libboard_zynq::timer::GlobalTimer;
use libcortex_a9::{cache::dcci_slice, mutex::Mutex};
use crate::kernel::DmaRecorder;
const ALIGNMENT: usize = 16 * 8;
static DMA_RECORD_STORE: Mutex<BTreeMap<String, (u32, Vec<u8>, i64)>> = Mutex::new(BTreeMap::new());

492
src/runtime/src/rtio_mgt.rs
View File

@ -1,25 +1,28 @@
use alloc::rc::Rc;
use alloc::{collections::BTreeMap, rc::Rc, string::String};
use core::cell::RefCell;
use io::{Cursor, ProtoRead};
use libboard_artiq::{drtio_routing, pl::csr};
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use libcortex_a9::mutex::Mutex;
use log::warn;
static mut RTIO_DEVICE_MAP: BTreeMap<u32, String> = BTreeMap::new();
#[cfg(has_drtio)]
pub mod drtio {
use alloc::vec::Vec;
use embedded_hal::blocking::delay::DelayMs;
use ksupport::{resolve_channel_name, ASYNC_ERROR_BUSY, ASYNC_ERROR_COLLISION, ASYNC_ERROR_SEQUENCE_ERROR,
SEEN_ASYNC_ERRORS};
use libasync::{delay, task};
use libboard_artiq::{drtioaux::Error, drtioaux_async, drtioaux_async::Packet,
drtioaux_proto::MASTER_PAYLOAD_MAX_SIZE};
use libboard_artiq::{drtioaux::Error, drtioaux_async, drtioaux_async::Packet, drtioaux_proto::DMA_TRACE_MAX_SIZE};
use libboard_zynq::time::Milliseconds;
use log::{error, info, warn};
use super::*;
use crate::{analyzer::remote_analyzer::RemoteBuffer, rtio_dma::remote_dma, subkernel};
use crate::{rtio_dma::remote_dma, ASYNC_ERROR_BUSY, ASYNC_ERROR_COLLISION, ASYNC_ERROR_SEQUENCE_ERROR,
SEEN_ASYNC_ERRORS};
pub fn startup(
aux_mutex: &Rc<Mutex<bool>>,
@ -41,42 +44,6 @@ pub mod drtio {
unsafe { (csr::DRTIO[linkno].rx_up_read)() == 1 }
}
async fn process_async_packets(aux_mutex: &Mutex<bool>, linkno: u8, packet: Packet) -> Option<Packet> {
// returns None if an async packet has been consumed
match packet {
Packet::DmaPlaybackStatus {
id,
destination,
error,
channel,
timestamp,
} => {
remote_dma::playback_done(id, destination, error, channel, timestamp).await;
None
}
Packet::SubkernelFinished { id, with_exception } => {
subkernel::subkernel_finished(id, with_exception).await;
None
}
Packet::SubkernelMessage {
id,
destination: from,
last,
length,
data,
} => {
subkernel::message_handle_incoming(id, last, length as usize, &data).await;
// acknowledge receiving part of the message
let _lock = aux_mutex.async_lock().await;
drtioaux_async::send(linkno, &Packet::SubkernelMessageAck { destination: from })
.await
.unwrap();
None
}
other => Some(other),
}
}
async fn recv_aux_timeout(linkno: u8, timeout: u64, timer: GlobalTimer) -> Result<Packet, &'static str> {
if !link_rx_up(linkno).await {
return Err("link went down");
@ -99,7 +66,22 @@ pub mod drtio {
}
let _lock = aux_mutex.async_lock().await;
drtioaux_async::send(linkno, request).await.unwrap();
Ok(recv_aux_timeout(linkno, 200, timer).await?)
loop {
let reply = recv_aux_timeout(linkno, 200, timer).await;
match reply {
Ok(Packet::DmaPlaybackStatus {
id,
destination,
error,
channel,
timestamp,
}) => {
remote_dma::playback_done(id, destination, error, channel, timestamp).await;
}
Ok(packet) => return Ok(packet),
Err(e) => return Err(e),
}
}
}
async fn drain_buffer(linkno: u8, draining_time: Milliseconds, timer: GlobalTimer) {
@ -208,11 +190,14 @@ pub mod drtio {
async fn process_unsolicited_aux(aux_mutex: &Rc<Mutex<bool>>, linkno: u8) {
let _lock = aux_mutex.async_lock().await;
match drtioaux_async::recv(linkno).await {
Ok(Some(packet)) => {
if let Some(packet) = process_async_packets(aux_mutex, linkno, packet).await {
warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet);
}
}
Ok(Some(Packet::DmaPlaybackStatus {
id,
destination,
error,
channel,
timestamp,
})) => remote_dma::playback_done(id, destination, error, channel, timestamp).await,
Ok(Some(packet)) => warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet),
Ok(None) => (),
Err(_) => warn!("[LINK#{}] aux packet error", linkno),
}
@ -281,72 +266,55 @@ pub mod drtio {
let linkno = hop - 1;
if destination_up(up_destinations, destination).await {
if up_links[linkno as usize] {
loop {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DestinationStatusRequest {
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::DestinationDownReply) => {
destination_set_up(routing_table, up_destinations, destination, false).await;
remote_dma::destination_changed(
aux_mutex,
routing_table,
timer,
destination,
false,
)
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DestinationStatusRequest {
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::DestinationDownReply) => {
destination_set_up(routing_table, up_destinations, destination, false).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, false)
.await;
subkernel::destination_changed(aux_mutex, routing_table, timer, destination, false)
.await;
}
Ok(Packet::DestinationOkReply) => (),
Ok(Packet::DestinationSequenceErrorReply { channel }) => {
error!(
"[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
}
Ok(Packet::DestinationCollisionReply { channel }) => {
error!(
"[DEST#{}] RTIO collision involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
}
Ok(Packet::DestinationBusyReply { channel }) => {
error!(
"[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
}
Ok(packet) => match process_async_packets(aux_mutex, linkno, packet).await {
Some(packet) => {
error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet)
}
None => continue,
},
Err(e) => error!("[DEST#{}] communication failed ({})", destination, e),
}
break;
Ok(Packet::DestinationOkReply) => (),
Ok(Packet::DestinationSequenceErrorReply { channel }) => {
error!(
"[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
}
Ok(Packet::DestinationCollisionReply { channel }) => {
error!(
"[DEST#{}] RTIO collision involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
}
Ok(Packet::DestinationBusyReply { channel }) => {
error!(
"[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
}
Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
Err(e) => error!("[DEST#{}] communication failed ({})", destination, e),
}
} else {
destination_set_up(routing_table, up_destinations, destination, false).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, false).await;
subkernel::destination_changed(aux_mutex, routing_table, timer, destination, false).await;
}
} else {
if up_links[linkno as usize] {
@ -366,8 +334,6 @@ pub mod drtio {
init_buffer_space(destination as u8, linkno).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, true)
.await;
subkernel::destination_changed(aux_mutex, routing_table, timer, destination, true)
.await;
}
Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
Err(e) => error!("[DEST#{}] communication failed ({})", destination, e),
@ -454,36 +420,6 @@ pub mod drtio {
}
}
async fn partition_data<PacketF, HandlerF>(
linkno: u8,
aux_mutex: &Rc<Mutex<bool>>,
timer: GlobalTimer,
data: &[u8],
packet_f: PacketF,
reply_handler_f: HandlerF,
) -> Result<(), &'static str>
where
PacketF: Fn(&[u8; MASTER_PAYLOAD_MAX_SIZE], bool, usize) -> Packet,
HandlerF: Fn(&Packet) -> Result<(), &'static str>,
{
let mut i = 0;
while i < data.len() {
let mut slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
let len: usize = if i + MASTER_PAYLOAD_MAX_SIZE < data.len() {
MASTER_PAYLOAD_MAX_SIZE
} else {
data.len() - i
} as usize;
let last = i + len == data.len();
slice[..len].clone_from_slice(&data[i..i + len]);
i += len;
let packet = packet_f(&slice, last, len);
let reply = aux_transact(aux_mutex, linkno, &packet, timer).await?;
reply_handler_f(&reply)?;
}
Ok(())
}
pub async fn ddma_upload_trace(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
@ -493,25 +429,44 @@ pub mod drtio {
trace: &Vec<u8>,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
partition_data(
linkno,
aux_mutex,
timer,
trace,
|slice, last, len| Packet::DmaAddTraceRequest {
id: id,
destination: destination,
last: last,
length: len as u16,
trace: *slice,
},
|reply| match reply {
Packet::DmaAddTraceReply { succeeded: true } => Ok(()),
Packet::DmaAddTraceReply { succeeded: false } => Err("error adding trace on satellite"),
_ => Err("adding DMA trace failed, unexpected aux packet"),
},
)
.await
let mut i = 0;
while i < trace.len() {
let mut trace_slice: [u8; DMA_TRACE_MAX_SIZE] = [0; DMA_TRACE_MAX_SIZE];
let len: usize = if i + DMA_TRACE_MAX_SIZE < trace.len() {
DMA_TRACE_MAX_SIZE
} else {
trace.len() - i
} as usize;
let last = i + len == trace.len();
trace_slice[..len].clone_from_slice(&trace[i..i + len]);
i += len;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DmaAddTraceRequest {
id: id,
destination: destination,
last: last,
length: len as u16,
trace: trace_slice,
},
timer,
)
.await;
match reply {
Ok(Packet::DmaAddTraceReply { succeeded: true }) => (),
Ok(Packet::DmaAddTraceReply { succeeded: false }) => {
return Err("error adding trace on satellite");
}
Ok(_) => {
return Err("adding DMA trace failed, unexpected aux packet");
}
Err(_) => {
return Err("adding DMA trace failed, aux error");
}
}
}
Ok(())
}
pub async fn ddma_send_erase(
@ -567,195 +522,46 @@ pub mod drtio {
Err(_) => Err("aux error on DMA playback"),
}
}
}
async fn analyzer_get_data(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
destination: u8,
) -> Result<RemoteBuffer, &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::AnalyzerHeaderRequest {
destination: destination,
},
timer,
)
.await;
let (sent, total, overflow) = match reply {
Ok(Packet::AnalyzerHeader {
sent_bytes,
total_byte_count,
overflow_occurred,
}) => (sent_bytes, total_byte_count, overflow_occurred),
Ok(_) => return Err("received unexpected aux packet during remote analyzer header request"),
Err(e) => return Err(e),
};
let mut remote_data: Vec<u8> = Vec::new();
if sent > 0 {
let mut last_packet = false;
while !last_packet {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::AnalyzerDataRequest {
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::AnalyzerData { last, length, data }) => {
last_packet = last;
remote_data.extend(&data[0..length as usize]);
}
Ok(_) => return Err("received unexpected aux packet during remote analyzer data request"),
Err(e) => return Err(e),
fn read_device_map(cfg: &Config) -> BTreeMap<u32, String> {
let mut device_map: BTreeMap<u32, String> = BTreeMap::new();
let _ = cfg
.read("device_map")
.and_then(|raw_bytes| {
let mut bytes_cr = Cursor::new(raw_bytes);
let size = bytes_cr.read_u32().unwrap();
for _ in 0..size {
let channel = bytes_cr.read_u32().unwrap();
let device_name = bytes_cr.read_string().unwrap();
if let Some(old_entry) = device_map.insert(channel, device_name.clone()) {
warn!(
"conflicting device map entries for RTIO channel {}: '{}' and '{}'",
channel, old_entry, device_name
);
}
}
}
Ok(RemoteBuffer {
sent_bytes: sent,
total_byte_count: total,
error: overflow,
data: remote_data,
Ok(())
})
}
.or_else(|err| {
warn!(
"error reading device map ({}), device names will not be available in RTIO error messages",
err
);
Err(err)
});
device_map
}
pub async fn analyzer_query(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) -> Result<Vec<RemoteBuffer>, &'static str> {
let mut remote_buffers: Vec<RemoteBuffer> = Vec::new();
for i in 1..drtio_routing::DEST_COUNT {
if destination_up(up_destinations, i as u8).await {
remote_buffers.push(analyzer_get_data(aux_mutex, routing_table, timer, i as u8).await?);
}
}
Ok(remote_buffers)
fn _resolve_channel_name(channel: u32, device_map: &BTreeMap<u32, String>) -> String {
match device_map.get(&channel) {
Some(val) => val.clone(),
None => String::from("unknown"),
}
}
pub async fn subkernel_upload(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
data: &Vec<u8>,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
partition_data(
linkno,
aux_mutex,
timer,
data,
|slice, last, len| Packet::SubkernelAddDataRequest {
id: id,
destination: destination,
last: last,
length: len as u16,
data: *slice,
},
|reply| match reply {
Packet::SubkernelAddDataReply { succeeded: true } => Ok(()),
Packet::SubkernelAddDataReply { succeeded: false } => Err("error adding subkernel on satellite"),
_ => Err("adding subkernel failed, unexpected aux packet"),
},
)
.await
}
pub async fn subkernel_load(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
run: bool,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::SubkernelLoadRunRequest {
id: id,
destination: destination,
run: run,
},
timer,
)
.await?;
match reply {
Packet::SubkernelLoadRunReply { succeeded: true } => return Ok(()),
Packet::SubkernelLoadRunReply { succeeded: false } => return Err("error on subkernel run request"),
_ => return Err("received unexpected aux packet during subkernel run"),
}
}
pub async fn subkernel_retrieve_exception(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
destination: u8,
) -> Result<Vec<u8>, &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let mut remote_data: Vec<u8> = Vec::new();
loop {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::SubkernelExceptionRequest {
destination: destination,
},
timer,
)
.await?;
match reply {
Packet::SubkernelException { last, length, data } => {
remote_data.extend(&data[0..length as usize]);
if last {
return Ok(remote_data);
}
}
_ => return Err("received unexpected aux packet during subkernel exception request"),
}
}
}
pub async fn subkernel_send_message(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
message: &[u8],
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
partition_data(
linkno,
aux_mutex,
timer,
message,
|slice, last, len| Packet::SubkernelMessage {
destination: destination,
id: id,
last: last,
length: len as u16,
data: *slice,
},
|reply| match reply {
Packet::SubkernelMessageAck { .. } => Ok(()),
_ => Err("sending message to subkernel failed, unexpected aux packet"),
},
)
.await
}
pub fn resolve_channel_name(channel: u32) -> String {
_resolve_channel_name(channel, unsafe { &RTIO_DEVICE_MAP })
}
#[cfg(not(has_drtio))]
@ -779,7 +585,11 @@ pub fn startup(
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
cfg: &Config,
) {
unsafe {
RTIO_DEVICE_MAP = read_device_map(cfg);
}
drtio::startup(aux_mutex, routing_table, up_destinations, timer);
unsafe {
csr::rtio_core::reset_phy_write(1);

289
src/runtime/src/subkernel.rs
View File

@ -1,289 +0,0 @@
use alloc::{collections::BTreeMap, rc::Rc, vec::Vec};
use libasync::task;
use libboard_artiq::{drtio_routing::RoutingTable, drtioaux_proto::MASTER_PAYLOAD_MAX_SIZE};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::mutex::Mutex;
use log::error;
use crate::rtio_mgt::drtio;
#[derive(Debug, PartialEq, Clone, Copy)]
pub enum FinishStatus {
Ok,
CommLost,
Exception,
}
#[derive(Debug, PartialEq, Clone, Copy)]
pub enum SubkernelState {
NotLoaded,
Uploaded,
Running,
Finished { status: FinishStatus },
}
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub enum Error {
Timeout,
IncorrectState,
SubkernelNotFound,
CommLost,
DrtioError(&'static str),
}
impl From<&'static str> for Error {
fn from(value: &'static str) -> Error {
Error::DrtioError(value)
}
}
pub struct SubkernelFinished {
pub id: u32,
pub status: FinishStatus,
pub exception: Option<Vec<u8>>,
}
struct Subkernel {
pub destination: u8,
pub data: Vec<u8>,
pub state: SubkernelState,
}
impl Subkernel {
pub fn new(destination: u8, data: Vec<u8>) -> Self {
Subkernel {
destination: destination,
data: data,
state: SubkernelState::NotLoaded,
}
}
}
static SUBKERNELS: Mutex<BTreeMap<u32, Subkernel>> = Mutex::new(BTreeMap::new());
pub async fn add_subkernel(id: u32, destination: u8, kernel: Vec<u8>) {
SUBKERNELS
.async_lock()
.await
.insert(id, Subkernel::new(destination, kernel));
}
pub async fn upload(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
) -> Result<(), Error> {
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
drtio::subkernel_upload(
aux_mutex,
routing_table,
timer,
id,
subkernel.destination,
&subkernel.data,
)
.await?;
subkernel.state = SubkernelState::Uploaded;
Ok(())
} else {
Err(Error::SubkernelNotFound)
}
}
pub async fn load(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
run: bool,
) -> Result<(), Error> {
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
if subkernel.state != SubkernelState::Uploaded {
return Err(Error::IncorrectState);
}
drtio::subkernel_load(aux_mutex, routing_table, timer, id, subkernel.destination, run).await?;
if run {
subkernel.state = SubkernelState::Running;
}
Ok(())
} else {
Err(Error::SubkernelNotFound)
}
}
pub async fn clear_subkernels() {
SUBKERNELS.async_lock().await.clear();
MESSAGE_QUEUE.async_lock().await.clear();
CURRENT_MESSAGES.async_lock().await.clear();
}
pub async fn subkernel_finished(id: u32, with_exception: bool) {
// called upon receiving DRTIO SubkernelRunDone
// may be None if session ends and is cleared
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
subkernel.state = SubkernelState::Finished {
status: match with_exception {
true => FinishStatus::Exception,
false => FinishStatus::Ok,
},
}
}
}
pub async fn destination_changed(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
let mut locked_subkernels = SUBKERNELS.async_lock().await;
for (id, subkernel) in locked_subkernels.iter_mut() {
if subkernel.destination == destination {
if up {
match drtio::subkernel_upload(aux_mutex, routing_table, timer, *id, destination, &subkernel.data).await
{
Ok(_) => subkernel.state = SubkernelState::Uploaded,
Err(e) => error!("Error adding subkernel on destination {}: {}", destination, e),
}
} else {
subkernel.state = match subkernel.state {
SubkernelState::Running => SubkernelState::Finished {
status: FinishStatus::CommLost,
},
_ => SubkernelState::NotLoaded,
}
}
}
}
}
pub async fn await_finish(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
timeout: u64,
) -> Result<SubkernelFinished, Error> {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Running | SubkernelState::Finished { .. } => (),
_ => return Err(Error::IncorrectState),
}
let max_time = timer.get_time() + Milliseconds(timeout);
while timer.get_time() < max_time {
{
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished { .. } => break,
_ => (),
};
}
task::r#yield().await;
}
if timer.get_time() >= max_time {
error!("Remote subkernel finish await timed out");
return Err(Error::Timeout);
}
if let Some(subkernel) = SUBKERNELS.async_lock().await.get_mut(&id) {
match subkernel.state {
SubkernelState::Finished { status } => {
subkernel.state = SubkernelState::Uploaded;
Ok(SubkernelFinished {
id: id,
status: status,
exception: if status == FinishStatus::Exception {
Some(
drtio::subkernel_retrieve_exception(aux_mutex, routing_table, timer, subkernel.destination)
.await?,
)
} else {
None
},
})
}
_ => Err(Error::IncorrectState),
}
} else {
Err(Error::SubkernelNotFound)
}
}
pub struct Message {
from_id: u32,
pub count: u8,
pub tag: u8,
pub data: Vec<u8>,
}
// FIFO queue of messages
static MESSAGE_QUEUE: Mutex<Vec<Message>> = Mutex::new(Vec::new());
// currently under construction message(s) (can be from multiple sources)
static CURRENT_MESSAGES: Mutex<BTreeMap<u32, Message>> = Mutex::new(BTreeMap::new());
pub async fn message_handle_incoming(id: u32, last: bool, length: usize, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
// called when receiving a message from satellite
if SUBKERNELS.async_lock().await.get(&id).is_none() {
// do not add messages for non-existing or deleted subkernels
return;
}
let mut current_messages = CURRENT_MESSAGES.async_lock().await;
match current_messages.get_mut(&id) {
Some(message) => message.data.extend(&data[..length]),
None => {
current_messages.insert(
id,
Message {
from_id: id,
count: data[0],
tag: data[1],
data: data[2..length].to_vec(),
},
);
}
};
if last {
// when done, remove from working queue
MESSAGE_QUEUE
.async_lock()
.await
.push(current_messages.remove(&id).unwrap());
}
}
pub async fn message_await(id: u32, timeout: u64, timer: GlobalTimer) -> Result<Message, Error> {
match SUBKERNELS.async_lock().await.get(&id).unwrap().state {
SubkernelState::Finished {
status: FinishStatus::CommLost,
} => return Err(Error::CommLost),
SubkernelState::Running | SubkernelState::Finished { .. } => (),
_ => return Err(Error::IncorrectState),
}
let max_time = timer.get_time() + Milliseconds(timeout);
while timer.get_time() < max_time {
{
let mut message_queue = MESSAGE_QUEUE.async_lock().await;
for i in 0..message_queue.len() {
let msg = &message_queue[i];
if msg.from_id == id {
let message = message_queue.remove(i);
return Ok(message);
}
}
}
task::r#yield().await;
}
Err(Error::Timeout)
}
pub async fn message_send<'a>(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
message: Vec<u8>,
) -> Result<(), Error> {
let destination = SUBKERNELS.async_lock().await.get(&id).unwrap().destination;
// rpc data prepared by the kernel core already
Ok(drtio::subkernel_send_message(aux_mutex, routing_table, timer, id, destination, &message).await?)
}

18
src/satman/Cargo.toml.tpl → src/satman/Cargo.toml
View File

@ -14,19 +14,15 @@ build_zynq = { path = "../libbuild_zynq" }
[dependencies]
log = { version = "0.4", default-features = false }
core_io = { version = "0.1", features = ["collections"] }
cslice = "0.3"
embedded-hal = "0.2"
libboard_zynq = { path = "@@ZYNQ_RS@@/libboard_zynq", features = ["ipv6"]}
libsupport_zynq = { path = "@@ZYNQ_RS@@/libsupport_zynq", default-features = false, features = ["alloc_core"] }
libcortex_a9 = { path = "@@ZYNQ_RS@@/libcortex_a9" }
libasync = { path = "@@ZYNQ_RS@@/libasync" }
libregister = { path = "@@ZYNQ_RS@@/libregister" }
libconfig = { path = "@@ZYNQ_RS@@/libconfig", features = ["fat_lfn", "ipv6"] }
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"]}
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] }
libboard_artiq = { path = "../libboard_artiq" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }
io = { path = "../libio", features = ["alloc"] }
ksupport = { path = "../libksupport" }
libc = { path = "../libc" }

126
src/satman/src/analyzer.rs
View File

@ -1,126 +0,0 @@
use core::cmp::min;
use libboard_artiq::{drtioaux_proto::SAT_PAYLOAD_MAX_SIZE, pl::csr};
use libcortex_a9::cache;
const BUFFER_SIZE: usize = 512 * 1024;
#[repr(align(64))]
struct Buffer {
data: [u8; BUFFER_SIZE],
}
static mut BUFFER: Buffer = Buffer { data: [0; BUFFER_SIZE] };
fn arm() {
unsafe {
let base_addr = &mut BUFFER.data[0] as *mut _ as usize;
let last_addr = &mut BUFFER.data[BUFFER_SIZE - 1] as *mut _ as usize;
csr::rtio_analyzer::dma_base_address_write(base_addr as u32);
csr::rtio_analyzer::message_encoder_overflow_reset_write(1);
csr::rtio_analyzer::dma_last_address_write(last_addr as u32);
csr::rtio_analyzer::dma_reset_write(1);
csr::rtio_analyzer::enable_write(1);
}
}
fn disarm() {
unsafe {
csr::rtio_analyzer::enable_write(0);
while csr::rtio_analyzer::busy_read() != 0 {}
cache::dcci_slice(&BUFFER.data);
}
}
pub struct Analyzer {
// necessary for keeping track of sent data
data_len: usize,
sent_bytes: usize,
data_pointer: usize,
}
pub struct Header {
pub total_byte_count: u64,
pub sent_bytes: u32,
pub error: bool,
}
pub struct AnalyzerSliceMeta {
pub len: u16,
pub last: bool,
}
impl Drop for Analyzer {
fn drop(&mut self) {
disarm();
}
}
impl Analyzer {
pub fn new() -> Analyzer {
// create and arm new Analyzer
arm();
Analyzer {
data_len: 0,
sent_bytes: 0,
data_pointer: 0,
}
}
pub fn get_header(&mut self) -> Header {
disarm();
let overflow = unsafe { csr::rtio_analyzer::message_encoder_overflow_read() != 0 };
let bus_err = unsafe { csr::rtio_analyzer::dma_bus_error_read() != 0 };
let total_byte_count = unsafe { csr::rtio_analyzer::dma_byte_count_read() as u64 };
let wraparound = total_byte_count >= BUFFER_SIZE as u64;
self.data_len = if wraparound {
BUFFER_SIZE
} else {
total_byte_count as usize
};
self.data_pointer = if wraparound {
(total_byte_count % BUFFER_SIZE as u64) as usize
} else {
0
};
self.sent_bytes = 0;
if overflow {
warn!("overflow occured");
}
if bus_err {
warn!("bus error occured");
}
Header {
total_byte_count: total_byte_count,
sent_bytes: self.data_len as u32,
error: overflow | bus_err,
}
}
pub fn get_data(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> AnalyzerSliceMeta {
let data = unsafe { &BUFFER.data[..] };
let i = (self.data_pointer + self.sent_bytes) % BUFFER_SIZE;
let len = min(SAT_PAYLOAD_MAX_SIZE, self.data_len - self.sent_bytes);
let last = self.sent_bytes + len == self.data_len;
if i + len >= BUFFER_SIZE {
data_slice[..(BUFFER_SIZE - i)].clone_from_slice(&data[i..BUFFER_SIZE]);
data_slice[(BUFFER_SIZE - i)..len].clone_from_slice(&data[..(i + len) % BUFFER_SIZE]);
} else {
data_slice[..len].clone_from_slice(&data[i..i + len]);
}
self.sent_bytes += len;
if last {
arm();
}
AnalyzerSliceMeta {
len: len as u16,
last: last,
}
}
}

4
src/satman/src/dma.rs
View File

@ -170,8 +170,4 @@ impl Manager {
}
}
}
pub fn running(&self) -> bool {
self.state == ManagerState::Playback
}
}

421
src/satman/src/main.rs
View File

@ -1,15 +1,13 @@
#![no_std]
#![no_main]
#![feature(alloc_error_handler, try_trait, never_type, panic_info_message)]
#![feature(never_type, panic_info_message, asm, naked_functions)]
#![feature(alloc_error_handler)]
#[macro_use]
extern crate log;
extern crate core_io;
extern crate cslice;
extern crate embedded_hal;
extern crate io;
extern crate ksupport;
extern crate libboard_artiq;
extern crate libboard_zynq;
extern crate libcortex_a9;
@ -20,29 +18,28 @@ extern crate unwind;
extern crate alloc;
use analyzer::Analyzer;
use core::sync::atomic::{AtomicBool, Ordering};
use dma::Manager as DmaManager;
use embedded_hal::blocking::delay::DelayUs;
#[cfg(feature = "target_kasli_soc")]
use libboard_artiq::io_expander;
#[cfg(has_si5324)]
use libboard_artiq::si5324;
use libboard_artiq::{drtio_routing, drtioaux,
drtioaux_proto::{MASTER_PAYLOAD_MAX_SIZE, SAT_PAYLOAD_MAX_SIZE},
identifier_read, logger,
pl::csr};
use libboard_artiq::{drtio_routing, drtioaux, identifier_read, logger, pl::csr};
#[cfg(feature = "target_kasli_soc")]
use libboard_zynq::error_led::ErrorLED;
use libboard_zynq::{i2c::I2c, print, println, time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::{l2c::enable_l2_cache, regs::MPIDR};
use libregister::RegisterR;
use libboard_zynq::{gic, i2c::I2c, mpcore, print, println, stdio, time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::{asm, interrupt_handler,
l2c::enable_l2_cache,
notify_spin_lock,
regs::{MPIDR, SP},
spin_lock_yield};
use libregister::{RegisterR, RegisterW};
use libsupport_zynq::ram;
use subkernel::Manager as KernelManager;
mod analyzer;
mod dma;
mod repeater;
mod subkernel;
fn drtiosat_reset(reset: bool) {
unsafe {
@ -98,8 +95,6 @@ fn process_aux_packet(
timer: &mut GlobalTimer,
i2c: &mut I2c,
dma_manager: &mut DmaManager,
analyzer: &mut Analyzer,
kernel_manager: &mut KernelManager,
) -> Result<(), drtioaux::Error> {
// In the code below, *_chan_sel_write takes an u8 if there are fewer than 256 channels,
// and u16 otherwise; hence the `as _` conversion.
@ -118,84 +113,42 @@ fn process_aux_packet(
drtioaux::send(0, &drtioaux::Packet::ResetAck)
}
drtioaux::Packet::DestinationStatusRequest { destination } => {
drtioaux::Packet::DestinationStatusRequest {
destination: _destination,
} => {
#[cfg(has_drtio_routing)]
let hop = _routing_table.0[destination as usize][*_rank as usize];
let hop = _routing_table.0[_destination as usize][*_rank as usize];
#[cfg(not(has_drtio_routing))]
let hop = 0;
if hop == 0 {
if let Some(status) = dma_manager.check_state() {
info!(
"playback done, error: {}, channel: {}, timestamp: {}",
status.error, status.channel, status.timestamp
);
drtioaux::send(
0,
&drtioaux::Packet::DmaPlaybackStatus {
destination: destination,
id: status.id,
error: status.error,
channel: status.channel,
timestamp: status.timestamp,
},
)?;
} else if let Some(subkernel_finished) = kernel_manager.get_last_finished() {
info!(
"subkernel {} finished, with exception: {}",
subkernel_finished.id, subkernel_finished.with_exception
);
drtioaux::send(
0,
&drtioaux::Packet::SubkernelFinished {
id: subkernel_finished.id,
with_exception: subkernel_finished.with_exception,
},
)?;
} else if kernel_manager.message_is_ready() {
let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
match kernel_manager.message_get_slice(&mut data_slice) {
Some(meta) => drtioaux::send(
0,
&drtioaux::Packet::SubkernelMessage {
destination: destination,
id: kernel_manager.get_current_id().unwrap(),
last: meta.last,
length: meta.len as u16,
data: data_slice,
},
)?,
None => warn!("subkernel message is ready but no message is present"),
}
} else {
let errors;
let errors;
unsafe {
errors = csr::drtiosat::rtio_error_read();
}
if errors & 1 != 0 {
let channel;
unsafe {
errors = csr::drtiosat::rtio_error_read();
channel = csr::drtiosat::sequence_error_channel_read();
csr::drtiosat::rtio_error_write(1);
}
if errors & 1 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::sequence_error_channel_read();
csr::drtiosat::rtio_error_write(1);
}
drtioaux::send(0, &drtioaux::Packet::DestinationSequenceErrorReply { channel })?;
} else if errors & 2 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::collision_channel_read();
csr::drtiosat::rtio_error_write(2);
}
drtioaux::send(0, &drtioaux::Packet::DestinationCollisionReply { channel })?;
} else if errors & 4 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::busy_channel_read();
csr::drtiosat::rtio_error_write(4);
}
drtioaux::send(0, &drtioaux::Packet::DestinationBusyReply { channel })?;
} else {
drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
drtioaux::send(0, &drtioaux::Packet::DestinationSequenceErrorReply { channel })?;
} else if errors & 2 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::collision_channel_read();
csr::drtiosat::rtio_error_write(2);
}
drtioaux::send(0, &drtioaux::Packet::DestinationCollisionReply { channel })?;
} else if errors & 4 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::busy_channel_read();
csr::drtiosat::rtio_error_write(4);
}
drtioaux::send(0, &drtioaux::Packet::DestinationBusyReply { channel })?;
} else {
drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
}
}
@ -207,7 +160,7 @@ fn process_aux_packet(
let repno = hop - 1;
match _repeaters[repno].aux_forward(
&drtioaux::Packet::DestinationStatusRequest {
destination: destination,
destination: _destination,
},
timer,
) {
@ -267,8 +220,8 @@ fn process_aux_packet(
drtioaux::Packet::MonitorRequest {
destination: _destination,
channel,
probe,
channel: _channel,
probe: _probe,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let value;
@ -288,9 +241,9 @@ fn process_aux_packet(
}
drtioaux::Packet::InjectionRequest {
destination: _destination,
channel,
overrd,
value,
channel: _channel,
overrd: _overrd,
value: _value,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
#[cfg(has_rtio_moninj)]
@ -303,8 +256,8 @@ fn process_aux_packet(
}
drtioaux::Packet::InjectionStatusRequest {
destination: _destination,
channel,
overrd,
channel: _channel,
overrd: _overrd,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let value;
@ -459,36 +412,6 @@ fn process_aux_packet(
)
}
drtioaux::Packet::AnalyzerHeaderRequest {
destination: _destination,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let header = analyzer.get_header();
drtioaux::send(
0,
&drtioaux::Packet::AnalyzerHeader {
total_byte_count: header.total_byte_count,
sent_bytes: header.sent_bytes,
overflow_occurred: header.error,
},
)
}
drtioaux::Packet::AnalyzerDataRequest {
destination: _destination,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
let meta = analyzer.get_data(&mut data_slice);
drtioaux::send(
0,
&drtioaux::Packet::AnalyzerData {
last: meta.last,
length: meta.len,
data: data_slice,
},
)
}
drtioaux::Packet::DmaAddTraceRequest {
destination: _destination,
id,
@ -514,98 +437,10 @@ fn process_aux_packet(
timestamp,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = if !kernel_manager.running() {
dma_manager.playback(id, timestamp).is_ok()
} else {
false
};
let succeeded = dma_manager.playback(id, timestamp).is_ok();
drtioaux::send(0, &drtioaux::Packet::DmaPlaybackReply { succeeded: succeeded })
}
drtioaux::Packet::SubkernelAddDataRequest {
destination: _destination,
id,
last,
length,
data,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = kernel_manager.add(id, last, &data, length as usize).is_ok();
drtioaux::send(0, &drtioaux::Packet::SubkernelAddDataReply { succeeded: succeeded })
}
drtioaux::Packet::SubkernelLoadRunRequest {
destination: _destination,
id,
run,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let mut succeeded = kernel_manager.load(id).is_ok();
// allow preloading a kernel with delayed run
if run {
if dma_manager.running() {
// cannot run kernel while DDMA is running
succeeded = false;
} else {
succeeded |= kernel_manager.run(id).is_ok();
}
}
drtioaux::send(0, &drtioaux::Packet::SubkernelLoadRunReply { succeeded: succeeded })
}
drtioaux::Packet::SubkernelExceptionRequest {
destination: _destination,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let mut data_slice: [u8; SAT_PAYLOAD_MAX_SIZE] = [0; SAT_PAYLOAD_MAX_SIZE];
let meta = kernel_manager.exception_get_slice(&mut data_slice);
drtioaux::send(
0,
&drtioaux::Packet::SubkernelException {
last: meta.last,
length: meta.len,
data: data_slice,
},
)
}
drtioaux::Packet::SubkernelMessage {
destination,
id: _id,
last,
length,
data,
} => {
forward!(_routing_table, destination, *_rank, _repeaters, &packet, timer);
kernel_manager.message_handle_incoming(last, length as usize, &data);
drtioaux::send(
0,
&drtioaux::Packet::SubkernelMessageAck {
destination: destination,
},
)
}
drtioaux::Packet::SubkernelMessageAck {
destination: _destination,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
if kernel_manager.message_ack_slice() {
let mut data_slice: [u8; MASTER_PAYLOAD_MAX_SIZE] = [0; MASTER_PAYLOAD_MAX_SIZE];
if let Some(meta) = kernel_manager.message_get_slice(&mut data_slice) {
drtioaux::send(
0,
&drtioaux::Packet::SubkernelMessage {
destination: *_rank,
id: kernel_manager.get_current_id().unwrap(),
last: meta.last,
length: meta.len as u16,
data: data_slice,
},
)?;
} else {
error!("Error receiving message slice");
}
}
Ok(())
}
_ => {
warn!("received unexpected aux packet");
Ok(())
@ -620,22 +455,10 @@ fn process_aux_packets(
timer: &mut GlobalTimer,
i2c: &mut I2c,
dma_manager: &mut DmaManager,
analyzer: &mut Analyzer,
kernel_manager: &mut KernelManager,
) {
let result = drtioaux::recv(0).and_then(|packet| {
if let Some(packet) = packet {
process_aux_packet(
repeaters,
routing_table,
rank,
packet,
timer,
i2c,
dma_manager,
analyzer,
kernel_manager,
)
process_aux_packet(repeaters, routing_table, rank, packet, timer, i2c, dma_manager)
} else {
Ok(())
}
@ -743,28 +566,20 @@ pub extern "C" fn main_core0() -> i32 {
ram::init_alloc_core0();
ksupport::i2c::init();
let mut i2c = unsafe { (ksupport::i2c::I2C_BUS).as_mut().unwrap() };
#[cfg(feature = "target_kasli_soc")]
let (mut io_expander0, mut io_expander1);
let mut i2c = I2c::i2c0();
i2c.init().expect("I2C initialization failed");
#[cfg(feature = "target_kasli_soc")]
{
io_expander0 = io_expander::IoExpander::new(&mut i2c, 0).unwrap();
io_expander1 = io_expander::IoExpander::new(&mut i2c, 1).unwrap();
io_expander0
.init(&mut i2c)
.expect("I2C I/O expander #0 initialization failed");
io_expander1
.init(&mut i2c)
.expect("I2C I/O expander #1 initialization failed");
// Drive TX_DISABLE to false on SFP0..3
io_expander0.set(0, 1, false);
io_expander1.set(0, 1, false);
io_expander0.set(1, 1, false);
io_expander1.set(1, 1, false);
io_expander0.service(&mut i2c).unwrap();
io_expander1.service(&mut i2c).unwrap();
for expander_i in 0..=1 {
let mut io_expander = io_expander::IoExpander::new(&mut i2c, expander_i).unwrap();
io_expander.init().expect("I2C I/O expander #0 initialization failed");
// Actively drive TX_DISABLE to false on SFP0..3
io_expander.set_oe(0, 1 << 1).unwrap();
io_expander.set_oe(1, 1 << 1).unwrap();
io_expander.set(0, 1, false);
io_expander.set(1, 1, false);
io_expander.service().unwrap();
}
}
#[cfg(has_si5324)]
@ -773,9 +588,9 @@ pub extern "C" fn main_core0() -> i32 {
timer.delay_us(100_000);
info!("Switching SYS clocks...");
unsafe {
csr::gt_drtio::stable_clkin_write(1);
csr::drtio_transceiver::stable_clkin_write(1);
}
timer.delay_us(50_000); // wait for CPLL/QPLL/MMCM lock
timer.delay_us(20_000); // wait for CPLL/QPLL/MMCM lock
let clk = unsafe { csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
@ -784,7 +599,7 @@ pub extern "C" fn main_core0() -> i32 {
}
unsafe {
csr::gt_drtio::txenable_write(0xffffffffu32 as _);
csr::drtio_transceiver::txenable_write(0xffffffffu32 as _);
}
#[cfg(has_drtio_routing)]
@ -799,8 +614,6 @@ pub extern "C" fn main_core0() -> i32 {
let mut hardware_tick_ts = 0;
let mut control = ksupport::kernel::Control::start();
loop {
while !drtiosat_link_rx_up() {
drtiosat_process_errors();
@ -808,16 +621,6 @@ pub extern "C" fn main_core0() -> i32 {
for mut rep in repeaters.iter_mut() {
rep.service(&routing_table, rank, &mut timer);
}
#[cfg(feature = "target_kasli_soc")]
{
io_expander0
.service(&mut i2c)
.expect("I2C I/O expander #0 service failed");
io_expander1
.service(&mut i2c)
.expect("I2C I/O expander #1 service failed");
}
hardware_tick(&mut hardware_tick_ts, &mut timer);
}
@ -828,12 +631,10 @@ pub extern "C" fn main_core0() -> i32 {
si5324::siphaser::calibrate_skew(&mut timer).expect("failed to calibrate skew");
}
// Various managers created here, so when link is dropped, all DMA traces
// DMA manager created here, so when link is dropped, all DMA traces
// are cleared out for a clean slate on subsequent connections,
// without a manual intervention.
let mut dma_manager = DmaManager::new();
let mut analyzer = Analyzer::new();
let mut kernel_manager = KernelManager::new(&mut control);
drtioaux::reset(0);
drtiosat_reset(false);
@ -848,22 +649,11 @@ pub extern "C" fn main_core0() -> i32 {
&mut timer,
&mut i2c,
&mut dma_manager,
&mut analyzer,
&mut kernel_manager,
);
#[allow(unused_mut)]
for mut rep in repeaters.iter_mut() {
rep.service(&routing_table, rank, &mut timer);
}
#[cfg(feature = "target_kasli_soc")]
{
io_expander0
.service(&mut i2c)
.expect("I2C I/O expander #0 service failed");
io_expander1
.service(&mut i2c)
.expect("I2C I/O expander #1 service failed");
}
hardware_tick(&mut hardware_tick_ts, &mut timer);
if drtiosat_tsc_loaded() {
info!("TSC loaded from uplink");
@ -876,7 +666,24 @@ pub extern "C" fn main_core0() -> i32 {
error!("aux packet error: {:?}", e);
}
}
kernel_manager.process_kern_requests(rank, timer);
if let Some(status) = dma_manager.check_state() {
info!(
"playback done, error: {}, channel: {}, timestamp: {}",
status.error, status.channel, status.timestamp
);
if let Err(e) = drtioaux::send(
0,
&drtioaux::Packet::DmaPlaybackStatus {
destination: rank,
id: status.id,
error: status.error,
channel: status.channel,
timestamp: status.timestamp,
},
) {
error!("error sending DMA playback status: {:?}", e);
}
}
}
drtiosat_reset_phy(true);
@ -892,8 +699,46 @@ extern "C" {
static mut __stack1_start: u32;
}
interrupt_handler!(IRQ, irq, __irq_stack0_start, __irq_stack1_start, {
if MPIDR.read().cpu_id() == 1 {
let mpcore = mpcore::RegisterBlock::mpcore();
let mut gic = gic::InterruptController::gic(mpcore);
let id = gic.get_interrupt_id();
if id.0 == 0 {
gic.end_interrupt(id);
asm::exit_irq();
SP.write(&mut __stack1_start as *mut _ as u32);
asm::enable_irq();
CORE1_RESTART.store(false, Ordering::Relaxed);
notify_spin_lock();
main_core1();
}
stdio::drop_uart();
}
loop {}
});
static mut PANICKED: [bool; 2] = [false; 2];
static CORE1_RESTART: AtomicBool = AtomicBool::new(false);
pub fn restart_core1() {
let mut interrupt_controller = gic::InterruptController::gic(mpcore::RegisterBlock::mpcore());
CORE1_RESTART.store(true, Ordering::Relaxed);
interrupt_controller.send_sgi(gic::InterruptId(0), gic::CPUCore::Core1.into());
while CORE1_RESTART.load(Ordering::Relaxed) {
spin_lock_yield();
}
}
#[no_mangle]
pub fn main_core1() {
let mut interrupt_controller = gic::InterruptController::gic(mpcore::RegisterBlock::mpcore());
interrupt_controller.enable_interrupts();
loop {}
}
#[no_mangle]
pub extern "C" fn exception(_vect: u32, _regs: *const u32, pc: u32, ea: u32) {
fn hexdump(addr: u32) {
@ -949,3 +794,23 @@ pub fn panic_fmt(info: &core::panic::PanicInfo) -> ! {
loop {}
}
// linker symbols
extern "C" {
static __text_start: u32;
static __text_end: u32;
static __exidx_start: u32;
static __exidx_end: u32;
}
#[no_mangle]
extern "C" fn dl_unwind_find_exidx(_pc: *const u32, len_ptr: *mut u32) -> *const u32 {
let length;
let start: *const u32;
unsafe {
length = (&__exidx_end as *const u32).offset_from(&__exidx_start) as u32;
start = &__exidx_start;
*len_ptr = length;
}
start
}

692
src/satman/src/subkernel.rs
View File

@ -1,692 +0,0 @@
use alloc::{collections::{BTreeMap, VecDeque},
format,
string::{String, ToString},
vec::Vec};
use core::{cmp::min, option::NoneError, slice, str};
use core_io::{Error as IoError, Write};
use cslice::AsCSlice;
use io::{Cursor, ProtoRead, ProtoWrite};
use ksupport::{eh_artiq, kernel, rpc};
use libboard_artiq::{drtioaux_proto::{MASTER_PAYLOAD_MAX_SIZE, SAT_PAYLOAD_MAX_SIZE},
pl::csr};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::sync_channel::Receiver;
use log::warn;
#[derive(Debug, Clone, Copy, PartialEq)]
enum KernelState {
Absent,
Loaded,
Running,
MsgAwait(Milliseconds),
MsgSending,
}
#[derive(Debug)]
pub enum Error {
Load(String),
KernelNotFound,
Unexpected(String),
NoMessage,
AwaitingMessage,
SubkernelIoError,
KernelException(Sliceable),
}
impl From<NoneError> for Error {
fn from(_: NoneError) -> Error {
Error::KernelNotFound
}
}
impl From<IoError> for Error {
fn from(_value: IoError) -> Error {
Error::SubkernelIoError
}
}
impl From<()> for Error {
fn from(_: ()) -> Error {
Error::NoMessage
}
}
macro_rules! unexpected {
($($arg:tt)*) => (return Err(Error::Unexpected(format!($($arg)*))));
}
/* represents data that has to be sent to Master */
#[derive(Debug)]
pub struct Sliceable {
it: usize,
data: Vec<u8>,
}
/* represents interkernel messages */
struct Message {
count: u8,
tag: u8,
data: Vec<u8>,
}
#[derive(PartialEq)]
enum OutMessageState {
NoMessage,
MessageReady,
MessageBeingSent,
MessageSent,
MessageAcknowledged,
}
/* for dealing with incoming and outgoing interkernel messages */
struct MessageManager {
out_message: Option<Sliceable>,
out_state: OutMessageState,
in_queue: VecDeque<Message>,
in_buffer: Option<Message>,
}
// Per-run state
struct Session {
id: u32,
kernel_state: KernelState,
last_exception: Option<Sliceable>,
messages: MessageManager,
}
impl Session {
pub fn new(id: u32) -> Session {
Session {
id: id,
kernel_state: KernelState::Absent,
last_exception: None,
messages: MessageManager::new(),
}
}
fn running(&self) -> bool {
match self.kernel_state {
KernelState::Absent | KernelState::Loaded => false,
KernelState::Running | KernelState::MsgAwait { .. } | KernelState::MsgSending => true,
}
}
}
#[derive(Debug)]
struct KernelLibrary {
library: Vec<u8>,
complete: bool,
}
pub struct Manager<'a> {
kernels: BTreeMap<u32, KernelLibrary>,
session: Session,
control: &'a mut kernel::Control,
cache: BTreeMap<String, Vec<i32>>,
last_finished: Option<SubkernelFinished>,
}
pub struct SubkernelFinished {
pub id: u32,
pub with_exception: bool,
}
pub struct SliceMeta {
pub len: u16,
pub last: bool,
}
macro_rules! get_slice_fn {
($name:tt, $size:expr) => {
pub fn $name(&mut self, data_slice: &mut [u8; $size]) -> SliceMeta {
if self.data.len() == 0 {
return SliceMeta { len: 0, last: true };
}
let len = min($size, self.data.len() - self.it);
let last = self.it + len == self.data.len();
data_slice[..len].clone_from_slice(&self.data[self.it..self.it + len]);
self.it += len;
SliceMeta {
len: len as u16,
last: last,
}
}
};
}
impl Sliceable {
pub fn new(data: Vec<u8>) -> Sliceable {
Sliceable { it: 0, data: data }
}
get_slice_fn!(get_slice_sat, SAT_PAYLOAD_MAX_SIZE);
get_slice_fn!(get_slice_master, MASTER_PAYLOAD_MAX_SIZE);
}
impl MessageManager {
pub fn new() -> MessageManager {
MessageManager {
out_message: None,
out_state: OutMessageState::NoMessage,
in_queue: VecDeque::new(),
in_buffer: None,
}
}
pub fn handle_incoming(&mut self, last: bool, length: usize, data: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
// called when receiving a message from master
match self.in_buffer.as_mut() {
Some(message) => message.data.extend(&data[..length]),
None => {
self.in_buffer = Some(Message {
count: data[0],
tag: data[1],
data: data[2..length].to_vec(),
});
}
};
if last {
// when done, remove from working queue
self.in_queue.push_back(self.in_buffer.take().unwrap());
}
}
pub fn is_outgoing_ready(&mut self) -> bool {
// called by main loop, to see if there's anything to send, will send it afterwards
match self.out_state {
OutMessageState::MessageReady => {
self.out_state = OutMessageState::MessageBeingSent;
true
}
_ => false,
}
}
pub fn was_message_acknowledged(&mut self) -> bool {
match self.out_state {
OutMessageState::MessageAcknowledged => {
self.out_state = OutMessageState::NoMessage;
true
}
_ => false,
}
}
pub fn get_outgoing_slice(&mut self, data_slice: &mut [u8; MASTER_PAYLOAD_MAX_SIZE]) -> Option<SliceMeta> {
if self.out_state != OutMessageState::MessageBeingSent {
return None;
}
let meta = self.out_message.as_mut()?.get_slice_master(data_slice);
if meta.last {
// clear the message slot
self.out_message = None;
// notify kernel with a flag that message is sent
self.out_state = OutMessageState::MessageSent;
}
Some(meta)
}
pub fn ack_slice(&mut self) -> bool {
// returns whether or not there's more to be sent
match self.out_state {
OutMessageState::MessageBeingSent => true,
OutMessageState::MessageSent => {
self.out_state = OutMessageState::MessageAcknowledged;
false
}
_ => {
warn!("received unsolicited SubkernelMessageAck");
false
}
}
}
pub fn accept_outgoing(&mut self, message: Vec<u8>) -> Result<(), Error> {
// service tag skipped in kernel
self.out_message = Some(Sliceable::new(message));
self.out_state = OutMessageState::MessageReady;
Ok(())
}
pub fn get_incoming(&mut self) -> Option<Message> {
self.in_queue.pop_front()
}
}
impl<'a> Manager<'_> {
pub fn new(control: &mut kernel::Control) -> Manager {
Manager {
kernels: BTreeMap::new(),
session: Session::new(0),
control: control,
cache: BTreeMap::new(),
last_finished: None,
}
}
pub fn add(&mut self, id: u32, last: bool, data: &[u8], data_len: usize) -> Result<(), Error> {
let kernel = match self.kernels.get_mut(&id) {
Some(kernel) => {
if kernel.complete {
// replace entry
self.kernels.remove(&id);
self.kernels.insert(
id,
KernelLibrary {
library: Vec::new(),
complete: false,
},
);
self.kernels.get_mut(&id)?
} else {
kernel
}
}
None => {
self.kernels.insert(
id,
KernelLibrary {
library: Vec::new(),
complete: false,
},
);
self.kernels.get_mut(&id)?
}
};
kernel.library.extend(&data[0..data_len]);
kernel.complete = last;
Ok(())
}
pub fn running(&self) -> bool {
self.session.running()
}
pub fn get_current_id(&self) -> Option<u32> {
match self.running() {
true => Some(self.session.id),
false => None,
}
}
pub fn run(&mut self, id: u32) -> Result<(), Error> {
info!("starting subkernel #{}", id);
if self.session.kernel_state != KernelState::Loaded || self.session.id != id {
self.load(id)?;
}
self.session.kernel_state = KernelState::Running;
unsafe {
csr::cri_con::selected_write(2);
}
self.control.tx.send(kernel::Message::StartRequest);
Ok(())
}
pub fn message_handle_incoming(&mut self, last: bool, length: usize, slice: &[u8; MASTER_PAYLOAD_MAX_SIZE]) {
if !self.running() {
return;
}
self.session.messages.handle_incoming(last, length, slice);
}
pub fn message_get_slice(&mut self, slice: &mut [u8; MASTER_PAYLOAD_MAX_SIZE]) -> Option<SliceMeta> {
if !self.running() {
return None;
}
self.session.messages.get_outgoing_slice(slice)
}
pub fn message_ack_slice(&mut self) -> bool {
if !self.running() {
warn!("received unsolicited SubkernelMessageAck");
return false;
}
self.session.messages.ack_slice()
}
pub fn message_is_ready(&mut self) -> bool {
self.session.messages.is_outgoing_ready()
}
pub fn load(&mut self, id: u32) -> Result<(), Error> {
if self.session.id == id && self.session.kernel_state == KernelState::Loaded {
return Ok(());
}
if !self.kernels.get(&id)?.complete {
return Err(Error::KernelNotFound);
}
self.session = Session::new(id);
self.control.restart();
self.control
.tx
.send(kernel::Message::LoadRequest(self.kernels.get(&id)?.library.clone()));
let reply = self.control.rx.recv();
match reply {
kernel::Message::LoadCompleted => Ok(()),
kernel::Message::LoadFailed => Err(Error::Load("kernel load failed".to_string())),
_ => Err(Error::Load(format!(
"unexpected kernel CPU reply to load request: {:?}",
reply
))),
}
}
pub fn exception_get_slice(&mut self, data_slice: &mut [u8; SAT_PAYLOAD_MAX_SIZE]) -> SliceMeta {
match self.session.last_exception.as_mut() {
Some(exception) => exception.get_slice_sat(data_slice),
None => SliceMeta { len: 0, last: true },
}
}
pub fn get_last_finished(&mut self) -> Option<SubkernelFinished> {
self.last_finished.take()
}
fn kernel_stop(&mut self) {
self.session.kernel_state = KernelState::Absent;
unsafe {
csr::cri_con::selected_write(0);
}
}
fn runtime_exception(&mut self, cause: Error) {
let raw_exception: Vec<u8> = Vec::new();
let mut writer = Cursor::new(raw_exception);
match write_exception(
&mut writer,
&[Some(eh_artiq::Exception {
id: 11, // SubkernelError, defined in ksupport
message: format!("in subkernel id {}: {:?}", self.session.id, cause).as_c_slice(),
param: [0, 0, 0],
file: file!().as_c_slice(),
line: line!(),
column: column!(),
function: format!("subkernel id {}", self.session.id).as_c_slice(),
})],
&[eh_artiq::StackPointerBacktrace {
stack_pointer: 0,
initial_backtrace_size: 0,
current_backtrace_size: 0,
}],
&[],
0,
) {
Ok(_) => self.session.last_exception = Some(Sliceable::new(writer.into_inner())),
Err(_) => error!("Error writing exception data"),
}
self.kernel_stop();
}
pub fn process_kern_requests(&mut self, rank: u8, timer: GlobalTimer) {
if !self.running() {
return;
}
match self.process_external_messages(timer) {
Ok(()) => (),
Err(Error::AwaitingMessage) => return, // kernel still waiting, do not process kernel messages
Err(Error::KernelException(exception)) => {
self.session.last_exception = Some(exception);
self.last_finished = Some(SubkernelFinished {
id: self.session.id,
with_exception: true,
});
}
Err(e) => {
error!("Error while running processing external messages: {:?}", e);
self.runtime_exception(e);
self.last_finished = Some(SubkernelFinished {
id: self.session.id,
with_exception: true,
});
}
}
match self.process_kern_message(rank, timer) {
Ok(true) => {
self.last_finished = Some(SubkernelFinished {
id: self.session.id,
with_exception: false,
});
}
Ok(false) | Err(Error::NoMessage) => (),
Err(Error::KernelException(exception)) => {
self.session.last_exception = Some(exception);
self.last_finished = Some(SubkernelFinished {
id: self.session.id,
with_exception: true,
});
}
Err(e) => {
error!("Error while running kernel: {:?}", e);
self.runtime_exception(e);
self.last_finished = Some(SubkernelFinished {
id: self.session.id,
with_exception: true,
});
}
}
}
fn process_kern_message(&mut self, rank: u8, timer: GlobalTimer) -> Result<bool, Error> {
let reply = self.control.rx.try_recv()?;
match reply {
kernel::Message::KernelFinished(_async_errors) => {
self.kernel_stop();
return Ok(true);
}
kernel::Message::KernelException(exceptions, stack_pointers, backtrace, async_errors) => {
error!("exception in kernel");
for exception in exceptions {
error!("{:?}", exception.unwrap());
}
error!("stack pointers: {:?}", stack_pointers);
error!("backtrace: {:?}", backtrace);
let buf: Vec<u8> = Vec::new();
let mut writer = Cursor::new(buf);
match write_exception(&mut writer, exceptions, stack_pointers, backtrace, async_errors) {
Ok(()) => (),
Err(_) => error!("Error writing exception data"),
}
self.kernel_stop();
return Err(Error::KernelException(Sliceable::new(writer.into_inner())));
}
kernel::Message::CachePutRequest(key, value) => {
self.cache.insert(key, value);
}
kernel::Message::CacheGetRequest(key) => {
const DEFAULT: Vec<i32> = Vec::new();
let value = self.cache.get(&key).unwrap_or(&DEFAULT).clone();
self.control.tx.send(kernel::Message::CacheGetReply(value));
}
kernel::Message::SubkernelMsgSend { id: _, data } => {
self.session.messages.accept_outgoing(data)?;
self.session.kernel_state = KernelState::MsgSending;
}
kernel::Message::SubkernelMsgRecvRequest { id: _, timeout } => {
let max_time = timer.get_time() + Milliseconds(timeout);
self.session.kernel_state = KernelState::MsgAwait(max_time);
}
kernel::Message::UpDestinationsRequest(destination) => {
self.control
.tx
.send(kernel::Message::UpDestinationsReply(destination == (rank as i32)));
}
_ => {
unexpected!("unexpected message from core1 while kernel was running: {:?}", reply);
}
}
Ok(false)
}
fn process_external_messages(&mut self, timer: GlobalTimer) -> Result<(), Error> {
match self.session.kernel_state {
KernelState::MsgAwait(timeout) => {
if timer.get_time() > timeout {
self.control.tx.send(kernel::Message::SubkernelMsgRecvReply {
status: kernel::SubkernelStatus::Timeout,
count: 0,
});
self.session.kernel_state = KernelState::Running;
return Ok(());
}
if let Some(message) = self.session.messages.get_incoming() {
self.control.tx.send(kernel::Message::SubkernelMsgRecvReply {
status: kernel::SubkernelStatus::NoError,
count: message.count,
});
self.session.kernel_state = KernelState::Running;
self.pass_message_to_kernel(&message, timer)
} else {
Err(Error::AwaitingMessage)
}
}
KernelState::MsgSending => {
if self.session.messages.was_message_acknowledged() {
self.session.kernel_state = KernelState::Running;
self.control.tx.send(kernel::Message::SubkernelMsgSent);
Ok(())
} else {
Err(Error::AwaitingMessage)
}
}
_ => Ok(()),
}
}
fn pass_message_to_kernel(&mut self, message: &Message, timer: GlobalTimer) -> Result<(), Error> {
let mut reader = Cursor::new(&message.data);
let mut tag: [u8; 1] = [message.tag];
let mut i = message.count;
loop {
let slot = match recv_w_timeout(&mut self.control.rx, timer, 100)? {
kernel::Message::RpcRecvRequest(slot) => slot,
other => unexpected!("expected root value slot from core1, not {:?}", other),
};
let mut exception: Option<Sliceable> = None;
let mut unexpected: Option<String> = None;
rpc::recv_return(&mut reader, &tag, slot, &mut |size| {
if size == 0 {
0 as *mut ()
} else {
self.control.tx.send(kernel::Message::RpcRecvReply(Ok(size)));
match recv_w_timeout(&mut self.control.rx, timer, 100) {
Ok(kernel::Message::RpcRecvRequest(slot)) => slot,
Ok(kernel::Message::KernelException(exceptions, stack_pointers, backtrace, async_errors)) => {
let buf: Vec<u8> = Vec::new();
let mut writer = Cursor::new(buf);
match write_exception(&mut writer, exceptions, stack_pointers, backtrace, async_errors) {
Ok(()) => {
exception = Some(Sliceable::new(writer.into_inner()));
}
Err(_) => {
unexpected = Some("Error writing exception data".to_string());
}
};
0 as *mut ()
}
other => {
unexpected = Some(format!("expected nested value slot from kernel CPU, not {:?}", other));
0 as *mut ()
}
}
}
})?;
if let Some(exception) = exception {
self.kernel_stop();
return Err(Error::KernelException(exception));
} else if let Some(unexpected) = unexpected {
self.kernel_stop();
unexpected!("{}", unexpected);
}
self.control.tx.send(kernel::Message::RpcRecvReply(Ok(0)));
i -= 1;
if i == 0 {
break;
} else {
// update the tag for next read
tag[0] = reader.read_u8()?;
}
}
Ok(())
}
}
fn write_exception<W>(
writer: &mut W,
exceptions: &[Option<eh_artiq::Exception>],
stack_pointers: &[eh_artiq::StackPointerBacktrace],
backtrace: &[(usize, usize)],
async_errors: u8,
) -> Result<(), Error>
where
W: Write + ?Sized,
{
/* header */
writer.write_bytes(&[0x5a, 0x5a, 0x5a, 0x5a, /*Reply::KernelException*/ 9])?;
writer.write_u32(exceptions.len() as u32)?;
for exception in exceptions.iter() {
let exception = exception.as_ref().unwrap();
writer.write_u32(exception.id)?;
if exception.message.len() == usize::MAX {
// exception with host string
writer.write_u32(u32::MAX)?;
writer.write_u32(exception.message.as_ptr() as u32)?;
} else {
let msg =
str::from_utf8(unsafe { slice::from_raw_parts(exception.message.as_ptr(), exception.message.len()) })
.unwrap()
.replace(
"{rtio_channel_info:0}",
&format!(
"0x{:04x}:{}",
exception.param[0],
ksupport::resolve_channel_name(exception.param[0] as u32)
),
);
writer.write_string(&msg)?;
}
writer.write_u64(exception.param[0] as u64)?;
writer.write_u64(exception.param[1] as u64)?;
writer.write_u64(exception.param[2] as u64)?;
writer.write_bytes(exception.file.as_ref())?;
writer.write_u32(exception.line)?;
writer.write_u32(exception.column)?;
writer.write_bytes(exception.function.as_ref())?;
}
for sp in stack_pointers.iter() {
writer.write_u32(sp.stack_pointer as u32)?;
writer.write_u32(sp.initial_backtrace_size as u32)?;
writer.write_u32(sp.current_backtrace_size as u32)?;
}
writer.write_u32(backtrace.len() as u32)?;
for &(addr, sp) in backtrace {
writer.write_u32(addr as u32)?;
writer.write_u32(sp as u32)?;
}
writer.write_u8(async_errors as u8)?;
Ok(())
}
fn recv_w_timeout(
rx: &mut Receiver<'_, kernel::Message>,
timer: GlobalTimer,
timeout: u64,
) -> Result<kernel::Message, Error> {
let max_time = timer.get_time() + Milliseconds(timeout);
while timer.get_time() < max_time {
match rx.try_recv() {
Err(_) => (),
Ok(message) => return Ok(message),
}
}
Err(Error::NoMessage)
}