jcoates
/
artiq-zynq
forked from M-Labs/artiq-zynq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: jcoates:master
Branches Tags
jcoates:master
jcoates:release-6
jcoates:release-7
M-Labs:release-7
M-Labs:release-8
M-Labs:master
M-Labs:release-6
..
compare: jcoates:release-6
Branches Tags
jcoates:master
jcoates:release-6
jcoates:release-7
M-Labs:release-7
M-Labs:release-8
M-Labs:master
M-Labs:release-6

11 Commits
master ... release-6

Author SHA1 Message Date
mwojcik 0c1036cb0c qc2: add 4 edge counters to the end of rtio 2023-04-03 12:39:43 +08:00
Egor Savkin c484153348 Fix idle/startup_kernel typos in config 
Signed-off-by: Egor Savkin <es@m-labs.hk>
 2023-01-03 11:04:10 +08:00
Sebastien Bourdeauducq 49d9a644c6 mgmt: remove spurious config write warning 2022-03-16 08:25:05 +08:00
pca006132 14c2abe578 runtime/rpc: fixes alignment and size problem 2022-01-05 08:24:41 +08:00
Sebastien Bourdeauducq 1155802cf7 Revert "runtime: expose rint from libm" 
This reverts commit 95d024e38b.
 2021-10-11 08:13:15 +08:00
Sebastien Bourdeauducq 95d024e38b runtime: expose rint from libm 2021-10-10 20:40:43 +08:00
pca006132 d765b83e36 libdyld: fixed symbol relocation 
Note that in libdyld/src/lib.rs #117-118, image pointer is already added
to the symbol offset, so we do not need to add the pointer again
 2021-09-25 13:50:30 +08:00
Sebastien Bourdeauducq 62a30bda9b update dependencies 2021-07-27 16:29:09 +08:00
Sebastien Bourdeauducq 801d7ac911 nixpkgs 21.05 2021-07-27 15:46:32 +08:00
Sebastien Bourdeauducq 298085b62d fix compilation with nixpkgs 21.05 
The environment variable is optional to keep compatibility with other build environments.

Closes #131
 2021-07-27 15:40:56 +08:00
Sebastien Bourdeauducq 93e26087ee update copyright year 2021-07-07 18:07:34 +08:00
84 changed files with 2389 additions and 9311 deletions
Show Stats Expand all files Collapse all files

30
README.md
View File

@ -4,8 +4,8 @@ ARTIQ on Zynq
How to use How to use
---------- ----------
1. Install the ARTIQ version that corresponds to the artiq-zynq version are targeting. 1. Install ARTIQ-6 or newer.
2. Select the latest successful build on [Hydra](https://nixbld.m-labs.hk/) for the targeted artiq-zynq version, or use AFWS to obtain firmware binaries. 2. Select the latest successful build on Hydra: https://nixbld.m-labs.hk/jobset/artiq/zynq
3. Search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``). 3. Search for the job named ``<board>-<variant>-sd`` (for example: ``zc706-nist_clock-sd`` or ``zc706-nist_qc2-sd``).
4. Download the ``boot.bin`` "binary distribution" and place it at the root of a FAT-formatted SD card. 4. Download the ``boot.bin`` "binary distribution" and place it at the root of a FAT-formatted SD card.
5. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network. 5. Optionally, create a ``config.txt`` configuration file at the root of the SD card containing ``key=value`` pairs on each line. Use the ``ip``, ``ip6`` and ``mac`` keys to respectively set the IPv4, IPv6 and MAC address of the board. Configuring an IPv6 address is entirely optional. If these keys are not found, the firmware will use default values that may or may not be compatible with your network.
@ -24,7 +24,7 @@ The following configuration keys are available:
- ``ip``: IPv4 address. - ``ip``: IPv4 address.
- ``ip6``: IPv6 address. - ``ip6``: IPv6 address.
- ``startup``: startup kernel in ELF format (as produced by ``artiq_compile``). - ``startup``: startup kernel in ELF format (as produced by ``artiq_compile``).
- ``rtio_clock``: source of RTIO clock; valid values are ``ext0_bypass`` and ``int_125``. - ``rtioclk``: source of RTIO clock; valid values are ``external`` and ``internal``.
- ``boot``: SD card "boot.bin" file, for replacing the boot firmware/gateware. Write only. - ``boot``: SD card "boot.bin" file, for replacing the boot firmware/gateware. Write only.
Configurations can be read/written/removed via ``artiq_coremgmt``. Config erase is Configurations can be read/written/removed via ``artiq_coremgmt``. Config erase is
@ -33,38 +33,44 @@ not implemented as it seems not very useful.
Development instructions Development instructions
------------------------ ------------------------
ARTIQ on Zynq is packaged using the [Nix](https://nixos.org) Flakes system. Install Nix 2.8+ and enable flakes by adding ``experimental-features = nix-command flakes`` to ``nix.conf`` (e.g. ``~/.config/nix/nix.conf``). Configure Nix channels:
```shell
nix-channel --add https://nixbld.m-labs.hk/channel/custom/artiq/fast-beta/artiq-fast
nix-channel --update
```
Note: if you are using Nix channels the first time, you need to be aware of this bug: https://github.com/NixOS/nix/issues/3831
Pure build with Nix and execution on a remote JTAG server: Pure build with Nix and execution on a remote JTAG server:
```shell ```shell
nix build .#zc706-nist_clock-jtag # or zc706-nist_qc2-jtag or zc706-nist_clock_satellite-jtag etc. nix-build -A zc706-simple-jtag # or zc706-nist_qc2-jtag or zc706-nist_clock-jtag
./remote_run.sh ./remote_run.sh
``` ```
Impure incremental build and execution on a remote JTAG server: Impure incremental build and execution on a remote JTAG server:
```shell ```shell
nix develop nix-shell
cd src cd src
gateware/zc706.py -g ../build/gateware -V <variant> # build gateware gateware/zc706.py -g ../build/gateware # build gateware
make GWARGS="-V <variant>" <runtime/satman> # build firmware make # build firmware
cd .. cd ..
./remote_run.sh -i ./remote_run.sh -i
``` ```
Notes: Notes:
- This is developed with Nixpkgs 21.05, and the ``nixbld.m-labs.hk`` binary substituter can also be used here (see the ARTIQ manual for the public key and instructions).
- The impure build process is also compatible with non-Nix systems. - The impure build process is also compatible with non-Nix systems.
- When calling make, you need to specify both the variant and firmware type.
- Firmware type must be either ``runtime`` for DRTIO-less or DRTIO master variants, or ``satman`` for DRTIO satellite.
- If the board is connected to the local machine, use the ``local_run.sh`` script. - If the board is connected to the local machine, use the ``local_run.sh`` script.
- To update ``zynq-rs``, update the cargo files as per usual for Rust projects, but also keep ``flake.lock`` in sync. - To update ``zynq-rs``, update the cargo files as per usual for Rust projects, but also keep ``zynq-rs.nix`` in sync.
License License
------- -------
Copyright (C) 2019-2022 M-Labs Limited. Copyright (C) 2019-2021 M-Labs Limited.
ARTIQ is free software: you can redistribute it and/or modify ARTIQ is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by it under the terms of the GNU Lesser General Public License as published by

147
default.nix Normal file
View File

@ -0,0 +1,147 @@
let
zynq-rs = (import ./zynq-rs.nix);
pkgs = import <nixpkgs> { overlays = [ (import "${zynq-rs}/nix/mozilla-overlay.nix") ]; };
rustPlatform = (import "${zynq-rs}/nix/rust-platform.nix" { inherit pkgs; });
cargo-xbuild = (import zynq-rs).cargo-xbuild;
mkbootimage = import "${zynq-rs}/nix/mkbootimage.nix" { inherit pkgs; };
artiqpkgs = import <artiq-fast/default.nix> { inherit pkgs; };
vivado = import <artiq-fast/vivado.nix> { inherit pkgs; };
# FSBL configuration supplied by Vivado 2020.1 for these boards:
fsblTargets = ["zc702" "zc706" "zed"];
build = { target, variant }: let
szl = (import zynq-rs)."${target}-szl";
fsbl = import "${zynq-rs}/nix/fsbl.nix" {
inherit pkgs;
board = target;
};
firmware = rustPlatform.buildRustPackage rec {
# note: due to fetchCargoTarball, cargoSha256 depends on package name
name = "firmware";
src = ./src;
cargoSha256 = "1g1nb8jz1na1v60hq3zrpgrwnw9qz7xrnnr2lkiyhsp72bg52ssl";
nativeBuildInputs = [
pkgs.gnumake
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi misoc artiq ])))
cargo-xbuild
pkgs.llvmPackages_9.llvm
pkgs.llvmPackages_9.clang-unwrapped
];
buildPhase = ''
export XARGO_RUST_SRC="${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library"
export CLANG_EXTRA_INCLUDE_DIR="${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include"
export CARGO_HOME=$(mktemp -d cargo-home.XXX)
make TARGET=${target} VARIANT=${variant}
'';
installPhase = ''
mkdir -p $out $out/nix-support
cp ../build/runtime.bin $out/runtime.bin
cp ../build/firmware/armv7-none-eabihf/release/runtime $out/runtime.elf
echo file binary-dist $out/runtime.bin >> $out/nix-support/hydra-build-products
echo file binary-dist $out/runtime.elf >> $out/nix-support/hydra-build-products
'';
doCheck = false;
dontFixup = true;
};
gateware = pkgs.runCommand "${target}-${variant}-gateware"
{
nativeBuildInputs = [
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi misoc artiq ])))
vivado
];
}
''
python ${./src/gateware}/${target}.py -g build -V ${variant}
mkdir -p $out $out/nix-support
cp build/top.bit $out
echo file binary-dist $out/top.bit >> $out/nix-support/hydra-build-products
'';
# SZL startup
jtag = pkgs.runCommand "${target}-${variant}-jtag" {}
''
mkdir $out
ln -s ${szl}/szl.elf $out
ln -s ${firmware}/runtime.bin $out
ln -s ${gateware}/top.bit $out
'';
sd = pkgs.runCommand "${target}-${variant}-sd"
{
buildInputs = [ mkbootimage ];
}
''
# Do not use "long" paths in boot.bif, because embedded developers
# can't write software (mkbootimage will segfault).
bifdir=`mktemp -d`
cd $bifdir
ln -s ${szl}/szl.elf szl.elf
ln -s ${firmware}/runtime.elf runtime.elf
ln -s ${gateware}/top.bit top.bit
cat > boot.bif << EOF
the_ROM_image:
{
[bootloader]szl.elf
top.bit
runtime.elf
}
EOF
mkdir $out $out/nix-support
mkbootimage boot.bif $out/boot.bin
echo file binary-dist $out/boot.bin >> $out/nix-support/hydra-build-products
'';
# FSBL startup
fsbl-sd = pkgs.runCommand "${target}-${variant}-fsbl-sd"
{
buildInputs = [ mkbootimage ];
}
''
bifdir=`mktemp -d`
cd $bifdir
ln -s ${fsbl}/fsbl.elf fsbl.elf
ln -s ${gateware}/top.bit top.bit
ln -s ${firmware}/runtime.elf runtime.elf
cat > boot.bif << EOF
the_ROM_image:
{
[bootloader]fsbl.elf
top.bit
runtime.elf
}
EOF
mkdir $out $out/nix-support
mkbootimage boot.bif $out/boot.bin
echo file binary-dist $out/boot.bin >> $out/nix-support/hydra-build-products
'';
in {
"${target}-${variant}-firmware" = firmware;
"${target}-${variant}-gateware" = gateware;
"${target}-${variant}-jtag" = jtag;
"${target}-${variant}-sd" = sd;
} // (
if builtins.elem target fsblTargets
then {
"${target}-${variant}-fsbl-sd" = fsbl-sd;
}
else {}
);
in
(
(build { target = "zc706"; variant = "simple"; }) //
(build { target = "zc706"; variant = "nist_clock"; }) //
(build { target = "zc706"; variant = "nist_qc2"; }) //
(build { target = "zc706"; variant = "acpki_simple"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2"; }) //
(build { target = "coraz7"; variant = "10"; }) //
(build { target = "coraz7"; variant = "07s"; }) //
(build { target = "coraz7"; variant = "acpki_10"; }) //
(build { target = "coraz7"; variant = "acpki_07s"; }) //
(build { target = "redpitaya"; variant = "simple"; }) //
(build { target = "redpitaya"; variant = "acpki_simple"; }) //
{ inherit zynq-rs; }
)

60
demo.json
View File

@ -1,60 +0,0 @@
{
"target": "kasli_soc",
"variant": "demo",
"hw_rev": "v1.0",
"base": "standalone",
"peripherals": [
{
"type": "grabber",
"ports": [0]
},
{
"type": "dio",
"ports": [1],
"bank_direction_low": "input",
"bank_direction_high": "output"
},
{
"type": "dio",
"ports": [2],
"bank_direction_low": "output",
"bank_direction_high": "output"
},
{
"type": "urukul",
"dds": "ad9910",
"ports": [3, 4],
"clk_sel": 2
},
{
"type": "zotino",
"ports": [5]
},
{
"type": "sampler",
"ports": [6, 7]
},
{
"type": "mirny",
"ports": [8],
"clk_sel": 1,
"refclk": 125e6
},
{
"type": "fastino",
"ports": [9]
},
{
"type": "dio",
"ports": [10],
"bank_direction_low": "input",
"bank_direction_high": "input"
},
{
"type": "dio",
"ports": [11],
"bank_direction_low": "output",
"bank_direction_high": "input"
}
]
}

25
examples/device_db.py
View File

@ -8,7 +8,7 @@ device_db = {
"arguments": { "arguments": {
"host": "192.168.1.52", "host": "192.168.1.52",
"ref_period": 1e-9, "ref_period": 1e-9,
"ref_multiplier": 8, "ref_multiplier": 1,
"target": "cortexa9" "target": "cortexa9"
} }
}, },
@ -29,11 +29,30 @@ device_db = {
"class": "PCA9548" "class": "PCA9548"
}, },
# led? are common to all variants
"led0": { "led0": {
"type": "local", "type": "local",
"module": "artiq.coredevice.ttl", "module": "artiq.coredevice.ttl",
"class": "TTLOut", "class": "TTLOut",
"arguments": {"channel": 41}, "arguments": {"channel": 0},
},
"led1": {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLOut",
"arguments": {"channel": 1},
},
"led2": {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLOut",
"arguments": {"channel": 2}
},
"led3": {
"type": "local",
"module": "artiq.coredevice.ttl",
"class": "TTLOut",
"arguments": {"channel": 3}
}, },
} }
@ -43,7 +62,7 @@ for i in range(40):
"type": "local", "type": "local",
"module": "artiq.coredevice.ttl", "module": "artiq.coredevice.ttl",
"class": "TTLInOut", "class": "TTLInOut",
"arguments": {"channel": i} "arguments": {"channel": 4+i}
} }
device_db["ad9914dds0"] = { device_db["ad9914dds0"] = {

235
flake.lock
View File

@ -1,235 +0,0 @@
{
"nodes": {
"artiq": {
"inputs": {
"artiq-comtools": "artiq-comtools",
"mozilla-overlay": "mozilla-overlay",
"nixpkgs": "nixpkgs",
"sipyco": "sipyco",
"src-migen": "src-migen",
"src-misoc": "src-misoc",
"src-pythonparser": "src-pythonparser"
},
"locked": {
"lastModified": 1680398257,
"narHash": "sha256-ANODm+Xjx/OYHkmQpybDuIPXiu22IkoQ0wdI0ReQsNk=",
"ref": "refs/heads/master",
"rev": "7ba06bfe61cad4ad41f478188917cac6571a6875",
"revCount": 8323,
"type": "git",
"url": "https://github.com/m-labs/artiq.git"
},
"original": {
"type": "git",
"url": "https://github.com/m-labs/artiq.git"
}
},
"artiq-comtools": {
"inputs": {
"flake-utils": "flake-utils",
"nixpkgs": [
"artiq",
"nixpkgs"
],
"sipyco": [
"artiq",
"sipyco"
]
},
"locked": {
"lastModified": 1664405593,
"narHash": "sha256-yP441NerlLGig7n+9xHsx8yCtZ+Ggd0VqfBSzc20E04=",
"owner": "m-labs",
"repo": "artiq-comtools",
"rev": "15ddac62813ef623a076ccf982b3bc63d314e651",
"type": "github"
},
"original": {
"owner": "m-labs",
"repo": "artiq-comtools",
"type": "github"
}
},
"flake-utils": {
"locked": {
"lastModified": 1659877975,
"narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"mozilla-overlay": {
"flake": false,
"locked": {
"lastModified": 1672878308,
"narHash": "sha256-0+fl6PHokhtSV+w58z2QD2rTf8QhcOGsT9o4LwHHZHE=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "d38863db88e100866b3e494a651ee4962b762fcc",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"mozilla-overlay_2": {
"flake": false,
"locked": {
"lastModified": 1677493379,
"narHash": "sha256-A1gO8zlWLv3+tZ3cGVB1WYvvoN9pbFyv0xIJHcTsckw=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "78e723925daf5c9e8d0a1837ec27059e61649cb6",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"mozilla-overlay_3": {
"flake": false,
"locked": {
"lastModified": 1664789696,
"narHash": "sha256-UGWJHQShiwLCr4/DysMVFrYdYYHcOqAOVsWNUu+l6YU=",
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"rev": "80627b282705101e7b38e19ca6e8df105031b072",
"type": "github"
},
"original": {
"owner": "mozilla",
"repo": "nixpkgs-mozilla",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1673345971,
"narHash": "sha256-4DfFcKLRfVUTyuGrGNNmw37IeIZSoku9tgTVmu/iD98=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "54644f409ab471e87014bb305eac8c50190bcf48",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-22.11",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"artiq": "artiq",
"mozilla-overlay": "mozilla-overlay_2",
"zynq-rs": "zynq-rs"
}
},
"sipyco": {
"inputs": {
"nixpkgs": [
"artiq",
"nixpkgs"
]
},
"locked": {
"lastModified": 1673433867,
"narHash": "sha256-a7Oq35YoDzPtISbqAsaT+2/v15HZ7G1q0ukXmKWdb7Q=",
"owner": "m-labs",
"repo": "sipyco",
"rev": "38f8f4185d7db6b68bd7f71546da9077b1e2561c",
"type": "github"
},
"original": {
"owner": "m-labs",
"repo": "sipyco",
"type": "github"
}
},
"src-migen": {
"flake": false,
"locked": {
"lastModified": 1673433200,
"narHash": "sha256-ribBG06gsucz5oBS+O6aL8s2oJjx+qfl+vXmspts8gg=",
"owner": "m-labs",
"repo": "migen",
"rev": "f3e9145c9825514a1b4225378936569da4df8e12",
"type": "github"
},
"original": {
"owner": "m-labs",
"repo": "migen",
"type": "github"
}
},
"src-misoc": {
"flake": false,
"locked": {
"lastModified": 1671158014,
"narHash": "sha256-50w0K2E2ympYrG1Tte/HVbsp4FS2U+yohqZByXTOo4I=",
"ref": "refs/heads/master",
"rev": "26f039f9f6931a20a04ccd0f0a5402f67f553916",
"revCount": 2436,
"submodules": true,
"type": "git",
"url": "https://github.com/m-labs/misoc.git"
},
"original": {
"submodules": true,
"type": "git",
"url": "https://github.com/m-labs/misoc.git"
}
},
"src-pythonparser": {
"flake": false,
"locked": {
"lastModified": 1628745371,
"narHash": "sha256-p6TgeeaK4NEmbhimEXp31W8hVRo4DgWmcCoqZ+UdN60=",
"owner": "m-labs",
"repo": "pythonparser",
"rev": "5413ee5c9f8760e95c6acd5d6e88dabb831ad201",
"type": "github"
},
"original": {
"owner": "m-labs",
"repo": "pythonparser",
"type": "github"
}
},
"zynq-rs": {
"inputs": {
"mozilla-overlay": "mozilla-overlay_3",
"nixpkgs": [
"artiq",
"nixpkgs"
]
},
"locked": {
"lastModified": 1669819016,
"narHash": "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=",
"ref": "refs/heads/master",
"rev": "67dbb5932fa8ff5f143983476f741f945871d286",
"revCount": 624,
"type": "git",
"url": "https://git.m-labs.hk/m-labs/zynq-rs"
},
"original": {
"type": "git",
"url": "https://git.m-labs.hk/m-labs/zynq-rs"
}
}
},
"root": "root",
"version": 7
}

390
flake.nix
View File

@ -1,390 +0,0 @@
{
description = "ARTIQ port to the Zynq-7000 platform";
inputs.artiq.url = git+https://github.com/m-labs/artiq.git;
inputs.mozilla-overlay = { url = github:mozilla/nixpkgs-mozilla; flake = false; };
inputs.zynq-rs.url = git+https://git.m-labs.hk/m-labs/zynq-rs;
inputs.zynq-rs.inputs.nixpkgs.follows = "artiq/nixpkgs";
outputs = { self, mozilla-overlay, zynq-rs, artiq }:
let
pkgs = import artiq.inputs.nixpkgs { system = "x86_64-linux"; overlays = [ (import mozilla-overlay) ]; };
zynqpkgs = zynq-rs.packages.x86_64-linux;
artiqpkgs = artiq.packages.x86_64-linux;
rustPlatform = zynq-rs.rustPlatform;
fastnumbers = pkgs.python3Packages.buildPythonPackage rec {
pname = "fastnumbers";
version = "2.2.1";
src = pkgs.python3Packages.fetchPypi {
inherit pname version;
sha256 = "0j15i54p7nri6hkzn1wal9pxri4pgql01wgjccig6ar0v5jjbvsy";
};
};
artiq-netboot = pkgs.python3Packages.buildPythonPackage rec {
pname = "artiq-netboot";
version = "unstable-2020-10-15";
src = pkgs.fetchgit {
url = "https://git.m-labs.hk/m-labs/artiq-netboot.git";
rev = "04f69eb07df73abe4b89fde2c24084f7664f2104";
sha256 = "0ql4fr8m8gpb2yql8aqsdqsssxb8zqd6l65kl1f6s9845zy7shs9";
};
};
ramda = pkgs.python3Packages.buildPythonPackage {
pname = "ramda";
version = "unstable-2020-04-11";
src = pkgs.fetchFromGitHub {
owner = "peteut";
repo = "ramda.py";
rev = "d315a9717ebd639366bf3fe26bad9e3d08ec3c49";
sha256 = "sha256-bmSt/IHDnULsZjsC6edELnNH7LoJSVF4L4XhwBAXRkY=";
};
nativeBuildInputs = with pkgs.python3Packages; [ pbr ];
propagatedBuildInputs = with pkgs.python3Packages; [ future fastnumbers ];
checkInputs = with pkgs.python3Packages; [ pytest pytest-flake8 ];
checkPhase = "pytest";
doCheck = false;
preBuild = ''
export PBR_VERSION=0.5.5
'';
};
migen-axi = pkgs.python3Packages.buildPythonPackage {
pname = "migen-axi";
version = "unstable-2023-01-06";
src = pkgs.fetchFromGitHub {
owner = "peteut";
repo = "migen-axi";
rev = "27eaa84a70a3abfe1930c86c36c4de2cd652da35";
sha256 = "sha256-3Y9W5ns+1wbVd14iePzgSBzE+LxnGMUDtUw3BccFt80=";
};
format = "pyproject";
propagatedBuildInputs = with pkgs.python3Packages; [ setuptools click numpy toolz jinja2 ramda artiqpkgs.migen artiqpkgs.misoc ];
checkInputs = with pkgs.python3Packages; [ pytest pytest-timeout pytest-flake8 ];
checkPhase = "pytest";
# migen/misoc version checks are broken with pyproject for some reason
postPatch = ''
substituteInPlace pyproject.toml \
--replace '"migen@git+https://github.com/m-labs/migen",' ""
substituteInPlace pyproject.toml \
--replace '"misoc@git+https://github.com/m-labs/misoc.git",' ""
'';
};
binutils = { platform, target, zlib }: pkgs.stdenv.mkDerivation rec {
basename = "binutils";
version = "2.30";
name = "${basename}-${platform}-${version}";
src = pkgs.fetchurl {
url = "https://ftp.gnu.org/gnu/binutils/binutils-${version}.tar.bz2";
sha256 = "028cklfqaab24glva1ks2aqa1zxa6w6xmc8q34zs1sb7h22dxspg";
};
configureFlags =
[ "--enable-shared" "--enable-deterministic-archives" "--target=${target}"];
outputs = [ "out" "info" "man" ];
depsBuildBuild = [ pkgs.buildPackages.stdenv.cc ];
buildInputs = [ zlib ];
enableParallelBuilding = true;
};
binutils-arm = pkgs.callPackage binutils { platform = "arm"; target = "armv7-unknown-linux-gnueabihf"; };
# FSBL configuration supplied by Vivado 2020.1 for these boards:
fsblTargets = ["zc702" "zc706" "zed"];
sat_variants = [
# kasli-soc satellite variants
"satellite"
# zc706 satellite variants
"nist_clock_satellite" "nist_qc2_satellite" "acpki_nist_clock_satellite" "acpki_nist_qc2_satellite"
"nist_clock_satellite_100mhz" "nist_qc2_satellite_100mhz" "acpki_nist_clock_satellite_100mhz" "acpki_nist_qc2_satellite_100mhz"
];
build = { target, variant, json ? null }: let
szl = zynqpkgs."${target}-szl";
fsbl = zynqpkgs."${target}-fsbl";
fwtype = if builtins.elem variant sat_variants then "satman" else "runtime";
firmware = rustPlatform.buildRustPackage rec {
name = "firmware";
src = ./src;
cargoLock = {
lockFile = src/Cargo.lock;
outputHashes = {
"libasync-0.0.0" = "sha256-WvNMUekL4Elc55RdqX8XP43QPnBrK8Rbd0bsoI61E5U=";
};
};
nativeBuildInputs = [
pkgs.gnumake
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
zynqpkgs.cargo-xbuild
pkgs.llvmPackages_9.llvm
pkgs.llvmPackages_9.clang-unwrapped
];
buildPhase = ''
export XARGO_RUST_SRC="${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library"
export CLANG_EXTRA_INCLUDE_DIR="${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include"
export CARGO_HOME=$(mktemp -d cargo-home.XXX)
make TARGET=${target} GWARGS="${if json == null then "-V ${variant}" else json}" ${fwtype}
'';
installPhase = ''
mkdir -p $out $out/nix-support
cp ../build/${fwtype}.bin $out/${fwtype}.bin
cp ../build/firmware/armv7-none-eabihf/release/${fwtype} $out/${fwtype}.elf
echo file binary-dist $out/${fwtype}.bin >> $out/nix-support/hydra-build-products
echo file binary-dist $out/${fwtype}.elf >> $out/nix-support/hydra-build-products
'';
doCheck = false;
dontFixup = true;
};
gateware = pkgs.runCommand "${target}-${variant}-gateware"
{
nativeBuildInputs = [
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ ps.jsonschema migen migen-axi misoc artiq ])))
artiqpkgs.artiq
artiqpkgs.vivado
];
}
''
python ${./src/gateware}/${target}.py -g build ${if json == null then "-V ${variant}" else json}
mkdir -p $out $out/nix-support
cp build/top.bit $out
echo file binary-dist $out/top.bit >> $out/nix-support/hydra-build-products
'';
# SZL startup
jtag = pkgs.runCommand "${target}-${variant}-jtag" {}
''
mkdir $out
ln -s ${szl}/szl.elf $out
ln -s ${firmware}/${fwtype}.bin $out
ln -s ${gateware}/top.bit $out
'';
sd = pkgs.runCommand "${target}-${variant}-sd"
{
buildInputs = [ zynqpkgs.mkbootimage ];
}
''
# Do not use "long" paths in boot.bif, because embedded developers
# can't write software (mkbootimage will segfault).
bifdir=`mktemp -d`
cd $bifdir
ln -s ${szl}/szl.elf szl.elf
ln -s ${firmware}/${fwtype}.elf ${fwtype}.elf
ln -s ${gateware}/top.bit top.bit
cat > boot.bif << EOF
the_ROM_image:
{
[bootloader]szl.elf
top.bit
${fwtype}.elf
}
EOF
mkdir $out $out/nix-support
mkbootimage boot.bif $out/boot.bin
echo file binary-dist $out/boot.bin >> $out/nix-support/hydra-build-products
'';
# FSBL startup
fsbl-sd = pkgs.runCommand "${target}-${variant}-fsbl-sd"
{
buildInputs = [ zynqpkgs.mkbootimage ];
}
''
bifdir=`mktemp -d`
cd $bifdir
ln -s ${fsbl}/fsbl.elf fsbl.elf
ln -s ${gateware}/top.bit top.bit
ln -s ${firmware}/${fwtype}.elf ${fwtype}.elf
cat > boot.bif << EOF
the_ROM_image:
{
[bootloader]fsbl.elf
top.bit
${fwtype}.elf
}
EOF
mkdir $out $out/nix-support
mkbootimage boot.bif $out/boot.bin
echo file binary-dist $out/boot.bin >> $out/nix-support/hydra-build-products
'';
in {
"${target}-${variant}-firmware" = firmware;
"${target}-${variant}-gateware" = gateware;
"${target}-${variant}-jtag" = jtag;
"${target}-${variant}-sd" = sd;
} // (
if builtins.elem target fsblTargets
then {
"${target}-${variant}-fsbl-sd" = fsbl-sd;
}
else {}
);
gateware-sim = pkgs.stdenv.mkDerivation {
name = "gateware-sim";
nativeBuildInputs = [
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi artiq ])))
artiqpkgs.artiq
];
phases = [ "buildPhase" ];
buildPhase =
''
python -m unittest discover ${self}/src/gateware -v
touch $out
'';
};
fmt-check = pkgs.stdenv.mkDerivation {
name = "fmt-check";
nativeBuildInputs = [
rustPlatform.rust.cargo
];
phases = [ "buildPhase" ];
buildPhase =
''
cd ${self}/src
cargo fmt -- --check
touch $out
'';
};
# for hitl-tests
zc706-nist_qc2 = (build { target = "zc706"; variant = "nist_qc2"; });
zc706-hitl-tests = pkgs.stdenv.mkDerivation {
name = "zc706-hitl-tests";
__networked = true; # compatibility with old patched Nix
# breaks hydra, https://github.com/NixOS/hydra/issues/1216
#__impure = true; # Nix 2.8+
buildInputs = [
pkgs.netcat pkgs.openssh pkgs.rsync artiqpkgs.artiq artiq-netboot zynqpkgs.zc706-szl
];
phases = [ "buildPhase" ];
buildPhase =
''
export NIX_SSHOPTS="-F /dev/null -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -i /opt/hydra_id_ed25519"
LOCKCTL=$(mktemp -d)
mkfifo $LOCKCTL/lockctl
cat $LOCKCTL/lockctl | ${pkgs.openssh}/bin/ssh \
$NIX_SSHOPTS \
rpi-4 \
'mkdir -p /tmp/board_lock && flock /tmp/board_lock/zc706-1 -c "echo Ok; cat"' \
| (
# End remote flock via FIFO
atexit_unlock() {
echo > $LOCKCTL/lockctl
}
trap atexit_unlock EXIT
# Read "Ok" line when remote successfully locked
read LOCK_OK
echo Power cycling board...
(echo b; sleep 5; echo B; sleep 5) | nc -N -w6 192.168.1.31 3131
echo Power cycle done.
export USER=hydra
export OPENOCD_ZYNQ=${zynq-rs}/openocd
export SZL=${zynqpkgs.szl}
bash ${self}/remote_run.sh -h rpi-4 -o "$NIX_SSHOPTS" -d ${zc706-nist_qc2.zc706-nist_qc2-jtag}
echo Waiting for the firmware to boot...
sleep 15
echo Running test kernel...
artiq_run --device-db ${self}/examples/device_db.py ${self}/examples/mandelbrot.py
echo Running ARTIQ unit tests...
export ARTIQ_ROOT=${self}/examples
export ARTIQ_LOW_LATENCY=1
python -m unittest discover artiq.test.coredevice -v
touch $out
echo Completed
(echo b; sleep 5) | nc -N -w6 192.168.1.31 3131
echo Board powered off
)
'';
};
in rec {
packages.x86_64-linux =
{
inherit fastnumbers artiq-netboot ramda migen-axi binutils-arm;
} //
(build { target = "zc706"; variant = "nist_clock"; }) //
(build { target = "zc706"; variant = "nist_clock_master"; }) //
(build { target = "zc706"; variant = "nist_clock_satellite"; }) //
(build { target = "zc706"; variant = "nist_clock_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "nist_qc2"; }) //
(build { target = "zc706"; variant = "nist_qc2_master"; }) //
(build { target = "zc706"; variant = "nist_qc2_satellite"; }) //
(build { target = "zc706"; variant = "nist_qc2_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_master"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_satellite"; }) //
(build { target = "zc706"; variant = "acpki_nist_clock_satellite_100mhz"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_master"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_satellite"; }) //
(build { target = "zc706"; variant = "acpki_nist_qc2_satellite_100mhz"; }) //
(build { target = "kasli_soc"; variant = "demo"; json = ./demo.json; }) //
(build { target = "kasli_soc"; variant = "master"; json = ./kasli-soc-master.json; }) //
(build { target = "kasli_soc"; variant = "satellite"; json = ./kasli-soc-satellite.json; });
hydraJobs = packages.x86_64-linux // { inherit zc706-hitl-tests; inherit gateware-sim; inherit fmt-check; };
devShell.x86_64-linux = pkgs.mkShell {
name = "artiq-zynq-dev-shell";
buildInputs = with pkgs; [
rustPlatform.rust.rustc
rustPlatform.rust.cargo
llvmPackages_9.llvm
llvmPackages_9.clang-unwrapped
gnumake
cacert
zynqpkgs.cargo-xbuild
zynqpkgs.mkbootimage
openocd
openssh rsync
(python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi misoc artiq artiq-netboot ps.jsonschema ps.pyftdi ])))
artiqpkgs.artiq
artiqpkgs.vivado
binutils-arm
];
XARGO_RUST_SRC = "${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library";
CLANG_EXTRA_INCLUDE_DIR = "${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include";
OPENOCD_ZYNQ = "${zynq-rs}/openocd";
SZL = "${zynqpkgs.szl}";
};
makeArtiqZynqPackage = build;
};
}

60
kasli-soc-master.json
View File

@ -1,60 +0,0 @@
{
"target": "kasli_soc",
"variant": "master",
"hw_rev": "v1.0",
"base": "master",
"peripherals": [
{
"type": "grabber",
"ports": [0]
},
{
"type": "dio",
"ports": [1],
"bank_direction_low": "input",
"bank_direction_high": "output"
},
{
"type": "dio",
"ports": [2],
"bank_direction_low": "output",
"bank_direction_high": "output"
},
{
"type": "urukul",
"dds": "ad9910",
"ports": [3, 4],
"clk_sel": 2
},
{
"type": "zotino",
"ports": [5]
},
{
"type": "sampler",
"ports": [6, 7]
},
{
"type": "mirny",
"ports": [8],
"clk_sel": 1,
"refclk": 125e6
},
{
"type": "fastino",
"ports": [9]
},
{
"type": "dio",
"ports": [10],
"bank_direction_low": "input",
"bank_direction_high": "input"
},
{
"type": "dio",
"ports": [11],
"bank_direction_low": "output",
"bank_direction_high": "input"
}
]
}

60
kasli-soc-satellite.json
View File

@ -1,60 +0,0 @@
{
"target": "kasli_soc",
"variant": "satellite",
"hw_rev": "v1.0",
"base": "satellite",
"peripherals": [
{
"type": "grabber",
"ports": [0]
},
{
"type": "dio",
"ports": [1],
"bank_direction_low": "input",
"bank_direction_high": "output"
},
{
"type": "dio",
"ports": [2],
"bank_direction_low": "output",
"bank_direction_high": "output"
},
{
"type": "urukul",
"dds": "ad9910",
"ports": [3, 4],
"clk_sel": 2
},
{
"type": "zotino",
"ports": [5]
},
{
"type": "sampler",
"ports": [6, 7]
},
{
"type": "mirny",
"ports": [8],
"clk_sel": 1,
"refclk": 125e6
},
{
"type": "fastino",
"ports": [9]
},
{
"type": "dio",
"ports": [10],
"bank_direction_low": "input",
"bank_direction_high": "input"
},
{
"type": "dio",
"ports": [11],
"bank_direction_low": "output",
"bank_direction_high": "input"
}
]
}

25
local_run.sh
View File

@ -13,10 +13,9 @@ fi
impure=0 impure=0
load_bitstream=1 load_bitstream=1
board_type="kasli_soc" board_host="192.168.1.52"
fw_type="runtime"
while getopts "ilb:t:f:" opt; do while getopts "ilb:" opt; do
case "$opt" in case "$opt" in
\?) exit 1 \?) exit 1
;; ;;
@ -26,36 +25,24 @@ while getopts "ilb:t:f:" opt; do
;; ;;
b) board_host=$OPTARG b) board_host=$OPTARG
;; ;;
t) board_type=$OPTARG
;;
f) fw_type=$OPTARG
;;
esac esac
done done
if [ -z "$board_host" ]; then
case $board_type in
kasli_soc) board_host="192.168.1.56";;
zc706) board_host="192.168.1.52";;
*) echo "Unknown board type"; exit 1;;
esac
fi
load_bitstream_cmd="" load_bitstream_cmd=""
build_dir=`pwd`/build build_dir=`pwd`/build
result_dir=`pwd`/result result_dir=`pwd`/result
cd $OPENOCD_ZYNQ cd $OPENOCD_ZYNQ
openocd -f $board_type.cfg -c "load_image $SZL/szl-$board_type.elf; resume 0; exit" openocd -f zc706.cfg -c "load_image $SZL; resume 0; exit"
sleep 5 sleep 5
if [ $impure -eq 1 ]; then if [ $impure -eq 1 ]; then
if [ $load_bitstream -eq 1 ]; then if [ $load_bitstream -eq 1 ]; then
load_bitstream_cmd="-g $build_dir/gateware/top.bit" load_bitstream_cmd="-g $build_dir/gateware/top.bit"
fi fi
artiq_netboot $load_bitstream_cmd -f $build_dir/$fw_type.bin -b $board_host artiq_netboot $load_bitstream_cmd -f $build_dir/runtime.bin -b $board_host
else else
if [ $load_bitstream -eq 1 ]; then if [ $load_bitstream -eq 1 ]; then
load_bitstream_cmd="-g $result_dir/top.bit" load_bitstream_cmd="-g $result_dir/top.bit"
fi fi
artiq_netboot $load_bitstream_cmd -f $result_dir/$fw_type.bin -b $board_host artiq_netboot $load_bitstream_cmd -f $result_dir/runtime.bin -b $board_host
fi fi

13
remote_run.sh
View File

@ -1,7 +1,5 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Only ZC706 supported for now.
set -e set -e
if [ -z "$OPENOCD_ZYNQ" ]; then if [ -z "$OPENOCD_ZYNQ" ]; then
@ -20,9 +18,8 @@ impure_dir="build"
sshopts="" sshopts=""
load_bitstream=1 load_bitstream=1
board_host="192.168.1.52" board_host="192.168.1.52"
fw_type="runtime"
while getopts "h:id:o:lt:" opt; do while getopts "h:id:o:l" opt; do
case "$opt" in case "$opt" in
\?) exit 1 \?) exit 1
;; ;;
@ -39,8 +36,6 @@ while getopts "h:id:o:lt:" opt; do
;; ;;
b) board_host=$OPTARG b) board_host=$OPTARG
;; ;;
t) fw_type=$OPTARG
;;
esac esac
done done
@ -51,17 +46,17 @@ echo "Creating $target_folder..."
ssh $sshopts $target_host "mkdir -p $target_folder" ssh $sshopts $target_host "mkdir -p $target_folder"
echo "Copying files..." echo "Copying files..."
rsync -e "ssh $sshopts" -Lc $OPENOCD_ZYNQ/* $target_host:$target_folder rsync -e "ssh $sshopts" -Lc $OPENOCD_ZYNQ/* $target_host:$target_folder
rsync -e "ssh $sshopts" -Lc $SZL/szl-zc706.elf $target_host:$target_folder/szl.elf rsync -e "ssh $sshopts" -Lc $SZL $target_host:$target_folder
if [ $impure -eq 1 ]; then if [ $impure -eq 1 ]; then
if [ $load_bitstream -eq 1 ]; then if [ $load_bitstream -eq 1 ]; then
load_bitstream_cmd="-g build/gateware/top.bit" load_bitstream_cmd="-g build/gateware/top.bit"
fi fi
firmware="build/$fw_type.bin" firmware="build/runtime.bin"
else else
if [ $load_bitstream -eq 1 ]; then if [ $load_bitstream -eq 1 ]; then
load_bitstream_cmd="-g $pure_dir/top.bit" load_bitstream_cmd="-g $pure_dir/top.bit"
fi fi
firmware="$pure_dir/$fw_type.bin" firmware="$pure_dir/runtime.bin"
fi fi
echo "Programming board..." echo "Programming board..."
ssh $sshopts $target_host "cd $target_folder; openocd -f zc706.cfg -c'load_image szl.elf; resume 0; exit'" ssh $sshopts $target_host "cd $target_folder; openocd -f zc706.cfg -c'load_image szl.elf; resume 0; exit'"

36
shell.nix Normal file
View File

@ -0,0 +1,36 @@
let
zynq-rs = (import ./zynq-rs.nix);
pkgs = import <nixpkgs> { overlays = [ (import "${zynq-rs}/nix/mozilla-overlay.nix") ]; };
rustPlatform = (import "${zynq-rs}/nix/rust-platform.nix" { inherit pkgs; });
cargo-xbuild = (import zynq-rs).cargo-xbuild;
artiq-fast = <artiq-fast>;
artiqpkgs = import "${artiq-fast}/default.nix" { inherit pkgs; };
vivado = import "${artiq-fast}/vivado.nix" { inherit pkgs; };
zc706-szl = (import zynq-rs).zc706-szl;
in
pkgs.stdenv.mkDerivation {
name = "artiq-zynq-env";
buildInputs = [
pkgs.gnumake
rustPlatform.rust.rustc
rustPlatform.rust.cargo
pkgs.llvmPackages_9.llvm
pkgs.llvmPackages_9.clang-unwrapped
pkgs.cacert
cargo-xbuild
pkgs.openocd
pkgs.openssh pkgs.rsync
(pkgs.python3.withPackages(ps: (with artiqpkgs; [ migen migen-axi misoc artiq artiq-netboot ])))
vivado
artiqpkgs.binutils-arm
(import "${zynq-rs}/nix/mkbootimage.nix" { inherit pkgs; })
];
XARGO_RUST_SRC = "${rustPlatform.rust.rustc}/lib/rustlib/src/rust/library";
CLANG_EXTRA_INCLUDE_DIR = "${pkgs.llvmPackages_9.clang-unwrapped.lib}/lib/clang/9.0.1/include";
OPENOCD_ZYNQ = "${zynq-rs}/openocd";
SZL = "${zc706-szl}/szl.elf";
}

193
src/Cargo.lock generated
View File

@ -1,7 +1,5 @@
# This file is automatically @generated by Cargo. # This file is automatically @generated by Cargo.
# It is not intended for manual editing. # It is not intended for manual editing.
version = 3
[[package]] [[package]]
name = "async-recursion" name = "async-recursion"
version = "0.3.2" version = "0.3.2"
@ -15,9 +13,9 @@ dependencies = [
[[package]] [[package]]
name = "autocfg" name = "autocfg"
version = "1.1.0" version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa" checksum = "cdb031dd78e28731d87d56cc8ffef4a8f36ca26c38fe2de700543e627f8a464a"
[[package]] [[package]]
name = "bit_field" name = "bit_field"
@ -27,19 +25,9 @@ checksum = "dcb6dd1c2376d2e096796e234a70e17e94cc2d5d54ff8ce42b28cef1d0d359a4"
[[package]] [[package]]
name = "bitflags" name = "bitflags"
version = "1.3.2" version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" checksum = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
[[package]]
name = "build_const"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ae4235e6dac0694637c763029ecea1a2ec9e4e06ec2729bd21ba4d9c863eb7"
[[package]]
name = "build_zynq"
version = "0.0.0"
[[package]] [[package]]
name = "byteorder" name = "byteorder"
@ -49,9 +37,9 @@ checksum = "14c189c53d098945499cdfa7ecc63567cf3886b3332b312a5b4585d8d3a6a610"
[[package]] [[package]]
name = "cc" name = "cc"
version = "1.0.77" version = "1.0.69"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9f73505338f7d905b19d18738976aae232eb46b8efc15554ffc56deb5d9ebe4" checksum = "e70cc2f62c6ce1868963827bd677764c62d07c3d9a3e1fb1177ee1a9ab199eb2"
[[package]] [[package]]
name = "cfg-if" name = "cfg-if"
@ -80,15 +68,6 @@ dependencies = [
"rustc_version", "rustc_version",
] ]
[[package]]
name = "crc"
version = "1.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d663548de7f5cca343f1e0a48d14dcfb0e9eb4e079ec58883b7251539fa10aeb"
dependencies = [
"build_const",
]
[[package]] [[package]]
name = "cslice" name = "cslice"
version = "0.3.0" version = "0.3.0"
@ -101,7 +80,6 @@ version = "0.0.0"
dependencies = [ dependencies = [
"cfg-if 0.1.10", "cfg-if 0.1.10",
"compiler_builtins", "compiler_builtins",
"cslice",
"libc", "libc",
"unwind", "unwind",
] ]
@ -116,9 +94,9 @@ dependencies = [
[[package]] [[package]]
name = "embedded-hal" name = "embedded-hal"
version = "0.2.7" version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35949884794ad573cf46071e41c9b60efb0cb311e3ca01f7af807af1debc66ff" checksum = "db184d3fa27bc7a2344250394c0264144dfe0bc81a4401801dcb964b8dd172ad"
dependencies = [ dependencies = [
"nb 0.1.3", "nb 0.1.3",
"void", "void",
@ -138,9 +116,9 @@ dependencies = [
[[package]] [[package]]
name = "futures" name = "futures"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "38390104763dc37a5145a53c29c63c1290b5d316d6086ec32c293f6736051bb0" checksum = "1adc00f486adfc9ce99f77d717836f0c5aa84965eb0b4f051f4e83f7cab53f8b"
dependencies = [ dependencies = [
"futures-channel", "futures-channel",
"futures-core", "futures-core",
@ -152,9 +130,9 @@ dependencies = [
[[package]] [[package]]
name = "futures-channel" name = "futures-channel"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52ba265a92256105f45b719605a571ffe2d1f0fea3807304b522c1d778f79eed" checksum = "74ed2411805f6e4e3d9bc904c95d5d423b89b3b25dc0250aa74729de20629ff9"
dependencies = [ dependencies = [
"futures-core", "futures-core",
"futures-sink", "futures-sink",
@ -162,22 +140,24 @@ dependencies = [
[[package]] [[package]]
name = "futures-core" name = "futures-core"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04909a7a7e4633ae6c4a9ab280aeb86da1236243a77b694a49eacd659a4bd3ac" checksum = "af51b1b4a7fdff033703db39de8802c673eb91855f2e0d47dcf3bf2c0ef01f99"
[[package]] [[package]]
name = "futures-io" name = "futures-io"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00f5fb52a06bdcadeb54e8d3671f8888a39697dcb0b81b23b55174030427f4eb" checksum = "0b0e06c393068f3a6ef246c75cdca793d6a46347e75286933e5e75fd2fd11582"
[[package]] [[package]]
name = "futures-macro" name = "futures-macro"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bdfb8ce053d86b91919aad980c220b1fb8401a9394410e1c289ed7e66b61835d" checksum = "c54913bae956fb8df7f4dc6fc90362aa72e69148e3f39041fbe8742d21e0ac57"
dependencies = [ dependencies = [
"autocfg",
"proc-macro-hack",
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn", "syn",
@ -185,43 +165,37 @@ dependencies = [
[[package]] [[package]]
name = "futures-sink" name = "futures-sink"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39c15cf1a4aa79df40f1bb462fb39676d0ad9e366c2a33b590d7c66f4f81fcf9" checksum = "c0f30aaa67363d119812743aa5f33c201a7a66329f97d1a887022971feea4b53"
[[package]] [[package]]
name = "futures-task" name = "futures-task"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2ffb393ac5d9a6eaa9d3fdf37ae2776656b706e200c8e16b1bdb227f5198e6ea" checksum = "bbe54a98670017f3be909561f6ad13e810d9a51f3f061b902062ca3da80799f2"
[[package]] [[package]]
name = "futures-util" name = "futures-util"
version = "0.3.25" version = "0.3.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "197676987abd2f9cadff84926f410af1c183608d36641465df73ae8211dc65d6" checksum = "67eb846bfd58e44a8481a00049e82c43e0ccb5d61f8dc071057cb19249dd4d78"
dependencies = [ dependencies = [
"autocfg",
"futures-core", "futures-core",
"futures-macro", "futures-macro",
"futures-sink", "futures-sink",
"futures-task", "futures-task",
"pin-project-lite", "pin-project-lite",
"pin-utils", "pin-utils",
] "proc-macro-hack",
"proc-macro-nested",
[[package]]
name = "io"
version = "0.0.0"
dependencies = [
"byteorder",
"core_io",
"libsupport_zynq",
] ]
[[package]] [[package]]
name = "libasync" name = "libasync"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"embedded-hal", "embedded-hal",
"libcortex_a9", "libcortex_a9",
@ -230,30 +204,10 @@ dependencies = [
"smoltcp", "smoltcp",
] ]
[[package]]
name = "libboard_artiq"
version = "0.0.0"
dependencies = [
"build_zynq",
"core_io",
"crc",
"embedded-hal",
"io",
"libasync",
"libboard_zynq",
"libconfig",
"libcortex_a9",
"libregister",
"log",
"log_buffer",
"nb 1.0.0",
"void",
]
[[package]] [[package]]
name = "libboard_zynq" name = "libboard_zynq"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"embedded-hal", "embedded-hal",
@ -278,7 +232,7 @@ dependencies = [
[[package]] [[package]]
name = "libconfig" name = "libconfig"
version = "0.1.0" version = "0.1.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"core_io", "core_io",
"fatfs", "fatfs",
@ -289,7 +243,7 @@ dependencies = [
[[package]] [[package]]
name = "libcortex_a9" name = "libcortex_a9"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"libregister", "libregister",
@ -298,14 +252,14 @@ dependencies = [
[[package]] [[package]]
name = "libm" name = "libm"
version = "0.2.6" version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "348108ab3fba42ec82ff6e9564fc4ca0247bdccdc68dd8af9764bbc79c3c8ffb" checksum = "c7d73b3f436185384286bd8098d17ec07c9a7d2388a6599f824d8502b529702a"
[[package]] [[package]]
name = "libregister" name = "libregister"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"bit_field", "bit_field",
"vcell", "vcell",
@ -315,7 +269,7 @@ dependencies = [
[[package]] [[package]]
name = "libsupport_zynq" name = "libsupport_zynq"
version = "0.0.0" version = "0.0.0"
source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#67dbb5932fa8ff5f143983476f741f945871d286" source = "git+https://git.m-labs.hk/M-Labs/zynq-rs.git#2c161720fa12f8b7abecaf60f77b062b08ac9bc1"
dependencies = [ dependencies = [
"cc", "cc",
"compiler_builtins", "compiler_builtins",
@ -334,9 +288,9 @@ checksum = "822add9edb1860698b79522510da17bef885171f75aa395cff099d770c609c24"
[[package]] [[package]]
name = "log" name = "log"
version = "0.4.17" version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "abb12e687cfb44aa40f41fc3978ef76448f9b6038cad6aef4259d3c095a2382e" checksum = "51b9bbe6c47d51fc3e1a9b945965946b4c44142ab8792c50835a980d362c2710"
dependencies = [ dependencies = [
"cfg-if 1.0.0", "cfg-if 1.0.0",
] ]
@ -381,18 +335,18 @@ dependencies = [
[[package]] [[package]]
name = "num-traits" name = "num-traits"
version = "0.2.15" version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "578ede34cf02f8924ab9447f50c28075b4d3e5b269972345e7e0372b38c6cdcd" checksum = "9a64b1ec5cda2586e284722486d802acf1f7dbdc623e2bfc57e65ca1cd099290"
dependencies = [ dependencies = [
"autocfg", "autocfg",
] ]
[[package]] [[package]]
name = "pin-project-lite" name = "pin-project-lite"
version = "0.2.9" version = "0.2.7"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e0a7ae3ac2f1173085d398531c705756c94a4c56843785df85a60c1a0afac116" checksum = "8d31d11c69a6b52a174b42bdc0c30e5e11670f90788b2c471c31c1d17d449443"
[[package]] [[package]]
name = "pin-utils" name = "pin-utils"
@ -401,19 +355,31 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]] [[package]]
name = "proc-macro2" name = "proc-macro-hack"
version = "1.0.43" version = "0.5.19"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0a2ca2c61bc9f3d74d2886294ab7b9853abd9c1ad903a3ac7815c58989bb7bab" checksum = "dbf0c48bc1d91375ae5c3cd81e3722dff1abcf81a30960240640d223f59fe0e5"
[[package]]
name = "proc-macro-nested"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bc881b2c22681370c6a780e47af9840ef841837bc98118431d4e1868bd0c1086"
[[package]]
name = "proc-macro2"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c7ed8b8c7b886ea3ed7dde405212185f423ab44682667c8c6dd14aa1d9f6612"
dependencies = [ dependencies = [
"unicode-ident", "unicode-xid",
] ]
[[package]] [[package]]
name = "quote" name = "quote"
version = "1.0.21" version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbe448f377a7d6961e30f5955f9b8d106c3f5e449d493ee1b125c1d43c2b5179" checksum = "c3d0b9745dc2debf507c8422de05d7226cc1f0644216dfdfead988f9b1ab32a7"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
] ]
@ -429,7 +395,6 @@ name = "runtime"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"async-recursion", "async-recursion",
"build_zynq",
"byteorder", "byteorder",
"core_io", "core_io",
"cslice", "cslice",
@ -437,9 +402,7 @@ dependencies = [
"dyld", "dyld",
"embedded-hal", "embedded-hal",
"futures", "futures",
"io",
"libasync", "libasync",
"libboard_artiq",
"libboard_zynq", "libboard_zynq",
"libc", "libc",
"libconfig", "libconfig",
@ -466,24 +429,6 @@ dependencies = [
"semver", "semver",
] ]
[[package]]
name = "satman"
version = "0.0.0"
dependencies = [
"build_zynq",
"embedded-hal",
"libasync",
"libboard_artiq",
"libboard_zynq",
"libc",
"libconfig",
"libcortex_a9",
"libregister",
"libsupport_zynq",
"log",
"unwind",
]
[[package]] [[package]]
name = "semver" name = "semver"
version = "0.1.20" version = "0.1.20"
@ -503,20 +448,20 @@ dependencies = [
[[package]] [[package]]
name = "syn" name = "syn"
version = "1.0.101" version = "1.0.74"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e90cde112c4b9690b8cbe810cba9ddd8bc1d7472e2cae317b69e9438c1cba7d2" checksum = "1873d832550d4588c3dbc20f01361ab00bfe741048f71e3fecf145a7cc18b29c"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"unicode-ident", "unicode-xid",
] ]
[[package]] [[package]]
name = "unicode-ident" name = "unicode-xid"
version = "1.0.5" version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ceab39d59e4c9499d4e5a8ee0e2735b891bb7308ac83dfb4e80cad195c9f6f3" checksum = "8ccb82d61f80a663efe1f787a51b16b5a51e3314d6ac365b08639f52387b33f3"
[[package]] [[package]]
name = "unwind" name = "unwind"
@ -542,9 +487,9 @@ checksum = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d"
[[package]] [[package]]
name = "volatile-register" name = "volatile-register"
version = "0.2.1" version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ee8f19f9d74293faf70901bc20ad067dc1ad390d2cbf1e3f75f721ffee908b6" checksum = "0d67cb4616d99b940db1d6bd28844ff97108b498a6ca850e5b6191a532063286"
dependencies = [ dependencies = [
"vcell", "vcell",
] ]

2
src/Cargo.toml
View File

@ -3,10 +3,8 @@ members = [
"libc", "libc",
"libdyld", "libdyld",
"libdwarf", "libdwarf",
"libio",
"libunwind", "libunwind",
"runtime", "runtime",
"satman"
] ]
[profile.release] [profile.release]

25
src/Makefile
View File

@ -1,19 +1,16 @@
TARGET := zc706 TARGET := zc706
GWARGS := -V nist_clock VARIANT := simple
all: runtime all: ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin
runtime: ../build/runtime.bin .PHONY: all
satman: ../build/satman.bin
.PHONY: all runtime_target satman_target ../build/pl.rs ../build/rustc-cfg: gateware/*
../build/pl.rs ../build/rustc-cfg ../build/mem.rs: gateware/*
mkdir -p ../build mkdir -p ../build
python gateware/$(TARGET).py -r ../build/pl.rs -c ../build/rustc-cfg -m ../build/mem.rs $(GWARGS) python gateware/$(TARGET).py -r ../build/pl.rs -c ../build/rustc-cfg -V $(VARIANT)
../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print) ../build/firmware/armv7-none-eabihf/release/runtime: ../build/pl.rs ../build/rustc-cfg $(shell find . -print)
cd runtime && \ cd runtime && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \ XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \ cargo xbuild --release \
@ -22,13 +19,3 @@ satman: ../build/satman.bin
../build/runtime.bin: ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin: ../build/firmware/armv7-none-eabihf/release/runtime
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/runtime ../build/runtime.bin
../build/firmware/armv7-none-eabihf/release/satman: ../build/pl.rs ../build/rustc-cfg ../build/mem.rs $(shell find . -type f -print)
cd satman && \
XBUILD_SYSROOT_PATH=`pwd`/../../build/sysroot \
cargo xbuild --release \
--target-dir ../../build/firmware \
--no-default-features --features=target_$(TARGET)
../build/satman.bin: ../build/firmware/armv7-none-eabihf/release/satman
llvm-objcopy -O binary ../build/firmware/armv7-none-eabihf/release/satman ../build/satman.bin

191
src/gateware/coraz7.py Executable file
View File

@ -0,0 +1,191 @@
#!/usr/bin/env python
import argparse
from operator import itemgetter
from migen import *
from migen.build.generic_platform import *
from migen.genlib.resetsync import AsyncResetSynchronizer
from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import coraz7
from misoc.interconnect.csr import *
from misoc.integration import cpu_interface
from artiq.gateware import rtio
from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2
import dma
import analyzer
import acpki
class RTIOCRG(Module, AutoCSR):
def __init__(self, platform, rtio_internal_clk):
self.clock_sel = CSRStorage()
self.pll_reset = CSRStorage(reset=1)
self.pll_locked = CSRStatus()
self.clock_domains.cd_rtio = ClockDomain()
self.clock_domains.cd_rtiox4 = ClockDomain(reset_less=True)
rtio_external_clk = Signal()
# user_sma_clock = platform.request("user_sma_clock")
# platform.add_period_constraint(user_sma_clock.p, 8.0)
# self.specials += Instance("IBUFDS",
# i_I=user_sma_clock.p, i_IB=user_sma_clock.n,
# o_O=rtio_external_clk)
pll_locked = Signal()
rtio_clk = Signal()
rtiox4_clk = Signal()
self.specials += [
Instance("PLLE2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
p_REF_JITTER1=0.01,
p_CLKIN1_PERIOD=8.0, p_CLKIN2_PERIOD=8.0,
i_CLKIN1=rtio_internal_clk, i_CLKIN2=rtio_external_clk,
# Warning: CLKINSEL=0 means CLKIN2 is selected
i_CLKINSEL=~self.clock_sel.storage,
# VCO @ 1GHz when using 125MHz input
p_CLKFBOUT_MULT=8, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=self.cd_rtio.clk,
i_RST=self.pll_reset.storage,
o_CLKFBOUT=rtio_clk,
p_CLKOUT0_DIVIDE=2, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=rtiox4_clk),
Instance("BUFG", i_I=rtio_clk, o_O=self.cd_rtio.clk),
Instance("BUFG", i_I=rtiox4_clk, o_O=self.cd_rtiox4.clk),
AsyncResetSynchronizer(self.cd_rtio, ~pll_locked),
MultiReg(pll_locked, self.pll_locked.status)
]
class CoraZ7(SoCCore):
def __init__(self, device_variant="10", acpki=False):
self.acpki = acpki
self.rustc_cfg = dict()
platform = coraz7.Platform(device_variant=device_variant)
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = self.__class__.__name__
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident)
platform.add_platform_command("create_clock -name clk_fpga_0 -period 8 [get_pins \"PS7/FCLKCLK[0]\"]")
platform.add_platform_command("set_input_jitter clk_fpga_0 0.24")
self.submodules.rtio_crg = RTIOCRG(self.platform, self.ps7.cd_sys.clk)
self.csr_devices.append("rtio_crg")
self.platform.add_period_constraint(self.rtio_crg.cd_rtio.clk, 8.)
self.platform.add_false_path_constraints(
self.ps7.cd_sys.clk,
self.rtio_crg.cd_rtio.clk)
def add_rtio(self, rtio_channels):
self.submodules.rtio_tsc = rtio.TSC("async", glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri])
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.rtio_core.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
class Simple(CoraZ7):
def __init__(self, **kwargs):
CoraZ7.__init__(self, **kwargs)
platform = self.platform
rtio_channels = []
for i in range(2):
phy = ttl_simple.Output(platform.request("user_led", i))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels)
rtio_channels.append(rtio.LogChannel())
self.add_rtio(rtio_channels)
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main():
parser = argparse.ArgumentParser(
description="ARTIQ port to the Cora Z7 Zynq development kit")
parser.add_argument("-r", default=None,
help="build Rust interface into the specified file")
parser.add_argument("-c", default=None,
help="build Rust compiler configuration into the specified file")
parser.add_argument("-g", default=None,
help="build gateware into the specified directory")
parser.add_argument("-V", "--variant", default="10",
help="variant: "
"[acpki_]10/07s "
"(default: %(default)s)")
args = parser.parse_args()
variant = args.variant.lower()
acpki = variant.startswith("acpki_")
if acpki:
variant = variant[6:]
try:
soc = Simple(device_variant=variant, acpki=acpki)
except KeyError:
raise SystemExit("Invalid variant (-V/--variant)")
soc.finalize()
if args.r is not None:
write_csr_file(soc, args.r)
if args.c is not None:
write_rustc_cfg_file(soc, args.c)
if args.g is not None:
soc.build(build_dir=args.g)
if __name__ == "__main__":
main()

85
src/gateware/drtio_aux_controller.py
View File

@ -1,85 +0,0 @@
"""Auxiliary controller, common to satellite and master"""
from artiq.gateware.drtio.aux_controller import Transmitter, Receiver
from migen.fhdl.simplify import FullMemoryWE
from misoc.interconnect.csr import *
from migen_axi.interconnect.sram import SRAM
from migen_axi.interconnect import axi
max_packet = 1024
class _DRTIOAuxControllerBase(Module):
def __init__(self, link_layer):
self.bus = axi.Interface()
self.submodules.transmitter = Transmitter(link_layer, len(self.bus.w.data))
self.submodules.receiver = Receiver(link_layer, len(self.bus.w.data))
def get_csrs(self):
return self.transmitter.get_csrs() + self.receiver.get_csrs()
# TODO: FullMemoryWE should be applied by migen.build
@FullMemoryWE()
class DRTIOAuxControllerAxi(_DRTIOAuxControllerBase):
def __init__(self, link_layer):
_DRTIOAuxControllerBase.__init__(self, link_layer)
tx_sdram_if = SRAM(self.transmitter.mem, read_only=False)
rx_sdram_if = SRAM(self.receiver.mem, read_only=True)
aw_decoder = axi.AddressDecoder(self.bus.aw,
[(lambda a: a[log2_int(max_packet)] == 0, tx_sdram_if.bus.aw),
(lambda a: a[log2_int(max_packet)] == 1, rx_sdram_if.bus.aw)],
register=True)
ar_decoder = axi.AddressDecoder(self.bus.ar,
[(lambda a: a[log2_int(max_packet)] == 0, tx_sdram_if.bus.ar),
(lambda a: a[log2_int(max_packet)] == 1, rx_sdram_if.bus.ar)],
register=True)
# unlike wb, axi address decoder only connects ar/aw lanes,
# the rest must also be connected!
# not quite unlike an address decoder itself.
# connect bus.b with tx.b
self.comb += [tx_sdram_if.bus.b.ready.eq(self.bus.b.ready),
self.bus.b.id.eq(tx_sdram_if.bus.b.id),
self.bus.b.resp.eq(tx_sdram_if.bus.b.resp),
self.bus.b.valid.eq(tx_sdram_if.bus.b.valid)]
# connect bus.w with tx.w
# no worries about w.valid and slave sel here, only tx will be written to
self.comb += [tx_sdram_if.bus.w.id.eq(self.bus.w.id),
tx_sdram_if.bus.w.data.eq(self.bus.w.data),
tx_sdram_if.bus.w.strb.eq(self.bus.w.strb),
tx_sdram_if.bus.w.last.eq(self.bus.w.last),
tx_sdram_if.bus.w.valid.eq(self.bus.w.valid),
self.bus.w.ready.eq(tx_sdram_if.bus.w.ready)]
# connect bus.r with rx.r and tx.r w/o data
self.comb += [self.bus.r.id.eq(rx_sdram_if.bus.r.id | tx_sdram_if.bus.r.id),
#self.bus.r.data.eq(rx_sdram_if.bus.r.data | tx_sdram_if.bus.r.data),
self.bus.r.resp.eq(rx_sdram_if.bus.r.resp | tx_sdram_if.bus.r.resp),
self.bus.r.last.eq(rx_sdram_if.bus.r.last | tx_sdram_if.bus.r.last),
self.bus.r.valid.eq(rx_sdram_if.bus.r.valid | tx_sdram_if.bus.r.valid),
rx_sdram_if.bus.r.ready.eq(self.bus.r.ready),
tx_sdram_if.bus.r.ready.eq(self.bus.r.ready)]
# connect read data after being masked
masked = [Replicate(rx_sdram_if.bus.r.valid,
len(self.bus.r.data)
) & rx_sdram_if.bus.r.data,
Replicate(tx_sdram_if.bus.r.valid,
len(self.bus.r.data)
) & tx_sdram_if.bus.r.data]
self.comb += self.bus.r.data.eq(reduce(or_, masked))
self.submodules += tx_sdram_if, rx_sdram_if, aw_decoder, ar_decoder
@FullMemoryWE()
class DRTIOAuxControllerBare(_DRTIOAuxControllerBase):
# Barebones version of the AuxController. No SRAM, no decoders.
# add memories manually from tx and rx in target code.
def get_tx_port(self):
return self.transmitter.mem.get_port(write_capable=True)
def get_rx_port(self):
return self.receiver.mem.get_port(write_capable=False)
def get_mem_size(self):
return max_packet

529
src/gateware/kasli_soc.py
View File

@ -1,529 +0,0 @@
#!/usr/bin/env python
import argparse
from operator import itemgetter
from migen import *
from migen.build.generic_platform import *
from migen.genlib.resetsync import AsyncResetSynchronizer
from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import kasli_soc
from misoc.interconnect.csr import *
from misoc.integration import cpu_interface
from artiq.coredevice import jsondesc
from artiq.gateware import rtio, eem_7series
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from artiq.gateware.rtio.phy import ttl_simple
from artiq.gateware.drtio.transceiver import gtx_7series
from artiq.gateware.drtio.siphaser import SiPhaser7Series
from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer
from artiq.gateware.drtio import *
import dma
import analyzer
import acpki
import drtio_aux_controller
import zynq_clocking
eem_iostandard_dict = {
0: "LVDS_25",
1: "LVDS_25",
2: "LVDS",
3: "LVDS",
4: "LVDS",
5: "LVDS",
6: "LVDS",
7: "LVDS",
8: "LVDS_25",
9: "LVDS_25",
10: "LVDS",
11: "LVDS",
}
def eem_iostandard(eem):
return IOStandard(eem_iostandard_dict[eem])
class SMAClkinForward(Module):
def __init__(self, platform):
sma_clkin = platform.request("sma_clkin")
sma_clkin_se = Signal()
cdr_clk_se = Signal()
cdr_clk = platform.request("cdr_clk")
self.specials += [
Instance("IBUFDS", i_I=sma_clkin.p, i_IB=sma_clkin.n, o_O=sma_clkin_se),
Instance("ODDR", i_C=sma_clkin_se, i_CE=1, i_D1=1, i_D2=0, o_Q=cdr_clk_se),
Instance("OBUFDS", i_I=cdr_clk_se, o_O=cdr_clk.p, o_OB=cdr_clk.n)
]
class GTP125BootstrapClock(Module):
def __init__(self, platform):
self.clock_domains.cd_bootstrap = ClockDomain(reset_less=True)
self.cd_bootstrap.clk.attr.add("keep")
bootstrap_125 = platform.request("clk125_gtp")
bootstrap_se = Signal()
platform.add_period_constraint(bootstrap_125.p, 8.0)
self.specials += [
Instance("IBUFDS_GTE2",
p_CLKSWING_CFG="0b11",
i_CEB=0,
i_I=bootstrap_125.p, i_IB=bootstrap_125.n, o_O=bootstrap_se),
Instance("BUFG", i_I=bootstrap_se, o_O=self.cd_bootstrap.clk)
]
class GenericStandalone(SoCCore):
def __init__(self, description, acpki=False):
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = description["variant"]
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.submodules += SMAClkinForward(self.platform)
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_soft_reset"] = None
clk_synth = platform.request("cdr_clk_clean_fabric")
clk_synth_se = Signal()
platform.add_period_constraint(clk_synth.p, 8.0)
self.specials += Instance("IBUFGDS",
p_DIFF_TERM="TRUE", p_IBUF_LOW_PWR="FALSE",
i_I=clk_synth.p, i_IB=clk_synth.n, o_O=clk_synth_se)
fix_serdes_timing_path(platform)
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, clk_synth_se)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys
self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_grabber:
self.grabber_csr_group = []
eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard)
for i in (0, 1):
print("USER LED at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
user_led = self.platform.request("user_led", i)
phy = ttl_simple.Output(user_led)
self.submodules += phy
self.rtio_channels.append(rtio.Channel.from_phy(phy))
self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri])
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.rtio_core.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
if has_grabber:
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
for grabber in self.grabber_csr_group:
self.platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, getattr(self, grabber).deserializer.cd_cl.clk)
class GenericMaster(SoCCore):
def __init__(self, description, acpki=False):
clk_freq = description["rtio_frequency"]
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = description["variant"]
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
self.submodules += SMAClkinForward(self.platform)
data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
txout_buf,
clk_sw=gtx0.tx_init.done)
self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
fix_serdes_timing_path(platform)
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_soft_reset"] = None
self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_grabber:
self.grabber_csr_group = []
eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard)
for i in (0, 1):
print("USER LED at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
user_led = self.platform.request("user_led", i)
phy = ttl_simple.Output(user_led)
self.submodules += phy
self.rtio_channels.append(rtio.Channel.from_phy(phy))
self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
drtio_csr_group = []
drtioaux_csr_group = []
drtioaux_memory_group = []
self.drtio_cri = []
for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtio_csr_group.append(core_name)
drtioaux_csr_group.append(coreaux_name)
drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster(self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
size = coreaux.get_mem_size()
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), size)
self.axi2csr.register_port(coreaux.get_rx_port(), size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, size * 2)
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, self.rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.rtio_core.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
if has_grabber:
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
class GenericSatellite(SoCCore):
def __init__(self, description, acpki=False):
clk_freq = description["rtio_frequency"]
self.acpki = acpki
self.rustc_cfg = dict()
platform = kasli_soc.Platform()
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = description["variant"]
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
data_pads = [platform.request("sfp", i) for i in range(4)]
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("clk_gtp"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = GTP125BootstrapClock(self.platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
txout_buf,
clk_sw=gtx0.tx_init.done)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
self.crg = self.ps7 # HACK for eem_7series to find the clock
self.crg.cd_sys = self.sys_crg.cd_sys
fix_serdes_timing_path(platform)
self.rtio_channels = []
has_grabber = any(peripheral["type"] == "grabber" for peripheral in description["peripherals"])
if has_grabber:
self.grabber_csr_group = []
eem_7series.add_peripherals(self, description["peripherals"], iostandard=eem_iostandard)
for i in (0, 1):
print("USER LED at RTIO channel 0x{:06x}".format(len(self.rtio_channels)))
user_led = self.platform.request("user_led", i)
phy = ttl_simple.Output(user_led)
self.submodules += phy
self.rtio_channels.append(rtio.Channel.from_phy(phy))
self.config["RTIO_LOG_CHANNEL"] = len(self.rtio_channels)
self.rtio_channels.append(rtio.LogChannel())
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
drtioaux_csr_group = []
drtioaux_memory_group = []
drtiorep_csr_group = []
self.drtio_cri = []
for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name)
drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite(
self.rtio_tsc, self.drtio_transceiver.channels[i],
self.rx_synchronizer))
self.submodules.drtiosat = core
self.csr_devices.append("drtiosat")
else:
corerep_name = "drtiorep" + str(i-1)
drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater(
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
mem_size = coreaux.get_mem_size()
tx_port = coreaux.get_tx_port()
rx_port = coreaux.get_rx_port()
memory_address = self.axi2csr.register_port(tx_port, mem_size)
# rcv in upper half of the memory, thus added second
self.axi2csr.register_port(rx_port, mem_size)
# and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.add_csr_group("drtiorep", drtiorep_csr_group)
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.local_io = SyncRTIO(self.rtio_tsc, self.rtio_channels)
self.comb += self.drtiosat.async_errors.eq(self.local_io.async_errors)
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri],
[self.local_io.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table")
self.submodules.rtio_moninj = rtio.MonInj(self.rtio_channels)
self.csr_devices.append("rtio_moninj")
rtio_clk_period = 1e9/clk_freq
self.rustc_cfg["rtio_frequency"] = str(clk_freq/1e6)
self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("cdr_clk"),
rx_synchronizer=self.rx_synchronizer,
ultrascale=False,
rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
self.csr_devices.append("siphaser")
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["has_siphaser"] = None
self.rustc_cfg["si5324_soft_reset"] = None
gtx0 = self.drtio_transceiver.gtxs[0]
platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk)
if has_grabber:
self.rustc_cfg["has_grabber"] = None
self.add_csr_group("grabber", self.grabber_csr_group)
# no RTIO CRG here
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main():
parser = argparse.ArgumentParser(
description="ARTIQ device binary builder for generic Kasli-SoC systems")
parser.add_argument("-r", default=None,
help="build Rust interface into the specified file")
parser.add_argument("-c", default=None,
help="build Rust compiler configuration into the specified file")
parser.add_argument("-m", default=None,
help="build Rust memory interface into the specified file")
parser.add_argument("-g", default=None,
help="build gateware into the specified directory")
parser.add_argument("--acpki", default=False, action="store_true",
help="enable ACPKI")
parser.add_argument("description", metavar="DESCRIPTION",
help="JSON system description file")
args = parser.parse_args()
description = jsondesc.load(args.description)
if description["target"] != "kasli_soc":
raise ValueError("Description is for a different target")
if description["base"] == "standalone":
cls = GenericStandalone
elif description["base"] == "master":
cls = GenericMaster
elif description["base"] == "satellite":
cls = GenericSatellite
else:
raise ValueError("Invalid base")
soc = cls(description, acpki=args.acpki)
soc.finalize()
if args.r is not None:
write_csr_file(soc, args.r)
if args.m is not None:
write_mem_file(soc, args.m)
if args.c is not None:
write_rustc_cfg_file(soc, args.c)
if args.g is not None:
soc.build(build_dir=args.g)
if __name__ == "__main__":
main()

191
src/gateware/redpitaya.py Executable file
View File

@ -0,0 +1,191 @@
#!/usr/bin/env python
import argparse
from operator import itemgetter
from migen import *
from migen.build.generic_platform import *
from migen.genlib.resetsync import AsyncResetSynchronizer
from migen.genlib.cdc import MultiReg
from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import redpitaya
from misoc.interconnect.csr import *
from misoc.integration import cpu_interface
from artiq.gateware import rtio
from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2
import dma
import analyzer
import acpki
class RTIOCRG(Module, AutoCSR):
def __init__(self, platform, rtio_internal_clk):
self.clock_sel = CSRStorage()
self.pll_reset = CSRStorage(reset=1)
self.pll_locked = CSRStatus()
self.clock_domains.cd_rtio = ClockDomain()
self.clock_domains.cd_rtiox4 = ClockDomain(reset_less=True)
rtio_external_clk = Signal()
# user_sma_clock = platform.request("user_sma_clock")
# platform.add_period_constraint(user_sma_clock.p, 8.0)
# self.specials += Instance("IBUFDS",
# i_I=user_sma_clock.p, i_IB=user_sma_clock.n,
# o_O=rtio_external_clk)
pll_locked = Signal()
rtio_clk = Signal()
rtiox4_clk = Signal()
self.specials += [
Instance("PLLE2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
p_REF_JITTER1=0.01,
p_CLKIN1_PERIOD=8.0, p_CLKIN2_PERIOD=8.0,
i_CLKIN1=rtio_internal_clk, i_CLKIN2=rtio_external_clk,
# Warning: CLKINSEL=0 means CLKIN2 is selected
i_CLKINSEL=~self.clock_sel.storage,
# VCO @ 1GHz when using 125MHz input
p_CLKFBOUT_MULT=8, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=self.cd_rtio.clk,
i_RST=self.pll_reset.storage,
o_CLKFBOUT=rtio_clk,
p_CLKOUT0_DIVIDE=2, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=rtiox4_clk),
Instance("BUFG", i_I=rtio_clk, o_O=self.cd_rtio.clk),
Instance("BUFG", i_I=rtiox4_clk, o_O=self.cd_rtiox4.clk),
AsyncResetSynchronizer(self.cd_rtio, ~pll_locked),
MultiReg(pll_locked, self.pll_locked.status)
]
class Redpitaya(SoCCore):
def __init__(self, acpki=False):
self.acpki = acpki
self.rustc_cfg = dict()
platform = redpitaya.Platform()
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = self.__class__.__name__
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident)
platform.add_platform_command("create_clock -name clk_fpga_0 -period 8 [get_pins \"PS7/FCLKCLK[0]\"]")
platform.add_platform_command("set_input_jitter clk_fpga_0 0.24")
self.submodules.rtio_crg = RTIOCRG(self.platform, self.ps7.cd_sys.clk)
self.csr_devices.append("rtio_crg")
self.platform.add_period_constraint(self.rtio_crg.cd_rtio.clk, 8.)
self.platform.add_false_path_constraints(
self.ps7.cd_sys.clk,
self.rtio_crg.cd_rtio.clk)
def add_rtio(self, rtio_channels):
self.submodules.rtio_tsc = rtio.TSC("async", glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri])
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.rtio_core.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
class Simple(Redpitaya):
def __init__(self, **kwargs):
Redpitaya.__init__(self, **kwargs)
platform = self.platform
rtio_channels = []
for i in range(2):
phy = ttl_simple.Output(platform.request("user_led", i))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels)
rtio_channels.append(rtio.LogChannel())
self.add_rtio(rtio_channels)
VARIANTS = {cls.__name__.lower(): cls for cls in [Simple]}
def write_csr_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f:
for k, v in sorted(soc.rustc_cfg.items(), key=itemgetter(0)):
if v is None:
f.write("{}\n".format(k))
else:
f.write("{}=\"{}\"\n".format(k, v))
def main():
parser = argparse.ArgumentParser(
description="ARTIQ port to the Redpitaya Zynq development kit")
parser.add_argument("-r", default=None,
help="build Rust interface into the specified file")
parser.add_argument("-c", default=None,
help="build Rust compiler configuration into the specified file")
parser.add_argument("-g", default=None,
help="build gateware into the specified directory")
parser.add_argument("-V", "--variant", default="10",
help="variant: "
"[acpki_]simple "
"(default: %(default)s)")
args = parser.parse_args()
variant = args.variant.lower()
acpki = variant.startswith("acpki_")
if acpki:
variant = variant[6:]
soc = Simple(acpki=acpki)
soc.finalize()
if args.r is not None:
write_csr_file(soc, args.r)
if args.c is not None:
write_rustc_cfg_file(soc, args.c)
if args.g is not None:
soc.build(build_dir=args.g)
if __name__ == "__main__":
main()

4
src/gateware/test_dma.py
View File

@ -168,7 +168,7 @@ class FullStackTB(Module):
bus = axi.Interface(ws*8) bus = axi.Interface(ws*8)
self.memory = AXIMemorySim(bus, sequence) self.memory = AXIMemorySim(bus, sequence)
self.submodules.dut = dma.DMA(bus) self.submodules.dut = dma.DMA(bus)
self.submodules.tsc = rtio.TSC() self.submodules.tsc = rtio.TSC("async")
self.submodules.rtio = rtio.Core(self.tsc, rtio_channels) self.submodules.rtio = rtio.Core(self.tsc, rtio_channels)
self.comb += self.dut.cri.connect(self.rtio.cri) self.comb += self.dut.cri.connect(self.rtio.cri)
@ -229,7 +229,7 @@ class TestDMA(unittest.TestCase):
do_dma(tb.dut, 0), monitor(), do_dma(tb.dut, 0), monitor(),
(None for _ in range(70)), (None for _ in range(70)),
tb.memory.ar(), tb.memory.r() tb.memory.ar(), tb.memory.r()
]}, {"sys": 8, "rsys": 8, "rio": 8, "rio_phy": 8}) ]}, {"sys": 8, "rsys": 8, "rtio": 8, "rio": 8, "rio_phy": 8})
correct_changes = [(timestamp + 11, channel) correct_changes = [(timestamp + 11, channel)
for channel, timestamp, _, _ in test_writes_full_stack] for channel, timestamp, _, _ in test_writes_full_stack]

612
src/gateware/zc706.py
View File

@ -11,161 +11,85 @@ from migen_axi.integration.soc_core import SoCCore
from migen_axi.platforms import zc706 from migen_axi.platforms import zc706
from misoc.interconnect.csr import * from misoc.interconnect.csr import *
from misoc.integration import cpu_interface from misoc.integration import cpu_interface
from misoc.cores import gpio
from artiq.gateware import rtio, nist_clock, nist_qc2 from artiq.gateware import rtio, nist_clock, nist_qc2
from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2, edge_counter from artiq.gateware.rtio.phy import ttl_simple, ttl_serdes_7series, dds, spi2, edge_counter
from artiq.gateware.rtio.xilinx_clocking import fix_serdes_timing_path
from artiq.gateware.drtio.transceiver import gtx_7series
from artiq.gateware.drtio.siphaser import SiPhaser7Series
from artiq.gateware.drtio.rx_synchronizer import XilinxRXSynchronizer
from artiq.gateware.drtio import *
import dma import dma
import analyzer import analyzer
import acpki import acpki
import drtio_aux_controller
import zynq_clocking
class SMAClkinForward(Module): class RTIOCRG(Module, AutoCSR):
def __init__(self, platform): def __init__(self, platform, rtio_internal_clk):
sma_clkin = platform.request("user_sma_clock") self.clock_sel = CSRStorage()
sma_clkin_se = Signal() self.pll_reset = CSRStorage(reset=1)
si5324_clkin_se = Signal() self.pll_locked = CSRStatus()
si5324_clkin = platform.request("si5324_clkin") self.clock_domains.cd_rtio = ClockDomain()
self.clock_domains.cd_rtiox4 = ClockDomain(reset_less=True)
rtio_external_clk = Signal()
user_sma_clock = platform.request("user_sma_clock")
platform.add_period_constraint(user_sma_clock.p, 8.0)
self.specials += Instance("IBUFDS",
i_I=user_sma_clock.p, i_IB=user_sma_clock.n,
o_O=rtio_external_clk)
pll_locked = Signal()
rtio_clk = Signal()
rtiox4_clk = Signal()
self.specials += [ self.specials += [
Instance("IBUFDS", i_I=sma_clkin.p, i_IB=sma_clkin.n, o_O=sma_clkin_se), Instance("PLLE2_ADV",
Instance("ODDR", i_C=sma_clkin_se, i_CE=1, i_D1=1, i_D2=0, o_Q=si5324_clkin_se), p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
Instance("OBUFDS", i_I=si5324_clkin_se, o_O=si5324_clkin.p, o_OB=si5324_clkin.n)
p_REF_JITTER1=0.01,
p_CLKIN1_PERIOD=8.0, p_CLKIN2_PERIOD=8.0,
i_CLKIN1=rtio_internal_clk, i_CLKIN2=rtio_external_clk,
# Warning: CLKINSEL=0 means CLKIN2 is selected
i_CLKINSEL=~self.clock_sel.storage,
# VCO @ 1GHz when using 125MHz input
p_CLKFBOUT_MULT=8, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=self.cd_rtio.clk,
i_RST=self.pll_reset.storage,
o_CLKFBOUT=rtio_clk,
p_CLKOUT0_DIVIDE=2, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=rtiox4_clk),
Instance("BUFG", i_I=rtio_clk, o_O=self.cd_rtio.clk),
Instance("BUFG", i_I=rtiox4_clk, o_O=self.cd_rtiox4.clk),
AsyncResetSynchronizer(self.cd_rtio, ~pll_locked),
MultiReg(pll_locked, self.pll_locked.status)
] ]
class CLK200BootstrapClock(Module):
def __init__(self, platform, freq=125e6):
self.clock_domains.cd_bootstrap = ClockDomain(reset_less=True)
self.cd_bootstrap.clk.attr.add("keep")
clk200 = platform.request("clk200")
clk200_se = Signal()
pll_fb = Signal()
pll_clkout = Signal()
assert freq in [125e6, 100e6]
divide = int(1e9/freq)
self.specials += [
Instance("IBUFDS",
i_I=clk200.p, i_IB=clk200.n, o_O=clk200_se),
Instance("PLLE2_BASE",
p_CLKIN1_PERIOD=5.0,
i_CLKIN1=clk200_se,
i_CLKFBIN=pll_fb,
o_CLKFBOUT=pll_fb,
# VCO @ 1GHz
p_CLKFBOUT_MULT=5, p_DIVCLK_DIVIDE=1,
# 125MHz/100MHz for bootstrap
p_CLKOUT1_DIVIDE=divide, p_CLKOUT1_PHASE=0.0, o_CLKOUT1=pll_clkout,
),
Instance("BUFG", i_I=pll_clkout, o_O=self.cd_bootstrap.clk)
]
# The NIST backplanes require setting VADJ to 3.3V by reprogramming the power supply.
# This also changes the I/O standard for some on-board LEDs.
leds_fmc33 = [
("user_led_33", 0, Pins("Y21"), IOStandard("LVCMOS33")),
("user_led_33", 1, Pins("G2"), IOStandard("LVCMOS15")),
("user_led_33", 2, Pins("W21"), IOStandard("LVCMOS33")),
("user_led_33", 3, Pins("A17"), IOStandard("LVCMOS15")),
]
# same deal as with LEDs - changed I/O standard.
si5324_fmc33 = [
("si5324_33", 0,
Subsignal("rst_n", Pins("W23"), IOStandard("LVCMOS33")),
Subsignal("int", Pins("AJ25"), IOStandard("LVCMOS33"))
),
]
pmod1_33 = [
("pmod1_33", 0, Pins("AJ21"), IOStandard("LVCMOS33")),
("pmod1_33", 1, Pins("AK21"), IOStandard("LVCMOS33")),
("pmod1_33", 2, Pins("AB21"), IOStandard("LVCMOS33")),
("pmod1_33", 3, Pins("AB16"), IOStandard("LVCMOS33")),
# rest removed for use with dummy spi
]
_ams101_dac = [
("ams101_dac", 0,
Subsignal("ldac", Pins("XADC:GPIO0")),
Subsignal("clk", Pins("XADC:GPIO1")),
Subsignal("mosi", Pins("XADC:GPIO2")),
Subsignal("cs_n", Pins("XADC:GPIO3")),
IOStandard("LVCMOS15")
)
]
_pmod_spi = [
("pmod_spi", 0,
# PMOD_1 4-7 pins, same bank as sfp_tx_disable or user_sma_clock
Subsignal("miso", Pins("Y20"), IOStandard("LVCMOS25")),
Subsignal("clk", Pins("AA20"), IOStandard("LVCMOS25")),
Subsignal("mosi", Pins("AC18"), IOStandard("LVCMOS25")),
Subsignal("cs_n", Pins("AC19"), IOStandard("LVCMOS25")),
IOStandard("LVCMOS25")
)
]
def prepare_zc706_platform(platform):
platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
class ZC706(SoCCore): class ZC706(SoCCore):
def __init__(self, acpki=False): def __init__(self, acpki=False):
self.acpki = acpki self.acpki = acpki
self.rustc_cfg = dict() self.rustc_cfg = dict()
platform = zc706.Platform() platform = zc706.Platform()
prepare_zc706_platform(platform) platform.toolchain.bitstream_commands.extend([
"set_property BITSTREAM.GENERAL.COMPRESS True [current_design]",
])
ident = self.__class__.__name__ ident = self.__class__.__name__
if self.acpki: if self.acpki:
ident = "acpki_" + ident ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False) SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident)
platform.add_extension(si5324_fmc33) platform.add_platform_command("create_clock -name clk_fpga_0 -period 8 [get_pins \"PS7/FCLKCLK[0]\"]")
self.comb += platform.request("si5324_33").rst_n.eq(1) platform.add_platform_command("set_input_jitter clk_fpga_0 0.24")
cdr_clk = Signal() self.submodules.rtio_crg = RTIOCRG(self.platform, self.ps7.cd_sys.clk)
cdr_clk_buf = Signal() self.csr_devices.append("rtio_crg")
si5324_out = platform.request("si5324_clkout") self.platform.add_period_constraint(self.rtio_crg.cd_rtio.clk, 8.)
platform.add_period_constraint(si5324_out.p, 8.0) self.platform.add_false_path_constraints(
self.specials += [ self.ps7.cd_sys.clk,
Instance("IBUFDS_GTE2", self.rtio_crg.cd_rtio.clk)
i_CEB=0,
i_I=si5324_out.p, i_IB=si5324_out.n,
o_O=cdr_clk,
p_CLKCM_CFG="0b1",
p_CLKRCV_TRST="0b1",
p_CLKSWING_CFG="0b11"),
Instance("BUFG", i_I=cdr_clk, o_O=cdr_clk_buf)
]
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_as_synthesizer"] = None
self.rustc_cfg["si5324_soft_reset"] = None
self.submodules.bootstrap = CLK200BootstrapClock(platform)
self.submodules.sys_crg = zynq_clocking.SYSCRG(self.platform, self.ps7, cdr_clk_buf)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
def add_rtio(self, rtio_channels): def add_rtio(self, rtio_channels):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3) self.submodules.rtio_tsc = rtio.TSC("async", glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, rtio_channels) self.submodules.rtio_core = rtio.Core(self.rtio_tsc, rtio_channels)
self.csr_devices.append("rtio_core") self.csr_devices.append("rtio_core")
@ -197,314 +121,52 @@ class ZC706(SoCCore):
self.csr_devices.append("rtio_analyzer") self.csr_devices.append("rtio_analyzer")
class _MasterBase(SoCCore): class Simple(ZC706):
def __init__(self, acpki=False, drtio100mhz=False): def __init__(self, **kwargs):
self.acpki = acpki ZC706.__init__(self, **kwargs)
self.rustc_cfg = dict()
clk_freq = 100e6 if drtio100mhz else 125e6 platform = self.platform
platform = zc706.Platform() rtio_channels = []
prepare_zc706_platform(platform) for i in range(4):
ident = self.__class__.__name__ phy = ttl_simple.Output(platform.request("user_led", i))
if self.acpki: self.submodules += phy
ident = "acpki_" + ident rtio_channels.append(rtio.Channel.from_phy(phy))
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
platform.add_extension(si5324_fmc33) self.config["RTIO_LOG_CHANNEL"] = len(rtio_channels)
rtio_channels.append(rtio.LogChannel())
self.comb += platform.request("sfp_tx_disable_n").eq(1) self.add_rtio(rtio_channels)
data_pads = [
platform.request("sfp"),
platform.request("user_sma_mgt")
]
self.submodules += SMAClkinForward(self.platform)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("drtio_transceiver")
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
txout_buf = Signal()
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance("BUFG", i_I=gtx0.txoutclk, o_O=txout_buf)
self.submodules.bootstrap = CLK200BootstrapClock(platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
txout_buf,
clk_sw=gtx0.tx_init.done,
freq=clk_freq)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
drtio_csr_group = []
drtioaux_csr_group = []
drtioaux_memory_group = []
self.drtio_cri = []
for i in range(len(self.drtio_transceiver.channels)):
core_name = "drtio" + str(i)
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtio_csr_group.append(core_name)
drtioaux_csr_group.append(coreaux_name)
drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
core = cdr(DRTIOMaster(
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, core_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(core_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
mem_size = coreaux.get_mem_size()
memory_address = self.axi2csr.register_port(coreaux.get_tx_port(), mem_size)
self.axi2csr.register_port(coreaux.get_rx_port(), mem_size)
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtio", drtio_csr_group)
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n")
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["si5324_as_synthesizer"] = None
# Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX)
platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX)
for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints(
gtx0.txoutclk, gtx.rxoutclk)
fix_serdes_timing_path(platform)
def add_rtio(self, rtio_channels):
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
self.submodules.rtio_core = rtio.Core(self.rtio_tsc, rtio_channels)
self.csr_devices.append("rtio_core")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.rtio.cri, self.rtio_dma.cri],
[self.rtio_core.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
self.submodules.rtio_moninj = rtio.MonInj(rtio_channels)
self.csr_devices.append("rtio_moninj")
self.submodules.rtio_analyzer = analyzer.Analyzer(self.rtio_tsc, self.rtio_core.cri,
self.ps7.s_axi_hp1)
self.csr_devices.append("rtio_analyzer")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table")
class _SatelliteBase(SoCCore): # The NIST backplanes require setting VADJ to 3.3V by reprogramming the power supply.
def __init__(self, acpki=False, drtio100mhz=False): # This also changes the I/O standard for some on-board LEDs.
self.acpki = acpki leds_fmc33 = [
self.rustc_cfg = dict() ("user_led_33", 0, Pins("Y21"), IOStandard("LVCMOS33")),
("user_led_33", 1, Pins("G2"), IOStandard("LVCMOS15")),
clk_freq = 100e6 if drtio100mhz else 125e6 ("user_led_33", 2, Pins("W21"), IOStandard("LVCMOS33")),
("user_led_33", 3, Pins("A17"), IOStandard("LVCMOS15")),
platform = zc706.Platform() ]
prepare_zc706_platform(platform)
ident = self.__class__.__name__
if self.acpki:
ident = "acpki_" + ident
SoCCore.__init__(self, platform=platform, csr_data_width=32, ident=ident, ps_cd_sys=False)
platform.add_extension(si5324_fmc33)
# SFP
self.comb += platform.request("sfp_tx_disable_n").eq(0)
data_pads = [
platform.request("sfp"),
platform.request("user_sma_mgt")
]
self.submodules.rtio_tsc = rtio.TSC(glbl_fine_ts_width=3)
# 1000BASE_BX10 Ethernet compatible, 125MHz RTIO clock
self.submodules.drtio_transceiver = gtx_7series.GTX(
clock_pads=platform.request("si5324_clkout"),
pads=data_pads,
clk_freq=clk_freq)
self.csr_devices.append("drtio_transceiver")
txout_buf = Signal()
txout_buf.attr.add("keep")
gtx0 = self.drtio_transceiver.gtxs[0]
self.specials += Instance(
"BUFG",
i_I=gtx0.txoutclk,
o_O=txout_buf)
self.submodules.bootstrap = CLK200BootstrapClock(platform, clk_freq)
self.submodules.sys_crg = zynq_clocking.SYSCRG(
self.platform,
self.ps7,
txout_buf,
clk_sw=gtx0.tx_init.done,
freq=clk_freq)
platform.add_false_path_constraints(
self.bootstrap.cd_bootstrap.clk, self.sys_crg.cd_sys.clk)
self.csr_devices.append("sys_crg")
drtioaux_csr_group = []
drtioaux_memory_group = []
drtiorep_csr_group = []
self.drtio_cri = []
for i in range(len(self.drtio_transceiver.channels)):
coreaux_name = "drtioaux" + str(i)
memory_name = "drtioaux" + str(i) + "_mem"
drtioaux_csr_group.append(coreaux_name)
drtioaux_memory_group.append(memory_name)
cdr = ClockDomainsRenamer({"rtio_rx": "rtio_rx" + str(i)})
# Satellite
if i == 0:
self.submodules.rx_synchronizer = cdr(XilinxRXSynchronizer())
core = cdr(DRTIOSatellite(
self.rtio_tsc, self.drtio_transceiver.channels[0], self.rx_synchronizer))
self.submodules.drtiosat = core
self.csr_devices.append("drtiosat")
# Repeaters
else:
corerep_name = "drtiorep" + str(i-1)
drtiorep_csr_group.append(corerep_name)
core = cdr(DRTIORepeater(
self.rtio_tsc, self.drtio_transceiver.channels[i]))
setattr(self.submodules, corerep_name, core)
self.drtio_cri.append(core.cri)
self.csr_devices.append(corerep_name)
coreaux = cdr(drtio_aux_controller.DRTIOAuxControllerBare(core.link_layer))
setattr(self.submodules, coreaux_name, coreaux)
self.csr_devices.append(coreaux_name)
mem_size = coreaux.get_mem_size()
tx_port = coreaux.get_tx_port()
rx_port = coreaux.get_rx_port()
memory_address = self.axi2csr.register_port(tx_port, mem_size)
# rcv in upper half of the memory, thus added second
self.axi2csr.register_port(rx_port, mem_size)
# and registered in PS interface
# manually, because software refers to rx/tx by halves of entire memory block, not names
self.add_memory_region(memory_name, self.mem_map["csr"] + memory_address, mem_size * 2)
self.rustc_cfg["has_drtio"] = None
self.rustc_cfg["has_drtio_routing"] = None
self.add_csr_group("drtioaux", drtioaux_csr_group)
self.add_csr_group("drtiorep", drtiorep_csr_group)
self.add_memory_group("drtioaux_mem", drtioaux_memory_group)
self.rustc_cfg["rtio_frequency"] = str(self.drtio_transceiver.rtio_clk_freq/1e6)
# Si5324 Phaser
self.submodules.siphaser = SiPhaser7Series(
si5324_clkin=platform.request("si5324_clkin"),
rx_synchronizer=self.rx_synchronizer,
ultrascale=False,
rtio_clk_freq=self.drtio_transceiver.rtio_clk_freq)
platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, self.siphaser.mmcm_freerun_output)
self.csr_devices.append("siphaser")
self.submodules.si5324_rst_n = gpio.GPIOOut(platform.request("si5324_33").rst_n)
self.csr_devices.append("si5324_rst_n")
self.rustc_cfg["has_si5324"] = None
self.rustc_cfg["has_siphaser"] = None
rtio_clk_period = 1e9/self.drtio_transceiver.rtio_clk_freq
# Constrain TX & RX timing for the first transceiver channel
# (First channel acts as master for phase alignment for all channels' TX)
platform.add_false_path_constraints(
gtx0.txoutclk, gtx0.rxoutclk)
# Constrain RX timing for the each transceiver channel
# (Each channel performs single-lane phase alignment for RX)
for gtx in self.drtio_transceiver.gtxs[1:]:
platform.add_false_path_constraints(
self.sys_crg.cd_sys.clk, gtx.rxoutclk)
fix_serdes_timing_path(platform)
def add_rtio(self, rtio_channels):
self.submodules.rtio_moninj = rtio.MonInj(rtio_channels)
self.csr_devices.append("rtio_moninj")
if self.acpki:
self.rustc_cfg["ki_impl"] = "acp"
self.submodules.rtio = acpki.KernelInitiator(self.rtio_tsc,
bus=self.ps7.s_axi_acp,
user=self.ps7.s_axi_acp_user,
evento=self.ps7.event.o)
self.csr_devices.append("rtio")
else:
self.rustc_cfg["ki_impl"] = "csr"
self.submodules.rtio = rtio.KernelInitiator(self.rtio_tsc, now64=True)
self.csr_devices.append("rtio")
self.submodules.rtio_dma = dma.DMA(self.ps7.s_axi_hp0)
self.csr_devices.append("rtio_dma")
self.submodules.local_io = SyncRTIO(self.rtio_tsc, rtio_channels)
self.submodules.cri_con = rtio.CRIInterconnectShared(
[self.drtiosat.cri, self.rtio_dma.cri],
[self.local_io.cri] + self.drtio_cri,
enable_routing=True)
self.csr_devices.append("cri_con")
self.submodules.routing_table = rtio.RoutingTableAccess(self.cri_con)
self.csr_devices.append("routing_table")
class NIST_CLOCK(ZC706):
class _NIST_CLOCK_RTIO:
""" """
NIST clock hardware, with old backplane and 11 DDS channels NIST clock hardware, with old backplane and 11 DDS channels
""" """
def __init__(self): def __init__(self, **kwargs):
ZC706.__init__(self, **kwargs)
platform = self.platform platform = self.platform
platform.add_extension(nist_clock.fmc_adapter_io) platform.add_extension(nist_clock.fmc_adapter_io)
platform.add_extension(leds_fmc33) platform.add_extension(leds_fmc33)
platform.add_extension(pmod1_33)
platform.add_extension(_ams101_dac)
platform.add_extension(_pmod_spi)
rtio_channels = [] rtio_channels = []
for i in range(4):
phy = ttl_simple.Output(platform.request("user_led_33", i))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
for i in range(16): for i in range(16):
if i % 4 == 3: if i % 4 == 3:
phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i)) phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i))
@ -520,40 +182,16 @@ class _NIST_CLOCK_RTIO:
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512)) rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512))
# no SMA GPIO, replaced with PMOD1_0
phy = ttl_serdes_7series.InOut_8X(platform.request("pmod1_33", 0))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512))
phy = ttl_simple.Output(platform.request("user_led_33", 0))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
ams101_dac = self.platform.request("ams101_dac", 0)
phy = ttl_simple.Output(ams101_dac.ldac)
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
phy = ttl_simple.ClockGen(platform.request("la32_p")) phy = ttl_simple.ClockGen(platform.request("la32_p"))
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy)) rtio_channels.append(rtio.Channel.from_phy(phy))
phy = spi2.SPIMaster(ams101_dac)
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(
phy, ififo_depth=4))
for i in range(3): for i in range(3):
phy = spi2.SPIMaster(self.platform.request("spi", i)) phy = spi2.SPIMaster(self.platform.request("spi", i))
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy( rtio_channels.append(rtio.Channel.from_phy(
phy, ififo_depth=128)) phy, ififo_depth=128))
# no SDIO on PL side, dummy SPI placeholder instead
phy = spi2.SPIMaster(platform.request("pmod_spi"))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=4))
phy = dds.AD9914(platform.request("dds"), 11, onehot=True) phy = dds.AD9914(platform.request("dds"), 11, onehot=True)
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=4)) rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=4))
@ -564,21 +202,26 @@ class _NIST_CLOCK_RTIO:
self.add_rtio(rtio_channels) self.add_rtio(rtio_channels)
class _NIST_QC2_RTIO: class NIST_QC2(ZC706):
""" """
NIST QC2 hardware, as used in Quantum I and Quantum II, with new backplane NIST QC2 hardware, as used in Quantum I and Quantum II, with new backplane
and 24 DDS channels. Two backplanes are used. and 24 DDS channels. Two backplanes are used.
""" """
def __init__(self): def __init__(self, **kwargs):
ZC706.__init__(self, **kwargs)
platform = self.platform platform = self.platform
platform.add_extension(nist_qc2.fmc_adapter_io) platform.add_extension(nist_qc2.fmc_adapter_io)
platform.add_extension(leds_fmc33) platform.add_extension(leds_fmc33)
platform.add_extension(_ams101_dac)
platform.add_extension(pmod1_33)
rtio_channels = [] rtio_channels = []
edge_counter_phy = [] edge_counter_phy = []
for i in range(4):
phy = ttl_simple.Output(platform.request("user_led_33", i))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
# All TTL channels are In+Out capable # All TTL channels are In+Out capable
for i in range(40): for i in range(40):
phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i)) phy = ttl_serdes_7series.InOut_8X(platform.request("ttl", i))
@ -588,31 +231,12 @@ class _NIST_QC2_RTIO:
if i < 4: if i < 4:
edge_counter_phy.append(phy) edge_counter_phy.append(phy)
# no SMA GPIO, replaced with PMOD1_0
phy = ttl_serdes_7series.InOut_8X(platform.request("pmod1_33", 0))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy, ififo_depth=512))
phy = ttl_simple.Output(platform.request("user_led_33", 0))
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
ams101_dac = self.platform.request("ams101_dac", 0)
phy = ttl_simple.Output(ams101_dac.ldac)
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy))
# CLK0, CLK1 are for clock generators, on backplane SMP connectors # CLK0, CLK1 are for clock generators, on backplane SMP connectors
for i in range(2): for i in range(2):
phy = ttl_simple.ClockGen( phy = ttl_simple.ClockGen(
platform.request("clkout", i)) platform.request("clkout", i))
self.submodules += phy self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(phy)) rtio_channels.append(rtio.Channel.from_phy(phy))
phy = spi2.SPIMaster(ams101_dac)
self.submodules += phy
rtio_channels.append(rtio.Channel.from_phy(
phy, ififo_depth=4))
for i in range(4): for i in range(4):
phy = spi2.SPIMaster(self.platform.request("spi", i)) phy = spi2.SPIMaster(self.platform.request("spi", i))
@ -637,40 +261,7 @@ class _NIST_QC2_RTIO:
self.add_rtio(rtio_channels) self.add_rtio(rtio_channels)
class NIST_CLOCK(ZC706, _NIST_CLOCK_RTIO): VARIANTS = {cls.__name__.lower(): cls for cls in [Simple, NIST_CLOCK, NIST_QC2]}
def __init__(self, acpki, drtio100mhz):
ZC706.__init__(self, acpki)
self.submodules += SMAClkinForward(self.platform)
_NIST_CLOCK_RTIO.__init__(self)
class NIST_CLOCK_Master(_MasterBase, _NIST_CLOCK_RTIO):
def __init__(self, acpki, drtio100mhz):
_MasterBase.__init__(self, acpki, drtio100mhz)
_NIST_CLOCK_RTIO.__init__(self)
class NIST_CLOCK_Satellite(_SatelliteBase, _NIST_CLOCK_RTIO):
def __init__(self, acpki, drtio100mhz):
_SatelliteBase.__init__(self, acpki, drtio100mhz)
_NIST_CLOCK_RTIO.__init__(self)
class NIST_QC2(ZC706, _NIST_QC2_RTIO):
def __init__(self, acpki, drtio100mhz):
ZC706.__init__(self, acpki)
self.submodules += SMAClkinForward(self.platform)
_NIST_QC2_RTIO.__init__(self)
class NIST_QC2_Master(_MasterBase, _NIST_QC2_RTIO):
def __init__(self, acpki, drtio100mhz):
_MasterBase.__init__(self, acpki, drtio100mhz)
_NIST_QC2_RTIO.__init__(self)
class NIST_QC2_Satellite(_SatelliteBase, _NIST_QC2_RTIO):
def __init__(self, acpki, drtio100mhz):
_SatelliteBase.__init__(self, acpki, drtio100mhz)
_NIST_QC2_RTIO.__init__(self)
VARIANTS = {cls.__name__.lower(): cls for cls in [NIST_CLOCK, NIST_CLOCK_Master, NIST_CLOCK_Satellite,
NIST_QC2, NIST_QC2_Master, NIST_QC2_Satellite]}
def write_csr_file(soc, filename): def write_csr_file(soc, filename):
@ -678,11 +269,6 @@ def write_csr_file(soc, filename):
f.write(cpu_interface.get_csr_rust( f.write(cpu_interface.get_csr_rust(
soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants())) soc.get_csr_regions(), soc.get_csr_groups(), soc.get_constants()))
def write_mem_file(soc, filename):
with open(filename, "w") as f:
f.write(cpu_interface.get_mem_rust(
soc.get_memory_regions(), soc.get_memory_groups(), None))
def write_rustc_cfg_file(soc, filename): def write_rustc_cfg_file(soc, filename):
with open(filename, "w") as f: with open(filename, "w") as f:
@ -698,15 +284,13 @@ def main():
description="ARTIQ port to the ZC706 Zynq development kit") description="ARTIQ port to the ZC706 Zynq development kit")
parser.add_argument("-r", default=None, parser.add_argument("-r", default=None,
help="build Rust interface into the specified file") help="build Rust interface into the specified file")
parser.add_argument("-m", default=None,
help="build Rust memory interface into the specified file")
parser.add_argument("-c", default=None, parser.add_argument("-c", default=None,
help="build Rust compiler configuration into the specified file") help="build Rust compiler configuration into the specified file")
parser.add_argument("-g", default=None, parser.add_argument("-g", default=None,
help="build gateware into the specified directory") help="build gateware into the specified directory")
parser.add_argument("-V", "--variant", default="nist_clock", parser.add_argument("-V", "--variant", default="simple",
help="variant: " help="variant: "
"[acpki_]nist_clock/nist_qc2[_master/_satellite][_100mhz]" "[acpki_]simple/nist_clock/nist_qc2 "
"(default: %(default)s)") "(default: %(default)s)")
args = parser.parse_args() args = parser.parse_args()
@ -714,25 +298,21 @@ def main():
acpki = variant.startswith("acpki_") acpki = variant.startswith("acpki_")
if acpki: if acpki:
variant = variant[6:] variant = variant[6:]
drtio100mhz = variant.endswith("_100mhz")
if drtio100mhz:
variant = variant[:-7]
try: try:
cls = VARIANTS[variant] cls = VARIANTS[variant]
except KeyError: except KeyError:
raise SystemExit("Invalid variant (-V/--variant)") raise SystemExit("Invalid variant (-V/--variant)")
soc = cls(acpki=acpki, drtio100mhz=drtio100mhz) soc = cls(acpki=acpki)
soc.finalize() soc.finalize()
if args.r is not None: if args.r is not None:
write_csr_file(soc, args.r) write_csr_file(soc, args.r)
if args.m is not None:
write_mem_file(soc, args.m)
if args.c is not None: if args.c is not None:
write_rustc_cfg_file(soc, args.c) write_rustc_cfg_file(soc, args.c)
if args.g is not None: if args.g is not None:
soc.build(build_dir=args.g) soc.build(build_dir=args.g)
if __name__ == "__main__": if __name__ == "__main__":
main() main()

122
src/gateware/zynq_clocking.py
View File

@ -1,122 +0,0 @@
from migen import *
from migen.genlib.cdc import MultiReg
from migen.genlib.resetsync import AsyncResetSynchronizer
from misoc.interconnect.csr import *
class ClockSwitchFSM(Module):
def __init__(self):
self.i_clk_sw = Signal()
self.o_clk_sw = Signal()
self.o_reset = Signal()
###
i_switch = Signal()
o_switch = Signal()
reset = Signal()
# at 125MHz bootstrap cd, will get around 0.5ms
delay_counter = Signal(16, reset=0xFFFF)
# register to prevent glitches
self.sync.bootstrap += [
self.o_clk_sw.eq(o_switch),
self.o_reset.eq(reset),
]
self.o_clk_sw.attr.add("no_retiming")
self.o_reset.attr.add("no_retiming")
self.i_clk_sw.attr.add("no_retiming")
i_switch.attr.add("no_retiming")
self.specials += MultiReg(self.i_clk_sw, i_switch, "bootstrap")
fsm = ClockDomainsRenamer("bootstrap")(FSM(reset_state="START"))
self.submodules += fsm
fsm.act("START",
If(i_switch & ~o_switch,
NextState("RESET_START"))
)
fsm.act("RESET_START",
reset.eq(1),
If(delay_counter == 0,
NextValue(delay_counter, 0xFFFF),
NextState("CLOCK_SWITCH")
).Else(
NextValue(delay_counter, delay_counter-1),
)
)
fsm.act("CLOCK_SWITCH",
reset.eq(1),
NextValue(o_switch, 1),
NextValue(delay_counter, delay_counter-1),
If(delay_counter == 0,
NextState("END"))
)
fsm.act("END",
NextValue(o_switch, 1),
reset.eq(0))
class SYSCRG(Module, AutoCSR):
def __init__(self, platform, ps7, main_clk, clk_sw=None, freq=125e6):
# assumes bootstrap clock is same freq as main and sys output
self.clock_domains.cd_sys = ClockDomain()
self.clock_domains.cd_sys4x = ClockDomain(reset_less=True)
self.current_clock = CSRStatus()
self.cd_sys.clk.attr.add("keep")
bootstrap_clk = ClockSignal("bootstrap")
period = 1e9/freq
pll_locked = Signal()
pll_sys = Signal()
pll_sys4x = Signal()
fb_clk = Signal()
self.submodules.clk_sw_fsm = ClockSwitchFSM()
if clk_sw is None:
self.clock_switch = CSRStorage()
self.comb += self.clk_sw_fsm.i_clk_sw.eq(self.clock_switch.storage)
else:
self.comb += self.clk_sw_fsm.i_clk_sw.eq(clk_sw)
self.specials += [
Instance("PLLE2_ADV",
p_STARTUP_WAIT="FALSE", o_LOCKED=pll_locked,
p_BANDWIDTH="HIGH",
p_REF_JITTER1=0.001,
p_CLKIN1_PERIOD=period, i_CLKIN1=main_clk,
p_CLKIN2_PERIOD=period, i_CLKIN2=bootstrap_clk,
i_CLKINSEL=self.clk_sw_fsm.o_clk_sw,
# VCO @ 1.5GHz when using 125MHz input
# 1.2GHz for 100MHz (zc706)
p_CLKFBOUT_MULT=12, p_DIVCLK_DIVIDE=1,
i_CLKFBIN=fb_clk,
i_RST=self.clk_sw_fsm.o_reset,
o_CLKFBOUT=fb_clk,
p_CLKOUT0_DIVIDE=3, p_CLKOUT0_PHASE=0.0,
o_CLKOUT0=pll_sys4x,
p_CLKOUT1_DIVIDE=12, p_CLKOUT1_PHASE=0.0,
o_CLKOUT1=pll_sys),
Instance("BUFG", i_I=pll_sys, o_O=self.cd_sys.clk),
Instance("BUFG", i_I=pll_sys4x, o_O=self.cd_sys4x.clk),
AsyncResetSynchronizer(self.cd_sys, ~pll_locked),
]
self.comb += self.current_clock.status.eq(self.clk_sw_fsm.o_clk_sw)

31
src/libboard_artiq/Cargo.toml
View File

@ -1,31 +0,0 @@
[package]
name = "libboard_artiq"
version = "0.0.0"
authors = ["M-Labs"]
edition = "2018"
[lib]
name = "libboard_artiq"
[features]
target_zc706 = ["libboard_zynq/target_zc706", "libconfig/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libconfig/target_kasli_soc"]
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
log = "0.4"
log_buffer = { version = "1.2" }
crc = { version = "1.7", default-features = false }
core_io = { version = "0.1", features = ["collections"] }
embedded-hal = "0.2"
nb = "1.0"
void = { version = "1", default-features = false }
io = { path = "../libio", features = ["byteorder"] }
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git"}
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn"] }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }

5
src/libboard_artiq/build.rs
View File

@ -1,5 +0,0 @@
extern crate build_zynq;
fn main() {
build_zynq::cfg();
}

107
src/libboard_artiq/src/drtio_routing.rs
View File

@ -1,107 +0,0 @@
use core::fmt;
use libconfig::Config;
use log::{info, warn};
#[cfg(has_drtio_routing)]
use crate::pl::csr;
#[cfg(has_drtio_routing)]
pub const DEST_COUNT: usize = 256;
#[cfg(not(has_drtio_routing))]
pub const DEST_COUNT: usize = 0;
pub const MAX_HOPS: usize = 32;
pub const INVALID_HOP: u8 = 0xff;
pub struct RoutingTable(pub [[u8; MAX_HOPS]; DEST_COUNT]);
impl RoutingTable {
// default routing table is for star topology with no repeaters
pub fn default_master(default_n_links: usize) -> RoutingTable {
let mut ret = RoutingTable([[INVALID_HOP; MAX_HOPS]; DEST_COUNT]);
let n_entries = default_n_links + 1; // include local RTIO
for i in 0..n_entries {
ret.0[i][0] = i as u8;
}
for i in 1..n_entries {
ret.0[i][1] = 0x00;
}
ret
}
// use this by default on satellite, as they receive
// the routing table from the master
pub fn default_empty() -> RoutingTable {
RoutingTable([[INVALID_HOP; MAX_HOPS]; DEST_COUNT])
}
}
impl fmt::Display for RoutingTable {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "RoutingTable {{")?;
for i in 0..DEST_COUNT {
if self.0[i][0] != INVALID_HOP {
write!(f, " {}:", i)?;
for j in 0..MAX_HOPS {
if self.0[i][j] == INVALID_HOP {
break;
}
write!(f, " {}", self.0[i][j])?;
}
write!(f, ";")?;
}
}
write!(f, " }}")?;
Ok(())
}
}
pub fn config_routing_table(default_n_links: usize, cfg: &Config) -> RoutingTable {
let mut ret = RoutingTable::default_master(default_n_links);
if let Ok(data) = cfg.read("routing_table") {
if data.len() == DEST_COUNT * MAX_HOPS {
for i in 0..DEST_COUNT {
for j in 0..MAX_HOPS {
ret.0[i][j] = data[i * MAX_HOPS + j];
}
}
} else {
warn!("length of the configured routing table is incorrect, using default");
}
} else {
info!("could not read routing table from configuration, using default");
}
info!("routing table: {}", ret);
ret
}
#[cfg(has_drtio_routing)]
pub fn interconnect_enable(routing_table: &RoutingTable, rank: u8, destination: u8) {
let hop = routing_table.0[destination as usize][rank as usize];
unsafe {
csr::routing_table::destination_write(destination);
csr::routing_table::hop_write(hop);
}
}
#[cfg(has_drtio_routing)]
pub fn interconnect_disable(destination: u8) {
unsafe {
csr::routing_table::destination_write(destination);
csr::routing_table::hop_write(INVALID_HOP);
}
}
#[cfg(has_drtio_routing)]
pub fn interconnect_enable_all(routing_table: &RoutingTable, rank: u8) {
for i in 0..DEST_COUNT {
interconnect_enable(routing_table, rank, i as u8);
}
}
#[cfg(has_drtio_routing)]
pub fn interconnect_disable_all() {
for i in 0..DEST_COUNT {
interconnect_disable(i as u8);
}
}

161
src/libboard_artiq/src/drtioaux.rs
View File

@ -1,161 +0,0 @@
use core_io::{Error as IoError, ErrorKind as IoErrorKind};
use crc;
use io::{proto::{ProtoRead, ProtoWrite},
Cursor};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::asm::dmb;
pub use crate::drtioaux_proto::Packet;
use crate::{drtioaux_proto::Error as ProtocolError, mem::mem::DRTIOAUX_MEM, pl::csr::DRTIOAUX};
#[derive(Debug)]
pub enum Error {
GatewareError,
CorruptedPacket,
LinkDown,
TimedOut,
UnexpectedReply,
RoutingError,
Protocol(ProtocolError),
}
impl From<ProtocolError> for Error {
fn from(value: ProtocolError) -> Error {
Error::Protocol(value)
}
}
impl From<IoError> for Error {
fn from(value: IoError) -> Error {
Error::Protocol(ProtocolError::Io(value))
}
}
pub fn reset(linkno: u8) {
let linkno = linkno as usize;
unsafe {
// clear buffer first to limit race window with buffer overflow
// error. We assume the CPU is fast enough so that no two packets
// will be received between the buffer and the error flag are cleared.
(DRTIOAUX[linkno].aux_rx_present_write)(1);
(DRTIOAUX[linkno].aux_rx_error_write)(1);
}
}
pub fn has_rx_error(linkno: u8) -> bool {
let linkno = linkno as usize;
unsafe {
let error = (DRTIOAUX[linkno].aux_rx_error_read)() != 0;
if error {
(DRTIOAUX[linkno].aux_rx_error_write)(1)
}
error
}
}
pub fn copy_work_buffer(src: *mut u32, dst: *mut u32, len: isize) {
// AXI writes must be 4-byte aligned (drtio proto doesn't care for that),
// and AXI burst reads/writes are not implemented yet in gateware
// thus the need for a work buffer for transmitting and copying it over
unsafe {
for i in 0..(len / 4) {
*dst.offset(i) = *src.offset(i);
//data memory barrier to prevent bursts
dmb();
}
}
}
fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error>
where F: FnOnce(&[u8]) -> Result<T, Error> {
let linkidx = linkno as usize;
unsafe {
if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2) as *mut u32;
let len = (DRTIOAUX[linkidx].aux_rx_length_read)() as usize;
// work buffer to accomodate axi burst reads
let mut buf: [u8; 1024] = [0; 1024];
copy_work_buffer(ptr, buf.as_mut_ptr() as *mut u32, len as isize);
let result = f(&buf[0..len]);
(DRTIOAUX[linkidx].aux_rx_present_write)(1);
Ok(Some(result?))
} else {
Ok(None)
}
}
}
pub fn recv(linkno: u8) -> Result<Option<Packet>, Error> {
if has_rx_error(linkno) {
return Err(Error::GatewareError);
}
receive(linkno, |buffer| {
if buffer.len() < 8 {
return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into());
}
let mut reader = Cursor::new(buffer);
let checksum_at = buffer.len() - 4;
let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
reader.set_position(checksum_at);
if reader.read_u32()? != checksum {
return Err(Error::CorruptedPacket);
}
reader.set_position(0);
Ok(Packet::read_from(&mut reader)?)
})
}
pub fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTimer) -> Result<Packet, Error> {
let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10));
let limit = timer.get_time() + timeout_ms;
while timer.get_time() < limit {
match recv(linkno)? {
None => (),
Some(packet) => return Ok(packet),
}
}
Err(Error::TimedOut)
}
fn transmit<F>(linkno: u8, f: F) -> Result<(), Error>
where F: FnOnce(&mut [u8]) -> Result<usize, Error> {
let linkno = linkno as usize;
unsafe {
while (DRTIOAUX[linkno].aux_tx_read)() != 0 {}
let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
let len = DRTIOAUX_MEM[linkno].size / 2;
// work buffer, works with unaligned mem access
let mut buf: [u8; 1024] = [0; 1024];
let len = f(&mut buf[0..len])?;
copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
(DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
(DRTIOAUX[linkno].aux_tx_write)(1);
Ok(())
}
}
pub fn send(linkno: u8, packet: &Packet) -> Result<(), Error> {
transmit(linkno, |buffer| {
let mut writer = Cursor::new(buffer);
packet.write_to(&mut writer)?;
// Pad till offset 4, insert checksum there
let padding = (12 - (writer.position() % 8)) % 8;
for _ in 0..padding {
writer.write_u8(0)?;
}
let checksum = crc::crc32::checksum_ieee(&writer.get_ref()[0..writer.position()]);
writer.write_u32(checksum)?;
Ok(writer.position())
})
}

135
src/libboard_artiq/src/drtioaux_async.rs
View File

@ -1,135 +0,0 @@
use core_io::{Error as IoError, ErrorKind as IoErrorKind};
use crc;
use io::{proto::{ProtoRead, ProtoWrite},
Cursor};
use libasync::{block_async, task};
use libboard_zynq::{time::Milliseconds, timer::GlobalTimer};
use nb;
use void::Void;
pub use crate::drtioaux_proto::Packet;
use crate::{drtioaux::{copy_work_buffer, has_rx_error, Error},
mem::mem::DRTIOAUX_MEM,
pl::csr::DRTIOAUX};
pub async fn reset(linkno: u8) {
let linkno = linkno as usize;
unsafe {
// clear buffer first to limit race window with buffer overflow
// error. We assume the CPU is fast enough so that no two packets
// will be received between the buffer and the error flag are cleared.
(DRTIOAUX[linkno].aux_rx_present_write)(1);
(DRTIOAUX[linkno].aux_rx_error_write)(1);
}
}
fn tx_ready(linkno: usize) -> nb::Result<(), Void> {
unsafe {
if (DRTIOAUX[linkno].aux_tx_read)() != 0 {
Err(nb::Error::WouldBlock)
} else {
Ok(())
}
}
}
async fn receive<F, T>(linkno: u8, f: F) -> Result<Option<T>, Error>
where F: FnOnce(&[u8]) -> Result<T, Error> {
let linkidx = linkno as usize;
unsafe {
if (DRTIOAUX[linkidx].aux_rx_present_read)() == 1 {
let ptr = (DRTIOAUX_MEM[linkidx].base + DRTIOAUX_MEM[linkidx].size / 2) as *mut u32;
let len = (DRTIOAUX[linkidx].aux_rx_length_read)() as usize;
// work buffer to accomodate axi burst reads
let mut buf: [u8; 1024] = [0; 1024];
copy_work_buffer(ptr, buf.as_mut_ptr() as *mut u32, len as isize);
let result = f(&buf[0..len]);
(DRTIOAUX[linkidx].aux_rx_present_write)(1);
Ok(Some(result?))
} else {
Ok(None)
}
}
}
pub async fn recv(linkno: u8) -> Result<Option<Packet>, Error> {
if has_rx_error(linkno) {
return Err(Error::GatewareError);
}
receive(linkno, |buffer| {
if buffer.len() < 8 {
return Err(IoError::new(IoErrorKind::UnexpectedEof, "Unexpected end").into());
}
let mut reader = Cursor::new(buffer);
let checksum_at = buffer.len() - 4;
let checksum = crc::crc32::checksum_ieee(&reader.get_ref()[0..checksum_at]);
reader.set_position(checksum_at);
if reader.read_u32()? != checksum {
return Err(Error::CorruptedPacket);
}
reader.set_position(0);
Ok(Packet::read_from(&mut reader)?)
})
.await
}
pub async fn recv_timeout(linkno: u8, timeout_ms: Option<u64>, timer: GlobalTimer) -> Result<Packet, Error> {
let timeout_ms = Milliseconds(timeout_ms.unwrap_or(10));
let limit = timer.get_time() + timeout_ms;
let mut would_block = false;
while timer.get_time() < limit {
// to ensure one last time recv would run one last time
// in case async would return after timeout
if would_block {
task::r#yield().await;
}
match recv(linkno).await? {
None => {
would_block = true;
}
Some(packet) => return Ok(packet),
}
}
Err(Error::TimedOut)
}
async fn transmit<F>(linkno: u8, f: F) -> Result<(), Error>
where F: FnOnce(&mut [u8]) -> Result<usize, Error> {
let linkno = linkno as usize;
unsafe {
let _ = block_async!(tx_ready(linkno)).await;
let ptr = DRTIOAUX_MEM[linkno].base as *mut u32;
let len = DRTIOAUX_MEM[linkno].size / 2;
// work buffer, works with unaligned mem access
let mut buf: [u8; 1024] = [0; 1024];
let len = f(&mut buf[0..len])?;
copy_work_buffer(buf.as_mut_ptr() as *mut u32, ptr, len as isize);
(DRTIOAUX[linkno].aux_tx_length_write)(len as u16);
(DRTIOAUX[linkno].aux_tx_write)(1);
Ok(())
}
}
pub async fn send(linkno: u8, packet: &Packet) -> Result<(), Error> {
transmit(linkno, |buffer| {
let mut writer = Cursor::new(buffer);
packet.write_to(&mut writer)?;
// Pad till offset 4, insert checksum there
let padding = (12 - (writer.position() % 8)) % 8;
for _ in 0..padding {
writer.write_u8(0)?;
}
let checksum = crc::crc32::checksum_ieee(&writer.get_ref()[0..writer.position()]);
writer.write_u32(checksum)?;
Ok(writer.position())
})
.await
}

589
src/libboard_artiq/src/drtioaux_proto.rs
View File

@ -1,589 +0,0 @@
use core_io::{Error as IoError, Read, Write};
use io::proto::{ProtoRead, ProtoWrite};
/* 512 (max size) - 4 (CRC) - 1 (packet ID) - 1 (destination) - 4 (trace ID) - 1 (last) - 2 (length) */
pub const DMA_TRACE_MAX_SIZE: usize = 499;
#[derive(Debug)]
pub enum Error {
UnknownPacket(u8),
Io(IoError),
}
impl From<IoError> for Error {
fn from(value: IoError) -> Error {
Error::Io(value)
}
}
#[derive(PartialEq, Debug)]
pub enum Packet {
EchoRequest,
EchoReply,
ResetRequest,
ResetAck,
TSCAck,
DestinationStatusRequest {
destination: u8,
},
DestinationDownReply,
DestinationOkReply,
DestinationSequenceErrorReply {
channel: u16,
},
DestinationCollisionReply {
channel: u16,
},
DestinationBusyReply {
channel: u16,
},
RoutingSetPath {
destination: u8,
hops: [u8; 32],
},
RoutingSetRank {
rank: u8,
},
RoutingAck,
MonitorRequest {
destination: u8,
channel: u16,
probe: u8,
},
MonitorReply {
value: u64,
},
InjectionRequest {
destination: u8,
channel: u16,
overrd: u8,
value: u8,
},
InjectionStatusRequest {
destination: u8,
channel: u16,
overrd: u8,
},
InjectionStatusReply {
value: u8,
},
I2cStartRequest {
destination: u8,
busno: u8,
},
I2cRestartRequest {
destination: u8,
busno: u8,
},
I2cStopRequest {
destination: u8,
busno: u8,
},
I2cWriteRequest {
destination: u8,
busno: u8,
data: u8,
},
I2cWriteReply {
succeeded: bool,
ack: bool,
},
I2cReadRequest {
destination: u8,
busno: u8,
ack: bool,
},
I2cReadReply {
succeeded: bool,
data: u8,
},
I2cBasicReply {
succeeded: bool,
},
I2cSwitchSelectRequest {
destination: u8,
busno: u8,
address: u8,
mask: u8,
},
SpiSetConfigRequest {
destination: u8,
busno: u8,
flags: u8,
length: u8,
div: u8,
cs: u8,
},
SpiWriteRequest {
destination: u8,
busno: u8,
data: u32,
},
SpiReadRequest {
destination: u8,
busno: u8,
},
SpiReadReply {
succeeded: bool,
data: u32,
},
SpiBasicReply {
succeeded: bool,
},
DmaAddTraceRequest {
destination: u8,
id: u32,
last: bool,
length: u16,
trace: [u8; DMA_TRACE_MAX_SIZE],
},
DmaAddTraceReply {
succeeded: bool,
},
DmaRemoveTraceRequest {
destination: u8,
id: u32,
},
DmaRemoveTraceReply {
succeeded: bool,
},
DmaPlaybackRequest {
destination: u8,
id: u32,
timestamp: u64,
},
DmaPlaybackReply {
succeeded: bool,
},
DmaPlaybackStatus {
destination: u8,
id: u32,
error: u8,
channel: u32,
timestamp: u64,
},
}
impl Packet {
pub fn read_from<R>(reader: &mut R) -> Result<Self, Error>
where R: Read + ?Sized {
Ok(match reader.read_u8()? {
0x00 => Packet::EchoRequest,
0x01 => Packet::EchoReply,
0x02 => Packet::ResetRequest,
0x03 => Packet::ResetAck,
0x04 => Packet::TSCAck,
0x20 => Packet::DestinationStatusRequest {
destination: reader.read_u8()?,
},
0x21 => Packet::DestinationDownReply,
0x22 => Packet::DestinationOkReply,
0x23 => Packet::DestinationSequenceErrorReply {
channel: reader.read_u16()?,
},
0x24 => Packet::DestinationCollisionReply {
channel: reader.read_u16()?,
},
0x25 => Packet::DestinationBusyReply {
channel: reader.read_u16()?,
},
0x30 => {
let destination = reader.read_u8()?;
let mut hops = [0; 32];
reader.read_exact(&mut hops)?;
Packet::RoutingSetPath {
destination: destination,
hops: hops,
}
}
0x31 => Packet::RoutingSetRank {
rank: reader.read_u8()?,
},
0x32 => Packet::RoutingAck,
0x40 => Packet::MonitorRequest {
destination: reader.read_u8()?,
channel: reader.read_u16()?,
probe: reader.read_u8()?,
},
0x41 => Packet::MonitorReply {
value: reader.read_u64()?,
},
0x50 => Packet::InjectionRequest {
destination: reader.read_u8()?,
channel: reader.read_u16()?,
overrd: reader.read_u8()?,
value: reader.read_u8()?,
},
0x51 => Packet::InjectionStatusRequest {
destination: reader.read_u8()?,
channel: reader.read_u16()?,
overrd: reader.read_u8()?,
},
0x52 => Packet::InjectionStatusReply {
value: reader.read_u8()?,
},
0x80 => Packet::I2cStartRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
},
0x81 => Packet::I2cRestartRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
},
0x82 => Packet::I2cStopRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
},
0x83 => Packet::I2cWriteRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
data: reader.read_u8()?,
},
0x84 => Packet::I2cWriteReply {
succeeded: reader.read_bool()?,
ack: reader.read_bool()?,
},
0x85 => Packet::I2cReadRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
ack: reader.read_bool()?,
},
0x86 => Packet::I2cReadReply {
succeeded: reader.read_bool()?,
data: reader.read_u8()?,
},
0x87 => Packet::I2cBasicReply {
succeeded: reader.read_bool()?,
},
0x88 => Packet::I2cSwitchSelectRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
address: reader.read_u8()?,
mask: reader.read_u8()?,
},
0x90 => Packet::SpiSetConfigRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
flags: reader.read_u8()?,
length: reader.read_u8()?,
div: reader.read_u8()?,
cs: reader.read_u8()?,
},
/* 0x91: was Packet::SpiSetXferRequest */
0x92 => Packet::SpiWriteRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
data: reader.read_u32()?,
},
0x93 => Packet::SpiReadRequest {
destination: reader.read_u8()?,
busno: reader.read_u8()?,
},
0x94 => Packet::SpiReadReply {
succeeded: reader.read_bool()?,
data: reader.read_u32()?,
},
0x95 => Packet::SpiBasicReply {
succeeded: reader.read_bool()?,
},
0xb0 => {
let destination = reader.read_u8()?;
let id = reader.read_u32()?;
let last = reader.read_bool()?;
let length = reader.read_u16()?;
let mut trace: [u8; DMA_TRACE_MAX_SIZE] = [0; DMA_TRACE_MAX_SIZE];
reader.read_exact(&mut trace[0..length as usize])?;
Packet::DmaAddTraceRequest {
destination: destination,
id: id,
last: last,
length: length as u16,
trace: trace,
}
}
0xb1 => Packet::DmaAddTraceReply {
succeeded: reader.read_bool()?,
},
0xb2 => Packet::DmaRemoveTraceRequest {
destination: reader.read_u8()?,
id: reader.read_u32()?,
},
0xb3 => Packet::DmaRemoveTraceReply {
succeeded: reader.read_bool()?,
},
0xb4 => Packet::DmaPlaybackRequest {
destination: reader.read_u8()?,
id: reader.read_u32()?,
timestamp: reader.read_u64()?,
},
0xb5 => Packet::DmaPlaybackReply {
succeeded: reader.read_bool()?,
},
0xb6 => Packet::DmaPlaybackStatus {
destination: reader.read_u8()?,
id: reader.read_u32()?,
error: reader.read_u8()?,
channel: reader.read_u32()?,
timestamp: reader.read_u64()?,
},
ty => return Err(Error::UnknownPacket(ty)),
})
}
pub fn write_to<W>(&self, writer: &mut W) -> Result<(), IoError>
where W: Write + ?Sized {
match *self {
Packet::EchoRequest => writer.write_u8(0x00)?,
Packet::EchoReply => writer.write_u8(0x01)?,
Packet::ResetRequest => writer.write_u8(0x02)?,
Packet::ResetAck => writer.write_u8(0x03)?,
Packet::TSCAck => writer.write_u8(0x04)?,
Packet::DestinationStatusRequest { destination } => {
writer.write_u8(0x20)?;
writer.write_u8(destination)?;
}
Packet::DestinationDownReply => writer.write_u8(0x21)?,
Packet::DestinationOkReply => writer.write_u8(0x22)?,
Packet::DestinationSequenceErrorReply { channel } => {
writer.write_u8(0x23)?;
writer.write_u16(channel)?;
}
Packet::DestinationCollisionReply { channel } => {
writer.write_u8(0x24)?;
writer.write_u16(channel)?;
}
Packet::DestinationBusyReply { channel } => {
writer.write_u8(0x25)?;
writer.write_u16(channel)?;
}
Packet::RoutingSetPath { destination, hops } => {
writer.write_u8(0x30)?;
writer.write_u8(destination)?;
writer.write_all(&hops)?;
}
Packet::RoutingSetRank { rank } => {
writer.write_u8(0x31)?;
writer.write_u8(rank)?;
}
Packet::RoutingAck => writer.write_u8(0x32)?,
Packet::MonitorRequest {
destination,
channel,
probe,
} => {
writer.write_u8(0x40)?;
writer.write_u8(destination)?;
writer.write_u16(channel)?;
writer.write_u8(probe)?;
}
Packet::MonitorReply { value } => {
writer.write_u8(0x41)?;
writer.write_u64(value)?;
}
Packet::InjectionRequest {
destination,
channel,
overrd,
value,
} => {
writer.write_u8(0x50)?;
writer.write_u8(destination)?;
writer.write_u16(channel)?;
writer.write_u8(overrd)?;
writer.write_u8(value)?;
}
Packet::InjectionStatusRequest {
destination,
channel,
overrd,
} => {
writer.write_u8(0x51)?;
writer.write_u8(destination)?;
writer.write_u16(channel)?;
writer.write_u8(overrd)?;
}
Packet::InjectionStatusReply { value } => {
writer.write_u8(0x52)?;
writer.write_u8(value)?;
}
Packet::I2cStartRequest { destination, busno } => {
writer.write_u8(0x80)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
}
Packet::I2cRestartRequest { destination, busno } => {
writer.write_u8(0x81)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
}
Packet::I2cStopRequest { destination, busno } => {
writer.write_u8(0x82)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
}
Packet::I2cWriteRequest {
destination,
busno,
data,
} => {
writer.write_u8(0x83)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
writer.write_u8(data)?;
}
Packet::I2cWriteReply { succeeded, ack } => {
writer.write_u8(0x84)?;
writer.write_bool(succeeded)?;
writer.write_bool(ack)?;
}
Packet::I2cReadRequest {
destination,
busno,
ack,
} => {
writer.write_u8(0x85)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
writer.write_bool(ack)?;
}
Packet::I2cReadReply { succeeded, data } => {
writer.write_u8(0x86)?;
writer.write_bool(succeeded)?;
writer.write_u8(data)?;
}
Packet::I2cBasicReply { succeeded } => {
writer.write_u8(0x87)?;
writer.write_bool(succeeded)?;
}
Packet::I2cSwitchSelectRequest {
destination,
busno,
address,
mask,
} => {
writer.write_u8(0x88)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
writer.write_u8(address)?;
writer.write_u8(mask)?;
}
Packet::SpiSetConfigRequest {
destination,
busno,
flags,
length,
div,
cs,
} => {
writer.write_u8(0x90)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
writer.write_u8(flags)?;
writer.write_u8(length)?;
writer.write_u8(div)?;
writer.write_u8(cs)?;
}
Packet::SpiWriteRequest {
destination,
busno,
data,
} => {
writer.write_u8(0x92)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
writer.write_u32(data)?;
}
Packet::SpiReadRequest { destination, busno } => {
writer.write_u8(0x93)?;
writer.write_u8(destination)?;
writer.write_u8(busno)?;
}
Packet::SpiReadReply { succeeded, data } => {
writer.write_u8(0x94)?;
writer.write_bool(succeeded)?;
writer.write_u32(data)?;
}
Packet::SpiBasicReply { succeeded } => {
writer.write_u8(0x95)?;
writer.write_bool(succeeded)?;
}
Packet::DmaAddTraceRequest {
destination,
id,
last,
trace,
length,
} => {
writer.write_u8(0xb0)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_bool(last)?;
// trace may be broken down to fit within drtio aux memory limit
// will be reconstructed by satellite
writer.write_u16(length)?;
writer.write_all(&trace[0..length as usize])?;
}
Packet::DmaAddTraceReply { succeeded } => {
writer.write_u8(0xb1)?;
writer.write_bool(succeeded)?;
}
Packet::DmaRemoveTraceRequest { destination, id } => {
writer.write_u8(0xb2)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
}
Packet::DmaRemoveTraceReply { succeeded } => {
writer.write_u8(0xb3)?;
writer.write_bool(succeeded)?;
}
Packet::DmaPlaybackRequest {
destination,
id,
timestamp,
} => {
writer.write_u8(0xb4)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_u64(timestamp)?;
}
Packet::DmaPlaybackReply { succeeded } => {
writer.write_u8(0xb5)?;
writer.write_bool(succeeded)?;
}
Packet::DmaPlaybackStatus {
destination,
id,
error,
channel,
timestamp,
} => {
writer.write_u8(0xb6)?;
writer.write_u8(destination)?;
writer.write_u32(id)?;
writer.write_u8(error)?;
writer.write_u32(channel)?;
writer.write_u64(timestamp)?;
}
}
Ok(())
}
}

140
src/libboard_artiq/src/io_expander.rs
View File

@ -1,140 +0,0 @@
use libboard_zynq::i2c;
use log::info;
// Only the bare minimum registers. Bits/IO connections equivalent between IC types.
struct Registers {
// PCA9539 equivalent register names in comments
iodira: u8, // Configuration Port 0
iodirb: u8, // Configuration Port 1
gpioa: u8, // Output Port 0
gpiob: u8, // Output Port 1
}
pub struct IoExpander<'a> {
i2c: &'a mut i2c::I2c,
address: u8,
iodir: [u8; 2],
out_current: [u8; 2],
out_target: [u8; 2],
registers: Registers,
}
impl<'a> IoExpander<'a> {
pub fn new(i2c: &'a mut i2c::I2c, index: u8) -> Result<Self, &'static str> {
// Both expanders on SHARED I2C bus
let mut io_expander = match index {
0 => IoExpander {
i2c,
address: 0x40,
iodir: [0xff; 2],
out_current: [0; 2],
out_target: [0; 2],
registers: Registers {
iodira: 0x00,
iodirb: 0x01,
gpioa: 0x12,
gpiob: 0x13,
},
},
1 => IoExpander {
i2c,
address: 0x42,
iodir: [0xff; 2],
out_current: [0; 2],
out_target: [0; 2],
registers: Registers {
iodira: 0x00,
iodirb: 0x01,
gpioa: 0x12,
gpiob: 0x13,
},
},
_ => return Err("incorrect I/O expander index"),
};
if !io_expander.check_ack()? {
info!("MCP23017 io expander {} not found. Checking for PCA9539.", index);
io_expander.address += 0xa8; // translate to PCA9539 addresses (see schematic)
io_expander.registers = Registers {
iodira: 0x06,
iodirb: 0x07,
gpioa: 0x02,
gpiob: 0x03,
};
if !io_expander.check_ack()? {
return Err("Neither MCP23017 nor PCA9539 io expander found.");
};
}
Ok(io_expander)
}
fn select(&mut self) -> Result<(), &'static str> {
self.i2c.pca954x_select(0x70, None)?;
self.i2c.pca954x_select(0x71, Some(3))?;
Ok(())
}
fn write(&mut self, addr: u8, value: u8) -> Result<(), &'static str> {
self.i2c.start()?;
self.i2c.write(self.address)?;
self.i2c.write(addr)?;
self.i2c.write(value)?;
self.i2c.stop()?;
Ok(())
}
fn check_ack(&mut self) -> Result<bool, &'static str> {
// Check for ack from io expander
self.select()?;
self.i2c.start()?;
let ack = self.i2c.write(self.address)?;
self.i2c.stop()?;
Ok(ack)
}
fn update_iodir(&mut self) -> Result<(), &'static str> {
self.write(self.registers.iodira, self.iodir[0])?;
self.write(self.registers.iodirb, self.iodir[1])?;
Ok(())
}
pub fn init(&mut self) -> Result<(), &'static str> {
self.select()?;
self.update_iodir()?;
self.out_current[0] = 0x00;
self.write(self.registers.gpioa, 0x00)?;
self.out_current[1] = 0x00;
self.write(self.registers.gpiob, 0x00)?;
Ok(())
}
pub fn set_oe(&mut self, port: u8, outputs: u8) -> Result<(), &'static str> {
self.iodir[port as usize] &= !outputs;
self.update_iodir()?;
Ok(())
}
pub fn set(&mut self, port: u8, bit: u8, high: bool) {
if high {
self.out_target[port as usize] |= 1 << bit;
} else {
self.out_target[port as usize] &= !(1 << bit);
}
}
pub fn service(&mut self) -> Result<(), &'static str> {
if self.out_target != self.out_current {
self.select()?;
if self.out_target[0] != self.out_current[0] {
self.write(self.registers.gpioa, self.out_target[0])?;
self.out_current[0] = self.out_target[0];
}
if self.out_target[1] != self.out_current[1] {
self.write(self.registers.gpiob, self.out_target[1])?;
self.out_current[1] = self.out_target[1];
}
}
Ok(())
}
}

48
src/libboard_artiq/src/lib.rs
View File

@ -1,48 +0,0 @@
#![no_std]
#![feature(never_type)]
extern crate core_io;
extern crate crc;
extern crate embedded_hal;
extern crate io;
extern crate libasync;
extern crate libboard_zynq;
extern crate libconfig;
extern crate libcortex_a9;
extern crate libregister;
extern crate log;
extern crate log_buffer;
pub mod drtio_routing;
#[cfg(has_drtio)]
pub mod drtioaux;
#[cfg(has_drtio)]
pub mod drtioaux_async;
pub mod drtioaux_proto;
#[cfg(feature = "target_kasli_soc")]
pub mod io_expander;
pub mod logger;
#[cfg(has_drtio)]
#[rustfmt::skip]
#[path = "../../../build/mem.rs"]
pub mod mem;
#[rustfmt::skip]
#[path = "../../../build/pl.rs"]
pub mod pl;
#[cfg(has_si5324)]
pub mod si5324;
use core::{cmp, str};
pub fn identifier_read(buf: &mut [u8]) -> &str {
unsafe {
pl::csr::identifier::address_write(0);
let len = pl::csr::identifier::data_read();
let len = cmp::min(len, buf.len() as u8);
for i in 0..len {
pl::csr::identifier::address_write(1 + i);
buf[i as usize] = pl::csr::identifier::data_read();
}
str::from_utf8_unchecked(&buf[..len as usize])
}
}

362
src/libboard_artiq/src/si5324.rs
View File

@ -1,362 +0,0 @@
use core::result;
use embedded_hal::blocking::delay::DelayUs;
use libboard_zynq::{i2c::I2c, time::Milliseconds, timer::GlobalTimer};
use log::info;
#[cfg(not(si5324_soft_reset))]
use crate::pl::csr;
type Result<T> = result::Result<T, &'static str>;
const ADDRESS: u8 = 0x68;
#[cfg(not(si5324_soft_reset))]
fn hard_reset(timer: &mut GlobalTimer) {
unsafe {
csr::si5324_rst_n::out_write(0);
}
timer.delay_us(1_000);
unsafe {
csr::si5324_rst_n::out_write(1);
}
timer.delay_us(10_000);
}
// NOTE: the logical parameters DO NOT MAP to physical values written
// into registers. They have to be mapped; see the datasheet.
// DSPLLsim reports the logical parameters in the design summary, not
// the physical register values.
pub struct FrequencySettings {
pub n1_hs: u8,
pub nc1_ls: u32,
pub n2_hs: u8,
pub n2_ls: u32,
pub n31: u32,
pub n32: u32,
pub bwsel: u8,
pub crystal_as_ckin2: bool,
}
pub enum Input {
Ckin1,
Ckin2,
}
fn map_frequency_settings(settings: &FrequencySettings) -> Result<FrequencySettings> {
if settings.nc1_ls != 0 && (settings.nc1_ls % 2) == 1 {
return Err("NC1_LS must be 0 or even");
}
if settings.nc1_ls > (1 << 20) {
return Err("NC1_LS is too high");
}
if (settings.n2_ls % 2) == 1 {
return Err("N2_LS must be even");
}
if settings.n2_ls > (1 << 20) {
return Err("N2_LS is too high");
}
if settings.n31 > (1 << 19) {
return Err("N31 is too high");
}
if settings.n32 > (1 << 19) {
return Err("N32 is too high");
}
let r = FrequencySettings {
n1_hs: match settings.n1_hs {
4 => 0b000,
5 => 0b001,
6 => 0b010,
7 => 0b011,
8 => 0b100,
9 => 0b101,
10 => 0b110,
11 => 0b111,
_ => return Err("N1_HS has an invalid value"),
},
nc1_ls: settings.nc1_ls - 1,
n2_hs: match settings.n2_hs {
4 => 0b000,
5 => 0b001,
6 => 0b010,
7 => 0b011,
8 => 0b100,
9 => 0b101,
10 => 0b110,
11 => 0b111,
_ => return Err("N2_HS has an invalid value"),
},
n2_ls: settings.n2_ls - 1,
n31: settings.n31 - 1,
n32: settings.n32 - 1,
bwsel: settings.bwsel,
crystal_as_ckin2: settings.crystal_as_ckin2,
};
Ok(r)
}
fn write(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address");
}
if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register");
}
if !i2c.write(val).unwrap() {
return Err("Si5324 failed to ack value");
}
i2c.stop().unwrap();
Ok(())
}
#[allow(dead_code)]
fn write_no_ack_value(i2c: &mut I2c, reg: u8, val: u8) -> Result<()> {
i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address");
}
if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register");
}
i2c.write(val).unwrap();
i2c.stop().unwrap();
Ok(())
}
fn read(i2c: &mut I2c, reg: u8) -> Result<u8> {
i2c.start().unwrap();
if !i2c.write(ADDRESS << 1).unwrap() {
return Err("Si5324 failed to ack write address");
}
if !i2c.write(reg).unwrap() {
return Err("Si5324 failed to ack register");
}
i2c.restart().unwrap();
if !i2c.write((ADDRESS << 1) | 1).unwrap() {
return Err("Si5324 failed to ack read address");
}
let val = i2c.read(false).unwrap();
i2c.stop().unwrap();
Ok(val)
}
fn rmw<F>(i2c: &mut I2c, reg: u8, f: F) -> Result<()>
where F: Fn(u8) -> u8 {
let value = read(i2c, reg)?;
write(i2c, reg, f(value))?;
Ok(())
}
fn ident(i2c: &mut I2c) -> Result<u16> {
Ok(((read(i2c, 134)? as u16) << 8) | (read(i2c, 135)? as u16))
}
#[cfg(si5324_soft_reset)]
fn soft_reset(i2c: &mut I2c, timer: &mut GlobalTimer) -> Result<()> {
let val = read(i2c, 136)?;
write_no_ack_value(i2c, 136, val | 0x80)?;
timer.delay_us(10_000);
Ok(())
}
fn has_xtal(i2c: &mut I2c) -> Result<bool> {
Ok((read(i2c, 129)? & 0x01) == 0) // LOSX_INT=0
}
fn has_ckin(i2c: &mut I2c, input: Input) -> Result<bool> {
match input {
Input::Ckin1 => Ok((read(i2c, 129)? & 0x02) == 0), // LOS1_INT=0
Input::Ckin2 => Ok((read(i2c, 129)? & 0x04) == 0), // LOS2_INT=0
}
}
fn locked(i2c: &mut I2c) -> Result<bool> {
Ok((read(i2c, 130)? & 0x01) == 0) // LOL_INT=0
}
fn monitor_lock(i2c: &mut I2c, timer: &mut GlobalTimer) -> Result<()> {
info!("waiting for Si5324 lock...");
let timeout = timer.get_time() + Milliseconds(20_000);
while !locked(i2c)? {
// Yes, lock can be really slow.
if timer.get_time() > timeout {
return Err("Si5324 lock timeout");
}
}
info!(" ...locked");
Ok(())
}
fn init(i2c: &mut I2c, timer: &mut GlobalTimer) -> Result<()> {
#[cfg(not(si5324_soft_reset))]
hard_reset(timer);
#[cfg(feature = "target_kasli_soc")]
{
i2c.pca954x_select(0x70, None)?;
i2c.pca954x_select(0x71, Some(3))?;
}
#[cfg(feature = "target_zc706")]
{
i2c.pca954x_select(0x74, Some(4))?;
}
if ident(i2c)? != 0x0182 {
return Err("Si5324 does not have expected product number");
}
#[cfg(si5324_soft_reset)]
soft_reset(i2c, timer)?;
Ok(())
}
pub fn bypass(i2c: &mut I2c, input: Input, timer: &mut GlobalTimer) -> Result<()> {
let cksel_reg = match input {
Input::Ckin1 => 0b00,
Input::Ckin2 => 0b01,
};
init(i2c, timer)?;
rmw(i2c, 21, |v| v & 0xfe)?; // CKSEL_PIN=0
rmw(i2c, 3, |v| (v & 0x3f) | (cksel_reg << 6))?; // CKSEL_REG
rmw(i2c, 4, |v| (v & 0x3f) | (0b00 << 6))?; // AUTOSEL_REG=b00
rmw(i2c, 6, |v| (v & 0xc0) | 0b111111)?; // SFOUT2_REG=b111 SFOUT1_REG=b111
rmw(i2c, 0, |v| (v & 0xfd) | 0x02)?; // BYPASS_REG=1
Ok(())
}
pub fn setup(i2c: &mut I2c, settings: &FrequencySettings, input: Input, timer: &mut GlobalTimer) -> Result<()> {
let s = map_frequency_settings(settings)?;
let cksel_reg = match input {
Input::Ckin1 => 0b00,
Input::Ckin2 => 0b01,
};
init(i2c, timer)?;
if settings.crystal_as_ckin2 {
rmw(i2c, 0, |v| v | 0x40)?; // FREE_RUN=1
}
rmw(i2c, 2, |v| (v & 0x0f) | (s.bwsel << 4))?;
rmw(i2c, 21, |v| v & 0xfe)?; // CKSEL_PIN=0
rmw(i2c, 3, |v| (v & 0x2f) | (cksel_reg << 6) | 0x10)?; // CKSEL_REG, SQ_ICAL=1
rmw(i2c, 4, |v| (v & 0x3f) | (0b00 << 6))?; // AUTOSEL_REG=b00
rmw(i2c, 6, |v| (v & 0xc0) | 0b111111)?; // SFOUT2_REG=b111 SFOUT1_REG=b111
write(i2c, 25, (s.n1_hs << 5) as u8)?;
write(i2c, 31, (s.nc1_ls >> 16) as u8)?;
write(i2c, 32, (s.nc1_ls >> 8) as u8)?;
write(i2c, 33, (s.nc1_ls) as u8)?;
write(i2c, 34, (s.nc1_ls >> 16) as u8)?; // write to NC2_LS as well
write(i2c, 35, (s.nc1_ls >> 8) as u8)?;
write(i2c, 36, (s.nc1_ls) as u8)?;
write(i2c, 40, (s.n2_hs << 5) as u8 | (s.n2_ls >> 16) as u8)?;
write(i2c, 41, (s.n2_ls >> 8) as u8)?;
write(i2c, 42, (s.n2_ls) as u8)?;
write(i2c, 43, (s.n31 >> 16) as u8)?;
write(i2c, 44, (s.n31 >> 8) as u8)?;
write(i2c, 45, (s.n31) as u8)?;
write(i2c, 46, (s.n32 >> 16) as u8)?;
write(i2c, 47, (s.n32 >> 8) as u8)?;
write(i2c, 48, (s.n32) as u8)?;
rmw(i2c, 137, |v| v | 0x01)?; // FASTLOCK=1
rmw(i2c, 136, |v| v | 0x40)?; // ICAL=1
if !has_xtal(i2c)? {
return Err("Si5324 misses XA/XB signal");
}
if !has_ckin(i2c, input)? {
return Err("Si5324 misses clock input signal");
}
monitor_lock(i2c, timer)?;
Ok(())
}
pub fn select_input(i2c: &mut I2c, input: Input, timer: &mut GlobalTimer) -> Result<()> {
let cksel_reg = match input {
Input::Ckin1 => 0b00,
Input::Ckin2 => 0b01,
};
rmw(i2c, 3, |v| (v & 0x3f) | (cksel_reg << 6))?;
if !has_ckin(i2c, input)? {
return Err("Si5324 misses clock input signal");
}
monitor_lock(i2c, timer)?;
Ok(())
}
#[cfg(has_siphaser)]
pub mod siphaser {
use super::*;
use crate::pl::csr;
pub fn select_recovered_clock(i2c: &mut I2c, rc: bool, timer: &mut GlobalTimer) -> Result<()> {
let val = read(i2c, 3)?;
write(i2c, 3, (val & 0xdf) | (1 << 5))?; // DHOLD=1
unsafe {
csr::siphaser::switch_clocks_write(if rc { 1 } else { 0 });
}
let val = read(i2c, 3)?;
write(i2c, 3, (val & 0xdf) | (0 << 5))?; // DHOLD=0
monitor_lock(i2c, timer)?;
Ok(())
}
fn phase_shift(direction: u8, timer: &mut GlobalTimer) {
unsafe {
csr::siphaser::phase_shift_write(direction);
while csr::siphaser::phase_shift_done_read() == 0 {}
}
// wait for the Si5324 loop to stabilize
timer.delay_us(500);
}
fn has_error(timer: &mut GlobalTimer) -> bool {
unsafe {
csr::siphaser::error_write(1);
}
timer.delay_us(5_000);
unsafe { csr::siphaser::error_read() != 0 }
}
fn find_edge(target: bool, timer: &mut GlobalTimer) -> Result<u32> {
let mut nshifts = 0;
let mut previous = has_error(timer);
loop {
phase_shift(1, timer);
nshifts += 1;
let current = has_error(timer);
if previous != target && current == target {
return Ok(nshifts);
}
if nshifts > 5000 {
return Err("failed to find timing error edge");
}
previous = current;
}
}
pub fn calibrate_skew(timer: &mut GlobalTimer) -> Result<()> {
let jitter_margin = 32;
let lead = find_edge(false, timer)?;
for _ in 0..jitter_margin {
phase_shift(1, timer);
}
let width = find_edge(true, timer)? + jitter_margin;
// width is 360 degrees (one full rotation of the phase between s/h limits) minus jitter
info!(
"calibration successful, lead: {}, width: {} ({}deg)",
lead,
width,
width * 360 / (56 * 8)
);
// Apply reverse phase shift for half the width to get into the
// middle of the working region.
for _ in 0..width / 2 {
phase_shift(0, timer);
}
Ok(())
}
}

8
src/libbuild_zynq/Cargo.toml
View File

@ -1,8 +0,0 @@
[package]
authors = ["M-Labs"]
name = "build_zynq"
version = "0.0.0"
[lib]
name = "build_zynq"
path = "lib.rs"

29
src/libbuild_zynq/lib.rs
View File

@ -1,29 +0,0 @@
use std::{env,
fs::File,
io::{BufRead, BufReader, Write},
path::PathBuf};
pub fn add_linker_script() {
// Put the linker script somewhere the linker can find it
let out = &PathBuf::from(env::var_os("OUT_DIR").unwrap());
File::create(out.join("link.x"))
.unwrap()
.write_all(include_bytes!("link.x"))
.unwrap();
println!("cargo:rustc-link-search={}", out.display());
// Only re-run the build script when link.x is changed,
// instead of when any part of the source code changes.
println!("cargo:rerun-if-changed=link.x");
}
pub fn cfg() {
// Handle rustc-cfg file
let cfg_path = "../../build/rustc-cfg";
println!("cargo:rerun-if-changed={}", cfg_path);
let f = BufReader::new(File::open(cfg_path).unwrap());
for line in f.lines() {
println!("cargo:rustc-cfg={}", line.unwrap());
}
}

7
src/libc/build.rs
View File

@ -4,7 +4,8 @@ fn main() {
} }
mod libc { mod libc {
use std::{env, path::Path}; use std::path::Path;
use std::env;
pub fn compile() { pub fn compile() {
let cfg = &mut cc::Build::new(); let cfg = &mut cc::Build::new();
@ -31,7 +32,9 @@ mod libc {
cfg.flag("-U_FORTIFY_SOURCE"); cfg.flag("-U_FORTIFY_SOURCE");
cfg.define("_FORTIFY_SOURCE", Some("0")); cfg.define("_FORTIFY_SOURCE", Some("0"));
let sources = vec!["printf.c"]; let sources = vec![
"printf.c"
];
let root = Path::new("./"); let root = Path::new("./");
for src in sources { for src in sources {

1
src/libdwarf/Cargo.toml
View File

@ -15,4 +15,3 @@ libc = { path = "../libc" }
unwind = { path = "../libunwind" } unwind = { path = "../libunwind" }
compiler_builtins = "0.1.0" compiler_builtins = "0.1.0"
cfg-if = "0.1.8" cfg-if = "0.1.8"
cslice = "0.3"

146
src/libdwarf/src/eh.rs
View File

@ -11,11 +11,8 @@
#![allow(non_upper_case_globals)] #![allow(non_upper_case_globals)]
#![allow(unused)] #![allow(unused)]
use core::mem;
use cslice::CSlice;
use crate::DwarfReader; use crate::DwarfReader;
use core::mem;
pub const DW_EH_PE_omit: u8 = 0xFF; pub const DW_EH_PE_omit: u8 = 0xFF;
pub const DW_EH_PE_absptr: u8 = 0x00; pub const DW_EH_PE_absptr: u8 = 0x00;
@ -54,48 +51,10 @@ pub enum EHAction {
pub const USING_SJLJ_EXCEPTIONS: bool = cfg!(all(target_os = "ios", target_arch = "arm")); pub const USING_SJLJ_EXCEPTIONS: bool = cfg!(all(target_os = "ios", target_arch = "arm"));
fn size_of_encoded_value(encoding: u8) -> usize {
if encoding == DW_EH_PE_omit {
0
} else {
let encoding = encoding & 0x07;
match encoding {
DW_EH_PE_absptr => core::mem::size_of::<*const ()>(),
DW_EH_PE_udata2 => 2,
DW_EH_PE_udata4 => 4,
DW_EH_PE_udata8 => 8,
_ => unreachable!(),
}
}
}
unsafe fn get_ttype_entry(
offset: usize,
encoding: u8,
ttype_base: usize,
ttype: *const u8,
) -> Result<Option<*const u8>, ()> {
let i = (offset * size_of_encoded_value(encoding)) as isize;
read_encoded_pointer_with_base(
&mut DwarfReader::new(ttype.offset(-i)),
// the DW_EH_PE_pcrel is a hack.
// It seems that the default encoding is absolute, but we have to take reallocation into
// account. Unsure if we can fix this in the compiler setting or if this would be affected
// by updating the compiler
encoding | DW_EH_PE_pcrel,
ttype_base,
)
.map(|v| match v {
ttype_base => None,
ttype_entry => Some(ttype_entry as *const u8),
})
}
pub unsafe fn find_eh_action( pub unsafe fn find_eh_action(
lsda: *const u8, lsda: *const u8,
context: &EHContext<'_>, context: &EHContext<'_>,
foreign_exception: bool, foreign_exception: bool,
id: u32,
) -> Result<EHAction, ()> { ) -> Result<EHAction, ()> {
if lsda.is_null() { if lsda.is_null() {
return Ok(EHAction::None); return Ok(EHAction::None);
@ -113,17 +72,10 @@ pub unsafe fn find_eh_action(
}; };
let ttype_encoding = reader.read::<u8>(); let ttype_encoding = reader.read::<u8>();
// we do care about the type table if ttype_encoding != DW_EH_PE_omit {
let ttype_offset = if ttype_encoding != DW_EH_PE_omit { // Rust doesn't analyze exception types, so we don't care about the type table
reader.read_uleb128() reader.read_uleb128();
} else { }
0
};
// for rust functions, it seems that there is no type table, so I just put whatever value here.
// we should not return an error, otherwise we would abort unwinding and cannot unwind through
// rust functions
let ttype_base = get_base(ttype_encoding, context).unwrap_or(1);
let ttype_table = reader.ptr.offset(ttype_offset as isize);
let call_site_encoding = reader.read::<u8>(); let call_site_encoding = reader.read::<u8>();
let call_site_table_length = reader.read_uleb128(); let call_site_table_length = reader.read_uleb128();
@ -142,49 +94,11 @@ pub unsafe fn find_eh_action(
break; break;
} }
if ip < func_start + cs_start + cs_len { if ip < func_start + cs_start + cs_len {
// https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/libsupc%2B%2B/eh_personality.cc#L528
let lpad = lpad_base + cs_lpad;
if cs_lpad == 0 { if cs_lpad == 0 {
// no cleanups/handler
return Ok(EHAction::None);
} else if cs_action == 0 {
return Ok(EHAction::Cleanup(lpad));
} else if foreign_exception {
return Ok(EHAction::None); return Ok(EHAction::None);
} else { } else {
let mut saw_cleanup = false; let lpad = lpad_base + cs_lpad;
let mut action_record = action_table.offset(cs_action as isize - 1); return Ok(interpret_cs_action(cs_action, lpad, foreign_exception));
loop {
let mut reader = DwarfReader::new(action_record);
let ar_filter = reader.read_sleb128();
action_record = reader.ptr;
let ar_disp = reader.read_sleb128();
if ar_filter == 0 {
saw_cleanup = true;
} else if ar_filter > 0 {
let catch_type =
get_ttype_entry(ar_filter as usize, ttype_encoding, ttype_base, ttype_table)?;
match catch_type {
Some(clause_ptr) if *(clause_ptr as *const u32) == id => {
return Ok(EHAction::Catch(lpad));
}
None => return Ok(EHAction::Catch(lpad)),
_ => {}
}
} else if ar_filter < 0 {
// FIXME: how to handle this?
break;
}
if ar_disp == 0 {
break;
}
action_record = action_record.offset((ar_disp as usize) as isize);
}
if saw_cleanup {
return Ok(EHAction::Cleanup(lpad));
} else {
return Ok(EHAction::None);
}
} }
} }
} }
@ -192,7 +106,7 @@ pub unsafe fn find_eh_action(
// So rather than returning EHAction::Terminate, we do this. // So rather than returning EHAction::Terminate, we do this.
Ok(EHAction::None) Ok(EHAction::None)
} else { } else {
// SjLj version: (not yet modified) // SjLj version:
// The "IP" is an index into the call-site table, with two exceptions: // The "IP" is an index into the call-site table, with two exceptions:
// -1 means 'no-action', and 0 means 'terminate'. // -1 means 'no-action', and 0 means 'terminate'.
match ip as isize { match ip as isize {
@ -232,33 +146,18 @@ fn interpret_cs_action(cs_action: u64, lpad: usize, foreign_exception: bool) ->
#[inline] #[inline]
fn round_up(unrounded: usize, align: usize) -> Result<usize, ()> { fn round_up(unrounded: usize, align: usize) -> Result<usize, ()> {
if align.is_power_of_two() { if align.is_power_of_two() { Ok((unrounded + align - 1) & !(align - 1)) } else { Err(()) }
Ok((unrounded + align - 1) & !(align - 1))
} else {
Err(())
}
} }
fn get_base(encoding: u8, context: &EHContext<'_>) -> Result<usize, ()> { unsafe fn read_encoded_pointer(
match encoding & 0x70 { reader: &mut DwarfReader,
DW_EH_PE_absptr | DW_EH_PE_pcrel | DW_EH_PE_aligned => Ok(0), context: &EHContext<'_>,
DW_EH_PE_textrel => Ok((*context.get_text_start)()), encoding: u8,
DW_EH_PE_datarel => Ok((*context.get_data_start)()), ) -> Result<usize, ()> {
DW_EH_PE_funcrel if context.func_start != 0 => Ok(context.func_start),
_ => return Err(()),
}
}
unsafe fn read_encoded_pointer(reader: &mut DwarfReader, context: &EHContext<'_>, encoding: u8) -> Result<usize, ()> {
read_encoded_pointer_with_base(reader, encoding, get_base(encoding, context)?)
}
unsafe fn read_encoded_pointer_with_base(reader: &mut DwarfReader, encoding: u8, base: usize) -> Result<usize, ()> {
if encoding == DW_EH_PE_omit { if encoding == DW_EH_PE_omit {
return Err(()); return Err(());
} }
let original_ptr = reader.ptr;
// DW_EH_PE_aligned implies it's an absolute pointer value // DW_EH_PE_aligned implies it's an absolute pointer value
if encoding == DW_EH_PE_aligned { if encoding == DW_EH_PE_aligned {
reader.ptr = round_up(reader.ptr as usize, mem::size_of::<usize>())? as *const u8; reader.ptr = round_up(reader.ptr as usize, mem::size_of::<usize>())? as *const u8;
@ -278,10 +177,19 @@ unsafe fn read_encoded_pointer_with_base(reader: &mut DwarfReader, encoding: u8,
_ => return Err(()), _ => return Err(()),
}; };
result += if (encoding & 0x70) == DW_EH_PE_pcrel { result += match encoding & 0x70 {
original_ptr as usize DW_EH_PE_absptr => 0,
} else { // relative to address of the encoded value, despite the name
base DW_EH_PE_pcrel => reader.ptr as usize,
DW_EH_PE_funcrel => {
if context.func_start == 0 {
return Err(());
}
context.func_start
}
DW_EH_PE_textrel => (*context.get_text_start)(),
DW_EH_PE_datarel => (*context.get_data_start)(),
_ => return Err(()),
}; };
if encoding & DW_EH_PE_indirect != 0 { if encoding & DW_EH_PE_indirect != 0 {

4
src/libdwarf/src/lib.rs
View File

@ -26,10 +26,6 @@ impl DwarfReader {
DwarfReader { ptr } DwarfReader { ptr }
} }
pub unsafe fn offset(&mut self, offset: isize) {
self.ptr = self.ptr.offset(offset);
}
// DWARF streams are packed, so e.g., a u32 would not necessarily be aligned // DWARF streams are packed, so e.g., a u32 would not necessarily be aligned
// on a 4-byte boundary. This may cause problems on platforms with strict // on a 4-byte boundary. This may cause problems on platforms with strict
// alignment requirements. By wrapping data in a "packed" struct, we are // alignment requirements. By wrapping data in a "packed" struct, we are

198
src/libdyld/src/elf.rs
View File

@ -1451,7 +1451,8 @@ pub const R_AARCH64_TLSDESC_CALL: usize = 569;
pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12: usize = 570; pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12: usize = 570;
pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12_NC: usize = 571; pub const R_AARCH64_TLSLE_LDST128_TPREL_LO12_NC: usize = 571;
pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12: usize = 572; pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12: usize = 572;
pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC: usize = 573; pub const R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC: usize =
573;
pub const R_AARCH64_COPY: usize = 1024; pub const R_AARCH64_COPY: usize = 1024;
pub const R_AARCH64_GLOB_DAT: usize = 1025; pub const R_AARCH64_GLOB_DAT: usize = 1025;
pub const R_AARCH64_JUMP_SLOT: usize = 1026; pub const R_AARCH64_JUMP_SLOT: usize = 1026;
@ -2266,9 +2267,7 @@ pub struct Elf32_Ehdr {
pub e_shstrndx: Elf32_Half, pub e_shstrndx: Elf32_Half,
} }
impl Clone for Elf32_Ehdr { impl Clone for Elf32_Ehdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2289,9 +2288,7 @@ pub struct Elf64_Ehdr {
pub e_shstrndx: Elf64_Half, pub e_shstrndx: Elf64_Half,
} }
impl Clone for Elf64_Ehdr { impl Clone for Elf64_Ehdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2308,9 +2305,7 @@ pub struct Elf32_Shdr {
pub sh_entsize: Elf32_Word, pub sh_entsize: Elf32_Word,
} }
impl Clone for Elf32_Shdr { impl Clone for Elf32_Shdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2327,9 +2322,7 @@ pub struct Elf64_Shdr {
pub sh_entsize: Elf64_Xword, pub sh_entsize: Elf64_Xword,
} }
impl Clone for Elf64_Shdr { impl Clone for Elf64_Shdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2342,9 +2335,7 @@ pub struct Elf32_Sym {
pub st_shndx: Elf32_Section, pub st_shndx: Elf32_Section,
} }
impl Clone for Elf32_Sym { impl Clone for Elf32_Sym {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2357,9 +2348,7 @@ pub struct Elf64_Sym {
pub st_size: Elf64_Xword, pub st_size: Elf64_Xword,
} }
impl Clone for Elf64_Sym { impl Clone for Elf64_Sym {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2368,9 +2357,7 @@ pub struct Elf32_Syminfo {
pub si_flags: Elf32_Half, pub si_flags: Elf32_Half,
} }
impl Clone for Elf32_Syminfo { impl Clone for Elf32_Syminfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2379,9 +2366,7 @@ pub struct Elf64_Syminfo {
pub si_flags: Elf64_Half, pub si_flags: Elf64_Half,
} }
impl Clone for Elf64_Syminfo { impl Clone for Elf64_Syminfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2390,9 +2375,7 @@ pub struct Elf32_Rel {
pub r_info: Elf32_Word, pub r_info: Elf32_Word,
} }
impl Clone for Elf32_Rel { impl Clone for Elf32_Rel {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2401,9 +2384,7 @@ pub struct Elf64_Rel {
pub r_info: Elf64_Xword, pub r_info: Elf64_Xword,
} }
impl Clone for Elf64_Rel { impl Clone for Elf64_Rel {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2413,9 +2394,7 @@ pub struct Elf32_Rela {
pub r_addend: Elf32_Sword, pub r_addend: Elf32_Sword,
} }
impl Clone for Elf32_Rela { impl Clone for Elf32_Rela {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2425,9 +2404,7 @@ pub struct Elf64_Rela {
pub r_addend: Elf64_Sxword, pub r_addend: Elf64_Sxword,
} }
impl Clone for Elf64_Rela { impl Clone for Elf64_Rela {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2442,9 +2419,7 @@ pub struct Elf32_Phdr {
pub p_align: Elf32_Word, pub p_align: Elf32_Word,
} }
impl Clone for Elf32_Phdr { impl Clone for Elf32_Phdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2459,9 +2434,7 @@ pub struct Elf64_Phdr {
pub p_align: Elf64_Xword, pub p_align: Elf64_Xword,
} }
impl Clone for Elf64_Phdr { impl Clone for Elf64_Phdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2476,14 +2449,10 @@ pub union Elf32_Dyn__bindgen_ty_1 {
pub d_ptr: Elf32_Addr, pub d_ptr: Elf32_Addr,
} }
impl Clone for Elf32_Dyn__bindgen_ty_1 { impl Clone for Elf32_Dyn__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_Dyn { impl Clone for Elf32_Dyn {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2498,14 +2467,10 @@ pub union Elf64_Dyn__bindgen_ty_1 {
pub d_ptr: Elf64_Addr, pub d_ptr: Elf64_Addr,
} }
impl Clone for Elf64_Dyn__bindgen_ty_1 { impl Clone for Elf64_Dyn__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf64_Dyn { impl Clone for Elf64_Dyn {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2519,9 +2484,7 @@ pub struct Elf32_Verdef {
pub vd_next: Elf32_Word, pub vd_next: Elf32_Word,
} }
impl Clone for Elf32_Verdef { impl Clone for Elf32_Verdef {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2535,9 +2498,7 @@ pub struct Elf64_Verdef {
pub vd_next: Elf64_Word, pub vd_next: Elf64_Word,
} }
impl Clone for Elf64_Verdef { impl Clone for Elf64_Verdef {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2546,9 +2507,7 @@ pub struct Elf32_Verdaux {
pub vda_next: Elf32_Word, pub vda_next: Elf32_Word,
} }
impl Clone for Elf32_Verdaux { impl Clone for Elf32_Verdaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2557,9 +2516,7 @@ pub struct Elf64_Verdaux {
pub vda_next: Elf64_Word, pub vda_next: Elf64_Word,
} }
impl Clone for Elf64_Verdaux { impl Clone for Elf64_Verdaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2571,9 +2528,7 @@ pub struct Elf32_Verneed {
pub vn_next: Elf32_Word, pub vn_next: Elf32_Word,
} }
impl Clone for Elf32_Verneed { impl Clone for Elf32_Verneed {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2585,9 +2540,7 @@ pub struct Elf64_Verneed {
pub vn_next: Elf64_Word, pub vn_next: Elf64_Word,
} }
impl Clone for Elf64_Verneed { impl Clone for Elf64_Verneed {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2599,9 +2552,7 @@ pub struct Elf32_Vernaux {
pub vna_next: Elf32_Word, pub vna_next: Elf32_Word,
} }
impl Clone for Elf32_Vernaux { impl Clone for Elf32_Vernaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2613,9 +2564,7 @@ pub struct Elf64_Vernaux {
pub vna_next: Elf64_Word, pub vna_next: Elf64_Word,
} }
impl Clone for Elf64_Vernaux { impl Clone for Elf64_Vernaux {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2629,14 +2578,10 @@ pub union Elf32_auxv_t__bindgen_ty_1 {
pub a_val: u32, pub a_val: u32,
} }
impl Clone for Elf32_auxv_t__bindgen_ty_1 { impl Clone for Elf32_auxv_t__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_auxv_t { impl Clone for Elf32_auxv_t {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2650,14 +2595,10 @@ pub union Elf64_auxv_t__bindgen_ty_1 {
pub a_val: u64, pub a_val: u64,
} }
impl Clone for Elf64_auxv_t__bindgen_ty_1 { impl Clone for Elf64_auxv_t__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf64_auxv_t { impl Clone for Elf64_auxv_t {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2667,9 +2608,7 @@ pub struct Elf32_Nhdr {
pub n_type: Elf32_Word, pub n_type: Elf32_Word,
} }
impl Clone for Elf32_Nhdr { impl Clone for Elf32_Nhdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2679,9 +2618,7 @@ pub struct Elf64_Nhdr {
pub n_type: Elf64_Word, pub n_type: Elf64_Word,
} }
impl Clone for Elf64_Nhdr { impl Clone for Elf64_Nhdr {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2693,9 +2630,7 @@ pub struct Elf32_Move {
pub m_stride: Elf32_Half, pub m_stride: Elf32_Half,
} }
impl Clone for Elf32_Move { impl Clone for Elf32_Move {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2707,9 +2642,7 @@ pub struct Elf64_Move {
pub m_stride: Elf64_Half, pub m_stride: Elf64_Half,
} }
impl Clone for Elf64_Move { impl Clone for Elf64_Move {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Copy)] #[derive(Copy)]
@ -2724,9 +2657,7 @@ pub struct Elf32_gptab__bindgen_ty_1 {
pub gt_unused: Elf32_Word, pub gt_unused: Elf32_Word,
} }
impl Clone for Elf32_gptab__bindgen_ty_1 { impl Clone for Elf32_gptab__bindgen_ty_1 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2735,14 +2666,10 @@ pub struct Elf32_gptab__bindgen_ty_2 {
pub gt_bytes: Elf32_Word, pub gt_bytes: Elf32_Word,
} }
impl Clone for Elf32_gptab__bindgen_ty_2 { impl Clone for Elf32_gptab__bindgen_ty_2 {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
impl Clone for Elf32_gptab { impl Clone for Elf32_gptab {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2752,9 +2679,7 @@ pub struct Elf32_RegInfo {
pub ri_gp_value: Elf32_Sword, pub ri_gp_value: Elf32_Sword,
} }
impl Clone for Elf32_RegInfo { impl Clone for Elf32_RegInfo {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2765,9 +2690,7 @@ pub struct Elf_Options {
pub info: Elf32_Word, pub info: Elf32_Word,
} }
impl Clone for Elf_Options { impl Clone for Elf_Options {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2776,9 +2699,7 @@ pub struct Elf_Options_Hw {
pub hwp_flags2: Elf32_Word, pub hwp_flags2: Elf32_Word,
} }
impl Clone for Elf_Options_Hw { impl Clone for Elf_Options_Hw {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2790,9 +2711,7 @@ pub struct Elf32_Lib {
pub l_flags: Elf32_Word, pub l_flags: Elf32_Word,
} }
impl Clone for Elf32_Lib { impl Clone for Elf32_Lib {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
#[repr(C)] #[repr(C)]
#[derive(Debug, Copy)] #[derive(Debug, Copy)]
@ -2804,31 +2723,14 @@ pub struct Elf64_Lib {
pub l_flags: Elf64_Word, pub l_flags: Elf64_Word,
} }
impl Clone for Elf64_Lib { impl Clone for Elf64_Lib {
fn clone(&self) -> Self { fn clone(&self) -> Self { *self }
*self
}
} }
pub type Elf32_Conflict = Elf32_Addr; pub type Elf32_Conflict = Elf32_Addr;
#[repr(C)]
#[derive(Clone, Copy)]
pub struct EXIDX_Entry(u32, u32);
pub fn ELF32_R_SYM(info: Elf32_Word) -> Elf32_Word { pub fn ELF32_R_SYM(info: Elf32_Word) -> Elf32_Word { info >> 8 }
info >> 8 pub fn ELF32_R_TYPE(info: Elf32_Word) -> u8 { info as u8 }
} pub fn ELF32_R_INFO(sym: Elf32_Word, ty: u8) -> Elf32_Word { sym << 8 | ty as Elf32_Word }
pub fn ELF32_R_TYPE(info: Elf32_Word) -> u8 {
info as u8
}
pub fn ELF32_R_INFO(sym: Elf32_Word, ty: u8) -> Elf32_Word {
sym << 8 | ty as Elf32_Word
}
pub fn ELF32_ST_BIND(info: u8) -> u8 { pub fn ELF32_ST_BIND(info: u8) -> u8 { info >> 4 }
info >> 4 pub fn ELF32_ST_TYPE(info: u8) -> u8 { info & 0xf }
} pub fn ELF32_ST_INFO(bind: u8, ty: u8) -> u8 { (bind << 4) | (ty & 0xf) }
pub fn ELF32_ST_TYPE(info: u8) -> u8 {
info & 0xf
}
pub fn ELF32_ST_INFO(bind: u8, ty: u8) -> u8 {
(bind << 4) | (ty & 0xf)
}

54
src/libdyld/src/file.rs
View File

@ -1,8 +1,8 @@
use core::{mem, use core::{mem, ptr, ops::{Deref, Range}};
ops::{Deref, Range}, use super::{
ptr}; Arch,
elf::*,
use super::{elf::*, Arch}; };
fn read_unaligned<T: Copy>(data: &[u8], offset: usize) -> Option<T> { fn read_unaligned<T: Copy>(data: &[u8], offset: usize) -> Option<T> {
if data.len() < offset + mem::size_of::<T>() { if data.len() < offset + mem::size_of::<T>() {
@ -31,40 +31,14 @@ impl<'a> File<'a> {
pub fn arch(&self) -> Option<Arch> { pub fn arch(&self) -> Option<Arch> {
const IDENT_OPENRISC: [u8; EI_NIDENT] = [ const IDENT_OPENRISC: [u8; EI_NIDENT] = [
ELFMAG0, ELFMAG0, ELFMAG1, ELFMAG2, ELFMAG3,
ELFMAG1, ELFCLASS32, ELFDATA2MSB, EV_CURRENT, ELFOSABI_NONE,
ELFMAG2, /* ABI version */ 0, /* padding */ 0, 0, 0, 0, 0, 0, 0
ELFMAG3,
ELFCLASS32,
ELFDATA2MSB,
EV_CURRENT,
ELFOSABI_NONE,
/* ABI version */ 0,
/* padding */ 0,
0,
0,
0,
0,
0,
0,
]; ];
const IDENT_ARM: [u8; EI_NIDENT] = [ const IDENT_ARM: [u8; EI_NIDENT] = [
ELFMAG0, ELFMAG0, ELFMAG1, ELFMAG2, ELFMAG3,
ELFMAG1, ELFCLASS32, ELFDATA2LSB, EV_CURRENT, ELFOSABI_NONE,
ELFMAG2, /* ABI version */ 0, /* padding */ 0, 0, 0, 0, 0, 0, 0
ELFMAG3,
ELFCLASS32,
ELFDATA2LSB,
EV_CURRENT,
ELFOSABI_NONE,
/* ABI version */ 0,
/* padding */ 0,
0,
0,
0,
0,
0,
0,
]; ];
match (self.ehdr.e_ident, self.ehdr.e_machine) { match (self.ehdr.e_ident, self.ehdr.e_machine) {
@ -74,14 +48,16 @@ impl<'a> File<'a> {
} }
} }
pub fn program_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Phdr>> + 'b { pub fn program_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Phdr>> + 'b
{
(0..self.ehdr.e_phnum).map(move |i| { (0..self.ehdr.e_phnum).map(move |i| {
let phdr_off = self.ehdr.e_phoff as usize + mem::size_of::<Elf32_Phdr>() * i as usize; let phdr_off = self.ehdr.e_phoff as usize + mem::size_of::<Elf32_Phdr>() * i as usize;
self.read_unaligned::<Elf32_Phdr>(phdr_off) self.read_unaligned::<Elf32_Phdr>(phdr_off)
}) })
} }
pub fn section_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Shdr>> + 'b { pub fn section_headers<'b>(&'b self) -> impl Iterator<Item = Option<Elf32_Shdr>> + 'b
{
(0..self.ehdr.e_shnum).map(move |i| { (0..self.ehdr.e_shnum).map(move |i| {
let shdr_off = self.ehdr.e_shoff as usize + mem::size_of::<Elf32_Shdr>() * i as usize; let shdr_off = self.ehdr.e_shoff as usize + mem::size_of::<Elf32_Shdr>() * i as usize;
self.read_unaligned::<Elf32_Shdr>(shdr_off) self.read_unaligned::<Elf32_Shdr>(shdr_off)

103
src/libdyld/src/image.rs
View File

@ -1,9 +1,13 @@
use core::{
ops::{Deref, DerefMut, Range},
mem,
slice,
};
use alloc::alloc::{alloc_zeroed, dealloc, Layout, LayoutError}; use alloc::alloc::{alloc_zeroed, dealloc, Layout, LayoutError};
use core::{mem, use super::{
ops::{Deref, DerefMut, Range}, elf::*,
slice}; Error,
};
use super::{elf::*, Error};
pub struct DynamicSection { pub struct DynamicSection {
pub strtab: Range<usize>, pub strtab: Range<usize>,
@ -30,12 +34,17 @@ impl Image {
slice::from_raw_parts_mut(ptr, size) slice::from_raw_parts_mut(ptr, size)
}; };
Ok(Image { layout, data }) Ok(Image {
layout,
data,
})
} }
/// assumes that self.data is properly aligned /// assumes that self.data is properly aligned
pub(crate) fn get_ref<T>(&self, offset: usize) -> Option<&T> pub(crate) fn get_ref<T>(&self, offset: usize) -> Option<&T>
where T: Copy { where
T: Copy,
{
if self.data.len() < offset + mem::size_of::<T>() { if self.data.len() < offset + mem::size_of::<T>() {
None None
} else if (self.data.as_ptr() as usize + offset) & (mem::align_of::<T>() - 1) != 0 { } else if (self.data.as_ptr() as usize + offset) & (mem::align_of::<T>() - 1) != 0 {
@ -57,53 +66,55 @@ impl Image {
unsafe { slice::from_raw_parts(ptr, len) } unsafe { slice::from_raw_parts(ptr, len) }
} }
fn dyn_headers<'a>(&'a self, range: Range<usize>) -> impl Iterator<Item = &'a Elf32_Dyn> + 'a { fn dyn_headers<'a>(&'a self, range: Range<usize>) ->
impl Iterator<Item = &'a Elf32_Dyn> + 'a
{
range range
.step_by(mem::size_of::<Elf32_Dyn>()) .step_by(mem::size_of::<Elf32_Dyn>())
.filter_map(move |offset| self.get_ref::<Elf32_Dyn>(offset)) .filter_map(move |offset| {
self.get_ref::<Elf32_Dyn>(offset)
})
.take_while(|d| unsafe { d.d_un.d_val } as i32 != DT_NULL) .take_while(|d| unsafe { d.d_un.d_val } as i32 != DT_NULL)
} }
pub fn dyn_section(&self, range: Range<usize>) -> Result<DynamicSection, Error> { pub fn dyn_section(&self, range: Range<usize>) -> Result<DynamicSection, Error> {
let (mut strtab_off, mut strtab_sz) = (0, 0); let (mut strtab_off, mut strtab_sz) = (0, 0);
let (mut rel_off, mut rel_sz) = (0, 0); let (mut rel_off, mut rel_sz) = (0, 0);
let (mut rela_off, mut rela_sz) = (0, 0); let (mut rela_off, mut rela_sz) = (0, 0);
let (mut pltrel_off, mut pltrel_sz) = (0, 0); let (mut pltrel_off, mut pltrel_sz) = (0, 0);
let (mut hash_off, mut hash_sz) = (0, 0); let (mut hash_off, mut hash_sz) = (0, 0);
let mut symtab_off = 0; let mut symtab_off = 0;
let mut sym_ent = 0; let mut sym_ent = 0;
let mut rel_ent = 0; let mut rel_ent = 0;
let mut rela_ent = 0; let mut rela_ent = 0;
let mut nbucket = 0; let mut nbucket = 0;
let mut nchain = 0; let mut nchain = 0;
for dyn_header in self.dyn_headers(range) { for dyn_header in self.dyn_headers(range) {
let val = unsafe { dyn_header.d_un.d_val } as usize; let val = unsafe { dyn_header.d_un.d_val } as usize;
match dyn_header.d_tag { match dyn_header.d_tag {
DT_NULL => break, DT_NULL => break,
DT_STRTAB => strtab_off = val, DT_STRTAB => strtab_off = val,
DT_STRSZ => strtab_sz = val, DT_STRSZ => strtab_sz = val,
DT_SYMTAB => symtab_off = val, DT_SYMTAB => symtab_off = val,
DT_SYMENT => sym_ent = val, DT_SYMENT => sym_ent = val,
DT_REL => rel_off = val, DT_REL => rel_off = val,
DT_RELSZ => rel_sz = val, DT_RELSZ => rel_sz = val,
DT_RELENT => rel_ent = val, DT_RELENT => rel_ent = val,
DT_RELA => rela_off = val, DT_RELA => rela_off = val,
DT_RELASZ => rela_sz = val, DT_RELASZ => rela_sz = val,
DT_RELAENT => rela_ent = val, DT_RELAENT => rela_ent = val,
DT_JMPREL => pltrel_off = val, DT_JMPREL => pltrel_off = val,
DT_PLTRELSZ => pltrel_sz = val, DT_PLTRELSZ => pltrel_sz = val,
DT_HASH => { DT_HASH => {
nbucket = *self nbucket = *self.get_ref::<Elf32_Word>(val + 0)
.get_ref::<Elf32_Word>(val + 0)
.ok_or("cannot read hash bucket count")? as usize; .ok_or("cannot read hash bucket count")? as usize;
nchain = *self nchain = *self.get_ref::<Elf32_Word>(val + 4)
.get_ref::<Elf32_Word>(val + 4)
.ok_or("cannot read hash chain count")? as usize; .ok_or("cannot read hash chain count")? as usize;
hash_off = val + 8; hash_off = val + 8;
hash_sz = (nbucket + nchain) * mem::size_of::<Elf32_Word>(); hash_sz = (nbucket + nchain) * mem::size_of::<Elf32_Word>();
} }
_ => (), _ => ()
} }
} }
@ -112,28 +123,28 @@ impl Image {
let symtab_sz = nchain * mem::size_of::<Elf32_Sym>(); let symtab_sz = nchain * mem::size_of::<Elf32_Sym>();
if strtab_off + strtab_sz > self.data.len() { if strtab_off + strtab_sz > self.data.len() {
return Err("invalid strtab offset/size")?; return Err("invalid strtab offset/size")?
} }
if symtab_off + symtab_sz > self.data.len() { if symtab_off + symtab_sz > self.data.len() {
return Err("invalid symtab offset/size")?; return Err("invalid symtab offset/size")?
} }
if sym_ent != mem::size_of::<Elf32_Sym>() { if sym_ent != mem::size_of::<Elf32_Sym>() {
return Err("incorrect symbol entry size")?; return Err("incorrect symbol entry size")?
} }
if rel_off + rel_sz > self.data.len() { if rel_off + rel_sz > self.data.len() {
return Err("invalid rel offset/size")?; return Err("invalid rel offset/size")?
} }
if rel_ent != 0 && rel_ent != mem::size_of::<Elf32_Rel>() { if rel_ent != 0 && rel_ent != mem::size_of::<Elf32_Rel>() {
return Err("incorrect relocation entry size")?; return Err("incorrect relocation entry size")?
} }
if rela_off + rela_sz > self.data.len() { if rela_off + rela_sz > self.data.len() {
return Err("invalid rela offset/size")?; return Err("invalid rela offset/size")?
} }
if rela_ent != 0 && rela_ent != mem::size_of::<Elf32_Rela>() { if rela_ent != 0 && rela_ent != mem::size_of::<Elf32_Rela>() {
return Err("incorrect relocation entry size")?; return Err("incorrect relocation entry size")?
} }
if pltrel_off + pltrel_sz > self.data.len() { if pltrel_off + pltrel_sz > self.data.len() {
return Err("invalid pltrel offset/size")?; return Err("invalid pltrel offset/size")?
} }
Ok(DynamicSection { Ok(DynamicSection {
@ -154,7 +165,7 @@ impl Image {
pub fn write(&self, offset: usize, value: Elf32_Word) -> Result<(), Error> { pub fn write(&self, offset: usize, value: Elf32_Word) -> Result<(), Error> {
if offset + mem::size_of::<Elf32_Addr>() > self.data.len() { if offset + mem::size_of::<Elf32_Addr>() > self.data.len() {
return Err("relocation out of image bounds")?; return Err("relocation out of image bounds")?
} }
let ptr = (self.data.as_ptr() as usize + offset) as *mut Elf32_Addr; let ptr = (self.data.as_ptr() as usize + offset) as *mut Elf32_Addr;

117
src/libdyld/src/lib.rs
View File

@ -1,14 +1,13 @@
#![no_std] #![no_std]
extern crate alloc; extern crate alloc;
extern crate libcortex_a9;
extern crate log; extern crate log;
extern crate libcortex_a9;
use alloc::string::String;
use core::{convert, fmt, ops::Range, str}; use core::{convert, fmt, ops::Range, str};
use alloc::string::String;
use elf::*;
use log::{debug, trace}; use log::{debug, trace};
use elf::*;
pub mod elf; pub mod elf;
mod file; mod file;
@ -22,10 +21,11 @@ pub enum Arch {
OpenRisc, OpenRisc,
} }
#[derive(Debug)] #[derive(Debug)]
pub enum Error { pub enum Error {
Parsing(&'static str), Parsing(&'static str),
Lookup(String), Lookup(String)
} }
impl convert::From<&'static str> for Error { impl convert::From<&'static str> for Error {
@ -37,8 +37,10 @@ impl convert::From<&'static str> for Error {
impl fmt::Display for Error { impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {
&Error::Parsing(desc) => write!(f, "parse error: {}", desc), &Error::Parsing(desc) =>
&Error::Lookup(ref sym) => write!(f, "symbol lookup error: {}", sym), write!(f, "parse error: {}", desc),
&Error::Lookup(ref sym) =>
write!(f, "symbol lookup error: {}", sym),
} }
} }
} }
@ -101,22 +103,20 @@ impl Library {
let mut index = self.hash_bucket()[hash as usize % self.hash_bucket().len()] as usize; let mut index = self.hash_bucket()[hash as usize % self.hash_bucket().len()] as usize;
loop { loop {
if index == STN_UNDEF { if index == STN_UNDEF { return None }
return None;
}
let sym = &self.symtab()[index]; let sym = &self.symtab()[index];
let sym_name_off = sym.st_name as usize; let sym_name_off = sym.st_name as usize;
match self.strtab().get(sym_name_off..sym_name_off + name.len()) { match self.strtab().get(sym_name_off..sym_name_off + name.len()) {
Some(sym_name) if sym_name == name => { Some(sym_name) if sym_name == name => {
if ELF32_ST_BIND(sym.st_info) & STB_GLOBAL == 0 { if ELF32_ST_BIND(sym.st_info) & STB_GLOBAL == 0 {
return None; return None
} }
match sym.st_shndx { match sym.st_shndx {
SHN_UNDEF => return None, SHN_UNDEF => return None,
SHN_ABS => return Some(self.image.ptr() as u32 + sym.st_value), SHN_ABS => return Some(self.image.ptr() as u32 + sym.st_value),
_ => return Some(self.image.ptr() as u32 + sym.st_value), _ => return Some(self.image.ptr() as u32 + sym.st_value)
} }
} }
_ => (), _ => (),
@ -127,16 +127,10 @@ impl Library {
} }
pub fn name_starting_at(&self, offset: usize) -> Result<&[u8], Error> { pub fn name_starting_at(&self, offset: usize) -> Result<&[u8], Error> {
let size = self let size = self.strtab().iter().skip(offset).position(|&x| x == 0)
.strtab() .ok_or("symbol in symbol table not null-terminated")?;
.iter() Ok(self.strtab().get(offset..offset + size)
.skip(offset) .ok_or("cannot read symbol name")?)
.position(|&x| x == 0)
.ok_or("symbol in symbol table not null-terminated")?;
Ok(self
.strtab()
.get(offset..offset + size)
.ok_or("cannot read symbol name")?)
} }
/// Rebind Rela by `name` to a new `addr` /// Rebind Rela by `name` to a new `addr`
@ -144,64 +138,58 @@ impl Library {
reloc::rebind(self.arch, self, name, addr as Elf32_Word) reloc::rebind(self.arch, self, name, addr as Elf32_Word)
} }
pub fn exidx(&self) -> &[EXIDX_Entry] { pub fn exidx(&self) -> &[u32] {
self.image.get_ref_slice_unchecked(&self.exidx) self.image.get_ref_slice_unchecked(&self.exidx)
} }
} }
pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Result<Library, Error> { pub fn load(
data: &[u8],
resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>
) -> Result<Library, Error> {
// validate ELF file // validate ELF file
let file = file::File::new(data).ok_or("cannot read ELF header")?; let file = file::File::new(data)
.ok_or("cannot read ELF header")?;
if file.ehdr.e_type != ET_DYN { if file.ehdr.e_type != ET_DYN {
return Err("not a shared library")?; return Err("not a shared library")?
} }
let arch = file.arch().ok_or("not for a supported architecture")?; let arch = file.arch()
.ok_or("not for a supported architecture")?;
// prepare target memory // prepare target memory
let image_size = file let image_size = file.program_headers()
.program_headers()
.filter_map(|phdr| phdr.map(|phdr| phdr.p_vaddr + phdr.p_memsz)) .filter_map(|phdr| phdr.map(|phdr| phdr.p_vaddr + phdr.p_memsz))
.max() .max()
.unwrap_or(0) as usize; .unwrap_or(0) as usize;
let image_align = file let image_align = file.program_headers()
.program_headers() .filter_map(|phdr| phdr.and_then(|phdr| {
.filter_map(|phdr| { if phdr.p_type == PT_LOAD {
phdr.and_then(|phdr| { Some(phdr.p_align)
if phdr.p_type == PT_LOAD { } else {
Some(phdr.p_align) None
} else { }
None }))
}
})
})
.max() .max()
.unwrap_or(4) as usize; .unwrap_or(4) as usize;
// 1 image for all segments // 1 image for all segments
let mut image = image::Image::new(image_size, image_align).map_err(|_| "cannot allocate target image")?; let mut image = image::Image::new(image_size, image_align)
debug!( .map_err(|_| "cannot allocate target image")?;
"ELF target: {} bytes, align to {:X}, allocated at {:08X}", debug!("ELF target: {} bytes, align to {:X}, allocated at {:08X}", image_size, image_align, image.ptr() as usize);
image_size,
image_align,
image.ptr() as usize
);
// LOAD // LOAD
for phdr in file.program_headers() { for phdr in file.program_headers() {
let phdr = phdr.ok_or("cannot read program header")?; let phdr = phdr.ok_or("cannot read program header")?;
trace!( trace!("Program header: {:08X}+{:08X} to {:08X}",
"Program header: {:08X}+{:08X} to {:08X}", phdr.p_offset, phdr.p_filesz,
phdr.p_offset, image.ptr() as u32
phdr.p_filesz,
image.ptr() as u32
); );
let file_range = phdr.p_offset as usize..(phdr.p_offset + phdr.p_filesz) as usize; let file_range = phdr.p_offset as usize..(phdr.p_offset + phdr.p_filesz) as usize;
match phdr.p_type { match phdr.p_type {
PT_LOAD => { PT_LOAD => {
let src = file let src = file.get(file_range)
.get(file_range)
.ok_or("program header requests an out of bounds load (in file)")?; .ok_or("program header requests an out of bounds load (in file)")?;
let dst = image let dst = image.get_mut(phdr.p_vaddr as usize..
.get_mut(phdr.p_vaddr as usize..(phdr.p_vaddr + phdr.p_filesz) as usize) (phdr.p_vaddr + phdr.p_filesz) as usize)
.ok_or("program header requests an out of bounds load (in target)")?; .ok_or("program header requests an out of bounds load (in target)")?;
dst.copy_from_slice(src); dst.copy_from_slice(src);
} }
@ -215,9 +203,9 @@ pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Resul
let shdr = shdr.ok_or("cannot read section header")?; let shdr = shdr.ok_or("cannot read section header")?;
match shdr.sh_type as usize { match shdr.sh_type as usize {
SHT_ARM_EXIDX => { SHT_ARM_EXIDX => {
let range = shdr.sh_addr as usize..(shdr.sh_addr + shdr.sh_size) as usize; let range = shdr.sh_addr as usize..
let _ = image (shdr.sh_addr + shdr.sh_size) as usize;
.get(range.clone()) let _ = image.get(range.clone())
.ok_or("section header specifies EXIDX outside of image (in target)")?; .ok_or("section header specifies EXIDX outside of image (in target)")?;
exidx = Some(range); exidx = Some(range);
} }
@ -226,14 +214,11 @@ pub fn load(data: &[u8], resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>) -> Resul
} }
// relocate DYNAMIC // relocate DYNAMIC
let dyn_range = file.dyn_header_vaddr().ok_or("cannot find a dynamic header")?; let dyn_range = file.dyn_header_vaddr()
.ok_or("cannot find a dynamic header")?;
let dyn_section = image.dyn_section(dyn_range.clone())?; let dyn_section = image.dyn_section(dyn_range.clone())?;
debug!( debug!("Relocating {} rela, {} rel, {} pltrel",
"Relocating {} rela, {} rel, {} pltrel", dyn_section.rela.len(), dyn_section.rel.len(), dyn_section.pltrel.len());
dyn_section.rela.len(),
dyn_section.rel.len(),
dyn_section.pltrel.len()
);
let lib = Library { let lib = Library {
arch, arch,
image, image,

145
src/libdyld/src/reloc.rs
View File

@ -1,10 +1,16 @@
use alloc::string::String; use alloc::string::String;
use libcortex_a9::{asm::{dsb, isb},
cache::{bpiall, dcci_slice, iciallu}};
use log::trace; use log::trace;
use super::{
use super::{elf::*, image::Image, Arch, Error, Library}; Arch,
elf::*,
Error,
image::Image,
Library,
};
use libcortex_a9::{
cache::{dcci_slice, iciallu, bpiall},
asm::{dsb, isb},
};
pub trait Relocatable { pub trait Relocatable {
fn offset(&self) -> usize; fn offset(&self) -> usize;
@ -53,25 +59,29 @@ impl Relocatable for Elf32_Rela {
enum RelType { enum RelType {
None, None,
Relative, Relative,
LookupAbs, Lookup,
LookupRel,
} }
impl RelType { impl RelType {
pub fn new(arch: Arch, type_info: u8) -> Option<Self> { pub fn new(arch: Arch, type_info: u8) -> Option<Self> {
match type_info { match type_info {
R_OR1K_NONE if arch == Arch::OpenRisc => Some(RelType::None), R_OR1K_NONE if arch == Arch::OpenRisc =>
R_ARM_NONE if arch == Arch::Arm => Some(RelType::None), Some(RelType::None),
R_ARM_NONE if arch == Arch::Arm =>
Some(RelType::None),
R_OR1K_RELATIVE if arch == Arch::OpenRisc => Some(RelType::Relative), R_OR1K_RELATIVE if arch == Arch::OpenRisc =>
R_ARM_RELATIVE if arch == Arch::Arm => Some(RelType::Relative), Some(RelType::Relative),
R_ARM_RELATIVE if arch == Arch::Arm =>
Some(RelType::Relative),
R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT if arch == Arch::OpenRisc => Some(RelType::LookupAbs), R_OR1K_32 | R_OR1K_GLOB_DAT | R_OR1K_JMP_SLOT
R_ARM_GLOB_DAT | R_ARM_JUMP_SLOT | R_ARM_ABS32 if arch == Arch::Arm => Some(RelType::LookupAbs), if arch == Arch::OpenRisc => Some(RelType::Lookup),
R_ARM_GLOB_DAT | R_ARM_JUMP_SLOT
if arch == Arch::Arm => Some(RelType::Lookup),
R_ARM_PREL31 if arch == Arch::Arm => Some(RelType::LookupRel), _ =>
None
_ => None,
} }
} }
} }
@ -83,108 +93,71 @@ fn format_sym_name(sym_name: &[u8]) -> String {
} }
pub fn relocate<R: Relocatable>( pub fn relocate<R: Relocatable>(
arch: Arch, arch: Arch, lib: &Library,
lib: &Library, rel: &R, resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>
rel: &R,
resolve: &dyn Fn(&[u8]) -> Option<Elf32_Word>,
) -> Result<(), Error> { ) -> Result<(), Error> {
let sym; let sym;
if rel.sym_info() == 0 { if rel.sym_info() == 0 {
sym = None; sym = None;
} else { } else {
sym = Some( sym = Some(lib.symtab().get(rel.sym_info() as usize)
lib.symtab() .ok_or("symbol out of bounds of symbol table")?)
.get(rel.sym_info() as usize)
.ok_or("symbol out of bounds of symbol table")?,
)
} }
let rel_type = RelType::new(arch, rel.type_info()).ok_or("unsupported relocation type")?; let rel_type = RelType::new(arch, rel.type_info())
let value = match rel_type { .ok_or("unsupported relocation type")?;
RelType::None => return Ok(()), let value;
match rel_type {
RelType::None =>
return Ok(()),
RelType::Relative => { RelType::Relative => {
let addend = rel.addend(&lib.image); let addend = rel.addend(&lib.image);
lib.image.ptr().wrapping_offset(addend as isize) as Elf32_Word value = lib.image.ptr().wrapping_offset(addend as isize) as Elf32_Word;
} }
RelType::LookupAbs | RelType::LookupRel => { RelType::Lookup => {
let sym = sym.ok_or("relocation requires an associated symbol")?; let sym = sym.ok_or("relocation requires an associated symbol")?;
let sym_name = lib.name_starting_at(sym.st_name as usize)?; let sym_name = lib.name_starting_at(sym.st_name as usize)?;
let sym_addr = if let Some(addr) = lib.lookup(sym_name) { if let Some(addr) = lib.lookup(sym_name) {
// First, try to resolve against itself. // First, try to resolve against itself.
trace!("looked up symbol {} in image", format_sym_name(sym_name)); trace!("looked up symbol {} in image", format_sym_name(sym_name));
addr value = addr;
} else if let Some(addr) = resolve(sym_name) { } else if let Some(addr) = resolve(sym_name) {
// Second, call the user-provided function. // Second, call the user-provided function.
trace!("resolved symbol {:?}", format_sym_name(sym_name)); trace!("resolved symbol {:?}", format_sym_name(sym_name));
addr value = addr;
} else { } else {
// We couldn't find it anywhere. // We couldn't find it anywhere.
return Err(Error::Lookup(format_sym_name(sym_name))); return Err(Error::Lookup(format_sym_name(sym_name)))
};
match rel_type {
RelType::LookupAbs => sym_addr,
RelType::LookupRel => {
sym_addr.wrapping_sub(lib.image.ptr().wrapping_offset(rel.offset() as isize) as Elf32_Addr)
}
_ => unreachable!(),
} }
} }
};
match rel.type_info() {
R_ARM_PREL31 => {
let reloc_word = lib
.image
.get_ref::<Elf32_Word>(rel.offset())
.ok_or("relocation offset cannot be read")?;
lib.image
.write(rel.offset(), (reloc_word & 0x80000000) | (value & 0x7FFFFFFF))
}
_ => lib.image.write(rel.offset(), value),
} }
lib.image.write(rel.offset(), value)
} }
pub fn rebind(arch: Arch, lib: &Library, name: &[u8], value: Elf32_Word) -> Result<(), Error> { pub fn rebind(
fn rebind_symbol_to_value<R: Relocatable>( arch: Arch, lib: &Library, name: &[u8], value: Elf32_Word
arch: Arch, ) -> Result<(), Error> {
lib: &Library, for rela in lib.pltrel() {
name: &[u8], let rel_type = RelType::new(arch, rela.type_info())
value: Elf32_Word, .ok_or("unsupported relocation type")?;
relocs: &[R], match rel_type {
) -> Result<(), Error> { RelType::Lookup => {
for reloc in relocs { let sym = lib.symtab().get(ELF32_R_SYM(rela.r_info) as usize)
let rel_type = RelType::new(arch, reloc.type_info()).ok_or("unsupported relocation type")?; .ok_or("symbol out of bounds of symbol table")?;
match rel_type { let sym_name = lib.name_starting_at(sym.st_name as usize)?;
RelType::LookupAbs => {
let sym = lib
.symtab()
.get(reloc.sym_info() as usize)
.ok_or("symbol out of bounds of symbol table")?;
let sym_name = lib.name_starting_at(sym.st_name as usize)?;
if sym_name == name { if sym_name == name {
lib.image.write(reloc.offset(), value)? lib.image.write(rela.offset(), value)?
}
} }
// No associated symbols for other relocation types.
_ => {}
} }
// No associated symbols for other relocation types.
_ => {}
} }
Ok(())
} }
if lib.pltrel().is_empty() {
rebind_symbol_to_value(arch, lib, name, value, lib.rela())?;
} else {
rebind_symbol_to_value(arch, lib, name, value, lib.pltrel())?;
}
// FIXME: the cache maintainance operations may be more than enough, // FIXME: the cache maintainance operations may be more than enough,
// may cause performance degradation. // may cause performance degradation.
dcci_slice(lib.image.data); dcci_slice(lib.image.data);

17
src/libio/Cargo.toml
View File

@ -1,17 +0,0 @@
[package]
authors = ["M-Labs"]
name = "io"
version = "0.0.0"
[lib]
name = "io"
path = "lib.rs"
[dependencies]
core_io = { version = "0.1", features = ["collections"] }
byteorder = { version = "1.0", default-features = false, optional = true }
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
[features]
alloc = []

78
src/libio/cursor.rs
View File

@ -1,78 +0,0 @@
use core_io::{Error as IoError, Read, Write};
#[derive(Debug, Clone)]
pub struct Cursor<T> {
inner: T,
pos: usize,
}
impl<T> Cursor<T> {
#[inline]
pub fn new(inner: T) -> Cursor<T> {
Cursor { inner, pos: 0 }
}
#[inline]
pub fn into_inner(self) -> T {
self.inner
}
#[inline]
pub fn get_ref(&self) -> &T {
&self.inner
}
#[inline]
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
#[inline]
pub fn position(&self) -> usize {
self.pos
}
#[inline]
pub fn set_position(&mut self, pos: usize) {
self.pos = pos
}
}
impl<T: AsRef<[u8]>> Read for Cursor<T> {
fn read(&mut self, buf: &mut [u8]) -> Result<usize, IoError> {
let data = &self.inner.as_ref()[self.pos..];
let len = buf.len().min(data.len());
buf[..len].copy_from_slice(&data[..len]);
self.pos += len;
Ok(len)
}
}
impl Write for Cursor<&mut [u8]> {
fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> {
let data = &mut self.inner[self.pos..];
let len = buf.len().min(data.len());
data[..len].copy_from_slice(&buf[..len]);
self.pos += len;
Ok(len)
}
#[inline]
fn flush(&mut self) -> Result<(), IoError> {
Ok(())
}
}
#[cfg(feature = "alloc")]
impl Write for Cursor<::alloc::Vec<u8>> {
#[inline]
fn write(&mut self, buf: &[u8]) -> Result<usize, IoError> {
self.inner.extend_from_slice(buf);
Ok(buf.len())
}
#[inline]
fn flush(&mut self) -> Result<(), IoError> {
Ok(())
}
}

22
src/libio/lib.rs
View File

@ -1,22 +0,0 @@
#![no_std]
#![feature(never_type)]
#![cfg_attr(feature = "alloc", feature(alloc))]
extern crate alloc;
extern crate core_io;
#[cfg(feature = "alloc")]
#[macro_use]
use alloc;
#[cfg(feature = "byteorder")]
extern crate byteorder;
pub mod cursor;
#[cfg(feature = "byteorder")]
pub mod proto;
pub use cursor::Cursor;
#[cfg(all(feature = "byteorder", feature = "alloc"))]
pub use proto::ReadStringError;
#[cfg(feature = "byteorder")]
pub use proto::{ProtoRead, ProtoWrite};

18
src/libunwind/backtrace.rs
View File

@ -1,25 +1,27 @@
use libc::{c_int, c_void}; use libc::{c_void, c_int};
use crate::libunwind as uw; use crate::libunwind as uw;
const UW_REG_SP: c_int = 13; const UW_REG_SP: c_int = 13;
pub fn backtrace<F>(f: F) -> Result<(), uw::_Unwind_Reason_Code> pub fn backtrace<F>(f: F) -> Result<(), uw::_Unwind_Reason_Code>
where F: FnMut(usize) -> () { where F: FnMut(usize) -> ()
{
struct TraceContext<F> { struct TraceContext<F> {
step_fn: F, step_fn: F,
prev_sp: uw::_Unwind_Word, prev_sp: uw::_Unwind_Word
} }
extern "C" fn trace<F>(context: *mut uw::_Unwind_Context, arg: *mut c_void) -> uw::_Unwind_Reason_Code extern fn trace<F>(context: *mut uw::_Unwind_Context, arg: *mut c_void)
where F: FnMut(usize) -> () { -> uw::_Unwind_Reason_Code
where F: FnMut(usize) -> ()
{
unsafe { unsafe {
let trace_context = &mut *(arg as *mut TraceContext<F>); let trace_context = &mut *(arg as *mut TraceContext<F>);
// Detect the root of a libfringe thread // Detect the root of a libfringe thread
let cur_sp = uw::_Unwind_GetGR(context, UW_REG_SP); let cur_sp = uw::_Unwind_GetGR(context, UW_REG_SP);
if cur_sp == trace_context.prev_sp { if cur_sp == trace_context.prev_sp {
return uw::_URC_END_OF_STACK; return uw::_URC_END_OF_STACK
} else { } else {
trace_context.prev_sp = cur_sp; trace_context.prev_sp = cur_sp;
} }
@ -33,7 +35,7 @@ where F: FnMut(usize) -> () {
let mut trace_context = TraceContext { step_fn: f, prev_sp: 0 }; let mut trace_context = TraceContext { step_fn: f, prev_sp: 0 };
match uw::_Unwind_Backtrace(trace::<F>, &mut trace_context as *mut _ as *mut c_void) { match uw::_Unwind_Backtrace(trace::<F>, &mut trace_context as *mut _ as *mut c_void) {
uw::_URC_NO_REASON => Ok(()), uw::_URC_NO_REASON => Ok(()),
err => Err(err), err => Err(err)
} }
} }
} }

10
src/libunwind/build.rs
View File

@ -5,7 +5,8 @@ fn main() {
} }
mod llvm_libunwind { mod llvm_libunwind {
use std::{env, path::Path}; use std::path::Path;
use std::env;
fn setup_options(cfg: &mut cc::Build) { fn setup_options(cfg: &mut cc::Build) {
cfg.no_default_flags(true); cfg.no_default_flags(true);
@ -22,7 +23,6 @@ mod llvm_libunwind {
cfg.flag("-fno-stack-protector"); cfg.flag("-fno-stack-protector");
cfg.flag("--target=armv7-none-eabihf"); cfg.flag("--target=armv7-none-eabihf");
cfg.flag("-O2"); cfg.flag("-O2");
cfg.flag("-flto");
cfg.flag("-std=c99"); cfg.flag("-std=c99");
cfg.flag("-fstrict-aliasing"); cfg.flag("-fstrict-aliasing");
@ -81,7 +81,11 @@ mod llvm_libunwind {
cfg.flag("-fvisibility=hidden"); cfg.flag("-fvisibility=hidden");
cfg.flag_if_supported("-fvisibility-global-new-delete-hidden"); cfg.flag_if_supported("-fvisibility-global-new-delete-hidden");
let unwind_sources = vec!["Unwind-EHABI.cpp", "Unwind-seh.cpp", "libunwind.cpp"]; let unwind_sources = vec![
"Unwind-EHABI.cpp",
"Unwind-seh.cpp",
"libunwind.cpp"
];
let root = Path::new("../llvm_libunwind"); let root = Path::new("../llvm_libunwind");
cfg.include(root.join("include")); cfg.include(root.join("include"));

6
src/libunwind/libunwind.rs
View File

@ -21,7 +21,8 @@ pub use _Unwind_Reason_Code::*;
pub type _Unwind_Exception_Class = u64; pub type _Unwind_Exception_Class = u64;
pub type _Unwind_Word = uintptr_t; pub type _Unwind_Word = uintptr_t;
pub type _Unwind_Ptr = uintptr_t; pub type _Unwind_Ptr = uintptr_t;
pub type _Unwind_Trace_Fn = extern "C" fn(ctx: *mut _Unwind_Context, arg: *mut c_void) -> _Unwind_Reason_Code; pub type _Unwind_Trace_Fn =
extern "C" fn(ctx: *mut _Unwind_Context, arg: *mut c_void) -> _Unwind_Reason_Code;
#[cfg(target_arch = "x86")] #[cfg(target_arch = "x86")]
pub const unwinder_private_data_size: usize = 5; pub const unwinder_private_data_size: usize = 5;
@ -278,6 +279,7 @@ if #[cfg(all(windows, target_arch = "x86_64", target_env = "gnu"))] {
} // cfg_if! } // cfg_if!
#[no_mangle] #[no_mangle]
extern "C" fn abort() { extern fn abort() {
panic!("Abort!"); panic!("Abort!");
} }

9
src/llvm_libunwind/include/unwind.h
View File

@ -107,12 +107,9 @@ struct _Unwind_Control_Block {
} __attribute__((__aligned__(8))); } __attribute__((__aligned__(8)));
typedef _Unwind_Reason_Code (*_Unwind_Stop_Fn) typedef _Unwind_Reason_Code (*_Unwind_Stop_Fn)
(int version, (_Unwind_State state,
_Unwind_Action actions, _Unwind_Exception* exceptionObject,
uint64_t exceptionClass, struct _Unwind_Context* context);
_Unwind_Exception* exceptionObject,
struct _Unwind_Context* context,
void* stop_parameter);
typedef _Unwind_Reason_Code (*__personality_routine) typedef _Unwind_Reason_Code (*__personality_routine)
(_Unwind_State state, (_Unwind_State state,

170
src/llvm_libunwind/src/Unwind-EHABI.cpp
View File

@ -95,11 +95,9 @@ _Unwind_Reason_Code ProcessDescriptors(
case Descriptor::LU32: case Descriptor::LU32:
descriptor = getNextWord(descriptor, &length); descriptor = getNextWord(descriptor, &length);
descriptor = getNextWord(descriptor, &offset); descriptor = getNextWord(descriptor, &offset);
break;
case Descriptor::LU16: case Descriptor::LU16:
descriptor = getNextNibble(descriptor, &length); descriptor = getNextNibble(descriptor, &length);
descriptor = getNextNibble(descriptor, &offset); descriptor = getNextNibble(descriptor, &offset);
break;
default: default:
assert(false); assert(false);
return _URC_FAILURE; return _URC_FAILURE;
@ -185,14 +183,8 @@ static _Unwind_Reason_Code unwindOneFrame(_Unwind_State state,
if (result != _URC_CONTINUE_UNWIND) if (result != _URC_CONTINUE_UNWIND)
return result; return result;
switch (__unw_step(reinterpret_cast<unw_cursor_t *>(context))) { if (__unw_step(reinterpret_cast<unw_cursor_t *>(context)) != UNW_STEP_SUCCESS)
case UNW_STEP_SUCCESS:
return _URC_CONTINUE_UNWIND;
case UNW_STEP_END:
return _URC_END_OF_STACK;
default:
return _URC_FAILURE; return _URC_FAILURE;
}
return _URC_CONTINUE_UNWIND; return _URC_CONTINUE_UNWIND;
} }
@ -685,128 +677,6 @@ static _Unwind_Reason_Code unwind_phase2(unw_context_t *uc, unw_cursor_t *cursor
return _URC_FATAL_PHASE2_ERROR; return _URC_FATAL_PHASE2_ERROR;
} }
static _Unwind_Reason_Code
unwind_phase2_forced(unw_context_t *uc, unw_cursor_t *cursor,
_Unwind_Exception *exception_object, _Unwind_Stop_Fn stop,
void *stop_parameter) {
// See comment at the start of unwind_phase1 regarding VRS integrity.
__unw_init_local(cursor, uc);
_LIBUNWIND_TRACE_UNWINDING("unwind_phase2_forced(ex_ojb=%p)",
static_cast<void *>(exception_object));
// Walk each frame until we reach where search phase said to stop.
bool end_of_stack = false;
// TODO: why can't libunwind handle end of stack properly?
// We should fix this kind of hack.
unw_word_t forced_phase2_prev_sp = 0x0;
while (!end_of_stack) {
// Get info about this frame.
unw_word_t sp;
unw_proc_info_t frameInfo;
__unw_get_reg(cursor, UNW_REG_SP, &sp);
if (sp == forced_phase2_prev_sp) {
break;
}
forced_phase2_prev_sp = sp;
if (__unw_get_proc_info(cursor, &frameInfo) != UNW_ESUCCESS) {
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): __unw_get_proc_info "
"failed => _URC_FATAL_PHASE2_ERROR",
static_cast<void *>(exception_object));
return _URC_FATAL_PHASE2_ERROR;
}
// When tracing, print state information.
if (_LIBUNWIND_TRACING_UNWINDING) {
char functionBuf[512];
const char *functionName = functionBuf;
unw_word_t offset;
if ((__unw_get_proc_name(cursor, functionBuf, sizeof(functionBuf),
&offset) != UNW_ESUCCESS) ||
(frameInfo.start_ip + offset > frameInfo.end_ip))
functionName = ".anonymous.";
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): start_ip=0x%" PRIxPTR ", func=%s, sp=0x%" PRIxPTR ", "
"lsda=0x%" PRIxPTR ", personality=0x%" PRIxPTR "",
static_cast<void *>(exception_object), frameInfo.start_ip,
functionName, sp, frameInfo.lsda,
frameInfo.handler);
}
_Unwind_Action action =
(_Unwind_Action)(_UA_FORCE_UNWIND | _UA_CLEANUP_PHASE);
_Unwind_Reason_Code stopResult =
(*stop)(1, action, exception_object->exception_class, exception_object,
(_Unwind_Context *)(cursor), stop_parameter);
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): stop function returned %d",
(void *)exception_object, stopResult);
if (stopResult != _URC_NO_REASON) {
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): stopped by stop function",
(void *)exception_object);
return _URC_FATAL_PHASE2_ERROR;
}
// If there is a personality routine, tell it we are unwinding.
if (frameInfo.handler != 0) {
__personality_routine p =
(__personality_routine)(long)(frameInfo.handler);
struct _Unwind_Context *context = (struct _Unwind_Context *)(cursor);
// EHABI #7.2
exception_object->pr_cache.fnstart = frameInfo.start_ip;
exception_object->pr_cache.ehtp =
(_Unwind_EHT_Header *)frameInfo.unwind_info;
exception_object->pr_cache.additional = frameInfo.flags;
_Unwind_Reason_Code personalityResult =
(*p)(_US_FORCE_UNWIND | _US_UNWIND_FRAME_STARTING, exception_object,
context);
switch (personalityResult) {
case _URC_CONTINUE_UNWIND:
// Continue unwinding
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): _URC_CONTINUE_UNWIND",
static_cast<void *>(exception_object));
break;
case _URC_INSTALL_CONTEXT:
_LIBUNWIND_TRACE_UNWINDING(
"unwind_phase2_forced(ex_ojb=%p): _URC_INSTALL_CONTEXT",
static_cast<void *>(exception_object));
{
// EHABI #7.4.1 says we need to preserve pc for when _Unwind_Resume
// is called back, to find this same frame.
unw_word_t pc;
__unw_get_reg(cursor, UNW_REG_IP, &pc);
exception_object->unwinder_cache.reserved2 = (uint32_t)pc;
}
// We may get control back if landing pad calls _Unwind_Resume().
__unw_resume(cursor);
break;
case _URC_END_OF_STACK:
end_of_stack = true;
break;
default:
// Personality routine returned an unknown result code.
_LIBUNWIND_DEBUG_LOG("personality function returned unknown result %d",
personalityResult);
return _URC_FATAL_PHASE2_ERROR;
}
}
}
_LIBUNWIND_TRACE_UNWINDING("unwind_phase2_forced(ex_ojb=%p): calling stop "
"function with _UA_END_OF_STACK",
(void *)exception_object);
_Unwind_Action lastAction =
(_Unwind_Action)(_UA_FORCE_UNWIND | _UA_CLEANUP_PHASE | _UA_END_OF_STACK);
(*stop)(1, lastAction, exception_object->exception_class, exception_object,
(struct _Unwind_Context *)(cursor), stop_parameter);
return _URC_FATAL_PHASE2_ERROR;
}
/// Called by __cxa_throw. Only returns if there is a fatal error. /// Called by __cxa_throw. Only returns if there is a fatal error.
_LIBUNWIND_EXPORT _Unwind_Reason_Code _LIBUNWIND_EXPORT _Unwind_Reason_Code
_Unwind_RaiseException(_Unwind_Exception *exception_object) { _Unwind_RaiseException(_Unwind_Exception *exception_object) {
@ -854,36 +724,15 @@ _Unwind_Resume(_Unwind_Exception *exception_object) {
unw_cursor_t cursor; unw_cursor_t cursor;
__unw_getcontext(&uc); __unw_getcontext(&uc);
if (exception_object->unwinder_cache.reserved1) // _Unwind_RaiseException on EHABI will always set the reserved1 field to 0,
unwind_phase2_forced( // which is in the same position as private_1 below.
&uc, &cursor, exception_object, // TODO(ajwong): Who wronte the above? Why is it true?
(_Unwind_Stop_Fn)exception_object->unwinder_cache.reserved1, unwind_phase2(&uc, &cursor, exception_object, true);
(void *)exception_object->unwinder_cache.reserved3);
else
unwind_phase2(&uc, &cursor, exception_object, true);
// Clients assume _Unwind_Resume() does not return, so all we can do is abort. // Clients assume _Unwind_Resume() does not return, so all we can do is abort.
_LIBUNWIND_ABORT("_Unwind_Resume() can't return"); _LIBUNWIND_ABORT("_Unwind_Resume() can't return");
} }
_LIBUNWIND_EXPORT _Unwind_Reason_Code
_Unwind_ForcedUnwind(_Unwind_Exception *exception_object, _Unwind_Stop_Fn stop,
void *stop_parameter) {
_LIBUNWIND_TRACE_API("_Unwind_ForcedUnwind(ex_obj=%p, stop=%p)",
(void *)exception_object, (void *)(uintptr_t)stop);
unw_context_t uc;
unw_cursor_t cursor;
__unw_getcontext(&uc);
// Mark that this is a forced unwind, so _Unwind_Resume() can do
// the right thing.
exception_object->unwinder_cache.reserved1 = (uintptr_t)stop;
exception_object->unwinder_cache.reserved3 = (uintptr_t)stop_parameter;
return unwind_phase2_forced(&uc, &cursor, exception_object, stop,
stop_parameter);
}
/// Called by personality handler during phase 2 to get LSDA for current frame. /// Called by personality handler during phase 2 to get LSDA for current frame.
_LIBUNWIND_EXPORT uintptr_t _LIBUNWIND_EXPORT uintptr_t
_Unwind_GetLanguageSpecificData(struct _Unwind_Context *context) { _Unwind_GetLanguageSpecificData(struct _Unwind_Context *context) {
@ -1153,14 +1002,9 @@ extern "C" _LIBUNWIND_EXPORT _Unwind_Reason_Code
__gnu_unwind_frame(_Unwind_Exception *exception_object, __gnu_unwind_frame(_Unwind_Exception *exception_object,
struct _Unwind_Context *context) { struct _Unwind_Context *context) {
unw_cursor_t *cursor = (unw_cursor_t *)context; unw_cursor_t *cursor = (unw_cursor_t *)context;
switch (__unw_step(cursor)) { if (__unw_step(cursor) != UNW_STEP_SUCCESS)
case UNW_STEP_SUCCESS:
return _URC_OK;
case UNW_STEP_END:
return _URC_END_OF_STACK;
default:
return _URC_FAILURE; return _URC_FAILURE;
} return _URC_OK;
} }
#endif // defined(_LIBUNWIND_ARM_EHABI) #endif // defined(_LIBUNWIND_ARM_EHABI)

13
src/runtime/Cargo.toml
View File

@ -6,13 +6,12 @@ authors = ["M-Labs"]
edition = "2018" edition = "2018"
[features] [features]
target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"] target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"] target_coraz7 = ["libboard_zynq/target_coraz7", "libsupport_zynq/target_coraz7", "libconfig/target_coraz7"]
target_redpitaya = ["libboard_zynq/target_redpitaya", "libsupport_zynq/target_redpitaya", "libconfig/target_redpitaya"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc"]
default = ["target_zc706"] default = ["target_zc706"]
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies] [dependencies]
num-traits = { version = "0.2", default-features = false } num-traits = { version = "0.2", default-features = false }
num-derive = "0.3" num-derive = "0.3"
@ -34,11 +33,9 @@ libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" } libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" } libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" } libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] } libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"] }
dyld = { path = "../libdyld" } dyld = { path = "../libdyld" }
dwarf = { path = "../libdwarf" } dwarf = { path = "../libdwarf" }
unwind = { path = "../libunwind" } unwind = { path = "../libunwind" }
libc = { path = "../libc" } libc = { path = "../libc" }
io = { path = "../libio" }
libboard_artiq = { path = "../libboard_artiq" }

28
src/runtime/build.rs
View File

@ -1,6 +1,28 @@
extern crate build_zynq; use std::env;
use std::fs::File;
use std::io::Write;
use std::io::{BufRead, BufReader};
use std::path::PathBuf;
fn main() { fn main() {
build_zynq::add_linker_script(); // Put the linker script somewhere the linker can find it
build_zynq::cfg(); let out = &PathBuf::from(env::var_os("OUT_DIR").unwrap());
File::create(out.join("link.x"))
.unwrap()
.write_all(include_bytes!("link.x"))
.unwrap();
println!("cargo:rustc-link-search={}", out.display());
// Only re-run the build script when link.x is changed,
// instead of when any part of the source code changes.
println!("cargo:rerun-if-changed=link.x");
// Handle rustc-cfg file
let cfg_path = "../../build/rustc-cfg";
println!("cargo:rerun-if-changed={}", cfg_path);
let f = BufReader::new(File::open(cfg_path).unwrap());
for line in f.lines() {
println!("cargo:rustc-cfg={}", line.unwrap());
}
} }

0
src/libbuild_zynq/link.x → src/runtime/link.x
View File

17
src/runtime/src/analyzer.rs
View File

@ -3,7 +3,8 @@ use libboard_zynq::smoltcp::Error;
use libcortex_a9::cache; use libcortex_a9::cache;
use log::{debug, info, warn}; use log::{debug, info, warn};
use crate::{pl, proto_async::*}; use crate::proto_async::*;
use crate::pl;
const BUFFER_SIZE: usize = 512 * 1024; const BUFFER_SIZE: usize = 512 * 1024;
@ -12,7 +13,9 @@ struct Buffer {
data: [u8; BUFFER_SIZE], data: [u8; BUFFER_SIZE],
} }
static mut BUFFER: Buffer = Buffer { data: [0; BUFFER_SIZE] }; static mut BUFFER: Buffer = Buffer {
data: [0; BUFFER_SIZE]
};
fn arm() { fn arm() {
debug!("arming RTIO analyzer"); debug!("arming RTIO analyzer");
@ -43,7 +46,7 @@ struct Header {
total_byte_count: u64, total_byte_count: u64,
error_occurred: bool, error_occurred: bool,
log_channel: u8, log_channel: u8,
dds_onehot_sel: bool, dds_onehot_sel: bool
} }
async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Error> { async fn write_header(stream: &mut TcpStream, header: &Header) -> Result<(), Error> {
@ -75,14 +78,10 @@ async fn handle_connection(stream: &mut TcpStream) -> Result<(), Error> {
let header = Header { let header = Header {
total_byte_count: total_byte_count, total_byte_count: total_byte_count,
sent_bytes: if wraparound { sent_bytes: if wraparound { BUFFER_SIZE as u32 } else { total_byte_count as u32 },
BUFFER_SIZE as u32
} else {
total_byte_count as u32
},
error_occurred: overflow_occurred | bus_error_occurred, error_occurred: overflow_occurred | bus_error_occurred,
log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8, log_channel: pl::csr::CONFIG_RTIO_LOG_CHANNEL as u8,
dds_onehot_sel: true, // kept for backward compatibility of analyzer dumps dds_onehot_sel: true // kept for backward compatibility of analyzer dumps
}; };
debug!("{:?}", header); debug!("{:?}", header);

465
src/runtime/src/comms.rs
View File

@ -1,33 +1,34 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String, vec, vec::Vec}; use core::fmt;
use core::{cell::RefCell, fmt, slice, str}; use core::cell::RefCell;
use core::str::Utf8Error;
use alloc::{vec, vec::Vec, string::String, collections::BTreeMap, rc::Rc};
use log::{info, warn, error};
use cslice::CSlice;
use futures::{future::FutureExt, select_biased};
use libasync::{smoltcp::{Sockets, TcpStream},
task};
use libboard_artiq::drtio_routing;
#[cfg(feature = "target_kasli_soc")]
use libboard_zynq::error_led::ErrorLED;
use libboard_zynq::{self as zynq,
smoltcp::{self,
iface::{EthernetInterfaceBuilder, NeighborCache},
time::Instant,
wire::IpCidr},
timer::GlobalTimer};
use libconfig::{net_settings, Config};
use libcortex_a9::{mutex::Mutex,
semaphore::Semaphore,
sync_channel::{Receiver, Sender}};
use log::{error, info, warn};
use num_derive::{FromPrimitive, ToPrimitive}; use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive}; use num_traits::{FromPrimitive, ToPrimitive};
#[cfg(has_drtio)] use libboard_zynq::{
use crate::pl; self as zynq,
use crate::{analyzer, kernel, mgmt, moninj, smoltcp::{
proto_async::*, self,
rpc, rtio_dma, wire::IpCidr,
rtio_mgt::{self, resolve_channel_name}}; iface::{NeighborCache, EthernetInterfaceBuilder},
time::Instant,
},
timer::GlobalTimer,
};
use libcortex_a9::{semaphore::Semaphore, mutex::Mutex, sync_channel::{Sender, Receiver}};
use futures::{select_biased, future::FutureExt};
use libasync::{smoltcp::{Sockets, TcpStream}, task};
use libconfig::{Config, net_settings};
use crate::proto_async::*;
use crate::kernel;
use crate::rpc;
use crate::moninj;
use crate::mgmt;
use crate::analyzer;
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error { pub enum Error {
@ -35,6 +36,7 @@ pub enum Error {
UnexpectedPattern, UnexpectedPattern,
UnrecognizedPacket, UnrecognizedPacket,
BufferExhausted, BufferExhausted,
Utf8Error(Utf8Error),
} }
pub type Result<T> = core::result::Result<T, Error>; pub type Result<T> = core::result::Result<T, Error>;
@ -43,9 +45,10 @@ impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {
Error::NetworkError(error) => write!(f, "network error: {}", error), Error::NetworkError(error) => write!(f, "network error: {}", error),
Error::UnexpectedPattern => write!(f, "unexpected pattern"), Error::UnexpectedPattern => write!(f, "unexpected pattern"),
Error::UnrecognizedPacket => write!(f, "unrecognized packet"), Error::UnrecognizedPacket => write!(f, "unrecognized packet"),
Error::BufferExhausted => write!(f, "buffer exhausted"), Error::BufferExhausted => write!(f, "buffer exhausted"),
Error::Utf8Error(error) => write!(f, "UTF-8 error: {}", error),
} }
} }
} }
@ -79,32 +82,31 @@ enum Reply {
} }
static CACHE_STORE: Mutex<BTreeMap<String, Vec<i32>>> = Mutex::new(BTreeMap::new()); static CACHE_STORE: Mutex<BTreeMap<String, Vec<i32>>> = Mutex::new(BTreeMap::new());
static DMA_RECORD_STORE: Mutex<BTreeMap<String, (Vec<u8>, i64)>> = Mutex::new(BTreeMap::new());
async fn write_header(stream: &TcpStream, reply: Reply) -> Result<()> { async fn write_header(stream: &TcpStream, reply: Reply) -> Result<()> {
stream stream.send_slice(&[0x5a, 0x5a, 0x5a, 0x5a, reply.to_u8().unwrap()]).await?;
.send_slice(&[0x5a, 0x5a, 0x5a, 0x5a, reply.to_u8().unwrap()])
.await?;
Ok(()) Ok(())
} }
async fn read_request(stream: &TcpStream, allow_close: bool) -> Result<Option<Request>> { async fn read_request(stream: &TcpStream, allow_close: bool) -> Result<Option<Request>> {
match expect(stream, &[0x5a, 0x5a, 0x5a, 0x5a]).await { match expect(stream, &[0x5a, 0x5a, 0x5a, 0x5a]).await {
Ok(true) => {} Ok(true) => {}
Ok(false) => return Err(Error::UnexpectedPattern), Ok(false) =>
Err(smoltcp::Error::Finished) => { return Err(Error::UnexpectedPattern),
Err(smoltcp::Error::Illegal) => {
if allow_close { if allow_close {
info!("peer closed connection"); info!("peer closed connection");
return Ok(None); return Ok(None);
} else { } else {
error!("peer unexpectedly closed connection"); error!("peer unexpectedly closed connection");
return Err(smoltcp::Error::Finished)?; return Err(smoltcp::Error::Illegal)?;
} }
} },
Err(e) => return Err(e)?, Err(e) =>
return Err(e)?,
} }
Ok(Some( Ok(Some(FromPrimitive::from_i8(read_i8(&stream).await?).ok_or(Error::UnrecognizedPacket)?))
FromPrimitive::from_i8(read_i8(&stream).await?).ok_or(Error::UnrecognizedPacket)?,
))
} }
async fn read_bytes(stream: &TcpStream, max_length: usize) -> Result<Vec<u8>> { async fn read_bytes(stream: &TcpStream, max_length: usize) -> Result<Vec<u8>> {
@ -117,13 +119,20 @@ async fn read_bytes(stream: &TcpStream, max_length: usize) -> Result<Vec<u8>> {
Ok(buffer) Ok(buffer)
} }
async fn read_string(stream: &TcpStream, max_length: usize) -> Result<String> {
let bytes = read_bytes(stream, max_length).await?;
Ok(String::from_utf8(bytes).map_err(|err| Error::Utf8Error(err.utf8_error()))?)
}
const RETRY_LIMIT: usize = 100; const RETRY_LIMIT: usize = 100;
async fn fast_send(sender: &mut Sender<'_, kernel::Message>, content: kernel::Message) { async fn fast_send(sender: &mut Sender<'_, kernel::Message>, content: kernel::Message) {
let mut content = content; let mut content = content;
for _ in 0..RETRY_LIMIT { for _ in 0..RETRY_LIMIT {
match sender.try_send(content) { match sender.try_send(content) {
Ok(()) => return, Ok(()) => {
return
},
Err(v) => { Err(v) => {
content = v; content = v;
} }
@ -135,31 +144,16 @@ async fn fast_send(sender: &mut Sender<'_, kernel::Message>, content: kernel::Me
async fn fast_recv(receiver: &mut Receiver<'_, kernel::Message>) -> kernel::Message { async fn fast_recv(receiver: &mut Receiver<'_, kernel::Message>) -> kernel::Message {
for _ in 0..RETRY_LIMIT { for _ in 0..RETRY_LIMIT {
match receiver.try_recv() { match receiver.try_recv() {
Ok(v) => return v, Ok(v) => {
Err(()) => (), return v;
},
Err(()) => ()
} }
} }
receiver.async_recv().await receiver.async_recv().await
} }
async fn write_exception_string(stream: &TcpStream, s: CSlice<'static, u8>) -> Result<()> { async fn handle_run_kernel(stream: Option<&TcpStream>, control: &Rc<RefCell<kernel::Control>>) -> Result<()> {
if s.len() == usize::MAX {
write_i32(stream, -1).await?;
write_i32(stream, s.as_ptr() as i32).await?
} else {
write_chunk(stream, s.as_ref()).await?;
};
Ok(())
}
async fn handle_run_kernel(
stream: Option<&TcpStream>,
control: &Rc<RefCell<kernel::Control>>,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
) -> Result<()> {
control.borrow_mut().tx.async_send(kernel::Message::StartRequest).await; control.borrow_mut().tx.async_send(kernel::Message::StartRequest).await;
loop { loop {
let reply = control.borrow_mut().rx.async_recv().await; let reply = control.borrow_mut().rx.async_recv().await;
@ -167,7 +161,7 @@ async fn handle_run_kernel(
kernel::Message::RpcSend { is_async, data } => { kernel::Message::RpcSend { is_async, data } => {
if stream.is_none() { if stream.is_none() {
error!("Unexpected RPC from startup/idle kernel!"); error!("Unexpected RPC from startup/idle kernel!");
break; break
} }
let stream = stream.unwrap(); let stream = stream.unwrap();
write_header(stream, Reply::RPCRequest).await?; write_header(stream, Reply::RPCRequest).await?;
@ -194,185 +188,90 @@ async fn handle_run_kernel(
fast_send(&mut control.tx, kernel::Message::RpcRecvReply(Ok(size))).await; fast_send(&mut control.tx, kernel::Message::RpcRecvReply(Ok(size))).await;
match fast_recv(&mut control.rx).await { match fast_recv(&mut control.rx).await {
kernel::Message::RpcRecvRequest(slot) => slot, kernel::Message::RpcRecvRequest(slot) => slot,
other => { other => panic!("expected nested value slot from kernel CPU, not {:?}", other),
panic!("expected nested value slot from kernel CPU, not {:?}", other)
}
} }
} }
} }
}) }).await?;
.await?; control.borrow_mut().tx.async_send(kernel::Message::RpcRecvReply(Ok(0))).await;
control },
.borrow_mut()
.tx
.async_send(kernel::Message::RpcRecvReply(Ok(0)))
.await;
}
Request::RPCException => { Request::RPCException => {
let mut control = control.borrow_mut(); let mut control = control.borrow_mut();
match control.rx.async_recv().await { match control.rx.async_recv().await {
kernel::Message::RpcRecvRequest(_) => (), kernel::Message::RpcRecvRequest(_) => (),
other => panic!("expected (ignored) root value slot from kernel CPU, not {:?}", other), other => panic!("expected (ignored) root value slot from kernel CPU, not {:?}", other),
} }
let id = read_i32(stream).await? as u32; let name = read_string(stream, 16384).await?;
let message = read_i32(stream).await? as u32; let message = read_string(stream, 16384).await?;
let param = [ let param = [read_i64(stream).await?,
read_i64(stream).await?, read_i64(stream).await?,
read_i64(stream).await?, read_i64(stream).await?];
read_i64(stream).await?, let file = read_string(stream, 16384).await?;
]; let line = read_i32(stream).await?;
let file = read_i32(stream).await? as u32; let column = read_i32(stream).await?;
let line = read_i32(stream).await?; let function = read_string(stream, 16384).await?;
let column = read_i32(stream).await?; control.tx.async_send(kernel::Message::RpcRecvReply(Err(kernel::RPCException {
let function = read_i32(stream).await? as u32; name, message, param, file, line, column, function
control }))).await;
.tx },
.async_send(kernel::Message::RpcRecvReply(Err(kernel::RPCException {
id,
message,
param,
file,
line,
column,
function,
})))
.await;
}
_ => { _ => {
error!("unexpected RPC request from host: {:?}", host_request); error!("unexpected RPC request from host: {:?}", host_request);
return Err(Error::UnrecognizedPacket); return Err(Error::UnrecognizedPacket)
} }
} }
} }
} },
kernel::Message::KernelFinished(async_errors) => { kernel::Message::KernelFinished => {
if let Some(stream) = stream { if let Some(stream) = stream {
write_header(stream, Reply::KernelFinished).await?; write_header(stream, Reply::KernelFinished).await?;
write_i8(stream, async_errors as i8).await?;
} }
break; break;
} },
kernel::Message::KernelException(exceptions, stack_pointers, backtrace, async_errors) => { kernel::Message::KernelException(exception, backtrace) => {
match stream { match stream {
Some(stream) => { Some(stream) => {
// only send the exception data to host if there is host, // only send the exception data to host if there is host,
// i.e. not idle/startup kernel. // i.e. not idle/startup kernel.
write_header(stream, Reply::KernelException).await?; write_header(stream, Reply::KernelException).await?;
write_i32(stream, exceptions.len() as i32).await?; write_chunk(stream, exception.name.as_ref()).await?;
for exception in exceptions.iter() { write_chunk(stream, exception.message.as_ref()).await?;
let exception = exception.as_ref().unwrap(); write_i64(stream, exception.param[0] as i64).await?;
write_i32(stream, exception.id as i32).await?; write_i64(stream, exception.param[1] as i64).await?;
write_i64(stream, exception.param[2] as i64).await?;
if exception.message.len() == usize::MAX { write_chunk(stream, exception.file.as_ref()).await?;
// exception with host string write_i32(stream, exception.line as i32).await?;
write_exception_string(stream, exception.message).await?; write_i32(stream, exception.column as i32).await?;
} else { write_chunk(stream, exception.function.as_ref()).await?;
let msg = str::from_utf8(unsafe {
slice::from_raw_parts(exception.message.as_ptr(), exception.message.len())
})
.unwrap()
.replace(
"{rtio_channel_info:0}",
&format!(
"0x{:04x}:{}",
exception.param[0],
resolve_channel_name(exception.param[0] as u32)
),
);
write_exception_string(stream, unsafe { CSlice::new(msg.as_ptr(), msg.len()) }).await?;
}
write_i64(stream, exception.param[0] as i64).await?;
write_i64(stream, exception.param[1] as i64).await?;
write_i64(stream, exception.param[2] as i64).await?;
write_exception_string(stream, exception.file).await?;
write_i32(stream, exception.line as i32).await?;
write_i32(stream, exception.column as i32).await?;
write_exception_string(stream, exception.function).await?;
}
for sp in stack_pointers.iter() {
write_i32(stream, sp.stack_pointer as i32).await?;
write_i32(stream, sp.initial_backtrace_size as i32).await?;
write_i32(stream, sp.current_backtrace_size as i32).await?;
}
write_i32(stream, backtrace.len() as i32).await?; write_i32(stream, backtrace.len() as i32).await?;
for &(addr, sp) in backtrace { for &addr in backtrace {
write_i32(stream, addr as i32).await?; write_i32(stream, addr as i32).await?;
write_i32(stream, sp as i32).await?;
} }
write_i8(stream, async_errors as i8).await?; },
}
None => { None => {
error!("Uncaught kernel exceptions: {:?}", exceptions); error!("Uncaught kernel exception: {:?}", exception);
} }
} }
break; break;
} }
kernel::Message::CachePutRequest(key, value) => { kernel::Message::CachePutRequest(key, value) => {
CACHE_STORE.lock().insert(key, value); CACHE_STORE.lock().insert(key, value);
} },
kernel::Message::CacheGetRequest(key) => { kernel::Message::CacheGetRequest(key) => {
const DEFAULT: Vec<i32> = Vec::new(); const DEFAULT: Vec<i32> = Vec::new();
let value = CACHE_STORE.lock().get(&key).unwrap_or(&DEFAULT).clone(); let value = CACHE_STORE.lock().get(&key).unwrap_or(&DEFAULT).clone();
control control.borrow_mut().tx.async_send(kernel::Message::CacheGetReply(value)).await;
.borrow_mut() },
.tx
.async_send(kernel::Message::CacheGetReply(value))
.await;
}
kernel::Message::DmaPutRequest(recorder) => { kernel::Message::DmaPutRequest(recorder) => {
let _id = rtio_dma::put_record(aux_mutex, routing_table, timer, recorder).await; DMA_RECORD_STORE.lock().insert(recorder.name, (recorder.buffer, recorder.duration));
#[cfg(has_drtio)] },
rtio_dma::remote_dma::upload_traces(aux_mutex, routing_table, timer, _id).await;
}
kernel::Message::DmaEraseRequest(name) => { kernel::Message::DmaEraseRequest(name) => {
// prevent possible OOM when we have large DMA record replacement. // prevent possible OOM when we have large DMA record replacement.
rtio_dma::erase(name, aux_mutex, routing_table, timer).await; DMA_RECORD_STORE.lock().remove(&name);
} },
kernel::Message::DmaGetRequest(name) => { kernel::Message::DmaGetRequest(name) => {
let result = rtio_dma::retrieve(name); let result = DMA_RECORD_STORE.lock().get(&name).map(|v| v.clone());
control control.borrow_mut().tx.async_send(kernel::Message::DmaGetReply(result)).await;
.borrow_mut() },
.tx
.async_send(kernel::Message::DmaGetReply(result))
.await;
}
#[cfg(has_drtio)]
kernel::Message::DmaStartRemoteRequest { id, timestamp } => {
rtio_dma::remote_dma::playback(aux_mutex, routing_table, timer, id as u32, timestamp as u64).await;
}
#[cfg(has_drtio)]
kernel::Message::DmaAwaitRemoteRequest(id) => {
let result = rtio_dma::remote_dma::await_done(id as u32, Some(10_000), timer).await;
let reply = match result {
Ok(rtio_dma::remote_dma::RemoteState::PlaybackEnded {
error,
channel,
timestamp,
}) => kernel::Message::DmaAwaitRemoteReply {
timeout: false,
error: error,
channel: channel,
timestamp: timestamp,
},
_ => kernel::Message::DmaAwaitRemoteReply {
timeout: true,
error: 0,
channel: 0,
timestamp: 0,
},
};
control.borrow_mut().tx.async_send(reply).await;
}
#[cfg(has_drtio)]
kernel::Message::UpDestinationsRequest(destination) => {
let result = _up_destinations.borrow()[destination as usize];
control
.borrow_mut()
.tx
.async_send(kernel::Message::UpDestinationsReply(result))
.await;
}
_ => { _ => {
panic!("unexpected message from core1 while kernel was running: {:?}", reply); panic!("unexpected message from core1 while kernel was running: {:?}", reply);
} }
@ -381,17 +280,11 @@ async fn handle_run_kernel(
Ok(()) Ok(())
} }
async fn load_kernel(
buffer: &Vec<u8>, async fn load_kernel(buffer: &Vec<u8>, control: &Rc<RefCell<kernel::Control>>, stream: Option<&TcpStream>) -> Result<()> {
control: &Rc<RefCell<kernel::Control>>,
stream: Option<&TcpStream>,
) -> Result<()> {
let mut control = control.borrow_mut(); let mut control = control.borrow_mut();
control.restart(); control.restart();
control control.tx.async_send(kernel::Message::LoadRequest(buffer.to_vec())).await;
.tx
.async_send(kernel::Message::LoadRequest(buffer.to_vec()))
.await;
let reply = control.rx.async_recv().await; let reply = control.rx.async_recv().await;
match reply { match reply {
kernel::Message::LoadCompleted => { kernel::Message::LoadCompleted => {
@ -399,7 +292,7 @@ async fn load_kernel(
write_header(stream, Reply::LoadCompleted).await?; write_header(stream, Reply::LoadCompleted).await?;
} }
Ok(()) Ok(())
} },
kernel::Message::LoadFailed => { kernel::Message::LoadFailed => {
if let Some(stream) = stream { if let Some(stream) = stream {
write_header(stream, Reply::LoadFailed).await?; write_header(stream, Reply::LoadFailed).await?;
@ -408,7 +301,7 @@ async fn load_kernel(
error!("Kernel load failed"); error!("Kernel load failed");
} }
Err(Error::UnexpectedPattern) Err(Error::UnexpectedPattern)
} },
_ => { _ => {
error!("unexpected message from core1: {:?}", reply); error!("unexpected message from core1: {:?}", reply);
if let Some(stream) = stream { if let Some(stream) = stream {
@ -420,16 +313,7 @@ async fn load_kernel(
} }
} }
async fn handle_connection( async fn handle_connection(stream: &TcpStream, control: Rc<RefCell<kernel::Control>>) -> Result<()> {
stream: &mut TcpStream,
control: Rc<RefCell<kernel::Control>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
) -> Result<()> {
stream.set_ack_delay(None);
if !expect(stream, b"ARTIQ coredev\n").await? { if !expect(stream, b"ARTIQ coredev\n").await? {
return Err(Error::UnexpectedPattern); return Err(Error::UnexpectedPattern);
} }
@ -444,32 +328,24 @@ async fn handle_connection(
Request::SystemInfo => { Request::SystemInfo => {
write_header(stream, Reply::SystemInfo).await?; write_header(stream, Reply::SystemInfo).await?;
stream.send_slice("ARZQ".as_bytes()).await?; stream.send_slice("ARZQ".as_bytes()).await?;
} },
Request::LoadKernel => { Request::LoadKernel => {
let buffer = read_bytes(stream, 1024 * 1024).await?; let buffer = read_bytes(stream, 1024*1024).await?;
load_kernel(&buffer, &control, Some(stream)).await?; load_kernel(&buffer, &control, Some(stream)).await?;
} },
Request::RunKernel => { Request::RunKernel => {
handle_run_kernel( handle_run_kernel(Some(stream), &control).await?;
Some(stream), },
&control,
&up_destinations,
aux_mutex,
routing_table,
timer,
)
.await?;
}
_ => { _ => {
error!("unexpected request from host: {:?}", request); error!("unexpected request from host: {:?}", request);
return Err(Error::UnrecognizedPacket); return Err(Error::UnrecognizedPacket)
} }
} }
} }
} }
pub fn main(timer: GlobalTimer, cfg: Config) { pub fn main(timer: GlobalTimer, cfg: Config) {
let net_addresses = net_settings::get_addresses(&cfg); let net_addresses = net_settings::get_adresses(&cfg);
info!("network addresses: {}", net_addresses); info!("network addresses: {}", net_addresses);
let eth = zynq::eth::Eth::eth0(net_addresses.hardware_addr.0.clone()); let eth = zynq::eth::Eth::eth0(net_addresses.hardware_addr.0.clone());
@ -486,46 +362,31 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0),
IpCidr::new(addr, 0), IpCidr::new(addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
.ip_addrs(ip_addrs) .ip_addrs(ip_addrs)
.neighbor_cache(neighbor_cache) .neighbor_cache(neighbor_cache)
.finalize() .finalize()
} }
None => { None => {
let ip_addrs = [ let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0), IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0), IpCidr::new(net_addresses.ipv6_ll_addr, 0)
]; ];
EthernetInterfaceBuilder::new(&mut eth) EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr) .ethernet_addr(net_addresses.hardware_addr)
.ip_addrs(ip_addrs) .ip_addrs(ip_addrs)
.neighbor_cache(neighbor_cache) .neighbor_cache(neighbor_cache)
.finalize() .finalize()
} }
}; };
Sockets::init(32); Sockets::init(32);
// before, mutex was on io, but now that io isn't used...?
let aux_mutex: Rc<Mutex<bool>> = Rc::new(Mutex::new(false));
#[cfg(has_drtio)]
let drtio_routing_table = Rc::new(RefCell::new(drtio_routing::config_routing_table(
pl::csr::DRTIO.len(),
&cfg,
)));
#[cfg(not(has_drtio))]
let drtio_routing_table = Rc::new(RefCell::new(drtio_routing::RoutingTable::default_empty()));
let up_destinations = Rc::new(RefCell::new([false; drtio_routing::DEST_COUNT]));
#[cfg(has_drtio_routing)]
drtio_routing::interconnect_disable_all();
rtio_mgt::startup(&aux_mutex, &drtio_routing_table, &up_destinations, timer, &cfg);
analyzer::start(); analyzer::start();
moninj::start(timer, &aux_mutex, &drtio_routing_table); moninj::start(timer);
let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start())); let control: Rc<RefCell<kernel::Control>> = Rc::new(RefCell::new(kernel::Control::start()));
let idle_kernel = Rc::new(cfg.read("idle_kernel").ok()); let idle_kernel = Rc::new(cfg.read("idle_kernel").ok());
@ -533,15 +394,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
info!("Loading startup kernel..."); info!("Loading startup kernel...");
if let Ok(()) = task::block_on(load_kernel(&buffer, &control, None)) { if let Ok(()) = task::block_on(load_kernel(&buffer, &control, None)) {
info!("Starting startup kernel..."); info!("Starting startup kernel...");
let routing_table = drtio_routing_table.borrow(); let _ = task::block_on(handle_run_kernel(None, &control));
let _ = task::block_on(handle_run_kernel(
None,
&control,
&up_destinations,
&aux_mutex,
&routing_table,
timer,
));
info!("Startup kernel finished!"); info!("Startup kernel finished!");
} else { } else {
error!("Error loading startup kernel!"); error!("Error loading startup kernel!");
@ -554,7 +407,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
let connection = Rc::new(Semaphore::new(1, 1)); let connection = Rc::new(Semaphore::new(1, 1));
let terminate = Rc::new(Semaphore::new(0, 1)); let terminate = Rc::new(Semaphore::new(0, 1));
loop { loop {
let mut stream = TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap(); let stream = TcpStream::accept(1381, 0x10_000, 0x10_000).await.unwrap();
if connection.try_wait().is_none() { if connection.try_wait().is_none() {
// there is an existing connection // there is an existing connection
@ -566,17 +419,13 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
let idle_kernel = idle_kernel.clone(); let idle_kernel = idle_kernel.clone();
let connection = connection.clone(); let connection = connection.clone();
let terminate = terminate.clone(); let terminate = terminate.clone();
let up_destinations = up_destinations.clone();
let aux_mutex = aux_mutex.clone();
let routing_table = drtio_routing_table.clone();
// we make sure the value of terminate is 0 before we start // we make sure the value of terminate is 0 before we start
let _ = terminate.try_wait(); let _ = terminate.try_wait();
task::spawn(async move { task::spawn(async move {
let routing_table = routing_table.borrow();
select_biased! { select_biased! {
_ = (async { _ = (async {
let _ = handle_connection(&mut stream, control.clone(), &up_destinations, &aux_mutex, &routing_table, timer) let _ = handle_connection(&stream, control.clone())
.await .await
.map_err(|e| warn!("connection terminated: {}", e)); .map_err(|e| warn!("connection terminated: {}", e));
if let Some(buffer) = &*idle_kernel { if let Some(buffer) = &*idle_kernel {
@ -584,7 +433,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
let _ = load_kernel(&buffer, &control, None) let _ = load_kernel(&buffer, &control, None)
.await.map_err(|_| warn!("error loading idle kernel")); .await.map_err(|_| warn!("error loading idle kernel"));
info!("Running idle kernel"); info!("Running idle kernel");
let _ = handle_run_kernel(None, &control, &up_destinations, &aux_mutex, &routing_table, timer) let _ = handle_run_kernel(None, &control)
.await.map_err(|_| warn!("error running idle kernel")); .await.map_err(|_| warn!("error running idle kernel"));
info!("Idle kernel terminated"); info!("Idle kernel terminated");
} }
@ -598,59 +447,7 @@ pub fn main(timer: GlobalTimer, cfg: Config) {
} }
}); });
Sockets::run(&mut iface, || Instant::from_millis(timer.get_time().0 as i32)); Sockets::run(&mut iface, || {
} Instant::from_millis(timer.get_time().0 as i32)
});
pub fn soft_panic_main(timer: GlobalTimer, cfg: Config) -> ! {
let net_addresses = net_settings::get_addresses(&cfg);
info!("network addresses: {}", net_addresses);
let eth = zynq::eth::Eth::eth0(net_addresses.hardware_addr.0.clone());
const RX_LEN: usize = 64;
// Number of transmission buffers (minimum is two because with
// one, duplicate packet transmission occurs)
const TX_LEN: usize = 64;
let eth = eth.start_rx(RX_LEN);
let mut eth = eth.start_tx(TX_LEN);
let neighbor_cache = NeighborCache::new(alloc::collections::BTreeMap::new());
let mut iface = match net_addresses.ipv6_addr {
Some(addr) => {
let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0),
IpCidr::new(addr, 0),
];
EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr)
.ip_addrs(ip_addrs)
.neighbor_cache(neighbor_cache)
.finalize()
}
None => {
let ip_addrs = [
IpCidr::new(net_addresses.ipv4_addr, 0),
IpCidr::new(net_addresses.ipv6_ll_addr, 0),
];
EthernetInterfaceBuilder::new(&mut eth)
.ethernet_addr(net_addresses.hardware_addr)
.ip_addrs(ip_addrs)
.neighbor_cache(neighbor_cache)
.finalize()
}
};
Sockets::init(32);
mgmt::start(cfg);
// getting eth settings disables the LED as it resets GPIO
// need to re-enable it here
#[cfg(feature = "target_kasli_soc")]
{
let mut err_led = ErrorLED::error_led();
err_led.toggle(true);
}
Sockets::run(&mut iface, || Instant::from_millis(timer.get_time().0 as i32));
} }

535
src/runtime/src/eh_artiq.rs
View File

@ -13,136 +13,61 @@
#![allow(non_camel_case_types)] #![allow(non_camel_case_types)]
use core::mem; use core::mem;
use cslice::CSlice; use cslice::CSlice;
use dwarf::eh::{self, EHAction, EHContext};
use libc::{c_int, c_void, uintptr_t};
use log::{error, trace};
use unwind as uw; use unwind as uw;
use libc::{c_int, uintptr_t};
use log::trace;
use dwarf::eh::{self, EHAction, EHContext};
use crate::kernel::KERNEL_IMAGE;
const EXCEPTION_CLASS: uw::_Unwind_Exception_Class = 0x4d_4c_42_53_41_52_54_51; /* 'MLBSARTQ' */ const EXCEPTION_CLASS: uw::_Unwind_Exception_Class = 0x4d_4c_42_53_41_52_54_51; /* 'MLBSARTQ' */
#[cfg(target_arch = "arm")] #[cfg(target_arch = "arm")]
const UNWIND_DATA_REG: (i32, i32) = (0, 1); // R0, R1 const UNWIND_DATA_REG: (i32, i32) = (0, 1); // R0, R1
// Note: CSlice within an exception may not be actual cslice, they may be strings that exist only
// in the host. If the length == usize:MAX, the pointer is actually a string key in the host.
#[repr(C)] #[repr(C)]
#[derive(Clone, Copy)] #[derive(Clone, Copy)]
pub struct Exception<'a> { pub struct Exception<'a> {
pub id: u32, pub name: CSlice<'a, u8>,
pub file: CSlice<'a, u8>, pub file: CSlice<'a, u8>,
pub line: u32, pub line: u32,
pub column: u32, pub column: u32,
pub function: CSlice<'a, u8>, pub function: CSlice<'a, u8>,
pub message: CSlice<'a, u8>, pub message: CSlice<'a, u8>,
pub param: [i64; 3], pub param: [i64; 3]
} }
fn str_err(_: core::str::Utf8Error) -> core::fmt::Error { fn str_err(_: core::str::Utf8Error) -> core::fmt::Error {
core::fmt::Error core::fmt::Error
} }
fn exception_str<'a>(s: &'a CSlice<'a, u8>) -> Result<&'a str, core::str::Utf8Error> {
if s.len() == usize::MAX {
Ok("<host string>")
} else {
core::str::from_utf8(s.as_ref())
}
}
impl<'a> core::fmt::Debug for Exception<'a> { impl<'a> core::fmt::Debug for Exception<'a> {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!( write!(f, "Exception {} from {} in {}:{}:{}, message: {}",
f, core::str::from_utf8(self.name.as_ref()).map_err(str_err)?,
"Exception {} from {} in {}:{}:{}, message: {}", core::str::from_utf8(self.function.as_ref()).map_err(str_err)?,
self.id, core::str::from_utf8(self.file.as_ref()).map_err(str_err)?,
exception_str(&self.function).map_err(str_err)?, self.line, self.column,
exception_str(&self.file).map_err(str_err)?, core::str::from_utf8(self.message.as_ref()).map_err(str_err)?)
self.line,
self.column,
exception_str(&self.message).map_err(str_err)?
)
} }
} }
const MAX_INFLIGHT_EXCEPTIONS: usize = 10;
const MAX_BACKTRACE_SIZE: usize = 128; const MAX_BACKTRACE_SIZE: usize = 128;
#[derive(Debug, Default)] #[repr(C)]
pub struct StackPointerBacktrace { struct ExceptionInfo {
pub stack_pointer: usize, uw_exception: uw::_Unwind_Exception,
pub initial_backtrace_size: usize, exception: Option<Exception<'static>>,
pub current_backtrace_size: usize, handled: bool,
backtrace: [usize; MAX_BACKTRACE_SIZE],
backtrace_size: usize
} }
struct ExceptionBuffer { unsafe fn find_eh_action(
// we need n _Unwind_Exception, because each will have their own private data
uw_exceptions: [uw::_Unwind_Exception; MAX_INFLIGHT_EXCEPTIONS],
exceptions: [Option<Exception<'static>>; MAX_INFLIGHT_EXCEPTIONS + 1],
exception_stack: [isize; MAX_INFLIGHT_EXCEPTIONS + 1],
// nested exceptions will share the backtrace buffer, treated as a tree
// backtrace contains a tuple of IP and SP
backtrace: [(usize, usize); MAX_BACKTRACE_SIZE],
backtrace_size: usize,
// stack pointers are stored to reconstruct backtrace for each exception
stack_pointers: [StackPointerBacktrace; MAX_INFLIGHT_EXCEPTIONS + 1],
// current allocated nested exceptions
exception_count: usize,
}
static mut EXCEPTION_BUFFER: ExceptionBuffer = ExceptionBuffer {
uw_exceptions: [uw::_Unwind_Exception {
exception_class: EXCEPTION_CLASS,
exception_cleanup: cleanup,
private: [0; uw::unwinder_private_data_size],
}; MAX_INFLIGHT_EXCEPTIONS],
exceptions: [None; MAX_INFLIGHT_EXCEPTIONS + 1],
exception_stack: [-1; MAX_INFLIGHT_EXCEPTIONS + 1],
backtrace: [(0, 0); MAX_BACKTRACE_SIZE],
backtrace_size: 0,
stack_pointers: [StackPointerBacktrace {
stack_pointer: 0,
initial_backtrace_size: 0,
current_backtrace_size: 0,
}; MAX_INFLIGHT_EXCEPTIONS + 1],
exception_count: 0,
};
pub unsafe extern "C" fn reset_exception_buffer() {
trace!("reset exception buffer");
EXCEPTION_BUFFER.uw_exceptions = [uw::_Unwind_Exception {
exception_class: EXCEPTION_CLASS,
exception_cleanup: cleanup,
private: [0; uw::unwinder_private_data_size],
}; MAX_INFLIGHT_EXCEPTIONS];
EXCEPTION_BUFFER.exceptions = [None; MAX_INFLIGHT_EXCEPTIONS + 1];
EXCEPTION_BUFFER.exception_stack = [-1; MAX_INFLIGHT_EXCEPTIONS + 1];
EXCEPTION_BUFFER.backtrace_size = 0;
EXCEPTION_BUFFER.exception_count = 0;
}
type _Unwind_Stop_Fn = extern "C" fn(
version: c_int,
actions: i32,
exception_class: uw::_Unwind_Exception_Class,
exception_object: *mut uw::_Unwind_Exception,
context: *mut uw::_Unwind_Context, context: *mut uw::_Unwind_Context,
stop_parameter: *mut c_void, foreign_exception: bool,
) -> uw::_Unwind_Reason_Code; ) -> Result<EHAction, ()> {
extern "C" {
// not defined in EHABI, but LLVM added it and is useful to us
fn _Unwind_ForcedUnwind(
exception: *mut uw::_Unwind_Exception,
stop_fn: _Unwind_Stop_Fn,
stop_parameter: *mut c_void,
) -> uw::_Unwind_Reason_Code;
}
unsafe fn find_eh_action(context: *mut uw::_Unwind_Context, foreign_exception: bool, id: u32) -> Result<EHAction, ()> {
let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8; let lsda = uw::_Unwind_GetLanguageSpecificData(context) as *const u8;
let mut ip_before_instr: c_int = 0; let mut ip_before_instr: c_int = 0;
let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr); let ip = uw::_Unwind_GetIPInfo(context, &mut ip_before_instr);
@ -154,317 +79,187 @@ unsafe fn find_eh_action(context: *mut uw::_Unwind_Context, foreign_exception: b
get_text_start: &|| uw::_Unwind_GetTextRelBase(context), get_text_start: &|| uw::_Unwind_GetTextRelBase(context),
get_data_start: &|| uw::_Unwind_GetDataRelBase(context), get_data_start: &|| uw::_Unwind_GetDataRelBase(context),
}; };
eh::find_eh_action(lsda, &eh_context, foreign_exception, id) eh::find_eh_action(lsda, &eh_context, foreign_exception)
} }
pub unsafe fn artiq_personality( pub unsafe fn artiq_personality(state: uw::_Unwind_State,
_state: uw::_Unwind_State, exception_object: *mut uw::_Unwind_Exception,
exception_object: *mut uw::_Unwind_Exception, context: *mut uw::_Unwind_Context)
context: *mut uw::_Unwind_Context, -> uw::_Unwind_Reason_Code {
) -> uw::_Unwind_Reason_Code { let state = state as c_int;
// we will only do phase 2 forced unwinding now let action = state & uw::_US_ACTION_MASK as c_int;
let search_phase = if action == uw::_US_VIRTUAL_UNWIND_FRAME as c_int {
// Backtraces on ARM will call the personality routine with
// state == _US_VIRTUAL_UNWIND_FRAME | _US_FORCE_UNWIND. In those cases
// we want to continue unwinding the stack, otherwise all our backtraces
// would end at __rust_try
if state & uw::_US_FORCE_UNWIND as c_int != 0 {
return continue_unwind(exception_object, context);
}
true
} else if action == uw::_US_UNWIND_FRAME_STARTING as c_int {
false
} else if action == uw::_US_UNWIND_FRAME_RESUME as c_int {
return continue_unwind(exception_object, context);
} else {
return uw::_URC_FAILURE;
};
// The DWARF unwinder assumes that _Unwind_Context holds things like the function // The DWARF unwinder assumes that _Unwind_Context holds things like the function
// and LSDA pointers, however ARM EHABI places them into the exception object. // and LSDA pointers, however ARM EHABI places them into the exception object.
// To preserve signatures of functions like _Unwind_GetLanguageSpecificData(), which // To preserve signatures of functions like _Unwind_GetLanguageSpecificData(), which
// take only the context pointer, GCC personality routines stash a pointer to // take only the context pointer, GCC personality routines stash a pointer to
// exception_object in the context, using location reserved for ARM's // exception_object in the context, using location reserved for ARM's
// "scratch register" (r12). // "scratch register" (r12).
uw::_Unwind_SetGR(context, uw::UNWIND_POINTER_REG, exception_object as uw::_Unwind_Ptr); uw::_Unwind_SetGR(context,
uw::UNWIND_POINTER_REG,
exception_object as uw::_Unwind_Ptr);
// ...A more principled approach would be to provide the full definition of ARM's // ...A more principled approach would be to provide the full definition of ARM's
// _Unwind_Context in our libunwind bindings and fetch the required data from there // _Unwind_Context in our libunwind bindings and fetch the required data from there
// directly, bypassing DWARF compatibility functions. // directly, bypassing DWARF compatibility functions.
let exception_class = (*exception_object).exception_class; let exception_class = (*exception_object).exception_class;
let foreign_exception = exception_class != EXCEPTION_CLASS; let foreign_exception = exception_class != EXCEPTION_CLASS;
assert!(!foreign_exception, "we do not expect foreign exceptions"); let eh_action = match find_eh_action(context, foreign_exception) {
let index = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
assert!(index != -1);
let exception = EXCEPTION_BUFFER.exceptions[index as usize].as_ref().unwrap();
let id = exception.id;
let eh_action = match find_eh_action(context, foreign_exception, id) {
Ok(action) => action, Ok(action) => action,
Err(_) => return uw::_URC_FAILURE, Err(_) => return uw::_URC_FAILURE,
}; };
match eh_action { let exception_info = &mut *(exception_object as *mut ExceptionInfo);
EHAction::None => return continue_unwind(exception_object, context), let exception = &exception_info.exception.unwrap();
EHAction::Cleanup(lpad) | EHAction::Catch(lpad) => { if search_phase {
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.0, exception_object as uintptr_t); match eh_action {
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.1, exception as *const _ as uw::_Unwind_Word); EHAction::None => return continue_unwind(exception_object, context),
uw::_Unwind_SetIP(context, lpad); // Actually, cleanup should not return handler found, this is to workaround
return uw::_URC_INSTALL_CONTEXT; // the issue of terminating directly when no catch cause is found while
// having some cleanup routines defined by finally.
// The best way to handle this is to force unwind the stack in the raise
// function when end of stack is reached, and call terminate at the end of
// the unwind. Unfortunately, there is no forced unwind function defined
// for EHABI, and I have no idea how to implement that, so this is a hack.
EHAction::Cleanup(_) => return uw::_URC_HANDLER_FOUND,
EHAction::Catch(_) => {
// EHABI requires the personality routine to update the
// SP value in the barrier cache of the exception object.
(*exception_object).private[5] =
uw::_Unwind_GetGR(context, uw::UNWIND_SP_REG);
return uw::_URC_HANDLER_FOUND;
}
EHAction::Terminate => return uw::_URC_FAILURE,
}
} else {
match eh_action {
EHAction::None => return continue_unwind(exception_object, context),
EHAction::Cleanup(lpad) |
EHAction::Catch(lpad) => {
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.0,
exception_object as uintptr_t);
uw::_Unwind_SetGR(context, UNWIND_DATA_REG.1, exception as *const _ as uw::_Unwind_Word);
uw::_Unwind_SetIP(context, lpad);
return uw::_URC_INSTALL_CONTEXT;
}
EHAction::Terminate => return uw::_URC_FAILURE,
} }
EHAction::Terminate => return uw::_URC_FAILURE,
} }
// On ARM EHABI the personality routine is responsible for actually // On ARM EHABI the personality routine is responsible for actually
// unwinding a single stack frame before returning (ARM EHABI Sec. 6.1). // unwinding a single stack frame before returning (ARM EHABI Sec. 6.1).
unsafe fn continue_unwind( unsafe fn continue_unwind(exception_object: *mut uw::_Unwind_Exception,
exception_object: *mut uw::_Unwind_Exception, context: *mut uw::_Unwind_Context)
context: *mut uw::_Unwind_Context, -> uw::_Unwind_Reason_Code {
) -> uw::_Unwind_Reason_Code { if __gnu_unwind_frame(exception_object, context) == uw::_URC_NO_REASON {
let reason = __gnu_unwind_frame(exception_object, context);
if reason == uw::_URC_NO_REASON {
uw::_URC_CONTINUE_UNWIND uw::_URC_CONTINUE_UNWIND
} else { } else {
reason uw::_URC_FAILURE
} }
} }
// defined in libgcc // defined in libgcc
extern "C" { extern "C" {
fn __gnu_unwind_frame( fn __gnu_unwind_frame(exception_object: *mut uw::_Unwind_Exception,
exception_object: *mut uw::_Unwind_Exception, context: *mut uw::_Unwind_Context)
context: *mut uw::_Unwind_Context, -> uw::_Unwind_Reason_Code;
) -> uw::_Unwind_Reason_Code;
} }
} }
pub unsafe extern "C" fn raise(exception: *const Exception) -> ! { extern fn cleanup(_unwind_code: uw::_Unwind_Reason_Code,
uw_exception: *mut uw::_Unwind_Exception) {
unsafe {
let exception_info = &mut *(uw_exception as *mut ExceptionInfo);
exception_info.exception = None;
}
}
static mut INFLIGHT: ExceptionInfo = ExceptionInfo {
uw_exception: uw::_Unwind_Exception {
exception_class: EXCEPTION_CLASS,
exception_cleanup: cleanup,
private: [0; uw::unwinder_private_data_size],
},
exception: None,
handled: true,
backtrace: [0; MAX_BACKTRACE_SIZE],
backtrace_size: 0
};
pub unsafe extern fn raise(exception: *const Exception) -> ! {
trace!("Trying to raise exception");
// FIXME: unsound transmute
// This would cause stack memory corruption.
INFLIGHT.exception = Some(mem::transmute::<Exception, Exception<'static>>(*exception));
INFLIGHT.handled = false;
let result = uw::_Unwind_RaiseException(&mut INFLIGHT.uw_exception);
assert!(result == uw::_URC_END_OF_STACK);
INFLIGHT.backtrace_size = 0;
// read backtrace
let _ = uw::backtrace(|ip| {
if INFLIGHT.backtrace_size < MAX_BACKTRACE_SIZE {
INFLIGHT.backtrace[INFLIGHT.backtrace_size] = ip;
INFLIGHT.backtrace_size += 1;
}
});
crate::kernel::core1::terminate(INFLIGHT.exception.as_ref().unwrap(), INFLIGHT.backtrace[..INFLIGHT.backtrace_size].as_mut());
}
pub unsafe extern fn reraise() -> ! {
use cslice::AsCSlice; use cslice::AsCSlice;
let count = EXCEPTION_BUFFER.exception_count; // Reraise is basically cxa_rethrow, which calls _Unwind_Resume_or_Rethrow,
let stack = &mut EXCEPTION_BUFFER.exception_stack; // which for EHABI would always call _Unwind_RaiseException.
let diff = exception as isize - EXCEPTION_BUFFER.exceptions.as_ptr() as isize; match INFLIGHT.exception {
if 0 <= diff && diff <= (mem::size_of::<Option<Exception>>() * MAX_INFLIGHT_EXCEPTIONS) as isize { Some(ref exception) => raise(exception),
let index = diff / (mem::size_of::<Option<Exception>>() as isize); None => raise(&Exception {
trace!("reraise at {}", index); name: "0:artiq.coredevice.exceptions.RuntimeError".as_c_slice(),
file: file!().as_c_slice(),
let mut found = false; line: line!(),
for i in 0..=MAX_INFLIGHT_EXCEPTIONS + 1 { column: column!(),
if found { // https://github.com/rust-lang/rfcs/pull/1719
if stack[i] == -1 { function: "__artiq_reraise".as_c_slice(),
stack[i - 1] = index; message: "No active exception to reraise".as_c_slice(),
assert!(i == count); param: [0, 0, 0]
break; })
} else {
stack[i - 1] = stack[i];
}
} else {
if stack[i] == index {
found = true;
}
}
}
assert!(found);
let _result = _Unwind_ForcedUnwind(
&mut EXCEPTION_BUFFER.uw_exceptions[stack[count - 1] as usize],
stop_fn,
core::ptr::null_mut(),
);
} else {
if count < MAX_INFLIGHT_EXCEPTIONS {
trace!("raising exception at level {}", count);
let exception = &*exception;
for (i, slot) in EXCEPTION_BUFFER.exceptions.iter_mut().enumerate() {
// we should always be able to find a slot
if slot.is_none() {
*slot = Some(*mem::transmute::<*const Exception, *const Exception<'static>>(
exception,
));
EXCEPTION_BUFFER.exception_stack[count] = i as isize;
EXCEPTION_BUFFER.uw_exceptions[i].private = [0; uw::unwinder_private_data_size];
EXCEPTION_BUFFER.stack_pointers[i] = StackPointerBacktrace {
stack_pointer: 0,
initial_backtrace_size: EXCEPTION_BUFFER.backtrace_size,
current_backtrace_size: 0,
};
EXCEPTION_BUFFER.exception_count += 1;
let _result =
_Unwind_ForcedUnwind(&mut EXCEPTION_BUFFER.uw_exceptions[i], stop_fn, core::ptr::null_mut());
}
}
} else {
error!("too many nested exceptions");
// TODO: better reporting?
let exception = Exception {
id: get_exception_id("RuntimeError"),
file: file!().as_c_slice(),
line: line!(),
column: column!(),
// https://github.com/rust-lang/rfcs/pull/1719
function: "__artiq_raise".as_c_slice(),
message: "too many nested exceptions".as_c_slice(),
param: [0, 0, 0],
};
EXCEPTION_BUFFER.exceptions[MAX_INFLIGHT_EXCEPTIONS] = Some(mem::transmute(exception));
EXCEPTION_BUFFER.stack_pointers[MAX_INFLIGHT_EXCEPTIONS] = Default::default();
EXCEPTION_BUFFER.exception_count += 1;
uncaught_exception()
}
} }
unreachable!();
}
pub unsafe extern "C" fn resume() -> ! {
trace!("resume");
assert!(EXCEPTION_BUFFER.exception_count != 0);
let i = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
assert!(i != -1);
let _result = _Unwind_ForcedUnwind(
&mut EXCEPTION_BUFFER.uw_exceptions[i as usize],
stop_fn,
core::ptr::null_mut(),
);
unreachable!()
}
pub unsafe extern "C" fn end_catch() {
let mut count = EXCEPTION_BUFFER.exception_count;
assert!(count != 0);
// we remove all exceptions with SP <= current exception SP
// i.e. the outer exception escapes the finally block
let index = EXCEPTION_BUFFER.exception_stack[count - 1] as usize;
EXCEPTION_BUFFER.exception_stack[count - 1] = -1;
EXCEPTION_BUFFER.exceptions[index] = None;
let outer_sp = EXCEPTION_BUFFER.stack_pointers[index].stack_pointer;
count -= 1;
for i in (0..count).rev() {
let index = EXCEPTION_BUFFER.exception_stack[i];
assert!(index != -1);
let index = index as usize;
let sp = EXCEPTION_BUFFER.stack_pointers[index].stack_pointer;
if sp >= outer_sp {
break;
}
EXCEPTION_BUFFER.exceptions[index] = None;
EXCEPTION_BUFFER.exception_stack[i] = -1;
count -= 1;
}
EXCEPTION_BUFFER.exception_count = count;
EXCEPTION_BUFFER.backtrace_size = if count > 0 {
let index = EXCEPTION_BUFFER.exception_stack[count - 1];
assert!(index != -1);
EXCEPTION_BUFFER.stack_pointers[index as usize].current_backtrace_size
} else {
0
};
}
extern "C" fn cleanup(_unwind_code: uw::_Unwind_Reason_Code, _uw_exception: *mut uw::_Unwind_Exception) {
unimplemented!()
}
fn uncaught_exception() -> ! {
unsafe {
// dump way to reorder the stack
for i in 0..EXCEPTION_BUFFER.exception_count {
if EXCEPTION_BUFFER.exception_stack[i] != i as isize {
// find the correct index
let index = EXCEPTION_BUFFER
.exception_stack
.iter()
.position(|v| *v == i as isize)
.unwrap();
let a = EXCEPTION_BUFFER.exception_stack[index];
let b = EXCEPTION_BUFFER.exception_stack[i];
assert!(a != -1 && b != -1);
core::mem::swap(
&mut EXCEPTION_BUFFER.exception_stack[index],
&mut EXCEPTION_BUFFER.exception_stack[i],
);
core::mem::swap(
&mut EXCEPTION_BUFFER.exceptions[a as usize],
&mut EXCEPTION_BUFFER.exceptions[b as usize],
);
core::mem::swap(
&mut EXCEPTION_BUFFER.stack_pointers[a as usize],
&mut EXCEPTION_BUFFER.stack_pointers[b as usize],
);
}
}
}
unsafe {
crate::kernel::core1::terminate(
EXCEPTION_BUFFER.exceptions[..EXCEPTION_BUFFER.exception_count].as_ref(),
EXCEPTION_BUFFER.stack_pointers[..EXCEPTION_BUFFER.exception_count].as_ref(),
EXCEPTION_BUFFER.backtrace[..EXCEPTION_BUFFER.backtrace_size].as_mut(),
)
}
}
// stop function which would be executed when we unwind each frame
extern "C" fn stop_fn(
_version: c_int,
actions: i32,
_uw_exception_class: uw::_Unwind_Exception_Class,
_uw_exception: *mut uw::_Unwind_Exception,
context: *mut uw::_Unwind_Context,
_stop_parameter: *mut c_void,
) -> uw::_Unwind_Reason_Code {
unsafe {
let load_addr = KERNEL_IMAGE.as_ref().unwrap().get_load_addr();
let backtrace_size = EXCEPTION_BUFFER.backtrace_size;
// we try to remove unrelated backtrace here to save some buffer size
if backtrace_size < MAX_BACKTRACE_SIZE {
let ip = uw::_Unwind_GetIP(context);
if ip >= load_addr {
let ip = ip - load_addr;
let sp = uw::_Unwind_GetGR(context, uw::UNWIND_SP_REG);
trace!("SP: {:X}, backtrace_size: {}", sp, backtrace_size);
EXCEPTION_BUFFER.backtrace[backtrace_size] = (ip, sp);
EXCEPTION_BUFFER.backtrace_size += 1;
let last_index = EXCEPTION_BUFFER.exception_stack[EXCEPTION_BUFFER.exception_count - 1];
assert!(last_index != -1);
let sp_info = &mut EXCEPTION_BUFFER.stack_pointers[last_index as usize];
sp_info.stack_pointer = sp;
sp_info.current_backtrace_size = backtrace_size + 1;
}
} else {
trace!("backtrace size exceeded");
}
if actions as u32 & uw::_US_END_OF_STACK as u32 != 0 {
uncaught_exception()
} else {
uw::_URC_NO_REASON
}
}
}
// Must be kept in sync with preallocate_runtime_exception_names() in artiq/language/embedding_map.py
static EXCEPTION_ID_LOOKUP: [(&str, u32); 11] = [
("RuntimeError", 0),
("RTIOUnderflow", 1),
("RTIOOverflow", 2),
("RTIODestinationUnreachable", 3),
("DMAError", 4),
("I2CError", 5),
("CacheError", 6),
("SPIError", 7),
("ZeroDivisionError", 8),
("IndexError", 9),
("UnwrapNoneError", 10),
];
pub fn get_exception_id(name: &str) -> u32 {
for (n, id) in EXCEPTION_ID_LOOKUP.iter() {
if *n == name {
return *id;
}
}
unimplemented!("unallocated internal exception id")
} }
#[macro_export] #[macro_export]
macro_rules! artiq_raise { macro_rules! artiq_raise {
($name:expr, $message:expr, $param0:expr, $param1:expr, $param2:expr) => {{ ($name:expr, $message:expr, $param0:expr, $param1:expr, $param2:expr) => ({
use cslice::AsCSlice; use cslice::AsCSlice;
let name_id = $crate::eh_artiq::get_exception_id($name);
let message_cl = $message.clone();
let exn = $crate::eh_artiq::Exception { let exn = $crate::eh_artiq::Exception {
id: name_id, name: concat!("0:artiq.coredevice.exceptions.", $name).as_c_slice(),
file: file!().as_c_slice(), file: file!().as_c_slice(),
line: line!(), line: line!(),
column: column!(), column: column!(),
// https://github.com/rust-lang/rfcs/pull/1719 // https://github.com/rust-lang/rfcs/pull/1719
function: "(Rust function)".as_c_slice(), function: "(Rust function)".as_c_slice(),
message: message_cl.as_c_slice(), message: $message.as_c_slice(),
param: [$param0, $param1, $param2], param: [$param0, $param1, $param2]
}; };
#[allow(unused_unsafe)] #[allow(unused_unsafe)]
unsafe { unsafe { $crate::eh_artiq::raise(&exn) }
$crate::eh_artiq::raise(&exn) });
} ($name:expr, $message:expr) => ({
}}; artiq_raise!($name, $message, 0, 0, 0)
($name:expr, $message:expr) => {{ artiq_raise!($name, $message, 0, 0, 0) }}; });
} }

155
src/runtime/src/i2c.rs
View File

@ -1,97 +1,100 @@
use libboard_zynq; #[cfg(feature = "target_zc706")]
mod i2c {
use libboard_zynq;
use crate::artiq_raise;
use crate::artiq_raise; static mut I2C_BUS: Option<libboard_zynq::i2c::I2c> = None;
pub static mut I2C_BUS: Option<libboard_zynq::i2c::I2c> = None; pub extern fn start(busno: i32) {
if busno > 0 {
pub extern "C" fn start(busno: i32) { artiq_raise!("I2CError", "I2C bus could not be accessed");
if busno > 0 { }
artiq_raise!("I2CError", "I2C bus could not be accessed"); unsafe {
} if (&mut I2C_BUS).as_mut().unwrap().start().is_err() {
unsafe { artiq_raise!("I2CError", "I2C start failed");
if (&mut I2C_BUS).as_mut().unwrap().start().is_err() { }
artiq_raise!("I2CError", "I2C start failed");
} }
} }
}
pub extern "C" fn restart(busno: i32) { pub extern fn restart(busno: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
unsafe { unsafe {
if (&mut I2C_BUS).as_mut().unwrap().restart().is_err() { if (&mut I2C_BUS).as_mut().unwrap().restart().is_err() {
artiq_raise!("I2CError", "I2C restart failed"); artiq_raise!("I2CError", "I2C restart failed");
}
} }
} }
}
pub extern "C" fn stop(busno: i32) { pub extern fn stop(busno: i32) {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
unsafe { unsafe {
if (&mut I2C_BUS).as_mut().unwrap().stop().is_err() { if (&mut I2C_BUS).as_mut().unwrap().stop().is_err() {
artiq_raise!("I2CError", "I2C stop failed"); artiq_raise!("I2CError", "I2C stop failed");
}
} }
} }
}
pub extern "C" fn write(busno: i32, data: i32) -> bool { pub extern fn write(busno: i32, data: i32) -> bool {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
unsafe { unsafe {
match (&mut I2C_BUS).as_mut().unwrap().write(data as u8) { match (&mut I2C_BUS).as_mut().unwrap().write(data as u8) {
Ok(r) => r, Ok(r) => r,
Err(_) => artiq_raise!("I2CError", "I2C write failed"), Err(_) => artiq_raise!("I2CError", "I2C write failed"),
}
} }
} }
}
pub extern "C" fn read(busno: i32, ack: bool) -> i32 { pub extern fn read(busno: i32, ack: bool) -> i32 {
if busno > 0 { if busno > 0 {
artiq_raise!("I2CError", "I2C bus could not be accessed"); artiq_raise!("I2CError", "I2C bus could not be accessed");
} }
unsafe { unsafe {
match (&mut I2C_BUS).as_mut().unwrap().read(ack) { match (&mut I2C_BUS).as_mut().unwrap().read(ack) {
Ok(r) => r as i32, Ok(r) => r as i32,
Err(_) => artiq_raise!("I2CError", "I2C read failed"), Err(_) => artiq_raise!("I2CError", "I2C read failed"),
}
} }
} }
}
pub extern "C" fn switch_select(busno: i32, address: i32, mask: i32) { pub fn init() {
if busno > 0 { let mut i2c = libboard_zynq::i2c::I2c::i2c0();
artiq_raise!("I2CError", "I2C bus could not be accessed"); i2c.init().expect("I2C bus initialization failed");
} unsafe { I2C_BUS = Some(i2c) };
let ch = match mask {
//decode from mainline, PCA9548-centric API
0x00 => None,
0x01 => Some(0),
0x02 => Some(1),
0x04 => Some(2),
0x08 => Some(3),
0x10 => Some(4),
0x20 => Some(5),
0x40 => Some(6),
0x80 => Some(7),
_ => artiq_raise!("I2CError", "switch select supports only one channel"),
};
unsafe {
if (&mut I2C_BUS)
.as_mut()
.unwrap()
.pca954x_select(address as u8, ch)
.is_err()
{
artiq_raise!("I2CError", "switch select failed");
}
} }
} }
pub fn init() { #[cfg(not(feature = "target_zc706"))]
let mut i2c = libboard_zynq::i2c::I2c::i2c0(); mod i2c {
i2c.init().expect("I2C bus initialization failed"); use crate::artiq_raise;
unsafe { I2C_BUS = Some(i2c) };
pub extern fn start(_busno: i32) {
artiq_raise!("I2CError", "No I2C bus");
}
pub extern fn restart(_busno: i32) {
artiq_raise!("I2CError", "No I2C bus");
}
pub extern fn stop(_busno: i32) {
artiq_raise!("I2CError", "No I2C bus");
}
pub extern fn write(_busno: i32, _data: i32) -> bool {
artiq_raise!("I2CError", "No I2C bus");
}
pub extern fn read(_busno: i32, _ack: bool) -> i32 {
artiq_raise!("I2CError", "No I2C bus");
}
pub fn init() {
}
} }
pub use i2c::*;

9
src/runtime/src/irq.rs
View File

@ -1,8 +1,11 @@
use core::sync::atomic::{AtomicBool, Ordering};
use libboard_zynq::{gic, mpcore, println, stdio}; use libboard_zynq::{gic, mpcore, println, stdio};
use libcortex_a9::{asm, interrupt_handler, notify_spin_lock, regs::MPIDR, spin_lock_yield}; use libcortex_a9::{
asm, interrupt_handler,
regs::MPIDR,
spin_lock_yield, notify_spin_lock
};
use libregister::RegisterR; use libregister::RegisterR;
use core::sync::atomic::{AtomicBool, Ordering};
extern "C" { extern "C" {
static mut __stack1_start: u32; static mut __stack1_start: u32;

96
src/runtime/src/kernel/api.rs
View File

@ -1,25 +1,29 @@
use alloc::vec; use core::ffi::VaList;
use core::{ffi::VaList, ptr, str}; use core::ptr;
use core::str;
use libc::{c_char, c_int, size_t}; use libc::{c_char, c_int, size_t};
use libm; use libm;
use log::{info, warn}; use log::{info, warn};
use super::{cache, use alloc::vec;
core1::rtio_get_destination_status,
dma, use crate::eh_artiq;
rpc::{rpc_recv, rpc_send, rpc_send_async}}; use crate::rtio;
use crate::{eh_artiq, i2c, rtio}; use crate::i2c;
use super::rpc::{rpc_send, rpc_send_async, rpc_recv};
use super::dma;
use super::cache;
extern "C" { extern "C" {
fn vsnprintf_(buffer: *mut c_char, count: size_t, format: *const c_char, va: VaList) -> c_int; fn vsnprintf_(buffer: *mut c_char, count: size_t, format: *const c_char, va: VaList) -> c_int;
} }
unsafe extern "C" fn core_log(fmt: *const c_char, mut args: ...) { unsafe extern fn core_log(fmt: *const c_char, mut args: ...) {
let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize; let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize;
let mut buf = vec![0; size + 1]; let mut buf = vec![0; size + 1];
vsnprintf_(buf.as_mut_ptr() as *mut i8, size + 1, fmt, args.as_va_list()); vsnprintf_(buf.as_mut_ptr() as *mut i8, size + 1, fmt, args.as_va_list());
let buf: &[u8] = &buf.as_slice()[..size - 1]; // strip \n and NUL let buf: &[u8] = &buf.as_slice()[..size-1]; // strip \n and NUL
match str::from_utf8(buf) { match str::from_utf8(buf) {
Ok(s) => info!("kernel: {}", s), Ok(s) => info!("kernel: {}", s),
Err(e) => { Err(e) => {
@ -29,13 +33,14 @@ unsafe extern "C" fn core_log(fmt: *const c_char, mut args: ...) {
} }
} }
unsafe extern "C" fn rtio_log(fmt: *const c_char, mut args: ...) { unsafe extern fn rtio_log(fmt: *const c_char, mut args: ...) {
let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize; let size = vsnprintf_(ptr::null_mut(), 0, fmt, args.as_va_list()) as usize;
let mut buf = vec![0; size + 1]; let mut buf = vec![0; size + 1];
vsnprintf_(buf.as_mut_ptr(), size + 1, fmt, args.as_va_list()); vsnprintf_(buf.as_mut_ptr(), size + 1, fmt, args.as_va_list());
rtio::write_log(buf.as_slice()); rtio::write_log(buf.as_slice());
} }
macro_rules! api { macro_rules! api {
($i:ident) => ({ ($i:ident) => ({
extern { static $i: u8; } extern { static $i: u8; }
@ -51,25 +56,24 @@ macro_rules! api {
} }
macro_rules! api_libm_f64f64 { macro_rules! api_libm_f64f64 {
($i:ident) => {{ ($i:ident) => ({
extern "C" fn $i(x: f64) -> f64 { extern fn $i(x: f64) -> f64 {
libm::$i(x) libm::$i(x)
} }
api!($i = $i) api!($i = $i)
}}; })
} }
macro_rules! api_libm_f64f64f64 { macro_rules! api_libm_f64f64f64 {
($i:ident) => {{ ($i:ident) => ({
extern "C" fn $i(x: f64, y: f64) -> f64 { extern fn $i(x: f64, y: f64) -> f64 {
libm::$i(x, y) libm::$i(x, y)
} }
api!($i = $i) api!($i = $i)
}}; })
} }
pub fn resolve(required: &[u8]) -> Option<u32> { pub fn resolve(required: &[u8]) -> Option<u32> {
#[rustfmt::skip]
let api = &[ let api = &[
// timing // timing
api!(now_mu = rtio::now_mu), api!(now_mu = rtio::now_mu),
@ -83,7 +87,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
// rtio // rtio
api!(rtio_init = rtio::init), api!(rtio_init = rtio::init),
api!(rtio_get_destination_status = rtio_get_destination_status), api!(rtio_get_destination_status = rtio::get_destination_status),
api!(rtio_get_counter = rtio::get_counter), api!(rtio_get_counter = rtio::get_counter),
api!(rtio_output = rtio::output), api!(rtio_output = rtio::output),
api!(rtio_output_wide = rtio::output_wide), api!(rtio_output_wide = rtio::output_wide),
@ -112,7 +116,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(i2c_stop = i2c::stop), api!(i2c_stop = i2c::stop),
api!(i2c_write = i2c::write), api!(i2c_write = i2c::write),
api!(i2c_read = i2c::read), api!(i2c_read = i2c::read),
api!(i2c_switch_select = i2c::switch_select),
// Double-precision floating-point arithmetic helper functions // Double-precision floating-point arithmetic helper functions
// RTABI chapter 4.1.2, Table 2 // RTABI chapter 4.1.2, Table 2
@ -120,7 +123,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_ddiv), api!(__aeabi_ddiv),
api!(__aeabi_dmul), api!(__aeabi_dmul),
api!(__aeabi_dsub), api!(__aeabi_dsub),
// Double-precision floating-point comparison helper functions // Double-precision floating-point comparison helper functions
// RTABI chapter 4.1.2, Table 3 // RTABI chapter 4.1.2, Table 3
api!(__aeabi_dcmpeq), api!(__aeabi_dcmpeq),
@ -130,14 +132,12 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_dcmpge), api!(__aeabi_dcmpge),
api!(__aeabi_dcmpgt), api!(__aeabi_dcmpgt),
api!(__aeabi_dcmpun), api!(__aeabi_dcmpun),
// Single-precision floating-point arithmetic helper functions // Single-precision floating-point arithmetic helper functions
// RTABI chapter 4.1.2, Table 4 // RTABI chapter 4.1.2, Table 4
api!(__aeabi_fadd), api!(__aeabi_fadd),
api!(__aeabi_fdiv), api!(__aeabi_fdiv),
api!(__aeabi_fmul), api!(__aeabi_fmul),
api!(__aeabi_fsub), api!(__aeabi_fsub),
// Single-precision floating-point comparison helper functions // Single-precision floating-point comparison helper functions
// RTABI chapter 4.1.2, Table 5 // RTABI chapter 4.1.2, Table 5
api!(__aeabi_fcmpeq), api!(__aeabi_fcmpeq),
@ -147,7 +147,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_fcmpge), api!(__aeabi_fcmpge),
api!(__aeabi_fcmpgt), api!(__aeabi_fcmpgt),
api!(__aeabi_fcmpun), api!(__aeabi_fcmpun),
// Floating-point to integer conversions. // Floating-point to integer conversions.
// RTABI chapter 4.1.2, Table 6 // RTABI chapter 4.1.2, Table 6
api!(__aeabi_d2iz), api!(__aeabi_d2iz),
@ -158,11 +157,9 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_f2uiz), api!(__aeabi_f2uiz),
api!(__aeabi_f2lz), api!(__aeabi_f2lz),
api!(__aeabi_f2ulz), api!(__aeabi_f2ulz),
// Conversions between floating types. // Conversions between floating types.
// RTABI chapter 4.1.2, Table 7 // RTABI chapter 4.1.2, Table 7
api!(__aeabi_f2d), api!(__aeabi_f2d),
// Integer to floating-point conversions. // Integer to floating-point conversions.
// RTABI chapter 4.1.2, Table 8 // RTABI chapter 4.1.2, Table 8
api!(__aeabi_i2d), api!(__aeabi_i2d),
@ -173,14 +170,12 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_ui2f), api!(__aeabi_ui2f),
api!(__aeabi_l2f), api!(__aeabi_l2f),
api!(__aeabi_ul2f), api!(__aeabi_ul2f),
// Long long helper functions // Long long helper functions
// RTABI chapter 4.2, Table 9 // RTABI chapter 4.2, Table 9
api!(__aeabi_lmul), api!(__aeabi_lmul),
api!(__aeabi_llsl), api!(__aeabi_llsl),
api!(__aeabi_llsr), api!(__aeabi_llsr),
api!(__aeabi_lasr), api!(__aeabi_lasr),
// Integer division functions // Integer division functions
// RTABI chapter 4.3.1 // RTABI chapter 4.3.1
api!(__aeabi_idiv), api!(__aeabi_idiv),
@ -188,7 +183,6 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_idivmod), api!(__aeabi_idivmod),
api!(__aeabi_uidiv), api!(__aeabi_uidiv),
api!(__aeabi_uldivmod), api!(__aeabi_uldivmod),
// 4.3.4 Memory copying, clearing, and setting // 4.3.4 Memory copying, clearing, and setting
api!(__aeabi_memcpy8), api!(__aeabi_memcpy8),
api!(__aeabi_memcpy4), api!(__aeabi_memcpy4),
@ -204,43 +198,13 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api!(__aeabi_memclr), api!(__aeabi_memclr),
// libc // libc
api!( api!(memcmp, extern { fn memcmp(a: *const u8, b: *mut u8, size: usize); }),
memcpy,
extern "C" {
fn memcpy(dest: *mut u8, src: *const u8, n: usize) -> *mut u8;
}
),
api!(
memmove,
extern "C" {
fn memmove(dest: *mut u8, src: *const u8, n: usize) -> *mut u8;
}
),
api!(
memset,
extern "C" {
fn memset(s: *mut u8, c: i32, n: usize) -> *mut u8;
}
),
api!(
memcmp,
extern "C" {
fn memcmp(s1: *const u8, s2: *const u8, n: usize) -> i32;
}
),
// exceptions // exceptions
api!(_Unwind_Resume = unwind::_Unwind_Resume), api!(_Unwind_Resume = unwind::_Unwind_Resume),
api!(__nac3_personality = eh_artiq::artiq_personality),
api!(__nac3_raise = eh_artiq::raise),
api!(__nac3_resume = eh_artiq::resume),
api!(__nac3_end_catch = eh_artiq::end_catch),
// legacy exception symbols
api!(__artiq_personality = eh_artiq::artiq_personality), api!(__artiq_personality = eh_artiq::artiq_personality),
api!(__artiq_raise = eh_artiq::raise), api!(__artiq_raise = eh_artiq::raise),
api!(__artiq_resume = eh_artiq::resume), api!(__artiq_reraise = eh_artiq::reraise),
api!(__artiq_end_catch = eh_artiq::end_catch),
// Implementations for LLVM math intrinsics // Implementations for LLVM math intrinsics
api!(__powidf2), api!(__powidf2),
@ -267,7 +231,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(fabs), api_libm_f64f64!(fabs),
api_libm_f64f64!(floor), api_libm_f64f64!(floor),
{ {
extern "C" fn fma(x: f64, y: f64, z: f64) -> f64 { extern fn fma(x: f64, y: f64, z: f64) -> f64 {
libm::fma(x, y, z) libm::fma(x, y, z)
} }
api!(fma = fma) api!(fma = fma)
@ -279,7 +243,7 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(j0), api_libm_f64f64!(j0),
api_libm_f64f64!(j1), api_libm_f64f64!(j1),
{ {
extern "C" fn jn(n: i32, x: f64) -> f64 { extern fn jn(n: i32, x: f64) -> f64 {
libm::jn(n, x) libm::jn(n, x)
} }
api!(jn = jn) api!(jn = jn)
@ -301,13 +265,13 @@ pub fn resolve(required: &[u8]) -> Option<u32> {
api_libm_f64f64!(y0), api_libm_f64f64!(y0),
api_libm_f64f64!(y1), api_libm_f64f64!(y1),
{ {
extern "C" fn yn(n: i32, x: f64) -> f64 { extern fn yn(n: i32, x: f64) -> f64 {
libm::yn(n, x) libm::yn(n, x)
} }
api!(yn = yn) api!(yn = yn)
}, },
]; ];
api.iter() api.iter()
.find(|&&(exported, _)| exported.as_bytes() == required) .find(|&&(exported, _)| exported.as_bytes() == required)
.map(|&(_, ptr)| ptr as u32) .map(|&(_, ptr)| ptr as u32)
} }

33
src/runtime/src/kernel/cache.rs
View File

@ -1,37 +1,30 @@
use alloc::{boxed::Box, string::String}; use alloc::string::String;
use core::mem::{forget, transmute}; use cslice::{CSlice, AsCSlice};
use core::mem::{transmute, forget};
use super::{KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, Message};
use cslice::{AsCSlice, CSlice}; pub extern fn get(key: CSlice<u8>) -> CSlice<'static, i32> {
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0};
pub extern "C" fn get(key: CSlice<u8>) -> &CSlice<'static, i32> {
let key = String::from_utf8(key.as_ref().to_vec()).unwrap(); let key = String::from_utf8(key.as_ref().to_vec()).unwrap();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::CacheGetRequest(key));
.as_mut()
.unwrap()
.send(Message::CacheGetRequest(key));
let msg = KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv(); let msg = KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv();
if let Message::CacheGetReply(v) = msg { if let Message::CacheGetReply(v) = msg {
let leaked = Box::new(v.as_c_slice()); let slice = transmute(v.as_c_slice());
let reference = transmute(leaked.as_ref()); // we intentionally leak the memory here,
forget(leaked); // which does not matter as core1 would restart
forget(v); forget(v);
reference slice
} else { } else {
panic!("Expected CacheGetReply for CacheGetRequest"); panic!("Expected CacheGetReply for CacheGetRequest");
} }
} }
} }
pub extern "C" fn put(key: CSlice<u8>, list: &CSlice<i32>) { pub extern fn put(key: CSlice<u8>, list: CSlice<i32>) {
let key = String::from_utf8(key.as_ref().to_vec()).unwrap(); let key = String::from_utf8(key.as_ref().to_vec()).unwrap();
let value = list.as_ref().to_vec(); let value = list.as_ref().to_vec();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::CachePutRequest(key, value));
.as_mut()
.unwrap()
.send(Message::CachePutRequest(key, value));
} }
} }

9
src/runtime/src/kernel/control.rs
View File

@ -1,11 +1,11 @@
use core::mem::{forget, replace}; use libcortex_a9::sync_channel::{Sender, Receiver};
use libcortex_a9::sync_channel::{Receiver, Sender};
use libsupport_zynq::boot::Core1; use libsupport_zynq::boot::Core1;
use super::{Message, CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK}; use super::{CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK, Message};
use crate::irq::restart_core1; use crate::irq::restart_core1;
use core::mem::{forget, replace};
pub struct Control { pub struct Control {
pub tx: Sender<'static, Message>, pub tx: Sender<'static, Message>,
pub rx: Receiver<'static, Message>, pub rx: Receiver<'static, Message>,
@ -53,3 +53,4 @@ impl Control {
forget(replace(&mut self.rx, core0_rx)); forget(replace(&mut self.rx, core0_rx));
} }
} }

143
src/runtime/src/kernel/core1.rs
View File

@ -1,39 +1,50 @@
//! Kernel prologue/epilogue that runs on the 2nd CPU core //! Kernel prologue/epilogue that runs on the 2nd CPU core
use core::{mem, ptr, cell::UnsafeCell};
use alloc::borrow::ToOwned; use alloc::borrow::ToOwned;
use core::{cell::UnsafeCell, mem, ptr}; use log::{debug, info, error};
use cslice::CSlice; use cslice::CSlice;
use dyld::{self, elf::EXIDX_Entry, Library};
use libboard_zynq::{gic, mpcore};
use libcortex_a9::{asm::{dsb, isb},
cache::{bpiall, dcci_slice, iciallu},
enable_fpu, sync_channel};
use libsupport_zynq::ram;
use log::{debug, error, info};
use super::{api::resolve, dma, rpc::rpc_send_async, Message, CHANNEL_0TO1, CHANNEL_1TO0, CHANNEL_SEM, INIT_LOCK, use libcortex_a9::{
KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, KERNEL_IMAGE}; enable_fpu,
use crate::{eh_artiq, get_async_errors, rtio}; cache::{dcci_slice, iciallu, bpiall},
asm::{dsb, isb},
sync_channel,
};
use libboard_zynq::{mpcore, gic};
use libsupport_zynq::ram;
use dyld::{self, Library};
use crate::{eh_artiq, rtio};
use super::{
api::resolve,
rpc::rpc_send_async,
INIT_LOCK,
CHANNEL_0TO1, CHANNEL_1TO0,
CHANNEL_SEM,
KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0,
KERNEL_IMAGE,
Message,
dma,
};
// linker symbols // linker symbols
extern "C" { extern "C" {
static __text_start: u32; static __text_start: u32;
static __text_end: u32; static __text_end: u32;
static __exidx_start: EXIDX_Entry; static __exidx_start: u32;
static __exidx_end: EXIDX_Entry; static __exidx_end: u32;
} }
unsafe fn attribute_writeback(typeinfo: *const ()) { unsafe fn attribute_writeback(typeinfo: *const ()) {
struct Attr { struct Attr {
offset: usize, offset: usize,
tag: CSlice<'static, u8>, tag: CSlice<'static, u8>,
name: CSlice<'static, u8>, name: CSlice<'static, u8>
} }
struct Type { struct Type {
attributes: *const *const Attr, attributes: *const *const Attr,
objects: *const *const (), objects: *const *const ()
} }
let mut tys = typeinfo as *const *const Type; let mut tys = typeinfo as *const *const Type;
@ -52,16 +63,11 @@ unsafe fn attribute_writeback(typeinfo: *const ()) {
attributes = attributes.offset(1); attributes = attributes.offset(1);
if (*attribute).tag.len() > 0 { if (*attribute).tag.len() > 0 {
rpc_send_async( rpc_send_async(0, &(*attribute).tag, [
0, &object as *const _ as *const (),
&(*attribute).tag, &(*attribute).name as *const _ as *const (),
[ (object as usize + (*attribute).offset) as *const ()
&object as *const _ as *const (), ].as_ptr());
&(*attribute).name as *const _ as *const (),
(object as usize + (*attribute).offset) as *const (),
]
.as_ptr(),
);
} }
} }
} }
@ -76,8 +82,7 @@ pub struct KernelImage {
impl KernelImage { impl KernelImage {
pub fn new(library: Library) -> Result<Self, dyld::Error> { pub fn new(library: Library) -> Result<Self, dyld::Error> {
let __modinit__ = library let __modinit__ = library.lookup(b"__modinit__")
.lookup(b"__modinit__")
.ok_or(dyld::Error::Lookup("__modinit__".to_owned()))?; .ok_or(dyld::Error::Lookup("__modinit__".to_owned()))?;
let typeinfo = library.lookup(b"typeinfo"); let typeinfo = library.lookup(b"typeinfo");
@ -85,7 +90,8 @@ impl KernelImage {
let bss_start = library.lookup(b"__bss_start"); let bss_start = library.lookup(b"__bss_start");
let end = library.lookup(b"_end"); let end = library.lookup(b"_end");
if let Some(bss_start) = bss_start { if let Some(bss_start) = bss_start {
let end = end.ok_or(dyld::Error::Lookup("_end".to_owned()))?; let end = end
.ok_or(dyld::Error::Lookup("_end".to_owned()))?;
unsafe { unsafe {
ptr::write_bytes(bss_start as *mut u8, 0, (end - bss_start) as usize); ptr::write_bytes(bss_start as *mut u8, 0, (end - bss_start) as usize);
} }
@ -120,7 +126,9 @@ impl KernelImage {
} }
pub fn get_load_addr(&self) -> usize { pub fn get_load_addr(&self) -> usize {
unsafe { self.library.get().as_ref().unwrap().image.as_ptr() as usize } unsafe {
self.library.get().as_ref().unwrap().image.as_ptr() as usize
}
} }
} }
@ -156,24 +164,24 @@ pub extern "C" fn main_core1() {
let message = core1_rx.recv(); let message = core1_rx.recv();
match message { match message {
Message::LoadRequest(data) => { Message::LoadRequest(data) => {
let result = dyld::load(&data, &resolve).and_then(KernelImage::new); let result = dyld::load(&data, &resolve)
.and_then(KernelImage::new);
match result { match result {
Ok(kernel) => { Ok(kernel) => {
loaded_kernel = Some(kernel); loaded_kernel = Some(kernel);
debug!("kernel loaded"); debug!("kernel loaded");
core1_tx.send(Message::LoadCompleted); core1_tx.send(Message::LoadCompleted);
} },
Err(error) => { Err(error) => {
error!("failed to load shared library: {}", error); error!("failed to load shared library: {}", error);
core1_tx.send(Message::LoadFailed); core1_tx.send(Message::LoadFailed);
} }
} }
} },
Message::StartRequest => { Message::StartRequest => {
info!("kernel starting"); info!("kernel starting");
if let Some(kernel) = loaded_kernel.take() { if let Some(kernel) = loaded_kernel.take() {
unsafe { unsafe {
eh_artiq::reset_exception_buffer();
KERNEL_CHANNEL_0TO1 = Some(core1_rx); KERNEL_CHANNEL_0TO1 = Some(core1_rx);
KERNEL_CHANNEL_1TO0 = Some(core1_tx); KERNEL_CHANNEL_1TO0 = Some(core1_tx);
KERNEL_IMAGE = &kernel as *const KernelImage; KERNEL_IMAGE = &kernel as *const KernelImage;
@ -184,8 +192,7 @@ pub extern "C" fn main_core1() {
} }
} }
info!("kernel finished"); info!("kernel finished");
let async_errors = unsafe { get_async_errors() }; core1_tx.send(Message::KernelFinished);
core1_tx.send(Message::KernelFinished(async_errors));
} }
_ => error!("Core1 received unexpected message: {:?}", message), _ => error!("Core1 received unexpected message: {:?}", message),
} }
@ -193,62 +200,44 @@ pub extern "C" fn main_core1() {
} }
/// Called by eh_artiq /// Called by eh_artiq
pub fn terminate( pub fn terminate(exception: &'static eh_artiq::Exception<'static>, backtrace: &'static mut [usize]) -> ! {
exceptions: &'static [Option<eh_artiq::Exception<'static>>], let load_addr = unsafe {
stack_pointers: &'static [eh_artiq::StackPointerBacktrace], KERNEL_IMAGE.as_ref().unwrap().get_load_addr()
backtrace: &'static mut [(usize, usize)], };
) -> ! { let mut cursor = 0;
// The address in the backtrace is relocated, so we have to convert it back to the address in
// the original python script, and remove those Rust function backtrace.
for i in 0..backtrace.len() {
if backtrace[i] >= load_addr {
backtrace[cursor] = backtrace[i] - load_addr;
cursor += 1;
}
}
{ {
let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() }; let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() };
let errors = unsafe { get_async_errors() }; core1_tx.send(Message::KernelException(exception, &backtrace[..cursor]));
core1_tx.send(Message::KernelException(exceptions, stack_pointers, backtrace, errors));
} }
loop {} loop {}
} }
/// Called by llvm_libunwind /// Called by llvm_libunwind
#[no_mangle] #[no_mangle]
extern "C" fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const u32 { extern fn dl_unwind_find_exidx(pc: *const u32, len_ptr: *mut u32) -> *const u32 {
let length; let length;
let start: *const EXIDX_Entry; let start: *const u32;
unsafe { unsafe {
if &__text_start as *const u32 <= pc && pc < &__text_end as *const u32 { if &__text_start as *const u32 <= pc && pc < &__text_end as *const u32 {
length = (&__exidx_end as *const EXIDX_Entry).offset_from(&__exidx_start) as u32; length = (&__exidx_end as *const u32).offset_from(&__exidx_start) as u32;
start = &__exidx_start; start = &__exidx_start;
} else if KERNEL_IMAGE != ptr::null() { } else {
let exidx = KERNEL_IMAGE let exidx = KERNEL_IMAGE.as_ref()
.as_ref()
.expect("dl_unwind_find_exidx kernel image") .expect("dl_unwind_find_exidx kernel image")
.library .library.get().as_ref().unwrap().exidx();
.get()
.as_ref()
.unwrap()
.exidx();
length = exidx.len() as u32; length = exidx.len() as u32;
start = exidx.as_ptr(); start = exidx.as_ptr();
} else {
length = 0;
start = ptr::null();
} }
*len_ptr = length; *len_ptr = length;
} }
start as *const u32 start
}
pub extern "C" fn rtio_get_destination_status(destination: i32) -> bool {
#[cfg(has_drtio)]
if destination > 0 && destination < 255 {
let reply = unsafe {
let core1_rx = KERNEL_CHANNEL_0TO1.as_mut().unwrap();
let core1_tx = KERNEL_CHANNEL_1TO0.as_mut().unwrap();
core1_tx.send(Message::UpDestinationsRequest(destination));
core1_rx.recv()
};
return match reply {
Message::UpDestinationsReply(x) => x,
_ => panic!("received unexpected reply to UpDestinationsRequest: {:?}", reply),
};
}
destination == 0
} }

193
src/runtime/src/kernel/dma.rs
View File

@ -1,15 +1,21 @@
use alloc::{string::String, vec::Vec}; use crate::{
pl::csr,
artiq_raise,
rtio,
};
use alloc::{vec::Vec, string::String, boxed::Box};
use cslice::CSlice;
use super::{KERNEL_IMAGE, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, Message};
use core::mem; use core::mem;
use cslice::CSlice; use libcortex_a9::cache::dcci_slice;
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0, KERNEL_IMAGE}; const ALIGNMENT: usize = 16 * 8;
use crate::{artiq_raise, pl::csr, rtio};
#[repr(C)] #[repr(C)]
pub struct DmaTrace { pub struct DmaTrace {
duration: i64, duration: i64,
address: i32, address: i32,
} }
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
@ -17,7 +23,6 @@ pub struct DmaRecorder {
pub name: String, pub name: String,
pub buffer: Vec<u8>, pub buffer: Vec<u8>,
pub duration: i64, pub duration: i64,
pub enable_ddma: bool,
} }
static mut RECORDER: Option<DmaRecorder> = None; static mut RECORDER: Option<DmaRecorder> = None;
@ -28,13 +33,10 @@ pub unsafe fn init_dma_recorder() {
mem::forget(mem::replace(&mut RECORDER, None)); mem::forget(mem::replace(&mut RECORDER, None));
} }
pub extern "C" fn dma_record_start(name: CSlice<u8>) { pub extern fn dma_record_start(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap(); let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaEraseRequest(name.clone()));
.as_mut()
.unwrap()
.send(Message::DmaEraseRequest(name.clone()));
} }
unsafe { unsafe {
if RECORDER.is_some() { if RECORDER.is_some() {
@ -42,44 +44,42 @@ pub extern "C" fn dma_record_start(name: CSlice<u8>) {
} }
let library = KERNEL_IMAGE.as_ref().unwrap(); let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output", dma_record_output as *const ()).unwrap(); library.rebind(b"rtio_output",
library dma_record_output as *const ()).unwrap();
.rebind(b"rtio_output_wide", dma_record_output_wide as *const ()) library.rebind(b"rtio_output_wide",
.unwrap(); dma_record_output_wide as *const ()).unwrap();
RECORDER = Some(DmaRecorder { RECORDER = Some(DmaRecorder {
name, name,
buffer: Vec::new(), buffer: Vec::new(),
duration: 0, duration: 0,
enable_ddma: false,
}); });
} }
} }
pub extern "C" fn dma_record_stop(duration: i64, enable_ddma: bool) { pub extern fn dma_record_stop(duration: i64) {
unsafe { unsafe {
if RECORDER.is_none() { if RECORDER.is_none() {
artiq_raise!("DMAError", "DMA is not recording") artiq_raise!("DMAError", "DMA is not recording")
} }
let library = KERNEL_IMAGE.as_ref().unwrap(); let library = KERNEL_IMAGE.as_ref().unwrap();
library.rebind(b"rtio_output", rtio::output as *const ()).unwrap(); library.rebind(b"rtio_output",
library rtio::output as *const ()).unwrap();
.rebind(b"rtio_output_wide", rtio::output_wide as *const ()) library.rebind(b"rtio_output_wide",
.unwrap(); rtio::output_wide as *const ()).unwrap();
let mut recorder = RECORDER.take().unwrap(); let mut recorder = RECORDER.take().unwrap();
recorder.duration = duration; recorder.duration = duration;
recorder.enable_ddma = enable_ddma; KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(
KERNEL_CHANNEL_1TO0 Message::DmaPutRequest(recorder)
.as_mut() );
.unwrap()
.send(Message::DmaPutRequest(recorder));
} }
} }
#[inline(always)] #[inline(always)]
unsafe fn dma_record_output_prepare(timestamp: i64, target: i32, words: usize) { unsafe fn dma_record_output_prepare(timestamp: i64, target: i32,
words: usize) {
// See gateware/rtio/dma.py. // See gateware/rtio/dma.py.
const HEADER_LENGTH: usize = /*length*/1 + /*channel*/3 + /*timestamp*/8 + /*address*/1; const HEADER_LENGTH: usize = /*length*/1 + /*channel*/3 + /*timestamp*/8 + /*address*/1;
let length = HEADER_LENGTH + /*data*/words * 4; let length = HEADER_LENGTH + /*data*/words * 4;
@ -87,36 +87,36 @@ unsafe fn dma_record_output_prepare(timestamp: i64, target: i32, words: usize) {
let buffer = &mut RECORDER.as_mut().unwrap().buffer; let buffer = &mut RECORDER.as_mut().unwrap().buffer;
buffer.reserve(length); buffer.reserve(length);
buffer.extend_from_slice(&[ buffer.extend_from_slice(&[
(length >> 0) as u8, (length >> 0) as u8,
(target >> 8) as u8, (target >> 8) as u8,
(target >> 16) as u8, (target >> 16) as u8,
(target >> 24) as u8, (target >> 24) as u8,
(timestamp >> 0) as u8, (timestamp >> 0) as u8,
(timestamp >> 8) as u8, (timestamp >> 8) as u8,
(timestamp >> 16) as u8, (timestamp >> 16) as u8,
(timestamp >> 24) as u8, (timestamp >> 24) as u8,
(timestamp >> 32) as u8, (timestamp >> 32) as u8,
(timestamp >> 40) as u8, (timestamp >> 40) as u8,
(timestamp >> 48) as u8, (timestamp >> 48) as u8,
(timestamp >> 56) as u8, (timestamp >> 56) as u8,
(target >> 0) as u8, (target >> 0) as u8,
]); ]);
} }
pub extern "C" fn dma_record_output(target: i32, word: i32) { pub extern fn dma_record_output(target: i32, word: i32) {
unsafe { unsafe {
let timestamp = rtio::now_mu(); let timestamp = rtio::now_mu();
dma_record_output_prepare(timestamp, target, 1); dma_record_output_prepare(timestamp, target, 1);
RECORDER.as_mut().unwrap().buffer.extend_from_slice(&[ RECORDER.as_mut().unwrap().buffer.extend_from_slice(&[
(word >> 0) as u8, (word >> 0) as u8,
(word >> 8) as u8, (word >> 8) as u8,
(word >> 16) as u8, (word >> 16) as u8,
(word >> 24) as u8, (word >> 24) as u8,
]); ]);
} }
} }
pub extern "C" fn dma_record_output_wide(target: i32, words: CSlice<i32>) { pub extern fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
assert!(words.len() <= 16); // enforce the hardware limit assert!(words.len() <= 16); // enforce the hardware limit
unsafe { unsafe {
@ -125,8 +125,8 @@ pub extern "C" fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
let buffer = &mut RECORDER.as_mut().unwrap().buffer; let buffer = &mut RECORDER.as_mut().unwrap().buffer;
for word in words.as_ref().iter() { for word in words.as_ref().iter() {
buffer.extend_from_slice(&[ buffer.extend_from_slice(&[
(word >> 0) as u8, (word >> 0) as u8,
(word >> 8) as u8, (word >> 8) as u8,
(word >> 16) as u8, (word >> 16) as u8,
(word >> 24) as u8, (word >> 24) as u8,
]); ]);
@ -134,114 +134,79 @@ pub extern "C" fn dma_record_output_wide(target: i32, words: CSlice<i32>) {
} }
} }
pub extern "C" fn dma_erase(name: CSlice<u8>) { pub extern fn dma_erase(name: CSlice<u8>) {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap(); let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0 KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaEraseRequest(name));
.as_mut()
.unwrap()
.send(Message::DmaEraseRequest(name));
} }
} }
pub extern "C" fn dma_retrieve(name: CSlice<u8>) -> DmaTrace { pub extern fn dma_retrieve(name: CSlice<u8>) -> DmaTrace {
let name = String::from_utf8(name.as_ref().to_vec()).unwrap(); let name = String::from_utf8(name.as_ref().to_vec()).unwrap();
unsafe { unsafe {
KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaGetRequest(name)); KERNEL_CHANNEL_1TO0.as_mut().unwrap().send(Message::DmaGetRequest(name));
} }
match unsafe { KERNEL_CHANNEL_0TO1.as_mut().unwrap() }.recv() { match unsafe {KERNEL_CHANNEL_0TO1.as_mut().unwrap()}.recv() {
Message::DmaGetReply(None) => (), Message::DmaGetReply(None) => (),
Message::DmaGetReply(Some((address, duration))) => { Message::DmaGetReply(Some((mut v, duration))) => {
return DmaTrace { address, duration }; v.reserve(ALIGNMENT - 1);
} let original_length = v.len();
let padding = ALIGNMENT - v.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
v.push(0);
}
// trailing zero to indicate end of buffer
v.push(0);
v.copy_within(0..original_length, padding);
dcci_slice(&v);
let v = Box::new(v);
let address = Box::into_raw(v) as *mut Vec<u8> as i32;
return DmaTrace {
address,
duration,
};
},
_ => panic!("Expected DmaGetReply after DmaGetRequest!"), _ => panic!("Expected DmaGetReply after DmaGetRequest!"),
} }
// we have to defer raising error as we have to drop the message first... // we have to defer raising error as we have to drop the message first...
artiq_raise!("DMAError", "DMA trace not found"); artiq_raise!("DMAError", "DMA trace not found");
} }
pub extern "C" fn dma_playback(timestamp: i64, ptr: i32) { pub extern fn dma_playback(timestamp: i64, ptr: i32) {
unsafe { unsafe {
let v = Box::from_raw(ptr as *mut Vec<u8>);
let padding = ALIGNMENT - v.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
let ptr = v.as_ptr().add(padding) as i32;
csr::rtio_dma::base_address_write(ptr as u32); csr::rtio_dma::base_address_write(ptr as u32);
csr::rtio_dma::time_offset_write(timestamp as u64); csr::rtio_dma::time_offset_write(timestamp as u64);
csr::cri_con::selected_write(1); csr::cri_con::selected_write(1);
csr::rtio_dma::enable_write(1); csr::rtio_dma::enable_write(1);
#[cfg(has_drtio)]
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaStartRemoteRequest {
id: ptr,
timestamp: timestamp,
});
while csr::rtio_dma::enable_read() != 0 {} while csr::rtio_dma::enable_read() != 0 {}
csr::cri_con::selected_write(0); csr::cri_con::selected_write(0);
// leave the handle as we may try to do playback for another time.
mem::forget(v);
let error = csr::rtio_dma::error_read(); let error = csr::rtio_dma::error_read();
if error != 0 { if error != 0 {
let timestamp = csr::rtio_dma::error_timestamp_read(); let timestamp = csr::rtio_dma::error_timestamp_read();
let channel = csr::rtio_dma::error_channel_read(); let channel = csr::rtio_dma::error_channel_read();
csr::rtio_dma::error_write(1); csr::rtio_dma::error_write(1);
if error & 1 != 0 { if error & 1 != 0 {
artiq_raise!( artiq_raise!("RTIOUnderflow",
"RTIOUnderflow", "RTIO underflow at {0} mu, channel {1}",
"RTIO underflow at {1} mu, channel {rtio_channel_info:0}", timestamp as i64, channel as i64, 0);
channel as i64,
timestamp as i64,
0
);
} }
if error & 2 != 0 { if error & 2 != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, output, at {0} mu, channel {1}",
"RTIO destination unreachable, output, at {1} mu, channel {rtio_channel_info:0}", timestamp as i64, channel as i64, 0);
channel as i64,
timestamp as i64,
0
);
}
}
#[cfg(has_drtio)]
{
KERNEL_CHANNEL_1TO0
.as_mut()
.unwrap()
.send(Message::DmaAwaitRemoteRequest(ptr));
match KERNEL_CHANNEL_0TO1.as_mut().unwrap().recv() {
Message::DmaAwaitRemoteReply {
timeout,
error,
channel,
timestamp,
} => {
if timeout {
artiq_raise!(
"DMAError",
"Error running DMA on satellite device, timed out waiting for results"
);
}
if error & 1 != 0 {
artiq_raise!(
"RTIOUnderflow",
"RTIO underflow at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
if error & 2 != 0 {
artiq_raise!(
"RTIODestinationUnreachable",
"RTIO destination unreachable, output, at {1} mu, channel {rtio_channel_info:0}",
channel as i64,
timestamp as i64,
0
);
}
}
_ => panic!("Expected DmaAwaitRemoteReply after DmaAwaitRemoteRequest!"),
} }
} }
} }
} }

55
src/runtime/src/kernel/mod.rs
View File

@ -1,28 +1,27 @@
use alloc::{string::String, vec::Vec};
use core::ptr; use core::ptr;
use alloc::{vec::Vec, string::String};
use libcortex_a9::{mutex::Mutex, semaphore::Semaphore, sync_channel}; use libcortex_a9::{mutex::Mutex, sync_channel, semaphore::Semaphore};
use crate::eh_artiq; use crate::eh_artiq;
mod control; mod control;
pub use control::Control; pub use control::Control;
mod api;
pub mod core1; pub mod core1;
mod dma; mod api;
mod rpc; mod rpc;
mod dma;
pub use dma::DmaRecorder; pub use dma::DmaRecorder;
mod cache; mod cache;
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct RPCException { pub struct RPCException {
pub id: u32, pub name: String,
pub message: u32, pub message: String,
pub param: [i64; 3], pub param: [i64; 3],
pub file: u32, pub file: String,
pub line: i32, pub line: i32,
pub column: i32, pub column: i32,
pub function: u32, pub function: String
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@ -31,17 +30,9 @@ pub enum Message {
LoadCompleted, LoadCompleted,
LoadFailed, LoadFailed,
StartRequest, StartRequest,
KernelFinished(u8), KernelFinished,
KernelException( KernelException(&'static eh_artiq::Exception<'static>, &'static [usize]),
&'static [Option<eh_artiq::Exception<'static>>], RpcSend { is_async: bool, data: Vec<u8> },
&'static [eh_artiq::StackPointerBacktrace],
&'static [(usize, usize)],
u8,
),
RpcSend {
is_async: bool,
data: Vec<u8>,
},
RpcRecvRequest(*mut ()), RpcRecvRequest(*mut ()),
RpcRecvReply(Result<usize, RPCException>), RpcRecvReply(Result<usize, RPCException>),
@ -52,26 +43,7 @@ pub enum Message {
DmaPutRequest(DmaRecorder), DmaPutRequest(DmaRecorder),
DmaEraseRequest(String), DmaEraseRequest(String),
DmaGetRequest(String), DmaGetRequest(String),
DmaGetReply(Option<(i32, i64)>), DmaGetReply(Option<(Vec<u8>, i64)>),
#[cfg(has_drtio)]
DmaStartRemoteRequest {
id: i32,
timestamp: i64,
},
#[cfg(has_drtio)]
DmaAwaitRemoteRequest(i32),
#[cfg(has_drtio)]
DmaAwaitRemoteReply {
timeout: bool,
error: u8,
channel: u32,
timestamp: u64,
},
#[cfg(has_drtio)]
UpDestinationsRequest(i32),
#[cfg(has_drtio)]
UpDestinationsReply(bool),
} }
static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None); static CHANNEL_0TO1: Mutex<Option<sync_channel::Sender<'static, Message>>> = Mutex::new(None);
@ -81,6 +53,7 @@ static CHANNEL_SEM: Semaphore = Semaphore::new(0, 1);
static mut KERNEL_CHANNEL_0TO1: Option<sync_channel::Receiver<'static, Message>> = None; static mut KERNEL_CHANNEL_0TO1: Option<sync_channel::Receiver<'static, Message>> = None;
static mut KERNEL_CHANNEL_1TO0: Option<sync_channel::Sender<'static, Message>> = None; static mut KERNEL_CHANNEL_1TO0: Option<sync_channel::Sender<'static, Message>> = None;
pub static mut KERNEL_IMAGE: *const core1::KernelImage = ptr::null(); static mut KERNEL_IMAGE: *const core1::KernelImage = ptr::null();
static INIT_LOCK: Mutex<()> = Mutex::new(()); static INIT_LOCK: Mutex<()> = Mutex::new(());

33
src/runtime/src/kernel/rpc.rs
View File

@ -1,11 +1,14 @@
//! Kernel-side RPC API //! Kernel-side RPC API
use alloc::vec::Vec; use alloc::vec::Vec;
use cslice::{CSlice, AsCSlice};
use cslice::CSlice; use crate::eh_artiq;
use crate::rpc::send_args;
use super::{Message, KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0}; use super::{
use crate::{eh_artiq, rpc::send_args}; KERNEL_CHANNEL_0TO1, KERNEL_CHANNEL_1TO0,
Message,
};
fn rpc_send_common(is_async: bool, service: u32, tag: &CSlice<u8>, data: *const *const ()) { fn rpc_send_common(is_async: bool, service: u32, tag: &CSlice<u8>, data: *const *const ()) {
let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() }; let core1_tx = unsafe { KERNEL_CHANNEL_1TO0.as_mut().unwrap() };
@ -14,15 +17,15 @@ fn rpc_send_common(is_async: bool, service: u32, tag: &CSlice<u8>, data: *const
core1_tx.send(Message::RpcSend { is_async, data: buffer }); core1_tx.send(Message::RpcSend { is_async, data: buffer });
} }
pub extern "C" fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) { pub extern fn rpc_send(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
rpc_send_common(false, service, tag, data); rpc_send_common(false, service, tag, data);
} }
pub extern "C" fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) { pub extern fn rpc_send_async(service: u32, tag: &CSlice<u8>, data: *const *const ()) {
rpc_send_common(true, service, tag, data); rpc_send_common(true, service, tag, data);
} }
pub extern "C" fn rpc_recv(slot: *mut ()) -> usize { pub extern fn rpc_recv(slot: *mut ()) -> usize {
let reply = unsafe { let reply = unsafe {
let core1_rx = KERNEL_CHANNEL_0TO1.as_mut().unwrap(); let core1_rx = KERNEL_CHANNEL_0TO1.as_mut().unwrap();
let core1_tx = KERNEL_CHANNEL_1TO0.as_mut().unwrap(); let core1_tx = KERNEL_CHANNEL_1TO0.as_mut().unwrap();
@ -33,15 +36,15 @@ pub extern "C" fn rpc_recv(slot: *mut ()) -> usize {
Message::RpcRecvReply(Ok(alloc_size)) => alloc_size, Message::RpcRecvReply(Ok(alloc_size)) => alloc_size,
Message::RpcRecvReply(Err(exception)) => unsafe { Message::RpcRecvReply(Err(exception)) => unsafe {
eh_artiq::raise(&eh_artiq::Exception { eh_artiq::raise(&eh_artiq::Exception {
id: exception.id, name: exception.name.as_bytes().as_c_slice(),
file: CSlice::new(exception.file as *const u8, usize::MAX), file: exception.file.as_bytes().as_c_slice(),
line: exception.line as u32, line: exception.line as u32,
column: exception.column as u32, column: exception.column as u32,
function: CSlice::new(exception.function as *const u8, usize::MAX), function: exception.function.as_bytes().as_c_slice(),
message: CSlice::new(exception.message as *const u8, usize::MAX), message: exception.message.as_bytes().as_c_slice(),
param: exception.param, param: exception.param
}) })
}, },
_ => panic!("received unexpected reply to RpcRecvRequest: {:?}", reply), _ => panic!("received unexpected reply to RpcRecvRequest: {:?}", reply)
} }
} }

56
src/libboard_artiq/src/logger.rs → src/runtime/src/logger.rs
View File

@ -1,13 +1,13 @@
use core::{cell::Cell, fmt::Write}; use core::cell::Cell;
use core::fmt::Write;
use libboard_zynq::{println, timer::GlobalTimer}; use log::{Log, LevelFilter};
use libcortex_a9::mutex::{Mutex, MutexGuard};
use log::{LevelFilter, Log};
use log_buffer::LogBuffer; use log_buffer::LogBuffer;
use libcortex_a9::mutex::{Mutex, MutexGuard};
use libboard_zynq::{println, timer::GlobalTimer};
pub struct LogBufferRef<'a> { pub struct LogBufferRef<'a> {
buffer: MutexGuard<'a, LogBuffer<&'static mut [u8]>>, buffer: MutexGuard<'a, LogBuffer<&'static mut [u8]>>,
old_log_level: LevelFilter, old_log_level: LevelFilter
} }
impl<'a> LogBufferRef<'a> { impl<'a> LogBufferRef<'a> {
@ -37,7 +37,7 @@ impl<'a> Drop for LogBufferRef<'a> {
} }
pub struct BufferLogger { pub struct BufferLogger {
buffer: Mutex<LogBuffer<&'static mut [u8]>>, buffer: Mutex<LogBuffer<&'static mut [u8]>>,
uart_filter: Cell<LevelFilter>, uart_filter: Cell<LevelFilter>,
buffer_filter: Cell<LevelFilter>, buffer_filter: Cell<LevelFilter>,
} }
@ -56,7 +56,8 @@ impl BufferLogger {
pub fn register(self) { pub fn register(self) {
unsafe { unsafe {
LOGGER = Some(self); LOGGER = Some(self);
log::set_logger(LOGGER.as_ref().unwrap()).expect("global logger can only be initialized once"); log::set_logger(LOGGER.as_ref().unwrap())
.expect("global logger can only be initialized once");
} }
} }
@ -65,7 +66,9 @@ impl BufferLogger {
} }
pub fn buffer<'a>(&'a self) -> Option<LogBufferRef<'a>> { pub fn buffer<'a>(&'a self) -> Option<LogBufferRef<'a>> {
self.buffer.try_lock().map(LogBufferRef::new) self.buffer
.try_lock()
.map(LogBufferRef::new)
} }
pub fn uart_log_level(&self) -> LevelFilter { pub fn uart_log_level(&self) -> LevelFilter {
@ -96,36 +99,25 @@ impl Log for BufferLogger {
fn log(&self, record: &log::Record) { fn log(&self, record: &log::Record) {
if self.enabled(record.metadata()) { if self.enabled(record.metadata()) {
let timestamp = unsafe { GlobalTimer::get() }.get_us().0; let timestamp = unsafe {
let seconds = timestamp / 1_000_000; GlobalTimer::get()
let micros = timestamp % 1_000_000; }.get_us().0;
let seconds = timestamp / 1_000_000;
let micros = timestamp % 1_000_000;
if record.level() <= self.buffer_log_level() { if record.level() <= self.buffer_log_level() {
let mut buffer = self.buffer.lock(); let mut buffer = self.buffer.lock();
writeln!( writeln!(buffer, "[{:6}.{:06}s] {:>5}({}): {}", seconds, micros,
buffer, record.level(), record.target(), record.args()).unwrap();
"[{:6}.{:06}s] {:>5}({}): {}",
seconds,
micros,
record.level(),
record.target(),
record.args()
)
.unwrap();
} }
if record.level() <= self.uart_log_level() { if record.level() <= self.uart_log_level() {
println!( println!("[{:6}.{:06}s] {:>5}({}): {}", seconds, micros,
"[{:6}.{:06}s] {:>5}({}): {}", record.level(), record.target(), record.args());
seconds,
micros,
record.level(),
record.target(),
record.args()
);
} }
} }
} }
fn flush(&self) {} fn flush(&self) {
}
} }

190
src/runtime/src/main.rs
View File

@ -1,6 +1,6 @@
#![no_std] #![no_std]
#![no_main] #![no_main]
#![recursion_limit = "1024"] // for futures_util::select! #![recursion_limit="1024"] // for futures_util::select!
#![feature(alloc_error_handler)] #![feature(alloc_error_handler)]
#![feature(panic_info_message)] #![feature(panic_info_message)]
#![feature(c_variadic)] #![feature(c_variadic)]
@ -9,52 +9,122 @@
#![feature(naked_functions)] #![feature(naked_functions)]
#![feature(asm)] #![feature(asm)]
#[macro_use]
extern crate alloc; extern crate alloc;
use libasync::{block_async, task}; use core::{cmp, str};
#[cfg(feature = "target_kasli_soc")] use log::{info, warn, error};
use libboard_artiq::io_expander;
use libboard_artiq::{identifier_read, logger, pl}; use libboard_zynq::{timer::GlobalTimer, mpcore, gic, slcr};
use libboard_zynq::{gic, mpcore, timer::GlobalTimer}; use libasync::{task, block_async};
use libconfig::Config;
use libcortex_a9::l2c::enable_l2_cache;
use libsupport_zynq::ram; use libsupport_zynq::ram;
use log::{error, info, warn};
use nb; use nb;
use void::Void; use void::Void;
use embedded_hal::blocking::delay::DelayMs;
use libconfig::Config;
use libregister::RegisterW;
use libcortex_a9::l2c::enable_l2_cache;
const ASYNC_ERROR_COLLISION: u8 = 1 << 0; mod proto_core_io;
const ASYNC_ERROR_BUSY: u8 = 1 << 1;
const ASYNC_ERROR_SEQUENCE_ERROR: u8 = 1 << 2;
mod analyzer;
mod comms;
mod eh_artiq;
mod i2c;
mod irq;
mod kernel;
mod mgmt;
mod moninj;
mod panic;
mod proto_async; mod proto_async;
mod comms;
mod rpc; mod rpc;
#[path = "../../../build/pl.rs"]
mod pl;
#[cfg(ki_impl = "csr")] #[cfg(ki_impl = "csr")]
#[path = "rtio_csr.rs"] #[path = "rtio_csr.rs"]
mod rtio; mod rtio;
#[cfg(ki_impl = "acp")] #[cfg(ki_impl = "acp")]
#[path = "rtio_acp.rs"] #[path = "rtio_acp.rs"]
mod rtio; mod rtio;
mod rtio_clocking; mod kernel;
mod rtio_dma; mod moninj;
mod rtio_mgt; mod eh_artiq;
mod panic;
mod logger;
mod mgmt;
mod analyzer;
mod irq;
mod i2c;
static mut SEEN_ASYNC_ERRORS: u8 = 0; fn init_gateware() {
// Set up PS->PL clocks
slcr::RegisterBlock::unlocked(|slcr| {
// As we are touching the mux, the clock may glitch, so reset the PL.
slcr.fpga_rst_ctrl.write(
slcr::FpgaRstCtrl::zeroed()
.fpga0_out_rst(true)
.fpga1_out_rst(true)
.fpga2_out_rst(true)
.fpga3_out_rst(true)
);
slcr.fpga0_clk_ctrl.write(
slcr::Fpga0ClkCtrl::zeroed()
.src_sel(slcr::PllSource::IoPll)
.divisor0(8)
.divisor1(1)
);
slcr.fpga_rst_ctrl.write(
slcr::FpgaRstCtrl::zeroed()
);
});
}
pub unsafe fn get_async_errors() -> u8 { fn identifier_read(buf: &mut [u8]) -> &str {
let errors = SEEN_ASYNC_ERRORS; unsafe {
SEEN_ASYNC_ERRORS = 0; pl::csr::identifier::address_write(0);
errors let len = pl::csr::identifier::data_read();
let len = cmp::min(len, buf.len() as u8);
for i in 0..len {
pl::csr::identifier::address_write(1 + i);
buf[i as usize] = pl::csr::identifier::data_read();
}
str::from_utf8_unchecked(&buf[..len as usize])
}
}
fn init_rtio(timer: &mut GlobalTimer, cfg: &Config) {
let clock_sel =
if let Ok(rtioclk) = cfg.read_str("rtioclk") {
match rtioclk.as_ref() {
"internal" => {
info!("using internal RTIO clock");
0
},
"external" => {
info!("using external RTIO clock");
1
},
other => {
warn!("RTIO clock specification '{}' not recognized", other);
info!("using internal RTIO clock");
0
},
}
} else {
info!("using internal RTIO clock (default)");
0
};
loop {
unsafe {
pl::csr::rtio_crg::pll_reset_write(1);
pl::csr::rtio_crg::clock_sel_write(clock_sel);
pl::csr::rtio_crg::pll_reset_write(0);
}
timer.delay_ms(1);
let locked = unsafe { pl::csr::rtio_crg::pll_locked_read() != 0 };
if locked {
info!("RTIO PLL locked");
break;
} else {
warn!("RTIO PLL failed to lock, retrying...");
timer.delay_ms(500);
}
}
unsafe {
pl::csr::rtio_core::reset_phy_write(1);
}
} }
fn wait_for_async_rtio_error() -> nb::Result<(), Void> { fn wait_for_async_rtio_error() -> nb::Result<(), Void> {
@ -72,44 +142,33 @@ async fn report_async_rtio_errors() {
let _ = block_async!(wait_for_async_rtio_error()).await; let _ = block_async!(wait_for_async_rtio_error()).await;
unsafe { unsafe {
let errors = pl::csr::rtio_core::async_error_read(); let errors = pl::csr::rtio_core::async_error_read();
if errors & ASYNC_ERROR_COLLISION != 0 { if errors & 1 != 0 {
let channel = pl::csr::rtio_core::collision_channel_read(); error!("RTIO collision involving channel {}",
error!( pl::csr::rtio_core::collision_channel_read());
"RTIO collision involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
} }
if errors & ASYNC_ERROR_BUSY != 0 { if errors & 2 != 0 {
let channel = pl::csr::rtio_core::busy_channel_read(); error!("RTIO busy error involving channel {}",
error!( pl::csr::rtio_core::busy_channel_read());
"RTIO busy error involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
} }
if errors & ASYNC_ERROR_SEQUENCE_ERROR != 0 { if errors & 4 != 0 {
let channel = pl::csr::rtio_core::sequence_error_channel_read(); error!("RTIO sequence error involving channel {}",
error!( pl::csr::rtio_core::sequence_error_channel_read());
"RTIO sequence error involving channel 0x{:04x}:{}",
channel,
rtio_mgt::resolve_channel_name(channel as u32)
);
} }
SEEN_ASYNC_ERRORS = errors;
pl::csr::rtio_core::async_error_write(errors); pl::csr::rtio_core::async_error_write(errors);
} }
} }
} }
static mut LOG_BUFFER: [u8; 1 << 17] = [0; 1 << 17]; static mut LOG_BUFFER: [u8; 1<<17] = [0; 1<<17];
#[no_mangle] #[no_mangle]
pub fn main_core0() { pub fn main_core0() {
enable_l2_cache(0x8); enable_l2_cache(0x8);
let mut timer = GlobalTimer::start(); let mut timer = GlobalTimer::start();
let buffer_logger = unsafe { logger::BufferLogger::new(&mut LOG_BUFFER[..]) }; let buffer_logger = unsafe {
logger::BufferLogger::new(&mut LOG_BUFFER[..])
};
buffer_logger.set_uart_log_level(log::LevelFilter::Info); buffer_logger.set_uart_log_level(log::LevelFilter::Info);
buffer_logger.register(); buffer_logger.register();
log::set_max_level(log::LevelFilter::Info); log::set_max_level(log::LevelFilter::Info);
@ -119,25 +178,11 @@ pub fn main_core0() {
ram::init_alloc_core0(); ram::init_alloc_core0();
gic::InterruptController::gic(mpcore::RegisterBlock::mpcore()).enable_interrupts(); gic::InterruptController::gic(mpcore::RegisterBlock::mpcore()).enable_interrupts();
info!("gateware ident: {}", identifier_read(&mut [0; 64])); init_gateware();
info!("detected gateware: {}", identifier_read(&mut [0; 64]));
i2c::init(); i2c::init();
#[cfg(feature = "target_kasli_soc")]
{
let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
for expander_i in 0..=1 {
let mut io_expander = io_expander::IoExpander::new(i2c, expander_i).unwrap();
io_expander.init().expect("I2C I/O expander #0 initialization failed");
// Actively drive TX_DISABLE to false on SFP0..3
io_expander.set_oe(0, 1 << 1).unwrap();
io_expander.set_oe(1, 1 << 1).unwrap();
io_expander.set(0, 1, false);
io_expander.set(1, 1, false);
io_expander.service().unwrap();
}
}
let cfg = match Config::new() { let cfg = match Config::new() {
Ok(cfg) => cfg, Ok(cfg) => cfg,
Err(err) => { Err(err) => {
@ -146,8 +191,7 @@ pub fn main_core0() {
} }
}; };
rtio_clocking::init(&mut timer, &cfg); init_rtio(&mut timer, &cfg);
task::spawn(report_async_rtio_errors()); task::spawn(report_async_rtio_errors());
comms::main(timer, cfg); comms::main(timer, cfg);

57
src/runtime/src/mgmt.rs
View File

@ -1,17 +1,16 @@
use alloc::{rc::Rc, string::String, vec::Vec};
use core::cell::RefCell;
use futures::{future::poll_fn, task::Poll}; use futures::{future::poll_fn, task::Poll};
use libasync::{smoltcp::TcpStream, task}; use libasync::{smoltcp::TcpStream, task};
use libboard_artiq::logger::{BufferLogger, LogBufferRef}; use libboard_zynq::smoltcp;
use libboard_zynq::{slcr, smoltcp};
use libconfig::Config; use libconfig::Config;
use log::{self, debug, error, info, warn, LevelFilter}; use core::cell::RefCell;
use alloc::{rc::Rc, vec::Vec, string::String};
use log::{self, info, debug, warn, error, LevelFilter};
use crate::logger::{BufferLogger, LogBufferRef};
use crate::proto_async::*;
use num_derive::FromPrimitive; use num_derive::FromPrimitive;
use num_traits::FromPrimitive; use num_traits::FromPrimitive;
use crate::proto_async::*;
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error { pub enum Error {
NetworkError(smoltcp::Error), NetworkError(smoltcp::Error),
@ -25,10 +24,10 @@ type Result<T> = core::result::Result<T, Error>;
impl core::fmt::Display for Error { impl core::fmt::Display for Error {
fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result { fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
match self { match self {
&Error::NetworkError(error) => write!(f, "network error: {}", error), &Error::NetworkError(error) => write!(f, "network error: {}", error),
&Error::UnknownLogLevel(lvl) => write!(f, "unknown log level {}", lvl), &Error::UnknownLogLevel(lvl) => write!(f, "unknown log level {}", lvl),
&Error::UnexpectedPattern => write!(f, "unexpected pattern"), &Error::UnexpectedPattern => write!(f, "unexpected pattern"),
&Error::UnrecognizedPacket => write!(f, "unrecognized packet"), &Error::UnrecognizedPacket => write!(f, "unrecognized packet"),
} }
} }
} }
@ -45,7 +44,6 @@ pub enum Request {
ClearLog = 2, ClearLog = 2,
PullLog = 7, PullLog = 7,
SetLogFilter = 3, SetLogFilter = 3,
Reboot = 5,
SetUartLogFilter = 6, SetUartLogFilter = 6,
ConfigRead = 12, ConfigRead = 12,
@ -57,7 +55,6 @@ pub enum Request {
pub enum Reply { pub enum Reply {
Success = 1, Success = 1,
LogContent = 2, LogContent = 2,
RebootImminent = 3,
Error = 6, Error = 6,
ConfigData = 7, ConfigData = 7,
} }
@ -75,7 +72,9 @@ async fn read_log_level_filter(stream: &mut TcpStream) -> Result<log::LevelFilte
} }
async fn get_logger_buffer_pred<F>(f: F) -> LogBufferRef<'static> async fn get_logger_buffer_pred<F>(f: F) -> LogBufferRef<'static>
where F: Fn(&LogBufferRef) -> bool { where
F: Fn(&LogBufferRef) -> bool,
{
poll_fn(|ctx| { poll_fn(|ctx| {
let logger = unsafe { BufferLogger::get_logger().as_mut().unwrap() }; let logger = unsafe { BufferLogger::get_logger().as_mut().unwrap() };
match logger.buffer() { match logger.buffer() {
@ -111,7 +110,10 @@ async fn read_key(stream: &mut TcpStream) -> Result<String> {
Ok(String::from_utf8(buffer).unwrap()) Ok(String::from_utf8(buffer).unwrap())
} }
async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cfg: Rc<Config>) -> Result<()> { async fn handle_connection(
stream: &mut TcpStream,
pull_id: Rc<RefCell<u32>>,
cfg: Rc<Config>) -> Result<()> {
if !expect(&stream, b"ARTIQ management\n").await? { if !expect(&stream, b"ARTIQ management\n").await? {
return Err(Error::UnexpectedPattern); return Err(Error::UnexpectedPattern);
} }
@ -119,7 +121,7 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
loop { loop {
let msg = read_i8(stream).await; let msg = read_i8(stream).await;
if let Err(smoltcp::Error::Finished) = msg { if let Err(smoltcp::Error::Illegal) = msg {
return Ok(()); return Ok(());
} }
let msg: Request = FromPrimitive::from_i8(msg?).ok_or(Error::UnrecognizedPacket)?; let msg: Request = FromPrimitive::from_i8(msg?).ok_or(Error::UnrecognizedPacket)?;
@ -159,7 +161,7 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
logger.set_buffer_log_level(LevelFilter::Trace); logger.set_buffer_log_level(LevelFilter::Trace);
} }
} }
} },
Request::SetLogFilter => { Request::SetLogFilter => {
let lvl = read_log_level_filter(stream).await?; let lvl = read_log_level_filter(stream).await?;
info!("Changing log level to {}", lvl); info!("Changing log level to {}", lvl);
@ -170,7 +172,10 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
let lvl = read_log_level_filter(stream).await?; let lvl = read_log_level_filter(stream).await?;
info!("Changing UART log level to {}", lvl); info!("Changing UART log level to {}", lvl);
unsafe { unsafe {
BufferLogger::get_logger().as_ref().unwrap().set_uart_log_level(lvl); BufferLogger::get_logger()
.as_ref()
.unwrap()
.set_uart_log_level(lvl);
} }
write_i8(stream, Reply::Success as i8).await?; write_i8(stream, Reply::Success as i8).await?;
} }
@ -186,12 +191,16 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
warn!("read error: no such key"); warn!("read error: no such key");
write_i8(stream, Reply::Error as i8).await?; write_i8(stream, Reply::Error as i8).await?;
} }
} },
Request::ConfigWrite => { Request::ConfigWrite => {
let key = read_key(stream).await?; let key = read_key(stream).await?;
debug!("write key: {}", key); debug!("write key: {}", key);
let len = read_i32(stream).await?; let len = read_i32(stream).await?;
let len = if len <= 0 { 0 } else { len as usize }; let len = if len <= 0 {
0
} else {
len as usize
};
let mut buffer = Vec::with_capacity(len); let mut buffer = Vec::with_capacity(len);
unsafe { unsafe {
buffer.set_len(len); buffer.set_len(len);
@ -206,7 +215,7 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
error!("failed to write: {:?}", value); error!("failed to write: {:?}", value);
write_i8(stream, Reply::Error as i8).await?; write_i8(stream, Reply::Error as i8).await?;
} }
} },
Request::ConfigRemove => { Request::ConfigRemove => {
let key = read_key(stream).await?; let key = read_key(stream).await?;
debug!("erase key: {}", key); debug!("erase key: {}", key);
@ -219,12 +228,6 @@ async fn handle_connection(stream: &mut TcpStream, pull_id: Rc<RefCell<u32>>, cf
write_i8(stream, Reply::Error as i8).await?; write_i8(stream, Reply::Error as i8).await?;
} }
} }
Request::Reboot => {
info!("rebooting");
write_i8(stream, Reply::RebootImminent as i8).await?;
stream.flush().await?;
slcr::reboot();
}
} }
} }
} }

239
src/runtime/src/moninj.rs
View File

@ -1,23 +1,25 @@
use alloc::{collections::BTreeMap, rc::Rc}; use core::fmt;
use core::{cell::RefCell, fmt}; use alloc::collections::BTreeMap;
use futures::{pin_mut, select_biased, FutureExt};
use libasync::{block_async, nb, smoltcp::TcpStream, task};
use libboard_artiq::drtio_routing;
use libboard_zynq::{smoltcp, time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::mutex::Mutex;
use log::{debug, info, warn}; use log::{debug, info, warn};
use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive};
use void::Void; use void::Void;
use libboard_zynq::{smoltcp, timer::GlobalTimer, time::Milliseconds};
use libasync::{task, smoltcp::TcpStream, block_async, nb};
use num_derive::{FromPrimitive, ToPrimitive};
use num_traits::{FromPrimitive, ToPrimitive};
use futures::{pin_mut, select_biased, FutureExt};
use crate::proto_async::*; use crate::proto_async::*;
use crate::pl::csr;
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Error { pub enum Error {
NetworkError(smoltcp::Error), NetworkError(smoltcp::Error),
UnexpectedPattern, UnexpectedPattern,
UnrecognizedPacket, UnrecognizedPacket,
} }
pub type Result<T> = core::result::Result<T, Error>; pub type Result<T> = core::result::Result<T, Error>;
@ -26,8 +28,8 @@ impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {
&Error::NetworkError(error) => write!(f, "network error: {}", error), &Error::NetworkError(error) => write!(f, "network error: {}", error),
&Error::UnexpectedPattern => write!(f, "unexpected pattern"), &Error::UnexpectedPattern => write!(f, "unexpected pattern"),
&Error::UnrecognizedPacket => write!(f, "unrecognized packet"), &Error::UnrecognizedPacket => write!(f, "unrecognized packet"),
} }
} }
} }
@ -43,186 +45,62 @@ enum HostMessage {
MonitorProbe = 0, MonitorProbe = 0,
MonitorInjection = 3, MonitorInjection = 3,
Inject = 1, Inject = 1,
GetInjectionStatus = 2, GetInjectionStatus = 2
} }
#[derive(Debug, FromPrimitive, ToPrimitive)] #[derive(Debug, FromPrimitive, ToPrimitive)]
enum DeviceMessage { enum DeviceMessage {
MonitorStatus = 0, MonitorStatus = 0,
InjectionStatus = 1, InjectionStatus = 1
} }
#[cfg(has_drtio)] fn read_probe(channel: i32, probe: i8) -> i32 {
mod remote_moninj { unsafe {
use libboard_artiq::drtioaux_async; csr::rtio_moninj::mon_chan_sel_write(channel as _);
use log::error; csr::rtio_moninj::mon_probe_sel_write(probe as _);
csr::rtio_moninj::mon_value_update_write(1);
use super::*; csr::rtio_moninj::mon_value_read() as i32
use crate::rtio_mgt::drtio;
pub async fn read_probe(
aux_mutex: &Rc<Mutex<bool>>,
timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
probe: i8,
) -> i64 {
let reply = drtio::aux_transact(
aux_mutex,
linkno,
&drtioaux_async::Packet::MonitorRequest {
destination: destination,
channel: channel as _,
probe: probe as _,
},
timer,
)
.await;
match reply {
Ok(drtioaux_async::Packet::MonitorReply { value }) => return value as i64,
Ok(packet) => error!("received unexpected aux packet: {:?}", packet),
Err("link went down") => {
debug!("link is down");
}
Err(e) => error!("aux packet error ({})", e),
}
0
}
pub async fn inject(
aux_mutex: &Rc<Mutex<bool>>,
_timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
overrd: i8,
value: i8,
) {
let _lock = aux_mutex.lock();
drtioaux_async::send(
linkno,
&drtioaux_async::Packet::InjectionRequest {
destination: destination,
channel: channel as _,
overrd: overrd as _,
value: value as _,
},
)
.await
.unwrap();
}
pub async fn read_injection_status(
aux_mutex: &Rc<Mutex<bool>>,
timer: GlobalTimer,
linkno: u8,
destination: u8,
channel: i32,
overrd: i8,
) -> i8 {
let reply = drtio::aux_transact(
aux_mutex,
linkno,
&drtioaux_async::Packet::InjectionStatusRequest {
destination: destination,
channel: channel as _,
overrd: overrd as _,
},
timer,
)
.await;
match reply {
Ok(drtioaux_async::Packet::InjectionStatusReply { value }) => return value as i8,
Ok(packet) => error!("received unexpected aux packet: {:?}", packet),
Err("link went down") => {
debug!("link is down");
}
Err(e) => error!("aux packet error ({})", e),
}
0
} }
} }
mod local_moninj { fn inject(channel: i32, overrd: i8, value: i8) {
use libboard_artiq::pl::csr; unsafe {
csr::rtio_moninj::inj_chan_sel_write(channel as _);
pub fn read_probe(channel: i32, probe: i8) -> i64 { csr::rtio_moninj::inj_override_sel_write(overrd as _);
unsafe { csr::rtio_moninj::inj_value_write(value as _);
csr::rtio_moninj::mon_chan_sel_write(channel as _);
csr::rtio_moninj::mon_probe_sel_write(probe as _);
csr::rtio_moninj::mon_value_update_write(1);
csr::rtio_moninj::mon_value_read() as i64
}
}
pub fn inject(channel: i32, overrd: i8, value: i8) {
unsafe {
csr::rtio_moninj::inj_chan_sel_write(channel as _);
csr::rtio_moninj::inj_override_sel_write(overrd as _);
csr::rtio_moninj::inj_value_write(value as _);
}
}
pub fn read_injection_status(channel: i32, overrd: i8) -> i8 {
unsafe {
csr::rtio_moninj::inj_chan_sel_write(channel as _);
csr::rtio_moninj::inj_override_sel_write(overrd as _);
csr::rtio_moninj::inj_value_read() as i8
}
} }
} }
#[cfg(has_drtio)] fn read_injection_status(channel: i32, overrd: i8) -> i8 {
macro_rules! dispatch { unsafe {
($timer:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{ csr::rtio_moninj::inj_chan_sel_write(channel as _);
let destination = ($channel >> 16) as u8; csr::rtio_moninj::inj_override_sel_write(overrd as _);
let channel = $channel; csr::rtio_moninj::inj_value_read() as i8
let hop = $routing_table.0[destination as usize][0]; }
if hop == 0 {
local_moninj::$func(channel.into(), $($param, )*)
} else {
let linkno = hop - 1 as u8;
remote_moninj::$func($aux_mutex, $timer, linkno, destination, channel, $($param, )*).await
}
}}
} }
#[cfg(not(has_drtio))] async fn handle_connection(stream: &TcpStream, timer: GlobalTimer) -> Result<()> {
macro_rules! dispatch {
($timer:ident, $aux_mutex:ident, $routing_table:ident, $channel:expr, $func:ident $(, $param:expr)*) => {{
let channel = $channel as u16;
local_moninj::$func(channel.into(), $($param, )*)
}}
}
async fn handle_connection(
stream: &TcpStream,
timer: GlobalTimer,
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &drtio_routing::RoutingTable,
) -> Result<()> {
if !expect(&stream, b"ARTIQ moninj\n").await? { if !expect(&stream, b"ARTIQ moninj\n").await? {
return Err(Error::UnexpectedPattern); return Err(Error::UnexpectedPattern);
} }
stream.send_slice("e".as_bytes()).await?;
let mut probe_watch_list: BTreeMap<(i32, i8), Option<i64>> = BTreeMap::new(); let mut probe_watch_list: BTreeMap<(i32, i8), Option<i32>> = BTreeMap::new();
let mut inject_watch_list: BTreeMap<(i32, i8), Option<i8>> = BTreeMap::new(); let mut inject_watch_list: BTreeMap<(i32, i8), Option<i8>> = BTreeMap::new();
let mut next_check = timer.get_time(); let mut next_check = Milliseconds(0);
let timeout = |next_check: Milliseconds| -> nb::Result<(), Void> {
if timer.get_time() < next_check {
Err(nb::Error::WouldBlock)
} else {
Ok(())
}
};
loop { loop {
// TODO: we don't need fuse() here. // TODO: we don't need fuse() here.
// remove after https://github.com/rust-lang/futures-rs/issues/1989 lands // remove after https://github.com/rust-lang/futures-rs/issues/1989 lands
let read_message_f = read_i8(&stream).fuse(); let read_message_f = read_i8(&stream).fuse();
let next_check_c = next_check.clone(); let next_check_c = next_check.clone();
let timeout = || -> nb::Result<(), Void> {
let timeout_f = block_async!(timeout(next_check_c)).fuse(); if timer.get_time() < next_check_c {
Err(nb::Error::WouldBlock)
} else {
Ok(())
}
};
let timeout_f = block_async!(timeout()).fuse();
pin_mut!(read_message_f, timeout_f); pin_mut!(read_message_f, timeout_f);
select_biased! { select_biased! {
message = read_message_f => { message = read_message_f => {
@ -257,13 +135,13 @@ async fn handle_connection(
let channel = read_i32(&stream).await?; let channel = read_i32(&stream).await?;
let overrd = read_i8(&stream).await?; let overrd = read_i8(&stream).await?;
let value = read_i8(&stream).await?; let value = read_i8(&stream).await?;
dispatch!(timer, _aux_mutex, _routing_table, channel, inject, overrd, value); inject(channel, overrd, value);
debug!("INJECT channel {}, overrd {}, value {}", channel, overrd, value); debug!("INJECT channel {}, overrd {}, value {}", channel, overrd, value);
}, },
HostMessage::GetInjectionStatus => { HostMessage::GetInjectionStatus => {
let channel = read_i32(&stream).await?; let channel = read_i32(&stream).await?;
let overrd = read_i8(&stream).await?; let overrd = read_i8(&stream).await?;
let value = dispatch!(timer, _aux_mutex, _routing_table, channel, read_injection_status, overrd); let value = read_injection_status(channel, overrd);
write_i8(&stream, DeviceMessage::InjectionStatus.to_i8().unwrap()).await?; write_i8(&stream, DeviceMessage::InjectionStatus.to_i8().unwrap()).await?;
write_i32(&stream, channel).await?; write_i32(&stream, channel).await?;
write_i8(&stream, overrd).await?; write_i8(&stream, overrd).await?;
@ -273,17 +151,17 @@ async fn handle_connection(
}, },
_ = timeout_f => { _ = timeout_f => {
for (&(channel, probe), previous) in probe_watch_list.iter_mut() { for (&(channel, probe), previous) in probe_watch_list.iter_mut() {
let current = dispatch!(timer, _aux_mutex, _routing_table, channel, read_probe, probe); let current = read_probe(channel, probe);
if previous.is_none() || previous.unwrap() != current { if previous.is_none() || previous.unwrap() != current {
write_i8(&stream, DeviceMessage::MonitorStatus.to_i8().unwrap()).await?; write_i8(&stream, DeviceMessage::MonitorStatus.to_i8().unwrap()).await?;
write_i32(&stream, channel).await?; write_i32(&stream, channel).await?;
write_i8(&stream, probe).await?; write_i8(&stream, probe).await?;
write_i64(&stream, current).await?; write_i32(&stream, current).await?;
*previous = Some(current); *previous = Some(current);
} }
} }
for (&(channel, overrd), previous) in inject_watch_list.iter_mut() { for (&(channel, overrd), previous) in inject_watch_list.iter_mut() {
let current = dispatch!(timer, _aux_mutex, _routing_table, channel, read_injection_status, overrd); let current = read_injection_status(channel, overrd);
if previous.is_none() || previous.unwrap() != current { if previous.is_none() || previous.unwrap() != current {
write_i8(&stream, DeviceMessage::InjectionStatus.to_i8().unwrap()).await?; write_i8(&stream, DeviceMessage::InjectionStatus.to_i8().unwrap()).await?;
write_i32(&stream, channel).await?; write_i32(&stream, channel).await?;
@ -292,30 +170,21 @@ async fn handle_connection(
*previous = Some(current); *previous = Some(current);
} }
} }
next_check = timer.get_time() + Milliseconds(200); next_check = next_check + Milliseconds(200);
} }
} }
} }
} }
pub fn start( pub fn start(timer: GlobalTimer) {
timer: GlobalTimer,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
) {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
task::spawn(async move { task::spawn(async move {
loop { loop {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
let stream = TcpStream::accept(1383, 2048, 2048).await.unwrap(); let stream = TcpStream::accept(1383, 2048, 2048).await.unwrap();
task::spawn(async move { task::spawn(async move {
info!("received connection"); info!("received connection");
let routing_table = routing_table.borrow(); let result = handle_connection(&stream, timer).await;
let result = handle_connection(&stream, timer, &aux_mutex, &routing_table).await;
match result { match result {
Err(Error::NetworkError(smoltcp::Error::Finished)) => info!("peer closed connection"), Err(Error::NetworkError(smoltcp::Error::Illegal)) => info!("peer closed connection"),
Err(error) => warn!("connection terminated: {}", error), Err(error) => warn!("connection terminated: {}", error),
_ => (), _ => (),
} }

58
src/runtime/src/panic.rs
View File

@ -1,22 +1,22 @@
#[cfg(feature = "target_kasli_soc")] use libboard_zynq::{print, println};
use libboard_zynq::error_led::ErrorLED;
use libboard_zynq::{print, println, timer::GlobalTimer};
use libconfig::Config;
use libcortex_a9::regs::MPIDR;
use libregister::RegisterR; use libregister::RegisterR;
use log::error; use libcortex_a9::regs::MPIDR;
use unwind::backtrace; use unwind::backtrace;
use crate::comms::soft_panic_main;
static mut PANICKED: [bool; 2] = [false; 2]; static mut PANICKED: [bool; 2] = [false; 2];
static mut SOFT_PANICKED: bool = false;
#[panic_handler] #[panic_handler]
fn panic(info: &core::panic::PanicInfo) -> ! { fn panic(info: &core::panic::PanicInfo) -> ! {
let id = MPIDR.read().cpu_id() as usize; let id = MPIDR.read().cpu_id() as usize;
let soft_panicked = unsafe { SOFT_PANICKED }; print!("Core {} ", id);
print!("Core {} panic at ", id); unsafe {
if PANICKED[id] {
println!("nested panic!");
loop {}
}
PANICKED[id] = true;
}
print!("panic at ");
if let Some(location) = info.location() { if let Some(location) = info.location() {
print!("{}:{}:{}", location.file(), location.line(), location.column()); print!("{}:{}:{}", location.file(), location.line(), location.column());
} else { } else {
@ -27,20 +27,6 @@ fn panic(info: &core::panic::PanicInfo) -> ! {
} else { } else {
println!(""); println!("");
} }
unsafe {
// soft panics only allowed for core 0
if PANICKED[id] && (SOFT_PANICKED || id == 1) {
println!("nested panic!");
loop {}
}
SOFT_PANICKED = true;
PANICKED[id] = true;
}
#[cfg(feature = "target_kasli_soc")]
{
let mut err_led = ErrorLED::error_led();
err_led.toggle(true);
}
println!("Backtrace: "); println!("Backtrace: ");
let _ = backtrace(|ip| { let _ = backtrace(|ip| {
// Backtrace gives us the return address, i.e. the address after the delay slot, // Backtrace gives us the return address, i.e. the address after the delay slot,
@ -48,26 +34,6 @@ fn panic(info: &core::panic::PanicInfo) -> ! {
print!("{:#08x} ", ip - 2 * 4); print!("{:#08x} ", ip - 2 * 4);
}); });
println!("\nEnd backtrace"); println!("\nEnd backtrace");
if !soft_panicked && id == 0 {
soft_panic(info);
}
loop {} loop {}
} }
fn soft_panic(info: &core::panic::PanicInfo) -> ! {
// write panic info to log, so coremgmt can also read it
if let Some(location) = info.location() {
error!("panic at {}:{}:{}", location.file(), location.line(), location.column());
} else {
error!("panic at unknown location");
}
if let Some(message) = info.message() {
error!("panic message: {}", message);
}
let timer = GlobalTimer::start();
let cfg = match Config::new() {
Ok(cfg) => cfg,
Err(_) => Config::new_dummy(),
};
soft_panic_main(timer, cfg);
}

61
src/runtime/src/proto_async.rs
View File

@ -1,7 +1,8 @@
use core::{cell::RefCell, cmp::min}; use core::cmp::min;
use core::cell::RefCell;
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp; use libboard_zynq::smoltcp;
use libasync::smoltcp::TcpStream;
type Result<T> = core::result::Result<T, smoltcp::Error>; type Result<T> = core::result::Result<T, smoltcp::Error>;
@ -13,27 +14,25 @@ enum RecvState<T> {
pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> { pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> {
let mut state = RecvState::NeedsMore(0, true); let mut state = RecvState::NeedsMore(0, true);
loop { loop {
state = stream state = stream.recv(|buf| {
.recv(|buf| { let mut consumed = 0;
let mut consumed = 0; if let RecvState::NeedsMore(mut cur_index, _) = state {
if let RecvState::NeedsMore(mut cur_index, _) = state { for b in buf.iter() {
for b in buf.iter() { consumed += 1;
consumed += 1; if *b == pattern[cur_index] {
if *b == pattern[cur_index] { if cur_index + 1 == pattern.len() {
if cur_index + 1 == pattern.len() { return (consumed, RecvState::Completed(true));
return (consumed, RecvState::Completed(true));
}
} else {
return (consumed, RecvState::Completed(false));
} }
cur_index += 1; } else {
return (consumed, RecvState::Completed(false));
} }
(consumed, RecvState::NeedsMore(cur_index, true)) cur_index += 1;
} else {
unreachable!();
} }
}) (consumed, RecvState::NeedsMore(cur_index, true))
.await?; } else {
unreachable!();
}
}).await?;
if let RecvState::Completed(result) = state { if let RecvState::Completed(result) = state {
return Ok(result); return Ok(result);
} }
@ -41,11 +40,15 @@ pub async fn expect(stream: &TcpStream, pattern: &[u8]) -> Result<bool> {
} }
pub async fn read_bool(stream: &TcpStream) -> Result<bool> { pub async fn read_bool(stream: &TcpStream) -> Result<bool> {
Ok(stream.recv(|buf| (1, buf[0] != 0)).await?) Ok(stream.recv(|buf| {
(1, buf[0] != 0)
}).await?)
} }
pub async fn read_i8(stream: &TcpStream) -> Result<i8> { pub async fn read_i8(stream: &TcpStream) -> Result<i8> {
Ok(stream.recv(|buf| (1, buf[0] as i8)).await?) Ok(stream.recv(|buf| {
(1, buf[0] as i8)
}).await?)
} }
pub async fn read_i32(stream: &TcpStream) -> Result<i32> { pub async fn read_i32(stream: &TcpStream) -> Result<i32> {
@ -65,14 +68,12 @@ pub async fn read_chunk(stream: &TcpStream, destination: &mut [u8]) -> Result<()
let destination = RefCell::new(destination); let destination = RefCell::new(destination);
let mut done = 0; let mut done = 0;
while done < total { while done < total {
let count = stream let count = stream.recv(|buf| {
.recv(|buf| { let mut destination = destination.borrow_mut();
let mut destination = destination.borrow_mut(); let count = min(total - done, buf.len());
let count = min(total - done, buf.len()); destination[done..done + count].copy_from_slice(&buf[..count]);
destination[done..done + count].copy_from_slice(&buf[..count]); (count, count)
(count, count) }).await?;
})
.await?;
done += count; done += count;
} }
Ok(()) Ok(())

17
src/libio/proto.rs → src/runtime/src/proto_core_io.rs
View File

@ -1,14 +1,15 @@
use alloc::{string::String, vec};
use core::str::Utf8Error; use core::str::Utf8Error;
use byteorder::{ByteOrder, NativeEndian}; use byteorder::{ByteOrder, NativeEndian};
use core_io::{Error as IoError, Read, Write}; use alloc::vec;
use alloc::string::String;
use core_io::{Read, Write, Error as IoError};
#[allow(dead_code)] #[allow(dead_code)]
#[derive(Debug, Clone, PartialEq)] #[derive(Debug, Clone, PartialEq)]
pub enum ReadStringError<T> { pub enum ReadStringError<T> {
Utf8(Utf8Error), Utf8(Utf8Error),
Other(T), Other(T)
} }
pub trait ProtoRead { pub trait ProtoRead {
@ -140,9 +141,7 @@ pub trait ProtoWrite {
} }
} }
impl<T> ProtoRead for T impl<T> ProtoRead for T where T: Read + ?Sized {
where T: Read + ?Sized
{
type ReadError = IoError; type ReadError = IoError;
fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Self::ReadError> { fn read_exact(&mut self, buf: &mut [u8]) -> Result<(), Self::ReadError> {
@ -150,9 +149,7 @@ where T: Read + ?Sized
} }
} }
impl<T> ProtoWrite for T impl<T> ProtoWrite for T where T: Write + ?Sized {
where T: Write + ?Sized
{
type WriteError = IoError; type WriteError = IoError;
fn write_all(&mut self, buf: &[u8]) -> Result<(), Self::WriteError> { fn write_all(&mut self, buf: &[u8]) -> Result<(), Self::WriteError> {

511
src/runtime/src/rpc.rs
View File

@ -1,130 +1,67 @@
use alloc::boxed::Box; use core::str;
use core::{future::Future, str}; use core::future::Future;
use cslice::{CSlice, CMutSlice};
use async_recursion::async_recursion;
use byteorder::{ByteOrder, NativeEndian};
use core_io::{Error, Write};
use cslice::{CMutSlice, CSlice};
use io::proto::ProtoWrite;
use libasync::smoltcp::TcpStream;
use libboard_zynq::smoltcp;
use log::trace; use log::trace;
use byteorder::{NativeEndian, ByteOrder};
use self::tag::{split_tag, Tag, TagIterator}; use core_io::{Write, Error};
use libboard_zynq::smoltcp;
use libasync::smoltcp::TcpStream;
use alloc::boxed::Box;
use async_recursion::async_recursion;
use crate::proto_core_io::ProtoWrite;
use crate::proto_async; use crate::proto_async;
use self::tag::{Tag, TagIterator, split_tag};
#[inline] #[inline]
fn round_up(val: usize, power_of_two: usize) -> usize { fn alignment_offset(alignment: isize, ptr: isize) -> isize {
assert!(power_of_two.is_power_of_two()); (alignment - ptr % alignment) % alignment
let max_rem = power_of_two - 1;
(val + max_rem) & (!max_rem)
} }
#[inline]
unsafe fn round_up_mut<T>(ptr: *mut T, power_of_two: usize) -> *mut T {
round_up(ptr as usize, power_of_two) as *mut T
}
#[inline]
unsafe fn round_up_const<T>(ptr: *const T, power_of_two: usize) -> *const T {
round_up(ptr as usize, power_of_two) as *const T
}
#[inline]
unsafe fn align_ptr<T>(ptr: *const ()) -> *const T { unsafe fn align_ptr<T>(ptr: *const ()) -> *const T {
round_up_const(ptr, core::mem::align_of::<T>()) as *const T let alignment = core::mem::align_of::<T>() as isize;
let fix = alignment_offset(alignment, ptr as isize);
((ptr as isize) + fix) as *const T
} }
#[inline]
unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T { unsafe fn align_ptr_mut<T>(ptr: *mut ()) -> *mut T {
round_up_mut(ptr, core::mem::align_of::<T>()) as *mut T let alignment = core::mem::align_of::<T>() as isize;
let fix = alignment_offset(alignment, ptr as isize);
((ptr as isize) + fix) as *mut T
} }
/// Reads (deserializes) `length` array or list elements of type `tag` from `stream`,
/// writing them into the buffer given by `storage`.
///
/// `alloc` is used for nested allocations (if elements themselves contain
/// lists/arrays), see [recv_value].
#[async_recursion(?Send)] #[async_recursion(?Send)]
async unsafe fn recv_elements<F>( async unsafe fn recv_value<F>(stream: &TcpStream, tag: Tag<'async_recursion>, data: &mut *mut (),
stream: &TcpStream, alloc: &(impl Fn(usize) -> F + 'async_recursion))
elt_tag: Tag<'async_recursion>, -> Result<(), smoltcp::Error>
length: usize, where F: Future<Output=*mut ()>
storage: *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{
// List of simple types are special-cased in the protocol for performance.
match elt_tag {
Tag::Bool => {
let dest = core::slice::from_raw_parts_mut(storage as *mut u8, length);
proto_async::read_chunk(stream, dest).await?;
}
Tag::Int32 => {
let ptr = storage as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest);
}
Tag::Int64 | Tag::Float64 => {
let ptr = storage as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
}
_ => {
let mut data = storage;
for _ in 0..length {
recv_value(stream, elt_tag, &mut data, alloc).await?
}
}
}
Ok(())
}
/// Reads (deserializes) a value of type `tag` from `stream`, writing the results to
/// the kernel-side buffer `data` (the passed pointer to which is incremented to point
/// past the just-received data). For nested allocations (lists/arrays), `alloc` is
/// invoked any number of times with the size of the required allocation as a parameter
/// (which is assumed to be correctly aligned for all payload types).
#[async_recursion(?Send)]
async unsafe fn recv_value<F>(
stream: &TcpStream,
tag: Tag<'async_recursion>,
data: &mut *mut (),
alloc: &(impl Fn(usize) -> F + 'async_recursion),
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{ {
macro_rules! consume_value { macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{ ($ty:ty, |$ptr:ident| $map:expr) => ({
let $ptr = align_ptr_mut::<$ty>(*data); let $ptr = align_ptr_mut::<$ty>(*data);
*data = $ptr.offset(1) as *mut (); *data = $ptr.offset(1) as *mut ();
$map $map
}}; })
} }
match tag { match tag {
Tag::None => Ok(()), Tag::None => Ok(()),
Tag::Bool => consume_value!(i8, |ptr| { Tag::Bool =>
*ptr = proto_async::read_i8(stream).await?; consume_value!(i8, |ptr| {
Ok(()) *ptr = proto_async::read_i8(stream).await?;
}), Ok(())
Tag::Int32 => consume_value!(i32, |ptr| { }),
*ptr = proto_async::read_i32(stream).await?; Tag::Int32 =>
Ok(()) consume_value!(i32, |ptr| {
}), *ptr = proto_async::read_i32(stream).await?;
Tag::Int64 | Tag::Float64 => consume_value!(i64, |ptr| { Ok(())
*ptr = proto_async::read_i64(stream).await?; }),
Ok(()) Tag::Int64 | Tag::Float64 =>
}), consume_value!(i64, |ptr| {
*ptr = proto_async::read_i64(stream).await?;
Ok(())
}),
Tag::String | Tag::Bytes | Tag::ByteArray => { Tag::String | Tag::Bytes | Tag::ByteArray => {
consume_value!(CMutSlice<u8>, |ptr| { consume_value!(CMutSlice<u8>, |ptr| {
let length = proto_async::read_i32(stream).await? as usize; let length = proto_async::read_i32(stream).await? as usize;
@ -134,66 +71,117 @@ where
}) })
} }
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let alignment = tag.alignment(); *data = (*data).offset(alignment_offset(tag.alignment() as isize, *data as isize));
*data = round_up_mut(*data, alignment);
let mut it = it.clone(); let mut it = it.clone();
for _ in 0..arity { for _ in 0..arity {
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await? recv_value(stream, tag, data, alloc).await?;
} }
// Take into account any tail padding (if element(s) with largest alignment
// are not at the end).
*data = round_up_mut(*data, alignment);
Ok(()) Ok(())
} }
Tag::List(it) => { Tag::List(it) => {
#[repr(C)] #[repr(C)]
struct List { struct List { elements: *mut (), length: u32 }
elements: *mut (), consume_value!(List, |ptr| {
length: usize,
}
consume_value!(*mut List, |ptr_to_list| {
let tag = it.clone().next().expect("truncated tag");
let length = proto_async::read_i32(stream).await? as usize; let length = proto_async::read_i32(stream).await? as usize;
(*ptr).length = length as u32;
let tag = it.clone().next().expect("truncated tag");
let data_size = tag.size() * length as usize +
match tag {
Tag::Int64 | Tag::Float64 => 4,
_ => 0
};
let mut data = alloc(data_size).await;
// To avoid multiple kernel CPU roundtrips, use a single allocation for let alignment = tag.alignment();
// both the pointer/length List (slice) and the backing storage for the data = data.offset(alignment_offset(alignment as isize, data as isize));
// elements. We can assume that alloc() is aligned suitably, so just (*ptr).elements = data;
// need to take into account any extra padding required. match tag {
// (Note: At the time of writing, there will never actually be any types Tag::Bool => {
// with alignment larger than 8 bytes, so storage_offset == 0 always.) let ptr = data as *mut u8;
let list_size = 4 + 4; let dest = core::slice::from_raw_parts_mut(ptr, length);
let storage_offset = round_up(list_size, tag.alignment()); proto_async::read_chunk(stream, dest).await?;
let storage_size = tag.size() * length; },
Tag::Int32 => {
let allocation = alloc(storage_offset + storage_size).await as *mut u8; let ptr = data as *mut u32;
*ptr_to_list = allocation as *mut List; // reading as raw bytes and do endianness conversion later
let storage = allocation.offset(storage_offset as isize) as *mut (); let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
proto_async::read_chunk(stream, dest).await?;
(**ptr_to_list).length = length; drop(dest);
(**ptr_to_list).elements = storage; let dest = core::slice::from_raw_parts_mut(ptr, length);
recv_elements(stream, tag, length, storage, alloc).await NativeEndian::from_slice_u32(dest);
},
Tag::Int64 | Tag::Float64 => {
let ptr = data as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
},
_ => {
for _ in 0..(*ptr).length as usize {
recv_value(stream, tag, &mut data, alloc).await?
}
}
}
Ok(())
}) })
} }
Tag::Array(it, num_dims) => { Tag::Array(it, num_dims) => {
consume_value!(*mut (), |buffer| { consume_value!(*mut (), |buffer| {
// Deserialize length along each dimension and compute total number of let mut total_len: u32 = 1;
// elements.
let mut total_len: usize = 1;
for _ in 0..num_dims { for _ in 0..num_dims {
let len = proto_async::read_i32(stream).await? as usize; let len = proto_async::read_i32(stream).await? as u32;
total_len *= len; total_len *= len;
consume_value!(usize, |ptr| *ptr = len) consume_value!(u32, |ptr| *ptr = len )
} }
// Allocate backing storage for elements; deserialize them.
let elt_tag = it.clone().next().expect("truncated tag"); let elt_tag = it.clone().next().expect("truncated tag");
*buffer = alloc(elt_tag.size() * total_len).await; let data_size = elt_tag.size() * total_len as usize +
recv_elements(stream, elt_tag, total_len, *buffer, alloc).await match elt_tag {
Tag::Int64 | Tag::Float64 => 4,
_ => 0
};
let mut data = alloc(data_size).await;
let alignment = tag.alignment();
data = data.offset(alignment_offset(alignment as isize, data as isize));
*buffer = data;
let length = total_len as usize;
match elt_tag {
Tag::Bool => {
let ptr = data as *mut u8;
let dest = core::slice::from_raw_parts_mut(ptr, length);
proto_async::read_chunk(stream, dest).await?;
},
Tag::Int32 => {
let ptr = data as *mut u32;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 4);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u32(dest);
},
Tag::Int64 | Tag::Float64 => {
let ptr = data as *mut u64;
let dest = core::slice::from_raw_parts_mut(ptr as *mut u8, length * 8);
proto_async::read_chunk(stream, dest).await?;
drop(dest);
let dest = core::slice::from_raw_parts_mut(ptr, length);
NativeEndian::from_slice_u64(dest);
},
_ => {
for _ in 0..length {
recv_value(stream, elt_tag, &mut data, alloc).await?
}
}
}
Ok(())
}) })
} }
Tag::Range(it) => { Tag::Range(it) => {
*data = round_up_mut(*data, tag.alignment()); *data = (*data).offset(alignment_offset(tag.alignment() as isize, *data as isize));
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
recv_value(stream, tag, data, alloc).await?; recv_value(stream, tag, data, alloc).await?;
recv_value(stream, tag, data, alloc).await?; recv_value(stream, tag, data, alloc).await?;
@ -201,18 +189,14 @@ where
Ok(()) Ok(())
} }
Tag::Keyword(_) => unreachable!(), Tag::Keyword(_) => unreachable!(),
Tag::Object => unreachable!(), Tag::Object => unreachable!()
} }
} }
pub async fn recv_return<F>( pub async fn recv_return<F>(stream: &TcpStream, tag_bytes: &[u8], data: *mut (),
stream: &TcpStream, alloc: &impl Fn(usize) -> F)
tag_bytes: &[u8], -> Result<(), smoltcp::Error>
data: *mut (), where F: Future<Output=*mut ()>
alloc: &impl Fn(usize) -> F,
) -> Result<(), smoltcp::Error>
where
F: Future<Output = *mut ()>,
{ {
let mut it = TagIterator::new(tag_bytes); let mut it = TagIterator::new(tag_bytes);
trace!("recv ...->{}", it); trace!("recv ...->{}", it);
@ -224,82 +208,84 @@ where
Ok(()) Ok(())
} }
unsafe fn send_elements<W>(writer: &mut W, elt_tag: Tag, length: usize, data: *const ()) -> Result<(), Error> unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ())
where W: Write + ?Sized { -> Result<(), Error>
writer.write_u8(elt_tag.as_u8())?; where W: Write + ?Sized
match elt_tag { {
// we cannot use NativeEndian::from_slice_i32 as the data is not mutable,
// and that is not needed as the data is already in native endian
Tag::Bool => {
let slice = core::slice::from_raw_parts(data as *const u8, length);
writer.write_all(slice)?;
}
Tag::Int32 => {
let slice = core::slice::from_raw_parts(data as *const u8, length * 4);
writer.write_all(slice)?;
}
Tag::Int64 | Tag::Float64 => {
let slice = core::slice::from_raw_parts(data as *const u8, length * 8);
writer.write_all(slice)?;
}
_ => {
let mut data = data;
for _ in 0..length {
send_value(writer, elt_tag, &mut data)?;
}
}
}
Ok(())
}
unsafe fn send_value<W>(writer: &mut W, tag: Tag, data: &mut *const ()) -> Result<(), Error>
where W: Write + ?Sized {
macro_rules! consume_value { macro_rules! consume_value {
($ty:ty, | $ptr:ident | $map:expr) => {{ ($ty:ty, |$ptr:ident| $map:expr) => ({
let $ptr = align_ptr::<$ty>(*data); let $ptr = align_ptr::<$ty>(*data);
*data = $ptr.offset(1) as *const (); *data = $ptr.offset(1) as *const ();
$map $map
}}; })
} }
writer.write_u8(tag.as_u8())?; writer.write_u8(tag.as_u8())?;
match tag { match tag {
Tag::None => Ok(()), Tag::None => Ok(()),
Tag::Bool => consume_value!(u8, |ptr| writer.write_u8(*ptr)), Tag::Bool =>
Tag::Int32 => consume_value!(u32, |ptr| writer.write_u32(*ptr)), consume_value!(u8, |ptr|
Tag::Int64 | Tag::Float64 => consume_value!(u64, |ptr| writer.write_u64(*ptr)), writer.write_u8(*ptr)),
Tag::String => consume_value!(CSlice<u8>, |ptr| { Tag::Int32 =>
writer.write_string(str::from_utf8((*ptr).as_ref()).unwrap()) consume_value!(u32, |ptr|
}), writer.write_u32(*ptr)),
Tag::Bytes | Tag::ByteArray => consume_value!(CSlice<u8>, |ptr| writer.write_bytes((*ptr).as_ref())), Tag::Int64 | Tag::Float64 =>
consume_value!(u64, |ptr|
writer.write_u64(*ptr)),
Tag::String =>
consume_value!(CSlice<u8>, |ptr|
writer.write_string(str::from_utf8((*ptr).as_ref()).unwrap())),
Tag::Bytes | Tag::ByteArray =>
consume_value!(CSlice<u8>, |ptr|
writer.write_bytes((*ptr).as_ref())),
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let mut it = it.clone(); let mut it = it.clone();
writer.write_u8(arity)?; writer.write_u8(arity)?;
let mut max_alignment = 0;
for _ in 0..arity { for _ in 0..arity {
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
max_alignment = core::cmp::max(max_alignment, tag.alignment());
send_value(writer, tag, data)? send_value(writer, tag, data)?
} }
*data = round_up_const(*data, max_alignment);
Ok(()) Ok(())
} }
Tag::List(it) => { Tag::List(it) => {
#[repr(C)] #[repr(C)]
struct List { struct List { elements: *const (), length: u32 }
elements: *const (), consume_value!(List, |ptr| {
length: u32, let length = (*ptr).length as isize;
}
consume_value!(&List, |ptr| {
let length = (**ptr).length as usize;
writer.write_u32((*ptr).length)?; writer.write_u32((*ptr).length)?;
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
send_elements(writer, tag, length, (**ptr).elements) let mut data = (*ptr).elements;
writer.write_u8(tag.as_u8())?;
match tag {
Tag::Bool => {
// we can pretend this is u8...
let ptr1 = align_ptr::<u8>(data);
let slice = core::slice::from_raw_parts(ptr1, length as usize);
writer.write_all(slice)?;
},
Tag::Int32 => {
let ptr1 = align_ptr::<i32>(data);
let slice = core::slice::from_raw_parts(ptr1 as *const u8, length as usize * 4);
writer.write_all(slice)?;
},
Tag::Int64 | Tag::Float64 => {
let ptr1 = align_ptr::<i64>(data);
let slice = core::slice::from_raw_parts(ptr1 as *const u8, length as usize * 8);
writer.write_all(slice)?;
},
// non-primitive types, not sure if this would happen but we can handle it...
_ => {
for _ in 0..length {
send_value(writer, tag, &mut data)?;
}
}
};
Ok(())
}) })
} }
Tag::Array(it, num_dims) => { Tag::Array(it, num_dims) => {
writer.write_u8(num_dims)?; writer.write_u8(num_dims)?;
consume_value!(*const (), |buffer| { consume_value!(*const(), |buffer| {
let elt_tag = it.clone().next().expect("truncated tag"); let elt_tag = it.clone().next().expect("truncated tag");
let mut total_len = 1; let mut total_len = 1;
@ -309,8 +295,33 @@ where W: Write + ?Sized {
total_len *= *len; total_len *= *len;
}) })
} }
let length = total_len as usize; let mut data = *buffer;
send_elements(writer, elt_tag, length, *buffer) let length = total_len as isize;
writer.write_u8(elt_tag.as_u8())?;
match elt_tag {
Tag::Bool => {
let ptr1 = align_ptr::<u8>(data);
let slice = core::slice::from_raw_parts(ptr1, length as usize);
writer.write_all(slice)?;
},
Tag::Int32 => {
let ptr1 = align_ptr::<i32>(data);
let slice = core::slice::from_raw_parts(ptr1 as *const u8, length as usize * 4);
writer.write_all(slice)?;
},
Tag::Int64 | Tag::Float64 => {
let ptr1 = align_ptr::<i64>(data);
let slice = core::slice::from_raw_parts(ptr1 as *const u8, length as usize * 8);
writer.write_all(slice)?;
},
// non-primitive types, not sure if this would happen but we can handle it...
_ => {
for _ in 0..length {
send_value(writer, elt_tag, &mut data)?;
}
}
};
Ok(())
}) })
} }
Tag::Range(it) => { Tag::Range(it) => {
@ -322,9 +333,7 @@ where W: Write + ?Sized {
} }
Tag::Keyword(it) => { Tag::Keyword(it) => {
#[repr(C)] #[repr(C)]
struct Keyword<'a> { struct Keyword<'a> { name: CSlice<'a, u8> }
name: CSlice<'a, u8>,
}
consume_value!(Keyword, |ptr| { consume_value!(Keyword, |ptr| {
writer.write_string(str::from_utf8((*ptr).name.as_ref()).unwrap())?; writer.write_string(str::from_utf8((*ptr).name.as_ref()).unwrap())?;
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
@ -336,16 +345,17 @@ where W: Write + ?Sized {
} }
Tag::Object => { Tag::Object => {
#[repr(C)] #[repr(C)]
struct Object { struct Object { id: u32 }
id: u32, consume_value!(*const Object, |ptr|
} writer.write_u32((**ptr).id))
consume_value!(*const Object, |ptr| writer.write_u32((**ptr).id))
} }
} }
} }
pub fn send_args<W>(writer: &mut W, service: u32, tag_bytes: &[u8], data: *const *const ()) -> Result<(), Error> pub fn send_args<W>(writer: &mut W, service: u32, tag_bytes: &[u8], data: *const *const ())
where W: Write + ?Sized { -> Result<(), Error>
where W: Write + ?Sized
{
let (arg_tags_bytes, return_tag_bytes) = split_tag(tag_bytes); let (arg_tags_bytes, return_tag_bytes) = split_tag(tag_bytes);
let mut args_it = TagIterator::new(arg_tags_bytes); let mut args_it = TagIterator::new(arg_tags_bytes);
@ -358,7 +368,7 @@ where W: Write + ?Sized {
let mut data = unsafe { *data.offset(index) }; let mut data = unsafe { *data.offset(index) };
unsafe { send_value(writer, arg_tag, &mut data)? }; unsafe { send_value(writer, arg_tag, &mut data)? };
} else { } else {
break; break
} }
} }
writer.write_u8(0)?; writer.write_u8(0)?;
@ -371,10 +381,10 @@ mod tag {
use core::fmt; use core::fmt;
pub fn split_tag(tag_bytes: &[u8]) -> (&[u8], &[u8]) { pub fn split_tag(tag_bytes: &[u8]) -> (&[u8], &[u8]) {
let tag_separator = tag_bytes let tag_separator =
.iter() tag_bytes.iter()
.position(|&b| b == b':') .position(|&b| b == b':')
.expect("tag without a return separator"); .expect("tag without a return separator");
let (arg_tags_bytes, rest) = tag_bytes.split_at(tag_separator); let (arg_tags_bytes, rest) = tag_bytes.split_at(tag_separator);
let return_tag_bytes = &rest[1..]; let return_tag_bytes = &rest[1..];
(arg_tags_bytes, return_tag_bytes) (arg_tags_bytes, return_tag_bytes)
@ -395,7 +405,7 @@ mod tag {
Array(TagIterator<'a>, u8), Array(TagIterator<'a>, u8),
Range(TagIterator<'a>), Range(TagIterator<'a>),
Keyword(TagIterator<'a>), Keyword(TagIterator<'a>),
Object, Object
} }
impl<'a> Tag<'a> { impl<'a> Tag<'a> {
@ -430,21 +440,23 @@ mod tag {
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let it = it.clone(); let it = it.clone();
it.take(arity.into()).map(|t| t.alignment()).max().unwrap() it.take(arity.into()).map(|t| t.alignment()).max().unwrap()
} },
Tag::Range(it) => { Tag::Range(it) => {
let it = it.clone(); let it = it.clone();
it.take(3).map(|t| t.alignment()).max().unwrap() it.take(3).map(|t| t.alignment()).max().unwrap()
} }
// the ptr/length(s) pair is basically CSlice // CSlice basically
Tag::Bytes | Tag::String | Tag::ByteArray | Tag::List(_) | Tag::Array(_, _) => { Tag::Bytes | Tag::String | Tag::ByteArray | Tag::List(_) =>
core::mem::align_of::<CSlice<()>>() core::mem::align_of::<CSlice<()>>(),
} // array buffer is allocated, so no need for alignment first
Tag::Keyword(_) => unreachable!("Tag::Keyword should not appear in composite types"), Tag::Array(_, _) => 1,
Tag::Object => core::mem::align_of::<u32>(), // will not be sent from the host
_ => unreachable!("unexpected tag from host")
} }
} }
pub fn size(self) -> usize { pub fn size(self) -> usize {
use super::alignment_offset;
match self { match self {
Tag::None => 0, Tag::None => 0,
Tag::Bool => 1, Tag::Bool => 1,
@ -456,21 +468,16 @@ mod tag {
Tag::ByteArray => 8, Tag::ByteArray => 8,
Tag::Tuple(it, arity) => { Tag::Tuple(it, arity) => {
let mut size = 0; let mut size = 0;
let mut max_alignment = 0;
let mut it = it.clone(); let mut it = it.clone();
for _ in 0..arity { for _ in 0..arity {
let tag = it.next().expect("truncated tag"); let tag = it.next().expect("truncated tag");
let alignment = tag.alignment();
max_alignment = core::cmp::max(max_alignment, alignment);
size = super::round_up(size, alignment);
size += tag.size(); size += tag.size();
// includes padding
size += alignment_offset(tag.alignment() as isize, size as isize) as usize;
} }
// Take into account any tail padding (if element(s) with largest
// alignment are not at the end).
size = super::round_up(size, max_alignment);
size size
} }
Tag::List(_) => 4, Tag::List(_) => 8,
Tag::Array(_, num_dims) => 4 * (1 + num_dims as usize), Tag::Array(_, num_dims) => 4 * (1 + num_dims as usize),
Tag::Range(it) => { Tag::Range(it) => {
let tag = it.clone().next().expect("truncated tag"); let tag = it.clone().next().expect("truncated tag");
@ -484,7 +491,7 @@ mod tag {
#[derive(Debug, Clone, Copy)] #[derive(Debug, Clone, Copy)]
pub struct TagIterator<'a> { pub struct TagIterator<'a> {
data: &'a [u8], data: &'a [u8]
} }
impl<'a> TagIterator<'a> { impl<'a> TagIterator<'a> {
@ -492,14 +499,13 @@ mod tag {
TagIterator { data } TagIterator { data }
} }
fn sub(&mut self, count: u8) -> TagIterator<'a> { fn sub(&mut self, count: u8) -> TagIterator<'a> {
let data = self.data; let data = self.data;
for _ in 0..count { for _ in 0..count {
self.next().expect("truncated tag"); self.next().expect("truncated tag");
} }
TagIterator { TagIterator { data: &data[..(data.len() - self.data.len())] }
data: &data[..(data.len() - self.data.len())],
}
} }
} }
@ -508,7 +514,7 @@ mod tag {
fn next(&mut self) -> Option<Tag<'a>> { fn next(&mut self) -> Option<Tag<'a>> {
if self.data.len() == 0 { if self.data.len() == 0 {
return None; return None
} }
let tag_byte = self.data[0]; let tag_byte = self.data[0];
@ -536,7 +542,7 @@ mod tag {
b'r' => Tag::Range(self.sub(1)), b'r' => Tag::Range(self.sub(1)),
b'k' => Tag::Keyword(self.sub(1)), b'k' => Tag::Keyword(self.sub(1)),
b'O' => Tag::Object, b'O' => Tag::Object,
_ => unreachable!(), _ => unreachable!()
}) })
} }
} }
@ -553,14 +559,22 @@ mod tag {
} }
match tag { match tag {
Tag::None => write!(f, "None")?, Tag::None =>
Tag::Bool => write!(f, "Bool")?, write!(f, "None")?,
Tag::Int32 => write!(f, "Int32")?, Tag::Bool =>
Tag::Int64 => write!(f, "Int64")?, write!(f, "Bool")?,
Tag::Float64 => write!(f, "Float64")?, Tag::Int32 =>
Tag::String => write!(f, "String")?, write!(f, "Int32")?,
Tag::Bytes => write!(f, "Bytes")?, Tag::Int64 =>
Tag::ByteArray => write!(f, "ByteArray")?, write!(f, "Int64")?,
Tag::Float64 =>
write!(f, "Float64")?,
Tag::String =>
write!(f, "String")?,
Tag::Bytes =>
write!(f, "Bytes")?,
Tag::ByteArray =>
write!(f, "ByteArray")?,
Tag::Tuple(it, _) => { Tag::Tuple(it, _) => {
write!(f, "Tuple(")?; write!(f, "Tuple(")?;
it.fmt(f)?; it.fmt(f)?;
@ -586,7 +600,8 @@ mod tag {
it.fmt(f)?; it.fmt(f)?;
write!(f, ")")?; write!(f, ")")?;
} }
Tag::Object => write!(f, "Object")?, Tag::Object =>
write!(f, "Object")?,
} }
} }

175
src/runtime/src/rtio_acp.rs
View File

@ -1,18 +1,18 @@
use cslice::CSlice;
use vcell::VolatileCell;
use libcortex_a9::asm;
use crate::artiq_raise;
use core::sync::atomic::{fence, Ordering}; use core::sync::atomic::{fence, Ordering};
use cslice::CSlice; use crate::pl::csr;
use libcortex_a9::asm;
use vcell::VolatileCell;
use crate::{artiq_raise, pl::csr, rtio_mgt::resolve_channel_name}; pub const RTIO_O_STATUS_WAIT: i32 = 1;
pub const RTIO_O_STATUS_UNDERFLOW: i32 = 2;
pub const RTIO_O_STATUS_WAIT: i32 = 1; pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: i32 = 4;
pub const RTIO_O_STATUS_UNDERFLOW: i32 = 2; pub const RTIO_I_STATUS_WAIT_EVENT: i32 = 1;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: i32 = 4; pub const RTIO_I_STATUS_OVERFLOW: i32 = 2;
pub const RTIO_I_STATUS_WAIT_EVENT: i32 = 1; pub const RTIO_I_STATUS_WAIT_STATUS: i32 = 4; // TODO
pub const RTIO_I_STATUS_OVERFLOW: i32 = 2; pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: i32 = 8;
pub const RTIO_I_STATUS_WAIT_STATUS: i32 = 4; // TODO
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: i32 = 8;
#[repr(C)] #[repr(C)]
pub struct TimestampedData { pub struct TimestampedData {
@ -46,10 +46,10 @@ static mut TRANSACTION_BUFFER: Transaction = Transaction {
reply_timestamp: VolatileCell::new(0), reply_timestamp: VolatileCell::new(0),
padding0: [0; 2], padding0: [0; 2],
padding1: [0; 2], padding1: [0; 2],
padding2: [0; 2], padding2: [0; 2]
}; };
pub extern "C" fn init() { pub extern fn init() {
unsafe { unsafe {
csr::rtio_core::reset_write(1); csr::rtio_core::reset_write(1);
csr::rtio::engine_addr_base_write(&TRANSACTION_BUFFER as *const Transaction as u32); csr::rtio::engine_addr_base_write(&TRANSACTION_BUFFER as *const Transaction as u32);
@ -57,7 +57,12 @@ pub extern "C" fn init() {
} }
} }
pub extern "C" fn get_counter() -> i64 { pub extern fn get_destination_status(destination: i32) -> bool {
// TODO
destination == 0
}
pub extern fn get_counter() -> i64 {
unsafe { unsafe {
csr::rtio::counter_update_write(1); csr::rtio::counter_update_write(1);
csr::rtio::counter_read() as i64 csr::rtio::counter_read() as i64
@ -66,15 +71,15 @@ pub extern "C" fn get_counter() -> i64 {
static mut NOW: i64 = 0; static mut NOW: i64 = 0;
pub extern "C" fn now_mu() -> i64 { pub extern fn now_mu() -> i64 {
unsafe { NOW } unsafe { NOW }
} }
pub extern "C" fn at_mu(t: i64) { pub extern fn at_mu(t: i64) {
unsafe { NOW = t } unsafe { NOW = t }
} }
pub extern "C" fn delay_mu(dt: i64) { pub extern fn delay_mu(dt: i64) {
unsafe { NOW += dt } unsafe { NOW += dt }
} }
@ -86,34 +91,18 @@ unsafe fn process_exceptional_status(channel: i32, status: i32) {
while csr::rtio::o_status_read() as i32 & RTIO_O_STATUS_WAIT != 0 {} while csr::rtio::o_status_read() as i32 & RTIO_O_STATUS_WAIT != 0 {}
} }
if status & RTIO_O_STATUS_UNDERFLOW != 0 { if status & RTIO_O_STATUS_UNDERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOUnderflow",
"RTIOUnderflow", "RTIO underflow at {0} mu, channel {1}, slack {2} mu",
format!( timestamp, channel as i64, timestamp - get_counter());
"RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
timestamp,
timestamp - get_counter()
);
} }
if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, output, at {0} mu, channel {1}",
format!( timestamp, channel as i64, 0);
"RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
timestamp,
channel as i64,
0
);
} }
} }
pub extern "C" fn output(target: i32, data: i32) { pub extern fn output(target: i32, data: i32) {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -141,7 +130,7 @@ pub extern "C" fn output(target: i32, data: i32) {
} }
} }
pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) { pub extern fn output_wide(target: i32, data: CSlice<i32>) {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -158,7 +147,7 @@ pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
@ -169,8 +158,8 @@ pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) {
} }
} }
pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 { pub extern fn input_timestamp(timeout: i64, channel: i32) -> i64 {
unsafe { unsafe {
// Clear status so we can observe response // Clear status so we can observe response
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -186,45 +175,29 @@ pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_WAIT_EVENT != 0 { if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return -1; return -1
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TRANSACTION_BUFFER.reply_timestamp.get() TRANSACTION_BUFFER.reply_timestamp.get()
} }
} }
pub extern "C" fn input_data(channel: i32) -> i32 { pub extern fn input_data(channel: i32) -> i32 {
unsafe { unsafe {
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -240,42 +213,26 @@ pub extern "C" fn input_data(channel: i32) -> i32 {
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TRANSACTION_BUFFER.reply_data.get() TRANSACTION_BUFFER.reply_data.get()
} }
} }
pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData { pub extern fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData {
unsafe { unsafe {
TRANSACTION_BUFFER.reply_status.set(0); TRANSACTION_BUFFER.reply_status.set(0);
@ -291,35 +248,19 @@ pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> Timestam
loop { loop {
status = TRANSACTION_BUFFER.reply_status.get(); status = TRANSACTION_BUFFER.reply_status.get();
if status != 0 { if status != 0 {
break; break
} }
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TimestampedData { TimestampedData {

258
src/runtime/src/rtio_clocking.rs
View File

@ -1,258 +0,0 @@
use embedded_hal::blocking::delay::DelayMs;
use libboard_artiq::pl;
#[cfg(has_si5324)]
use libboard_artiq::si5324;
#[cfg(has_si5324)]
use libboard_zynq::i2c::I2c;
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use log::{info, warn};
#[cfg(has_si5324)]
use crate::i2c;
#[derive(Debug, PartialEq, Copy, Clone)]
#[allow(non_camel_case_types)]
pub enum RtioClock {
Default,
Int_125,
Int_100,
Int_150,
Ext0_Bypass,
Ext0_Synth0_10to125,
Ext0_Synth0_100to125,
Ext0_Synth0_125to125,
}
#[allow(unreachable_code)]
fn get_rtio_clock_cfg(cfg: &Config) -> RtioClock {
let mut res = RtioClock::Default;
if let Ok(clk) = cfg.read_str("rtio_clock") {
res = match clk.as_ref() {
"int_125" => RtioClock::Int_125,
"int_100" => RtioClock::Int_100,
"int_150" => RtioClock::Int_150,
"ext0_bypass" => RtioClock::Ext0_Bypass,
"ext0_bypass_125" => RtioClock::Ext0_Bypass,
"ext0_bypass_100" => RtioClock::Ext0_Bypass,
"ext0_synth0_10to125" => RtioClock::Ext0_Synth0_10to125,
"ext0_synth0_100to125" => RtioClock::Ext0_Synth0_100to125,
"ext0_synth0_125to125" => RtioClock::Ext0_Synth0_125to125,
_ => {
warn!("Unrecognised rtio_clock setting. Falling back to default.");
RtioClock::Default
}
};
} else {
warn!("error reading configuration. Falling back to default.");
}
if res == RtioClock::Default {
#[cfg(rtio_frequency = "100.0")]
{
warn!("Using default configuration - internal 100MHz RTIO clock.");
return RtioClock::Int_100;
}
#[cfg(rtio_frequency = "125.0")]
{
warn!("Using default configuration - internal 125MHz RTIO clock.");
return RtioClock::Int_125;
}
// anything else
{
warn!("Using default configuration - internal 125MHz RTIO clock.");
return RtioClock::Int_125;
}
}
res
}
#[cfg(not(has_drtio))]
fn init_rtio(timer: &mut GlobalTimer) {
info!("Switching SYS clocks...");
unsafe {
pl::csr::sys_crg::clock_switch_write(1);
}
// if it's not locked, it will hang at the CSR.
timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
} else {
panic!("SYS CLK did not switch");
}
unsafe {
pl::csr::rtio_core::reset_phy_write(1);
}
info!("SYS PLL locked");
}
#[cfg(has_drtio)]
fn init_drtio(timer: &mut GlobalTimer) {
unsafe {
pl::csr::drtio_transceiver::stable_clkin_write(1);
}
timer.delay_ms(20); // wait for CPLL/QPLL/SYS PLL lock
let clk = unsafe { pl::csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
} else {
panic!("SYS CLK did not switch");
}
unsafe {
pl::csr::rtio_core::reset_phy_write(1);
pl::csr::drtio_transceiver::txenable_write(0xffffffffu32 as _);
}
}
// Si5324 input to select for locking to an external clock.
#[cfg(has_si5324)]
const SI5324_EXT_INPUT: si5324::Input = si5324::Input::Ckin1;
#[cfg(has_si5324)]
fn setup_si5324(i2c: &mut I2c, timer: &mut GlobalTimer, clk: RtioClock) {
let (si5324_settings, si5324_ref_input) = match clk {
RtioClock::Ext0_Synth0_10to125 => {
// 125 MHz output from 10 MHz CLKINx reference, 504 Hz BW
info!("using 10MHz reference to make 125MHz RTIO clock with PLL");
(
si5324::FrequencySettings {
n1_hs: 10,
nc1_ls: 4,
n2_hs: 10,
n2_ls: 300,
n31: 6,
n32: 6,
bwsel: 4,
crystal_as_ckin2: false,
},
SI5324_EXT_INPUT,
)
}
RtioClock::Ext0_Synth0_100to125 => {
// 125MHz output, from 100MHz CLKINx reference, 586 Hz loop bandwidth
info!("using 100MHz reference to make 125MHz RTIO clock with PLL");
(
si5324::FrequencySettings {
n1_hs: 10,
nc1_ls: 4,
n2_hs: 10,
n2_ls: 260,
n31: 52,
n32: 52,
bwsel: 4,
crystal_as_ckin2: false,
},
SI5324_EXT_INPUT,
)
}
RtioClock::Ext0_Synth0_125to125 => {
// 125MHz output, from 125MHz CLKINx reference, 606 Hz loop bandwidth
info!("using 125MHz reference to make 125MHz RTIO clock with PLL");
(
si5324::FrequencySettings {
n1_hs: 5,
nc1_ls: 8,
n2_hs: 7,
n2_ls: 360,
n31: 63,
n32: 63,
bwsel: 4,
crystal_as_ckin2: false,
},
SI5324_EXT_INPUT,
)
}
RtioClock::Int_150 => {
// 150MHz output, from crystal
info!("using internal 150MHz RTIO clock");
(
si5324::FrequencySettings {
n1_hs: 9,
nc1_ls: 4,
n2_hs: 10,
n2_ls: 33732,
n31: 7139,
n32: 7139,
bwsel: 3,
crystal_as_ckin2: true,
},
si5324::Input::Ckin2,
)
}
RtioClock::Int_100 => {
// 100MHz output, from crystal
info!("using internal 100MHz RTIO clock");
(
si5324::FrequencySettings {
n1_hs: 9,
nc1_ls: 6,
n2_hs: 10,
n2_ls: 33732,
n31: 7139,
n32: 7139,
bwsel: 3,
crystal_as_ckin2: true,
},
si5324::Input::Ckin2,
)
}
RtioClock::Int_125 => {
// 125MHz output, from crystal, 7 Hz
info!("using internal 125MHz RTIO clock");
(
si5324::FrequencySettings {
n1_hs: 10,
nc1_ls: 4,
n2_hs: 10,
n2_ls: 19972,
n31: 4565,
n32: 4565,
bwsel: 4,
crystal_as_ckin2: true,
},
si5324::Input::Ckin2,
)
}
_ => {
// same setting as Int_125, but fallback to default
warn!(
"rtio_clock setting '{:?}' is unsupported. Falling back to default internal 125MHz RTIO clock.",
clk
);
(
si5324::FrequencySettings {
n1_hs: 10,
nc1_ls: 4,
n2_hs: 10,
n2_ls: 19972,
n31: 4565,
n32: 4565,
bwsel: 4,
crystal_as_ckin2: true,
},
si5324::Input::Ckin2,
)
}
};
si5324::setup(i2c, &si5324_settings, si5324_ref_input, timer).expect("cannot initialize Si5324");
}
pub fn init(timer: &mut GlobalTimer, cfg: &Config) {
let clk = get_rtio_clock_cfg(cfg);
#[cfg(has_si5324)]
{
let i2c = unsafe { (&mut i2c::I2C_BUS).as_mut().unwrap() };
match clk {
RtioClock::Ext0_Bypass => si5324::bypass(i2c, SI5324_EXT_INPUT, timer).expect("cannot bypass Si5324"),
_ => setup_si5324(i2c, timer, clk),
}
}
#[cfg(has_drtio)]
init_drtio(timer);
#[cfg(not(has_drtio))]
init_rtio(timer);
}

172
src/runtime/src/rtio_csr.rs
View File

@ -1,16 +1,17 @@
use core::ptr::{read_volatile, write_volatile}; use core::ptr::{read_volatile, write_volatile};
use cslice::CSlice; use cslice::CSlice;
use crate::{artiq_raise, pl::csr, rtio_mgt::resolve_channel_name}; use crate::artiq_raise;
pub const RTIO_O_STATUS_WAIT: u8 = 1; use crate::pl::csr;
pub const RTIO_O_STATUS_UNDERFLOW: u8 = 2;
pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: u8 = 4; pub const RTIO_O_STATUS_WAIT: u8 = 1;
pub const RTIO_I_STATUS_WAIT_EVENT: u8 = 1; pub const RTIO_O_STATUS_UNDERFLOW: u8 = 2;
pub const RTIO_I_STATUS_OVERFLOW: u8 = 2; pub const RTIO_O_STATUS_DESTINATION_UNREACHABLE: u8 = 4;
pub const RTIO_I_STATUS_WAIT_STATUS: u8 = 4; pub const RTIO_I_STATUS_WAIT_EVENT: u8 = 1;
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: u8 = 8; pub const RTIO_I_STATUS_OVERFLOW: u8 = 2;
pub const RTIO_I_STATUS_WAIT_STATUS: u8 = 4;
pub const RTIO_I_STATUS_DESTINATION_UNREACHABLE: u8 = 8;
#[repr(C)] #[repr(C)]
pub struct TimestampedData { pub struct TimestampedData {
@ -18,30 +19,37 @@ pub struct TimestampedData {
data: i32, data: i32,
} }
pub extern "C" fn init() { pub extern fn init() {
unsafe { unsafe {
csr::rtio_core::reset_write(1); csr::rtio_core::reset_write(1);
} }
} }
pub extern "C" fn get_counter() -> i64 { pub extern fn get_destination_status(destination: i32) -> bool {
// TODO
destination == 0
}
pub extern fn get_counter() -> i64 {
unsafe { unsafe {
csr::rtio::counter_update_write(1); csr::rtio::counter_update_write(1);
csr::rtio::counter_read() as i64 csr::rtio::counter_read() as i64
} }
} }
pub extern "C" fn now_mu() -> i64 { pub extern fn now_mu() -> i64 {
unsafe { csr::rtio::now_read() as i64 } unsafe {
csr::rtio::now_read() as i64
}
} }
pub extern "C" fn at_mu(t: i64) { pub extern fn at_mu(t: i64) {
unsafe { unsafe {
csr::rtio::now_write(t as u64); csr::rtio::now_write(t as u64);
} }
} }
pub extern "C" fn delay_mu(dt: i64) { pub extern fn delay_mu(dt: i64) {
unsafe { unsafe {
csr::rtio::now_write(csr::rtio::now_read() + dt as u64); csr::rtio::now_write(csr::rtio::now_read() + dt as u64);
} }
@ -52,13 +60,13 @@ pub extern "C" fn delay_mu(dt: i64) {
pub unsafe fn rtio_o_data_write(offset: usize, data: u32) { pub unsafe fn rtio_o_data_write(offset: usize, data: u32) {
write_volatile( write_volatile(
csr::rtio::O_DATA_ADDR.offset((csr::rtio::O_DATA_SIZE - 1 - offset) as isize), csr::rtio::O_DATA_ADDR.offset((csr::rtio::O_DATA_SIZE - 1 - offset) as isize),
data, data);
);
} }
#[inline(always)] #[inline(always)]
pub unsafe fn rtio_i_data_read(offset: usize) -> u32 { pub unsafe fn rtio_i_data_read(offset: usize) -> u32 {
read_volatile(csr::rtio::I_DATA_ADDR.offset((csr::rtio::I_DATA_SIZE - 1 - offset) as isize)) read_volatile(
csr::rtio::I_DATA_ADDR.offset((csr::rtio::I_DATA_SIZE - 1 - offset) as isize))
} }
#[inline(never)] #[inline(never)]
@ -68,34 +76,18 @@ unsafe fn process_exceptional_status(channel: i32, status: u8) {
while csr::rtio::o_status_read() & RTIO_O_STATUS_WAIT != 0 {} while csr::rtio::o_status_read() & RTIO_O_STATUS_WAIT != 0 {}
} }
if status & RTIO_O_STATUS_UNDERFLOW != 0 { if status & RTIO_O_STATUS_UNDERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOUnderflow",
"RTIOUnderflow", "RTIO underflow at {0} mu, channel {1}, slack {2} mu",
format!( timestamp, channel as i64, timestamp - get_counter());
"RTIO underflow at {{1}} mu, channel 0x{:04x}:{}, slack {{2}} mu",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
timestamp,
timestamp - get_counter()
);
} }
if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_O_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, output, at {0} mu, channel {1}",
format!( timestamp, channel as i64, 0);
"RTIO destination unreachable, output, at {{0}} mu, channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
timestamp,
channel as i64,
0
);
} }
} }
pub extern "C" fn output(target: i32, data: i32) { pub extern fn output(target: i32, data: i32) {
unsafe { unsafe {
csr::rtio::target_write(target as u32); csr::rtio::target_write(target as u32);
// writing target clears o_data // writing target clears o_data
@ -107,7 +99,7 @@ pub extern "C" fn output(target: i32, data: i32) {
} }
} }
pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) { pub extern fn output_wide(target: i32, data: CSlice<i32>) {
unsafe { unsafe {
csr::rtio::target_write(target as u32); csr::rtio::target_write(target as u32);
// writing target clears o_data // writing target clears o_data
@ -121,7 +113,7 @@ pub extern "C" fn output_wide(target: i32, data: CSlice<i32>) {
} }
} }
pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 { pub extern fn input_timestamp(timeout: i64, channel: i32) -> i64 {
unsafe { unsafe {
csr::rtio::target_write((channel as u32) << 8); csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64); csr::rtio::i_timeout_write(timeout as u64);
@ -132,40 +124,24 @@ pub extern "C" fn input_timestamp(timeout: i64, channel: i32) -> i64 {
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_WAIT_EVENT != 0 { if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return -1; return -1
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
csr::rtio::i_timestamp_read() as i64 csr::rtio::i_timestamp_read() as i64
} }
} }
pub extern "C" fn input_data(channel: i32) -> i32 { pub extern fn input_data(channel: i32) -> i32 {
unsafe { unsafe {
csr::rtio::target_write((channel as u32) << 8); csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(0xffffffff_ffffffff); csr::rtio::i_timeout_write(0xffffffff_ffffffff);
@ -176,37 +152,21 @@ pub extern "C" fn input_data(channel: i32) -> i32 {
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
rtio_i_data_read(0) as i32 rtio_i_data_read(0) as i32
} }
} }
pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData { pub extern fn input_timestamped_data(timeout: i64, channel: i32) -> TimestampedData {
unsafe { unsafe {
csr::rtio::target_write((channel as u32) << 8); csr::rtio::target_write((channel as u32) << 8);
csr::rtio::i_timeout_write(timeout as u64); csr::rtio::i_timeout_write(timeout as u64);
@ -217,38 +177,22 @@ pub extern "C" fn input_timestamped_data(timeout: i64, channel: i32) -> Timestam
} }
if status & RTIO_I_STATUS_OVERFLOW != 0 { if status & RTIO_I_STATUS_OVERFLOW != 0 {
artiq_raise!( artiq_raise!("RTIOOverflow",
"RTIOOverflow", "RTIO input overflow on channel {0}",
format!( channel as i64, 0, 0);
"RTIO input overflow on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
if status & RTIO_I_STATUS_WAIT_EVENT != 0 { if status & RTIO_I_STATUS_WAIT_EVENT != 0 {
return TimestampedData { timestamp: -1, data: 0 }; return TimestampedData { timestamp: -1, data: 0 }
} }
if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 { if status & RTIO_I_STATUS_DESTINATION_UNREACHABLE != 0 {
artiq_raise!( artiq_raise!("RTIODestinationUnreachable",
"RTIODestinationUnreachable", "RTIO destination unreachable, input, on channel {0}",
format!( channel as i64, 0, 0);
"RTIO destination unreachable, input, on channel 0x{:04x}:{}",
channel,
resolve_channel_name(channel as u32)
),
channel as i64,
0,
0
);
} }
TimestampedData { TimestampedData {
timestamp: csr::rtio::i_timestamp_read() as i64, timestamp: csr::rtio::i_timestamp_read() as i64,
data: rtio_i_data_read(0) as i32, data: rtio_i_data_read(0) as i32
} }
} }
} }

349
src/runtime/src/rtio_dma.rs
View File

@ -1,349 +0,0 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String, vec::Vec};
#[cfg(has_drtio)]
use core::mem;
#[cfg(has_drtio)]
use libasync::task;
use libboard_artiq::drtio_routing::RoutingTable;
use libboard_zynq::timer::GlobalTimer;
use libcortex_a9::{cache::dcci_slice, mutex::Mutex};
use crate::kernel::DmaRecorder;
const ALIGNMENT: usize = 16 * 8;
static DMA_RECORD_STORE: Mutex<BTreeMap<String, (u32, Vec<u8>, i64)>> = Mutex::new(BTreeMap::new());
#[cfg(has_drtio)]
pub mod remote_dma {
use libboard_zynq::time::Milliseconds;
use log::error;
use super::*;
use crate::rtio_mgt::drtio;
#[derive(Debug, PartialEq, Clone)]
pub enum RemoteState {
NotLoaded,
Loaded,
PlaybackEnded { error: u8, channel: u32, timestamp: u64 },
}
#[derive(Debug, Clone)]
struct RemoteTrace {
trace: Vec<u8>,
pub state: RemoteState,
}
impl From<Vec<u8>> for RemoteTrace {
fn from(trace: Vec<u8>) -> Self {
RemoteTrace {
trace: trace,
state: RemoteState::NotLoaded,
}
}
}
impl RemoteTrace {
pub fn get_trace(&self) -> &Vec<u8> {
&self.trace
}
}
// represents all traces for a given ID
struct TraceSet {
id: u32,
done_count: Mutex<usize>,
traces: Mutex<BTreeMap<u8, RemoteTrace>>,
}
impl TraceSet {
pub fn new(id: u32, traces: BTreeMap<u8, Vec<u8>>) -> TraceSet {
let mut trace_map: BTreeMap<u8, RemoteTrace> = BTreeMap::new();
for (destination, trace) in traces {
trace_map.insert(destination, trace.into());
}
TraceSet {
id: id,
done_count: Mutex::new(0),
traces: Mutex::new(trace_map),
}
}
pub async fn await_done(&self, timeout: Option<u64>, timer: GlobalTimer) -> Result<RemoteState, &'static str> {
let timeout_ms = Milliseconds(timeout.unwrap_or(10_000));
let limit = timer.get_time() + timeout_ms;
while (timer.get_time() < limit)
& (*(self.done_count.async_lock().await) < self.traces.async_lock().await.len())
{
task::r#yield().await;
}
if timer.get_time() >= limit {
error!("Remote DMA await done timed out");
return Err("Timed out waiting for results.");
}
let mut playback_state: RemoteState = RemoteState::PlaybackEnded {
error: 0,
channel: 0,
timestamp: 0,
};
let mut lock = self.traces.async_lock().await;
let trace_iter = lock.iter_mut();
for (_dest, trace) in trace_iter {
match trace.state {
RemoteState::PlaybackEnded {
error: e,
channel: _c,
timestamp: _ts,
} => {
if e != 0 {
playback_state = trace.state.clone();
}
}
_ => (),
}
trace.state = RemoteState::Loaded;
}
Ok(playback_state)
}
pub async fn upload_traces(
&mut self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
) {
let mut lock = self.traces.async_lock().await;
let trace_iter = lock.iter_mut();
for (destination, trace) in trace_iter {
match drtio::ddma_upload_trace(
aux_mutex,
routing_table,
timer,
self.id,
*destination,
trace.get_trace(),
)
.await
{
Ok(_) => trace.state = RemoteState::Loaded,
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
}
*(self.done_count.async_lock().await) = 0;
}
pub async fn erase(&mut self, aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer) {
let lock = self.traces.async_lock().await;
let trace_iter = lock.keys();
for destination in trace_iter {
match drtio::ddma_send_erase(aux_mutex, routing_table, timer, self.id, *destination).await {
Ok(_) => (),
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
}
}
pub async fn playback_done(&mut self, destination: u8, error: u8, channel: u32, timestamp: u64) {
let mut traces_locked = self.traces.async_lock().await;
let mut trace = traces_locked.get_mut(&destination).unwrap();
trace.state = RemoteState::PlaybackEnded {
error: error,
channel: channel,
timestamp: timestamp,
};
*(self.done_count.async_lock().await) += 1;
}
pub async fn playback(
&self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
timestamp: u64,
) {
let mut dest_list: Vec<u8> = Vec::new();
{
let lock = self.traces.async_lock().await;
let trace_iter = lock.iter();
for (dest, trace) in trace_iter {
if trace.state != RemoteState::Loaded {
error!("Destination {} not ready for DMA, state: {:?}", dest, trace.state);
continue;
}
dest_list.push(dest.clone());
}
}
// mutex lock must be dropped before sending a playback request to avoid a deadlock,
// if PlaybackStatus is sent from another satellite and the state must be updated.
for destination in dest_list {
match drtio::ddma_send_playback(aux_mutex, routing_table, timer, self.id, destination, timestamp).await
{
Ok(_) => (),
Err(e) => error!("Error during remote DMA playback: {}", e),
}
}
}
pub async fn destination_changed(
&mut self,
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
// update state of the destination, resend traces if it's up
if let Some(trace) = self.traces.lock().get_mut(&destination) {
if up {
match drtio::ddma_upload_trace(
aux_mutex,
routing_table,
timer,
self.id,
destination,
trace.get_trace(),
)
.await
{
Ok(_) => trace.state = RemoteState::Loaded,
Err(e) => error!("Error adding DMA trace on destination {}: {}", destination, e),
}
} else {
trace.state = RemoteState::NotLoaded;
}
}
}
}
static mut TRACES: BTreeMap<u32, TraceSet> = BTreeMap::new();
pub fn add_traces(id: u32, traces: BTreeMap<u8, Vec<u8>>) {
unsafe { TRACES.insert(id, TraceSet::new(id, traces)) };
}
pub async fn await_done(id: u32, timeout: Option<u64>, timer: GlobalTimer) -> Result<RemoteState, &'static str> {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.await_done(timeout, timer).await
}
pub async fn erase(aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer, id: u32) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.erase(aux_mutex, routing_table, timer).await;
unsafe {
TRACES.remove(&id);
}
}
pub async fn upload_traces(aux_mutex: &Rc<Mutex<bool>>, routing_table: &RoutingTable, timer: GlobalTimer, id: u32) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.upload_traces(aux_mutex, routing_table, timer).await;
}
pub async fn playback(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
id: u32,
timestamp: u64,
) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.playback(aux_mutex, routing_table, timer, timestamp).await;
}
pub async fn playback_done(id: u32, destination: u8, error: u8, channel: u32, timestamp: u64) {
let trace_set = unsafe { TRACES.get_mut(&id).unwrap() };
trace_set.playback_done(destination, error, channel, timestamp).await;
}
pub async fn destination_changed(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &RoutingTable,
timer: GlobalTimer,
destination: u8,
up: bool,
) {
let trace_iter = unsafe { TRACES.values_mut() };
for trace_set in trace_iter {
trace_set
.destination_changed(aux_mutex, routing_table, timer, destination, up)
.await;
}
}
}
pub async fn put_record(
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &RoutingTable,
_timer: GlobalTimer,
mut recorder: DmaRecorder,
) -> u32 {
#[cfg(has_drtio)]
let mut remote_traces: BTreeMap<u8, Vec<u8>> = BTreeMap::new();
#[cfg(has_drtio)]
if recorder.enable_ddma {
let mut local_trace: Vec<u8> = Vec::new();
// analyze each entry and put in proper buckets, as the kernel core
// sends whole chunks, to limit comms/kernel CPU communication,
// and as only comms core has access to varios DMA buffers.
let mut ptr = 0;
recorder.buffer.push(0);
while recorder.buffer[ptr] != 0 {
// ptr + 3 = tgt >> 24 (destination)
let len = recorder.buffer[ptr] as usize;
let destination = recorder.buffer[ptr + 3];
if destination == 0 {
local_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
if let Some(remote_trace) = remote_traces.get_mut(&destination) {
remote_trace.extend(&recorder.buffer[ptr..ptr + len]);
} else {
remote_traces.insert(destination, recorder.buffer[ptr..ptr + len].to_vec());
}
}
// and jump to the next event
ptr += len;
}
mem::swap(&mut recorder.buffer, &mut local_trace);
}
// trailing zero to indicate end of buffer
recorder.buffer.push(0);
recorder.buffer.reserve(ALIGNMENT - 1);
let original_length = recorder.buffer.len();
let padding = ALIGNMENT - recorder.buffer.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
recorder.buffer.push(0);
}
recorder.buffer.copy_within(0..original_length, padding);
dcci_slice(&recorder.buffer);
let ptr = recorder.buffer[padding..].as_ptr() as u32;
let _old_record = DMA_RECORD_STORE
.lock()
.insert(recorder.name, (ptr, recorder.buffer, recorder.duration));
#[cfg(has_drtio)]
{
if let Some((old_id, _v, _d)) = _old_record {
remote_dma::erase(_aux_mutex, _routing_table, _timer, old_id).await;
}
remote_dma::add_traces(ptr, remote_traces);
}
ptr
}
pub async fn erase(name: String, _aux_mutex: &Rc<Mutex<bool>>, _routing_table: &RoutingTable, _timer: GlobalTimer) {
let _entry = DMA_RECORD_STORE.lock().remove(&name);
#[cfg(has_drtio)]
if let Some((id, _v, _d)) = _entry {
remote_dma::erase(_aux_mutex, _routing_table, _timer, id).await;
}
}
pub fn retrieve(name: String) -> Option<(i32, i64)> {
let (ptr, _v, duration) = DMA_RECORD_STORE.lock().get(&name)?.clone();
Some((ptr as i32, duration))
}

605
src/runtime/src/rtio_mgt.rs
View File

@ -1,605 +0,0 @@
use alloc::{collections::BTreeMap, rc::Rc, string::String};
use core::cell::RefCell;
use io::{Cursor, ProtoRead};
use libboard_artiq::{drtio_routing, pl::csr};
use libboard_zynq::timer::GlobalTimer;
use libconfig::Config;
use libcortex_a9::mutex::Mutex;
use log::warn;
static mut RTIO_DEVICE_MAP: BTreeMap<u32, String> = BTreeMap::new();
#[cfg(has_drtio)]
pub mod drtio {
use alloc::vec::Vec;
use embedded_hal::blocking::delay::DelayMs;
use libasync::{delay, task};
use libboard_artiq::{drtioaux::Error, drtioaux_async, drtioaux_async::Packet, drtioaux_proto::DMA_TRACE_MAX_SIZE};
use libboard_zynq::time::Milliseconds;
use log::{error, info, warn};
use super::*;
use crate::{rtio_dma::remote_dma, ASYNC_ERROR_BUSY, ASYNC_ERROR_COLLISION, ASYNC_ERROR_SEQUENCE_ERROR,
SEEN_ASYNC_ERRORS};
pub fn startup(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) {
let aux_mutex = aux_mutex.clone();
let routing_table = routing_table.clone();
let up_destinations = up_destinations.clone();
task::spawn(async move {
let routing_table = routing_table.borrow();
link_task(&aux_mutex, &routing_table, &up_destinations, timer).await;
});
}
async fn link_rx_up(linkno: u8) -> bool {
let linkno = linkno as usize;
unsafe { (csr::DRTIO[linkno].rx_up_read)() == 1 }
}
async fn recv_aux_timeout(linkno: u8, timeout: u64, timer: GlobalTimer) -> Result<Packet, &'static str> {
if !link_rx_up(linkno).await {
return Err("link went down");
}
match drtioaux_async::recv_timeout(linkno, Some(timeout), timer).await {
Ok(packet) => return Ok(packet),
Err(Error::TimedOut) => return Err("timed out"),
Err(_) => return Err("aux packet error"),
}
}
pub async fn aux_transact(
aux_mutex: &Mutex<bool>,
linkno: u8,
request: &Packet,
timer: GlobalTimer,
) -> Result<Packet, &'static str> {
if !link_rx_up(linkno).await {
return Err("link went down");
}
let _lock = aux_mutex.async_lock().await;
drtioaux_async::send(linkno, request).await.unwrap();
loop {
let reply = recv_aux_timeout(linkno, 200, timer).await;
match reply {
Ok(Packet::DmaPlaybackStatus {
id,
destination,
error,
channel,
timestamp,
}) => {
remote_dma::playback_done(id, destination, error, channel, timestamp).await;
}
Ok(packet) => return Ok(packet),
Err(e) => return Err(e),
}
}
}
async fn drain_buffer(linkno: u8, draining_time: Milliseconds, timer: GlobalTimer) {
let max_time = timer.get_time() + draining_time;
loop {
if timer.get_time() > max_time {
return;
} //could this be cut short?
let _ = drtioaux_async::recv(linkno).await;
}
}
async fn ping_remote(aux_mutex: &Rc<Mutex<bool>>, linkno: u8, timer: GlobalTimer) -> u32 {
let mut count = 0;
loop {
if !link_rx_up(linkno).await {
return 0;
}
count += 1;
if count > 100 {
return 0;
}
let reply = aux_transact(aux_mutex, linkno, &Packet::EchoRequest, timer).await;
match reply {
Ok(Packet::EchoReply) => {
// make sure receive buffer is drained
let draining_time = Milliseconds(200);
drain_buffer(linkno, draining_time, timer).await;
return count;
}
_ => {}
}
}
}
async fn sync_tsc(aux_mutex: &Rc<Mutex<bool>>, linkno: u8, timer: GlobalTimer) -> Result<(), &'static str> {
let _lock = aux_mutex.async_lock().await;
unsafe {
(csr::DRTIO[linkno as usize].set_time_write)(1);
while (csr::DRTIO[linkno as usize].set_time_read)() == 1 {}
}
// TSCAck is the only aux packet that is sent spontaneously
// by the satellite, in response to a TSC set on the RT link.
let reply = recv_aux_timeout(linkno, 10000, timer).await?;
if reply == Packet::TSCAck {
return Ok(());
} else {
return Err("unexpected reply");
}
}
async fn load_routing_table(
aux_mutex: &Rc<Mutex<bool>>,
linkno: u8,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
) -> Result<(), &'static str> {
for i in 0..drtio_routing::DEST_COUNT {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::RoutingSetPath {
destination: i as u8,
hops: routing_table.0[i],
},
timer,
)
.await?;
if reply != Packet::RoutingAck {
return Err("unexpected reply");
}
}
Ok(())
}
async fn set_rank(
aux_mutex: &Rc<Mutex<bool>>,
linkno: u8,
rank: u8,
timer: GlobalTimer,
) -> Result<(), &'static str> {
let reply = aux_transact(aux_mutex, linkno, &Packet::RoutingSetRank { rank: rank }, timer).await?;
if reply != Packet::RoutingAck {
return Err("unexpected reply");
}
Ok(())
}
async fn init_buffer_space(destination: u8, linkno: u8) {
let linkno = linkno as usize;
unsafe {
(csr::DRTIO[linkno].destination_write)(destination);
(csr::DRTIO[linkno].force_destination_write)(1);
(csr::DRTIO[linkno].o_get_buffer_space_write)(1);
while (csr::DRTIO[linkno].o_wait_read)() == 1 {}
info!(
"[DEST#{}] buffer space is {}",
destination,
(csr::DRTIO[linkno].o_dbg_buffer_space_read)()
);
(csr::DRTIO[linkno].force_destination_write)(0);
}
}
async fn process_unsolicited_aux(aux_mutex: &Rc<Mutex<bool>>, linkno: u8) {
let _lock = aux_mutex.async_lock().await;
match drtioaux_async::recv(linkno).await {
Ok(Some(Packet::DmaPlaybackStatus {
id,
destination,
error,
channel,
timestamp,
})) => remote_dma::playback_done(id, destination, error, channel, timestamp).await,
Ok(Some(packet)) => warn!("[LINK#{}] unsolicited aux packet: {:?}", linkno, packet),
Ok(None) => (),
Err(_) => warn!("[LINK#{}] aux packet error", linkno),
}
}
async fn process_local_errors(linkno: u8) {
let errors;
let linkidx = linkno as usize;
unsafe {
errors = (csr::DRTIO[linkidx].protocol_error_read)();
(csr::DRTIO[linkidx].protocol_error_write)(errors);
}
if errors != 0 {
error!("[LINK#{}] error(s) found (0x{:02x}):", linkno, errors);
if errors & 1 != 0 {
error!("[LINK#{}] received packet of an unknown type", linkno);
}
if errors & 2 != 0 {
error!("[LINK#{}] received truncated packet", linkno);
}
if errors & 4 != 0 {
error!("[LINK#{}] timeout attempting to get remote buffer space", linkno);
}
}
}
async fn destination_set_up(
routing_table: &drtio_routing::RoutingTable,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
destination: u8,
up: bool,
) {
let mut up_destinations = up_destinations.borrow_mut();
up_destinations[destination as usize] = up;
if up {
drtio_routing::interconnect_enable(routing_table, 0, destination);
info!("[DEST#{}] destination is up", destination);
} else {
drtio_routing::interconnect_disable(destination);
info!("[DEST#{}] destination is down", destination);
}
}
async fn destination_up(up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>, destination: u8) -> bool {
let up_destinations = up_destinations.borrow();
up_destinations[destination as usize]
}
async fn destination_survey(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
up_links: &[bool],
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) {
for destination in 0..drtio_routing::DEST_COUNT {
let hop = routing_table.0[destination][0];
let destination = destination as u8;
if hop == 0 {
/* local RTIO */
if !destination_up(up_destinations, destination).await {
destination_set_up(routing_table, up_destinations, destination, true).await;
}
} else if hop as usize <= csr::DRTIO.len() {
let linkno = hop - 1;
if destination_up(up_destinations, destination).await {
if up_links[linkno as usize] {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DestinationStatusRequest {
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::DestinationDownReply) => {
destination_set_up(routing_table, up_destinations, destination, false).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, false)
.await;
}
Ok(Packet::DestinationOkReply) => (),
Ok(Packet::DestinationSequenceErrorReply { channel }) => {
error!(
"[DEST#{}] RTIO sequence error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_SEQUENCE_ERROR };
}
Ok(Packet::DestinationCollisionReply { channel }) => {
error!(
"[DEST#{}] RTIO collision involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_COLLISION };
}
Ok(Packet::DestinationBusyReply { channel }) => {
error!(
"[DEST#{}] RTIO busy error involving channel 0x{:04x}:{}",
destination,
channel,
resolve_channel_name(channel as u32)
);
unsafe { SEEN_ASYNC_ERRORS |= ASYNC_ERROR_BUSY };
}
Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
Err(e) => error!("[DEST#{}] communication failed ({})", destination, e),
}
} else {
destination_set_up(routing_table, up_destinations, destination, false).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, false).await;
}
} else {
if up_links[linkno as usize] {
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DestinationStatusRequest {
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::DestinationDownReply) => (),
Ok(Packet::DestinationOkReply) => {
destination_set_up(routing_table, up_destinations, destination, true).await;
init_buffer_space(destination as u8, linkno).await;
remote_dma::destination_changed(aux_mutex, routing_table, timer, destination, true)
.await;
}
Ok(packet) => error!("[DEST#{}] received unexpected aux packet: {:?}", destination, packet),
Err(e) => error!("[DEST#{}] communication failed ({})", destination, e),
}
}
}
}
}
}
pub async fn link_task(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
) {
let mut up_links = [false; csr::DRTIO.len()];
loop {
for linkno in 0..csr::DRTIO.len() {
let linkno = linkno as u8;
if up_links[linkno as usize] {
/* link was previously up */
if link_rx_up(linkno).await {
process_unsolicited_aux(aux_mutex, linkno).await;
process_local_errors(linkno).await;
} else {
info!("[LINK#{}] link is down", linkno);
up_links[linkno as usize] = false;
}
} else {
/* link was previously down */
if link_rx_up(linkno).await {
info!("[LINK#{}] link RX became up, pinging", linkno);
let ping_count = ping_remote(aux_mutex, linkno, timer).await;
if ping_count > 0 {
info!("[LINK#{}] remote replied after {} packets", linkno, ping_count);
up_links[linkno as usize] = true;
if let Err(e) = sync_tsc(aux_mutex, linkno, timer).await {
error!("[LINK#{}] failed to sync TSC ({})", linkno, e);
}
if let Err(e) = load_routing_table(aux_mutex, linkno, routing_table, timer).await {
error!("[LINK#{}] failed to load routing table ({})", linkno, e);
}
if let Err(e) = set_rank(aux_mutex, linkno, 1 as u8, timer).await {
error!("[LINK#{}] failed to set rank ({})", linkno, e);
}
info!("[LINK#{}] link initialization completed", linkno);
} else {
error!("[LINK#{}] ping failed", linkno);
}
}
}
}
destination_survey(aux_mutex, routing_table, &up_links, up_destinations, timer).await;
let mut countdown = timer.countdown();
delay(&mut countdown, Milliseconds(200)).await;
}
}
#[allow(dead_code)]
pub fn reset(aux_mutex: Rc<Mutex<bool>>, mut timer: GlobalTimer) {
for linkno in 0..csr::DRTIO.len() {
unsafe {
(csr::DRTIO[linkno].reset_write)(1);
}
}
timer.delay_ms(1);
for linkno in 0..csr::DRTIO.len() {
unsafe {
(csr::DRTIO[linkno].reset_write)(0);
}
}
for linkno in 0..csr::DRTIO.len() {
let linkno = linkno as u8;
if task::block_on(link_rx_up(linkno)) {
let reply = task::block_on(aux_transact(&aux_mutex, linkno, &Packet::ResetRequest, timer));
match reply {
Ok(Packet::ResetAck) => (),
Ok(_) => error!("[LINK#{}] reset failed, received unexpected aux packet", linkno),
Err(e) => error!("[LINK#{}] reset failed, aux packet error ({})", linkno, e),
}
}
}
}
pub async fn ddma_upload_trace(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
trace: &Vec<u8>,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let mut i = 0;
while i < trace.len() {
let mut trace_slice: [u8; DMA_TRACE_MAX_SIZE] = [0; DMA_TRACE_MAX_SIZE];
let len: usize = if i + DMA_TRACE_MAX_SIZE < trace.len() {
DMA_TRACE_MAX_SIZE
} else {
trace.len() - i
} as usize;
let last = i + len == trace.len();
trace_slice[..len].clone_from_slice(&trace[i..i + len]);
i += len;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DmaAddTraceRequest {
id: id,
destination: destination,
last: last,
length: len as u16,
trace: trace_slice,
},
timer,
)
.await;
match reply {
Ok(Packet::DmaAddTraceReply { succeeded: true }) => (),
Ok(Packet::DmaAddTraceReply { succeeded: false }) => {
return Err("error adding trace on satellite");
}
Ok(_) => {
return Err("adding DMA trace failed, unexpected aux packet");
}
Err(_) => {
return Err("adding DMA trace failed, aux error");
}
}
}
Ok(())
}
pub async fn ddma_send_erase(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DmaRemoveTraceRequest {
id: id,
destination: destination,
},
timer,
)
.await;
match reply {
Ok(Packet::DmaRemoveTraceReply { succeeded: true }) => Ok(()),
Ok(Packet::DmaRemoveTraceReply { succeeded: false }) => Err("satellite DMA erase error"),
Ok(_) => Err("adding trace failed, unexpected aux packet"),
Err(_) => Err("erasing trace failed, aux error"),
}
}
pub async fn ddma_send_playback(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &drtio_routing::RoutingTable,
timer: GlobalTimer,
id: u32,
destination: u8,
timestamp: u64,
) -> Result<(), &'static str> {
let linkno = routing_table.0[destination as usize][0] - 1;
let reply = aux_transact(
aux_mutex,
linkno,
&Packet::DmaPlaybackRequest {
id: id,
destination: destination,
timestamp: timestamp,
},
timer,
)
.await;
match reply {
Ok(Packet::DmaPlaybackReply { succeeded: true }) => Ok(()),
Ok(Packet::DmaPlaybackReply { succeeded: false }) => Err("error on DMA playback request"),
Ok(_) => Err("received unexpected aux packet during DMA playback"),
Err(_) => Err("aux error on DMA playback"),
}
}
}
fn read_device_map(cfg: &Config) -> BTreeMap<u32, String> {
let mut device_map: BTreeMap<u32, String> = BTreeMap::new();
let _ = cfg
.read("device_map")
.and_then(|raw_bytes| {
let mut bytes_cr = Cursor::new(raw_bytes);
let size = bytes_cr.read_u32().unwrap();
for _ in 0..size {
let channel = bytes_cr.read_u32().unwrap();
let device_name = bytes_cr.read_string().unwrap();
if let Some(old_entry) = device_map.insert(channel, device_name.clone()) {
warn!(
"conflicting device map entries for RTIO channel {}: '{}' and '{}'",
channel, old_entry, device_name
);
}
}
Ok(())
})
.or_else(|err| {
warn!(
"error reading device map ({}), device names will not be available in RTIO error messages",
err
);
Err(err)
});
device_map
}
fn _resolve_channel_name(channel: u32, device_map: &BTreeMap<u32, String>) -> String {
match device_map.get(&channel) {
Some(val) => val.clone(),
None => String::from("unknown"),
}
}
pub fn resolve_channel_name(channel: u32) -> String {
_resolve_channel_name(channel, unsafe { &RTIO_DEVICE_MAP })
}
#[cfg(not(has_drtio))]
pub mod drtio {
use super::*;
pub fn startup(
_aux_mutex: &Rc<Mutex<bool>>,
_routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
_up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
_timer: GlobalTimer,
) {
}
#[allow(dead_code)]
pub fn reset(_aux_mutex: Rc<Mutex<bool>>, mut _timer: GlobalTimer) {}
}
pub fn startup(
aux_mutex: &Rc<Mutex<bool>>,
routing_table: &Rc<RefCell<drtio_routing::RoutingTable>>,
up_destinations: &Rc<RefCell<[bool; drtio_routing::DEST_COUNT]>>,
timer: GlobalTimer,
cfg: &Config,
) {
unsafe {
RTIO_DEVICE_MAP = read_device_map(cfg);
}
drtio::startup(aux_mutex, routing_table, up_destinations, timer);
unsafe {
csr::rtio_core::reset_phy_write(1);
}
}
#[allow(dead_code)]
pub fn reset(aux_mutex: Rc<Mutex<bool>>, timer: GlobalTimer) {
unsafe {
csr::rtio_core::reset_write(1);
}
drtio::reset(aux_mutex, timer)
}

68
src/rustfmt.toml
View File

@ -1,68 +0,0 @@
max_width = 120
hard_tabs = false
tab_spaces = 4
newline_style = "Auto"
use_small_heuristics = "Default"
indent_style = "Block"
wrap_comments = false
format_code_in_doc_comments = false
comment_width = 100
normalize_comments = false
normalize_doc_attributes = false
license_template_path = ""
format_strings = true
format_macro_matchers = true
format_macro_bodies = true
empty_item_single_line = true
struct_lit_single_line = true
fn_single_line = false
where_single_line = true
imports_indent = "Visual"
imports_layout = "Mixed"
merge_imports = true
group_imports = "StdExternalCrate"
reorder_imports = true
reorder_modules = true
reorder_impl_items = false
type_punctuation_density = "Wide"
space_before_colon = false
space_after_colon = true
spaces_around_ranges = false
binop_separator = "Front"
remove_nested_parens = true
combine_control_expr = true
overflow_delimited_expr = false
struct_field_align_threshold = 0
enum_discrim_align_threshold = 0
match_arm_blocks = true
match_arm_leading_pipes = "Never"
force_multiline_blocks = false
fn_args_layout = "Tall"
brace_style = "SameLineWhere"
control_brace_style = "AlwaysSameLine"
trailing_semicolon = true
trailing_comma = "Vertical"
match_block_trailing_comma = false
blank_lines_upper_bound = 1
blank_lines_lower_bound = 0
edition = "2018"
version = "Two"
inline_attribute_width = 0
merge_derives = true
use_try_shorthand = false
use_field_init_shorthand = false
force_explicit_abi = true
condense_wildcard_suffixes = false
color = "Auto"
required_version = "1.4.32"
unstable_features = false
disable_all_formatting = false
skip_children = false
hide_parse_errors = false
error_on_line_overflow = false
error_on_unformatted = false
report_todo = "Never"
report_fixme = "Never"
ignore = []
emit_mode = "Files"
make_backup = false

28
src/satman/Cargo.toml
View File

@ -1,28 +0,0 @@
[package]
authors = ["M-Labs"]
name = "satman"
version = "0.0.0"
build = "build.rs"
[features]
target_zc706 = ["libboard_zynq/target_zc706", "libsupport_zynq/target_zc706", "libconfig/target_zc706", "libboard_artiq/target_zc706"]
target_kasli_soc = ["libboard_zynq/target_kasli_soc", "libsupport_zynq/target_kasli_soc", "libconfig/target_kasli_soc", "libboard_artiq/target_kasli_soc"]
default = ["target_zc706", ]
[build-dependencies]
build_zynq = { path = "../libbuild_zynq" }
[dependencies]
log = { version = "0.4", default-features = false }
embedded-hal = "0.2"
libboard_zynq = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["ipv6"]}
libsupport_zynq = { default-features = false, features = ["alloc_core"], git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libcortex_a9 = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libasync = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libregister = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git" }
libconfig = { git = "https://git.m-labs.hk/M-Labs/zynq-rs.git", features = ["fat_lfn", "ipv6"] }
libboard_artiq = { path = "../libboard_artiq" }
unwind = { path = "../libunwind" }
libc = { path = "../libc" }

6
src/satman/build.rs
View File

@ -1,6 +0,0 @@
extern crate build_zynq;
fn main() {
build_zynq::add_linker_script();
build_zynq::cfg();
}

173
src/satman/src/dma.rs
View File

@ -1,173 +0,0 @@
use alloc::{collections::btree_map::BTreeMap, vec::Vec};
use libboard_artiq::pl::csr;
use libcortex_a9::cache::dcci_slice;
const ALIGNMENT: usize = 64;
#[derive(Debug, PartialEq)]
enum ManagerState {
Idle,
Playback,
}
pub struct RtioStatus {
pub id: u32,
pub error: u8,
pub channel: u32,
pub timestamp: u64,
}
pub enum Error {
IdNotFound,
PlaybackInProgress,
EntryNotComplete,
}
#[derive(Debug)]
struct Entry {
trace: Vec<u8>,
padding_len: usize,
complete: bool,
}
#[derive(Debug)]
pub struct Manager {
entries: BTreeMap<u32, Entry>,
state: ManagerState,
currentid: u32,
}
impl Manager {
pub fn new() -> Manager {
// in case Manager is created during a DMA in progress
// wait for it to end
unsafe { while csr::rtio_dma::enable_read() != 0 {} }
Manager {
entries: BTreeMap::new(),
currentid: 0,
state: ManagerState::Idle,
}
}
pub fn add(&mut self, id: u32, last: bool, trace: &[u8], trace_len: usize) -> Result<(), Error> {
let entry = match self.entries.get_mut(&id) {
Some(entry) => {
if entry.complete {
// replace entry
self.entries.remove(&id);
self.entries.insert(
id,
Entry {
trace: Vec::new(),
padding_len: 0,
complete: false,
},
);
self.entries.get_mut(&id).unwrap()
} else {
entry
}
}
None => {
self.entries.insert(
id,
Entry {
trace: Vec::new(),
padding_len: 0,
complete: false,
},
);
self.entries.get_mut(&id).unwrap()
}
};
entry.trace.extend(&trace[0..trace_len]);
if last {
entry.trace.push(0);
let data_len = entry.trace.len();
// Realign.
entry.trace.reserve(ALIGNMENT - 1);
let padding = ALIGNMENT - entry.trace.as_ptr() as usize % ALIGNMENT;
let padding = if padding == ALIGNMENT { 0 } else { padding };
for _ in 0..padding {
// Vec guarantees that this will not reallocate
entry.trace.push(0)
}
for i in 1..data_len + 1 {
entry.trace[data_len + padding - i] = entry.trace[data_len - i]
}
entry.complete = true;
entry.padding_len = padding;
dcci_slice(&entry.trace);
}
Ok(())
}
pub fn erase(&mut self, id: u32) -> Result<(), Error> {
match self.entries.remove(&id) {
Some(_) => Ok(()),
None => Err(Error::IdNotFound),
}
}
pub fn playback(&mut self, id: u32, timestamp: u64) -> Result<(), Error> {
if self.state != ManagerState::Idle {
return Err(Error::PlaybackInProgress);
}
let entry = match self.entries.get(&id) {
Some(entry) => entry,
None => {
return Err(Error::IdNotFound);
}
};
if !entry.complete {
return Err(Error::EntryNotComplete);
}
let ptr = entry.trace[entry.padding_len..].as_ptr();
assert!(ptr as u32 % 64 == 0);
self.state = ManagerState::Playback;
self.currentid = id;
unsafe {
csr::rtio_dma::base_address_write(ptr as u32);
csr::rtio_dma::time_offset_write(timestamp as u64);
csr::cri_con::selected_write(1);
csr::rtio_dma::enable_write(1);
// playback has begun here, for status call check_state
}
Ok(())
}
pub fn check_state(&mut self) -> Option<RtioStatus> {
if self.state != ManagerState::Playback {
// nothing to report
return None;
}
let dma_enable = unsafe { csr::rtio_dma::enable_read() };
if dma_enable != 0 {
return None;
} else {
self.state = ManagerState::Idle;
unsafe {
csr::cri_con::selected_write(0);
let error = csr::rtio_dma::error_read();
let channel = csr::rtio_dma::error_channel_read();
let timestamp = csr::rtio_dma::error_timestamp_read();
if error != 0 {
csr::rtio_dma::error_write(1);
}
return Some(RtioStatus {
id: self.currentid,
error: error,
channel: channel,
timestamp: timestamp,
});
}
}
}
}

816
src/satman/src/main.rs
View File

@ -1,816 +0,0 @@
#![no_std]
#![no_main]
#![feature(never_type, panic_info_message, asm, naked_functions)]
#![feature(alloc_error_handler)]
#[macro_use]
extern crate log;
extern crate embedded_hal;
extern crate libboard_artiq;
extern crate libboard_zynq;
extern crate libcortex_a9;
extern crate libregister;
extern crate libsupport_zynq;
extern crate unwind;
extern crate alloc;
use core::sync::atomic::{AtomicBool, Ordering};
use dma::Manager as DmaManager;
use embedded_hal::blocking::delay::DelayUs;
#[cfg(feature = "target_kasli_soc")]
use libboard_artiq::io_expander;
#[cfg(has_si5324)]
use libboard_artiq::si5324;
use libboard_artiq::{drtio_routing, drtioaux, identifier_read, logger, pl::csr};
#[cfg(feature = "target_kasli_soc")]
use libboard_zynq::error_led::ErrorLED;
use libboard_zynq::{gic, i2c::I2c, mpcore, print, println, stdio, time::Milliseconds, timer::GlobalTimer};
use libcortex_a9::{asm, interrupt_handler,
l2c::enable_l2_cache,
notify_spin_lock,
regs::{MPIDR, SP},
spin_lock_yield};
use libregister::{RegisterR, RegisterW};
use libsupport_zynq::ram;
mod dma;
mod repeater;
fn drtiosat_reset(reset: bool) {
unsafe {
csr::drtiosat::reset_write(if reset { 1 } else { 0 });
}
}
fn drtiosat_reset_phy(reset: bool) {
unsafe {
csr::drtiosat::reset_phy_write(if reset { 1 } else { 0 });
}
}
fn drtiosat_link_rx_up() -> bool {
unsafe { csr::drtiosat::rx_up_read() == 1 }
}
fn drtiosat_tsc_loaded() -> bool {
unsafe {
let tsc_loaded = csr::drtiosat::tsc_loaded_read() == 1;
if tsc_loaded {
csr::drtiosat::tsc_loaded_write(1);
}
tsc_loaded
}
}
#[cfg(has_drtio_routing)]
macro_rules! forward {
($routing_table:expr, $destination:expr, $rank:expr, $repeaters:expr, $packet:expr, $timer:expr) => {{
let hop = $routing_table.0[$destination as usize][$rank as usize];
if hop != 0 {
let repno = (hop - 1) as usize;
if repno < $repeaters.len() {
return $repeaters[repno].aux_forward($packet, $timer);
} else {
return Err(drtioaux::Error::RoutingError);
}
}
}};
}
#[cfg(not(has_drtio_routing))]
macro_rules! forward {
($routing_table:expr, $destination:expr, $rank:expr, $repeaters:expr, $packet:expr, $timer:expr) => {};
}
fn process_aux_packet(
_repeaters: &mut [repeater::Repeater],
_routing_table: &mut drtio_routing::RoutingTable,
_rank: &mut u8,
packet: drtioaux::Packet,
timer: &mut GlobalTimer,
i2c: &mut I2c,
dma_manager: &mut DmaManager,
) -> Result<(), drtioaux::Error> {
// In the code below, *_chan_sel_write takes an u8 if there are fewer than 256 channels,
// and u16 otherwise; hence the `as _` conversion.
match packet {
drtioaux::Packet::EchoRequest => drtioaux::send(0, &drtioaux::Packet::EchoReply),
drtioaux::Packet::ResetRequest => {
info!("resetting RTIO");
drtiosat_reset(true);
timer.delay_us(100);
drtiosat_reset(false);
for rep in _repeaters.iter() {
if let Err(e) = rep.rtio_reset(timer) {
error!("failed to issue RTIO reset ({:?})", e);
}
}
drtioaux::send(0, &drtioaux::Packet::ResetAck)
}
drtioaux::Packet::DestinationStatusRequest {
destination: _destination,
} => {
#[cfg(has_drtio_routing)]
let hop = _routing_table.0[_destination as usize][*_rank as usize];
#[cfg(not(has_drtio_routing))]
let hop = 0;
if hop == 0 {
let errors;
unsafe {
errors = csr::drtiosat::rtio_error_read();
}
if errors & 1 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::sequence_error_channel_read();
csr::drtiosat::rtio_error_write(1);
}
drtioaux::send(0, &drtioaux::Packet::DestinationSequenceErrorReply { channel })?;
} else if errors & 2 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::collision_channel_read();
csr::drtiosat::rtio_error_write(2);
}
drtioaux::send(0, &drtioaux::Packet::DestinationCollisionReply { channel })?;
} else if errors & 4 != 0 {
let channel;
unsafe {
channel = csr::drtiosat::busy_channel_read();
csr::drtiosat::rtio_error_write(4);
}
drtioaux::send(0, &drtioaux::Packet::DestinationBusyReply { channel })?;
} else {
drtioaux::send(0, &drtioaux::Packet::DestinationOkReply)?;
}
}
#[cfg(has_drtio_routing)]
{
if hop != 0 {
let hop = hop as usize;
if hop <= csr::DRTIOREP.len() {
let repno = hop - 1;
match _repeaters[repno].aux_forward(
&drtioaux::Packet::DestinationStatusRequest {
destination: _destination,
},
timer,
) {
Ok(()) => (),
Err(drtioaux::Error::LinkDown) => {
drtioaux::send(0, &drtioaux::Packet::DestinationDownReply)?
}
Err(e) => {
drtioaux::send(0, &drtioaux::Packet::DestinationDownReply)?;
error!("aux error when handling destination status request: {:?}", e);
}
}
} else {
drtioaux::send(0, &drtioaux::Packet::DestinationDownReply)?;
}
}
}
Ok(())
}
#[cfg(has_drtio_routing)]
drtioaux::Packet::RoutingSetPath { destination, hops } => {
_routing_table.0[destination as usize] = hops;
for rep in _repeaters.iter() {
if let Err(e) = rep.set_path(destination, &hops, timer) {
error!("failed to set path ({:?})", e);
}
}
drtioaux::send(0, &drtioaux::Packet::RoutingAck)
}
#[cfg(has_drtio_routing)]
drtioaux::Packet::RoutingSetRank { rank } => {
*_rank = rank;
drtio_routing::interconnect_enable_all(_routing_table, rank);
let rep_rank = rank + 1;
for rep in _repeaters.iter() {
if let Err(e) = rep.set_rank(rep_rank, timer) {
error!("failed to set rank ({:?})", e);
}
}
info!("rank: {}", rank);
info!("routing table: {}", _routing_table);
drtioaux::send(0, &drtioaux::Packet::RoutingAck)
}
#[cfg(not(has_drtio_routing))]
drtioaux::Packet::RoutingSetPath {
destination: _,
hops: _,
} => drtioaux::send(0, &drtioaux::Packet::RoutingAck),
#[cfg(not(has_drtio_routing))]
drtioaux::Packet::RoutingSetRank { rank: _ } => drtioaux::send(0, &drtioaux::Packet::RoutingAck),
drtioaux::Packet::MonitorRequest {
destination: _destination,
channel: _channel,
probe: _probe,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let value;
#[cfg(has_rtio_moninj)]
unsafe {
csr::rtio_moninj::mon_chan_sel_write(channel as _);
csr::rtio_moninj::mon_probe_sel_write(probe);
csr::rtio_moninj::mon_value_update_write(1);
value = csr::rtio_moninj::mon_value_read() as u64;
}
#[cfg(not(has_rtio_moninj))]
{
value = 0;
}
let reply = drtioaux::Packet::MonitorReply { value: value };
drtioaux::send(0, &reply)
}
drtioaux::Packet::InjectionRequest {
destination: _destination,
channel: _channel,
overrd: _overrd,
value: _value,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
#[cfg(has_rtio_moninj)]
unsafe {
csr::rtio_moninj::inj_chan_sel_write(channel as _);
csr::rtio_moninj::inj_override_sel_write(overrd);
csr::rtio_moninj::inj_value_write(value);
}
Ok(())
}
drtioaux::Packet::InjectionStatusRequest {
destination: _destination,
channel: _channel,
overrd: _overrd,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let value;
#[cfg(has_rtio_moninj)]
unsafe {
csr::rtio_moninj::inj_chan_sel_write(channel as _);
csr::rtio_moninj::inj_override_sel_write(overrd);
value = csr::rtio_moninj::inj_value_read();
}
#[cfg(not(has_rtio_moninj))]
{
value = 0;
}
drtioaux::send(0, &drtioaux::Packet::InjectionStatusReply { value: value })
}
drtioaux::Packet::I2cStartRequest {
destination: _destination,
busno: _busno,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = i2c.start().is_ok();
drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
}
drtioaux::Packet::I2cRestartRequest {
destination: _destination,
busno: _busno,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = i2c.restart().is_ok();
drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
}
drtioaux::Packet::I2cStopRequest {
destination: _destination,
busno: _busno,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = i2c.stop().is_ok();
drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
}
drtioaux::Packet::I2cWriteRequest {
destination: _destination,
busno: _busno,
data,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
match i2c.write(data) {
Ok(ack) => drtioaux::send(
0,
&drtioaux::Packet::I2cWriteReply {
succeeded: true,
ack: ack,
},
),
Err(_) => drtioaux::send(
0,
&drtioaux::Packet::I2cWriteReply {
succeeded: false,
ack: false,
},
),
}
}
drtioaux::Packet::I2cReadRequest {
destination: _destination,
busno: _busno,
ack,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
match i2c.read(ack) {
Ok(data) => drtioaux::send(
0,
&drtioaux::Packet::I2cReadReply {
succeeded: true,
data: data,
},
),
Err(_) => drtioaux::send(
0,
&drtioaux::Packet::I2cReadReply {
succeeded: false,
data: 0xff,
},
),
}
}
drtioaux::Packet::I2cSwitchSelectRequest {
destination: _destination,
busno: _busno,
address,
mask,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let ch = match mask {
//decode from mainline, PCA9548-centric API
0x00 => None,
0x01 => Some(0),
0x02 => Some(1),
0x04 => Some(2),
0x08 => Some(3),
0x10 => Some(4),
0x20 => Some(5),
0x40 => Some(6),
0x80 => Some(7),
_ => return drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: false }),
};
let succeeded = i2c.pca954x_select(address, ch).is_ok();
drtioaux::send(0, &drtioaux::Packet::I2cBasicReply { succeeded: succeeded })
}
drtioaux::Packet::SpiSetConfigRequest {
destination: _destination,
busno: _busno,
flags: _flags,
length: _length,
div: _div,
cs: _cs,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
// todo: reimplement when/if SPI is available
//let succeeded = spi::set_config(busno, flags, length, div, cs).is_ok();
drtioaux::send(0, &drtioaux::Packet::SpiBasicReply { succeeded: false })
}
drtioaux::Packet::SpiWriteRequest {
destination: _destination,
busno: _busno,
data: _data,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
// todo: reimplement when/if SPI is available
//let succeeded = spi::write(busno, data).is_ok();
drtioaux::send(0, &drtioaux::Packet::SpiBasicReply { succeeded: false })
}
drtioaux::Packet::SpiReadRequest {
destination: _destination,
busno: _busno,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
// todo: reimplement when/if SPI is available
// match spi::read(busno) {
// Ok(data) => drtioaux::send(0,
// &drtioaux::Packet::SpiReadReply { succeeded: true, data: data }),
// Err(_) => drtioaux::send(0,
// &drtioaux::Packet::SpiReadReply { succeeded: false, data: 0 })
// }
drtioaux::send(
0,
&drtioaux::Packet::SpiReadReply {
succeeded: false,
data: 0,
},
)
}
drtioaux::Packet::DmaAddTraceRequest {
destination: _destination,
id,
last,
length,
trace,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = dma_manager.add(id, last, &trace, length as usize).is_ok();
drtioaux::send(0, &drtioaux::Packet::DmaAddTraceReply { succeeded: succeeded })
}
drtioaux::Packet::DmaRemoveTraceRequest {
destination: _destination,
id,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = dma_manager.erase(id).is_ok();
drtioaux::send(0, &drtioaux::Packet::DmaRemoveTraceReply { succeeded: succeeded })
}
drtioaux::Packet::DmaPlaybackRequest {
destination: _destination,
id,
timestamp,
} => {
forward!(_routing_table, _destination, *_rank, _repeaters, &packet, timer);
let succeeded = dma_manager.playback(id, timestamp).is_ok();
drtioaux::send(0, &drtioaux::Packet::DmaPlaybackReply { succeeded: succeeded })
}
_ => {
warn!("received unexpected aux packet");
Ok(())
}
}
}
fn process_aux_packets(
repeaters: &mut [repeater::Repeater],
routing_table: &mut drtio_routing::RoutingTable,
rank: &mut u8,
timer: &mut GlobalTimer,
i2c: &mut I2c,
dma_manager: &mut DmaManager,
) {
let result = drtioaux::recv(0).and_then(|packet| {
if let Some(packet) = packet {
process_aux_packet(repeaters, routing_table, rank, packet, timer, i2c, dma_manager)
} else {
Ok(())
}
});
match result {
Ok(()) => (),
Err(e) => warn!("aux packet error ({:?})", e),
}
}
fn drtiosat_process_errors() {
let errors;
unsafe {
errors = csr::drtiosat::protocol_error_read();
}
if errors & 1 != 0 {
error!("received packet of an unknown type");
}
if errors & 2 != 0 {
error!("received truncated packet");
}
if errors & 4 != 0 {
let destination;
unsafe {
destination = csr::drtiosat::buffer_space_timeout_dest_read();
}
error!(
"timeout attempting to get buffer space from CRI, destination=0x{:02x}",
destination
)
}
if errors & 8 != 0 {
let channel;
let timestamp_event;
let timestamp_counter;
unsafe {
channel = csr::drtiosat::underflow_channel_read();
timestamp_event = csr::drtiosat::underflow_timestamp_event_read() as i64;
timestamp_counter = csr::drtiosat::underflow_timestamp_counter_read() as i64;
}
error!(
"write underflow, channel={}, timestamp={}, counter={}, slack={}",
channel,
timestamp_event,
timestamp_counter,
timestamp_event - timestamp_counter
);
}
if errors & 16 != 0 {
error!("write overflow");
}
unsafe {
csr::drtiosat::protocol_error_write(errors);
}
}
fn hardware_tick(ts: &mut u64, timer: &mut GlobalTimer) {
let now = timer.get_time();
let mut ts_ms = Milliseconds(*ts);
if now > ts_ms {
ts_ms = now + Milliseconds(200);
*ts = ts_ms.0;
}
}
#[cfg(all(has_si5324, rtio_frequency = "125.0"))]
const SI5324_SETTINGS: si5324::FrequencySettings = si5324::FrequencySettings {
n1_hs: 5,
nc1_ls: 8,
n2_hs: 7,
n2_ls: 360,
n31: 63,
n32: 63,
bwsel: 4,
crystal_as_ckin2: true,
};
#[cfg(all(has_si5324, rtio_frequency = "100.0"))]
const SI5324_SETTINGS: si5324::FrequencySettings = si5324::FrequencySettings {
n1_hs: 5,
nc1_ls: 10,
n2_hs: 10,
n2_ls: 250,
n31: 50,
n32: 50,
bwsel: 4,
crystal_as_ckin2: true,
};
static mut LOG_BUFFER: [u8; 1 << 17] = [0; 1 << 17];
#[no_mangle]
pub extern "C" fn main_core0() -> i32 {
enable_l2_cache(0x8);
let mut timer = GlobalTimer::start();
let buffer_logger = unsafe { logger::BufferLogger::new(&mut LOG_BUFFER[..]) };
buffer_logger.set_uart_log_level(log::LevelFilter::Info);
buffer_logger.register();
log::set_max_level(log::LevelFilter::Info);
info!("ARTIQ satellite manager starting...");
info!("gateware ident {}", identifier_read(&mut [0; 64]));
ram::init_alloc_core0();
let mut i2c = I2c::i2c0();
i2c.init().expect("I2C initialization failed");
#[cfg(feature = "target_kasli_soc")]
{
for expander_i in 0..=1 {
let mut io_expander = io_expander::IoExpander::new(&mut i2c, expander_i).unwrap();
io_expander.init().expect("I2C I/O expander #0 initialization failed");
// Actively drive TX_DISABLE to false on SFP0..3
io_expander.set_oe(0, 1 << 1).unwrap();
io_expander.set_oe(1, 1 << 1).unwrap();
io_expander.set(0, 1, false);
io_expander.set(1, 1, false);
io_expander.service().unwrap();
}
}
#[cfg(has_si5324)]
si5324::setup(&mut i2c, &SI5324_SETTINGS, si5324::Input::Ckin1, &mut timer).expect("cannot initialize Si5324");
timer.delay_us(100_000);
info!("Switching SYS clocks...");
unsafe {
csr::drtio_transceiver::stable_clkin_write(1);
}
timer.delay_us(20_000); // wait for CPLL/QPLL/MMCM lock
let clk = unsafe { csr::sys_crg::current_clock_read() };
if clk == 1 {
info!("SYS CLK switched successfully");
} else {
panic!("SYS CLK did not switch");
}
unsafe {
csr::drtio_transceiver::txenable_write(0xffffffffu32 as _);
}
#[cfg(has_drtio_routing)]
let mut repeaters = [repeater::Repeater::default(); csr::DRTIOREP.len()];
#[cfg(not(has_drtio_routing))]
let mut repeaters = [repeater::Repeater::default(); 0];
for i in 0..repeaters.len() {
repeaters[i] = repeater::Repeater::new(i as u8);
}
let mut routing_table = drtio_routing::RoutingTable::default_empty();
let mut rank = 1;
let mut hardware_tick_ts = 0;
loop {
while !drtiosat_link_rx_up() {
drtiosat_process_errors();
#[allow(unused_mut)]
for mut rep in repeaters.iter_mut() {
rep.service(&routing_table, rank, &mut timer);
}
hardware_tick(&mut hardware_tick_ts, &mut timer);
}
info!("uplink is up, switching to recovered clock");
#[cfg(has_siphaser)]
{
si5324::siphaser::select_recovered_clock(&mut i2c, true, &mut timer).expect("failed to switch clocks");
si5324::siphaser::calibrate_skew(&mut timer).expect("failed to calibrate skew");
}
// DMA manager created here, so when link is dropped, all DMA traces
// are cleared out for a clean slate on subsequent connections,
// without a manual intervention.
let mut dma_manager = DmaManager::new();
drtioaux::reset(0);
drtiosat_reset(false);
drtiosat_reset_phy(false);
while drtiosat_link_rx_up() {
drtiosat_process_errors();
process_aux_packets(
&mut repeaters,
&mut routing_table,
&mut rank,
&mut timer,
&mut i2c,
&mut dma_manager,
);
#[allow(unused_mut)]
for mut rep in repeaters.iter_mut() {
rep.service(&routing_table, rank, &mut timer);
}
hardware_tick(&mut hardware_tick_ts, &mut timer);
if drtiosat_tsc_loaded() {
info!("TSC loaded from uplink");
for rep in repeaters.iter() {
if let Err(e) = rep.sync_tsc(&mut timer) {
error!("failed to sync TSC ({:?})", e);
}
}
if let Err(e) = drtioaux::send(0, &drtioaux::Packet::TSCAck) {
error!("aux packet error: {:?}", e);
}
}
if let Some(status) = dma_manager.check_state() {
info!(
"playback done, error: {}, channel: {}, timestamp: {}",
status.error, status.channel, status.timestamp
);
if let Err(e) = drtioaux::send(
0,
&drtioaux::Packet::DmaPlaybackStatus {
destination: rank,
id: status.id,
error: status.error,
channel: status.channel,
timestamp: status.timestamp,
},
) {
error!("error sending DMA playback status: {:?}", e);
}
}
}
drtiosat_reset_phy(true);
drtiosat_reset(true);
drtiosat_tsc_loaded();
info!("uplink is down, switching to local oscillator clock");
#[cfg(has_siphaser)]
si5324::siphaser::select_recovered_clock(&mut i2c, false, &mut timer).expect("failed to switch clocks");
}
}
extern "C" {
static mut __stack1_start: u32;
}
interrupt_handler!(IRQ, irq, __irq_stack0_start, __irq_stack1_start, {
if MPIDR.read().cpu_id() == 1 {
let mpcore = mpcore::RegisterBlock::mpcore();
let mut gic = gic::InterruptController::gic(mpcore);
let id = gic.get_interrupt_id();
if id.0 == 0 {
gic.end_interrupt(id);
asm::exit_irq();
SP.write(&mut __stack1_start as *mut _ as u32);
asm::enable_irq();
CORE1_RESTART.store(false, Ordering::Relaxed);
notify_spin_lock();
main_core1();
}
stdio::drop_uart();
}
loop {}
});
static mut PANICKED: [bool; 2] = [false; 2];
static CORE1_RESTART: AtomicBool = AtomicBool::new(false);
pub fn restart_core1() {
let mut interrupt_controller = gic::InterruptController::gic(mpcore::RegisterBlock::mpcore());
CORE1_RESTART.store(true, Ordering::Relaxed);
interrupt_controller.send_sgi(gic::InterruptId(0), gic::CPUCore::Core1.into());
while CORE1_RESTART.load(Ordering::Relaxed) {
spin_lock_yield();
}
}
#[no_mangle]
pub fn main_core1() {
let mut interrupt_controller = gic::InterruptController::gic(mpcore::RegisterBlock::mpcore());
interrupt_controller.enable_interrupts();
loop {}
}
#[no_mangle]
pub extern "C" fn exception(_vect: u32, _regs: *const u32, pc: u32, ea: u32) {
fn hexdump(addr: u32) {
let addr = (addr - addr % 4) as *const u32;
let mut ptr = addr;
println!("@ {:08p}", ptr);
for _ in 0..4 {
print!("+{:04x}: ", ptr as usize - addr as usize);
print!("{:08x} ", unsafe { *ptr });
ptr = ptr.wrapping_offset(1);
print!("{:08x} ", unsafe { *ptr });
ptr = ptr.wrapping_offset(1);
print!("{:08x} ", unsafe { *ptr });
ptr = ptr.wrapping_offset(1);
print!("{:08x}\n", unsafe { *ptr });
ptr = ptr.wrapping_offset(1);
}
}
hexdump(pc);
hexdump(ea);
panic!("exception at PC 0x{:x}, EA 0x{:x}", pc, ea)
}
#[no_mangle] // https://github.com/rust-lang/rust/issues/{38281,51647}
#[panic_handler]
pub fn panic_fmt(info: &core::panic::PanicInfo) -> ! {
let id = MPIDR.read().cpu_id() as usize;
print!("Core {} ", id);
unsafe {
if PANICKED[id] {
println!("nested panic!");
loop {}
}
PANICKED[id] = true;
}
print!("panic at ");
if let Some(location) = info.location() {
print!("{}:{}:{}", location.file(), location.line(), location.column());
} else {
print!("unknown location");
}
if let Some(message) = info.message() {
println!(": {}", message);
} else {
println!("");
}
#[cfg(feature = "target_kasli_soc")]
{
let mut err_led = ErrorLED::error_led();
err_led.toggle(true);
}
loop {}
}
// linker symbols
extern "C" {
static __text_start: u32;
static __text_end: u32;
static __exidx_start: u32;
static __exidx_end: u32;
}
#[no_mangle]
extern "C" fn dl_unwind_find_exidx(_pc: *const u32, len_ptr: *mut u32) -> *const u32 {
let length;
let start: *const u32;
unsafe {
length = (&__exidx_end as *const u32).offset_from(&__exidx_start) as u32;
start = &__exidx_start;
*len_ptr = length;
}
start
}

314
src/satman/src/repeater.rs
View File

@ -1,314 +0,0 @@
#[cfg(has_drtio_routing)]
use embedded_hal::prelude::_embedded_hal_blocking_delay_DelayUs;
#[cfg(has_drtio_routing)]
use libboard_artiq::pl::csr;
use libboard_artiq::{drtio_routing, drtioaux};
#[cfg(has_drtio_routing)]
use libboard_zynq::time::Milliseconds;
use libboard_zynq::timer::GlobalTimer;
#[cfg(has_drtio_routing)]
fn rep_link_rx_up(repno: u8) -> bool {
let repno = repno as usize;
unsafe { (csr::DRTIOREP[repno].rx_up_read)() == 1 }
}
#[cfg(has_drtio_routing)]
#[derive(Clone, Copy, PartialEq)]
enum RepeaterState {
Down,
SendPing { ping_count: u16 },
WaitPingReply { ping_count: u16, timeout: Milliseconds },
Up,
Failed,
}
#[cfg(has_drtio_routing)]
impl Default for RepeaterState {
fn default() -> RepeaterState {
RepeaterState::Down
}
}
#[cfg(has_drtio_routing)]
#[derive(Clone, Copy, Default)]
pub struct Repeater {
repno: u8,
auxno: u8,
state: RepeaterState,
}
#[cfg(has_drtio_routing)]
impl Repeater {
pub fn new(repno: u8) -> Repeater {
Repeater {
repno: repno,
auxno: repno + 1,
state: RepeaterState::Down,
}
}
#[allow(dead_code)]
pub fn is_up(&self) -> bool {
self.state == RepeaterState::Up
}
pub fn service(&mut self, routing_table: &drtio_routing::RoutingTable, rank: u8, timer: &mut GlobalTimer) {
self.process_local_errors();
match self.state {
RepeaterState::Down => {
if rep_link_rx_up(self.repno) {
info!("[REP#{}] link RX became up, pinging", self.repno);
self.state = RepeaterState::SendPing { ping_count: 0 };
}
}
RepeaterState::SendPing { ping_count } => {
if rep_link_rx_up(self.repno) {
drtioaux::send(self.auxno, &drtioaux::Packet::EchoRequest).unwrap();
self.state = RepeaterState::WaitPingReply {
ping_count: ping_count + 1,
timeout: timer.get_time() + Milliseconds(100),
}
} else {
error!("[REP#{}] link RX went down during ping", self.repno);
self.state = RepeaterState::Down;
}
}
RepeaterState::WaitPingReply { ping_count, timeout } => {
if rep_link_rx_up(self.repno) {
if let Ok(Some(drtioaux::Packet::EchoReply)) = drtioaux::recv(self.auxno) {
info!("[REP#{}] remote replied after {} packets", self.repno, ping_count);
self.state = RepeaterState::Up;
if let Err(e) = self.sync_tsc(timer) {
error!("[REP#{}] failed to sync TSC ({:?})", self.repno, e);
self.state = RepeaterState::Failed;
return;
}
if let Err(e) = self.load_routing_table(routing_table, timer) {
error!("[REP#{}] failed to load routing table ({:?})", self.repno, e);
self.state = RepeaterState::Failed;
return;
}
if let Err(e) = self.set_rank(rank + 1, timer) {
error!("[REP#{}] failed to set rank ({:?})", self.repno, e);
self.state = RepeaterState::Failed;
return;
}
} else {
if timer.get_time() > timeout {
if ping_count > 200 {
error!("[REP#{}] ping failed", self.repno);
self.state = RepeaterState::Failed;
} else {
self.state = RepeaterState::SendPing { ping_count: ping_count };
}
}
}
} else {
error!("[REP#{}] link RX went down during ping", self.repno);
self.state = RepeaterState::Down;
}
}
RepeaterState::Up => {
self.process_unsolicited_aux();
if !rep_link_rx_up(self.repno) {
info!("[REP#{}] link is down", self.repno);
self.state = RepeaterState::Down;
}
}
RepeaterState::Failed => {
if !rep_link_rx_up(self.repno) {
info!("[REP#{}] link is down", self.repno);
self.state = RepeaterState::Down;
}
}
}
}
fn process_unsolicited_aux(&self) {
match drtioaux::recv(self.auxno) {
Ok(Some(packet)) => warn!("[REP#{}] unsolicited aux packet: {:?}", self.repno, packet),
Ok(None) => (),
Err(_) => warn!("[REP#{}] aux packet error", self.repno),
}
}
fn process_local_errors(&self) {
let repno = self.repno as usize;
let errors;
unsafe {
errors = (csr::DRTIOREP[repno].protocol_error_read)();
}
if errors & 1 != 0 {
error!("[REP#{}] received packet of an unknown type", repno);
}
if errors & 2 != 0 {
error!("[REP#{}] received truncated packet", repno);
}
if errors & 4 != 0 {
let cmd;
let chan_sel;
unsafe {
cmd = (csr::DRTIOREP[repno].command_missed_cmd_read)();
chan_sel = (csr::DRTIOREP[repno].command_missed_chan_sel_read)();
}
error!(
"[REP#{}] CRI command missed, cmd={}, chan_sel=0x{:06x}",
repno, cmd, chan_sel
)
}
if errors & 8 != 0 {
let destination;
unsafe {
destination = (csr::DRTIOREP[repno].buffer_space_timeout_dest_read)();
}
error!(
"[REP#{}] timeout attempting to get remote buffer space, destination=0x{:02x}",
repno, destination
);
}
unsafe {
(csr::DRTIOREP[repno].protocol_error_write)(errors);
}
}
fn recv_aux_timeout(&self, timeout: u32, timer: &mut GlobalTimer) -> Result<drtioaux::Packet, drtioaux::Error> {
let max_time = timer.get_time() + Milliseconds(timeout.into());
loop {
if !rep_link_rx_up(self.repno) {
return Err(drtioaux::Error::LinkDown);
}
if timer.get_time() > max_time {
return Err(drtioaux::Error::TimedOut);
}
match drtioaux::recv(self.auxno) {
Ok(Some(packet)) => return Ok(packet),
Ok(None) => (),
Err(e) => return Err(e),
}
}
}
pub fn aux_forward(&self, request: &drtioaux::Packet, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up {
return Err(drtioaux::Error::LinkDown);
}
drtioaux::send(self.auxno, request).unwrap();
let reply = self.recv_aux_timeout(200, timer)?;
drtioaux::send(0, &reply).unwrap();
Ok(())
}
pub fn sync_tsc(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up {
return Ok(());
}
let repno = self.repno as usize;
unsafe {
(csr::DRTIOREP[repno].set_time_write)(1);
while (csr::DRTIOREP[repno].set_time_read)() == 1 {}
}
// TSCAck is the only aux packet that is sent spontaneously
// by the satellite, in response to a TSC set on the RT link.
let reply = self.recv_aux_timeout(10000, timer)?;
if reply == drtioaux::Packet::TSCAck {
return Ok(());
} else {
return Err(drtioaux::Error::UnexpectedReply);
}
}
pub fn set_path(
&self,
destination: u8,
hops: &[u8; drtio_routing::MAX_HOPS],
timer: &mut GlobalTimer,
) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up {
return Ok(());
}
drtioaux::send(
self.auxno,
&drtioaux::Packet::RoutingSetPath {
destination: destination,
hops: *hops,
},
)
.unwrap();
let reply = self.recv_aux_timeout(200, timer)?;
if reply != drtioaux::Packet::RoutingAck {
return Err(drtioaux::Error::UnexpectedReply);
}
Ok(())
}
pub fn load_routing_table(
&self,
routing_table: &drtio_routing::RoutingTable,
timer: &mut GlobalTimer,
) -> Result<(), drtioaux::Error> {
for i in 0..drtio_routing::DEST_COUNT {
self.set_path(i as u8, &routing_table.0[i], timer)?;
}
Ok(())
}
pub fn set_rank(&self, rank: u8, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
if self.state != RepeaterState::Up {
return Ok(());
}
drtioaux::send(self.auxno, &drtioaux::Packet::RoutingSetRank { rank: rank }).unwrap();
let reply = self.recv_aux_timeout(200, timer)?;
if reply != drtioaux::Packet::RoutingAck {
return Err(drtioaux::Error::UnexpectedReply);
}
Ok(())
}
pub fn rtio_reset(&self, timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
let repno = self.repno as usize;
unsafe {
(csr::DRTIOREP[repno].reset_write)(1);
}
timer.delay_us(100);
unsafe {
(csr::DRTIOREP[repno].reset_write)(0);
}
if self.state != RepeaterState::Up {
return Ok(());
}
drtioaux::send(self.auxno, &drtioaux::Packet::ResetRequest).unwrap();
let reply = self.recv_aux_timeout(200, timer)?;
if reply != drtioaux::Packet::ResetAck {
return Err(drtioaux::Error::UnexpectedReply);
}
Ok(())
}
}
#[cfg(not(has_drtio_routing))]
#[derive(Clone, Copy, Default)]
pub struct Repeater {}
#[cfg(not(has_drtio_routing))]
impl Repeater {
pub fn new(_repno: u8) -> Repeater {
Repeater::default()
}
pub fn service(&self, _routing_table: &drtio_routing::RoutingTable, _rank: u8, _timer: &mut GlobalTimer) {}
pub fn sync_tsc(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
Ok(())
}
pub fn rtio_reset(&self, _timer: &mut GlobalTimer) -> Result<(), drtioaux::Error> {
Ok(())
}
}

8
zynq-rs.nix Normal file
View File

@ -0,0 +1,8 @@
let
pkgs = import <nixpkgs> {};
in
pkgs.fetchgit {
url = "https://git.m-labs.hk/M-Labs/zynq-rs.git";
rev = "2c161720fa12f8b7abecaf60f77b062b08ac9bc1";
sha256 = "0zpazkicqzps86r7lgqf09y9ary94mjvxw6gc41z9kjjyxar5fhr";
}