M-Labs
/
artiq-zynq
10
1
Fork 16
Code Issues 14 Pull Requests 2 Releases Wiki Activity

Feature Request: EdgeCounters for NIST QC2 gateware #205

New Issue
Closed
opened 2022-11-03 10:00:49 +08:00 by ljstephenson · 11 comments
ljstephenson commented 2022-11-03 10:00:49 +08:00

Several end users at NIST using the QC2 gateware on zc706 have requested EdgeCounters for some TTLs.

I tried to add this to the gateware target myself, but the build failed for reasons that weren't clear to me and I didn't have time to investigate further.

I think a reasonable solution is to add 4 EdgeCounter RTIO channels at the end of the RTIO channel list (to maintain device_db back compatibility as much as possible), corresponding to the first 4 TTLInOut channels.

Four channels should be plenty - most experiments would only use one EdgeCounter, for counting a photomultiplying tube.

Several end users at NIST using the QC2 gateware on zc706 have requested EdgeCounters for some TTLs. I tried to add this to the gateware target myself, but the build failed for reasons that weren't clear to me and I didn't have time to investigate further. I think a reasonable solution is to add 4 EdgeCounter RTIO channels at the end of the RTIO channel list (to maintain device_db back compatibility as much as possible), corresponding to the first 4 TTLInOut channels. Four channels should be plenty - most experiments would only use one EdgeCounter, for counting a photomultiplying tube.
mwojcik commented 2022-11-03 10:38:16 +08:00
Owner

I made the changes, and it does compile, but I'd like you to test them out first before they're merged into the master branch here.

https://git.m-labs.hk/mwojcik/artiq-zynq/src/branch/qc2_edge_counters

I made the changes, and it does compile, but I'd like you to test them out first before they're merged into the master branch here. https://git.m-labs.hk/mwojcik/artiq-zynq/src/branch/qc2_edge_counters
ljstephenson commented 2022-11-03 11:15:30 +08:00
Poster

Build failed, but it seems like it may be something weird with my system. Freshly cloned from your fork, I get:

$ nix build .#zc706-nist_qc2_master-sd
error: builder for '/nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv' failed with exit code 1;
       last 10 log lines:
       >   File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen_axi/integration/soc_core.py", line 168, in build
       >     self.platform.build(self, *args, **kwargs)
       >   File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/platform.py", line 59, in build
       >     return self.toolchain.build(self, *args, **kwargs)
       >   File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/vivado.py", line 224, in build
       >     _run_vivado(build_name)
       >   File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/vivado.py", line 69, in _run_vivado
       >     raise OSError("Subprocess failed")
       > OSError: Subprocess failed
       >
       For full logs, run 'nix log /nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv'.
error: 1 dependencies of derivation '/nix/store/rwpwhw400wskiafyb2pwx5my5dvrkvaj-zc706-nist_qc2_master-sd.drv' failed to build

The detailed log seems to indicate something wrong with Vivado:

$ nix log /nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv
/etc/profile: line 19: /opt/Xilinx/Vivado/2021.2/settings64.sh: No such file or directory
Traceback (most recent call last):
  File "/nix/store/fxnbn9dhk1yxc8lw4a5lhf6cnigfi8lp-gateware/zc706.py", line 721, in <module>
    main()
  File "/nix/store/fxnbn9dhk1yxc8lw4a5lhf6cnigfi8lp-gateware/zc706.py", line 718, in main
    soc.build(build_dir=args.g)
  File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen_axi/integratio>
    self.platform.build(self, *args, **kwargs)
  File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/p>
    return self.toolchain.build(self, *args, **kwargs)
  File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/v>
    _run_vivado(build_name)
  File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/v>
    raise OSError("Subprocess failed")
OSError: Subprocess failed

However, the file clearly exists:

$ ls /opt/Xilinx/Vivado/2021.2/
bin   doc       fonts  ids_lite  lib    platforms         scripts         settings64.sh  tps
data  examples  gnu    include   lnx64  reportstrategies  settings64.csh  strategies

On my system I have /opt/Xilinx symlinked to the actual install location. Could that be causing issues? I guess I can change this pretty easily in the build script to point to the actual Vivado location.

EDIT: I just verified that this is the same error I got when I tried to do this myself, and that my diff more or less matches yours. At least I knew what to do in pronciple, but this error is mysterious.

Build failed, but it seems like it may be something weird with my system. Freshly cloned from your fork, I get: ``` $ nix build .#zc706-nist_qc2_master-sd error: builder for '/nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv' failed with exit code 1; last 10 log lines: > File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen_axi/integration/soc_core.py", line 168, in build > self.platform.build(self, *args, **kwargs) > File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/platform.py", line 59, in build > return self.toolchain.build(self, *args, **kwargs) > File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/vivado.py", line 224, in build > _run_vivado(build_name) > File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/vivado.py", line 69, in _run_vivado > raise OSError("Subprocess failed") > OSError: Subprocess failed > For full logs, run 'nix log /nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv'. error: 1 dependencies of derivation '/nix/store/rwpwhw400wskiafyb2pwx5my5dvrkvaj-zc706-nist_qc2_master-sd.drv' failed to build ``` The detailed log seems to indicate something wrong with Vivado: ``` $ nix log /nix/store/aqr0b63kyxirnjkzy3mf935hwis5j3a2-zc706-nist_qc2_master-gateware.drv /etc/profile: line 19: /opt/Xilinx/Vivado/2021.2/settings64.sh: No such file or directory Traceback (most recent call last): File "/nix/store/fxnbn9dhk1yxc8lw4a5lhf6cnigfi8lp-gateware/zc706.py", line 721, in <module> main() File "/nix/store/fxnbn9dhk1yxc8lw4a5lhf6cnigfi8lp-gateware/zc706.py", line 718, in main soc.build(build_dir=args.g) File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen_axi/integratio> self.platform.build(self, *args, **kwargs) File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/p> return self.toolchain.build(self, *args, **kwargs) File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/v> _run_vivado(build_name) File "/nix/store/qmxw40jm72xdfk0snc9vbx7wv9gpn4cs-python3-3.9.13-env/lib/python3.9/site-packages/migen/build/xilinx/v> raise OSError("Subprocess failed") OSError: Subprocess failed ``` However, the file clearly exists: ``` $ ls /opt/Xilinx/Vivado/2021.2/ bin doc fonts ids_lite lib platforms scripts settings64.sh tps data examples gnu include lnx64 reportstrategies settings64.csh strategies ``` On my system I have `/opt/Xilinx` symlinked to the actual install location. Could that be causing issues? I guess I can change this pretty easily in the build script to point to the actual Vivado location. EDIT: I just verified that this is the same error I got when I tried to do this myself, and that my diff more or less matches yours. At least I knew what to do in pronciple, but this error is mysterious.
ljstephenson commented 2022-11-03 11:35:10 +08:00
Poster

That said, building with the same command from artiq-zynq/master works fine, so my Vivado installation isn't completely broken.

Also my nix-fu isn't completely up to scratch: I see that the vivado location actually comes from flake.nix in the artiq repository. I'm not really sure how to add an entry to the local flake.nix to override this.

That said, building with the same command from artiq-zynq/master works fine, so my Vivado installation isn't completely broken. Also my nix-fu isn't completely up to scratch: I see that the vivado location actually comes from `flake.nix` in the `artiq` repository. I'm not really sure how to add an entry to the local `flake.nix` to override this.
mwojcik commented 2022-11-03 11:38:05 +08:00
Owner

I'm not really sure how to add an entry to the local flake.nix to override this.

I would create a Vivado entry similar to the one in artiq, but with changed location, as a variable in the local flake.nix - and then replace artiqpkgs.vivado (line 172) with your own package.

> I'm not really sure how to add an entry to the local flake.nix to override this. I would create a Vivado entry similar to the one in ``artiq``, but with changed location, as a variable in the local ``flake.nix`` - and then replace ``artiqpkgs.vivado`` (line 172) with your own package.
sb10q commented 2022-11-03 12:52:45 +08:00
Owner

On my system I have /opt/Xilinx symlinked to the actual install location. Could that be causing issues?

Yes - nix builds derivations in a sandbox where access to the filesystem is blocked except for dependencies and sandbox paths.

> On my system I have /opt/Xilinx symlinked to the actual install location. Could that be causing issues? Yes - nix builds derivations in a sandbox where access to the filesystem is blocked except for dependencies and sandbox paths.
ljstephenson commented 2022-11-04 00:46:20 +08:00
Poster

Still failing with same error after I exchange /opt/ for my path to Xilinx. See attached diff for the changes I made (dummy path in the diff, but you get the picture). The real path is in /home/<user>/Xilinx

Note as I mentioned that the symlinked structure worked fine when I was building from the artiq-zynq repository master branch.

Regardless, this seems to be an issue with my setup and is unrelated to the issue at hand - @mwojcik if you attach the boot.bin binary I can get someone here at NIST to test the gateware.

Still failing with same error after I exchange /opt/ for my path to Xilinx. See attached diff for the changes I made (dummy path in the diff, but you get the picture). The real path is in /home/\<user\>/Xilinx Note as I mentioned that the symlinked structure worked fine when I was building from the `artiq-zynq` repository master branch. Regardless, this seems to be an issue with my setup and is unrelated to the issue at hand - @mwojcik if you attach the boot.bin binary I can get someone here at NIST to test the gateware.
flake.diff
1.0 KiB
mwojcik commented 2022-11-04 10:08:27 +08:00
Owner

@ljstephenson - Try the attached compiled nist_qc2_master - let me know if you need any other variant.

@ljstephenson - Try the attached compiled nist_qc2_master - let me know if you need any other variant.
mwojcik commented 2022-11-04 16:58:29 +08:00
Owner

Sorry! I had this ready - but .bin files? or over 5mb files? are not allowed, giving no error - and I haven't noticed that it did not attach. Here's the zip.

Sorry! I had this ready - but .bin files? or over 5mb files? are not allowed, giving no error - and I haven't noticed that it did not attach. Here's the zip.
boot.zip
1.3 MiB
ljstephenson commented 2022-11-30 02:40:20 +08:00
Poster

One of my colleagues has been trying to test the gateware and ran into the following problem; calling fetch_count on the counter hangs as if there was no input to fetch, despite an earlier call to gate_rising.

from artiq.experiment import *


class EdgeCounterTest(EnvExperiment):
     def build(self):
          self.setattr_device('core')
          self.setattr_device('ttl0_counter')

     @kernel
     def run(self):
          self.core.reset()
          delay(10 * ms)

          self.ttl0_counter.gate_rising(100 * ms)
          delay(10 * ms)

          print("Counting rising edges")
          num_rising_edges = self.ttl0_counter.fetch_count() # hangs here

          print("Counts: ", num_rising_edges)

This is with master running software version 7.8123.3038639, on W10. Other experiments ran normally.

One of my colleagues has been trying to test the gateware and ran into the following problem; calling `fetch_count` on the counter hangs as if there was no input to fetch, despite an earlier call to `gate_rising`. ``` from artiq.experiment import * class EdgeCounterTest(EnvExperiment): def build(self): self.setattr_device('core') self.setattr_device('ttl0_counter') @kernel def run(self): self.core.reset() delay(10 * ms) self.ttl0_counter.gate_rising(100 * ms) delay(10 * ms) print("Counting rising edges") num_rising_edges = self.ttl0_counter.fetch_count() # hangs here print("Counts: ", num_rising_edges) ``` This is with master running software version 7.8123.3038639, on W10. Other experiments ran normally.
chinwen commented 2023-02-08 12:40:28 +08:00

I would like to follow up on this issue. Please let me know if we can provide more information to help resolve it.

I would like to follow up on this issue. Please let me know if we can provide more information to help resolve it.
mwojcik commented 2023-04-03 12:30:43 +08:00
Owner

That was my bad, off-by-one error in RTIO channel list in the device_db. Previously ttl1_counter would've yielded results from TTL0.

I tried that experiment code from @ljstephenson after fixing that problem and it doesn't hang anymore. After the pull request is merged, it should be available for you to easily download with AFWS, just remember to use the updated device_db.

That was my bad, off-by-one error in RTIO channel list in the ``device_db``. Previously ``ttl1_counter`` would've yielded results from TTL0. I tried that experiment code from @ljstephenson after fixing that problem and it doesn't hang anymore. After the pull request is merged, it should be available for you to easily download with AFWS, just remember to use the updated device_db.
sb10q closed this issue 2023-04-03 12:31:24 +08:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
master
release-7
release-6
Labels
Clear labels   
priority:critical
   
priority:high
   
priority:low
   
priority:medium
No Label
priority:critical
priority:high
priority:low
priority:medium
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/artiq-zynq#205
There is no content yet.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?