M-Labs/nix-scripts
9
0
Fork 9
Code Issues 6 Pull Requests Releases Wiki Activity

artiq-full: split code generation into artiq-board-generated jobset #41

Merged
sb10q merged 1 commits from astro/nix-scripts:board-generated into master 2021-02-15 17:41:05 +08:00
Conversation 5 Commits 1 Files Changed 8 +386 -266
astro commented 2021-02-07 04:11:14 +08:00
Contributor
Copy Link

Please review this for a solution to #36. I've let it run through Hydra thoroughly, so no breakage expected.

Please review this for a solution to #36. I've let it run through Hydra thoroughly, so no breakage expected.
astro commented 2021-02-07 04:16:52 +08:00
Author
Contributor
Copy Link

I started wondering about that static list of Kasli variants. Is there a reason it can't be sourced from a directory listing of sinaraSystemsSrc, with a white/black list of variants to include/exclude in non-beta?

I started wondering about that static list of Kasli variants. Is there a reason it can't be sourced from a directory listing of `sinaraSystemsSrc`, with a white/black list of variants to include/exclude in non-beta?
sb10q commented 2021-02-07 09:37:23 +08:00
Owner
Copy Link

How would you define those "minimum ARTIQ version" requirements though?

How would you define those "minimum ARTIQ version" requirements though?
astro commented 2021-02-07 09:39:22 +08:00
Author
Contributor
Copy Link

How about putting the minimum ARTIQ version in the variant JSON itself?

How about putting the minimum ARTIQ version in the variant JSON itself?
sb10q commented 2021-02-07 09:40:10 +08:00
Owner
Copy Link

Yes, that can be done.

Yes, that can be done.
sb10q reviewed 2021-02-07 12:38:14 +08:00
artiq-full.nix Outdated
@ -26,3 +26,3 @@
cp -a ${<artiq-fast>} $out/fast
cp ${./artiq-full}/artiq-board.nix $out
cp -a ${<artiq-board-generated>} $out/board-generated
sb10q commented 2021-02-07 12:38:14 +08:00
Owner
Copy Link

Won't that break on ARTIQ-5?

Won't that break on ARTIQ-5?
sb10q commented 2021-02-07 12:57:25 +08:00
Owner
Copy Link

...and on -6 bloat the channel expressions with all the generated sources for all variants?

...and on -6 bloat the channel expressions with all the generated sources for all variants?
astro commented 2021-02-08 09:16:43 +08:00
Author
Contributor
Copy Link

For the non-beta case no sources are built but board-generated/artiq-targets.nix is still used because it contains the static variants list.

"artiq:board-generated-beta:generated-nix" is another generated-nix just like <artiq-fast> here. As there are no dependencies on the generated sources (yet), I'm still not sure if artiq-full evaluation will wait until all of the board-generated jobs have been tried.

For the non-beta case no sources are built but `board-generated/artiq-targets.nix` is still used because it contains the static variants list. `"artiq:board-generated-beta:generated-nix"` is another `generated-nix` just like `<artiq-fast>` here. As there are no dependencies on the generated sources (yet), I'm still not sure if artiq-full evaluation will wait until all of the board-generated jobs have been tried.
sb10q commented 2021-02-08 10:46:34 +08:00
Owner
Copy Link

Hmm, it's a bit heavy to add a new jobset for that on -5. Can we be more conservative here?

Hmm, it's a bit heavy to add a new jobset for that on -5. Can we be more conservative here?
astro force-pushed board-generated from c6cdf8d0d0 to a7d9711b54 2021-02-12 05:44:12 +08:00 Compare
sb10q reviewed 2021-02-12 13:20:41 +08:00
hydra/artiq.json Outdated
@ -33,2 +51,3 @@
"sinaraSystemsSrc": { "type": "git", "value": "https://git.m-labs.hk/M-Labs/sinara-systems.git master 1", "emailresponsible": false },
"artiq-fast": { "type": "sysbuild", "value": "artiq:fast-beta:generated-nix", "emailresponsible": false }
"artiq-fast": { "type": "sysbuild", "value": "artiq:fast-beta:generated-nix", "emailresponsible": false },
"artiq-board-generated": { "type": "sysbuild", "value": "artiq:board-generated-beta:generated-nix", "emailresponsible": false }
sb10q commented 2021-02-12 13:20:41 +08:00
Owner
Copy Link

Won't this produce some spurious builds with inconsistent artiq-fast and artiq-board-generated versions?

I think there should be a single ARTIQ input to full-beta, i.e. board-generated needs to include the copy of fast that it used, and then full reuses it, in order to prevent de-synchronization.

Won't this produce some spurious builds with inconsistent ``artiq-fast`` and ``artiq-board-generated`` versions? I think there should be a single ARTIQ input to ``full-beta``, i.e. ``board-generated`` needs to include the copy of ``fast`` that it used, and then ``full`` reuses it, in order to prevent de-synchronization.
astro commented 2021-02-13 03:00:01 +08:00
Author
Contributor
Copy Link

I was wondering about that. Thank you for spelling it out. That will also result in yet more divergent codepaths between non-beta and beta.

I was wondering about that. Thank you for spelling it out. That will also result in yet more divergent codepaths between non-beta and beta.
sb10q commented 2021-02-13 09:45:43 +08:00
Owner
Copy Link

If it diverges too much, just have two independent copies of the relevant Nix files. This will also help avoiding inadvertently breaking stable.

If it diverges too much, just have two independent copies of the relevant Nix files. This will also help avoiding inadvertently breaking stable.
sb10q commented 2021-02-13 09:49:22 +08:00
Owner
Copy Link

board-generated needs to include the copy of fast that it used, and then full reuses it

You can probably symlink and not copy.

> board-generated needs to include the copy of fast that it used, and then full reuses it You can probably symlink and not copy.
astro commented 2021-02-15 07:06:50 +08:00
Author
Contributor
Copy Link

Symlinking seems to break Nix restricted mode in which only /nix/store entries of known inputs are available to a build.

Symlinking seems to break Nix restricted mode in which only /nix/store entries of known inputs are available to a build.
sb10q reviewed 2021-02-12 13:21:45 +08:00
hydra/artiq.json Outdated
@ -31,7 +49,8 @@
"nixpkgs": { "type": "git", "value": "git://github.com/NixOS/nixpkgs nixos-20.09", "emailresponsible": false },
"nixScripts": { "type": "git", "value": "https://git.m-labs.hk/M-Labs/nix-scripts.git", "emailresponsible": false },
"sinaraSystemsSrc": { "type": "git", "value": "https://git.m-labs.hk/M-Labs/sinara-systems.git master 1", "emailresponsible": false },
sb10q commented 2021-02-12 13:21:45 +08:00
Owner
Copy Link

Same as below - this can go out of sync with board-generated.

Same as below - this can go out of sync with ``board-generated``.
astro force-pushed board-generated from a7d9711b54 to 278197cb5a 2021-02-15 07:59:01 +08:00 Compare
astro commented 2021-02-15 08:15:44 +08:00
Author
Contributor
Copy Link

Still running tests...

Still running tests...
astro force-pushed board-generated from 278197cb5a to 64ee51b673 2021-02-15 11:15:21 +08:00 Compare
sb10q merged commit 64ee51b673 into master 2021-02-15 17:41:05 +08:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-scripts#41
No description provided.
Delete Branch "astro/nix-scripts:board-generated"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?