avoid unnecessary gateware rebuilds #1

Closed
opened 2019-04-19 13:22:37 +08:00 by sb10q · 8 comments

With the Nix model it is relatively straightforward to reuse and cache artifacts between builds (in particular the bitstream, which is slow to compile):

  1. split the gateware version tagging into a separate step that modifies the .bit and takes little time. That probably needs some modifications to the identifier gateware core, plus something like ISE's Data2BRAM.
  2. have Nix manage fine-grained build steps (instead of just running the ARTIQ target script that does everything) and make Nix aware of the precise inputs of each step.
With the Nix model it is relatively straightforward to reuse and cache artifacts between builds (in particular the bitstream, which is slow to compile): 1. split the gateware version tagging into a separate step that modifies the .bit and takes little time. That probably needs some modifications to the identifier gateware core, plus something like ISE's Data2BRAM. 2. have Nix manage fine-grained build steps (instead of just running the ARTIQ target script that does everything) and make Nix aware of the precise inputs of each step.
sb10q referenced this issue from a commit 2019-12-25 14:39:33 +08:00
nixbld: lock Linux kernel version to 4.19.79 On newer kernel versions (somewhere before 4.19.89) the shitty iwlwifi driver would crash the machine every few days with a message like: Dec 25 12:22:25 nixbld kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000068 Dec 25 12:22:25 nixbld kernel: PGD 0 P4D 0 Dec 25 12:22:25 nixbld kernel: Oops: 0002 [#1] SMP PTI Dec 25 12:22:25 nixbld kernel: CPU: 2 PID: 15625 Comm: kworker/2:1 Not tainted 4.19.90 #1-NixOS Dec 25 12:22:25 nixbld kernel: Hardware name: EVGA INTERNATIONAL CO.,LTD Default string/121-KS-E375, BIOS 1.07 03/15/2018 Dec 25 12:22:25 nixbld kernel: Workqueue: events iwl_mvm_add_new_dqa_stream_wk [iwlmvm] Dec 25 12:22:25 nixbld kernel: RIP: 0010:iwl_trans_pcie_txq_enable+0x5b/0x460 [iwlwifi] Dec 25 12:22:25 nixbld kernel: Code: 63 c6 4c 8b ac c7 40 91 00 00 f0 48 0f ab 87 40 a1 00 00 73 0d 80 3d 6b 65 03 00 00 0f 84 cb 03 00 00 44 89 c7 e8 15 c7 14 ce <49> 89 45 68 4d 85 e4 0f 84 eb 02 00> Dec 25 12:22:25 nixbld kernel: RSP: 0018:ffffa47386937c30 EFLAGS: 00010202 Dec 25 12:22:25 nixbld kernel: RAX: 0000000000002710 RBX: 000000000000001f RCX: 0000000000000000 Dec 25 12:22:25 nixbld kernel: RDX: 3ffffffffffffffe RSI: 000000000000001f RDI: 0000000000002710 Dec 25 12:22:25 nixbld kernel: RBP: 0000000000000000 R08: 0000000000002710 R09: 0000000000000001 Dec 25 12:22:25 nixbld kernel: R10: 0000000000000004 R11: ffff916f0a199ff0 R12: 0000000000000000 Dec 25 12:22:25 nixbld kernel: R13: 0000000000000000 R14: 0000000000000000 R15: ffff916f08480018 Dec 25 12:22:25 nixbld kernel: FS: 0000000000000000(0000) GS:ffff916f36280000(0000) knlGS:0000000000000000 Dec 25 12:22:25 nixbld kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Dec 25 12:22:25 nixbld kernel: CR2: 0000000000000068 CR3: 0000000834e0a004 CR4: 00000000003606e0 Dec 25 12:22:25 nixbld kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Dec 25 12:22:25 nixbld kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Dec 25 12:22:25 nixbld kernel: Call Trace: Dec 25 12:22:25 nixbld kernel: iwl_mvm_enable_txq+0x230/0x3b0 [iwlmvm] Dec 25 12:22:25 nixbld kernel: ? iwl_mvm_add_new_dqa_stream_wk+0x897/0x13b0 [iwlmvm] Dec 25 12:22:25 nixbld kernel: iwl_mvm_add_new_dqa_stream_wk+0x897/0x13b0 [iwlmvm] Dec 25 12:22:25 nixbld kernel: ? entry_SYSCALL_64_stage2+0xf/0x10 Dec 25 12:22:25 nixbld kernel: ? __switch_to_asm+0x41/0x70 Dec 25 12:22:25 nixbld kernel: ? __switch_to_asm+0x41/0x70 Dec 25 12:22:25 nixbld kernel: ? __switch_to_asm+0x41/0x70 Dec 25 12:22:25 nixbld kernel: ? __switch_to+0x8c/0x440 Dec 25 12:22:25 nixbld kernel: ? __switch_to_asm+0x41/0x70 Dec 25 12:22:25 nixbld kernel: ? __switch_to_asm+0x35/0x70 Dec 25 12:22:25 nixbld kernel: process_one_work+0x206/0x400 Dec 25 12:22:25 nixbld kernel: worker_thread+0x2d/0x3e0 Dec 25 12:22:25 nixbld kernel: ? process_one_work+0x400/0x400 Dec 25 12:22:25 nixbld kernel: kthread+0x112/0x130 Dec 25 12:22:25 nixbld kernel: ? kthread_bind+0x30/0x30 Dec 25 12:22:25 nixbld kernel: ret_from_fork+0x35/0x40

I've found a way to accomplish this in the Nix part of the build. I'm going to prepare a Merge Request for review once I'm done testing (these runs take long).

I have yet to find the version info in the bitstream sources.

I've found a way to accomplish this in the Nix part of the build. I'm going to prepare a Merge Request for review once I'm done testing (these runs take long). I have yet to find the version info in the bitstream sources.
Poster
Owner

The version info is in the "identifier" core from MiSoC.
You will have to change it to use BRAM since I suspect that the current code that may produce combinatorial logic with LUTs will be difficult to edit post-compilation.

The version info is in the "identifier" core from MiSoC. You will have to change it to use BRAM since I suspect that the current code that may produce combinatorial logic with LUTs will be difficult to edit post-compilation.
Poster
Owner

Since 9e66dd7075 the identifier can be rewritten without rebuilding the whole gateware.

Run vivado -mode tcl and then:

open_checkpoint top_route.dcp
set_property INIT_38 256'hFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF43424103 [get_cells identifier_str]
write_bitstream -force top.bit

The new identifier string appears in the log:

 __  __ _ ____         ____ 
|  \/  (_) ___|  ___  / ___|
| |\/| | \___ \ / _ \| |    
| |  | | |___) | (_) | |___ 
|_|  |_|_|____/ \___/ \____|

MiSoC Bootloader
Copyright (c) 2017-2020 M-Labs Limited

Bootloader CRC passed
Gateware ident ABC
Initializing SDRAM...

....

[     0.000009s]  INFO(runtime): ARTIQ runtime starting...
[     0.003934s]  INFO(runtime): software ident 5.0.dev+852.g380de177.dirty;tester
[     0.011170s]  INFO(runtime): gateware ident ABC
Since https://github.com/m-labs/artiq/commit/9e66dd70757a7fe9881a4bdd837fa382d59bdabf the identifier can be rewritten without rebuilding the whole gateware. Run ``vivado -mode tcl`` and then: ```text open_checkpoint top_route.dcp set_property INIT_38 256'hFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF43424103 [get_cells identifier_str] write_bitstream -force top.bit ``` The new identifier string appears in the log: ``` __ __ _ ____ ____ | \/ (_) ___| ___ / ___| | |\/| | \___ \ / _ \| | | | | | |___) | (_) | |___ |_| |_|_|____/ \___/ \____| MiSoC Bootloader Copyright (c) 2017-2020 M-Labs Limited Bootloader CRC passed Gateware ident ABC Initializing SDRAM... .... [ 0.000009s] INFO(runtime): ARTIQ runtime starting... [ 0.003934s] INFO(runtime): software ident 5.0.dev+852.g380de177.dirty;tester [ 0.011170s] INFO(runtime): gateware ident ABC ```
Poster
Owner

NB: I have not tried using more optimal primitives (e.g. ROM256X1).

NB: I have not tried using more optimal primitives (e.g. ROM256X1).
Poster
Owner

Ok, it also works with ROM256X1 :)

Ok, it also works with ROM256X1 :)

Are these TCL commands still supposed to work?

get_cells identifier_str
WARNING: [Vivado 12-180] No cells matched 'identifier_str'.

This returns some cells but none looks like having to do with the ident:

get_cells -hierarchical * -filter {INIT_38=~*}
Are these TCL commands still supposed to work? ```tcl get_cells identifier_str WARNING: [Vivado 12-180] No cells matched 'identifier_str'. ``` This returns some cells but none looks like having to do with the ident: ```tcl get_cells -hierarchical * -filter {INIT_38=~*} ```
Poster
Owner

See the next commit 4a8d361ace.
The memory is now made up of eight ROM256X1 called identifier_str0 ... identifier_str7.

See the next commit https://github.com/m-labs/artiq/commit/4a8d361acebf1e6b45e9fe35201315798b02d2f3. The memory is now made up of eight ROM256X1 called ```identifier_str0``` ... ```identifier_str7```.

Ok, in my case they were optimized away with .INIT(32'd0).

Using 32'hAAAAAAAA keeps that from happening.

Ok, in my case they were optimized away with `.INIT(32'd0)`. Using `32'hAAAAAAAA` keeps that from happening.
sb10q closed this issue 2020-12-17 21:39:47 +08:00
Sign in to join this conversation.
No Label
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-scripts#1
There is no content yet.