M-Labs/nix-scripts
9
0
Fork 9
Code Issues 6 Pull Requests Releases Wiki Activity

artiq-full evaluation is a memory hog #25

New Issue
Closed
opened 2020-06-10 21:58:54 +08:00 by sb10q · 5 comments
sb10q commented 2020-06-10 21:58:54 +08:00
Owner
Copy Link

Slow and makes hydra-eval-job use more than 20GB and sometimes crash with OOM.

Slow and makes hydra-eval-job use more than 20GB and sometimes crash with OOM.
astro commented 2020-06-11 01:45:19 +08:00
Contributor
Copy Link

Does 8f7308b improve the situation?

A migen patch may not be the most durable. I'm open for suggestions.

Does 8f7308b improve the situation? A migen patch may not be the most durable. I'm open for suggestions.
sb10q commented 2020-06-11 08:16:46 +08:00
Author
Owner
Copy Link

It's the evaluation (i.e. hydra-eval-job process, as I mentioned) that is the problem. Not the Vivado runs, where you actually want the little parallelism it supports to reduce build time.

It's the *evaluation* (i.e. hydra-eval-job process, as I mentioned) that is the problem. Not the Vivado runs, where you actually want the little parallelism it supports to reduce build time.
astro commented 2020-06-18 05:21:32 +08:00
Contributor
Copy Link

This improves memory usage and duration a lot for me: f1325dce29

This improves memory usage and duration a lot for me: f1325dce2941dbc127654e13b8acffdc151e5ffa
astro closed this issue 2020-06-18 05:21:56 +08:00
sb10q commented 2020-06-18 12:20:38 +08:00
Author
Owner
Copy Link

Thanks, also much better on nixbld.m-labs.hk.
I'm curious what was happening - lots of copies of the entire nixpkgs object?

Thanks, also much better on nixbld.m-labs.hk. I'm curious what was happening - lots of copies of the entire nixpkgs object?
astro commented 2020-06-18 17:29:57 +08:00
Contributor
Copy Link

For each of the board jobs, it was loading and evaluating the Vivado, Artiq and associated declarations over and over. I think it's putting import statements inside much-called functions (artiq-board) even when none of the function parameters are actually passed along.

For each of the board jobs, it was loading and evaluating the Vivado, Artiq and associated declarations over and over. I think it's putting `import` statements inside much-called functions (`artiq-board`) even when none of the function parameters are actually passed along.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-scripts#25
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?