M-Labs
/
nac3
8
0
Fork 5
Code Issues 87 Pull Requests 3 Releases Wiki Activity

bloat #17

New Issue
Open
opened 2021-09-24 09:58:37 +08:00 by sb10q · 10 comments
sb10q commented 2021-09-24 09:58:37 +08:00
Owner

The binary size is surprisingly large:

132M ../target/release/libnac3embedded.so

(Not a high priority issue)

The binary size is surprisingly large: ``` 132M ../target/release/libnac3embedded.so ``` (Not a high priority issue)
sb10q added the
low-priority
 label 2021-09-24 12:48:24 +08:00
sb10q commented 2021-10-03 10:07:17 +08:00
Poster
Owner

LLVM is statically linked and dynamic linking isn't supported by llvm-sys. This PR adds it:
https://gitlab.com/taricorp/llvm-sys.rs/-/merge_requests/4

LLVM is statically linked and dynamic linking isn't supported by llvm-sys. This PR adds it: https://gitlab.com/taricorp/llvm-sys.rs/-/merge_requests/4
sb10q commented 2021-10-03 11:13:17 +08:00
Poster
Owner

@ychenfo please work with llvm-sys upstream to get this PR edited as per review comments and merged. we want our own LLVM build and link it statically

<s>@ychenfo please work with llvm-sys upstream to get this PR edited as per review comments and merged.</s> we want our own LLVM build and link it statically
ychenfo was assigned by sb10q 2021-10-03 11:13:22 +08:00
sb10q commented 2021-11-05 18:31:28 +08:00
Poster
Owner

May also be worth initializing only the targets we use instead of Target::initialize_all. Could reduce startup time and/or memory consumption (TBD).
Alternatively just build LLVM with only the used targets.

May also be worth initializing only the targets we use instead of ``Target::initialize_all``. Could reduce startup time and/or memory consumption (TBD). Alternatively just build LLVM with only the used targets.
sb10q commented 2021-11-07 12:49:45 +08:00
Poster
Owner

With default nixpkgs LLVM (containing all targets):
149M ./target/release/libnac3artiq.so

Adding "-DLLVM_TARGETS_TO_BUILD=X86,ARM,RISCV" when building LLVM (editing pkgs/development/compilers/llvm/12/llvm/default.nix):
109M ./target/release/libnac3artiq.so

With default nixpkgs LLVM (containing all targets): 149M ./target/release/libnac3artiq.so Adding ``"-DLLVM_TARGETS_TO_BUILD=X86,ARM,RISCV"`` when building LLVM (editing ``pkgs/development/compilers/llvm/12/llvm/default.nix``): 109M ./target/release/libnac3artiq.so
pca006132 commented 2021-11-07 14:51:28 +08:00

With default nixpkgs LLVM (containing all targets):
149M ./target/release/libnac3artiq.so

Adding "-DLLVM_TARGETS_TO_BUILD=X86,ARM,RISCV" when building LLVM (editing pkgs/development/compilers/llvm/12/llvm/default.nix):
109M ./target/release/libnac3artiq.so

Removing debug info also reduce 149M to ~100M.

> With default nixpkgs LLVM (containing all targets): > 149M ./target/release/libnac3artiq.so > > Adding ``"-DLLVM_TARGETS_TO_BUILD=X86,ARM,RISCV"`` when building LLVM (editing ``pkgs/development/compilers/llvm/12/llvm/default.nix``): > 109M ./target/release/libnac3artiq.so > Removing debug info also reduce 149M to ~100M.
pca006132 commented 2021-11-07 15:08:37 +08:00

Apart from size bloat, I'm also a bit concerned about compilation time. Our development workflow will modify nac3core, and compile nac3artiq for example. Currently the compilation takes ~18s on my machine for release build and 11.5s for debug build, which is quite slow... (and slower on @ychenfo 's machine, taking over 1min iirc, maybe he should try tmpfs...)

Using lld and a nightly flag which shares generics between traits can reduce the compilation time to 11s for release build and 7s for debug build.

diff --git a/.cargo/config b/.cargo/config
index ac01e73..7be5770 100644
--- a/.cargo/config
+++ b/.cargo/config
@@ -1,2 +1,8 @@
 [unstable]
 extra-link-arg = true
+
+[target.x86_64-unknown-linux-gnu]
+rustflags = [
+    "-C", "link-arg=-fuse-ld=lld",
+    "-Zshare-generics=y"
+]
Apart from size bloat, I'm also a bit concerned about compilation time. Our development workflow will modify nac3core, and compile nac3artiq for example. Currently the compilation takes ~18s on my machine for release build and 11.5s for debug build, which is quite slow... (and slower on @ychenfo 's machine, taking over 1min iirc, maybe he should try tmpfs...) Using lld and a nightly flag which shares generics between traits can reduce the compilation time to 11s for release build and 7s for debug build. ```diff diff --git a/.cargo/config b/.cargo/config index ac01e73..7be5770 100644 --- a/.cargo/config +++ b/.cargo/config @@ -1,2 +1,8 @@ [unstable] extra-link-arg = true + +[target.x86_64-unknown-linux-gnu] +rustflags = [ + "-C", "link-arg=-fuse-ld=lld", + "-Zshare-generics=y" +] ```
pca006132 commented 2021-11-07 15:37:25 +08:00

Removing debug info, using the latest nightly with the above flags reduced the full release build compile time from 1m55s to 58.5s on zeus.

diff --git a/flake.nix b/flake.nix
index 8cf2ef1..8a362ff 100644
--- a/flake.nix
+++ b/flake.nix
@@ -1,27 +1,39 @@
 {
   description = "The third-generation ARTIQ compiler";
 
-  inputs.nixpkgs.url = github:NixOS/nixpkgs/master;
+  inputs = {
+    fenix = {
+      url = "github:nix-community/fenix";
+      inputs.nixpkgs.follows = "nixpkgs";
+    };
+    nixpkgs.url = github:NixOS/nixpkgs/master;
+  };
 
-  outputs = { self, nixpkgs }:
+  outputs = { self, fenix, nixpkgs }:
     let
-      pkgs = import nixpkgs { system = "x86_64-linux"; };
-    in rec {
+      pkgs = import nixpkgs {
+        system = "x86_64-linux";
+        overlays = [ fenix.overlay ];
+      };
+    in
+    rec {
       packages.x86_64-linux = {
         nac3artiq = pkgs.python3Packages.toPythonModule (
-          pkgs.rustPlatform.buildRustPackage {
+          (pkgs.makeRustPlatform {
+            inherit (fenix.packages.x86_64-linux.minimal) cargo rustc;
+          }).buildRustPackage {
             name = "nac3artiq";
             src = self;
             cargoSha256 = "sha256-otKLhr58HYMjVXAof6AdObNpggPnvK6qOl7I+4LWIP8=";
-            nativeBuildInputs = [ pkgs.python3 pkgs.llvm_12 ];
+            nativeBuildInputs = [ pkgs.python3 pkgs.llvm_12 pkgs.lld_12 ];
             buildInputs = [ pkgs.python3 pkgs.libffi pkgs.libxml2 pkgs.llvm_12 ];
             cargoBuildFlags = [ "--package" "nac3artiq" ];
             cargoTestFlags = [ "--package" "nac3ast" "--package" "nac3parser" "--package" "nac3core" "--package" "nac3artiq" ];
             installPhase =
               ''
-              TARGET_DIR=$out/${pkgs.python3Packages.python.sitePackages}
-              mkdir -p $TARGET_DIR
-              cp target/x86_64-unknown-linux-gnu/release/libnac3artiq.so $TARGET_DIR/nac3artiq.so
+                TARGET_DIR=$out/${pkgs.python3Packages.python.sitePackages}
+                mkdir -p $TARGET_DIR
+                cp target/x86_64-unknown-linux-gnu/release/libnac3artiq.so $TARGET_DIR/nac3artiq.so
               '';
           }
         );
@@ -33,12 +45,16 @@
           llvm_12
           clang_12
           lld_12
-          cargo
+          (pkgs.fenix.complete.withComponents [
+            "cargo"
+            "clippy"
+            "rust-src"
+            "rustc"
+            "rustfmt"
+          ])
           cargo-insta
-          rustc
           libffi
           libxml2
-          clippy
           (python3.withPackages(ps: [ ps.numpy ]))
         ];
       };
@@ -46,5 +62,6 @@
       hydraJobs = {
         inherit (packages.x86_64-linux) nac3artiq;
       };
-  };
+    };
 }
+

Note that nix build #.nac3artiq will now fail with

error: builder for '/nix/store/y0p6rasl5gp2x7yq5k5lw2m31vg51hg4-nac3artiq.drv' failed with exit code 127;
       last 10 log lines:
       >    Compiling serde_yaml v0.8.21
       >    Compiling nac3ast v0.1.0 (/build/source/nac3ast)
       >    Compiling nac3parser v0.1.2 (/build/source/nac3parser)
       >    Compiling insta v1.8.0
       >    Compiling nac3core v0.1.0 (/build/source/nac3core)
       >    Compiling nac3artiq v0.1.0 (/build/source/nac3artiq)
       >     Finished release [optimized] target(s) in 48.07s
       >      Running unittests (target/x86_64-unknown-linux-gnu/release/deps/nac3artiq-2146a041ef75dddf)
       > /build/source/target/x86_64-unknown-linux-gnu/release/deps/nac3artiq-2146a041ef75dddf: error while loading shared libraries: libffi.so.8: cannot open shared object file: No such file or directory
       > error: test failed, to rerun pass '-p nac3artiq --lib'

which I have no idea, maybe the LD_LIBRARY_PATH is messed up?

Removing debug info, using the latest nightly with the above flags reduced the full release build compile time from 1m55s to 58.5s on zeus. ```diff diff --git a/flake.nix b/flake.nix index 8cf2ef1..8a362ff 100644 --- a/flake.nix +++ b/flake.nix @@ -1,27 +1,39 @@ { description = "The third-generation ARTIQ compiler"; - inputs.nixpkgs.url = github:NixOS/nixpkgs/master; + inputs = { + fenix = { + url = "github:nix-community/fenix"; + inputs.nixpkgs.follows = "nixpkgs"; + }; + nixpkgs.url = github:NixOS/nixpkgs/master; + }; - outputs = { self, nixpkgs }: + outputs = { self, fenix, nixpkgs }: let - pkgs = import nixpkgs { system = "x86_64-linux"; }; - in rec { + pkgs = import nixpkgs { + system = "x86_64-linux"; + overlays = [ fenix.overlay ]; + }; + in + rec { packages.x86_64-linux = { nac3artiq = pkgs.python3Packages.toPythonModule ( - pkgs.rustPlatform.buildRustPackage { + (pkgs.makeRustPlatform { + inherit (fenix.packages.x86_64-linux.minimal) cargo rustc; + }).buildRustPackage { name = "nac3artiq"; src = self; cargoSha256 = "sha256-otKLhr58HYMjVXAof6AdObNpggPnvK6qOl7I+4LWIP8="; - nativeBuildInputs = [ pkgs.python3 pkgs.llvm_12 ]; + nativeBuildInputs = [ pkgs.python3 pkgs.llvm_12 pkgs.lld_12 ]; buildInputs = [ pkgs.python3 pkgs.libffi pkgs.libxml2 pkgs.llvm_12 ]; cargoBuildFlags = [ "--package" "nac3artiq" ]; cargoTestFlags = [ "--package" "nac3ast" "--package" "nac3parser" "--package" "nac3core" "--package" "nac3artiq" ]; installPhase = '' - TARGET_DIR=$out/${pkgs.python3Packages.python.sitePackages} - mkdir -p $TARGET_DIR - cp target/x86_64-unknown-linux-gnu/release/libnac3artiq.so $TARGET_DIR/nac3artiq.so + TARGET_DIR=$out/${pkgs.python3Packages.python.sitePackages} + mkdir -p $TARGET_DIR + cp target/x86_64-unknown-linux-gnu/release/libnac3artiq.so $TARGET_DIR/nac3artiq.so ''; } ); @@ -33,12 +45,16 @@ llvm_12 clang_12 lld_12 - cargo + (pkgs.fenix.complete.withComponents [ + "cargo" + "clippy" + "rust-src" + "rustc" + "rustfmt" + ]) cargo-insta - rustc libffi libxml2 - clippy (python3.withPackages(ps: [ ps.numpy ])) ]; }; @@ -46,5 +62,6 @@ hydraJobs = { inherit (packages.x86_64-linux) nac3artiq; }; - }; + }; } + ``` Note that `nix build #.nac3artiq` will now fail with ``` error: builder for '/nix/store/y0p6rasl5gp2x7yq5k5lw2m31vg51hg4-nac3artiq.drv' failed with exit code 127; last 10 log lines: > Compiling serde_yaml v0.8.21 > Compiling nac3ast v0.1.0 (/build/source/nac3ast) > Compiling nac3parser v0.1.2 (/build/source/nac3parser) > Compiling insta v1.8.0 > Compiling nac3core v0.1.0 (/build/source/nac3core) > Compiling nac3artiq v0.1.0 (/build/source/nac3artiq) > Finished release [optimized] target(s) in 48.07s > Running unittests (target/x86_64-unknown-linux-gnu/release/deps/nac3artiq-2146a041ef75dddf) > /build/source/target/x86_64-unknown-linux-gnu/release/deps/nac3artiq-2146a041ef75dddf: error while loading shared libraries: libffi.so.8: cannot open shared object file: No such file or directory > error: test failed, to rerun pass '-p nac3artiq --lib' ``` which I have no idea, maybe the LD_LIBRARY_PATH is messed up?
sb10q commented 2021-12-08 18:54:30 +08:00
Poster
Owner

I have removed the ffi dependency so it should work now. Might break the Windows build though and do we really want fenix and rust-nightly?
Note that in many cases rustc can be made to stop whining about "XYZ may not be used on the stable release channel" by simply setting the environment variable RUSTC_BOOTSTRAP=1.

I have removed the ffi dependency so it should work now. Might break the Windows build though and do we really want fenix and rust-nightly? Note that in many cases rustc can be made to stop whining about "XYZ may not be used on the stable release channel" by simply setting the environment variable ``RUSTC_BOOTSTRAP=1``.
sb10q commented 2022-10-16 17:16:57 +08:00
Poster
Owner

libgccjit might perhaps be an alternative to LLVM

> nix-shell -p gcc gmp mpfr libmpc
> export NIX_HARDENING_ENABLE=''
> ../gcc-12.2.0/configure --enable-host-shared --enable-languages=jit --disable-bootstrap --enable-checking=release --target=riscv-unknown-elf --disable-libgcc --disable-libquadmath --disable-libssp
> make -j32
> ls -sh gcc/libgccjit.so.0.0.1
19M gcc/libgccjit.so.0.0.1
libgccjit might perhaps be an alternative to LLVM ``` > nix-shell -p gcc gmp mpfr libmpc > export NIX_HARDENING_ENABLE='' > ../gcc-12.2.0/configure --enable-host-shared --enable-languages=jit --disable-bootstrap --enable-checking=release --target=riscv-unknown-elf --disable-libgcc --disable-libquadmath --disable-libssp > make -j32 > ls -sh gcc/libgccjit.so.0.0.1 19M gcc/libgccjit.so.0.0.1 ```
sb10q commented 2022-10-16 17:23:12 +08:00
Poster
Owner

Another alterntive is cranelift but 32-bit RISC-V and 32-bit ARM support is lacking.

Another alterntive is cranelift but 32-bit RISC-V and 32-bit ARM support is lacking.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
enhance/issue-149-ndarray/numpy-func-take-ndarray
master
enhance/cargo-test-standalone-demos
issue-136
rpc-obj-as-param
no_init
iterators
escape-analysis
refactor_anto
Labels
Clear labels   
UROP0622
   
future-extension
   
high-priority
   
low-priority
No Label
UROP0622
future-extension
high-priority
low-priority
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
ychenfo
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nac3#17
There is no content yet.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?