M-Labs
/
nac3
8
0
Fork 5
Code Issues 87 Pull Requests 3 Releases Wiki Activity

Meta changes from #321 #322

Merged
sb10q merged 7 commits from meta-changes-from-issue-315 into master 2023-09-22 20:59:29 +08:00
Conversation 8 Commits 7 Files Changed 11
derppening commented 2023-09-21 12:12:13 +08:00
Collaborator

Various changes that are made during the debugging process for #321 but is not relevant to the fix.

This PR contains infrastructural improvements, including updated Rust dependencies, replacement of deprecated LLVM intrinsics, implementation of __nac3_raise in Standalone, C implementation of the demo library and lli support, and improved documentation.

Various changes that are made during the debugging process for #321 but is not relevant to the fix. This PR contains infrastructural improvements, including updated Rust dependencies, replacement of deprecated LLVM intrinsics, ~~implementation of `__nac3_raise` in Standalone, C implementation of the `demo` library and `lli` support,~~ and improved documentation.
derppening self-assigned this 2023-09-21 12:12:14 +08:00
derppening added 8 commits 2023-09-21 12:12:15 +08:00
72570fbb16 core: Fix missing changes for codegen tests 
Apparently the changes were dropped after rebasing.
20aa094b1f core: Remove emit_llvm from CodeGenLLVMOptions 
We instead output an LLVM bitcode file when the option is specified on
the command-line.
derppening requested review from sb10q 2023-09-21 12:13:09 +08:00
derppening added a new dependency 2023-09-21 12:14:11 +08:00
#321 Use i8 for stack boolean variable allocation
derppening added a new dependency 2023-09-21 16:21:45 +08:00
#324 Meta changes from #301
sb10q commented 2023-09-21 21:27:43 +08:00
Owner

Please get rid of nac3standalone/demo/demo.c. We don't want to maintain both Rust and C versions. Can't rustc emit LLVM IR?

Please get rid of nac3standalone/demo/demo.c. We don't want to maintain both Rust and C versions. Can't rustc emit LLVM IR?
sb10q commented 2023-09-21 21:28:31 +08:00
Owner

And we don't want two C compilers either so please use clang not GCC. If GCC gets pulled as dependency somewhere then that's a bug in flake.nix that needs to be fixed.

And we don't want two C compilers either so please use clang not GCC. If GCC gets pulled as dependency somewhere then that's a bug in flake.nix that needs to be fixed.
sb10q reviewed 2023-09-21 21:31:17 +08:00
nac3core/src/codegen/expr.rs Outdated
@ -65,2 +65,3 @@
name: Option<&str>,
) -> BasicValueEnum<'ctx> {
unsafe { self.builder.build_load(self.builder.build_gep(ptr, index, "gep"), "load") }
unsafe { self.builder.build_load(self.builder.build_gep(ptr, index, ""), name.unwrap_or_default()) }
sb10q commented 2023-09-21 21:31:17 +08:00
Owner

I think unsafe can be restricted to build_gep?

I think unsafe can be restricted to build_gep?
sb10q commented 2023-09-21 21:32:46 +08:00
Owner

Please add invokations to run_demo_lli in the tests, otherwise it'll bitrot.

Please add invokations to run_demo_lli in the tests, otherwise it'll bitrot.
sb10q commented 2023-09-21 21:45:33 +08:00
Owner

implementation of __nac3_raise in Standalone

Where is that?

> implementation of __nac3_raise in Standalone Where is that?
derppening commented 2023-09-22 12:13:24 +08:00
Poster
Collaborator

Please get rid of nac3standalone/demo/demo.c. We don't want to maintain both Rust and C versions. Can't rustc emit LLVM IR?

rustc can emit LLVM IR, but a number of things will go wrong when using demo.rs for the implementation.

  • Emitting LLVM bitcode from demo.rs (rustc -O --emit=llvm-bc demo.rs):
lli: lli: demo.bc: error: Opaque pointers are only supported in -opaque-pointers mode (Producer: 'LLVM16.0.6' Reader: 'LLVM 14.0.6')
  • Emitting LLVM bitcode and enabling opaque pointer mode in lli:
lli: lli: demo.bc: error: Not an int attribute (Producer: 'LLVM16.0.6' Reader: 'LLVM 14.0.6')
  • Emitting object code (rustc -O --emit=obj demo.rs):
JIT session error: Symbols not found: [ _ZN3std2io5stdio6_print17h03234f5a885e836fE, _ZN3std2rt19lang_start_internal17hbcbc84fca12d560dE, _ZN43_$LT$char$u20$as$u20$core..fmt..Display$GT$3fmt17h646c6201d2eb8cf7E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$i32$GT$3fmt17h8f42ad49a80f761aE, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$i64$GT$3fmt17hca65c091437ab859E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$u32$GT$3fmt17h0a2397311660cdd4E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$u64$GT$3fmt17ha21735a82d3c171fE, _ZN4core3fmt5float50_$LT$impl$u20$core..fmt..Debug$u20$for$u20$f64$GT$3fmt17hb2a137fc77dfb50bE, _ZN4core9panicking5panic17h49ef28648bcd6145E ]
lli: Failed to materialize symbols: { (main, { main, output_int32, _ZN3std2rt10lang_start17h64195b79b23d67dcE, output_int64, output_uint64, output_uint32, __nac3_raise, output_int32_list, output_asciiart, __nac3_personality, output_float64 }) }
  • Emitting object code and including the Rust standard library to lli:
LLVM ERROR: allocation of TLS not implemented
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: lli --extra-object /nix/store/cxyn5knbq1w8cqajgasb5ccg01r5jrhn-rustc-1.72.0/lib/libstd-a31ad7eaa8e340af.so --extra-object demo.o --extra-module irrt.bc main.bc
[...]
./run_demo_lli.sh: line 26: 28271 Aborted                 (core dumped) lli --extra-object /nix/store/cxyn5knbq1w8cqajgasb5ccg01r5jrhn-rustc-1.72.0/lib/libstd-a31ad7eaa8e340af.so --extra-object demo.o --extra-module irrt.bc main.bc

This is why I opted to implement a minimal version of demo.rs in C to minimize the dependencies for lli.

And we don't want two C compilers either so please use clang not GCC. If GCC gets pulled as dependency somewhere then that's a bug in flake.nix that needs to be fixed.

For some reason Clang is not able to find headers for the C standard library.

$ echo | gcc -E -Wp,-v -  
#include "..." search starts here:
#include <...> search starts here:
 /nix/store/zg3b4xaazfzav5m44r5brkf5nn9r6704-llvm-14.0.6-dev/include
 /nix/store/cwsdr09dlvw3wgpdinzmpj8b8whh3m7a-ncurses-6.4-dev/include
 /nix/store/wv9bahc4cgf1bvp5g6jx0l5vi123kgl0-zlib-1.2.13-dev/include
 /nix/store/azxgj92jrp64xz9776mdy07qg54ak0ky-clang-14.0.6-dev/include
 /nix/store/1rmh01c749zqp36xj6rrlp1n41r7sbwz-lld-14.0.6-dev/include
 /nix/store/3jqppj27ily8k6q5cyniv7rjs7iaj0s2-python3-3.10.12-env/include
 /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/lib/gcc/x86_64-unknown-linux-gnu/12.3.0/include
 /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/include
 /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/lib/gcc/x86_64-unknown-linux-gnu/12.3.0/include-fixed
 /nix/store/p1wjbmik8bzpzcm2ck4yfxr6fnnc76mn-glibc-2.37-8-dev/include
End of search list.
# 0 "<stdin>"
# 0 "<built-in>"
# 0 "<command-line>"
# 1 "/nix/store/p1wjbmik8bzpzcm2ck4yfxr6fnnc76mn-glibc-2.37-8-dev/include/stdc-predef.h" 1 3 4
# 0 "<command-line>" 2
# 1 "<stdin>"

$ clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/nix/store/3mlw71m59v1b73bjpcnfk1xk98nf7rz8-clang-14.0.6/lib/clang/14.0.6/include"
#include "..." search starts here:
End of search list.
# 1 "<stdin>"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 357 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "<stdin>" 2

I can think of two ways to workaround this issue. We either (1) hardcode the necessary function prototypes into demo.c, or (2) we write a script to (a) lookup the path of GCC with respect to Nix, and (b) include those directories in the header search path.

I believe that it is rustc which is pulling GCC as a dependency, as any linker errors emitted by rustc -o demo demo.rs -Crelocation-model=static -Clink-arg=./module.o in run_demo.sh is prefixed with cc:, which is an indication that the linking process between Rust-generated code and external object files are performed by the GCC compiler.

./run_demo.sh jjj
Cannot open input file: No such file or directory (os error 2)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="[...]" "cc" [...] "./module.o"
  = note: /nix/store/8zxgqm6d879hskwh5g1l4crysl01kdv0-binutils-2.40/bin/ld: cannot find ./module.o: No such file or directory
          collect2: error: ld returned 1 exit status
          

error: aborting due to previous error

implementation of __nac3_raise in Standalone

Where is that?

I dropped this commit because I realized more work needs to be done in handling exceptions (the original implementation directly exits the program with no regards to except blocks). Updated the description to reflect this.

> Please get rid of nac3standalone/demo/demo.c. We don't want to maintain both Rust and C versions. Can't rustc emit LLVM IR? rustc can emit LLVM IR, but a number of things will go wrong when using `demo.rs` for the implementation. - Emitting LLVM bitcode from `demo.rs` (`rustc -O --emit=llvm-bc demo.rs`): ``` lli: lli: demo.bc: error: Opaque pointers are only supported in -opaque-pointers mode (Producer: 'LLVM16.0.6' Reader: 'LLVM 14.0.6') ``` - Emitting LLVM bitcode and enabling opaque pointer mode in lli: ``` lli: lli: demo.bc: error: Not an int attribute (Producer: 'LLVM16.0.6' Reader: 'LLVM 14.0.6') ``` - Emitting object code (`rustc -O --emit=obj demo.rs`): ``` JIT session error: Symbols not found: [ _ZN3std2io5stdio6_print17h03234f5a885e836fE, _ZN3std2rt19lang_start_internal17hbcbc84fca12d560dE, _ZN43_$LT$char$u20$as$u20$core..fmt..Display$GT$3fmt17h646c6201d2eb8cf7E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$i32$GT$3fmt17h8f42ad49a80f761aE, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$i64$GT$3fmt17hca65c091437ab859E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$u32$GT$3fmt17h0a2397311660cdd4E, _ZN4core3fmt3num3imp52_$LT$impl$u20$core..fmt..Display$u20$for$u20$u64$GT$3fmt17ha21735a82d3c171fE, _ZN4core3fmt5float50_$LT$impl$u20$core..fmt..Debug$u20$for$u20$f64$GT$3fmt17hb2a137fc77dfb50bE, _ZN4core9panicking5panic17h49ef28648bcd6145E ] lli: Failed to materialize symbols: { (main, { main, output_int32, _ZN3std2rt10lang_start17h64195b79b23d67dcE, output_int64, output_uint64, output_uint32, __nac3_raise, output_int32_list, output_asciiart, __nac3_personality, output_float64 }) } ``` - Emitting object code and including the Rust standard library to lli: ``` LLVM ERROR: allocation of TLS not implemented PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Program arguments: lli --extra-object /nix/store/cxyn5knbq1w8cqajgasb5ccg01r5jrhn-rustc-1.72.0/lib/libstd-a31ad7eaa8e340af.so --extra-object demo.o --extra-module irrt.bc main.bc [...] ./run_demo_lli.sh: line 26: 28271 Aborted (core dumped) lli --extra-object /nix/store/cxyn5knbq1w8cqajgasb5ccg01r5jrhn-rustc-1.72.0/lib/libstd-a31ad7eaa8e340af.so --extra-object demo.o --extra-module irrt.bc main.bc ``` This is why I opted to implement a minimal version of `demo.rs` in C to minimize the dependencies for `lli`. > And we don't want two C compilers either so please use clang not GCC. If GCC gets pulled as dependency somewhere then that's a bug in flake.nix that needs to be fixed. For some reason Clang is not able to find headers for the C standard library. ``` $ echo | gcc -E -Wp,-v - #include "..." search starts here: #include <...> search starts here: /nix/store/zg3b4xaazfzav5m44r5brkf5nn9r6704-llvm-14.0.6-dev/include /nix/store/cwsdr09dlvw3wgpdinzmpj8b8whh3m7a-ncurses-6.4-dev/include /nix/store/wv9bahc4cgf1bvp5g6jx0l5vi123kgl0-zlib-1.2.13-dev/include /nix/store/azxgj92jrp64xz9776mdy07qg54ak0ky-clang-14.0.6-dev/include /nix/store/1rmh01c749zqp36xj6rrlp1n41r7sbwz-lld-14.0.6-dev/include /nix/store/3jqppj27ily8k6q5cyniv7rjs7iaj0s2-python3-3.10.12-env/include /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/lib/gcc/x86_64-unknown-linux-gnu/12.3.0/include /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/include /nix/store/4khaz9q96z5nkgvh3150pz2ijhah30pr-gcc-12.3.0/lib/gcc/x86_64-unknown-linux-gnu/12.3.0/include-fixed /nix/store/p1wjbmik8bzpzcm2ck4yfxr6fnnc76mn-glibc-2.37-8-dev/include End of search list. # 0 "<stdin>" # 0 "<built-in>" # 0 "<command-line>" # 1 "/nix/store/p1wjbmik8bzpzcm2ck4yfxr6fnnc76mn-glibc-2.37-8-dev/include/stdc-predef.h" 1 3 4 # 0 "<command-line>" 2 # 1 "<stdin>" $ clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target x86_64-unknown-linux-gnu ignoring nonexistent directory "/nix/store/3mlw71m59v1b73bjpcnfk1xk98nf7rz8-clang-14.0.6/lib/clang/14.0.6/include" #include "..." search starts here: End of search list. # 1 "<stdin>" # 1 "<built-in>" 1 # 1 "<built-in>" 3 # 357 "<built-in>" 3 # 1 "<command line>" 1 # 1 "<built-in>" 2 # 1 "<stdin>" 2 ``` I can think of two ways to workaround this issue. We either (1) hardcode the necessary function prototypes into `demo.c`, or (2) we write a script to (a) lookup the path of GCC with respect to Nix, and (b) include those directories in the header search path. I believe that it is rustc which is pulling GCC as a dependency, as any linker errors emitted by `rustc -o demo demo.rs -Crelocation-model=static -Clink-arg=./module.o` in `run_demo.sh` is prefixed with `cc:`, which is an indication that the linking process between Rust-generated code and external object files are performed by the GCC compiler. ``` ./run_demo.sh jjj Cannot open input file: No such file or directory (os error 2) error: linking with `cc` failed: exit status: 1 | = note: LC_ALL="C" PATH="[...]" "cc" [...] "./module.o" = note: /nix/store/8zxgqm6d879hskwh5g1l4crysl01kdv0-binutils-2.40/bin/ld: cannot find ./module.o: No such file or directory collect2: error: ld returned 1 exit status error: aborting due to previous error ``` > > implementation of __nac3_raise in Standalone > > Where is that? I dropped this commit because I realized more work needs to be done in handling exceptions (the original implementation directly exits the program with no regards to `except` blocks). Updated the description to reflect this.
sb10q commented 2023-09-22 12:40:47 +08:00
Owner

For some reason Clang is not able to find headers for the C standard library.

The reason is those headers are provided to Clang by the Nix wrapper. But we are using clang-unwrapped for compiling IRRT since we don't actually want the system libraries there.

> For some reason Clang is not able to find headers for the C standard library. The reason is those headers are provided to Clang by the Nix wrapper. But we are using clang-unwrapped for compiling IRRT since we don't actually want the system libraries there.
sb10q commented 2023-09-22 12:44:05 +08:00
Owner

rustc can emit LLVM IR, but a number of things will go wrong

Have you tried making demo.rs a no_std Rust program? It's a very simple one so I don't think Rust should emit complicated IR, except for the standard library which we can do away with.

If it's really not possible to keep it in Rust then the alternative is to use only the C version and remove the Rust one.

> rustc can emit LLVM IR, but a number of things will go wrong Have you tried making demo.rs a no_std Rust program? It's a very simple one so I don't think Rust should emit complicated IR, except for the standard library which we can do away with. If it's really not possible to keep it in Rust then the alternative is to use only the C version and remove the Rust one.
derppening added 6 commits 2023-09-22 17:44:02 +08:00
3e430b9b40 core: Fix missing changes for codegen tests 
Apparently the changes were dropped after rebasing.
ab2360d7a0 core: Remove emit_llvm from CodeGenLLVMOptions 
We instead output an LLVM bitcode file when the option is specified on
the command-line.
derppening commented 2023-09-22 17:45:08 +08:00
Poster
Collaborator

v3: Dropped lli changes, restricted unsafe to build_gep

v3: Dropped `lli` changes, restricted unsafe to `build_gep`
sb10q merged commit fc93fc2f0e into master 2023-09-22 20:59:29 +08:00
Sign in to join this conversation.
Reviewers
No reviewers
sb10q
Labels
Clear labels   
UROP0622
   
future-extension
   
high-priority
   
low-priority
No Label
UROP0622
future-extension
high-priority
low-priority
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
derppening
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Blocks
#324 Meta changes from #301
M-Labs/nac3
Reference: M-Labs/nac3#322
There is no content yet.
Delete Branch "meta-changes-from-issue-315"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?