libafl_libfuzzer fails to build with error: 'sanitizer/allocator_interface.h' file not found

[0xalpharush]

I am trying to use the libafl_libfuzzer shim with cargo-fuzz and hitting this error:

    --- stderr
    src/sanitizer_interfaces.h:1:10: fatal error: 'sanitizer/allocator_interface.h' file not found
    thread 'main' panicked at /root/.cargo/git/checkouts/libafl-c33dc6f5ec2f7a70/2c676f0/libafl_targets/build.rs:237:14:
    Couldn't generate the sanitizer headers!: ClangDiagnostic("src/sanitizer_interfaces.h:1:10: fatal error: 'sanitizer/allocator_interface.h' file not found\n")
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  warning: build failed, waiting for other jobs to finish...
  thread 'main' panicked at /root/.cargo/git/checkouts/libafl-c33dc6f5ec2f7a70/2c676f0/libafl_libfuzzer/build.rs:137:5:
  Couldn't build runtime crate! Did you remember to use nightly? (`rustup default nightly` to install)
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: failed to build fuzz script: RUSTFLAGS="-Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table -Cllvm-args=-sanitizer-coverage-trace-compares --cfg fuzzing -Clink-dead-code -Cllvm-args=-sanitizer-coverage-stack-depth -Cdebug-assertions -C codegen-units=1" "cargo" "build" "--manifest-path" "/root/fuzz/fuzz/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "--release" "--config" "profile.release.debug=true" "--bin" "fuzz_target1"

As I have not been able to find any info on this error elsewhere, I'd appreciate any help. I am using rustc 1.83.0-nightly (c52c23b6f 2024-09-16) and running on an X86_64 Ubuntu server

[0xalpharush]

I ran rustup component add llvm-tools-preview --toolchain nightly && apt install clang pkg-config libssl-dev and get this assertion failure now:

LibAFL/libafl_libfuzzer/build.rs

Lines 137 to 140 in 2c676f0

	assert!(
	command.status().map_or(false, |s| s.success()),
	"Couldn't build runtime crate! Did you remember to use nightly? (`rustup default nightly` to install)"
	);

Caused by:
  process didn't exit successfully: `/root/fuzz/fuzz/target/release/build/libafl_libfuzzer-a21e2b403b9f0c1b/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=libafl_libfuzzer_runtime/src
  cargo:rerun-if-changed=libafl_libfuzzer_runtime/build.rs

  --- stderr
  warning: [email protected]: src/forkserver.c: In function ‘__afl_start_forkserver’:
  warning: [email protected]: src/forkserver.c:275:7: warning: ignoring return value of ‘write’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
  warning: [email protected]:   275 |       write(2, "Error: could not send autotokens len\n",
  warning: [email protected]:       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  warning: [email protected]:   276 |             strlen("Error: could not send autotokens len\n"));
  warning: [email protected]:       |             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Compiling libafl_libfuzzer_runtime v0.13.2 (/root/.cargo/git/checkouts/libafl-c33dc6f5ec2f7a70/2c676f0/libafl_libfuzzer_runtime)
  warning: elided lifetime has a name
    --> src/corpus.rs:91:17
     |
  87 |     fn _get<'a>(
     |             -- lifetime `'a` declared here
  ...
  91 |     ) -> Result<&RefCell<Testcase<I>>, Error> {
     |                 ^ this elided lifetime gets resolved as `'a`
     |
     = note: `#[warn(elided_named_lifetimes)]` on by default

  warning: `libafl_libfuzzer_runtime` (lib) generated 1 warning
  error: could not compile `libafl_libfuzzer_runtime` (lib); 1 warning emitted

  Caused by:
    process didn't exit successfully: `/root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc --crate-name afl_libfuzzer_runtime --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type staticlib --crate-type rlib --emit=dep-info,link -C opt-level=3 -C codegen-units=1 -C debuginfo=2 --cfg 'feature="fork"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("default", "fork", "track_hit_feedbacks"))' -C metadata=54bda43eadd7b8a6 -C extra-filename=-54bda43eadd7b8a6 --out-dir /root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps -L dependency=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/release/deps --extern ahash=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libahash-d4454d7199788b8c.rlib --extern env_logger=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libenv_logger-719e55909dfbbec1.rlib --extern hashbrown=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libhashbrown-903e9b269810aecf.rlib --extern libafl=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/liblibafl-63d1b96026e4f739.rlib --extern libafl_bolts=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/liblibafl_bolts-62092ff42cbc7450.rlib --extern libafl_targets=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/liblibafl_targets-5f48e8e7f5f0b9f8.rlib --extern libc=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/liblibc-a1f938fc5f4b2aea.rlib --extern log=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/liblog-2fb043475b03a7f5.rlib --extern mimalloc=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libmimalloc-a8390b24249b26dc.rlib --extern num_traits=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libnum_traits-08ac1ad7e0a7e7b7.rlib --extern rand=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/librand-29c25b879a1001f7.rlib --extern serde=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libserde-a0a1a1274dc86ed8.rlib --extern utf8_chars=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/deps/libutf8_chars-ea56d2f928010457.rlib -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer_runtime-6dcb95a3f04e49ac/out -l static=harness_wrap -l stdc++ -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libafl_targets-d24b81b125d961ce/out -L native=/root/fuzz/fuzz/target/x86_64-unknown-linux-gnu/release/build/libafl_libfuzzer-0ec8409e59a28312/out/libafl_libfuzzer/target/x86_64-unknown-linux-gnu/release/build/libmimalloc-sys-4939a0d87e815a1a/out` (signal: 9, SIGKILL: kill)
  thread 'main' panicked at /root/.cargo/git/checkouts/libafl-c33dc6f5ec2f7a70/2c676f0/libafl_libfuzzer/build.rs:137:5:
  Couldn't build runtime crate! Did you remember to use nightly? (`rustup default nightly` to install)
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtra

[addisoncrump]

Looks like you're OOMing during build. Check your memory usage during the build?

[0xalpharush]

How much memory is expected to be required?

Command exited with non-zero status 1
        Command being timed: "cargo fuzz build"
        User time (seconds): 26.33
        System time (seconds): 0.71
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:27.07
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 838316
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 1
        Minor (reclaiming a frame) page faults: 257070
        Voluntary context switches: 706
        Involuntary context switches: 602
        Swaps: 0
        File system inputs: 2536
        File system outputs: 48
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 1

[addisoncrump]

LTO is pretty greedy... unfortunately we don't have a good way to disable this. At build time I observed 2GB, at link time I observed up to 4.2GB.