The 'abi_mismatch' problem encountered when writing test code to detect the coexistence of 'stack-protector' and 'safe-stack'

[cezarbbb]

I encountered some issues while writing test code for the "Stabilize stack-protector" proposal（see #146369）. Specifically, when writing functions that allow both 'stack-protector' and 'safe-stack' functionality to coexist, I encountered an 'abi_mismatch' error from the referenced libraries. I tried using #[no_std] to avoid using some other libraries, but the same problem persisted.

My code is as follows:

//@ revisions: all strong none safestack safestack_strong safestack_all
//@ assembly-output: emit-asm
//@ignore-msvc safestack sanitizer not supported
//@ignore-musl safestack sanitizer not supported
//@ignore-nvptx64 stack protector is not supported
//@ignore-wasm32 safestack sanitizer not supported
//@ignore-aarch64
//@ [all] compile-flags: -Z stack-protector=all
//@ [strong] compile-flags: -Z stack-protector=strong
//@ [none] compile-flags: -Z stack-protector=none
//@ [safestack] compile-flags: -Z stack-protector=none -Z sanitizer=safestack
//@ [safestack_strong] compile-flags: -Z stack-protector=strong -Z sanitizer=safestack
//@ [safestack_all] compile-flags: -Z stack-protector=all -Z sanitizer=safestack
//@ compile-flags: -C opt-level=2 -Z merge-functions=disabled

#![no_std]
#![feature(link_llvm_intrinsics)]
#![feature(unsized_fn_params)]
#![crate_type = "lib"]

extern "C" { 
#[link_name = "llvm.memcpy.p0.p0.i64"] 
fn memcpy_intrinsic(dst: *mut u8, src: *const u8, size: u64, align: u32, is_volatile: bool);
}

//CHECK-LABEL: test1{{:|\[}}
#[no_mangle]
pub unsafe fn test1(src: *const u8, len: usize) -> u8 { 
let mut buf: [u8; 64] = [0; 64]; 

let copy_len = if len < 64 { len } else { 64 }; 
memcpy_intrinsic(buf.as_mut_ptr(), src, copy_len as u64, 1, false); 

buf[0] 

// none-NOT: __stack_chk_fail 
// strong: __stack_chk_fail 
// all: __stack_chk_fail 

// safestack: __safestack_unsafe_stack_ptr 
// safestack-NOT: __stack_chk_fail 

// safestack_strong: __safestack_unsafe_stack_ptr 
// safestack_strong: __stack_chk_fail 

// safestack_all: __safestack_unsafe_stack_ptr 
// safestack_all: __stack_chk_fail
}

The error is reported as follows:

------------------------------------------
2025-11-21T08:34:32.8540173Z	
2025-11-21T08:34:32.8540531Z	error in revision `safestack_strong`: compilation failed!
2025-11-21T08:34:32.8540835Z	status: exit status: 1
2025-11-21T08:34:32.8544229Z	command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/rustc" "/checkout/tests/assembly-llvm/stack-protector/stack-protector-safe-stack.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage1" "--target=x86_64-unknown-linux-gnu" "--cfg" "safestack_strong" "--check-cfg" "cfg(test,FALSE,all,strong,none,safestack,safestack_strong,safestack_all)" "-O" "-Cdebug-assertions=no" "-Zcodegen-source-order" "--emit" "asm" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/assembly-llvm/stack-protector/stack-protector-safe-stack.safestack_strong/stack-protector-safe-stack.s" "-A" "internal_features" "-A" "unused_parens" "-A" "unused_braces" "-Crpath" "-Cdebuginfo=0" "-Z" "stack-protector=strong" "-Z" "sanitizer=safestack" "-C" "opt-level=2" "-Z" "merge-functions=disabled"
2025-11-21T08:34:32.8547790Z	stdout: none
2025-11-21T08:34:32.8547999Z	--- stderr -------------------------------
2025-11-21T08:34:32.8548407Z	error: mixing `-Zsanitizer` will cause an ABI mismatch in crate `stack_protector_safe_stack`
2025-11-21T08:34:32.8549212Z	##[error]  --> /checkout/tests/assembly-llvm/stack-protector/stack-protector-safe-stack.rs:16:1
2025-11-21T08:34:32.8549956Z	   |
2025-11-21T08:34:32.8550121Z	16 | #![no_std]
2025-11-21T08:34:32.8550393Z	   | ^
2025-11-21T08:34:32.8550575Z	   |
2025-11-21T08:34:32.8551064Z	   = help: the `-Zsanitizer` flag modifies the ABI so Rust crates compiled with different values of this flag cannot be used together safely
2025-11-21T08:34:32.8551741Z	   = note: `-Zsanitizer=safestack` in this crate is incompatible with unset `-Zsanitizer` in dependency `core`
2025-11-21T08:34:32.8552283Z	   = help: unset `-Zsanitizer` in this crate or set `-Zsanitizer=safestack` in `core`
2025-11-21T08:34:32.8552860Z	   = help: if you are sure this will not cause problems, you may use `-Cunsafe-allow-abi-mismatch=sanitizer` to silence this error
2025-11-21T08:34:32.8553237Z	
2025-11-21T08:34:32.8553479Z	error: mixing `-Zsanitizer` will cause an ABI mismatch in crate `stack_protector_safe_stack`
2025-11-21T08:34:32.8554273Z	##[error]  --> /checkout/tests/assembly-llvm/stack-protector/stack-protector-safe-stack.rs:16:1
2025-11-21T08:34:32.8555023Z	   |
2025-11-21T08:34:32.8555183Z	16 | #![no_std]
2025-11-21T08:34:32.8555359Z	   | ^
2025-11-21T08:34:32.8555535Z	   |
2025-11-21T08:34:32.8556033Z	   = help: the `-Zsanitizer` flag modifies the ABI so Rust crates compiled with different values of this flag cannot be used together safely
2025-11-21T08:34:32.8556742Z	   = note: `-Zsanitizer=safestack` in this crate is incompatible with unset `-Zsanitizer` in dependency `compiler_builtins`
2025-11-21T08:34:32.8557332Z	   = help: unset `-Zsanitizer` in this crate or set `-Zsanitizer=safestack` in `compiler_builtins`
2025-11-21T08:34:32.8557925Z	   = help: if you are sure this will not cause problems, you may use `-Cunsafe-allow-abi-mismatch=sanitizer` to silence this error
2025-11-21T08:34:32.8558295Z	
2025-11-21T08:34:32.8558437Z	error: aborting due to 2 previous errors
2025-11-21T08:34:32.8558706Z	------------------------------------------

For a more detailed explanation of the error, please refer to my previously submitted pull request：#147115

PS: I reviewed the previous test code involving safe-stack as follows:

// This tests that the safestack attribute is applied when enabling the safe-stack sanitizer.
//
//@ needs-sanitizer-safestack
//@ compile-flags: -Zsanitizer=safestack -Copt-level=0

#![crate_type = "lib"]

// CHECK: ; Function Attrs:{{.*}}safestack
pub fn tagged() {}

// CHECK: attributes #0 = {{.*}}safestack

This empty function does not cause errors, but I have to do some corresponding operations to trigger the corresponding rules of stack-protector to take effect when checking 'safe-stack' and 'stack-protector'.

[bjorn3]

no_std will still use libcore. You could use a minicore based test. That way minicore gets compiled with stack protectors enbled too.

[cezarbbb]

no_std will still use libcore. You could use a minicore based test. That way minicore gets compiled with stack protectors enbled too.

Thanks! I'll have a try