Skip to content

Commit b7dd6a7

Browse files
jservjserv
committed
CI: Add explicit TSAN race detection validation
Add comprehensive ThreadSanitizer output validation to prevent silent test failures. TSAN tests now explicitly check for data races and fail immediately with diagnostic output instead of masking errors. Implementation: - Capture TSAN stderr/stdout to log files for analysis - Pattern match race indicators: "ThreadSanitizer: data race", "ThreadSanitizer: race on", "WARNING: ThreadSanitizer:" - Exit with status 1 immediately upon race detection - Display race context (10 lines) for debugging - Progress indicators for 3-tier validation (Interpreter, JIT-T1, JIT-T2) Platform-Specific Handling: - Linux (x64/ARM64): Use setarch -R for ASLR mitigation (already gated in Makefile with ifeq ($(UNAME_S),Linux)) - macOS: NO setarch (not available), rely on MAP_FIXED allocations at 0x150000000000; gracefully handle SIP restrictions by distinguishing MAP_FAILED errors from actual race conditions Race Detection Patterns: ThreadSanitizer: data race # Standard race report ThreadSanitizer: race on # Race on specific object WARNING: ThreadSanitizer: # General TSAN warnings Error Handling (macOS): MAP_FAILED # mmap failure unexpected memory mapping # TSAN shadow conflict FATAL: ThreadSanitizer # Initialization failure → Skip test with warning (SIP restriction) → Still fail hard on actual races Benefits: - Immediate failure on race detection (fail-fast principle) - Clear diagnostic output in CI logs with race location/context - Platform-aware: Linux uses setarch -R, macOS handles SIP gracefully - No silent failures: Previously masked errors now cause test failure - Debugging support: Log files preserved for post-mortem analysis Validates race condition fixes from: - 2a2a5c4: TSAN with FULL4G and T2C support - f1b685e: ARM64 TSAN support and JIT cache coherency - 669efc1: Build system regressions (setarch gating, TSAN compatibility)
1 parent d1beb5a commit b7dd6a7

File tree

1 file changed

+134
-0
lines changed

1 file changed

+134
-0
lines changed

‎.github/workflows/main.yml‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -250,6 +250,46 @@ jobs:
250250
make distclean && make ENABLE_UBSAN=1 check $PARALLEL
251251
make ENABLE_JIT=1 clean && make ENABLE_JIT=1 ENABLE_UBSAN=1 check $PARALLEL
252252
253+
- name: ThreadSanitizer race detection test
254+
if: success() || failure()
255+
env:
256+
CC: ${{ steps.install_cc.outputs.cc }}
257+
run: |
258+
set -o pipefail
259+
260+
# TSAN requires ASLR disabled to prevent allocations in shadow memory
261+
# Interpreter with FULL4G: Basic race detection across emulation core
262+
echo "=== TSAN Test 1/3: Interpreter + FULL4G ==="
263+
make distclean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log
264+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
265+
echo "ERROR: Data race detected in interpreter mode!"
266+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
267+
exit 1
268+
fi
269+
echo "✓ No races detected in interpreter mode"
270+
271+
# JIT tier-1: Race detection in template-based JIT compilation
272+
echo "=== TSAN Test 2/3: JIT Tier-1 ==="
273+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log
274+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
275+
echo "ERROR: Data race detected in JIT tier-1 mode!"
276+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
277+
exit 1
278+
fi
279+
echo "✓ No races detected in JIT tier-1 mode"
280+
281+
# JIT tier-2 (T2C): Race detection across LLVM compilation thread
282+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) ==="
283+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log
284+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
285+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
286+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
287+
exit 1
288+
fi
289+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
290+
291+
echo "=== All TSAN tests passed ==="
292+
253293
- name: boot Linux kernel test
254294
if: success()
255295
env:
@@ -368,6 +408,33 @@ jobs:
368408
make ENABLE_JIT=1 clean && make ENABLE_EXT_A=0 ENABLE_JIT=1 check $PARALLEL
369409
make ENABLE_JIT=1 clean && make ENABLE_EXT_F=0 ENABLE_JIT=1 check $PARALLEL
370410
make ENABLE_JIT=1 clean && make ENABLE_EXT_C=0 ENABLE_JIT=1 check $PARALLEL
411+
# TSAN on ARM64: Fixed memory layout (0x150000000000 for main, 0x151000000000 for JIT)
412+
set -o pipefail
413+
echo "=== TSAN Test 1/3: Interpreter + FULL4G (ARM64) ==="
414+
make distclean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log
415+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
416+
echo "ERROR: Data race detected in interpreter mode!"
417+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
418+
exit 1
419+
fi
420+
echo "✓ No races detected in interpreter mode"
421+
echo "=== TSAN Test 2/3: JIT Tier-1 (ARM64) ==="
422+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log
423+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
424+
echo "ERROR: Data race detected in JIT tier-1 mode!"
425+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
426+
exit 1
427+
fi
428+
echo "✓ No races detected in JIT tier-1 mode"
429+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) (ARM64) ==="
430+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log
431+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
432+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
433+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
434+
exit 1
435+
fi
436+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
437+
echo "=== All TSAN tests passed (ARM64) ==="
371438
372439
macOS-arm64:
373440
needs: [detect-code-related-file-changes]
@@ -557,6 +624,73 @@ jobs:
557624
make distclean && make ENABLE_UBSAN=1 check $PARALLEL
558625
make ENABLE_JIT=1 clean && make ENABLE_JIT=1 ENABLE_UBSAN=1 check $PARALLEL
559626
627+
- name: ThreadSanitizer race detection test
628+
if: (success() || failure()) && steps.install_cc.outputs.cc == 'clang' # Only clang supports TSAN on macOS
629+
env:
630+
CC: ${{ steps.install_cc.outputs.cc }}
631+
run: |
632+
set -o pipefail
633+
634+
# macOS TSAN: Fixed memory at 0x150000000000 (main) and 0x151000000000 (JIT)
635+
# Note: ASLR disabled via mmap(MAP_FIXED), but SIP may restrict full ASLR control on GitHub runners
636+
637+
# Test 1: Interpreter + FULL4G
638+
echo "=== TSAN Test 1/3: Interpreter + FULL4G (macOS ARM64) ==="
639+
make distclean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log || {
640+
# Check if failure is due to MAP_FIXED restriction vs actual race
641+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-interpreter.log; then
642+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
643+
else
644+
echo "ERROR: Test execution failed"
645+
cat tsan-interpreter.log
646+
exit 1
647+
fi
648+
}
649+
if [ -f tsan-interpreter.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
650+
echo "ERROR: Data race detected in interpreter mode!"
651+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
652+
exit 1
653+
fi
654+
echo "✓ No races detected in interpreter mode"
655+
656+
# Test 2: JIT tier-1
657+
echo "=== TSAN Test 2/3: JIT Tier-1 (macOS ARM64) ==="
658+
make ENABLE_JIT=1 clean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log || {
659+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-jit.log; then
660+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
661+
else
662+
echo "ERROR: Test execution failed"
663+
cat tsan-jit.log
664+
exit 1
665+
fi
666+
}
667+
if [ -f tsan-jit.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
668+
echo "ERROR: Data race detected in JIT tier-1 mode!"
669+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
670+
exit 1
671+
fi
672+
echo "✓ No races detected in JIT tier-1 mode"
673+
674+
# Test 3: JIT tier-2 (T2C)
675+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) (macOS ARM64) ==="
676+
make ENABLE_JIT=1 clean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log || {
677+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-t2c.log; then
678+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
679+
else
680+
echo "ERROR: Test execution failed"
681+
cat tsan-t2c.log
682+
exit 1
683+
fi
684+
}
685+
if [ -f tsan-t2c.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
686+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
687+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
688+
exit 1
689+
fi
690+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
691+
692+
echo "=== All TSAN tests completed (macOS ARM64) ==="
693+
560694
- name: boot Linux kernel test
561695
if: success()
562696
env:

0 commit comments

Comments
 (0)