Enable `outline-atomics` by default on more AArch64 platforms #144938

tgross35 · 2025-08-05T02:29:48Z

The baseline Armv8.0 ISA doesn't have atomics instructions, but in
practice most hardware is at least Armv8.1-A (2014), which includes
single-instruction atomics as part of the LSE feature. As a performance
optimization for these cases, GCC and LLVM have the -moutline-atomics flag
to turn atomic operations into calls to symbols like __aarch64_cas1_acq.
These can do runtime feature detection and use the LSE instructions if
available, falling back to more portable load-exclusive/store-exclusive
loops.

Since the recent 3b50253 ("compiler-builtins: plumb LSE support
for aarch64 on linux") our builtins support this LSE optimization, and
since 6936bb9 ("Dynamically enable LSE for aarch64 rust provided
intrinsics"), std will set the flag as part of its startup code. The first
commit in this PR configures this to work on all platforms built with
outline-atomics, not just Linux.

Thus, enable outline-atomics by default on Android, OpenBSD, Windows,
and Fuchsia platforms that don't have LSE in the baseline. The feature is
already enabled on Linux. Platform-specific details are included in each
commit message.

The current implementation can still be accessed by setting
-Ctarget-feature=-outline-atomics. Setting -Ctarget-feature=+lse or
a relevant CPU will use the single-instruction atomics without the call
overhead. https://rust.godbolt.org/z/dsdrzszoe

Link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/lse/intro/
Original Clang outline-atomics benchmarks: https://reviews.llvm.org/D91157#2435844

try-job: aarch64-msvc-*
try-job: arm-android
try-job: dist-android
try-job: dist-aarch64-llvm-mingw
try-job: dist-aarch64-msvc
try-job: dist-various-*
try-job: test-various

rustbot · 2025-08-05T02:29:52Z

r? @ChrisDenton

rustbot has assigned @ChrisDenton.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-08-05T02:29:55Z

These commits modify compiler targets.
(See the Target Tier Policy.)

tgross35 · 2025-08-05T02:30:02Z

Whoops

r? @ghost

[experiment] enable outline-atomics on more aarch64 platforms try-job: arm-android try-job: dist-android try-job: dist-x86_64-freebsd try-job: dist-aarch64-windows-gnullvm try-job: dist-aarch64-apple try-job: aarch64-msvc-1 try-job: aarch64-msvc-2 try-job: dist-aarch64-msvc

Enable outline-atomics by default on more aarch64 platforms try-job: aarch64-apple try-job: aarch64-msvc-1 try-job: aarch64-msvc-2 try-job: arm-android try-job: dist-android try-job: dist-aarch64-windows-gnullvm try-job: dist-aarch64-apple try-job: dist-aarch64-msvc try-job: dist-various try-job: dist-x86_64-freebsd

Nashenas88 · 2025-09-10T05:38:48Z

I think this is good for fuchsia. I'm going to run it by some folks tomorrow morning.

Nashenas88 · 2025-09-10T17:30:05Z

No concerns from other folks on fuchsia. Thanks for working on this!

Nashenas88 · 2025-09-10T21:54:02Z

library/std/src/sys/configure_builtins.rs

+)]
 static RUST_LSE_INIT: extern "C" fn() = {
    extern "C" fn init_lse() {
        use crate::arch;


This comment is actually for line 37 below. Shouldn't the comment be updated to refer to the renamed file?

-// This is provided by compiler-builtins::aarch64_linux. +// This is provided by compiler-builtins::aarch64_outline_atomics.

Thanks for catching this! Updated

madsmtm

I verified that __rust_enable_lse is run on Apple platforms when having -lse,+outline-atomics.

It isn't actually used though, the code will always fall back to the slower, non LSE implementation (regardless of whether the current CPU actually supports it), since try_lse_op! is gated with .arch_extension lse, which doesn't seem to be enabled on Apple platforms?

And if I just remove it, I get:

error: ADR/ADRP relocations must be GOT relative
  |
note: instantiated into assembly here
 --> <inline asm>:6:14
  |
6 | adrp    x16, __ZN17compiler_builtins23aarch64_outline_atomics16HAVE_LSE_ATOMICS17h7f004a93068845c5E; nop; ldrb    w16, [x16, :lo12:__ZN17compiler_builtins...
  |              ^

error: unknown AArch64 fixup kind!
  |
note: instantiated into assembly here
 --> <inline asm>:6:14
  |
6 | adrp    x16, __ZN17compiler_builtins23aarch64_outline_atomics16HAVE_LSE_ATOMICS17h7f004a93068845c5E; nop; ldrb    w16, [x16, :lo12:__ZN17compiler_builtins...
  |              ^

That error seems to come from around here in AArch64MachObjectWriter.cpp, which seems to hint that the way the check is implemented isn't supported on Mach-O? Unsure.

View changes since this review

maurer · 2025-09-12T19:32:25Z

Stated in the Zulip thread already, but just to be clear, this change is fine for Android as well.

library/std/src/sys/configure_builtins.rs

davidtwco · 2025-09-17T12:16:57Z

It might make sense to land this for some targets first to avoid waiting on target maintainers replying and create a tracking issue for the rest.

bors · 2025-11-26T19:06:05Z

☔ The latest upstream changes (presumably #149351) made this pull request unmergeable. Please resolve the merge conflicts.

Build outline atomic symbols on all targets that have `outline-atomics` enabled, rather than only on Linux. Since this is no longer OS-specific, also rename the module.

rustbot · 2025-12-04T07:31:02Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Change the gating and link sections to enable this for any platforms that enable `outline-atomics`, rather than only Linux. Additionally, no longer run this if LSE is available, since in this case the outline versions will never be called.

Windows has a similar flag `/forceInterlockedFunctions`, which uses names such as `_InterlockedAdd64_rel`.

Per LLVM commit c5e7e64 ("[AArch64][Clang][Linux] Enable out-of-line atomics by default.") [1], Clang enables these on Android. Thus, do the same in Rust. [1]: llvm/llvm-project@c5e7e649d537067de

Clang has done this by default since LLVM commit 1a963d3 ("[Driver] Make -moutline-atomics default for aarch64-fuchsia targets"), [1], so do the same here. [1]: llvm/llvm-project@1a963d3

Clang has recently started doing this, as of LLVM commit 5d774ec8d183 ("[Driver] Enable outline atomics for OpenBSD/aarch64") [1]. Thus, do the same here. [1]: llvm/llvm-project@5d774ec

tgross35 · 2025-12-04T08:44:11Z

@bors2 try

Enable `outline-atomics` by default on more AArch64 platforms try-job: aarch64-msvc-* try-job: arm-android try-job: dist-android try-job: dist-aarch64-windows-gnullvm try-job: dist-aarch64-msvc try-job: dist-various-* try-job: test-various

tgross35 · 2025-12-04T08:45:56Z

Finally got a chance to circle back here. I split FreeBSD to a separate PR #149633, and dropped the Apple with an open issue #149634. For everything else here, the target maintainers have said this should be okay either on this PR or on Zulip.

@rustbot ready

tgross35 · 2025-12-04T08:47:08Z

@bors2 try

Enable `outline-atomics` by default on more AArch64 platforms try-job: aarch64-msvc-* try-job: arm-android try-job: dist-android try-job: dist-aarch64-llvm-mingw try-job: dist-aarch64-msvc try-job: dist-various-* try-job: test-various

rust-bors · 2025-12-04T11:04:54Z

☀️ Try build successful (CI)
Build commit: 73cd8c5 (73cd8c5953271a06db9935692296d56895fd742b, parent: 556beb9ec72360512d0294eb0855c92fb2c20c88)

rustbot assigned ChrisDenton Aug 5, 2025

rustbot unassigned ChrisDenton Aug 5, 2025

tgross35 marked this pull request as draft August 5, 2025 02:30

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 5, 2025

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

tgross35 force-pushed the more-outline-atomics branch from a25e101 to e20add7 Compare September 5, 2025 19:39

tgross35 changed the title ~~[experiment] enable outline-atomics on more aarch64 platforms~~ Enable outline-atomics by default on more aarch64 platforms Sep 5, 2025

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

Nashenas88 reviewed Sep 10, 2025

View reviewed changes

madsmtm reviewed Sep 11, 2025

View reviewed changes

madsmtm reviewed Sep 14, 2025

View reviewed changes

library/std/src/sys/configure_builtins.rs Show resolved Hide resolved

compiler-builtins: Change gating for outline atomic symbols

0030e8a

Build outline atomic symbols on all targets that have `outline-atomics` enabled, rather than only on Linux. Since this is no longer OS-specific, also rename the module.

tgross35 force-pushed the more-outline-atomics branch from b18a6c6 to fe78458 Compare December 4, 2025 07:30

tgross35 force-pushed the more-outline-atomics branch from fe78458 to 4401f9c Compare December 4, 2025 08:18

tgross35 added 5 commits December 4, 2025 03:36

Enable outline-atomics by default on AArch64 Windows platforms

1ed1b6e

Windows has a similar flag `/forceInterlockedFunctions`, which uses names such as `_InterlockedAdd64_rel`.

Enable outline-atomics by default on AArch64 Android

c455903

Per LLVM commit c5e7e64 ("[AArch64][Clang][Linux] Enable out-of-line atomics by default.") [1], Clang enables these on Android. Thus, do the same in Rust. [1]: llvm/llvm-project@c5e7e649d537067de

Enable outline-atomics by default on AArch64 Fuchsia

21525f8

Clang has done this by default since LLVM commit 1a963d3 ("[Driver] Make -moutline-atomics default for aarch64-fuchsia targets"), [1], so do the same here. [1]: llvm/llvm-project@1a963d3

Enable outline-atomics by default on AArch64 OpenBSD

66c150c

Clang has recently started doing this, as of LLVM commit 5d774ec8d183 ("[Driver] Enable outline atomics for OpenBSD/aarch64") [1]. Thus, do the same here. [1]: llvm/llvm-project@5d774ec

tgross35 force-pushed the more-outline-atomics branch from 4401f9c to 66c150c Compare December 4, 2025 08:36

This was referenced Dec 4, 2025

Enable outline-atomics by default on AArch64 FreeBSD #149633

Open

Enable outline-atomics by default on aarch64 Apple targets #149634

Open

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

Enable outline-atomics by default on more AArch64 platforms #144938

Are you sure you want to change the base?

Enable outline-atomics by default on more AArch64 platforms #144938

Uh oh!

Conversation

tgross35 commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 5, 2025

Uh oh!

rustbot commented Aug 5, 2025

Uh oh!

tgross35 commented Aug 5, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment has been minimized.

This comment was marked as outdated.

This comment was marked as outdated.

Nashenas88 commented Sep 10, 2025

Uh oh!

Nashenas88 commented Sep 10, 2025

Uh oh!

Nashenas88 Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

tgross35 Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

madsmtm left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

maurer commented Sep 12, 2025

Uh oh!

Uh oh!

davidtwco commented Sep 17, 2025

Uh oh!

bors commented Nov 26, 2025

Uh oh!

rustbot commented Dec 4, 2025

Uh oh!

tgross35 commented Dec 4, 2025

Uh oh!

This comment has been minimized.

This comment was marked as outdated.

This comment was marked as outdated.

tgross35 commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgross35 commented Dec 4, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Enable `outline-atomics` by default on more AArch64 platforms #144938

Enable `outline-atomics` by default on more AArch64 platforms #144938

tgross35 commented Aug 5, 2025 •

edited

Loading

madsmtm left a comment •

edited by rustbot

Loading

tgross35 commented Dec 4, 2025 •

edited

Loading