-
Start with
github:cpick/nix-rosetta-builder.- You might be interested in my fork; it targets Linux 6.12.29, applies Apple’s “Accelerating the performance of Rosetta” kernel patch, and incorporates upstream PR #34 (why not).
- I add a
rosettaAotoption to runrosettad, but don’t use it. The results crash far too often. See e.g..
-
Serving suggestion (adjust for your machine):
nix-rosetta-builder = {enable = true;cores = 12;diskSize = "100GiB";memory = "16GiB";onDemand = true;};That’s it! It’ll spin itself up when needed (you might need to retry the build since the first attempt might timeout while it starts up on-demand), and down again after a while of inactivity. Now you can do something like
nix build .#packages.x86_64-linux.chogand it’ll Just Work™. -
Whoops! I lied. You’ll get perhaps a lot of things like this:
error: build of '/nix/store/9dhn75m3isw55qz29dzw1h8xzs634pim-salchicha-0.4.0.drv' on 'ssh-ng://rosetta-builder' failed: builder for '/nix/store/9dhn75m3isw55qz29dzw1h8xzs634pim-salchicha-0.4.0.drv' failed with exit code 132;last 9 log lines:> Running phase: unpackPhase> unpacking source archive /nix/store/l05szcgqqaxxc9i1hd16ik8kd9vv5cn2-salchicha-0.4.0> source root is salchicha-0.4.0> Running phase: patchPhase> Running phase: updateAutotoolsGnuConfigScriptsPhase> Running phase: configurePhase> Running phase: buildPhase> Compiling 3 files (.ex)> /nix/store/qjjpd1310d6qi63pdmjigykcznfi3n4y-stdenv-linux/setup: line 1765: 44 Illegal instruction (core dumped) mix compile --no-deps-checkFor full logs, run:nix log /nix/store/9dhn75m3isw55qz29dzw1h8xzs634pim-salchicha-0.4.0.drverror: builder for '/nix/store/9dhn75m3isw55qz29dzw1h8xzs634pim-salchicha-0.4.0.drv' failed with exit code 1error: build of '/nix/store/3jfv7sghsansgg4sgihqkcsxg925zfrj-jason-1.4.4.drv' on 'ssh-ng://rosetta-builder' failed: builder for '/nix/store/3jfv7sghsansgg4sgihqkcsxg925zfrj-jason-1.4.4.drv' failed with exit code 134;last 12 log lines:> Running phase: unpackPhase> unpacking source archive /nix/store/km205nlk52awji63d5hynwnypdpfqqm3-jason-1.4.4> source root is jason-1.4.4> Running phase: patchPhase> Running phase: updateAutotoolsGnuConfigScriptsPhase> Running phase: configurePhase> Running phase: buildPhase> Compiling 10 files (.ex)> no next heap size found: 2305825417165001762, offset 0>> Crash dump is being written to: erl_crash.dump...done> /nix/store/qjjpd1310d6qi63pdmjigykcznfi3n4y-stdenv-linux/setup: line 1765: 44 Aborted (core dumped) mix compile --no-deps-checkFor full logs, run:nix log /nix/store/3jfv7sghsansgg4sgihqkcsxg925zfrj-jason-1.4.4.drverror: builder for '/nix/store/3jfv7sghsansgg4sgihqkcsxg925zfrj-jason-1.4.4.drv' failed with exit code 1
Just give me the fix.
Assuming you have pkgs from somewhere:
let
erlang = pkgs.beam_minimal.interpreters.erlang_27;
erlang-jmsingle =
(erlang.override {
patches = [
./nix/erlang_beam_jit_main.cpp.patch
];
}).overrideAttrs
(prev: {
preConfigure = ''
${prev.preConfigure or ""}
configureFlagsArray+=(CFLAGS="-O2 -g -DFORCE_ERTS_JIT_SINGLE_MAP=1")
'';
});
mkPackageWithErlang =
erlang:
let
beamPackages = pkgs.beam_minimal.packagesWith erlang;
elixir = beamPackages.elixir_1_18;
in
pkgs.callPackage ./nix/package.nix {
inherit
beamPackages
erlang
elixir
;
};
Note that -O2 -g are defaults for the Erlang build — we have to specify them again since we’re overriding CFLAGS as a whole.
Here’s the patch:
diff --git a/erts/emulator/beam/jit/beam_jit_main.cpp b/erts/emulator/beam/jit/beam_jit_main.cpp
index 8408b25056..d941532e5f 100644
--- a/erts/emulator/beam/jit/beam_jit_main.cpp
+++ b/erts/emulator/beam/jit/beam_jit_main.cpp
@@ -188,6 +188,8 @@ static JitAllocator *pick_allocator() {
* 64-bit x86 still uses dual-mapped memory as it lacks support for per-
* thread permissions and thus gets unprotected RWX pages with MAP_JIT. */
erts_jit_single_map = 1;
+#elif defined(FORCE_ERTS_JIT_SINGLE_MAP)
+ erts_jit_single_map = 1;
#endif
#if defined(HAVE_LINUX_PERF_SUPPORT)
Now you can expose packages like:
packages = rec {
default = chog;
chog = mkPackageWithErlang erlang;
chog-jmsingle = mkPackageWithErlang erlang-jmsingle;
};
Finally, your cross compile is nix build .#packages.x86_64-linux.chog-jmsingle.
???
Let’s read the erts erl command manual together:
+JMsingle true|false- Enables or disables the use of single-mapped RWX memory for JIT code.The default is to map JIT:ed machine code into two regions sharing the same physical pages, where one region is executable but not writable, and the other writable but not executable. As some tools, such as QEMU user mode emulation, cannot deal with the dual mapping, this flags allows it to be disabled. This flag is automatically enabled by the
+JPperfflag.Since: OTP 26.0
“Some tools” also includes Rosetta, it turns out. You’ll find some advice on Internet suggesting you use +JPperf with a variety of options to fix this issue, but it’s all cargo-culted — it’s the side-effect of setting +JMsingle true that makes it work. The above patch causes the flag to be set unilaterally.
The solution I present here has downsides:
- You’re no longer using separate RW/RX mappings, which has (somewhat theoretical) security implications. RIP points to writeable memory!
- We have to patch Erlang to make this happen consistently! I so wish we didn’t, but I couldn’t figure out a way to get the actual
+JMsingleflag to propagate and be set in every single place necessary — namely, every single Erlang/Elixir/etc. invocation. The result is toolchain bloat. - The resulting builds therefore differ from those without it.
I use this for testing and building x86_64 artifacts from the comfort of my laptop, but when I actually deploy on x86_64, I build the regular versions on a target machine.
May this help someone!