Prebuilt binaries to avoid requiring Rust tooling everywhere

[wraithan]

Given the change over to using rust, and the lack of rust tooling in most CI infrastructure by default, it might be nice to provide prebuilt binaries.

There are a few solutions to this, https://github.com/prebuild/prebuildify appears to be one of the most modern solutions to this problem, it includes the prebuilt binaries in the package so no extra downloads needed. https://github.com/prebuild/prebuild-install is an older solution but I've seen it used pretty successfully, artifacts are separate and you use github releases to host them. Since you've already moved to github actions, it should make the configuration pretty simple, if the artifacts end up hosted in the github releases. I've not looked into if there are any special considerations for doing this with Neon.

If this is interesting to you, I can spend some time trying to make this work and provide a PR within the next 2-3 weeks.

[magiclen]

I think this a great idea. Any PR is welcome!

[rafaelmaeuer]

Using prebuild binaries would probably solve #8 too.

[rafaelmaeuer]

@wraithan how are your plans regarding this? If you don't plan a PR soon, I would give it a try...

[rafaelmaeuer]

@wraithan are you still there? Any comment on this?

[wraithan]

Sorry, I've been busy at work and these notifications went to my personal email (since they weren't on work repo) and I missed them. We lowered the priority of moving to node-crc@2 and have been digging in to a few other things lately. I don't have any code to share, I only got as far as playing with prebuildify to learn a bit before I tried to get it working on this codebase.

[rafaelmaeuer]

I also gave prebuildify a short try today, but getting this error when running prebuildify --all --strip:

node:events:371
      throw er; // Unhandled 'error' event
      ^

Error: spawn node-gyp ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:480:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on ChildProcess instance at:
    at Process.ChildProcess._handle.onexit (node:internal/child_process:288:12)
    at onErrorNT (node:internal/child_process:480:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn node-gyp',
  path: 'node-gyp',
  spawnargs: [
    'rebuild',
    '--target=5.0.0',
    '--devdir=/var/folders/79/20rjf5910x9c32y41kvvtcv80000gn/T/prebuildify/node',
    '--target_arch=x64',
    '--release'
  ]
}

[rafaelmaeuer]

So I finally found the time to search for a solution, first I was playing around with napi-rs. It was possible to produce a prebuild binary which can be used with require (see rafaelmaeuer/node-crc/tree/dev/napi).

Then I found out, that cargo already produces the prebuild-binary index.node which can be used as main file in package.json. Mocca-Tests are still passing and I tested it in docker without the need of installing the rust toolchain.

I have made the following PR, please review and test it: #12

[rafaelmaeuer]

Just found this error in github-actions:

Error: /home/runner/work/***/***/node_modules/node-crc/index.node: invalid ELF header

Seems like pre-compiling need to handle all the possible OS (win/mac/linux) and platform types (x86/x64/arm64/aarch64).

[rafaelmaeuer]

After learning about the necessity of cross-platform binaries (invalid ELF header in Github actions) I ended up writing a build-script based on cross-rs and rustup/cross-compilation which can produce binaries for following platforms:

• macOS x64
• macOS ARM64
• Linux x64 (gnu/musl)
• Linux x86 (gnu/musl)
• Linux ARM64 (gnu/musl)
• Windows x64
• Windows x86

I slightly modified the lib to load the correct binary on each platform, I updated #12 accordingly.

[rafaelmaeuer]

@magiclen would you mind reviewing or commenting on it?