reboot wit-bindgen-go

This re-introduces Go support, with a new implementation written from scratch.
While the previous incarnation targeted TinyGo and reused the C generator for
lifting and lowering, this one targets "big" Go and handles lifting and lowering
itself.  In addition, it supports idiomatic, goroutine-based concurrency on top
of the component model async ABI, including streams and futures.

This implementation is also distinct from the
[go-modules](https://github.com/bytecodealliance/go-modules) project, which has
a number of limitations (e.g. no async support, not safe for use with "big" Go)
and is no longer being actively maintained.

Note that the async support currently requires [a small
patch](dicej/go@a1c8322)
to the Go `runtime` library.  I plan to work with the upstream project to make
that patch unecessary in the future.  Components that don't use async features
should work with stock, unpatched Go.

One of the tricky parts about lowering values and passing pointers to
e.g. `{stream,future}.{read,write}` is that we must tell the Go garbage
collector to pin the pointers (i.e. mark the pointee as both immovable and
uncollectable) for as long as the host has access to them, but then unpin them
promptly afterward to avoid leaks.  We do this using `runtime.Pinner` instances,
most of which are locally scoped and easy to reason about.  However, we must use
a couple of globally-scoped pinners as well: one for `cabi_realloc` allocations
by the host when it lowers parameters (which we unpin after lifting those
parameters) and another for returning values from sync-lifted exports (which we
unpin in post-return functions).

There are a couple of known missing features which I'll open GitHub issues for:

1. Resource handles are not being restored for unwritten items in the case of an incomplete stream or future write.
2. Importing and/or exporting multiple versions of the same package will cause name clashes.

In addition, I plan to expand the test coverage beyond the basics covered in
this commit.

Signed-off-by: Joel Dice <joel.dice@fermyon.com>

Author: dicej

Cargo.lock

@@ -48,6 +48,7 @@ wit-bindgen-rust = { path = "crates/rust", version = "0.49.0" }
 wit-bindgen-csharp = { path = 'crates/csharp', version = '0.49.0' }
 wit-bindgen-markdown = { path = 'crates/markdown', version = '0.49.0' }
 wit-bindgen-moonbit = { path = 'crates/moonbit', version = '0.49.0' }
+wit-bindgen-go = { path = 'crates/go', version = '0.49.0' }
 wit-bindgen = { path = 'crates/guest-rust', version = '0.49.0', default-features = false }
 wit-bindgen-test = { path = 'crates/test', version = '0.49.0' }
 
@@ -64,6 +65,7 @@ wit-bindgen-cpp = { workspace = true, features = ['clap'], optional = true }
 wit-bindgen-markdown = { workspace = true, features = ['clap'], optional = true }
 wit-bindgen-moonbit = { workspace = true, features = ['clap'], optional = true }
 wit-bindgen-csharp = { workspace = true, features = ['clap'], optional = true }
+wit-bindgen-go = { workspace = true, features = ['clap'], optional = true }
 wit-bindgen-test = { workspace = true }
 wit-component = { workspace = true }
 wasm-encoder = { workspace = true }
@@ -84,7 +86,7 @@ c = ['dep:wit-bindgen-c']
 cpp = ['dep:wit-bindgen-cpp']
 rust = ['dep:wit-bindgen-rust']
 markdown = ['dep:wit-bindgen-markdown']
-go = []
+go = ['dep:wit-bindgen-go']
 csharp = ['dep:wit-bindgen-csharp']
 csharp-mono = ['csharp']
 moonbit = ['dep:wit-bindgen-moonbit']

Cargo.toml

@@ -806,6 +806,12 @@ pub fn guest_export_needs_post_return(resolve: &Resolve, func: &Function) -> boo
         .unwrap_or(false)
 }
 
+pub fn guest_export_params_have_allocations(resolve: &Resolve, func: &Function) -> bool {
+    func.params
+        .iter()
+        .any(|(_, t)| needs_deallocate(resolve, &t, Deallocate::Lists))
+}
+
 fn needs_deallocate(resolve: &Resolve, ty: &Type, what: Deallocate) -> bool {
     match ty {
         Type::String => true,
@@ -1134,7 +1140,10 @@ impl<'a, B: Bindgen> Generator<'a, B> {
                     for (param_name, ty) in func.params.iter() {
                         let Some(types) = flat_types(self.resolve, ty, Some(max_flat_params))
                         else {
-                            panic!("failed to flatten types during direct parameter lifting ('{param_name}' in func '{}')", func.name);
+                            panic!(
+                                "failed to flatten types during direct parameter lifting ('{param_name}' in func '{}')",
+                                func.name
+                            );
                         };
                         for _ in 0..types.len() {
                             self.emit(&Instruction::GetArg { nth: offset });

crates/core/src/abi.rs

@@ -148,7 +148,7 @@ pub trait InterfaceGenerator<'a> {
     fn type_record(&mut self, id: TypeId, name: &str, record: &Record, docs: &Docs);
     fn type_resource(&mut self, id: TypeId, name: &str, docs: &Docs);
     fn type_flags(&mut self, id: TypeId, name: &str, flags: &Flags, docs: &Docs);
-    fn type_tuple(&mut self, id: TypeId, name: &str, flags: &Tuple, docs: &Docs);
+    fn type_tuple(&mut self, id: TypeId, name: &str, tuple: &Tuple, docs: &Docs);
     fn type_variant(&mut self, id: TypeId, name: &str, variant: &Variant, docs: &Docs);
     fn type_option(&mut self, id: TypeId, name: &str, payload: &Type, docs: &Docs);
     fn type_result(&mut self, id: TypeId, name: &str, result: &Result_, docs: &Docs);

crates/core/src/lib.rs

@@ -0,0 +1,19 @@
+[package]
+name = "wit-bindgen-go"
+edition = "2024"
+version.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version = "1.85.0"
+
+[dependencies]
+wit-bindgen-core = { workspace = true }
+wit-component = { workspace = true }
+wasm-encoder = { workspace = true }
+wasm-metadata = { workspace = true }
+anyhow = { workspace = true }
+heck = { workspace = true }
+clap = { workspace = true, optional = true }
+
+[features]
+clap = ['dep:clap', 'wit-bindgen-core/clap']