[POC] Uniffi binding

[kraenhansen]

To be clear: I don't expect this to merge.

I spent roughly 2 days implementing this, because I wanted to:

1. Gain experience with Uniffi & uniffi-bindgen-react-native and a deeper understanding for the Automerge API.
2. Show by example what a Uniffi binding could look like.

What is this? 🤔

An example Expo app located in ./example

import 'react-native-automerge';
import * as Automerge from '@automerge/automerge';

type MyDoc = Record<string, unknown>;

function createAndChange() {
  let doc = Automerge.init();
  doc = Automerge.change<MyDoc>(doc, d => (d.hello = "from automerge"));
  return JSON.stringify(doc);
}

export default function App() {
  const result = createAndChange();
  return (
    <View style={styles.container}>
      <Text>Open up App.tsx to start working on your app!</Text>
      <StatusBar style="auto" />
      <Text>Result: {result}</Text>
    </View>
  );
}

Importing "react-native-automerge" will inject the binding via the use export from @automerge/automerge.

A Uniffi binding located in ./rust/src/lib.rs

This file provides the Rust code backing the implementation of the Automerge class.
It's purpose is equivalent of that of the Automerge WASM binding's ./rust/automerge-wasm/src/lib.rs and this has served as an inspiration for much of the code.

The react-native-automerge package

• A native module located in ./package
• A sparse implementation of the Automerge "lower-level" API located in ./package/src/index.ts (injected into the Automerge SDK by calling the use function).
• An Automerge class located in ./package/src/Automerge.ts

This file provides the Rust code backing the implementation of the Automerge class.
It's purpose is equivalent of that of the Automerge WASM binding's ./rust/automerge-wasm/src/lib.rs and this has served as an inspiration for much of the code.

How does it even work? 🤷

The uniffi-bindgen-react-native tool

1. invokes cargo to build an xcframework containing .a binaries (and .so files in the jniLibs directory for Android) of the Rust code. These binaries externs a bunch of symbols for a C api and they can be prebuilt and included in the archive uploaded to NPM (or downloaded on demand or install - if you'd want to slim down the package size).
2. generates platform specific code (Objective C for iOS and Kotlin for Android) interfacing with React Native to register native TurboModules, eventually calling into generated library specific C++ (see 👇)
3. generates library specific C++ calling into JSI, creating a "HostObject" for the native module and creating "HostFunctions" on it, using the JSI runtime API. This enables JS to call into the C++ and further into the C API by "lowering" inputs and "lifting" outputs (as defined by the Uniffi project).

All of the code in 2 and 3 integrate with React Native Core APIs (either the platform specific APIs to register the TurboModule or the JS runtime via JSI) and it must be built on the developers machine, to use the correct API and ABIs declared by the exact React Native version that the developer is using to build their app.

How do I test / run it? 🏃

Note: I've been using iOS when developing this and I have no idea what the state of the Android build is.

1. Ensure you have the prerequisites for building with Uniffi for React Native, by following uniffi-bindgen-react-native's pre-install guide.
2. Install the root package (npm install).
3. Patch node_modules/@automerge/automerge/package.json by adding this react-native main field "react-native": "./dist/mjs/entrypoints/slim.js",, just below the "main" property.
4. Build the Uniffi binding for iOS (npm run uniffi:ios --prefix package)
5. Build the package's TypeScript (npm run build --prefix package)
6. Build and run the iOS Expo test app (npm run ios --prefix example)
7. Notice "Result: { "hello": "from automerge" }" 🎉

Caveats and drawbacks 🥺

• I've left a lot of code-paths "not yet implemented". Only shallow documents with properties of string value is implemented.
• Error handling in the current implementation is a mess. It's using lots of unwraps, which leads to crashes, instead of using the Result + Err pattern which would give more pleasant runtime errors.
• Uniffi bindings doesn't have access to JS APIs, this is specifically a drawback when the binding wants to:
  • Return a "JS Value" of unknown type, i.e. when the Rust code doesn't know ahead of time what type of primitive or object will be returned. The most obvious example of this is the materialize function, which currently returns a Map<string, string> but would have to return some "wrapper" which would be called to construct arbitrary JS values on the JS-side - not ideal is this will likely bring significant overhead for large documents.
  • Reflect about objects, specifically get / set values on objects passed into the binding. The most obvious example of this is the applyAndReturnPatches function, which in the WASM binding mutates the object in Rust code (calling out to the JS runtime via the wasmbindgen's runtime). This is not possible in the Uniffi binding and instead diffIncremental() is called and then the patches are applying on the JS-side. I actually suspect this might be faster than what the WASM binding is doing, because it doesn't need a call from native to JS on every change to the doc.

Key takeaways 💡

• I believe implementing a complete binding of Automerge for React Native via Uniffi is feasible.
• Suggestion for the future: I'd like to see more code being moved from the WASM Rust code into the JS SDK code. I.e. less use of reflection and instead return immutable data from Rust. This way more code could be shared between the different bindings.

[alexjg]

This is very impressive! I think the code itself here is very nice, there's not much boilerplate and the wrappers are close enough to the automerge-wasm wrappers that I think it would be straightforward to maintain. We can probably make the integration easier from the automerge side by looking at the issues you've raised (though I should note that in our experiments it was counterintuitively slower to run patches in JS than Rust).

My main worries are not about the code here, but about the state of the Uniffi react native project. It's currently pretty early with warnings all over its documentation to not use it in production. The project itself seems to be mostly the project of one company at the moment - that company has committed a lot of resources to the project but if they were to go away for some reason it's not clear to me what would happen. That means if we go ahead and use react native uniffi in this project then we are signing up for an unknown amount of maintenance work keeping up with changes in the uniffi project. That's not to say we shouldn't do it, just that we should think about who is going to be doing that maintenance. It might also be that we want to say that this project is also an experimental one which won't necessarily be maintained going forward.

The alternative - which is to use raw cbindgen and C++ might be a lot more boilerplate and work up front, but cbindgen is not going to be changing any time soon and so maybe we have less ongoing maintenance work to do.

I have a slight preference for the uniffi approach. I think it will get us something which works faster than the cbindgen approach and I think most of the work involved is work we will also have to do for cbindgen based bindings anyway, so if we find that uniffi is too much work we probably haven't lost too much. I am worried about having another giant build tool in the mix though (especially as application developers have to install it and run it, this will be a frequent source of issues if we go down this route). Hopefully once the turbo ABI is stabilized we can distribute prebuilt packages directly to mitigate that but I think we should also always keep half an eye on how we would eject to cbindgen.