Skip to content

Commit

Permalink
Merge pull request #67 from automerge/prettier
Browse files Browse the repository at this point in the history
Prettier
  • Loading branch information
HerbCaudill authored Mar 19, 2024
2 parents 9ff7090 + 9a24f19 commit f95396d
Show file tree
Hide file tree
Showing 37 changed files with 639 additions and 638 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: github pages
on:
push:
branches:
- main # Set a branch to deploy
- main # Set a branch to deploy
pull_request:

jobs:
Expand All @@ -14,8 +14,8 @@ jobs:

- uses: actions/setup-node@v2
with:
node-version: '18'
- run: 'git submodule init && git submodule update'
node-version: "18"
- run: "git submodule init && git submodule update"
- run: npm install
- run: npm run build

Expand Down
2 changes: 1 addition & 1 deletion babel.config.js
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module.exports = {
presets: [require.resolve('@docusaurus/core/lib/babel/preset')],
presets: [require.resolve("@docusaurus/core/lib/babel/preset")],
};
34 changes: 17 additions & 17 deletions blog/2023-01-17-automerge-2/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,13 @@ With Automerge 2.0 we've made a big investment in improving documentation. In ad

## Supporting Automerge

Those who have been following Automerge for a while may have noticed that we describe Automerge 2.0 as our first *supported* release. That’s because as part of the Automerge 2.0 release we’ve brought Alex Good onto the team full-time to provide support to external users, handle documentation, release management, and—of course—to continue implementing new Automerge features for the community.
Those who have been following Automerge for a while may have noticed that we describe Automerge 2.0 as our first _supported_ release. That’s because as part of the Automerge 2.0 release we’ve brought Alex Good onto the team full-time to provide support to external users, handle documentation, release management, and—of course—to continue implementing new Automerge features for the community.

This is a big moment for Ink & Switch and the Automerge project: we’re now able to provide support to our users thanks to sponsorship from enterprises like [Fly.io](https://fly.io/), [Prisma](https://www.prisma.io/), and [Bowtie](https://bowtie.works/) as well as so many others who have contributed either directly to Automerge or through supporting Martin Kleppmann on Patreon.

If your business is interested in sponsoring Automerge, you can [sponsor us directly](https://github.com/sponsors/automerge), or [get in touch with us for more information or other sponsorship methods](mailto:[email protected]). Every little bit helps, and the more sponsors we have, the more work we can do while still remaining an independent open source project.

> At Bowtie we support Automerge because it's the best way to achieve the resilliency properties that we're delivering to globally distributed private networks. It's clear to me that our sponsorship has furthered our software, and that this crew are among the best distributed-systems thinkers in the business.
> At Bowtie we support Automerge because it's the best way to achieve the resilliency properties that we're delivering to globally distributed private networks. It's clear to me that our sponsorship has furthered our software, and that this crew are among the best distributed-systems thinkers in the business.
> -- Issac Kelly, CTO, Bowtie.
## Performance: Speed, Memory and Disk
Expand All @@ -58,29 +58,29 @@ With Automerge 2.0, we've brought together an efficient binary data format with

Let's take a look at some numbers. One of the most challenging benchmarks for CRDTs is realtime text collaboration. That's because a long editing session can result in hundreds of thousands of individual keystrokes to record and synchronize. Martin Kleppmann recorded the keystrokes that went into writing an academic paper and replaying that data has become a [popular benchmark](https://github.com/automerge/automerge-perf) for CRDTs.

| Insert ~260k operations | Timing (ms) | Memory (bytes) |
| ---------------------------- | ----------- | -------------- |
| Automerge 0.14 | ~500,000 | ~1,100,000,000 |
| Automerge 1.0.1 | 13,052 | 184,721,408 |
| Automerge 2.0.1 | 1,816 | 44,523,520 |
| Yjs | 1,074 | 10,141,696 |
| Automerge 2.0.2-unstable | 661 | 22,953,984 |
| Insert ~260k operations | Timing (ms) | Memory (bytes) |
| ------------------------ | ----------- | -------------- |
| Automerge 0.14 | ~500,000 | ~1,100,000,000 |
| Automerge 1.0.1 | 13,052 | 184,721,408 |
| Automerge 2.0.1 | 1,816 | 44,523,520 |
| Yjs | 1,074 | 10,141,696 |
| Automerge 2.0.2-unstable | 661 | 22,953,984 |

Of course, even the most productive authors struggle to type an entire paper quite so quickly. Indeed, writing a paper can occur over months or even years, making both storage size on disk and load performance important as well.

| Size on Disk | bytes |
| -------------- | ----------- |
| plain text | 107,121 |
| automerge 2.0 | 129,062 |
| automerge 0.14 | 146,406,415 |
| automerge 0.14 | 146,406,415 |

The binary format works wonders in this example, encoding a full history for the document with only 30% overhead. That's less than one additional byte per character! The naive JSON encoding often used circa automerge 0.14 could exceed 1,300 bytes *per character*. If you'd like to learn more about the file format, we have a [specification](https://alexjg.github.io/automerge-storage-docs/) document.
The binary format works wonders in this example, encoding a full history for the document with only 30% overhead. That's less than one additional byte per character! The naive JSON encoding often used circa automerge 0.14 could exceed 1,300 bytes _per character_. If you'd like to learn more about the file format, we have a [specification](https://alexjg.github.io/automerge-storage-docs/) document.

| Load ~260k operations | Timing (ms) |
| ------------------------ | ------------ |
| Automerge 1.0.1 | 590 |
| Automerge 2.0.1 | 593 |
| Automerge 2.0.2-unstable | 438 |
| Load ~260k operations | Timing (ms) |
| ------------------------ | ----------- |
| Automerge 1.0.1 | 590 |
| Automerge 2.0.1 | 593 |
| Automerge 2.0.2-unstable | 438 |

Loading the compressed document is fast as well, ensuring the best possible start-up time.

Expand Down Expand Up @@ -128,7 +128,7 @@ While we retain the full history of Automerge documents and provide APIs to acce

## History Management

Today the best way to remove something from an Automerge document's history is to recreate the document from scratch or to reset to a time before that change went in. In the future, we plan to provide additional tools to give developers more control over document history. We expect this to include the ability to share just the latest version of a document (similar to a shallow clone in `git`), and to share updates that bypass changes you don't want to share (as when a developer squashes commits before publishing).
Today the best way to remove something from an Automerge document's history is to recreate the document from scratch or to reset to a time before that change went in. In the future, we plan to provide additional tools to give developers more control over document history. We expect this to include the ability to share just the latest version of a document (similar to a shallow clone in `git`), and to share updates that bypass changes you don't want to share (as when a developer squashes commits before publishing).

# Conclusion

Expand Down
7 changes: 4 additions & 3 deletions blog/2023-11-06-automerge-repo.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ The Automerge project provides a public sync server for you to experiment with `

:::


Next, create a document and make some changes to it:

```
Expand All @@ -40,6 +39,7 @@ Next, create a document and make some changes to it:
> console.log(handle.url)
automerge:2j9knpCseyhnK8izDmLpGP5WMdZQ
```

The code logs a URL to the document you created. On another computer, or in another browser, you could load this document using the same URL, as shown below:

```
Expand Down Expand Up @@ -92,7 +92,7 @@ The examples below illustrate asynchronously loading a document or synchronously
> console.log(doc)
```

Or
Or

```
> const handle = repo.find("automerge:2j9knpCseyhnK8izDmLpGP5WMdZQ")
Expand Down Expand Up @@ -166,6 +166,7 @@ Note that when `Repo` receives changes over the network or registers local chang
```

## What about <X>?

We'd love to help you make automerge work in your favorite development environment! Please reach out to us on GitHub or via [our Slack](https://join.slack.com/t/automerge/shared_invite/zt-e4p3760n-kKh7r3KRH1YwwNfiZM8ktw).

## Extending `automerge-repo`
Expand Down Expand Up @@ -195,7 +196,7 @@ This release of `automerge-repo` is just for javascript. Automerge is a multi-la
1. Documents with large histories (e.g. a collaboratively edited document with >60,000 edits) can be slow to sync.
2. The sync protocol currently requires that a document it is syncing be loaded into memory. This means that a sync server can struggle to handle a lot of traffic on large documents.

These two points mean that we're not ready to say this project is ready for production.
These two points mean that we're not ready to say this project is ready for production.

We're working hard on fixing the performance so that we _can_ say this is ready for production. But if you are interested in experimenting with the library now, or if you are only going to be working with relatively small documents or low traffic sync servers then you are good to go!

Expand Down
33 changes: 16 additions & 17 deletions blog/2023-12-21-merry-commitmas.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

Since [releasing](./2023-11-06-automerge-repo.md) automerge-repo last month, we've been working closely with our users to improve the library based on real-world usage. One in-house project, the [Tiny Essay Editor](https://github.com/inkandswitch/tiny-essay-editor), is a Markdown editor with comment support which was used to write the [latest Ink & Switch essay, Embark](https://www.inkandswitch.com/embark/).

Like many Ink & Switch essays, Embark is a large piece: the final version is over 11,000 words and 60,000 characters. The full edit history is just shy of 200,000 edits. That means the team produced roughly 3x as much text as made it into the final version. The final version of the document with the full history and all the comments included is only 376kb and takes a little under 4s to load from disk, but from then on edits are reasonably snappy: most edits take 25ms (equivalent to 30fps) from keypress to paint on my desktop.
Like many Ink & Switch essays, Embark is a large piece: the final version is over 11,000 words and 60,000 characters. The full edit history is just shy of 200,000 edits. That means the team produced roughly 3x as much text as made it into the final version. The final version of the document with the full history and all the comments included is only 376kb and takes a little under 4s to load from disk, but from then on edits are reasonably snappy: most edits take 25ms (equivalent to 30fps) from keypress to paint on my desktop.

There's still plenty of room to improve here. Ultimately our goal is to reliably achieve single-frame updates even on very large documents and we still have a long way to go on memory usage. Still, we thought folks might enjoy hearing a little behind-the-scenes description of what we've been up to.
There's still plenty of room to improve here. Ultimately our goal is to reliably achieve single-frame updates even on very large documents and we still have a long way to go on memory usage. Still, we thought folks might enjoy hearing a little behind-the-scenes description of what we've been up to.

But first, a few feature updates:

## CodeMirror Integration

Tiny Essay Editor is built around the [automerge-codemirror](https://github.com/automerge/automerge-codemirror) integration and uses incremental updates to make sure it stays fast even on extremely large documents. We've managed to maintain next-frame performance for most edits and document sizes, but on very large documents we still have a few stalls caused by calculating network synchronization messages to work through.
Tiny Essay Editor is built around the [automerge-codemirror](https://github.com/automerge/automerge-codemirror) integration and uses incremental updates to make sure it stays fast even on extremely large documents. We've managed to maintain next-frame performance for most edits and document sizes, but on very large documents we still have a few stalls caused by calculating network synchronization messages to work through.

That said, the CodeMirror integration is stable, efficient, and works well with both the marks and cursors APIs. If you need a well-supported plaintext editor (or want a reference to write your own integration for your favorite editor) start here. ProseMirror integration is coming too, more about that after Christmas.

Expand All @@ -23,20 +23,20 @@ The reason is that the interface Automerge exposes for modifying text is `Autome
Figuring out what changed between two strings is actually quite fiddly. There are algorithms you can study, such as the [Myers diff](https://www.nathaniel.ai/myers-diff/), and libraries [that implement them](https://www.npmjs.com/package/myers-diff)... but we decided that it would be worthwhile to just build one into Automerge and spare you the hassle. We've therefore introduced a function `Automerge.updateText`, which looks like this:

```javascript
let doc1 = Automerge.from({ text: "Hello world!" })
let doc1 = Automerge.from({ text: "Hello world!" });

let doc2 = Automerge.clone(doc1)
doc2 = Automerge.change(doc2, d => {
/// Note we just pass the new value we want the whole field to have
Automerge.updateText(d, ["text"], "Goodbye world!")
})
let doc2 = Automerge.clone(doc1);
doc2 = Automerge.change(doc2, (d) => {
/// Note we just pass the new value we want the whole field to have
Automerge.updateText(d, ["text"], "Goodbye world!");
});

doc1 = Automerge.change(doc1, d => {
Automerge.updateText(d, ["text"], "Hello friends!")
})
doc1 = Automerge.change(doc1, (d) => {
Automerge.updateText(d, ["text"], "Hello friends!");
});

/// text is now "Goodbye friends!"
const merged = Automerge.merge(doc1, doc2)
const merged = Automerge.merge(doc1, doc2);
```

This approach is really handy for places like form fields where a full rich-text editor would be overkill, but isn't as efficient at capturing inputs, particularly for larger documents. Let us know how it works for you!
Expand All @@ -55,12 +55,12 @@ Well, notice that I said loading the _editor_ took 40s. In profiling this proble

We solved this in a straightforward way: when a peer requests a document it doesn't have at all, we skip the elaborate sync protocol and simply send the whole compressed document. Future synchronizations are very fast: the peers remember their last sync state and can quickly calculate the comparison

Unfortunately, even loading the compressed version of this document was much slower than we expected: it was taking somewhere around 5s. That's about 5s too long.
Unfortunately, even loading the compressed version of this document was much slower than we expected: it was taking somewhere around 5s. That's about 5s too long.

Investigating where the time was being spent we spotted a few performance problems, including:

* Automerge stores operations internally in a B-tree, which has a vector of operations on each node. We were losing some time allocating these little vectors every time we received a new change.
* Each node in the B-tree has an index on it, where we store things like the number of ops and the number of characters in its subtree. When loading a document we were updating these indexes for every edit in the document's history.
- Automerge stores operations internally in a B-tree, which has a vector of operations on each node. We were losing some time allocating these little vectors every time we received a new change.
- Each node in the B-tree has an index on it, where we store things like the number of ops and the number of characters in its subtree. When loading a document we were updating these indexes for every edit in the document's history.

We solved these problems by making several changes:

Expand All @@ -72,4 +72,3 @@ Putting this all together, the load time for the Embark essay is reduced to arou
# That's it!

You can get all this good stuff by updating to Automerge 2.1.10 or later, as well as plenty of smaller improvements (like import/export) and bug fixes (like getting rid of a React hook race condition).

14 changes: 7 additions & 7 deletions docs/api.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
sidebar_position: 1000
---

# API Docs

* [`@automerge/automerge`](https://automerge.org/automerge/api-docs/js)
* [`@automerge/automerge-repo`](https://automerge.org/automerge-repo/)
- [`@automerge/automerge`](https://automerge.org/automerge/api-docs/js)
- [`@automerge/automerge-repo`](https://automerge.org/automerge-repo/)

# Other Languages

Here you can find links to the APIs for each implementation.

* [C](https://github.com/automerge/automerge/tree/main/rust/automerge-c)
* [Rust](https://github.com/automerge/automerge-rs)
* [Python](https://github.com/automerge/automerge-py)
* [Swift (macOS, iOS)](https://automerge.org/automerge-swift/documentation/automerge/)

- [C](https://github.com/automerge/automerge/tree/main/rust/automerge-c)
- [Rust](https://github.com/automerge/automerge-rs)
- [Python](https://github.com/automerge/automerge-py)
- [Swift (macOS, iOS)](https://automerge.org/automerge-swift/documentation/automerge/)
1 change: 0 additions & 1 deletion docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ This documentation is mostly focused on the javascript implementation of automer

:::


## Core concepts

Using automerge means storing your data in automerge [documents](#Documents). Documents have a [URL](#Document-URLs)s which you can use to share or request documents with/from other peers using a [repository](#Repositories). Repositories give you [`DocHandle`](#dochandles)s which you use to make changes to the document and listen for changes from other peers.
Expand Down
2 changes: 1 addition & 1 deletion docs/cookbook/_category_.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"label": "Cookbook",
"position": 5
}
}
Loading

0 comments on commit f95396d

Please sign in to comment.