When you're working on a lot of interrelated ideas for a long time, I think there's this natural point where, meiosis-like, that body of ideas starts to split and boundaries form around different groups of ideas.

In Earthstar's case, these groups would be:

• Formatting rules for document paths and 'cinn25519' keypairs.
• A sound, multi-writer, mutable data model and replication scheme for distributed systems.

Willow is the name for a new project which will formalise this latter set of ideas. We think it's a compelling new primitive for building distributed systems, and one which we think can supplant the dominant paradigm — the append-only log — for many use-cases.

And I'm delighted to say that I can use the pronoun 'we', as I'll be joined by my esteemed friend Aljoscha Meyer on this project. We'll be working towards three outcomes:

• A specification describing Willow’s data model, storage and replication.
• A reference implementation written in TypeScript.
• Using this implementation as the new core of Earthstar.

Willow won't just be a reshuffling of Earthstar's existing functionality. It will bring new features (like tombstone-less deletion) as well as new efficiencies. It will bring the best things about Earthstar to new use-cases and environments.

We're starting with a TypeScript implementation because a lot of the raw parts we need are already there. But Willow will have many design decisions that a language like JavaScript can't take advantage of, and we want to work on an implementation in a language like Rust in order to do that. But that is a big project which we are seeking support for separately.

I'm very happy and excited to be working on this. I'm particularly happy to be working with Aljoscha again, as we had a lot of fun collaborating on implementing range-based set reconciliation for Earthstar.

I'm also very grateful to have funding dedicated specifically towards writing a specification — many thanks to NLnet for providing that funding and making this kind of foundational work possible.