This is a ground-up rebuild of Earthstar which has been in the making since last April. It brings a lot of new features, the foundations for more to come, and is about as breaking as breaking changes get.
There are new features, APIs, and concepts. It's a lot to take in, so in addition to this new version of Earthstar, we've got a shiny website with explainers, API tours, technical documentation, and more!
So what's new? Lots.
There is a new
Peer class which acts as the point of contact between your
application and Earthstar.
- It's used to store, manage, and access replicas, with methods like
- And can be used to synchronise with other peers locally or over the network
Peer.sync. This method accepts lots of 'syncable' things like another
Peeror a URL to a replica server.
Syncing is now protocol-agnostic. Whereas previously you could only sync locally
or over HTTP, it's now possible to sync over theoretically any transport.
We've included transports for HTTP, Websockets and
BroadcastChannel in our new
earthstar-streaming-rpc. We're to add lots more so that Earthstar can
synchronise in many different contexts, like local networks.
We've also made it so that two Peers can determine which shares they have in common without revealing what those shares actually are. This is thanks to something we call a 'salted handshake', and makes it so that you can freely associate with other Peers without worrying that your share addresses are leaking.
There is also a low-level
Syncer class you can use to handle scenarios and
transports not catered to by the
Peer.sync convenience method, e.g. replica
servers or web workers (I mentioned we added a transport for
Previously data could be stored in two kinds of 'Storage', one with a
synchronous API and one with an asynchronous API. We've replaced both with a
Replica class with an asynchronous API.
The functionality of this new
Replica class is much like the previous storage,
but it is now far easier to write persistence drivers for using the
IReplicaDriver interface. Thanks to this we've been able to add a new
IndexedDBdriver and Sqlite drivers for both Node and Deno.
There are many contexts where an asynchronous API can be a royal pain.
ReplicaCache sticks a synchronous API in front of a backing
keeping a cache of results for queries. The best part is you can subscribe to
events where the cache has been updated, and use this to re-query the cache. We
suspect this will be a very useful tool in contexts like a React app!
There are many applications where you will want to subscribe to new events on a
Replica, such as new documents coming in. Using a
QueryFollower you can
subscribe to events filtered by a
Query. Callbacks triggered by this
abstraction have special qualities like being called blockingly so that they
apply backpressure (this is something that makes certain situations much easier
to deal with).
New default ed25519 driver
Earthstar uses ed25519 for author verification. Previous versions of Earthstar could use Node's native crypto, or chloride for Node or the browser. Chloride is fast, but adds a lot to a browser bundle. TweetNacl (also exposed via Chloride) is much smaller, but very slow in comparison.
We've added a new default driver for ed25519 operations using noble/ed25519. This driver is smaller than libsodium-wrappers and much faster than TweetNacl, and works in all JS runtimes
Earthstar can now be used in Deno projects:
import * as Earthstar from "https://email@example.com/mod.ts";
Deno is a really great way to write and run Typescript, and we think it will make it far easier to write personal scripts and utilities using Earthstar.
It's also worth mentioning that Deno is now our primary development environment, and that we were able to make that switch without sacrificing Node support. Deno's support for web platform APIs and all-in-one tooling is something we hope will make contributing to Earthstar that much easier.
Not interested in Node or Deno? We also have a new web bundle, which you can use
<script type="module"> import * as Earthstar from "https://cdn.earthstar-project.org/js/earthstar.bundle.v7.0.0.js"; </script>
- Thank you to Cinnamon, who led this reworking!
- Thank you to tlowrimore, who contributed a nice template querying feature!
- Thank you to NLnet, who funded my own work on this new release!