Gwil's garden

Enjoying the morning.

Where data lives

July 3rd, 2021

For the past year I've been a contributor to Earthstar, a pocket-sized peer-to-peer library. With it, you can take ownership of your data in a very meaningful way.

Friends and family naturally want to know why I care about this enough to devote so much time to it. And wrapped up in that question is: why should they care?

Ownership — in this case, data ownership — entails a certain amount of responsibility. But like owning a house, it also means you get to have everything the way you want it. And you get a kind of insulation from outside parties — e.g. landlords — who might want to interfere with your sweet setup.

It is very empowering. Below I'll compare the shapes of three networks: Facebook, Mastodon, and Earthstar, and explore how their power dynamics are defined by their shapes.

Facebook — a typical centralised network

When you think about centralised networks, you think about Facebook.

To do anything with their data, Alfie, Betty and Gemma must do it through machines Facebook owns, which is operated by a guy called “Mark”.

“Mark” has a lot of responsibilities. His users can only hope that their personal data is not being pawned off, that their communities will not be deleted, that they will be protected from bad actors, and that the apps they use to interact with their data will not take advantage of them.

We are counting on you to do the right thing, “Mark”! 🤞

Mastodon - the federated network

As we all began to realise that “Mark” could not be trusted, federated services like Mastodon appeared.

With Mastodon, our users' data is split between many servers ('instances'), operated here by Dalton and Epson. These servers connect to each other to exchange data — in Mastodon's case, tweet-length messages.

Users like Alfie can now choose which server to store their data with, and he has entrusted Dalton with data stewardship, technical maintenance and doing critical social work like moderation. Alfie trusts Dalton more than “Mark”, that's for sure!

But we must hope that operators like Dalton don't burn out while managing their server, forcing Alfie and Betty to find a new online home.

And that exchanging tweet-length messages is the only thing everyone wants to do for the foreseeable future.

And that everyone always wants to be connected to the internet. 🤞

Earthstar - Peer-to-peer

In a peer-to-peer system, users host the network's data on their own devices and connect directly to one another, and no peer has authority over another.

Earthstar is such a system. The diagram above represents a group of Earthstar users.

So what's everyone up to?

The kinds of things these users are doing — deciding how their data flows through the network, who they entrust it with, and what that data even is — are impossible with centralised and federated services.


Another big difference you'll notice is all the attractively-coloured bean shapes encirling our users. With Facebook and Mastodon, everyone existed in one big blue circle, a single universe of data. In Earthstar, you can make your own different universes, identified with names like +bookclub.a7uohu or +cooking.c8aoc7. Data from one universe cannot cross into another!

So Alfie is a member of the blue and red namespaces, while Gemma is only interested in the green namespace, which she shares with Betty. Alfie doesn't even know the green universe exists. Now that's privacy.

Data how you want it

Another consequence of having no centralised authority over data is that a network's data can be whatever its users want it to be. Not just tweets, but wikis, diagrams, spreadsheets, anything! Maybe even several formats of each. As long as someone's able to interpret and use it, it's worth the space it takes up. You don't need to design generic implementations that can serve a thousand to a billion people adequately, because you can design a bespoke implementation that will work really well for a few people.

Redundant infrastructure

Because Earthstar's network is all about replicating data from one peer to another, there are no authoritative servers keeping everything going. Alfie and Betty will be able to communicate with each other even if Dalton decides he doesn't care anymore. They could sync directly with one another, or even spin up their own little replica servers. Because the replica servers have no special authority, the only overhead they come with is keeping them online.

Speaking of which.

The cost of ownership

The trade we make for this kind of power is to take full responsibility. We are encouraged to believe that this responsibility is onerous, and not worth our time. Leave the decisions to those who know best, and enjoy the ride!

Probably sound advice if you're creating services for billions of people.


And if at the end of it, the data you made together was yours to do with as you wanted, forever?

That's the kind of thing that I hope Earthstar will help people do, and why I care about it. And also why I use it to power this blog, but that's a story for another day!