Definitely some good points here. Using a single primary database seems easier for a lot of more straightforward use cases, and read replicas are probably sufficient for read-heavy workloads that can tolerate some loss of consistency.
I think either the Cloudflare Workers team or the Fly folks have talked about how they envision edge apps as sharding data by region or by tenant, then sticking each shard in the optimal location. That sounds potentially useful for some cases, but then there's the complexity of sharding to deal with.
We host applications that do sharding like this, but when we build our own stuff at Fly.io, or publish tooling like LiteFS, we're generally not thinking about sharding right now. We like things that are easy to keep our heads around.
We're a hosting provider, not really a database provider. Systems that do sharding, Raft consistency, and stuff like that are built on top of systems like ours, not so much inside of them. :)
Gotcha, that makes sense; I'm probably thinking of Cloudflare Workers, then. It might've come up from kentonv's comments in the recent post about Cap'n Proto 1.0.
Hmm, not in that thread, but maybe in a thread about D1 or Durable Objects? We are indeed trying to make it as easy as possible for applications to shard databases by user or by "document".
There's various options... my current direction is CockroachLabs for the DB side... but Cloudflare offers D1 (sqlite interface over durable workers/kv) which is kind of what you are talking about in practice.
Actual data sharding, without persistence in different regions is more difficult. Different database systems lend themselves differently for different loads and sharding characteristics... I've seen MS-SQL in master-master across the globe, postgresql with sharding that is replicated to base host, and various ScyllaDB/Cassandra setups. It's always a mixed bag.
I think either the Cloudflare Workers team or the Fly folks have talked about how they envision edge apps as sharding data by region or by tenant, then sticking each shard in the optimal location. That sounds potentially useful for some cases, but then there's the complexity of sharding to deal with.