Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Off topic but... I had a friend who's main takeaway (tongue-in-check) from harvard mini-mba while he is in a SEAS program was that all you had to do to look smart around VC/tech-oriented business people was to ask this question.
Then we actually founded a startup and realized this was not a silly question. But perhaps that's circular when the default path is VC funding, hmmm.
by chermi - Me and Fable when the boss starts asking why my custom handrolled data engine needs to run on m6i.16xlarge for our 100 users.
- Nice! I like learning about databases but got a bit of "framework fatigue" when I started reading about all the number of dbs available today (and deciphering marketing from tech notes).
This article hooked me with the comparison at the beginning.
I used to scoff at redis' single threaded design but it makes sense in a memory bound db. This article is a great example of taking that high-performance approach and designing parallelization around it. SO COOOL.
Also, it's SO interesting that here's yet another example of how performant the actor model can be. It's an old design (Communicating Sequential Processes was published in 1984!) but it works so well in our current hardware.
From a developer's perspective actors are very easy to reason about. I'm curious to try out Spacetime in a project now. Organizing server logic into databases, sub-databases, tables and reducers is intriguing.
by gandreani - I had a coworker that would always ask this, like a javelin thrown in the bicycle spokes of every demo.
We hoped for tens of users.
by randusername - > You get to deploy your server logic directly into the database
> You may make use of the Licensed Work provided your application or service uses the Licensed Work with no more than one SpacetimeDB instance in production and provided that you do not use the Licensed Work for a Database Service.
Therefore, as an OSS product, SpacetimeDB does not scale.
by themgt - The intro section is a good summary of why distributed SQL databases (Spanner, roach, Yugabyte, TiDB) haven't taken off in the market in the same way as say distributed data warehouses have (Snowflake, Databricks, FabricDW, Clickhouse, etc.).
I would add a few other things to the list of scaling problems. Some SQL features are hard to scale out (auto_increment/serial columns, unique secondary keys, foreign keys, etc.). Some SQL query operators are hard to scale out for OLTP queries that want low latency and high throughput (DISTINCT, LIMIT/TOP-N, non-collocated joins). I take it SpacetimeDB is a nosql database, so these problems are less important to them?
As to how spacetimedb plans to scale out, I didn't follow it fully. It's hard to take the spacetimedb folks seriously (see: https://strn.cat/posts/spacetime/).
by AdamProut - As someone who has been using alot of spacetime for side projects (like https://heat.echohack.app), I am continually impressed with the speed at which it operates.
I think there's alot of interesting things happening in the database space. Vitess/Neki, vector stores, spacetime are all really good things to be happening. I think it's a shame that database developers seem to have a drama filled timeline out there. It's... all very exciting, together.
Anyway, Some things that need improvement (some of which are addressed by this blog post):
1. Backups (fast recovery) and Disaster Recovery (slow, durable recovery)
This is a big one, but sometimes speed is not the only objective you have to meet. You need to have certainty that, if everything goes down that you (eventually) can bring things back online. I don't really have a way of doing that today.
2. Read replicas sure would be nice for analytic workloads
3. Durable writes to s3 would be nice for intermittent bursting workloads (like ci systems)
I think the Spacetime folks have their work cut out for them, not necessarily because of the technology (that's hard too) but because the AI models have seemingly decided that Neon and Postgres are all that exists.
I think spacetime has a bright future ahead of it, and I am wishing the team all the best as they work hard to imprint something new on the universe.
by echohack5 - Spacetime sounds like really interesting technology, but I'm not sure that the comparison between CRDB is a good one.
I used to work at Cockroach Labs. The problem it's solving is fundamentally different. CRDB as a solution makes sense when you need to _guarantee_ that transactions are serializable and durable, and that your application can survive node or region failures while maintaining consistency. In a naive deployment it's significantly slower than operating on a single core, but that's the price that you pay for the ability to survive node loss without data loss.
I don't see anything that indicates how spacetime solves the core problem CRDB does, which is guaranteeing that single node failures can be tolerated with zero data loss or loss of availability. It sounds like transactions by default are required to be written to disk before completion, which makes them durable on a single node, but you can't ensure they're consistent across nodes without accepting the network overhead and losing transaction throughput (on writes, anyway).
Also, FWIW, in the several years I worked covering basically every incident, I can't recall seeing a network-bound cluster. Like anything else, there are tradeoffs. You give throughput, you get consistency and availability, and you don't need to engineer how to avoid data loss or availability with node failures. Unless I'm misunderstanding, spacetime is solving a totally different problem.
by timssopomo