Celld: Self-hosted, distributed Durable Objects

Join the discussion

• Hacker News
• I don't think AI is killing open source.

  It's just making it way easier to spam maintainers with code.

  Good contributors are probably worth even more now.

  by ironqcold
• Hey, is this something that compares to Temporal or Restate or Azure Durable Functions? If so, what are the differences and when would I use Cells instead of the aforementioned?
  by 7bit
• There's some overlap and some similar use cases. I've seen a few people being confused between Durable Objects and Durable Execution.

  Short writeup: https://crabmusket.net/2024/durable-execution-versus-session...

  Summary:

  I think the way to decide is, do you want to program with "objects" or "processes"?

  I'd use Durable Objects (or what I called Session Backends in my article, after Jamsocket) if I wanted to model an "entity" that lasts indefinitely (e.g. a Figma doc, a user, a concert/event). I'd use a temporal/restate function for something that has a linear sequence of events and eventually comes to an end.

  This isn't a hard rule. You can get the same outcome out of both technologies. (For example, Cloudflare built Workflows on top of Durable Objects; Rivet did the reverse.) You can use either one to implement the other, but you're going to be going out of your way based on what APIs are provided.

  by crabmusket
• I like it! Would be great if I could run locally from the start without configuring an s3 for easy playing/prototyping
  by hhthrowaway1230
• Maybe Wrangler is a better way to do that.
  by wmf
• claim is that very many things are not needed, but they are. they are just provided by the layer beneath (S3). so isn't correctness highly dependent on s3 correctness? was s3 built for that, vs just plain durability? for example, would it be just as reliable with minio?

  i would prefer a thing that was more self-contained, not dependent on a black box service layer underneath.

  much apologies if i just have a poor understanding.

  by jiveturkey
• I think you might be asking too much when it comes to correctness.

  It's like fixing all the problems with democracy by putting everyone in charge of their own 1-person election.

  by mrkeen
• Ryan & the Deno team thanks for this. not that I'm going to use it anytime soon - but yeah having durable objects outside Cloudflare and also self hosted ones means a lot.
  by dzonga
• Seeing something like this reminds me of Kleppmann's blog post: https://martin.kleppmann.com/2016/02/08/how-to-do-distribute....

  celld's README states "Object-storage compare-and-swap ensures that exactly one node owns a cell at a time," but I'm skeptical this actually holds at the point where data is written to storage - I had an AI skim through the code with me, and the actual segment writes looked like plain, unconditional PUTs with no epoch check.

  In many cases, I think using Cloudflare's Durable Objects is probably the right call instead.

  by somerandom1
• I recently spun up a simple app for our annual mango tasting event[0] using Cloudflare Workers and Durable Objects.

  It worked really well! Excited to see more options outside of Cloudflare.

  [0]https://github.com/jumploops/mangotango

  by jumploops
• Could someone explain to me in practice how one would use these to do… a thing? Like a straightforward use case? Having trouble figuring out what these are for
  by rtpg
• Multiplayer games, or apps like document editing for teams... I don't know, highly concurrent stuff
  by pelagicAustral
• Simple example 1:35 -7:40 https://youtu.be/qF2PuYnBahw

  More practical example (serverless WebSockets) https://youtu.be/FgWVoryZ8PU

  by wett
• what is the difference between celld and Cloudflare Workers open source version workerd?

  https://github.com/cloudflare/workerd

  by jitl
• workerd is missing the scheduler - the thing that allows it to operate across nodes. Durable Object storage are just local sqlite databases.

  celld is the full distributed system (albeit single tenant). It distributes DOs (cells) across any number of VMs. The databases for each DO are in object storage with RPO=0 guarantee.

  by rough-sea
• Finally!

  So happy to see support for running durable objects outside of one provider. Upvoted.

  The "durable object" concept has been repeatably demonstrated to be a valuable abstraction.

  "Each object is its own SQLite database, addressed by name and replicated to an S3-compatible bucket you own" -- this concept can take you a long way, both in its power and simplicity.

  by khalidx
• Check out https://rivet.dev/actors/
  by semicognitive
• This concept has been around for a long time.

  https://www.microsoft.com/en-us/research/wp-content/uploads/...

  by jeremycarter
• Well that couldn't have been any more timely. Did they know? Were they keeping this in a back pocket for the time?

  CloudFlare OS: https://blog.cloudflare.com/cloudflare-os/ https://news.ycombinator.com/item?id=49182996

  And yes, celld is isolates! The very lightweight within v8 isolation boundary. Hence the "very low idle cost". The deno team rolled their own new runtime! This one without deno_core! Shout out to this excellent 2022 post: https://deno.com/blog/roll-your-own-javascript-runtime https://news.ycombinator.com/item?id=35819990 https://news.ycombinator.com/item?id=35819990

  It's super super exciting having an age where many pieces of software can all run with low profiles. Very timely, again, for the unbelievably order of magnitude (and thats 10 to the not 2 to the, for you fellow computer nerds) ish ram crunch we are in. Next up, at some point, ideally we get some v8-like runtimes where we can share libraries across multiple isolates! Separating the shared code from the shared data, so we have multiple instances of the library, seems much harder. (It feels like wasm has some/much promise here and I eagerly await clearer signals that code-sharing while sandboxing is indeed possible)

  by jauntywundrkind
• > Next up, at some point, ideally we get some v8-like runtimes where we can share libraries across multiple isolates!

  I'm probably missing your point, but aren't wasm memories achieving exactly this? Just an example from wasmtime: https://docs.wasmtime.dev/examples-multimemory.html

  by tdrz
• The order of magnitude of this is simply mindblowing.
  by wartywhoa23
• Now you’re getting into the realm of old SOA… where data exchanges are just as stochastic as services on the pipe.
  by reactordev
• Wonder if this will become common practice from now on ?

    > Pull requests are disabled. Coding agents make it too easy to send a large, 
    > low-context change that costs maintainers more time than it saves. 
    > Thoughtful contributions are welcome; please understand the code, 
    > keep the patch focused, and respect the review time you are asking for.
    > 
    > Send a git format-patch attachment to ...

  by sakesun
• I wonder if the code was written by AI?
  by szmarczak
• time to bring back the dev email distribution list
  by dbmikus
• Funny, the same thing stuck out to me from Cloudflare OS's README (https://github.com/cloudflare/cloudflare-os#contributing):

  > At this time, we are not seeking outside contribution.

  > AI has made writing code easy. The hard part, today, is not writing the code, but reviewing it, making sure quality stays high, and keeping the product coherent. In that light, unfortunately, external code contributions are "donating" the easy part of the job, while creating more of the hard work.

  Feels very weird, but is logically sound: owners know exactly what they want and so they can work with Claude et al to iterate on features faster than with most drive-by contributors.

  It reads like an "end of an era" but I imagine the steady state will be somewhere in the middle: high trust, high context contributors will still be able to contribute meaningful work.

  by ryanrasti
• Could be done by an Openclaw agent named Jarvis.
  by aitchnyu