Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- S3's whole selling point is 11 9s of durability across the whole region which is probably why it's slow to begin with.by grenran
- Interesting project but lack of S3 protocol compatibility and fact it seems to YOLO your data means it's not acceptable for many.by stackskipton
- And means it is acceptable for many others. There is a whole world outside of s3 you know.by moi2388
- When you have a service and really care about shoving of S3 latencies in the millisecond range, then you propably have enough users that all the tiny images are cached @ edge anyways.by tuhgdetzhh
- > Despite serving from same-region datacenters 2 ms from the user, S3 would take 30-200 ms to respond to each request.
200ms seems fairly reasonable to me once we factor in all of the other aspects of S3. A lot of machines would have to die at Amazon for your data to become at risk.
by bob1029 - Similar systems include Facebook's Haystack and its open source equivalent, SeaweedFS.by Scaevolus
- That’s a lot of work creating a whole system that stores data on a raw block device. It would be nice to see this compared to… a filesystem. XFS, ZFS and btrfs are pretty popular.by amluto
- I don't quite understand the point, why would anybody use S3 then ?by bionsystem
- > Direct I/O means no more fsync: no more complexity via background flushes and optimal scheduling of syncs. There's no kernel overhead from copying and coalescing. It essentially provides the performance, control, and simplicity of issuing raw 1:1 I/O requests.
Not true, you still need fsync in direct I/O to ensure durability in power loss situations. Some drives have write caches that means acknowledged writes live in non-volatile memory. So maybe the perf is wildly better because you’re sacrificing durability?
by rockwotj - You mean in volatile memory?
- Looks like the author is well aware:
Full code here:/// Even when using direct I/O, `fsync` is still necessary, as it ensures the device itself has flushed any internal caches. async fn sync(&self) { let (fut, fut_ctl) = SignalFuture::new(); self.sender.send(Request::Sync { res: fut_ctl }).unwrap(); fut.await }https://github.com/wilsonzlin/blobd/blob/master/libblobd-dir...
by rrauch