Comments
Hacker News
the point the article makes is good (albeit not new). the style sounds very LLM to me.
by ch_sm
> No CFO approves a $1.4 million annual commitment on the strength of a chart showing that the commitment scales.
Except the company probably approved a budget for AWS or another cloud provider, and basically gave a blank check to developers to deploy whatever is needed. So developers are going to just deploy MSK or whatever is trendy, instead of trying to get the most throughput from the servers they got from IT.
by jfim
Other than batch jobs, I can't think of a problem that can be solved these days that doesn't also require high availability - at the very least they require a warm standby.
by edude03
A distributed system is slower than a laptop or phone by definition of “distributed” which entails a network which has a higher latency than your local system. It’s not about raw speed/latency as much as it is about volume.
I used to compare distributed system with freight trains vs F1 cars. But there are a ton of low latency distributed systems too like your LTE network.
The key factor is how much data is shared by how many endpoints. Having these two very high makes it for very hard problems. Like streaming high definition events to millions of people.
by random3
“My formative memory of Python was when the Quake Live team used it for the back end work, and we wound up having serious performance problems with a few million users. My bias is that a lot (not all!) of complex “scalable” systems can be done with a simple, single C++ server.”
by glouwbug
Even for reliability, distributed gets hairy.
I run a few VMs in a “homelab”. Was interested in redundancy so could work on system without taking down home networking… Familiar with GlusterFS, corosync/pacemaker, etc…
So to go from one system to 2, need a third system to at least ensure quorum.
Of course, storage has the same problem.
Active-active has its complexities. Maybe active-standby good enough with DRND. Still a lot to configure…
Filesystem replication now means my NVMe drives will be limited to 1Gbit/s networking link. 10GB somewhat difficult on SFF PC and expensive.
Physical location also matters. Standby probably should be somewhere else… But what about 3rd node? Wired networking not everywhere…
This ends up being a lot of moving parts, expense, and headache…
Guess what is fast? Single node. Move the storage drive to similar spare SFF PC. Two screws, takes about 3 minutes max. Local RAID-1 to guard against sudden failure… Backup a couple times a year for peace of mind.
It’s way too easy to overthink stuff…
by BobbyTables2
Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- the point the article makes is good (albeit not new). the style sounds very LLM to me.by ch_sm
- > No CFO approves a $1.4 million annual commitment on the strength of a chart showing that the commitment scales.
Except the company probably approved a budget for AWS or another cloud provider, and basically gave a blank check to developers to deploy whatever is needed. So developers are going to just deploy MSK or whatever is trendy, instead of trying to get the most throughput from the servers they got from IT.
by jfim - Other than batch jobs, I can't think of a problem that can be solved these days that doesn't also require high availability - at the very least they require a warm standby.by edude03
- A distributed system is slower than a laptop or phone by definition of “distributed” which entails a network which has a higher latency than your local system. It’s not about raw speed/latency as much as it is about volume.
I used to compare distributed system with freight trains vs F1 cars. But there are a ton of low latency distributed systems too like your LTE network.
The key factor is how much data is shared by how many endpoints. Having these two very high makes it for very hard problems. Like streaming high definition events to millions of people.
by random3 - “My formative memory of Python was when the Quake Live team used it for the back end work, and we wound up having serious performance problems with a few million users. My bias is that a lot (not all!) of complex “scalable” systems can be done with a simple, single C++ server.”by glouwbug
- Even for reliability, distributed gets hairy.
I run a few VMs in a “homelab”. Was interested in redundancy so could work on system without taking down home networking… Familiar with GlusterFS, corosync/pacemaker, etc…
So to go from one system to 2, need a third system to at least ensure quorum.
Of course, storage has the same problem.
Active-active has its complexities. Maybe active-standby good enough with DRND. Still a lot to configure…
Filesystem replication now means my NVMe drives will be limited to 1Gbit/s networking link. 10GB somewhat difficult on SFF PC and expensive.
Physical location also matters. Standby probably should be somewhere else… But what about 3rd node? Wired networking not everywhere…
This ends up being a lot of moving parts, expense, and headache…
Guess what is fast? Single node. Move the storage drive to similar spare SFF PC. Two screws, takes about 3 minutes max. Local RAID-1 to guard against sudden failure… Backup a couple times a year for peace of mind.
It’s way too easy to overthink stuff…
by BobbyTables2