Birbla

Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

Sign in
  • Hacker News
  • Nice that it is cc-commons.

    Unfortunately site would need some refurbishment for mobile… slide show doesn’t show, and page is floating.

    by sixtyj
  • Had a strange thought last year, that if a system runs too smoothly, eventually the knowledge of dealing with breakage will disappear, and when it inevitably breaks, everyone will be unprepared.

    So a little bit of breakage is like a healthy "exercise."

    Presumably, a functional system is supposed to have some kind of drills to fill in for that. But I have seen ones that don't!

    by andai
  • There’s a good Star Trek TNG episode about this: When the Bough Breaks (S1E16)
    by data-ottawa
  • Oh, the legendary backup restore drill? I'm sure it will happen someday ...
    by _def
  • that's the plot of The Machine Stops

    https://web.cs.ucdavis.edu/~rogaway/classes/188/materials/th...

    by otekengineering
  • https://en.wikipedia.org/wiki/Hormesis
    by trio8453
  • Very true. If things are too reliable, systems can come to depend on them always being so so reliable... So it can actually pay off to inject transient issues deliberately.

    https://netflix.github.io/chaosmonkey/ and similar can help.

    That said ... most instability is introduced with normal changes, so every engineer can be a chaos monkey. ;-)

    by gatio
  • Knowledge of how to fix the system comes from understanding how the system works, which does not necessarily require experience fixing it. Every problem ever fixed was at some point fixed without prior experience - if we can fix problems that have never happened before we can fix problems that haven't happened in a while.
    by jjk166
  • Yeah, I've lived that one. When your system is falling over all the time, you get good at quickly bringing it back. When you have the first failure in 2 years and you didn't do drills or etc, it takes a lot longer to get things going again.
    by toast0
  • This is known as the "Paradox of Automation." It's a real problem for many industries.

    The OG paper on this is called "The Ironies of Automation" (Bainbridge, 1983). It's a clunky but fascinating read. I give it a skim every couple years and usually come away with a bit of a fresh take on the problem.

    by jknoepfler