Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- > A team of three people maintains five microservices.
How about a team of 2 people maintaining 15+ microservices? :/
Add to this: at any given time, we have several features at different phases, so some microservices have multiple open PRs. And some features impact multiple microservices.
When we release a feature, we deploy all impacted microservices at the same time.
by caarmen - Perfection is always relative to a purpose or telos. So in the case of artifacts - which are the products of engineering, arts, and techne in general - the purpose isn't inherent to the object in the manner it is in oak trees and human beings, but projected onto the thing. It lives in the mind of the designer and the user. That's why we can say that technology is an "accidental" extension of human beings by which human purposes are realized and mediated in ways that inherent human powers cannot directly accomplish.
So, yes, if you have a purpose in mind, you can build something that can be used for that purpose, more or less effectively. The more general the purpose, the less specific the implementation; the more specific the purpose, the less general the implementation. Over-engineering here is something that indeed goes beyond the purpose of something, with feature creep as an example of moving goalposts and a slippery grasp of the limit.
However, production is only one layer of concern in engineering. You can talk about ideal artifacts whose forms perfectly enable the realization of some purpose, but there is the practicality of constructing one that must also be considered. Think of diminishing returns. Does it makes sense to invest a certain amount of time and resources into producing a more perfect artifact? If not, then pursing such perfection is itself an imperfection on the part of human judgement. That is, a more perfect engineer might know the limit of perfection for an idealized artifact, but he also knows what the perfect practical limit of justifiable effort is. Perfectionism is a sign of an imperfect engineer.
Returning to feature creep, the author might argue that the moving goalposts are a sign that the purpose is being refined, but feature creep can also mean that the shifting purpose is detached from real needs. So, in that case, the purpose served is no longer the purpose that ought to be served, and in that sense, while one may realize an end perfectly in a relative sense, the end pursued is itself the result of imperfect understanding, which is effectively what the author is gesturing at when discussing requirements.
by lo_zamoyski - "you need a very clear set of requirements. Every constraint on the table"
How nice it would be to live in this fantasy world
by s08148692 - "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away
- Antoine de Saint-Exupéryby W-Stool - This was my first thought too. I actually use this in my Agent prompts
- I think over-complicated and over-engineered are not the same thing.
Over-complicated is adding too many features, too much mechanism, too many moving parts. Over-engineering is far exceeding the requirements in an unhelpful way. For example: build me a treehouse. Said treehouse could probably be made of wood. If you made it out of concrete and steel, it might be a heck of a lot stronger and last a heck of a lot longer, without being more complicated (just more expensive). Too strong is over-engineered. If you made a treehouse with 13 bedrooms and an elevator, glass windows, solar power, and running water, that's overcomplicated.
by titzer - Reminds me of the Seinfeld episode where he hires a contractor to work on his kitchen and ends up with twice as many cabinets as he had before.by marknutter
- I think your concrete treehouse counts too though. Add a shower and suddenly you need plumbing, drainage, the structure has to hold more weight. The overcomplicated part drags the overengineered part along with it. They pretty much collapse into the same thing.by nulltrace
- I generally use: - Overbuilt: Designed to handle more of the same. Too strong. - Overengineered: Complexity not justified by the requirements.
Sometimes overbuilding (e.g. using really thick wood on a treehouse) means you can lower complexity (less bracing needed) at the cost of being more wasteful with material. That is often a great tradeoff.
by edg5000 - The caveat here is, as TFA states, is that you need to have "every constraint on the table". And that is almost never the case. Unless you create something in a mature domain, where everything has already been explored, you _won't know_ what the constraints are.
You can spend three months in whiteboard sessions planning out the perfect solution, and then your plan falls apart the second you start implementing, because the real world imposes constraints on you that were practically impossible to know a priori. Not theoretically impossible, mind you. If you'd been smart enough and spent enough time thinking about it, you would have thought of them. But you aren't smart enough, and you don't have the time. Those are some other constraints that the universe puts on you.
by laszlojamf - We spent 20 years talking about agile precisely because of this, and people took it, turned it into a checklist based processed, forgot about the short feedback loop it's precisely supposed to bring to the table to figure our your unknown unknown quickly, and declared it useless.
And now we are back to square one, where we argue again "but just make good specs".
The world is a silly place.
by BiteCode_dev - "With one big caveat: you need a very clear set of requirements"
Most new product launches are an exercise in figuring out what the product requirements should be through trial and error (really: through ongoing dialogue with your users). Even mature products can have requirements change over time as the market changes.
I think internalizing this reality is why most senior engineers who work in domains that touch the messy real world will reflexively push back against perfectionism.
by mlinsey - But this article allows framing this decision in a nice way. It's not a push back against over engineering, it's deliberate trade-off for speed by lowering quality, because quality is relative to the targeted outcome, which is yet fuzzy.by smallnix
- > Most new product launches are an exercise in figuring out what the product requirements should be through trial and error
Sure, but these have to be well-defined trials. In other words, yes, you will test several hypotheses, but your hypotheses have to be hypotheses, not hand waving.
by bad_username - I believe perfection is a dirty word because it has a pernicious effect on the human mind.
Striving towards perfection too often leads to yes over-engineering but I think more importantly it also: leads to way too much bike shedding due to the need to be perfect and allot of even emotional baggage when we don't make that perfect things.
Even the author admits that their definition of perfect can only arise with stringent requirements and I'll take it a step further, maybe it was perfect for that problem at that time but guess what change happens all the time, and as architects we have to think about future problems as much as current ones. With that in mind I think that having something that is good in many scenarios is better than the thing that is perfect in the scenario you start in.
- I don't share the definition of over engineering.
As engineering is the act of solving technical problems, over engineering is about putting too much engineering effort on aspects, features or products that don't have a linear payoff to the budget spent.
E.g. I worked in a company that was obsessed with unit test coverage metrics and the effort of maintaining the test suite was considerably biting in the ability to move the product. The ratio was 25% of product and 75% of unit tests.
And the payoff was small if not even negative, impacted morale, productivity and actively pushed back against refactors, because any large refactor was met with a disproportionate effort in unit test rewriting. Let alone the fact that as you were mocking external dependencies, and the mocks costed engineering effort and internals digging, it also suppressed work to keep it up to date.
I never got to convince the org to shift focus on E2E testing, which answered the real questions: does the product make money and works as expected. Uni test had to be used when writing a parser, not to validate some react scroll component as browser apis and the triggers mimicking costed days of work.
In fact we often had gigantic all green test suites for broken products.
That's to me over engineering an aspect of the engineering cycle.
by epolanski - This is a good way of putting it. People who advocate for over-engineering don't consider cost-benefit vs not doing it. This is where FMEA analysis comes in when developing products, you assign values for severity, occurrence, and detection to determine which potential issues need extra effort implemented, but just as important is identifying the potential issues that aren't worth putting effort into to prevent over-engineering.
I've recently had to deal with a problem where the parent company has dictated 100% production testing for a spec that we meet for the spec sheet but customers practically don't ever need. But since the parent company test their products this way they say we have to. I've been very vocally against this because it has delayed us shipping products that are otherwise ready, prevented starting new products, and added a lot of production cost when our RMA rate without the test is something on the order of 0.01% and most of those are due to installation errors, not true failures.
by starky - "We're not trying to build a perfect solution here" is not something said to assuage over-engineering or encourage sloppy work.
It's said to head off a specific complaint from a specific kind engineer who will object that the proposed solution won't work because it doesn't cover some obscure edge case which rarely comes up in production.
"We're not trying to build a perfect solution here" is saying "We acknowledge not everything will be covered, we're setting the requirements at the 90th percentile use case".
by nickelpro - In my experience when a manager says that it means “let’s just ignore the architecture and use a short cut if we can get the security team to rubber stamp it”.by jurgenburgen
- What is the qualifier of "rarely". One in ten, one in hundred, one in thousand? What is the failure in these cases? I could accept rarely if that is one in trillion to one in quadrillion range say. Truly rare cases. But not if it might be for seen reasonably.by Ekaros
- > We acknowledge not everything will be covered, we're setting the requirements at the 90th percentile use case
I'm ok with this as long as dropped requirements are documented and nobody wakes you up at night when those edge cases show up in production. Also, when the next feature needs to build on top of those dropped edge cases, you are given enough resources to redo the previous solution.
by sesm - I've been that engineer and with good reason. That obscure edge case which rarely comes up in production is very disruptive when it does come up. The product person waving it off is also not the one that will get paged at 2am to address the issue when it occurs. Accepting that 90th percentile use case is infuriating because it is tacit consent for an unfinished solution with the rest being made up later with additional toil, now constrained by load bearing things you cannot change. Thanks for rushing the thinking and making my life harder later for no reason.
- I find it that in wild it is quite often used by product people to push crap ASAP because they don’t want to spend time finding out even basic constraints in the system they supposed to know like the back of their hand.
Then engineers are on the hook because they run into those constraints while building and everyone always blames „those lazy software developers” ;)
by ozim - The bane of my engineering career is working under engineers like this. It's like we forget we are doing a very analog thing (collaboration and building) under the guise of something digital. We should accept that there will be edge cases, there will be crashes. And unless you're actually in a life-saving industry, that is ok. (I say this with the idea in mind of a 10+ year old code bases spanning many new coding patterns that achieves over 99.5% crash-free)by hectdev
- My grandpa used to say "we're not building a piano" when doing things like building a quick shelf or something in the basement and be done after a quick measure and a few cuts. My other grandpa would spend an entire day measuring, sanding, pulling out a router etc. Both valid, but if you just need something to put some paint cans on, it doesn't require a day of work.by esikich
- This was my initial reaction to reading this post as well.
Additionally, as I get older, I find the sentiment of "we're not trying to build a perfect system here" is less about "let's just go fast vroooom" and more akin to saying "I've been humbled before by thinking I had the perfect mental model of the universe before a single user touched the product."
by calebkaiser - I wouldn't say that "over-engineering means solving the wrong problem". It's possible that the idea is basically correct, but people are directing effort towards optimizing for constraints that don't really exist or can be dealt with once a better picture is in place, whether it's the mythical PMF or just "we now understand what the users want, let's build that".
The worst clusterfuck I've ever worked on was a web application that was actually solving a real problem fairly well, but the team was spending time building an absurd Rube-Goldberg contraption of microservices when the entire platform had less MAU than my hobby website. It wasn't the wrong problem, but it certainly was over-engineered!
by qsort - And sometimes optimizing for constraints happen because someone from above (perhaps someone with a title of Director who doesn’t actually write code) imposed their constraints on the team without understanding the ramification of these constraints. It takes a manager with very good people skills and/or very good writing skills to persuade that someone to drop their constraints.by kccqzy
- > optimizing for constraints that don't really exist or can be dealt with once a better picture is in place
I think these both fall comfortable under the umbrella of "solving the wrong problem"
- While I agree with you, a large number of people use the term to mean that something was engineered to a complexity they have difficulty understanding. Abstraction is also treated as a dirty word, with people focusing on only one interpretation of the word.
While I don't actually agree with OP, I do agree with their sentiment. I've seen people say something is "over engineered" when there's an elegant design. Elegance isn't over engineering, it is solving problems effectively. It's something we should chase! Elegance is solving the right problem, which usually people are having a hard time seeing. (It's not always easy)
If we constantly let people drag quality down then we get into this frustrating world where everything is constantly half broken.
by godelski