Join the discussion

Write your take first — we'll ask for email only when you're ready to publish.

  • Hacker News
  • Honestly?

    If the problem is scalability, just rate limit git commands on free accounts already. Nobody realistically need to push multiple times per minute, and that alone is bound to trickle down to anything that triggers on commits and pushes.

  • realistically the problem is vibe coding.
  • Sure. The problem is just an easy fix that nobody there thought about yet.
  • The historical uptime has been getting better but it is on a downwards trajectory in August:

    https://mrshu.github.io/github-statuses/?view=all

  • Oh thank god my pink unicorn site is back online, its had great uptime lately so thats nice.
  • I spent a bunch of time during the outage last week setting up forgejo and some custom action runners. At the time, I was worried I was wasting time and getting distracted from my real work...alas, I guess not. Gonna finish up that work and complete the move today.
  • Good for you. These kinds of migrations never feel productive at the time, but the right tools can make your life and work so much better.
  • GitHub Actions upkeep is down to one 9. I miss the days big tech aimed for four or five 9s of reliability :(
  • > Update - We've identified an issue with a database primary and are failing over to a replica immediately

    Seems like a weird thing to post on a status page. Shouldn't this have happened automatically and therefore precluded the need to inform users of it?

  • > Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues
  • Must be a day ending in Y
  • At this point, maybe it'd be more appropriate for people to post about github to HN when githubs actually working.
  • >Update - primary failover briefly improved performance but did not fully mitigate, we've throttled inbound traffic and are investigating upstream Vitess issues

    And now they're blaming their upstream vendor! Embarrassing stuff to be writing on a public page.

  • Why is that a problem if it _is_ an upstream vendor problem? (assuming it is)
  • I read that as an upstream service they own, but I agree the wording a bit weird.
  • Vitess is a distributed mysql database. Github could very well be managing it entirely on their own. I have only seen people managing their own vitess, its entirely open source afaik.
  • Things can go wrong, but really, its been a lot and we're normalizing that to an unhealthy degree...

    I wonder if it was down that much, if users would get credits the way we pay when we use the services - its kind of ridiculous for a critical service to be down that much and all we do is "ah okay, its just github". Like, as if that was normal to be down that much...

  • I don't think it's been "normalized". Github uptime is literally a joke in the tech community. They have first mover advantage and a behemoth behind them so they're not going away, but everyone knows how shit it's uptime is. It takes time for organizations to move away from services like this but I would bet anything that many are starting to try to move away, as well as new companies knowing they shouldn't use the service.
  • I think the authors of SMTP had a healthy attitude towards server uptimes:

       Retries continue until the message is transmitted or the sender gives
       up; the give-up time generally needs to be at least 4-5 days. 
    
    https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.4....
    by zaik
  • GitHub needs to completely bifurcate their enterprise/paid services from their free services at the infra level.
  • It would probably be better to run projects with extremely high commit/merge frequency on a separate "slop infrastructure", basically like MMOs move cheaters to their own servers ;)
  • enterprise is mostly separate, is it not? uptimes are significantly more reasonable on the enterprise status pages
  • surely if they did that everybody would complain how github "lost its touch with open source since they now prioritize paid services"
  • That's what Azure DevOps is supposed to do, but for some reason GitHub has a redundant enterprise division.
  • That's what I don't understand. They could mitigate their name so much if they just split free/paid/enterprise. It's already shown that enterprise is much more estable and is largely unaffected from service disruptions. Why don't they go one more layer? For sure it's worth the extra complexity.
    by weli
  • According to their status pages (e.g. https://eu.githubstatus.com/, https://us.githubstatus.com/), their Enterprise Cloud uptime for Actions is significantly higher.
  • They have that-ish as an option: https://docs.github.com/en/enterprise-cloud@latest/admin/dat...

    I'm told that GitHub has asserted to us that moving to this model means we would not be exposed to github.com outages. It's not at feature parity with github.com though.

  • Fun read about Azure and having 173 agents running a node: https://isolveproblems.substack.com/p/how-microsoft-vaporize...

    Probably just a coincidence that Github started to have issues after beginning their move to Azure at the end of last year.

  • Keep in mind the seniority of the author. This was not written by a staff+ engineer.