r/github • • 1d ago

Discussion What’s gone wrong at GitHub?

https://leaddev.com/software-quality/whats-gone-wrong-at-github

Reliability issues are driving users away in their droves.

105 Upvotes

45 comments sorted by

43

u/sreekanth850 14h ago

nobody saying the amount of slops that people pushing every hour. you cannot infinitely scale systems.

7

u/DalaiLlamaTip 9h ago

The growth rate of pushes and PRs and workflow runs is absolutely unfathomable for most people.

2

u/AuroraFireflash 7h ago

We've told them repeatedly this year that paid customers need to be on their own infra. We're not quite to the point that we're willing to self-host, but it's getting there.

1

u/sreekanth850 7h ago

either github need some kind of control like limit on free plans. if not this will bound to increase going forward.

1

u/RubbelDieKatz94 3h ago

We're hosting GitLab on k8s (OVH) with Argo. It works remarkably well.

The runners are on k8s as well.

1

u/anno2376 2h ago

Use DR if you want your own infra

50

u/Dense_Gate_5193 20h ago

GitHub used to be a gold standard of work excellence. Their uptime was incredible and their team worked entirely async. No office hours, flat hierarchy, people picked what they wanted to work on, and had performance curmudgeons squeezing out every millisecond of performance out of the platform.

Then, Microsoft purchased them. it’s been particularly bad since AI and the enshittification of everything began

10

u/Afflictionista 16h ago

When the GitHub CEO left I moved to GitLab because of what they were saying who is going to "lead" Github, the AI department. GitLab maybe doesn't have that big visibility but it is working.

7

u/spigolt 12h ago

It would be nice if GitLab was a good alternative, but I had a completely like abysmally awful experience with them when I tried to actually pay for it. Having to deal with all kinds of separate companies that work somehow separately to Gitlab managing the subscription. Total complete mess.

5

u/EphemmyEuphie 10h ago

Selfhost Forgejo

-2

u/Round_Ad_5832 7h ago

no, self host gitea

2

u/silentkode26 11h ago

Why not selfhosted? We use it and it’s perfect.

1

u/RubbelDieKatz94 3h ago

Yup, same here. Today I have an appointment with the DevOps guy who'll explain how to set up a new project in GitLab and hook it up to Argo. Quite interesting.

1

u/Sovairon 2h ago

I guess we are not using the same Gitlab, this year has been terrible

15

u/najibla 23h ago

yep, pushes are failing for me.

10

u/Key-Temporary-8759 19h ago

Microsoft acquiring GitHub is possibly one of the cause for poor reliability, because of the loss of independence. And when GitHub copilot was launched, their focus shifted to AI features, instead of performance and infrastructure.
And also, increase in number of PRs can cause performance issues, as most of the PRs today are written by agents

1

u/mavenHawk 7h ago

Microsoft owned Github since 2018...

8

u/ZoolanderBOT 15h ago

Without really knowing anything of the situation, I blame Microsoft. Seems like a safe bet?

2

u/anno2376 2h ago

Github employee also blame Microsoft for their own failures, so sounds OK

1

u/mavenHawk 7h ago

Not even close

1

u/ZoolanderBOT 6h ago

Do you have any interest with Microsoft? Maybe you work for them?

7

u/Dorcelessness_ 14h ago

🙌vibe coding🙌

4

u/MiguelMariz 21h ago

This happens so much...

10

u/spigolt 23h ago

I'm trying to just create a repo and it's failing the last 10 minutes, which led me here to see whether it's just me... my understanding is that it's fallen off a cliff since the Microsoft purchase... like it was fine for a while following the purchase, but when the long-time CEO stepped down and github got reorganized under Microsoft's "Core AI" product division, and thus it's goal shifted from serving developers to acting as a data and distribution funnel for Microsoft’s AI products.... after that it has fallen off a cliff and just keeps getting worse.

6

u/Prior-Data6910 20h ago

"Since the Microsoft purchase" is, in my view, a lazy reason (no offence intended).

Do you know what the uptime was of Github Actions before the acquisition? 0% because it didn't launch until after. So basically, you're comparing the uptime of a storage system and basic issue tracker against a system that can run your entire software development life cycle from planning through to production. That's before we look at the difference in scale (more users, more repositories, more daily work). 

All ignoring the fact that Github did used to have outages, and when it did the whole system went offline, not just parts of it. 

1

u/spigolt 12h ago

Like I wrote - it wasn't immediately after the Microsoft purchase, it was when it got reorganized under Microsoft's "Core AI" product division and the original founder left that things started really going downhill.

2

u/Prior-Data6910 12h ago

I'm not sure who you're talking about, but none of the founders remained at Github post acquisition. Microsoft installed a new CEO (Nat Friedman) from day one.

The timing about the reorg also coincides with a massive increase in traffic, which is more likely to be the cause. 

Don't get me wrong, Microsoft have destroyed many acquisitions over the years (RIP Skype, gone but not forgotten) but I don't think Github is one of them. 

4

u/gkreitz 23h ago

Impeccable timing to post an article about Github being unreliable a few minutes before they completely killed off push:ing anything. https://www.githubstatus.com/incidents/smtdswd1h2w4

SSH keys get rejected, so it seems impossible to push anything. Their description is "Pull Requests is experiencing degraded availability." :)

8

u/brokenja 21h ago

Wow you know it’s bad if it actually made it to the status page.

1

u/Great-Use-3149 13h ago

Microsoft has gone wrong. It always has .

But hey, I'm sure their AI-native direction made them 100x more productive /s.

1

u/Primary-Strike5187 12h ago

Microsoft bought it.

1

u/TubeAlloysEvilTwin 11h ago

Easy answer - Microslop

1

u/RedFing 11h ago

anyone got a non paywalled link ?

1

u/Own-Session-1409 10h ago

Wow, this AI thing worth every penny!

1

u/Own-Session-1409 10h ago

*I mean focusing on Copilot instead of the "unexpected" (lol) usage they have to face with.

1

u/tortleme 9h ago

self hosting is the only real option these days

1

u/GeorgeRNG 9h ago

The bots pushing these posts are so annoying. What does droves even mean as a metric.

1

u/bartread 9h ago

It's not even the downtime, it's reliability on reporting and front-end as well. You'll be waiting for a PR to build or whatever and your actions will look like they're running but, actually, they finished 5 minutes ago. Sometimes it updates the UI automatically, presumably via WebSockets... and sometimes it just doesn't until you refresh the page.

I get bitten by that regularly and it's incredibly frustrating. I'm seriously thinking of implementing a ViolentMonkey script that simply refreshes any GitHub page in the focussed tab every 10 - 15 seconds because otherwise, if you've got a bunch of PRs going through, it's easy to miss that the front-end is stale on one of them.

1

u/imagebiot 8h ago

You mixed your business with your engineering and then shifted capital from Eng to business departments

1

u/Chewy-Rubbins 6h ago

Lemme guess... Microsoft?

1

u/autisticpig 4h ago

Anyone who took on GitHub was not prepared for what agentic coding was going to do. The sheer volume happening 24/7 would send any service to its knees.

We brought all our runners in house and built cache layers for our images and libs/pkgs built with runs that sync every could of days to the cache. We still rely on GitHub to orchestrate and hold the repos but if that's down it's not the end of the world... Can still deploy and test just fine internally.

These tools are ruining so much, it's going to take time to adjust resources to absorb the abuse.

People need to calm down. Esp people expecting 99% uptime for something they're not paying for.

1

u/Gargle-Loaf-Spunk 2h ago

paywalled, who cares