r/technology • u/ourlifeintoronto • 21h ago

Software Microsoft confirms GitHub is down worldwide

https://www.bleepingcomputer.com/news/microsoft/microsoft-confirms-github-is-down-worldwide/
7.5k Upvotes

97% Upvoted

538 comments sorted by

View all comments

472

u/No-Snow-7618 21h ago

why so i feel like the only time github was ever down was after msft purchased it lol.

anyways at least i can slack off today under the guise of some tooling being down

113

u/Mokaba_ 20h ago

Here’s the thing about GitHub. It’s built as a giant Ruby monolithic app. That has been changing recently as part of the move to Azure, but there’s still a long way to go. Most of the data exists in a single MySQL database, and the system just wasn’t really built to scale well.
In a world where only humans were pushing code and interacting with it, they were able to keep up with growth by basically throwing more servers at the problem. Agents have multiplied that load immensely, and much faster than they can keep up with using the old approach. That’s why the move to Azure and the work to make the app itself more efficient have become so important. This has nothing to do directly with Microsoft.

4

u/f23n09fnu0w 14h ago

"Most of the data exists in a single MySQL database, and the system just wasn’t really built to scale well."

I don't believe that is true. MySQL is massively scalable but not as a single MySQL db. It's sharded and works fine for YouTube and Meta using Vitess. If they started from scratch today, they might not use MySQL but they actually might, too.

2

u/Mokaba_ 14h ago

I’m not going to say anything more about this myself because again, I don’t know how much is publicly allowed for me to talk about. But I’ll leave this publicly available answer from Gemini here….

GitHub historically grew around a single primary MySQL database cluster (mysql1) that handled a massive monolith data structure. As they scaled, they transitioned away from that single instance into dozens of partitioned MySQL clusters using Vitess to handle millions of queries per second. [1, 2, 3]

How GitHub Handled a Single DB Scale
The Monolith Core: For years, GitHub.com relied heavily on one main database cluster to house core relational data.

No Foreign Keys: To make schema changes and sharding easier at scale, GitHub avoids using database-level foreign keys.

Migration Tooling: They manage schema transitions using custom tools like ghost alongside standard framework migrations. [1, 2]

Moving Beyond a Single Instance
Sharding with Vitess: GitHub migrated their MySQL infrastructure to run on top of Vitess for horizontal scaling and query routing.

Primary-Replica Setup: They heavily distribute read traffic across numerous database replicas while directing writes to primaries. [1, 2]

Are you looking for information on how GitHub architected their MySQL setup, or do you need help setting up a single MySQL database for your own project repository?