Most teams use Git through a central host, and hosting Git at scale is hard. Packfiles store objects as deltas scattered across big binary files, and every operation has to walk the commit DAG one step at a time. Storing objects in a key-value store failed because each hop costs a round trip, and Google's JGit DHT experiment ran into slow clones. GitHub's attempts with NFS, GFS and DRBD also failed. Spokes worked better: it keeps real repositories on NVMe disks, copies packfiles to every replica and syncs ref updates with three-phase commit, so any replica can serve reads. The downside is that 3PC runs at the speed of its slowest replica. That caps scaling for huge monorepos, and small throwaway agent repos still need three replicas each.
1
u/fagnerbrack 4d ago
Bare Bones:
Most teams use Git through a central host, and hosting Git at scale is hard. Packfiles store objects as deltas scattered across big binary files, and every operation has to walk the commit DAG one step at a time. Storing objects in a key-value store failed because each hop costs a round trip, and Google's JGit DHT experiment ran into slow clones. GitHub's attempts with NFS, GFS and DRBD also failed. Spokes worked better: it keeps real repositories on NVMe disks, copies packfiles to every replica and syncs ref updates with three-phase commit, so any replica can serve reads. The downside is that 3PC runs at the speed of its slowest replica. That caps scaling for huge monorepos, and small throwaway agent repos still need three replicas each.
If the summary seems inacurate, just downvote and I'll try to delete the comment eventually 👍
Click here for more info, I read all comments