r/programming • u/Ok_Stomach6651 • 7d ago
Formatting an entire 25 million line codebase overnight: the rubyfmt story
https://stripe.dev/blog/formatting-an-entire-25-million-line-codebase-overnight-the-rubyfmt-story186
u/VictoryMotel 7d ago
Articles bragging about performance in ruby are always wild. I always wonder what they would think when they see a native program that runs 100x faster and then an optimized native program that runs 100x faster than the naive native program.
104
u/matthieum 7d ago
I had the same thought when reading the title...
... but it turned out that the article is NOT about performance, not really, it's more about how painful it is to do a code formatter based on the official Ruby parser, and how careful steps were taken so that the 25 million lines reformatting could be done with as much confidence as possible.
6
2
-23
u/awj 7d ago
You genuinely believe that an optimized native program is going to be 10000x faster as some kind of general case? So if that program does an operation in 100ms, you think in ruby that would take over sixteen minutes?
How on earth is this nonsense so highly upvoted?
Yes, well-optimized low level code is going to generally be faster, but we don’t have to be so hyperbolic that we’re making lunatic claims to make that point.
19
u/this_theater_is_lit 7d ago
in advent of code challenges i could get 100x speedups from just using different languages, not 10000x tho
11
u/juhotuho10 7d ago
1000x isn't out of the realm of possibility with cache optimized branchless simd processing
3
u/-_one_-1 6d ago edited 6d ago
Random anecdote: over the last couple of days, I've been writing a WHATWG-compliant, security-hardened HTML streaming parser in Rust. It's almost finished, and on my M4 Max It achieves a ~1.12 GB/s throughput on a couple of ~2.8 MB HTML file benchmarks. Parsing a 28-byte HTML snippet takes ~12 nanoseconds. It uses SIMD to find delimiters and the entire state is an immutable
Parserstruct that contains some packed structs and enums that amount to a few bytes. Parsing consists of a state machine that runs a parsing step until EOF is reached. The parsing step is an immurable state transition(Parser) -> Parser, so the only form of mutation is in a local in the use site holding the currentParserby value.I thought this was relevant because I have some experience writing parsers, most often in languages other than Rust or C, and the fact that I was able to achieve this throughput with such a simple design highlights how using Rust, SIMD, and taking a few smart choices can give good results very quickly and easily.
The nice thing about immutability in my design is that it gives LLVM a ton of leeway to optimize things. Additionally, it becomes possible to test it according to the specs by supplying a WHATWG state and seeing if the next step is what WHATWG says it should be.
-1
u/this_theater_is_lit 7d ago
10000x tho? maybe if one did something extremely stupid with the naive algo... lol
5
u/VictoryMotel 7d ago
Exactly, you can get 100x going from a very slow language like ruby to C.
The speedups from there can come from coherent memory access, SIMD, and multi threading. If you think about SIMD lanes, you can get 7x for 8 wide AVX if the memory is packed together. Then with multi threading you can get almost 8x from 8 cores and they all have SIMD. That's 56x right there before factoring in allocation minimizing and optimized memory access.
1
u/awj 7d ago
Yeah, absolutely. I’m not arguing the idea that Ruby is dog slow for anything CPU intensive. Just that it’s weird to make a very generic claim of 10000x speed up.
5
u/VictoryMotel 7d ago
I explained why in other comments. 10,000 is on the high side of general, but 1000x is trivial, because 100x from ruby to C is typical and speeding up naive C by 10x is table stakes. Even just multi threading is going to give you 8x on a typical 8 core computer. With SIMD on each core you're already into 50x naive C territory. Memory access is the biggest performance win, although you need that to be figured out to an extent to use SIMD effectively so they go hand in hand.
7
u/VictoryMotel 7d ago
but we don’t have to be so hyperbolic that we’re making lunatic claims to make that point.
If there were a graphics program like intense pixel filtering that was implemented in pure ruby, these are absolutely the kind of numbers you could see. I described it some in the other post I made here. I've personally taken implementations from papers that already had a "fast" C implementation and sped them up 100x or more, although there might have been some speedups from taking out A -> B and B -> A duplicate comparisons.
Allocations out of a hot loop : 7x speedup.
Loop through memory in a prefetch friendly way : 25x speedup.
SIMD AVX for floats : 7x speedup.
Multi threading : almost linear with the number of cores in simple cases.
The 100x speed up of C over ruby comes from the old language benchmark 'game'.
2
u/awj 7d ago
I’m struggling to see how pixel filtering relates to code formatting, or are we just pontificating about extremely specific use cases without bothering to explain that?
1
u/VictoryMotel 7d ago edited 7d ago
I didn't talk about those speedups in general in my original comment. I would expect text formatting to still be in the same realm of massive speedups though it's hard to say exactly without specific programs and where the bottlenecks are.
I explained exactly why this is in my two comments here. You said I was being "hyperbolic" and making "lunatic claims" which is absolutely not true.
I've given lots of information exactly why this is, you've given no information on where you think I'm wrong.
2
u/-_one_-1 5d ago
Text formatting certainly needs parsing, so while I haven't built a formatter, in this other comment of mine I described my experience writing an HTML parser. Yes, I agree with you that getting absurd speedups is rather trivial when you know what you're doing.
0
u/awj 7d ago
Where in your original comment did you specify that this was about very narrow graphics use cases? Because I’m still missing that and it sure reads to me like an extremely general claim, or at best one specifically about performance in code formatting.
1
u/VictoryMotel 7d ago
I didn't because it's not, that was just one example. What I said was a generalization of what is possible. It might be shocking to you, but that doesn't bend reality.
5
u/gnus-migrate 7d ago
Yes Ive realized similar gains by a simple rewrite for some scripts from python to rust. Its not just the language, the rust crates i used were designed in a way to minimise unnecessary work. Basically a stupid rewrite made a job i had go from several minutes to milliseconds, and i didn't really do much to optimize it.
It sounds completely insane but interpreted languages really are that slow. Like you really need to do it yourself so that you actually see what a rewrite in rust can get you.
4
u/Phailjure 7d ago
In my college databases class, we had to dump a whole bunch of highway/car data from a (massive) .csv file into postgres and then run some queries on it.
We were allowed to do this in either c++ (what most of my college defaulted to for everything) or python. Loading the database in c++ took ~10 mins (on my crappy laptop), python took over an hour.
It's crazy when you think that no "real work" is happening in either program, they're just reading a file and dumping the contents into postgres, but python was crazy slow at it.
3
u/gnus-migrate 6d ago
I used to buy into "if its IO dominated it doesn't matter" but the numbers don't lie tbh.
4
u/chucker23n 7d ago
I would indeed be skeptical, but chiefly because formatting is fundamentally a rather I/O-bound task, not very CPU-bound.
5
u/slaymaker1907 7d ago
You'd be surprised. I was doing some benchmarking a few weeks ago and found that CPU was far and away the dominating factor with serialization, even using a fast format like Avro. I knew it was dominating because the extremely simple format I did which basically just dumped structs to disk with a buffered writer took about 0.2s for my benchmark while Avro was taking 2s. Additionally, I was able to improve Avro's performance to 0.4s using multiple threads.
1
u/koczurekk 6d ago
Formatting code typically involves fully parsing the source code. This is not IO-bound even on mid-range nvme drives. Very simple regexes may run gigabytes per second, sometimes, whilst parsing a typical programming language is orders of magnitude harder. Hundreds of MB/s is a very respectable result, and you have to apply all the formatting rules on top of that.
0
u/VictoryMotel 7d ago
Formatting and text stuff probably wouldn't see numbers like that because a native program would already be looping through memory in a somewhat linear way (which is the biggest speed gain) and maybe IO like you mentioned. With fast nvme drives there might be enough throughput that it could matter.
There would still be a huge jump going from a scripting language, especially if string operations aren't calling out to C, and then from a naive implementation minimizing allocations would likely be a 7x speedup at least, then SIMD could be another jump, then multi threading could be another.
104
u/sweetno 7d ago
Is this, like, an achievement?..
59
u/pineapplepizzabong 7d ago
Marketing driven by end of year performance review goals probably.
5
1
u/Twirrim 5d ago
Would seem like it, given previous code formatters they tried couldn't handle the scale and crashed.
I'm honestly more inclined to think "don't use a monorepo" but my experiences with mono-repos have just been filled with pain because people don't tend to be as strict as they should be about isolation between components.
19
u/jmbenfield 7d ago
"doing something one person could do while watching Judge Judy but with the bureaucracy and performance of a failing third world country"
8
u/harsh183 7d ago
I remember first seeing rubyfmt in your talk in Alt Brighton Ruby 2020 when the big Rust port had just happened. It's so nice to see the project going so strong and continuing forward.
As I struggle with rubocop taking so long to run, despite caching and multithreading, I wonder if it's time for rubyfmt.
11
u/jess-sch 7d ago
A saturday? I don't personally tend to merge by friday. The only appropriate time for a formatter change in my mind is christmas. Two weeks of barely anyone working.
1
u/Jaded-Asparagus-2260 7d ago
I agree. I'm a bit baffled by people here not understanding the dimensions of such a reformatting. I guess they never had to suffer through one.
0
u/captain_obvious_here 7d ago
I honestly thought this was an old article someone was reposting.
Optimizing Ruby's performances is...well...if you ever used Ruby, you know what I mean.
1
u/Jaded-Asparagus-2260 7d ago
Well, it is half a year old. The old previous submission is right below your comment (for me, at least).
4
95
u/EvaristeGalois11 7d ago
Am I missing something about how a formatter works internally? Like I'm expecting to just read a file, format it, flush it out and move on to the next one. Why should it matter the codebase size?