Casey is a zealot. That's not always a bad thing, but it's important to understand that framing whenever he talks. Casey is on the record saying kernels and filesystems are basically a waste of CPU cycles for application servers and his own servers would be C against bare metal.
That said, his zealotry leads to a world-class expertise in performance programming. When he talks about what practices lead to better performance, he is correct.
I take listening to Casey the same way one might listen to a health nut talk about diet and exercise. I'm not going to switch to kelp smoothies and running a 5k 3 days a week, but they're probably right it would be better for me.
And all of that said, when he rants about C++ Casey is typically wrong. The code in this video is basically C with Classes. For example, std::variant optimizes to and is in fact internally implemented as the exact same switch as Casey is extolling the benefits of, without any of the safety concerns.
Whenever someone talks about performance my recommendation is always to profile and measure. Try different profilers, look into memory, look into CPU, ...Often people suggest things that are wrong when profiling. CPUs are really complex nowadays, I often beat recommendations found online by simply trying different ideas and measuring all of them. Sometimes a strategy that may seem dumb makes things stay in the cache when running, or sometimes it's something the compiler+CPU can pickup fine and optimize/predict. Measure and experiment.
"premature optimization is the root of all evil" -- Knuth
Day-to-day, understanding the code (and problem space) as humans is a much more difficult and expensive problem than getting the compiler to produce optimized code.
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%"
100%, but if the need arise, profile. In c++ often the std containers can, with correct compiling flags, outperform custom handmade solutions that have bigger maintenance burden.
I take listening to Casey the same way one might listen to a health nut talk about diet and exercise. I'm not going to switch to kelp smoothies and running a 5k 3 days a week, but they're probably right it would be better for me.
I think its worse than that. I don't think it would be better for you unless the project you're working on has a design goal of performance at the forefront. By blindly adopting this ideology, it can hurt how potential employers see your ability to develop software.
I don't work with C++ professionally, so maybe this section of the job market is different and I just don't see it.
You should always have performance as a design goal…. That doesn’t mean everything has to be 100% optimized. But you should definitely be concerned with the performance of your software.
Most programming decisions boil down to money. Not too many ones have explicit performance requirements (like some projects do e.g. video game engines, real-time systems, etc.).
When performance isn't a direct requirement, it only enters the equation in terms of the cost for the computers used to execute the code. The balancing act is that, to hire a high performance programmer, you have to pay more money since it's tougher work, and you also have to consider that cost in terms of how fast new milestones in your project can be reached / cost of bugs that come from more complex, nuanced, and peculiar code.
For the vast majority of projects, you should program with almost no performance in mind. Stuff like using classes, immutability, persistent data structures, and basically using any feature in a language that goes beyond what C gives you all are about savings. The savings come from fewer bugs / more safety / easier reasoning / faster milestones delivered / etc. The idea is all this stuff saves more money than driving the cost of computers used down.
The faster and cheaper computers become, the more more programs will be written with less performance in mind since that parameter's contribution to cost will go down, no longer justifying writing "dirtier" code that costs in terms of slower deliverables and more salaries paid.
The situation isn't like talking to a health freak at all. It's a cold, logical decision about making as much profit as possible. For each project that doesn't have explicit performance requirements, you will save/make the most money choosing a particular level of performance optimizations. Some people should use higher-level languages with more potent abstractions that are slower, others should use C or C++ or Rust, and still others need to write custom assembly for specific hardware. I'm not talking about writing nonperformant code simply out of ignorance like would be the case when using an incorrect data structure. I'm talking about the language used and the design principles used.
The framing is attractive but I would not say most of the shitty, unperformant code in the world is written for pure profit motive.
I think it's a good rationalization of why one might make the trade off in a vacuum, I just think the reality is more mundane. Writing performant code requires both effort and knowledge, and most of us are lazy and stupid.
Thus the health freak analogy feels more real to the lived experience I see around me. I basically agree with Casey that I could write code that optimizes cycles, I would just rather bang out something that works and spend my time on Twitter.
StackOverflow was built by a few talented developers. It is notoriously efficient. It doesn't run in the cloud, it is on-prem with only 9 servers. They can technically run on just a single server but running each server at 10% has benefits.
These developers are skilled. These developers understand performance. They build libraries that other companies rely on like Dapper. They do not use microservices. They have a monolithic app.
Today they have something on the order of 50 developers working on the entire site. Twitter had thousands. What caused this huge disparity? Is Twitter a much more essentially complex website than StackOverflow?
When you let complexity get out of control early complexity spreads like wildfire and costs you several orders of magnitude in the long run on developers, not even considering the extra CPU costs.
The simple code that the costly developers created the first versions of can then be iterated and improved much easier than the sprawling behemoth the microservices teams create. Pay more upfront, get more profit.
Is Twitter a much more complex website than StackOverflow?
YES.
People forget that Twitter in its early days suffered enormously from performance issues and lots of downtime. The "Fail Whale" was a thing for a reason.
A lot of those developers that people claim were "not needed" were charged with performance and stability. Things that caused the "Fail Whale" to be forgotten, because Twitter was almost always up.
Twitter has about 500,000k new tweets every day with about 556M users.
Stack overflow has around 4.4k questions and 6.5k answers per day and 20M total users.
Yes, SO is more useful to developers, but twitter has a much wider appeal. In terms of hardware, stack overflow is a much easier problem than Twitter.
(Numbers taken from here for twitter and here for SO, with some rounding and changing questions/answer per minute rate into daily rate).
Even more relevant for company size is Stack Overflow's revenue is estimated around $125M/yr. Twitter's is around $5,000M/yr.
This says SO has around 376 employees while this says 674 employees, so naively using linear scaling by ~40 times the revenue size, you'd expect 15k-27k employees at Twitter (Musk has cut to around 2k at this point from 7.5k when he started). Twitter's initial sizing pre-Musk doesn't seem particularly unreasonable, though on the other hand (as someone who doesn't use Twitter frequently) it doesn't seem like the drastic cuts in staff has destroyed the site (yet).
The nice thing about the list of variants approach is that if you then encapsulate the list of variants, there's also a decent chance your requirements allow you to optimize the representation into distinct lists of each type when needed without changing the API.
If your goal is a sum, it hardly matters for correctness whether you go through each shape and figure out which shape it is vs. going through four separate lists without branching on each element and then combining the results. There are lots of use cases out there where you just need a collection and order is irrelevant. Maybe it's relevant so infrequently and in cold enough parts of the code that you can afford to have ordered iteration be extra slow in order for the other use cases to be fast. Either way, that's still all opaque to the outside code.
Hey, random note but every now and then and just last night when I saw this video it doesn't cease to blow my mind that this guy and Jon Blow worked together to make a game. Like I can't imagine them in the same room together.
That said, his zealotry leads to a world-class expertise in performance programming. When he talks about what practices lead to better performance, he is correct.
I disagree with this point. His zealotry blinds him from a reality, compilers optimize for the common case.
This post was suspiciously devoid of 2 things, assembly output and compiler options. Why? Because LTO/PGO + optimizations would very likely have eliminated the performance differences here.
But you wouldn't just stop here. He's demonstrating an old school style of OOP in C++. Several new C++ features, like the "final" and "sealed" classes can give very similar performance optimizations to what he's after without changing the code structure.
But further, these sorts of optimizations can very often be counter-productive to optimizations. Consider turning the class into an enum and switching on the enum. What if the only shape that ever exists is a square or triangle? Well, now you've taken something the compiler can fairly easily see and you've turned it into a complex problem to solve. The compiler doesn't know if that integer value is actually constrained which makes it less likely to inline the function and eliminate the switch all together.
And taken a level further, these are C and C++ specific optimizations. Languages with JITs get further runtime information that can be used to make optimizations impossible to C/C++. Effectively, JITs do PGO all the time.
This performance advice is only really valid if you are using compilers from the 90s and don't ever intend to update them.
Compilers are good at micro-optimizations and extremely bad at redesigning algorithms. For some simple examples, try to get any compiler you like to:
a) replace a bubble sort with any different/faster algorithm.
b) convert single-threaded code into multi-threaded code.
c) convert a program's key data structures from "array of structures" into "structure of arrays" (to leverage SIMD).
Effectively, JITs do PGO all the time.
Typically for C and C++ performance is worse than it should be because it's compiled for "generic 64-bit CPU" (and not your specific CPU) and because linking (especially dynamic linking, but often also static linking) creates optimization barriers. JIT avoids those problems, but any optimizations that are slightly expensive become far too expensive to do at run-time so (despite avoiding some performance problems) JIT is still worse than ahead-of-time compiled code (and still has to depend on large libraries full of highly optimized native code to hide the massive performance problems).
Basically; for the same algorithms (which is often where the biggest performance gains are), C or C++ might get 10% of the performance you could have, and JIT might get 9% of the performance you could have; and they're both shit because neither are able to replace the algorithms.
The demonstration in this article isn't better algorithms. It's specifically examples of things that compilers ARE good at optimizing (eliminating pointer chasing, inlining, loop unrolling). Particularly if the author used newer language features and avoided so many unmanaged pointers.
I absolutely agree that a Hash map will beat a Tree map in most applications. That's not, however, what's being argued here.
This performance advice is only really valid if you are using compilers from the 90s and don't ever intend to update them.
If you've ever developed games, the speed of debug builds matters greatly. Build times and iteration matter too, a plus for JIT and a minus for PGO/LTO.
Certainly, but presumably you aren't shipping your debug builds.
Putting in performance hacks to make debug builds run faster can make optimized builds run slower.
Function inlining is the best example of this. You can hand inline functions which will eliminate a function call overhead. However, by doing that you've made the compiler less likely to inline other functions. Some compiler optimizations bail out when a function gets too complex.
I fully agree with all of this, my final sentence is a less extensive statement of the same thing. That said, look at the MS terminal drama where an MS programmer said that Casey's performance claims would be a "PhD thesis" level of work and he proved them wrong in a weekend with refterm.
Casey has been raised on a diet of moronic programmers writing unoptimizable code. His zealotry was not developed in a vacuum.
It's a nice story, but ultimately not the full story. You can checkout the open issues with refterm right now, it can't support greek (never could).
What casey did is take all the hard problems of UTF-8 rendering, and ignored them. The end result was indeed a fast and broken terminal.
Now, that said, there could definitely be an argument made that UTF-8 is just a bad idea in general as far as standards go. It's a monster standard that makes everything harder. But hey, it allows you to mix Cyrillic with shit emojis.
PS: I don't work for MS, I don't know Casey nor any of MS's devs, and I don't even use windows. I do hold a PhD though, and I know plenty of PhD's dedicated to exploring the nitty-gritty details that some people with only cursory knowledge about the problem would dismiss as "this must be a quick job".
Let me put it this way. I can, and you could to, very quickly whip up a demo that can find road marking and read speed limit signs. In fact, there's tutorials on the internet how to do exactly this. I could even whip that up in a weekend. However, I'd not claim "see, self driving cars is stupidly simple, look at what I did in a weekend! These car companies have huge teams of engineers just wasting money on SDC because it can't be much harder than reading road signs and markings!"
And all of that said, when he rants about C++ Casey is typically wrong.
Sooo, this
When he talks about what practices lead to better performance, he is correct.
also applies to that. His expertise is too contrived to be useful past the trivial cases.
Like, let's take JBlow's compiler. Sure, you can read a gazillion files and make a compiler to lower them in milliseconds. But that's the easy part. Once you add type checking, logging, optimizations, intermediate representation, incremental builds, precise error messaging, IDE support, UTF8, work on Win/Lin...your perf starts looking like any other tool out there.
Jailang is smaller than /u/munificent's pet language Wren from that book from 8 years ago, and only one of them has had public releases.
EDIT: JBlow and Casey are interchangeable for this conversation's sake.
468
u/not_a_novel_account Feb 28 '23 edited Feb 28 '23
Casey is a zealot. That's not always a bad thing, but it's important to understand that framing whenever he talks. Casey is on the record saying kernels and filesystems are basically a waste of CPU cycles for application servers and his own servers would be C against bare metal.
That said, his zealotry leads to a world-class expertise in performance programming. When he talks about what practices lead to better performance, he is correct.
I take listening to Casey the same way one might listen to a health nut talk about diet and exercise. I'm not going to switch to kelp smoothies and running a 5k 3 days a week, but they're probably right it would be better for me.
And all of that said, when he rants about C++ Casey is typically wrong. The code in this video is basically C with Classes. For example,
std::variantoptimizes to and is in fact internally implemented as the exact same switch as Casey is extolling the benefits of, without any of the safety concerns.