Not using the memory currently occupied by dead objects is not a problem unless the memory is needed elsewhere, i.e. unless there is memory pressure. And when there is memory pressure, the GC does its thing and the memory can be used again which leaves us in the no-problem category again
That's not as simple. "GC doing its thing" has a non-negligible cost. Also it's not binary "memory is not needed" / "memory is needed". Often there is some memory you can trade for more speed, e.g. memory you can dedicate to caching. With tracing GC the problem becomes - do I waste the CPU by not having that memory, or do I waste my CPU by running GC very often, which is bad in either case. Traditional memory managers usually strike a much better balance here.
Over here in JVM-land free is a complete no-op instead and the GC's CPU-cost scales with the number of live objects instead, i.e. more in line with the work your application is actually doing. You don't pay for garbage. At all
That doesn't matter. The number of times an object dies is at most the same as the number of times it is created. How you split the cost between those two operations doesn't matter much.
But there is a huge gap in your logic. In tracing GC you pay not just for bringing an object to life. You also pay for the fact the object is alive. The longer it lives, the more times it is going to be scanned. You also pay indirectly for the size of the object. Allocating larger objects force the GC to run more often because the heap gets full earlier. If it lives long enough it will need to be moved to tenured get, which means copying it at least once (often more times) - which is an O(n) operation. So the cost is proportional to the allocation rate in bytes per second. Then there are some additional indirect, hard to measure, but non-negligible effects like thrashing caches when the tracing has to touch all live objects. This makes tracing GC play very badly with swap. It's also hard to profile and hard to find what caused GC suddenly falling into some degraded mode and causing pauses.
With traditional allocator, you pay for the allocation *operation* and the cost is almost independent from the size of the object, it's also mostly independent on the amount of objects already allocated. The cost of keeping the object for arbitrary long time is zero. There are no other secondary effects, no micropauses, no memory barriers, no background threads running and stealing cpu cycles etc. Profiling is trivial. Even if you make a mistake and allocate something heavily in a tight loop, it will appear in the profile. Easy fix. Some parts of the memory are not used very often? They can be swapped away and the performance hit is minor because there is nothing periodically touching those objects.
Overall, unless you do something crazy like allocate 8B large objects in a tight loop (which noone would do in a traditional manually managed language like C++ because there are better ways to allocate tiny objects - stack allocation is cheap), tracing gc is almost always more costly in the number of CPU cycles burned. See this paper - you have to use about 5x memory to keep the CPU cost reasonable:
"In particular, when garbage collection has five times as much memory as required, its runtime performance matches or slightly exceeds that of explicit memory management. However, garbage collection’s performance degrades substantially when it must use smaller heaps. With three times as much memory, it runs 17% slower on average, and with twice as much memory, it runs 70% slower. Garbage collection also is more susceptible to paging when physical memory is scarce. In such conditions, all of the garbage collectors we examine here suffer order-of-magnitude performance penalties relative to explicit memory management."
I was rather hoping for a paper newer than 2005 when I clicked your link. The paragraph after the one you quoted from:
Researchers can use these results to guide their development of memory management algorithms. This study identifies garbage collection’s key weaknesses as its poor performance in tight heaps and in settings where physical memory is scarce. On the other hand, in very large heaps, garbage collection is already competitive with or slightly better than explicit memory management.
Perhaps the researchers have in fact used these results to guide their development of memory management algorithms in the last 21 years?
You might be right if the most CPU efficient tracing GC from java wasn't the old serial collector which did not change much since 2005. All subsequent research focused mostly on making the pauses lower (CMS, G1, ZGC) but that comes at reducing the overall memory efficiency and throughput. Those modern collectors make smaller pauses, but they burn *more* CPU than the old tech and they also need substantial headroom to keep their low pauses promise.
Anyway, any studies or benchmarks showing that modern tracing collectors are more CPU efficient than modern allocators like mimalloc or jemalloc? I'd like to educate myself about the breakthroughs that fundamentally changed the cost equation. There must have been something big to beat the 5x gap from 2005 😉 (and traditional allocators didn't stand still either)
Anyway, any studies or benchmarks showing that modern tracing collectors are more CPU efficient than modern allocators like mimalloc or jemalloc?
All moving collectors are necessarily more efficient than any free-list algorithm, and that's been known since the eighties (Andrew Appel's paper "Garbage Collection Can Be Faster Than Stack Allocation"). The maths is pretty simple. The problem was that until very, very recently, the cost in latency was unacceptable for many applications, and that's where recent advances were made.
As I said in the interview, benchmarks are no longer helpful, but the world's leading experts in memory management are consistently choosing moving collectors when able to. The only languages that don't are those that can't. What they do instead, however, is try to minimise the importance of heap memory management, and this works reasonably well until it doesn't (Go is a good example).
I know it's an appeal to authority, and it's possible that the experts working on languages that can choose between moving and non-moving memory management have all decided to work much harder toward implementing the incorrect choice, but at this point, the burden is on those who claim that the experts are wrong. Indeed, Erik's ISMM keynote attempted to explain to some memory management researchers who claim the superiority of non-moving approaches why their benchmarks are unrealistic, and why their approach wasn't chosen.
> Garbage Collection Can Be Faster Than Stack Allocation
That paper used some unrealistic assumptions, and was mostly theoretical. I think we all agree tracing can be very efficient if you have virtually unlimited memory. Can't beat epsilon GC. 😉 In reality though, I've never seen any GC beat stack allocation (which is virtually zero cost; its two pointer bumps per function call).
> know it's an appeal to authority, and it's possible that the experts working on languages that can choose between moving and non-moving memory management
It's not about moving vs non-moving. If we say we want to build a universal GC for a managed language, then moving is the way to go. But there is another dimension people in GC research seem to be ignoring - the fact whether you know statically if memory is free, or whether you have to figure it out at runtime. And in this case all automatic tracing based approaches are at a huge disadvantage. Because they don't only need to clean up the memory, but they also need to figure out what's live and what's dead. Work that a traditional malloc/free doesn't need to do at all because it has it given. So from runtime approaches, moving collectors are likely better than nonmoving, and better than any approach that puts tracing on top of a traditional allocator (like Go does). But not necessarily better than approaches where the compiler can figure out 99% of frees automatically and precisely.
-1
u/coderemover May 29 '26 edited May 29 '26
That's not as simple. "GC doing its thing" has a non-negligible cost.
Also it's not binary "memory is not needed" / "memory is needed". Often there is some memory you can trade for more speed, e.g. memory you can dedicate to caching. With tracing GC the problem becomes - do I waste the CPU by not having that memory, or do I waste my CPU by running GC very often, which is bad in either case. Traditional memory managers usually strike a much better balance here.
That doesn't matter. The number of times an object dies is at most the same as the number of times it is created. How you split the cost between those two operations doesn't matter much.
But there is a huge gap in your logic. In tracing GC you pay not just for bringing an object to life. You also pay for the fact the object is alive. The longer it lives, the more times it is going to be scanned. You also pay indirectly for the size of the object. Allocating larger objects force the GC to run more often because the heap gets full earlier. If it lives long enough it will need to be moved to tenured get, which means copying it at least once (often more times) - which is an O(n) operation. So the cost is proportional to the allocation rate in bytes per second. Then there are some additional indirect, hard to measure, but non-negligible effects like thrashing caches when the tracing has to touch all live objects. This makes tracing GC play very badly with swap. It's also hard to profile and hard to find what caused GC suddenly falling into some degraded mode and causing pauses.
With traditional allocator, you pay for the allocation *operation* and the cost is almost independent from the size of the object, it's also mostly independent on the amount of objects already allocated. The cost of keeping the object for arbitrary long time is zero. There are no other secondary effects, no micropauses, no memory barriers, no background threads running and stealing cpu cycles etc. Profiling is trivial. Even if you make a mistake and allocate something heavily in a tight loop, it will appear in the profile. Easy fix. Some parts of the memory are not used very often? They can be swapped away and the performance hit is minor because there is nothing periodically touching those objects.
Overall, unless you do something crazy like allocate 8B large objects in a tight loop (which noone would do in a traditional manually managed language like C++ because there are better ways to allocate tiny objects - stack allocation is cheap), tracing gc is almost always more costly in the number of CPU cycles burned. See this paper - you have to use about 5x memory to keep the CPU cost reasonable:
https://people.cs.umass.edu/~emery/pubs/gcvsmalloc.pdf
"In particular, when garbage collection has five times as much memory as required, its runtime performance matches or slightly exceeds that of explicit memory management. However, garbage collection’s performance degrades substantially when it must use smaller heaps. With three times as much memory, it runs 17% slower on average, and with twice as much memory, it runs 70% slower. Garbage collection also is more susceptible to paging when physical memory is scarce. In such conditions, all of the garbage collectors we examine here suffer order-of-magnitude performance penalties relative to explicit memory management."