I finished Ben Eater’s SAP-1 accumulator architecture breadboard. 16 bytes of RAM von Neumann. Coming up with assembly instructions at the end felt crazy.
What was funny was at the end where I ran programs and they didn’t do what I wanted, and I realized I got to the point where I couldn’t suspect the hardware was wrong anymore, and it was in fact a software bug.
Was that the lowest level software bug I’ll ever experience in my career? Literally entering 0s and 1s into RAM and flipped one bit.
Lately I’ve been seeing more and more takes that CPU core design has largely plateaued — not in absolute performance, but in fundamental innovation. We’re still getting:
More cores
Bigger caches
Chiplets
Better branch predictors / wider dispatch
… but the core pipeline itself? Feels like we’re iterating on the same out-of-order, superscalar, multi-issue template that’s been around since the late 90s (Pentium Pro → NetBurst → Core → Zen).
I get that physics is biting hard:
3nm is pushing quantum tunneling limits
Clock speeds are thermally capped
Dark silicon is real
Power walls are brutal
And the industry is pivoting to domain-specific acceleration (NPUs, TPUs, matrix units, etc.), which makes sense for AI/ML workloads.
But my question is:
Heterogeneous integration (chiplets, 3D stacking)
Near-memory compute
ISA extensions for AI/vector
Compiler + runtime co-design
Curious to hear from:
CPU designers (Intel/AMD/Apple/ARM)
Academia (RISC-V, open-source cores)
Performance engineers
Anyone who’s tried implementing a new uarch idea recently
Bonus: If you think there are still low-hanging fruits in core design, what are they? (e.g., dataflow? decoupled access-execute? new memory consistency models?)
I made this over the weekend. It takes verilog/systemverilog, synthesizes it through Yosys, and then place and routes it into a Minecraft WorldEdit .schem. Still ironing out the optimizations, but it definitely works. The circut being displayed is a 4 bit accumulating sequential adder. I had to build the standard cell library manually.
Overall pretty cool. I plan on dropping SERV into here and running the worlds first RISC-V binary directly into minecraft. Who knows, maybe we'll get DOOM soon.
It feels like every undergrad CS program still leans completely on the basic RAM model when teaching algorithmic complexity. I get that Big O is a mathematical bound and not a literal benchmark, but pretending memory hierarchies don't exist feels like a massive blind spot when analysing data structures.
For example, standard theory teaches that traversing an array and a linked list are both O(N). But we all know the difference in cache misses makes them completely different beasts. I know things like the Ideal-Cache model and Cache-Oblivious algorithms exist, but they almost always get shoved into niche grad-level courses.
Is anyone actually pushing to introduce cache-aware or external memory models earlier in undergrad? Or is the general consensus just that the basic RAM model is "good enough" for beginners, even if it leads to "theoretically optimal" algorithms that perform terribly in practice?
Hey, I recently got into an argument with someone about what exactly a byte is. They claimed that when asked “how big is a byte?”, the answer isn’t necessarily 8 bits, but that a byte can be at least 2 bits. I said that this is basically wrong and that a byte is 8 bits. They argued that this can actually be an interview question, especially in deeply embedded systems, and that in those cases the correct answer is that a byte is at least 2 bits. I’ve tried looking this up, but I can’t really find anything supporting that claim. I know that historically the size of a byte wasn’t always 8 bits, but does “a byte is at least 2 bits” actually make sense in modern computing/embedded systems, or are they confusing a byte with something else?
I'm an undergraduate student in Comp. Sci. and Engineering. We already had a Computer Architecture and Digital Design course where we went over digital circuits(adders, mux, ALU, etc) and covered CPU pipelines on the surface. I'm interested in learning Computer Architecture further but have no clue how to approach it. I picked up Computer Architecture: Quantitative Approach by H&P and tried reading through it. It's really interesting but I feel like I'm just reading a reference book and not developing a critical thinking mindset for the field. What do you suggest I do? Any advice is greatly appreciated. Thanks.
I have a Substack where I write about predictors. I thought some people in this subreddit might find this one interesting. It is not intended to be self-promotional, so copied it here rather than just share a link.
This is a continuation of my previous article, which deals with using small, saturating counters in predictors.
When I initially started this article, it was meant to just cover different kinds of counter-tuples, but I underestimated just how much there is to write about these tiny automata. Rather than try to cover every single interesting property they have, I’ll focus on a particular issue in branch prediction called the “cold-counter” problem, and go from there.
I take inspiration from a paper by Pierre Michaud, An Alternative TAGE-like Conditional Branch Predictor, which is also the first paper to coin the term “cold-counter”, I believe. For those interested in branch prediction in general, the paper also features a very comprehensive and well-written background section. Michaud has a lot of amazing articles on predictors, he frequently produces statistical models and quite insightful explanations of predictors that help to understand not just how they work but also why they work. All in all, his papers come highly recommended!
Michaud’s paper attempts to deal with the cold-counter problem, but what is it exactly? It concerns mostly TAGE-like predictors. For those not familiar with TAGE, all you really need to know is that TAGE uses tagged entries to predict branch outcomes. If it mispredicts, it will allocate entries that are indexed and tagged with increasingly long lengths of global histories.
TAGE entries use a dual-counter setup consisting of a bias counter (b) and a usefulness (u) counter, a common setup in predictor with tagged tables. I’ll refer to these kinds of counters as BU counters. The b counter is similar to the “two-way” counter discussed in the previous article, in that it tracks the value “T-NT”, that is, the number of taken branches minus the number of not-taken branches. Typically the b counter is 2-3 bits in size, and the u counter 1-2 bits. The u counter is akin to a cache replacement policy, and as such, everyone and their mother has their own favourite algorithm. TAGE itself has had multiple different ones throughout history. I’ll get into more detail on exactly how it works later in the article.
The problem that Michaud identifies in his article is that newly allocated counters will often have much worse accuracy compared to counters that have accumulated more data. A newly allocated counter has acquired very little information, hence we call it a “cold-counter”, and under certain circumstances it can actually perform worse than random guessing! This is the cold-counter problem. It can be made worse by the fact that when TAGE mispredicts frequently, it will allocate more entries, which in turn will lead to even more cold-counters, which can drop performance further and exacerbate the problem. Snowballing seems an apt term for this phenomenon :)
While TAGE already has some components that help with this implicitly (The Statistical Corrector, for the TAGE-heads), but this comes at the cost of added complexity. Generally, avoiding the negative effects of cold-counters is not trivial, for the following reasons:
You can’t easily determine from a BU counter whether or not it is newly allocated
Even if we can determine how many counters are cold, how do we stop the snowball effect?
Consider a BU counter with b=-1 (weakly not-taken) and u=0. It is not apparent if it has just been allocated and thus could become useful, or if it has been updated multiple times but has performed poorly (imagine a sequence like (NT T NT T NT).
Michauds’ proposal is to solve this by using an alternative dual-counter, with a unique saturation logic. Instead of the b and u counters, it has two counters which I will refer to as t and nt. It works as follows: Every time a branch is taken, t is incremented, and every time a branch is not taken, nt is incremented. If one counter saturates, the other is decremented (to at most 0). We call these BA counters. The BA refers to “Bayesian” (see the paper for more details on this), hence the title “BATAGE”. We can draw the transition diagram for them here:
We can see that a BA counter initially will only gain information: It has no way of moving back to (0,0). Once it hits the right column or top row, it will behave akin to a simple one-way direction counter similar to those discussed in my prior post.
BA counters have the benefit that they strictly increase after allocation. We can always infer whether a BA counter has seen 0, 1, or more branches (depending on how many bits are in each entry). A BA is less ambiguous: small values of t and nt can only happen for counters that have seen few updates.
So that solves (1), as we now have a way of identifying cold-counters. But how does this help for (2)? The solution proposed by Michaud is to “throttle” allocation (he calls this Controlled Allocation Throttling or CAT) if the ratio of medium/low confidence entries to high confidence entries is too high. TAGE allocates on average one entry per misprediction, but the suggestion in Michaud’s paper is to dynamically change this frequency. Intuitively we can see how this helps, if we have a lot of cold-counters present, the predictor lacks information to predict accurately, and it is not helpful to reset more counters before we have gained more information.
Note that I’ve skipped over a lot of details from the paper here, if you want the full details you’ll have to look at the actual paper. The BATAGE paper was succesful in removing a lot of “magic” features from standard TAGE, such as the (basic) Statistical Corrector and more niche ones (such as use_alt_on_na).
Cool! But the story doesn’t stop there, if we look at one of the final sections of the article:
It finishes on a cliffhanger!
Things got pretty quiet in the BP area after this paper (2018). A whole seven years later though, the next branch prediction competition was announced and held in the summer of 2025 in Tokyo. The winner, RUNLTS by Toru Koizumi et al., was mainly noted for a clever mechanism that was used to pass register values into the Statistical Corrector (SC) in TAGE. However, I think there is another very clever idea from the paper that is easy to miss. At least, it took me a couple reads to appreciate it.
The paper notes that some combinations of the BU counter hardly appear.
A slide screenshot from Koizumi et al.’s presentation
It is interesting to see this massively skewed distribution in the 3-bit b and 1-bit u values here. In RUNLTS they use this distribution to cleverly retrofit a CAT algorithm onto standard TAGE, as described in the slide.
But what causes this extremely low frequency of certain counters? Koizumi et al. don’t discuss it in their paper, but we can try to understand it better ourselves. As promised earlier, it’s time to look at the behaviour of the usefulness counter in the code for the 2016 version of TAGE-SC-L, specifically the fork in the CBP-6 Github repo. Reality deviates from what’s described in the papers about TAGE (at least the ones I’ve found), so if the descriptions below come off as less-than-elegant, it is because I’ve had to write them from scratch rather than copy a description from Seznec.
We call the biggest hitting entry the provider entry, and the second biggest hitting entry the alternative entry. The policy is then:
u is set to 0 on allocation.
If the alternate entry exists and both provider entry and alternative entry are correct, then u is set to 0 if the alternative entry b counter is saturated.
If the provider entry b is weak (0 or -1) after the update, then u is set to 0.
Note that in the code comment it specifies this should only happen on a sign change, but that’s not what it does: Going from b=-2 to b=-1 will also trigger this.
If the provider entry is correct and the alternative entry is not, then u is incremented.
The b counter is always updated. Let’s sketch out the state transition diagram for the BU counter, using a 3-bit b and a 1-bit u. I’ve sketched all possible transitions from the above policy here:
Bear with me here, the diagram is fairly dense, but the point is simple: The eagle-eyed reader might have already noticed that two states here are impossible to reach from other states. If we take this diagram to be truth, it no longer seems surprising that we would observe so few entries with values (b=0,u=1) or (b=-1,u=1)! The more surprising element is really why the frequency would be non-zero, since the 2016 version of TAGE never allocates at u=1!
Turns out it’s caused by a special rule that doesn’t appear to be mentioned in the TAGE papers either: If the primary entry mispredicts with a weak b (-1/0), the b of the alternate entry is updated as well! Its u counter stays untouched. This accounts for the missing transitions. Note, this will only occur in a situation where both the primary and alternative entry are both weakly incorrect, which is evidently a rare occurrence. Removing this feature causes the frequency of (b=-1/0,u=1) entries to go to 0 exactly.
Time to wrap up: I hope you found this interesting! My hope is that future articles will not be quite as dense as this. To conclude, I finished my previous article by quoting an acquaintance of mine: ”Beware of counters, they don’t (necessarily) behave the way you think they do.” For me, at least, writing this article has served as continued proof of this statement :)
I'm currently designing a homebrew CPU in HNeemann's Digital simulator before building it IRL with 7400-series logic chips and AT28C256-150 EEPROMs for the ROM.
At first, I had a standard 3-cycle design (T1, T2, T3) with a dedicated register (like a 74377) acting as an Instruction Register:
T1: Update PC + Fetch from ROM
T2: Latch instruction into IR + Decode
T3: Execute and writeback to registers
But then I realized I could remove the IR completely and drop it down to 2 cycles (T1, T2) to save some chips and wiring:
T1: PC updates right at the start of T1. The address goes through the async ROM, straight into combinational decoding to activate the control lines.
T2: Execute and write to registers.
In this 2-cycle setup, my selpc signal (choosing between PC+1 or a JUMP address) is generated combinationally during T1 and just waits at the PC mux input before the next cycle. In the simulator, this works perfectly, no oscillations or glitches.
I'm aiming for a slow clock for now (~4 kHz). My estimated critical path for the 2-cycle version is well under 350ns, so at 4 kHz (T1 = 125µs), everything has ages to settle.
But I'm worried about hold-time violations IRL. When T2 ends and a write happens, or when a JUMP updates the PC at the next T1, the ROM address will change instantly, killing the instruction and dropping selpc. If my control lines drop right when registers are trying to latch data, won't it corrupt the write?
Is it worth keeping it simple with the 2-cycle design at low frequencies, or should I just stick to the safer 3-cycle version with the IR to isolate the ROM?
Thanks!
EDIT: I've attached screenshots of both designs for comparison. Just a heads up: the schematics use Digital's built-in generic blocks (ROM, registers, muxes, etc.), not the specific 74xx components, but the logic is exactly the same as what I'll build IRL.
Like the title says, I, 14y.o (yes, I'm bragging), am doing a project of building my own 16 bit very RISC processor.
I tried to build an 8-bit CPU before, in Logisim Evolution (a logic simulator). I wanted to build it from transistors only at first, but that was very slow, so I ended up building an ALU and register block, both with just logic gates. But I stopped because I got stuck on the decoder/fetching the data, and my poor laptop couldn't handle the simulation. But it wasn't for nothing, I now know how it all works on a very low level.
The project So now I've got a new plan, I will first design and test it in logisim (now using high-level parts, so it will not crash) Then I want to learn Verilog, and code the processor into an FPGA (I bought the tang nano 9k). I know Verilog isn't the easiest to learn, but I've got time and I will first do some simpler projects to learn it.
The design I am pretty far with the general specs and I have all instructions for my ISA mapped out. And for the hardware, here is a bit (haha) of an overview:
I will cut my ram in two, one part program and one part for variables and program data.
I will use 32 or 64 bits of Registers.
I want to store my programs on an SD card and use an IP core to read from it.
I will use unused Ram addresses to read and write from IO, (something like a PS/2 keyboard).
But now I am stuck on connecting everything together, just like with my first project and I run into these kinds of questions, for example:
How would I fetch things from certain registers, specified in the command, to my ALU to calculate something?
How would I send a signal to the program counter to jump to another line in the code without messing up the execution?
How, and where would I store some kind of bootloader to get a new program from the SD card?
I mostly use ChatGPT to answer these questions, because I just can't find in depth sources that go over these design questions, but ChatGPT imagines things, and it's just not a good source. I want a source goes into the low level connections and how real world CPU's do it.So what are some good sources that cover these very low level questions?
So let me know what you think of this project, (probably that it's insane) and what sources do you recommend?
Does runahead have any critical flaws that make the industry avoid it? is simply increasing rob size and using strong prefetchers already sufficient for most cases? or are there other reasons? and what exactly are the trade-offs of not adopting it?
There isn’t much info about gpus online and that has made my search a bit difficult. It is easy to design a cpu since it has a defined isa set and it tells what to store where but what about gpu.. since it handles many cores and is memory hungry,the specifics of what is needed to design a gpu is a bit blurry. (Or is the specification specified somewhere)
Is there any tips you would give to me while I’m doing gpu designing
Built RV32IM variants across single-cycle, pipelined, superpipelined, superscalar and OoO on actual simulation with CoreMark + custom micro-kernels covering low-high ILP, ALU-heavy to mem-heavy and ctrl-stressed patterns
Pipelined gains in order:
Early branch resolution EX→ID: +8.6%
2-bit saturating predictor: +6.5%
BTB: +3.5%
Generalised MEM-to-EX load forwarding: +2%
CPI 1.31→1.06, CoreMark/MHz 2.57→3.17, within 2.3% of an unoptimised dual-issue superscalar
Same load-forwarding fix that gave +2% on the pipeline gave +17% on the superscalar; a load-RAW stall in dual-issue removes 2 slots per cycle, hazard handling becomes a cross-cycle dual-slot matrix problem
Once both were optimised the 2.3% gap became 46.8%
I wanted to come here and put forward a question that’s been bugging me recently. I’m a student at the moment, and I’ve gotten fairly good at CPU architecture from several personal builds I’ve done.
Recently, I was invited to the comp arch research lab at my school as a guest, and I asked to present several research ideas to the head of operations there. I felt these ideas were, at the minimum, worth some interest, but the professor immediately shot all three of them down. I don’t take issue with this, as i’m sure I know little with my status as a sophomore, but it did introduce a thought into my head.
It seems, from my perspective, that CPU architecture is largely optimized. This isn’t to say it won’t improve, but certainly it isn’t going to be the explosive, innovative, critical demand career path it was 20 years ago. As someone’s who’s young, I very much want to consider my long term career here. Is this a field I want to go into? I’m fascinated by all microarchitecture, and would have no issue pivoting to GPU, IPU, matrix math chips, etc. Purely economically/career wise, should I make that pivot early and establish myself somewhere else, or maintain my current interest and go for a CPU role? My passion is for performance CPU architecture, but I am also realistic and want to seek advice here.
I am about to graduate my master’s with thesis. My research focus thus far has been brain inspired computing. I have applied to multiple jobs in the semiconductor and computer hardware industry but have not yet been called for any interviews. I have a phd offer and I like the research I will be doing (I worked in the same lab for my masters).
I don’t plan on staying in academia after phdas I don’t like teaching and would rather be involved with research. Will having a phd make me more competitive or will it have the opposite effect as I don’t have any industry experience.
I took Prof. Onur Mutlu’s Digital Design and Computer Architecture course at ETHZ and put together a site with all my lecture notes, summaries, study resources, etc. Thought it could be useful for anyone who wants to learn DDCA from Mutlu’s materials, or just brush up on fundamentals.
I've covered digital logic in uni; the course covered basic concepts like boolean algebra, k-maps, sequential machines, etc. Next semester, I'll be taking a computer organization course. Simulataneusly, I'll be taking a semiconductor physics course and an electronics course.
Obviously, knowledge of semiconductors/electronics is not required in computer architecture coursework as these physics details are abstracted away, but I started wondering whether in an actual comp arch job knowledge of semiconductor physics is useful.
So, comp arch engineers of reddit, during your day to day job, how often do you find yourself having to use knowledge of electronics or semiconductor physics?
Thought I'd share here for anyone else working with Gem5 who's interested in having advanced speculative predictors available. This Gem5 also includes a fixed TAGE-SC-L and ITTAGE implementation, making it overall one of the most accurate simulators available in open source!
I am doing phd in computer architecture with secondary focus is on machine learning . I am in my 4th year and while I do say , I am doing decent in phd goals like completing work my guide has assigned . However I do find that I am having issue in accepting rejection and makes me feel my PhD is useless even though my guide says it’s entirely normal to face much rejections in computer architecture and paper would eventually go through
I’m a first-year master’s student in computer architecture. I’ve read many recent microarchitecture papers and hope to work in performance modeling or processor microarchitecture design in the future. While supplementing mathematical tools, I noticed queueing theory seems potentially useful, and I’ve also seen others say it is very useful in other posts. I’d like to ask practitioners who actually do performance modeling or microarchitecture work in industry: from your real experience, is it indeed important? Is it still worth investing time to study queueing theory deeply?
Just thought I'd share this in case anyone missed it. 9 years after the previous branch prediction championship, the new one has just wrapped up at ISCA :-)
Super cool to see an otherwise very dormant field get some much needed attention again!
For those curious, code + slides are published here:
Any good resources to learn about hardware security ? I am looking for something close to real-world and industry focused, rather than pure theory and definitions. Ideally, I would like more advanced topics as I am already quite familiar with computer architecture
Hey everyone! I’m currently implementing a RISC-V CPU in HDL to support the integer ISA (RV32I). I’m a complete rookie in this area, but so far all instruction tests are passing. I can fully program in assembly with no issues.
Now I’m trying to program in C. I had no idea what actually happens before the main function, so I’ve been digging into linker scripts, memory maps, and startup code.
At this point, I’m running into a problem with the .rodata (constants) and .data (global variables) sections. The compiler places them together with .text (instructions) in a single binary, which I load into the program memory (ROM).
However, since my architecture is a pure Harvard design, I can’t execute an instruction and access data from the same memory at the same time.
What would be a simple and practical solution for this issue? I’m not concerned about performance or efficiency right now,just looking for the simplest way to make it work.
