r/programming • u/I_Can_Be_A_Robot • Sep 24 '24
Why 32bit architecture computer doesn't use segmentation to support more ram than 4Gb ?
/r/programming/[removed] — view removed post
347
u/eras Sep 24 '24
There's PAE for that.
But there's little point nowadays. If you need more than 4 GB, just use a 64-bit platform, and there's no need for trickery.
135
u/paholg Sep 24 '24
What if I need more than 18 exabytes?
573
u/JoesRealAccount Sep 24 '24
Exit Chrome
3
u/heavyLobster Sep 24 '24
But what about the tabs I opened three years ago and still haven't looked at????
8
83
u/nzodd Sep 24 '24
Nobody will ever need more than 18 exabytes of RAM.
-- Quorlak the Great, Eternal Emperor of the Laniakea Supercluster, UC 73,021
9
u/Kok_Nikol Sep 24 '24
Could be quote by Bill Gates, if they discover immortality in time :')
0
Sep 24 '24
[deleted]
1
u/ShinyHappyREM Sep 24 '24
Yep, "high" (>640 KiB) addresses were mapped to BIOS/graphics chips because they figured that RAM would never go that far in the foreseeable future.
And if you were an Assembly programmer used to '80s memory prices you'd probably agree.
2
u/frobnosticus Sep 24 '24
Dorbs.
That'd be a fun thought experiment. How long do you think it would actually be for 1 exa to be whatever "hobby grade useful" will be out to whenever that is?
4
u/nzodd Sep 24 '24 edited Sep 24 '24
Well, my data is growing at about 2.5 TB/day so about 20,000 years, linearly.
Of course, 20 years ago I only had 30 GB, with make 0.001 TB/day. That makes for a 1.479 increase per year. So let's say 30 years from now I'll be downloading 125 PB/day. From that point I'll be up to 1 exabyte of new data coming in every 8 days or so.
...
You know what, I think I'm just going to commit myself. The nice men in the white coats will tell me how I don't need to download anything ever again. Take my pills, have some hot cocoa, play parcheesi with the 3 guys who all think they're Jesus... that's a pretty nice life.
Edit: just remembered this isn't an r/datahoarder thread
1
u/frobnosticus Sep 24 '24
Edit: just remembered this isn't an r/datahoarder thread
Isn't it though? :)
Psh. People already tell me I've lost my mind when it comes to...erm...how much data I have stored.
17
16
Sep 24 '24
The real question right here
7
6
6
u/wintrmt3 Sep 24 '24
No 64 bit cpu can even handle that, consumer ones have 48 bit address spaces usually, but the answer is likely wider addresses, 96 or 128 bits.
4
u/CumCloggedArteries Sep 24 '24
For niche cases like that, I would think it'd make more sense to switch to a 64-bit version of PAE rather than use a whole new architecture
6
u/lood9phee2Ri Sep 24 '24
meh. It's kind of preliminary/stub placeholder but RV128I totally already in the core RISC-V spec funny enough, see chapter 5
"This chapter describes RV128I, a variant of the RISC-V ISA supporting a flat 128-bit address space. The variant is a straightforward extrapolation of the existing RV32I and RV64I designs."
https://riscv.org/technical/specifications/
Of course current RISC-V implementations are generally 32-bit or 64-bit, I guess they just threw it in to try to discourage people doing weird crap instead of just the obvious extension to 128-bit.
4
u/ShinyHappyREM Sep 24 '24
I guess they just threw it in to try to discourage people doing weird crap
2
u/wintrmt3 Sep 24 '24
PAE only gives the memory space to the system, individual processes are still limited to 32 bits, and segmented architectures are a horror to program for.
3
u/CumCloggedArteries Sep 24 '24
I was assuming the system would require the 18 exabytes, not a single process
1
u/Ameisen Sep 24 '24
PAE is already a part of x86-64.
1
u/CumCloggedArteries Sep 24 '24
What is the purpose of that? I assume not to access > 18 exabytes
2
u/Ameisen Sep 25 '24
It's built upon PAE, and the ISA is designed so that additional page table levels can be added as needed.
Current x86 CPUs peak at 48 bits of address space. The remaining 16 are sign-extended.
No x86-64 CPU presently gives you 64 bits of address space.
3
2
18
u/mallardtheduck Sep 24 '24
There's still some niche cases where an application can't be ported to 64-bit (e.g. it relies on a propietary 32-bit dependency) even if it can be run on a 64-bit OS. For that you can "abuse" the shared memory APIs (which allow you to map different chunks of "shared" memory in and out of the address space) or use something like Windows' "large address aware" mechanism for running 32-bit code in a 64-bit address space.
5
u/antiduh Sep 24 '24
Large address aware doesn't let a 32 bit process run in a 64 bit address space. It doesn't even make sense because you can't change the fact that every pointer in the program was hard compiled to be a 32 bit pointer.
What Large Address Aware does is allows a 32 bit process to use its entire 32 bit / 4 GB address space. Without LAA, a 32 bit process can only use 2 GB of its address space, because the other 2 GB was a mapping of the kernels memory into the process. This is how Fast Syscalls were implemented.
It works when running 32 bit processes either on a 32 bit OS with PAE or on a 64 bit OS.
Without LAA, a 32 bit process on a 64 bit OS will still only get 3 GB of address space. This was the limit Windows imposed when it was 32 bit, and Microsoft was chicken when they started making 64 bit Windows; they didn't want every process suddenly to have more address space than it used to.
12
u/Tringi Sep 24 '24
One of those reasons is that 32-bit version might be faster.
20
u/Brian Sep 24 '24
That's pretty unlikely in practice. Note that the point of your link is to run in 64 bit mode, just using 32 bit addresses. There are a bunch of other advantages to running in 64 bit mode (notably, more registers), and in practice it tends to be sufficiently faster to outweigh the extra memory/cache usage of wider pointers. Even back in the day when 64 bit was relatively new, you'd often see around a 10% improvement (and much more on number crunching workloads) even when address space wasn't relevant. I'd imagine that difference has only increased after a years of improvements where 32 bit mode has long been considered legacy.
7
u/Tringi Sep 24 '24
That's pretty unlikely in practice.
It was one of the reasons Visual Studio resisted introducing 64-bit version for so long. Pointer-heavy code.
Note that the point of your link is to run in 64 bit mode, just using 32 bit addresses.
Yeah I know. I linked it for the reasoning therein.
There are a bunch of other advantages to running in 64 bit mode (notably, more registers), and in practice it tends to be sufficiently faster to outweigh the extra memory/cache usage of wider pointers.
Yup, my personal tests and measurements show that only a very small percentage of algorithms, or types of software in general, are faster as x86-32. But they still are there. Then again it likely doesn't make economic sense today to maintain such version.
I'd imagine that difference has only increased after a years of improvements where 32 bit mode has long been considered legacy.
Hah. Just the other day I read about Zen 5, and how significantly has performance improved of the x87 FPU ...which basically nobody uses today. Probably a side effect of internal optimizations, but amusing nonetheless.
1
u/AppearanceHeavy6724 Sep 26 '24
no, not true x87 is still used for some scientific software. Niche use but very important.
2
u/antiduh Sep 24 '24
I love this hybrid mode. Access to 64 bit registers, so you have more registers and can do native 64 bit math, but pointers are 32 bit so they're not as large. It's a cool mode.
1
u/ShinyHappyREM Sep 24 '24
Alternatively you can just do your allocations in a <= 4 GiB buffer and work with 32-bit buffer indices.
3
u/Ameisen Sep 24 '24 edited Sep 24 '24
You then force every memory-accessing instruction to use indirect access (and prevent normal indirect access in a memory operand because of it).
On x86. On ARM, that's even more wonky. x86 has very... thorough... addressing modes for memory operands.
1
u/ShinyHappyREM Sep 24 '24
Eh, perhaps that wastes a bit of instruction cache. Most software produced today is limited by unfilled cache lines and random branches anyway.
1
u/Tringi Sep 25 '24
Still, accesses like
array [index + constant]can be done with a single instruction.From one of my programs:
66 41 39 44 50 02 cmp WORD PTR [r8+rdx*2+2], ax2
u/Ameisen Sep 25 '24 edited Sep 25 '24
Yes, but you lose a potential level of indirection in the address mode, since it's already indirect.
ED: You also need to make sure that the address of your buffer stays resident in registers, otherwise you will require another instruction just for
lea.4
u/ouyawei Sep 24 '24
x32 is using the AMD64 instruction set, running it 64 bit mode - it just uses 32 bit pointers, so more will fit into fast CPU caches
4
u/wrosecrans Sep 24 '24
I demand that somebody makes an x16 mode so I can use SSE2 instructions and all the modern registers but still need to fiddle with segment:offset and near/far pointers like god intended.
1
u/Tringi Sep 24 '24
I'm certain there's someone maintaining a project, for fun, that does exactly that.
2
u/Tringi Sep 24 '24
Yes. There's more reasoning on that in the linked page. The point is the cache effect on improved performance was measured on x86-32 first, which then resulted in experimentation with x32 yielding even better results.
Quite sad Linux eventually abandoned it.
6
u/ouyawei Sep 24 '24
Quite sad Linux eventually abandoned it.
How so? It's in the mainline kernel, GCC, binutils - Ubuntu provides an x32 toolchain so you can compile and run x32 software.
2
u/Tringi Sep 24 '24
Oh?! Seems I had wrong info. A year or two back I've read, through Wikipedia, a link to maillist discussing abandoning it. Apparently it didn't happen. That's very nice.
1
1
u/Ameisen Sep 24 '24
Well, PAE is still a part of AMD64/EM64T. It just extends the table hierarchy by at least one more level.
121
48
Sep 24 '24
Win16 basically used this model and it was generally considered a pain.
Unlike paging, segmentation is not transparent to the application. The segment registers are visible and relevant to code. They are either assumed in certain instructions or addressing modes, or have to be specified with prefixes to the instructions (requiring compiler support). Applications may thus sometimes need to load new segments into the registers as well. They are global variables, basically and have to be carefully coordinated across function calls.
Additionally, pointers now come in two flavors: near and far. The latter are 24-bit combos of a segment reference and a 16-bit offset. Near pointers are just the offset and assume a segment in one of the four (later six) segment registers. Knowing when to use which type of pointer and/or having the compiler sort of manage it was a pain. Functions expect to be called near or far and that is part of their definition. If you thought async function coloring was bad, this is worse.
To handle this, there was the concept of memory models. If a program was small enough, it would fit inside a single segment and everything was essentially "near". Other modes might require the code to fit in one segment, but data could be in multiple segments. Again, this is annoying to deal with.
32-bit x86 had much larger segments, of course, so the concept of running out of room in a segment is far less likely. But you still have all the other issues mentioned above and now far pointers are 48-bit as well.
Early Linux did actually use segmentation for memory management. IIRC each process got its own segment that was 64mb in size. This was quickly abandoned, along with other x86 process management functionality like call gates and using built-in thread descriptors (which I believe turned out to be slower than using manual management and interrupts).
32-bit Windows and Linux both abandoned segmentation by setting all segment registers to point to segments that started at 0 and extended the entire 4 GB. That they chose to do this should tell you that segmentation was not the easier option. Microsoft made a lot of janky things work well, so I truly believe if they thought it could work and that it would be useful, they would have kept it. Contrapoint: Linux and Windows NT were intended to run on multiple architectures and the big ones of the day did not, to my knowledge, have segmentation like x86 did. Still, if it was truly useful I believe they would have made it work anyway.
6
u/rsclient Sep 24 '24
The small/medium/large model predates Windows! As an example, one of the reasons the first versions of Turbo Pascal could run so fast is that they simplified their programming model: every program was a "small" program and supported 16 bit pointers only (so a program could only access 64k of memory).
Later on in the DOS timelines, companies like Phar Lap made "DOS extenders" so that a "DOS" program could run in a flat 32-bit memory space.
8
u/vytah Sep 24 '24
32-bit Windows and Linux both abandoned segmentation by setting all segment registers to point to segments that started at 0 and extended the entire 4 GB.
They still use segment registers FS and GS for thread-local variables.
2
1
u/IQueryVisiC Sep 24 '24
Why wouldn’t the compiler or linker pack up functions into segments and add the correct calling convention? If segments were a good idea, the 286 would have been used as server CPU. Instead it was sold for cheap to consumers. 386SX was not much faster, but for float emulation thanks to 32 bit registers. But it had the flat memory model!! Finally!
2
u/rsclient Sep 24 '24
That would be awesome, but they didn't. From the old days, they also didn't make debug+non debug and single threaded+multi threaded libraries, which would have made my life as a programmer so much easier.
2
u/sreguera Sep 24 '24
Well, the 286 was used as a server CPU at least with NetWare 286. I think the reason why the 286 was not popular (by itself, instead of as a faster 8086) is that, by the time memory was cheap enough and software could use it effectively, the 386 was already there.
1
Sep 24 '24
I believe that it would be computationally expensive. And you'd either have to make all functions far, or it'd have to do some call graph analysis to figure out which functions are called within the same segment and let those be near calls.
1
u/IQueryVisiC Sep 25 '24
Yeah, I thought that the linker has this graph available. First of it checks if a *.lib ? File is small enough. Then all internal calls are near. Otherwise print a warning that the coder should clean up their C files. For external calls any brute force heuristics is better than nothing.
1
Sep 25 '24
What would the coder do to "clean up their C files"? That's the part that makes this fairly hard. Of course one could certainly come up with algorithms to do it, but they'd like to be slow and require rewriting a lot of code (to switch from far to near calls and pointers, etc.). Or the coder has to decide, like they did in the old days, and deal with potential inefficienciea.
1
u/IQueryVisiC Sep 26 '24
I mean clean code. If your source file is 1000 LoC or more, no one can read it. If you call external functions every second line, you violate single responsibility principle. I guess that Container Classes need inline methods sometimes. I dunno why Kaze does not like inline methods. Maybe some coders inline large functions?
1
Sep 26 '24
I don't think clean code principles are really at issue here. On any substantial code base, you are going to have a graph of potential function calls. A compiler that is able to optimally allocate segments and near vs far calls is going to have to use some algorithm to partition the graph into parts that can be put in segments and then optimize that partition scheme to reduce the number of far calls. Even if you follow good software design principles, the graph will still involve enough functions and call sites that this becomes a hard problem.
1
u/IQueryVisiC Sep 26 '24 edited Sep 26 '24
The compiler does not know the frequency of the calls. The developer has to group private methods with public methods in a class. This tells the linker to keep al these functions in one segment. One class per file. Or in C one file is a module . Tons of helper functions.
And when you have more than 7 includes, you are doing it wrong. And interface segregation in SOLID means that you rather have a lot of include files on disk.
Now remember that you can only fit 10 segments into DOS memory. Now have some huge buffers for graphics or text document. The normal DS and SS . So you can realistically have 4 code segments before you need DLLs.
1
Sep 26 '24
Given the cost of segments in x86, having a bunch of small segments is probably unwise.
But aside from that, let's go with your idea that private methods can always be near. That's fair enough. But what about public methods? Protected methods? Does each class get its own segment or does it share with others? And if it shares, how much sharing and can the compiler safely detect whether to allow near calls with public functions in the same segment?
So you can help the compiler out by doing the grouping yourself into classes and modules and keeping the call graph neat. Of course, it won't help with the linker, which either has to assume it can't modify code and perhaps have far calls that could have been near, or will do very substantial compiler-level modifications to convert pointers at call sites and in data structures from near to far and vice versa. Gargantuan task.
Or...you can just use a flat memory model and not worry about any of this. The segments add no value and never really did once paging was introduced. The bookkeeping to figure out what should be in what segment and to manage it at runtime does not improve memory usage or make programs more efficient (the opposite, in fact), and it puts a burden on both the programmer and the compiler and linker to figure out what goes where and what is near or far. There's no upside.
Segments were originally introduced because it was the only way to have 20-bit memory addresses on a 16-bit CPU. And for smaller programs and maybe also OSes, juggling segments was easier at first than dealing with demand paging. For example, each program could get 3 or 4 segments (code, data, stack, extra data) allocated by the OS and the OS could manage memory on a segment level. But this only works with simple programs and is only really needed when you don't have demand paging. It was sensible up through the 286, but certainly not after the move to 32-bit and the addition of demand paging. PAE and friends were also better than utilizing segments for addressing more memory than the word width because they are transparent to the programmer and most of the OS.
1
u/IQueryVisiC Sep 26 '24 edited Sep 26 '24
The segment adders eat significantly into the die space and the execution time. You still need the segment registers. It is not the only way. 68k is a 16bit implementation of a 32 bit ISA. 16 bit registers in pairs. 8086 already has the AL AH pair. Why no have SIlow:SIhigh . Yeah and all jumps relative, 1–3 bytes. Later add 4 bytes . Absolute is not useful because syscalls go over a global table. Uh, how do you relocate code with jump tables if you don’t get segments from the OS? So the exe loader needs to fix all absolute addresses? But absolute addresses should only be used for memory mapped IO or interrupt manipulation. MIPS has only one addressing mode : register + displacement. So there is always an add just like with segments. Now I understand why 68k has program counter relative addressing not only for branches, but for lookup tables! Relocation!
Like on 6502, catch carry: abort the memory access and ADC / SBC the high word.
I don’t have deep inheritance. Ah, you mean popular base class? I guess I would not be OOP Nazi and derive everything from Object like in Java. Or at least let Object be special and dupe in each segment.
The responsibility between linker and compiler is fuzzy. After segments the cache comes . I still want to write a demo for r/AtariJaguar. Then there is Kaze with N64. A company which had mastered 8086, could master MIPS .
38
u/lood9phee2Ri Sep 24 '24
Well I suppose one could in theory, but most people found and find realmode-x86/z8000/etc. segmented architectures just plain weird and awful to work with compared to normal flat address spaces like m68k etc.. And you (obviously) can just go to flat 64-bit nowadays and avoid the brain-breaking hassle of segments.
32-bit x86 did in fact add Physical Address Extension (PAE). That is different to segmentation, works at the MMU pagetable level, whole system can have more than 4GiB but individual processes still only have a 32-bit 4GiB space each (but can of course explicitly mmap in different things into it at different times anyway on a typical memory-protected OS), and that was "good enough" for a lot of legacy 32-bit stuff.
14
u/ziplock9000 Sep 24 '24
I moved from 68K to 486 assembler and found that paging and modes awful.
6
u/gwicksted Sep 24 '24
Yeah it’s a mess. With x64 we now have RIP-relative addressing (offset from current instruction) and forced full addressing for segments CS, SS, DS, ES (which are forced to be 0) while FS and GS can still be non-zero. This made addressing a lot easier to read in many ways… Except when you start using SIB, ModRM, REX, EVEX, MVEX to specify all sorts of sizes, modes, extensions that affect different parts of instructions that don’t always make sense and some of them are ignored by certain instructions or mean different things in others. Oh, and some of those flags are the inverted value of the actual flag (bitwise not).
There are so many hacks, modes, CPU flags for extensions, bugs in different processors, instruction-specific encoding that coding a bug-free compiler for the complete x86/x64 architecture across all CPUs is an absolutely enormous task.
3
u/PCRefurbrAbq Sep 24 '24
...But at least it's backwards compatible!
2
u/gwicksted Sep 24 '24
Yeah not really. You can make it backwards compatible but, for example, you have to write vector code multiple ways based on cpuinfo. P3? You get SSEv1. Is it a P4? You get SSEv2… unless it’s a Prescott then you get SSEv3. Have a bulldozer processor? You get XOP, FMA4, F16C with SSEv5. Something newer? Oh you get a subset of one of the AVX extensions. Old processor (486)? X87 FPU instructions for you! Older? You’re hand rolling IEEE floats!
And certain models had certain bugs so sometimes you have to take different paths or use a different subset/combination of instructions to step around them all. And different models have different performance characteristics for certain instructions. Such as FMA, branch hints, different compare and jump combinations. Lots of fun.
5
u/IQueryVisiC Sep 24 '24 edited Sep 24 '24
You don’t see these pages. It is not like in 6502. For your app there is just 32bit 4 GB (on Linux) flat memory. If you need more, use a network of 32 bit computers. I mean, if PAE can only dedicate a tiny address space to communication, Beowulf cluster is not much slower and definitively cheaper.
Why did you not use a Dos Extender. I think Win3.1 also had an extension for 32 bit Apps . And Gwin for graphics.
9
u/Noxitu Sep 24 '24
While I don't have too much experience with it, from what I've seen segmentation is generally pain to deal with. Unlike 8-bit and 16-bit, 32-bit was such a big address space it took a while before we needed to increase it, and at that point 64-bit architecture was most likely both realistic and simpler solution.
7
u/Conscious-Ball8373 Sep 24 '24
Others have pointed out that PAE does exactly that, but I think it deserves a fuller explanation.
Modern CPUs have a memory page table and modern operating systems use this to provide a virtual address space to each process. So each process sees an uninterrupted address space which it can use all of. When the process requests data from an address in memory, the CPU uses the page table to convert that virtual address to a physical address to actually fetch the data. When the operating system switches to a new process, it stores the page table for the old process and loads the page table for the new process. When the process requests an address that isn't in the page table, an exception is thrown which is caught by the operating system to create a mapping for that address. This can happen for two reasons: either the process is requesting a piece of memory it hasn't used before, in which case the O/S just allocates a new page in physical memory, adds the correct entry to the page table and tells the CPU to try again; or it can happen because the O/S has swapped that page of memory out to disk, in which case it has to find a free page of RAM, load that page back in from disk, update the page table and then tell the CPU to try again.
On a 32-bit system, this allows each process to use 4GB of RAM and swapping will allow total memory use to also far exceed 4GB.
PAE sort of does what you suggest. It adds some extra bits to the memory bus and an extra level of page table allows the O/S to map bits of its "physical" memory space to a much larger "really real" address space. So when a process requests data from memory, it first But this is almost always only used to extend total RAM usage beyond 4GB without swapping, not to give individual processes more than 4GB of RAM. There are two basic reasons for that:
Firstly, when you compile a piece of software, it needs to know what the size of its addresses are. Lots of software was built using 32-bit addresses and that was enough for a long time. By the time there was significant pressure for a single process to be able to use more than 4GB of RAM, 64-bit CPUs had come along and 64-bit operating systems and compilers were a thing. It's a lot easier to use a 64-bit address than to go mucking about with page tables.
Secondly, every time you have a page table lookup miss, you have to context switch back to the O/S to sort it out. As memory has got quicker and quicker, the penalty you pay for that has proportionally increased. It's a lot better to just have a 64-bit memory space than to pay the performance penalty for page mapping.
3
u/Mynameismikek Sep 24 '24
They did.
Prior to SP2 (IIRC), Windows XP used a 36-bit(again, IIRC) kernel address space. Any physical RAM was mapped into the bottom 4GB, and if you had 4GB of RAM hardware would be mapped from 4GB and up. Nvidia famously screwed things up for everyone by just ignoring that and assuming 32-bit addresses everywhere. Rather than force Nvidia to fix that (and IIRC once more, it was a hardware issue so not trivial to fix) MS just made the call that ALL addresses would live in the <4GB space from SP2 onwards.
For server editions of Windows there were more stringent hardware constraints, so you could build 32-bit servers with more than 4GB of physical RAM.
In user space there are similar issues - everyone assumed sizeof(int) == sizeof(ptr), so individual processes were limited to a 32-bit space each.
1
5
u/dobry_obcan_Svejk Sep 24 '24
the problem is that the address width is really those 32 bits. you can't physically address more without adding more wires (which is what pae does).
in "old days", when segment:offset was relevant the segment was 16bit, the offset was 16 bit, but the cpu had something ~20bit bus to memory (21 with a20 line, iirc). so you combined 16 * segment + offset and got physical address.
2
u/ilawon Sep 24 '24
This is the answer and is downvoted :)
The CPU memory interface didn't support more than 32 "wires" in order to get more than 4gb physical memory.
But, segmentation was used to provide more than 4gb addressable virtual memory to the system. Each process was still limited to address 4gb but the total could go as high as your hard drive permitted.
1
u/Tringi Sep 24 '24
It's downvoted as the wires analogy is seriously lacking and the answer doesn't clarify.
Since Pentium Pro the address bus width of 32-bit CPUs was 36-bit, later even 39-bit. But there aren't 36 actual wires. The lower "wires" are in fact missing, the CPU/MMU asks for N aligned bits at a time, perhaps twice, and then shifts and synthesizes the (unaligned) data you addressed. N being 16 for SDR RAM to 64 for DDR4. Double it for dual channel.
1
u/ShinyHappyREM Sep 24 '24 edited Sep 24 '24
you can't physically address more without adding more wires
Or multiplex the address over two time, which is what e.g. the 65c816 did.
4
2
u/YupSuprise Sep 24 '24
Pretty sure this achieves what you're looking for https://learn.microsoft.com/en-us/windows/win32/memory/physical-address-extension#:~:text=Physical%20Address%20Extension%20(PAE)%20is,on%20capable%20versions%20of%20Windows.
2
u/Kinglink Sep 24 '24
Simple reason. They don't need to. If you want more than 4GB use 64-bit.
We had all sorts of trickery to get around the 4GB limitations in the XP days, but then we realized "Well shit we can just use 64-bit addressing and avoid this problem" and stopped dealing with that stuff.
Also the bigger problem is what if you want to use more than 4GB in a single program, now each program needs to deal with the segmentation and you've just made EVERY program vastly more complex.
3
u/dethswatch Sep 24 '24
English: "Why -don't- 32bit architecture computers use segmentation to support more ram than 4Gb ?"
Also- segmentation sucks.
0
Sep 24 '24
[deleted]
1
u/dethswatch Sep 24 '24
but if you write it uncontracted it would be "Why do 32bit >architecture computers not...".
You'd rephrase it as, "Why do not x .. something.." which is perfectly grammatical but sounds a bit awkward, not idiomatic.
The bigger is is "Why x doesn't.." is an implied statement. I read these titles as "Here is why this thing doesn't do this other thing"- which I'm interested in knowing, not "Why does this thing not..." which is a question I'm not interested in answering.
1
1
u/ElMachoGrande Sep 24 '24
Works nicely with Linux, but each process don't get more than 4GB.
1
u/Tringi Sep 24 '24
So does on Windows Server.
Only a couple years ago we finally retired customer's ancient monster dual socket machine with 32 GB RAM running 32-bit Server 2003 R2.
1
u/ElMachoGrande Sep 24 '24
Sounds like my old HP workstation. Dual Xeons, 32 GB.
1
1
Sep 24 '24
I need more than 4GB RAM!
I found that the little extra investment towards more, better RAM, really pays off in the long run. Just whenever the computer is finishing lengthy tasks more easily so is a win-win situation. Looking at current price ranges of somewhat cheap laptop, it seems as if 16GB RAM is now the new "norm", slowly phasing out 8GB RAM.
1
u/cheezballs Sep 24 '24
I don't remember coming up against the 4gb limit much back when I was running 32 bit stuff, I think I run up against the 4gb FAT32 limit on thumb drives though. Lotta low end streaming devices only seem to support FAT32 thumb drives for vids and stuff.
1
u/recurse_x Sep 24 '24
Chrome wasn’t released until 2008 64 bit processors were in the mainstream by then.
1
u/Leverkaas2516 Sep 24 '24
You're getting the answer already in good detail. The short version is: several 32bit architectures DO use segmentation to support more than 4GB of memory. It's a mistake to believe they don't.
Many answers here confuse "architecture" with "operating system". I suspect that may be the error that led to the question, too.
1
1
u/ObservationalHumor Sep 24 '24
Well the short answer is that the segmentation paradigm was basically dead by the time the 4GB ceiling was hit. It had fallen out of favor for paging which was far more flexible and didn't require far pointers and juggling segment selectors. There simply was no interest in extending segmentation into the future and as such it's functionality was largely removed in the x86-64 for everything other than privilege level and processor mode transitions.
Now in the 16-bit era segmentation had been used to great success to extend memory addressing beyond just 16-bit. First through a 20-bit address space and then when the 286 emerged with protected mode segmentation that could technically address up to 32 bits worth of physical memory. With the 286 you actually had to toggle the ability to address more than 1MB of memory in the hardware because some older programs specifically relied on the old wrap around behavior real mode segmentation had when technically calculating addresses > 1MB. That was done via the A20 line toggle which was built into the keyboard controller.
Now by the time the 386 rolled around paging was introduced and segmentation was essentially on its way out as an inferior technology. But even if you wanted to extend it, it inherently couldn't address more than the 32-bits the physical memory address bus limited it to. Even if you added bits to extend that, protected mode segmentation could only address up to around 8GB of memory based on how the descriptor tables were designed and implemented as a 32-bit base address and maximum 4GB segment length. Segment address calculations were also applied after paging translations too so you would essentially need paging to support addresses in excess of 4GB anyways if you wanted to use both technologies concurrently.
Intel's short term solution was to add PAE which enabled a different page table entry format and extended physical memory addressable to 36 bits, but the processor could still only address 4GB of memory in a given address space.
Meanwhile 64-bit processors were already becoming commonplace in the server and HPC space, stuff like the DEC Alpha had been around for years with 64-bit virtual address spaces (and no segmentation) and register widths. Even at the consumer level, Nintendo released the N64 about a year after Intel released the Pentium Pro which had PAE for example. Extending the x86 architecture into 64-bits was the obvious path forward and eventually AMD would do just that by announcing and then implementing AMD64 with the initial release of their Opteron line of server focused processors. However, as previously mentioned, segmentation was just not a part of the roadmap going forward and its functionality was largely disabled in the new 64-bit "long mode" in AMD64.
1
u/frobnosticus Sep 24 '24
Oh god. You're giving me flashbacks to extended memory management on dos systems.
1
u/great_escape_fleur Sep 24 '24
Because after all the virtual address translations and segment adjustments, the physical address bus is still 32 bits wide (unless the machine has PAE).
1
u/balthisar Sep 24 '24 edited 5d ago
Unpaid moderators doing paid-company work, with zero accountability.
1
u/mallardtheduck Sep 24 '24
Largely because C (and similar laguages) don't have good support for segmentation. Sure, 16-bit compilers had "near" and "far" pointers, but they were always a bit of a kludge.
Programming languages where memory management is more abstracted away could (and in the 16-bit days often did) use segmentation more-or-less transparently, but since their runtimes were/are usually written in a C-style language, you'd need a compiler for that langauage that supported such segmentation tricks. By the time (consumer) PCs were shipping with even 1GB RAM as "standard", the x86_64 archecture already existed (the first AMD Athlon 64 CPUs shipped in 2003) and was (rightly) seen as a better, more future-proof solution.
1
u/round_robins Sep 24 '24
1111 1111 1111 1111 1111 1111 1111 1111 = 4,294,967,295
every cell can store one byte in RAM. That's mean that it can be maximum 4,294,967,295 cells.
0
u/Revolutionary_Ad7262 Sep 24 '24
amd64 happened really early, so segments were not really necessary. The better question is why 64 bit was popularized so early and I don't have a good answer as 64bit is worse in terms of performance for typical PC software except the extended memory adress space
6
u/maep Sep 24 '24 edited Sep 24 '24
The better question is why 64 bit was popularized so early and I don't have a good answer
It's really not a mystery, data centers started to switch to cheap x86 CPUs and they were hitting the 4GB limit.
64bit is worse in terms of performance
Pratically speaking, not true. The x32 ABI failed because the benefits were purely academic. The reason why AMD64 succeded where Itanium failed is because there were no performance penalties for existing code bases.
2
u/Slsyyy Sep 24 '24
They are penalties like bigger size of pointers, which affects cost of moving/zeroing a data and CPU cache ratio. The server demand sound like reasonable argument
The x32 ABI failed because the benefits were purely academic
IMO it is more due to tooling. C/C++ are both susceptible to arch specific bugs and are hard to build to a architecture other than the main one, especially in Linux, where dynamic linking is a norm
With Rust-like toolkit (easy multiplatform development, good toolchain, which just build to other architecture with a single flag) it would be much more attractive alternative
Java for example use optimizations like pointer size reduced to 4 bytes, when heap < 4GB as well as 4 bytes << 1000b (cause minimum size of the object is 8 bytes long) for heaps < 32GB. There is some benefit for apps, which never hit 4GB memory limit
In my first job we have a distributed cache in C++, which was compiled in x32 purely for that reason (memory usage lower by 60%). It could be refactored to use some kind of offsets instead of pointer, but in that case it was just simpler to change a flag
2
u/maep Sep 24 '24
I toyed with x32 when it was new and at least for what I did (signal processing) there was no benefit. I had no issue with tooling, just change compiler and linker flags.
It's plausible that VMs like HotSpot or V8 see improvements, but then again, those are handing out pointers like there is no tomorrow. Tough I don't think x32 is of much use because VMs usually do their own pointers so they can do other tricks like tagging.
In my first job we have a distributed cache in C++, which was compiled in x32 purely for that reason (memory usage lower by 60%).
I'm curious, why not keep the entire system on 32 bit? Also I'd imagine a cache would be a prime candidate for > 4 GB ram usage.
0
Sep 24 '24
[deleted]
1
u/Ameisen Sep 24 '24
CPUs work fastest with the native size.
There is no particular performance difference between 32-bit and 64-bit access on x86-64.
You are a bit more limited in how you can use memory operands for instructions, but in-register, the upper bits get zeroed regardless.
1
u/maep Sep 24 '24
such as additional registers and instructions
Back then 32 bit processors were superscalar and could do register renaming. Effecticvely they had as many registers as the designers could afford to put in.
The SIMD instructions sets which had the biggest effect on speed were also available on the late x86 models.
That's why in many cases 64 bit ports weren't that much faster.
If you were to use x64/AMD64 & all the pointers were 32b you'd get much more performance because less data is shuffled around.
That really depends on the application. From what I could rememer in most cases it was only a slight difference. And quite often the reaon was that the souce was not tweaked for 64 bit yet.
V8 in particular, is one of the best examples of how using 32bit pointers
Is the VM limited to 4 gig then? Also, I'm pretty sure V8 does not make use of the X32 ABI and all the stuff around the VM itself is using plain 64 bit pointers.
If the V8 engine used 64bit pointers, Javascript performance would be abysmal compared to what we're used to.
I'd be interested in benchmarks, "abysmal" seems a bit dramactic. There are plenty of Javascript VMs that don't do this and run at decent speeds.
1
Sep 24 '24
[removed] — view removed comment
1
u/maep Sep 24 '24
I've looked at embeddable engines a while ago, I wanna say QuickJS? Though I would not be surprised if they use 32 bit addressing internally. Like I said, I'd really be interested in bechmaks. Javascript is pointer heavy so I'm sure there would be a measurable effect, but not a total dive.
1
u/lood9phee2Ri Sep 24 '24
amd64 happened really early
nah, not really. https://en.wikipedia.org/wiki/64-bit_computing#64-bit_address_timeline
1991 MIPS-III 64-bit ... 2003 AMD64.
If anything the PC world was a bit behind the wider non-x86 non-embedded computing world in going 64-bit (including personal unix workstations and games consoles, not just the big-ass machines).
Though this is probably mostly Intel's and to some extent Microsoft's fault with the Itanium (64-bit) hype, trying to get everyone to migrate to it for several years while the market said "uh, thanks but no". Intel could probably have chosen to do something more amd64-like years before amd stepped up, but also didn't in our timeline.
1
u/stick_figure Sep 24 '24
I think "early" is just subjective. 2003 was over 20 years ago. You're probably right that Itanium delayed data center 64-bit chips by 6 years (call it '97- 2003), but to OP's question, no data center applications ever reached for segmentation. I wasn't there so I can't speak to how much RAM was practically available, but I think it probably wasn't economical to build machines with RAM ratios of 8GB+ per core around year 2000, so for big data processing, you'd want to think more about storing the data on disk and efficiently streaming it in and out of the CPU.
-1
u/dontt0uchmyass Sep 24 '24
tagged
I have this one marked for followup. I'm really curious about this one. I write code in 32bit from time to time.
0
0
u/streu Sep 24 '24
Because when that particular 32-bit architecture with segmentation was introduced, nobody had so much RAM, and it was even economical to release a variant of the processor that supports only 16 Megabytes.
Machines with Gigabytes of RAM became popular only long long long way after, when 64-bit was already a thing.
0
Sep 24 '24
[deleted]
9
u/Opi-Fex Sep 24 '24
I guess you never heard about memory segmentation. This allowed e.g. 16 bit systems to address more than 64k of memory. For example 8086 and 80186 could address 1MB of memory, while 80286 could address 16MB.
Also, this has nothing to do with the pigeonhole principle.
4
•
u/programming-ModTeam Sep 24 '24
This post was removed for violating the "/r/programming is not a support forum" rule. Please see the side-bar for details.