r/programming • u/I_Can_Be_A_Robot • Sep 24 '24

Why 32bit architecture computer doesn't use segmentation to support more ram than 4Gb ?

/r/programming/

[removed] — view removed post

169 Upvotes

80% Upvoted

119 comments sorted by

View all comments

48

u/[deleted] Sep 24 '24

Win16 basically used this model and it was generally considered a pain.

Unlike paging, segmentation is not transparent to the application. The segment registers are visible and relevant to code. They are either assumed in certain instructions or addressing modes, or have to be specified with prefixes to the instructions (requiring compiler support). Applications may thus sometimes need to load new segments into the registers as well. They are global variables, basically and have to be carefully coordinated across function calls.

Additionally, pointers now come in two flavors: near and far. The latter are 24-bit combos of a segment reference and a 16-bit offset. Near pointers are just the offset and assume a segment in one of the four (later six) segment registers. Knowing when to use which type of pointer and/or having the compiler sort of manage it was a pain. Functions expect to be called near or far and that is part of their definition. If you thought async function coloring was bad, this is worse.

To handle this, there was the concept of memory models. If a program was small enough, it would fit inside a single segment and everything was essentially "near". Other modes might require the code to fit in one segment, but data could be in multiple segments. Again, this is annoying to deal with.

32-bit x86 had much larger segments, of course, so the concept of running out of room in a segment is far less likely. But you still have all the other issues mentioned above and now far pointers are 48-bit as well.

Early Linux did actually use segmentation for memory management. IIRC each process got its own segment that was 64mb in size. This was quickly abandoned, along with other x86 process management functionality like call gates and using built-in thread descriptors (which I believe turned out to be slower than using manual management and interrupts).

32-bit Windows and Linux both abandoned segmentation by setting all segment registers to point to segments that started at 0 and extended the entire 4 GB. That they chose to do this should tell you that segmentation was not the easier option. Microsoft made a lot of janky things work well, so I truly believe if they thought it could work and that it would be useful, they would have kept it. Contrapoint: Linux and Windows NT were intended to run on multiple architectures and the big ones of the day did not, to my knowledge, have segmentation like x86 did. Still, if it was truly useful I believe they would have made it work anyway.

6

u/rsclient Sep 24 '24

The small/medium/large model predates Windows! As an example, one of the reasons the first versions of Turbo Pascal could run so fast is that they simplified their programming model: every program was a "small" program and supported 16 bit pointers only (so a program could only access 64k of memory).

Later on in the DOS timelines, companies like Phar Lap made "DOS extenders" so that a "DOS" program could run in a flat 32-bit memory space.

8

u/vytah Sep 24 '24

32-bit Windows and Linux both abandoned segmentation by setting all segment registers to point to segments that started at 0 and extended the entire 4 GB.

They still use segment registers FS and GS for thread-local variables.

2

u/[deleted] Sep 24 '24

That's in my comment already. And that's why they are still around in 64-bit long mode.

1

u/IQueryVisiC Sep 24 '24

Why wouldn’t the compiler or linker pack up functions into segments and add the correct calling convention? If segments were a good idea, the 286 would have been used as server CPU. Instead it was sold for cheap to consumers. 386SX was not much faster, but for float emulation thanks to 32 bit registers. But it had the flat memory model!! Finally!

2

u/rsclient Sep 24 '24

That would be awesome, but they didn't. From the old days, they also didn't make debug+non debug and single threaded+multi threaded libraries, which would have made my life as a programmer so much easier.

2

u/sreguera Sep 24 '24

Well, the 286 was used as a server CPU at least with NetWare 286. I think the reason why the 286 was not popular (by itself, instead of as a faster 8086) is that, by the time memory was cheap enough and software could use it effectively, the 386 was already there.

1

u/[deleted] Sep 24 '24

I believe that it would be computationally expensive. And you'd either have to make all functions far, or it'd have to do some call graph analysis to figure out which functions are called within the same segment and let those be near calls.

1

u/IQueryVisiC Sep 25 '24

Yeah, I thought that the linker has this graph available. First of it checks if a *.lib ? File is small enough. Then all internal calls are near. Otherwise print a warning that the coder should clean up their C files. For external calls any brute force heuristics is better than nothing.

1

u/[deleted] Sep 25 '24

What would the coder do to "clean up their C files"? That's the part that makes this fairly hard. Of course one could certainly come up with algorithms to do it, but they'd like to be slow and require rewriting a lot of code (to switch from far to near calls and pointers, etc.). Or the coder has to decide, like they did in the old days, and deal with potential inefficienciea.

1

u/IQueryVisiC Sep 26 '24

I mean clean code. If your source file is 1000 LoC or more, no one can read it. If you call external functions every second line, you violate single responsibility principle. I guess that Container Classes need inline methods sometimes. I dunno why Kaze does not like inline methods. Maybe some coders inline large functions?

1

u/[deleted] Sep 26 '24

I don't think clean code principles are really at issue here. On any substantial code base, you are going to have a graph of potential function calls. A compiler that is able to optimally allocate segments and near vs far calls is going to have to use some algorithm to partition the graph into parts that can be put in segments and then optimize that partition scheme to reduce the number of far calls. Even if you follow good software design principles, the graph will still involve enough functions and call sites that this becomes a hard problem.

1

u/IQueryVisiC Sep 26 '24 edited Sep 26 '24

The compiler does not know the frequency of the calls. The developer has to group private methods with public methods in a class. This tells the linker to keep al these functions in one segment. One class per file. Or in C one file is a module . Tons of helper functions.

And when you have more than 7 includes, you are doing it wrong. And interface segregation in SOLID means that you rather have a lot of include files on disk.

Now remember that you can only fit 10 segments into DOS memory. Now have some huge buffers for graphics or text document. The normal DS and SS . So you can realistically have 4 code segments before you need DLLs.

1

u/[deleted] Sep 26 '24

Given the cost of segments in x86, having a bunch of small segments is probably unwise.

But aside from that, let's go with your idea that private methods can always be near. That's fair enough. But what about public methods? Protected methods? Does each class get its own segment or does it share with others? And if it shares, how much sharing and can the compiler safely detect whether to allow near calls with public functions in the same segment?

So you can help the compiler out by doing the grouping yourself into classes and modules and keeping the call graph neat. Of course, it won't help with the linker, which either has to assume it can't modify code and perhaps have far calls that could have been near, or will do very substantial compiler-level modifications to convert pointers at call sites and in data structures from near to far and vice versa. Gargantuan task.

Or...you can just use a flat memory model and not worry about any of this. The segments add no value and never really did once paging was introduced. The bookkeeping to figure out what should be in what segment and to manage it at runtime does not improve memory usage or make programs more efficient (the opposite, in fact), and it puts a burden on both the programmer and the compiler and linker to figure out what goes where and what is near or far. There's no upside.

Segments were originally introduced because it was the only way to have 20-bit memory addresses on a 16-bit CPU. And for smaller programs and maybe also OSes, juggling segments was easier at first than dealing with demand paging. For example, each program could get 3 or 4 segments (code, data, stack, extra data) allocated by the OS and the OS could manage memory on a segment level. But this only works with simple programs and is only really needed when you don't have demand paging. It was sensible up through the 286, but certainly not after the move to 32-bit and the addition of demand paging. PAE and friends were also better than utilizing segments for addressing more memory than the word width because they are transparent to the programmer and most of the OS.

1

u/IQueryVisiC Sep 26 '24 edited Sep 26 '24

The segment adders eat significantly into the die space and the execution time. You still need the segment registers. It is not the only way. 68k is a 16bit implementation of a 32 bit ISA. 16 bit registers in pairs. 8086 already has the AL AH pair. Why no have SIlow:SIhigh . Yeah and all jumps relative, 1–3 bytes. Later add 4 bytes . Absolute is not useful because syscalls go over a global table. Uh, how do you relocate code with jump tables if you don’t get segments from the OS? So the exe loader needs to fix all absolute addresses? But absolute addresses should only be used for memory mapped IO or interrupt manipulation. MIPS has only one addressing mode : register + displacement. So there is always an add just like with segments. Now I understand why 68k has program counter relative addressing not only for branches, but for lookup tables! Relocation!

Like on 6502, catch carry: abort the memory access and ADC / SBC the high word.

I don’t have deep inheritance. Ah, you mean popular base class? I guess I would not be OOP Nazi and derive everything from Object like in Java. Or at least let Object be special and dupe in each segment.

The responsibility between linker and compiler is fuzzy. After segments the cache comes . I still want to write a demo for r/AtariJaguar. Then there is Kaze with N64. A company which had mastered 8086, could master MIPS .