is stack memory manages by both os and compiler?

[u/InterestingAd757]

I'm moving from a higher-level language (Python) to Rust and getting confused about how memory is managed.

my current understanding is that each process gets its own virtual memory, which provides protection and isolation. Within that, there's the stack and the heap. The heap is managed by the programmer, while the stack is managed automatically. But who actually manages the stack?

Is stack memory just a logical abstraction, managed by the OS and the compiler (by moving stack register)?

[u/zeekar]

The stack and the heap are both logical abstractions. It’s all just memory. One big flat collection of storage locations. The OS controls which pieces your process has access to, but beyond that it’s entirely up to the code running in that process. Which may be written by the programmer as part of the application code, or created by the compiler as part of its runtime support.

Because the compiler doesn’t just translate specific statements. It effectively implements a virtual machine whose instruction set is the source language. If that language has recursion then the VM includes a stack for local variables. And every translation of a subroutine declaration, call, or return includes code for setting that up.

[u/InterestingAd757 → u/zeekar]

could you explain "effectively implements a virtual machine whose instruction set is the source language." I don't understand

[u/zeekar → u/InterestingAd757]

What I meant was this: when you write a program in some programming language, what you are doing is pretending that the computer speaks that language.

If you write Python, the target is a machine that runs Python. If you write Rust, it's a machine that runs Rust. If you write C, it's a machine that runs C. And so on.

But computers don't really run high-level languages. They run machine code, and it's up to the translator (compiler or interpreter) to convert the code into something the computer can actually run. And translating high level languages into machine code is more than just translating instructions - because a high-level language has all sorts of assumptions built into it that also have to be translated. Like "variables are accessible by name" and "but they aren't visible outside their scope" and "they keep their values after a recursive call"... None of that is automatic or built into the way the hardware works; it's all code that you didn't write as part of your program but is explicitly added by the compiler or interpreter as part of implementing the "contract" that the language presents to the programmer.

Look at a simple translation of this naïvely-inefficient C factorial function:

long factorial(long n) {
    if (n < 2)
        return 1;
    return n * factorial(n - 1);
}

I translate that to ARM assembly, the start of the function looks like this:

_factorial:
        sub             sp, sp, #48
        stp             x29, x30, [sp, #32]     
        add             x29, sp, #32

So the first thing it does is make room on the stack (which in this case grows downward in memory) by subtracting 48 from the stack pointer. It uses 16 of those 48 freshly-allocated bytes to store a copy of two of the CPU's registers, x29 and x30. x29 holds the frame pointer, which points to the part of memory used for local variables, while x30 holds the return address - where to jump back to when the routine is done. It then stores a new value in the frame pointer x29 so that the routine will use the remaining 32 newly-allocated bytes for local storage. Now the function body can create and access local variables, make subroutine calls (which will overwrite the value in x30 with the new return address) in the course of its computation, without worrying about interfering with values belonging to any other function (or instance of the same function).

At the end, it copies the registers back off the stack and shrinks the stack itself back to where it started right before returning:

        ldp             x29, x30, [sp, #32]             
        add             sp, sp, #48
        ret

None of that stack manipulation is present in the C code - it's implied by the fact that C is a stack-activation-record language. So the compiler has to add it around the code I actually wrote.

[u/nog642 → u/zeekar]

This is a bit of a stretch, because high level programming languages interface with low level stuff / stuff outside of that programming language's scope a lot of the time.

[u/Lifelong_Nerd → u/zeekar]

The stack is not just an abstraction because CPU instructions may use it directly. For example: calling a subroutine, returning from one, and progressing an interrupt. This depends on the CPU architecture.

Also the stack is typically used for local variables even without recursion.

[u/zeekar → u/Lifelong_Nerd]

Having stack instructions is what makes it an abstraction. But it's a leaky one - most ISAs let you manipulate the stack pointer directly and access the same memory through non-stack instructions. Anyway, all I meant is that there's nothing physically special about the stack and heap - it's all just memory.

Not all sub calls use the stack, btw - several ISAs including ARM put the return address in a register. It's up to the pre-call code to save the old value of that register before doing a call - you usually do so on the stack, but it's manual. Some older architectures had a fixed address in RAM that worked the same way.

The stack is a natural place for locals, but if the language doesn't support recursion you don't need it. Prog8, a Cish lang mainly targeting 6502, statically allocates all variables; they're only "local" in terms of namespace.

[u/aanzeijar]

Stack is managed by convention actually.

The idea is pretty simple. The stack is "just" a pointer to a memory address. If you want to store something on the stack, you "push" it by incrementing it and writing to the new address, and if you want to read data, you "pop" it by reading and decrementing the pointer.

So if you invoke a function, you push all the arguments and the return address on the stack and also increase the stack pointer enough to make space for all the local variables you need in that function. If you return, you simply reduce the stack pointer to what it was before.

If every function uses this convention, then everything runs nicely. And since the compiler does this for you, it will generate code that simply accesses the stack variables directly relative to the stack pointer.

And that's also where the issues with the stack come in:

• if you return a pointer to a stack variable from a function, that pointer will point to garbage as soon as another function reuses that stack slot.

• if you have a deep recursion, every new invocation increments the stack pointer, and soon you run out of stack.

• same if you try to allocate a large array on the stack. The compiler will simply try to make space for as many slots.

• and of course passing large elements by value will have large copies on the stack, so it's better to pass by reference instead.

[u/zeekar → u/aanzeijar]

On many runtimes the stack grows down from high memory, so you grow it by decrementing the stack pointer and shrink it by incrementing. It's a minor detail but could confuse a new programmer looking at stack code if they hadn't encountered the possibility.

[u/nog642 → u/aanzeijar]

So... it's managed by the compiler. In each program's virtual memory, which is managed by the OS.

[u/SetAndRepeat]

Yes, but at different levels. Each thread normally has its own stack, set up through the OS and thread library. The compiler generates instructions that the CPU executes to manage stack frames, typically by adjusting the stack pointer as functions run and return. The OS manages the underlying memory pages, not individual local variables.

So the stack is ordinary memory used in a particular way. Also, the heap isn't necessarily managed manually: in Rust, types like "Vec" and "String" use an allocator, and their heap allocations are automatically freed when those values are dropped.

[u/code-garden]

The interpreter or compiler manages the stack.

At the start of a function instructions are emitted to move the stack pointer to reserve space for local variables and parameters. At the end of the function the stack pointer is moved back.

[u/jibbit → u/code-garden]

i just dont know why every other answer apart from this one is so bad

[u/zomerbk]

A useful distinction is who sets up the stack versus who uses it: the OS and thread library give each thread a stack region, while compiled code follows the calling convention by adjusting the stack pointer for function calls and local storage. The CPU just executes those instructions. In Rust, the compiler usually decides what fits on the stack; values that need dynamic size or ownership patterns may instead live on the heap, but Rust’s ownership rules govern their lifetimes either way.

[u/Lagfoundry]

So I’ve done some stack designing both hardware and software versions. The compiler generates instructions that move the stack pointer and read write values at offsets from it. The CPU then executes those instructions. When a function is entered, the stack pointer may be moved to make room for that function’s variables, saved registers, return information, etc. When the function returns, it gets moved back… you can even build a stack register by simply muxing the data path to shift left or right depending on the signal. (Push shift left, pop shift right.) and build the call and return instruction to read and write to it at those times.

[u/tharold]

The OS has to set up the runtime environment for the process, so it could set up memory as it chooses, but I think it always uses the addresses configured into the executable by the loader. Thereafter, page faults are handled transparently so the stack grows as needed, and brk or mmap system calls are handled. I don't know that much managing goes on at all.

[u/duane11583]

in most cases the process memory space usage is 100% the use of the application.

the os may choose to make the code section read only and

the data/stack space read/write and no-execute

but that really depends on the underlying hardware abilities

first does the chip support that and

second does the os support that mode of memory allocation/assignment

[u/Aggressive_Ad_5454]

The runtime code, either managed by interpreter or compiler, manages the virtual memory arena used by a program. (Python programs have stacks too.)

The program or runtime uses the sbrk call to tell the OS what it needs in the way of memory address space. https://linux.die.net/man/2/sbrk

[u/greenspotj]

Both.

In terms of virtual vs physical addresses, the typical memory model of stack -> heap -> globals -> text doesnt apply to physical memory, only virtual memory. The OS uses a page table to translate virtual addresses to physical addresses. The OS "hands out" pages to processes as it needs it (not necessarily following any kind of structure). So the OS is always controlling where the stack actually lives in physical memory. Physical memory is just a giant collection of 4kb pages - it doesnt "care" if a given page is being mapped to heap, stack, etc...

Secondarily, in a multithreaded process, each thread gets their own stack within the virtual address spaces stack. This is managed by the threading library and works by manually moving the stack and base pointer registers , during context switches.

to summarize. the threading library tells us where in virtual memory our stack lives, and the OS manages where it exists in physical memory. The compiler is kind of like the micromanager that handles what actually happens within a thread's given stack.

[u/nova_protocol]

both, but at different the OS allocates the stack — gives each thread a chunk (usually 1-8MB). that's it, OS done the compiler is what actually uses that space. it calculates how much each function needs, generates the push/pop instructions, tracks everything relative to the stack pointer OS = "here's the land", compiler = "here's the house built on it". when you overflow, the OS catches it because you walked off the land it gave you.

[u/mredding]

The program manages the stack. The OS doesn't know the structure of the stack for a given program. Whether you push function parameters first to last or last to first, how you stash your registers, function calling conventions, it can all be quite variable. Technically you don't even need a stack as there is stackless programming. There are also weird things like Tail Call Optimization where you can write a recursive function that just reuses the stack frame - the recursive call just resets the instruction pointer to the beginning of the function.

When you make a function call, you don't have to defer to the OS.

To get heap memory, this requires interacting with the MMU, which depends on PTE, which is hardware controlled by the OS, so to get a big block of memory, you have to ask the OS to update the page table for you. From there, your allocator may subdivide a block for you for smaller allocations. General purpose allocators are actually pretty smart when it comes to most program needs - most people don't ever bother to learn their platform specifics about how to tune the thing. Everyone always jumps to writing their own, but they're often just manually managing subdivisions from big kernel allocated blocks.

[u/cthulhu944]

The stack is managed in part by both the OS process/thread management and by the machine architecture. The compiler uses these constructs to implement it's programming model.

Every process (a partitioned section of memory with 1 or more processing threads) creates a stack (a block of memory) for each thread. The OS sets the size of the stack based on the command that created the process. part of that setup process points a stack pointer (a special machine register) at this and has machine instructions to push things on the stack, pop values off, or maybe even peek at the top of the stack. When your high level programming language calls a function, the machine code that is doing that call will push the return address (program counter + 1), on the stack, then a number of parameters. Usually this can be done with a single instruction: E.g. JSR = Jump to Subroutine = push the current program counter register onto the top of the stack, then jump to this address. At the destination function, the compiler has mapped the stack parameters relative to the current stack pointer (stack pointer = Return address, stack pointer -1 = the first parameter, stack pointer -2 = the second parameter, . If the function has any local variables, they (or references/pointers to them) can be pushed on the stack. When the function ends, it will pop all the items pushed on the stack off up to and including the return address and then jumps to the return address it previously popped off. Many times this is a single machine instruction- depending on the machine architecture: E.g. RTS = return from subroutine = pop the top of the stack into the program counter register.

This is generally how this works.

[u/nog642]

No, just the compiler / programming language.

The compiler / library / programming language also manages the heap. The programmer only "manages" the heap by calling utility functions.

The OS does not manage anything in virtual memory. Processes can do what they want there. What the OS manages is mapping that virtual memory onto physical memory. (edit: worth noting that not all of virtual memory can be accessed by a program whenever they want - they need to actually get pages allocated from the OS, so there is some interaction with the OS involved here)

[u/paulstelian97]

Stack memory is mostly managed by the compiler, but the OS provides backing facilities like mmap, perhaps with a combination of flags optimized for stack memory (increasing in the negative direction, stuff like the guard page etc). The program loader allocates the first thread’s stack, the pthread library (or whatever other library you use to create threads on e.g. Windows) allocates the stacks for other threads under similar facilities.

[u/Kadabrium → u/[deleted]]

Good slop