Skip to content
← Back to Community
How to determine how much data is on the stack, and prevent stackoverflow
Profile icon
xxpertHacker

Lang dev people, @fuzzyastrocat, @DynamicSquid, @firefish, do any of you know anything about stuff like this? Thank you in advance.

Let's consider a language that is JiT compiled, take Java; if we recurse until the stack is too large,

class Main { public static void main(String[] args) { main(args); } }

we get a runtime exception:

Exception in thread "main" java.lang.StackOverflowError

We get something similar in browser ​Wasm:

(func $main call $main ) (start $main)

In V8, we'll get:

RangeError: Maximum call stack size exceeded

Or debug compiled Rust:

fn main() -> ! { main(); }

stdout:

thread 'main' has overflowed its stack fatal runtime error: stack overflow

These are all compiled to native code, whether it's JiT or AoT, it's still native.

The key thing is that, if I were to do the same in raw C, we would end up with a segmentation fault, not a runtime exception.

In an interpreter, it would be simple to not use the native stack, and instead to use a heap-allocated structure, and bounds check on every push to the virtual stack or a function call.

But, how could one use the actual native stack for operations (e.g. function calls, alloca, etc.), while keeping the same level of safety, as say, Java or JiT compiled JavaScript, without significantly worsening performance?

Should I emit a check on every single function call?

foo()

->

if stack has space { foo() } else { throw stack_overflow }

I've heard that Mozilla uses an AoT compiler that uses the native stack, so this provoked my interest in how one would execute sandboxed code, safely and efficiently.

Do any systems (hardware/OS) provide a way to know when the stack has overflown, without causing a system segfault?

Answered by DynamicSquid [earned 5 cycles]
View Answer
Voters
Profile icon
SSR2021
Profile icon
DynamicSquid
Profile icon
xxpertHacker
Comments
hotnewtop
Profile icon
DynamicSquid

I've never though about stack overflows for my own language :/

The way I would do it is I would perform a check before every function call.

If you take a look at my language, half the lines are if statements that throw an error.

Screenshot (297)

You also bring up a really good point about how the performance will be impacted, so I should probably change my current way.

This is a really good question, I would like to hear what @fuzzyastrocat has to say.


According to Stack Overflow, it says that you shouldn't handle a stack overflow (no pun intended), but rather prevent it. That's because stack overflows throw asynchronous exceptions, which are arduous to catch.

So I'm guessing that you do have to perform a check every time you make a function call.

Perhaps have a variable like max_recursive_depth?


Woah, wait... Repl.it has a dark theme now?

Oh yuck, if you go to https://repl.it/bugs the dark theme looks terrible

Profile icon
xxpertHacker

@DynamicSquid

You also bring up a really good point about how the performance will be impacted, so I should probably change my current way.

No, your way doesn't matter, since those are compiler errors, not runtime errors. Emitting a conditonal branch that might throw a stack-unwinding runtime exception after every statement, or even function call, would be terrible.

"abc" + "def" oh no, there's actually a branch here, because operator +(str, str) is a function, and you might overflow the stack. That sounds awful.

Now, that SO you linked says something that sounds like exactly what we would want for this situation!

...several platforms provide means to intercept the stack overflow or even get a "stack overflow alert" when the stack is almost exhausted.

And, of course, the accepted answer:

...Note that some platforms do notify a program when a stack overflow occurs and allow the program to handle the error. On Windows, for example, an exception is thrown. This exception is not a C++ exception, though, it is an asynchronous exception. Whereas a C++ exception can only be thrown by a throw statement, an asynchronous exception may be thrown at any time during the execution of a program. This is expected, though, because a stack overflow can occur at any time: any function call or stack allocation may overflow the stack.

I presume that could be as simple as a longjump function call to a stack-unwinding exception throw... that sounds wierd, and somewhat complicated. How could one know what function was called last? Oh, nvm, stack unwinding would solve that.

If this were simple native code, (e.g. Rust/C/C++, etc), then you shouldn't handle the stack overflow; but sandboxing is one of those rare conditions where it should be handled.

Update, checking out Windows, this looks weird: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/resetstkoflw?redirectedfrom=MSDN&view=msvc-160

Profile icon
xxpertHacker

@DynamicSquid Well, I guess it's just platform-specific APIs. Wonder if I'll encounter a platform that doesn't have any.


Also, totally unrelated, but usually, when someone refers to polyglot C++, they mean C++ that can run in a C compiler, but then I did this:

#define fn auto using i32 = int; // chop here to run in rustc fn foo() -> i32 { return 1 + 2; }

Probably the worst abuse of C macros in C++, to date.

Profile icon
DynamicSquid

@xxpertHacker lol! I like that :)

Also the Windows _resetstkoflw seems really interesting. Never knew you could recover from a stack overflow like that.

Profile icon
BobTheTomatoPie

not tryna be mean, but you might not wanna ping 3 people it could make them mad

Profile icon
xxpertHacker

@BobTheTomatoPie I'm 100% confident that 2/3 of them will not mind.

Profile icon
BobTheTomatoPie

which one are you not sure of @xxpertHacker

Profile icon
DynamicSquid

@BobTheTomatoPie I'm not mad :)

I like getting pinged for these kinds of questions. Even if I don't know how to solve them, it's really interesting to read.

I'm only mad if you ping me to check out a low effort time wasting spam bot post or something like that.

@xxpertHacker, give me a second, I'll think about it

Profile icon
xxpertHacker

@DynamicSquid Oh no, Squid is mad at me ;-;

Profile icon
fuzzyastrocat

@xxpertHacker HOW DARE YOU AWAKEN ME FROM MY SLUMBER

(jokes aside, it looks like you've already found your answer and I don't really have much to add to it)

Profile icon
xxpertHacker

@fuzzyastrocat Ha, okay.

What were you up to for the last 2 weeks?

Profile icon
fuzzyastrocat

@xxpertHacker H i b e r n a t i o n

I've just been doing non-repl things for the last month or so, didn't really feel like coding much.