GCC “stack smashing detected” mechanism

I believe that programmers who develop software in C/C++ have experienced the problem of ‘stack overflow’. The ‘stack overflow’ problem is often very subtle, and sometimes it is very difficult to reproduce the problem. Whenever there is an unexplained problem in the software, there is always the question of whether the stack is overflowing, but it is very difficult to troubleshoot the problem, so ‘stack overflow’ is the nightmare of most C/C++ developers. Is there a general way to avoid or detect stack overflow? In fact, “stack overflow” sometimes there is no way to avoid the problem, even the experienced programmer can also fall into the trap of “stack overflow”, I believe everyone has heard of many well-known software systems due to “stack overflow” caused by various security problems. In order to reduce the occurrence of “stack overflow”, we should always pay attention to the use of the buffer on the stack when writing code, and be familiar with the various operation API of the stack buffer (the most frequently used is the string operation function). However, there is always a human error, so is there a mechanism to detect ‘stack overflow’? Today’s talk is about the various mechanisms that the GCC compiler provides for stack protection. First, let’s take a look at an obvious stack overflow code fragment compiled on the Ubuntu14.04 Gcc version 4.8.4. The code is as follows:

#include <string.h>
#include <stdio.h>

int main(void)
{
    char array[5] = {0};
    
    strcpy(array, "stackoverflow");
    
    return 0;
}
Copy the code

We use strcpy to copy the string “StackVoerflow” into the array buffer. Obviously, the array buffer overflowed. We’ll see how to use strcpy later.

Then, we compile and run, the result is as follows:

lhl@lhl-Aspire-4930:~/ technical summary/Linux/GCC $./ Buscore *** Stack smashing detected ***:.Copy the code

You can see that the program crashes and there is a message:

*** stack smashing detected ***: ./buscore terminated
Copy the code

The prompt is what the stack overflow protection provided by GCC will print when it detects a buffer overflow. What does GCC provide for stack overflow protection?

By looking at the GCC user manual, we can see that this mechanism provides buffer overflow detection for the -fstack-Protector range of GCC options. The principle of this mechanism is described as follows:

When -fstack-Protector is enabled, it immediately terminates the executing program when it detects a buffer overflow (for example, a buffer overflow attack) and informs it that it has detected a buffer overflow problem. This mechanism is accomplished by adding protection variables to the vulnerable target context in the function. These include functions that use the allcoa function and functions with buffers larger than 8bytes (it’s not clear here, but the above example with a buffer size of 5 also triggers this mechanism). These protection variables are initialized upon entry to the function and checked when the function exits. If some variables fail to check, an error message is printed and the current process is terminated.

Note: During testing, this mechanism is also triggered when the buffer size is less than 8bytes. After using the alloca function in the function, the buffer overflows but does not trigger the mechanism. Why?

char *stack_ptr = alloca(0); char *str = "hello, world"; strcpy(stack_ptr, str); printf("stack_ptr = %s\n", stack_ptr); Does the above code trigger this mechanism?Copy the code

The question then arises, is -fstack-Protector really omnipotent? Can it really detect all stack overflow problems? We can test this with the following example:

#include <alloca.h>
#include <string.h>
#include <stdio.h>

typedef struct
{
    char str[2];
}overflow_t;
int main(void) 
{
char *str = "welcom to China";

    overflow_t v;  

    printf("sizoef(v) = %d\n", sizeof(v));

    memcpy(&v, str, strlen(str));

    printf("v.str = %s\n", v.str);
}
Copy the code

GCC buscore. C -o buscore, sizoef(v) = 2 v.str = welcom to ChinaCopy the code

We see that the program executes normally, and there is no stack overflow detection. Right? Why is that? Further review of the GCC manual reveals that it also provides a -fstack-protractor all option, which explains:

Its functionality is similar to -fstack-protector, but it does stack overflow detection for all functions. Recompile the program with the -fstack-protractor all option and run it as follows:

sizoef(v) = 2

V.test= welcom to China *** Stack smashing detected ***:./ Buscore terminated You can see that the process successfully detected a stack overflow.

Therefore, it is better to add the -fstack-protractor all option when developing the software, and a large proportion of stack overflow problems will be detected.

It should also be noted that when a process crashes due to stack overflow, a coredump file is generated. For details on how to configure coredump, see this article on Coredump for Android. How do you locate the problem according to the coredump file? The -fstack-Protector mechanism is designed to detect when a function exits, so that we can locate the function that sent the stack overflow based on the function stack information in the Coredump. This reduces the scope of the problem and allows us to focus on the function that caused the problem. Then locate where there is a stack overflow.