Back to article

Preventing Buffer Overflow Exploits Using the Linux Distributed Security Module, Part 2

Existing Solutions to Buffer Overflow Exploits

January 12, 2006

The sad thing about buffer overflow exploits is that good programming practices could wipe out even potential exploits, however, that simply has not happened. The own defence against such exploits should revolve around controlling access to sensitive systems, installing software updates that replace exploitable software, and being aware of what a buffer overflow exploit looks like when your system is the intended victim.

Since buffer overflow exploits are one of the most popular attacks, there were many attempts trying to prevent them.

  1. The first solution is to write correct code where the programmer checks the size of the buffer; for instance, using strncpy instead of strcpy. In Part 1's Listing 3 (vulnerable.c), the strncpy function should be used instead the strcpy function. However, since one cannot expect that the code to be always written correctly, there is a need for system solutions.
  2. Buffer overflow exploits depend on an executable stack. One obvious solution is to make the stack segment non-executable; such kernel patches are available for Linux. This approach prevents buffer overflow exploits; however, it will create problems in the signal handling procedures. Signal handler returns in Linux require an executable stack. Furthermore, signal handlers are crucial in an operating system; thus, a temporary executable stack for signal handlers must be implemented. By removing stack execution from the system kernel, buffer overflow exploits can be prevented. However, this approach suffers from the code being non-portable, and operating system handling behaviour is modified and may be unpredictable. Another approach is the PaX project. The goal of the PaX project is to research various defense mechanisms against the exploitation of software bugs that give an attacker arbitrary read/write access to the attacked task's address space. This class of bugs contains among others various forms of buffer overflow bugs (be they stack or heap based), user supplied format string bugs, etc. (More on the PaX project.)
  3. Buffer overflow exploits are possible on processors on which the stack grows down. By moving more the data then a function´┐Żs local buffer can store, a return address can be overwritten. This situation cannot be achieved on the processors with the stack growing up because the return address has the lower address than the local buffer. Therefore, the strcpy function moves data to higher addresses and never reaches the return address. It can happen that some other part of the stack can be overwritten but not the return address. Therefore, it is impossible to jump to our shell code. Then another solution would be to make the stack gown up.
  4. Other solutions are implemented in compilers. A random extra field is placed on the stack when the function is called. After exiting from the call, this field is checked for corruption. This solution can be used at compile time. However, we need to recompile all existing programs and the problem arises when we are dealing with binaries and the source code is not available.
  5. The solutions we will discuss in this article are implemented in the kernel and are based on the mandatory security and Linux Security Module (LSM) hooks.
    1. One solution is offered by the National Security Agency (NSA) and is called the Security Enhanced Linux (SE Linux)
    2. The other solution is offered by Ericsson Research Open Systems Lab and is called the Distributed Security Module (DSM). At the time of writing, the stable version is DSI 0.3. DSI 0.4 is the beta version with ongoing development work.

    Both of these solutions are in principle similar because they are based on the same hooks in the Linux kernel. However, they differ in the actual implementation, and the security policies, performance and usability. We did not try to configure SE Linux for the purpose of this exercise since the policy setting is a bit more complex.

    For the purposes of this article, we will describe how the DSM module guards against buffer overflow exploits.