Executable-space protection

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Executable-space protection" – news · newspapers · books · scholar · JSTOR (April 2013) (Learn how and when to remove this message)

In computer security, executable-space protection marks memory regions as non-executable, such that an attempt to execute machine code in these regions will cause an exception. It relies on hardware features such as the NX bit (no-execute bit), or on software emulation when hardware support is unavailable. Software emulation often introduces a performance cost, or overhead (extra processing time or resources), while hardware-based NX bit implementations have no measurable performance impact.

The Burroughs large systems, starting with the Burroughs 5000 introduced in 1961, implemented executable-space protection using a tagged architecture. All accesses to code and data took place through descriptors, which had memory tags preventing them from being modified; descriptors for code did not allow the code to be modified, and descriptors for data did not allow the data to be executed as code.

Today, operating systems use executable-space protection to mark writable memory areas, such as the stack and heap, as non-executable, helping to prevent buffer overflow exploits. These attacks rely on some part of memory, usually the stack, being both writable and executable; if it is not, the attack fails.

Many operating systems implement or have an available executable space protection policy. Here is a list of such systems in alphabetical order, each with technologies ordered from newest to oldest.

A technology supplying Architecture Independent emulation will be functional on all processors which aren't hardware supported. The "Other Supported" line is for processors which allow some grey-area method, where an explicit NX bit doesn't exist yet hardware allows one to be emulated in some way.

As of Android 2.3 and later, architectures which support it have non-executable pages by default, including non-executable stack and heap.^[1][2][3]

Initial support for the NX bit, on x86-64 and IA-32 processors that support it, first appeared in FreeBSD -CURRENT on June 8, 2004. It has been in FreeBSD releases since the 5.3 release.

The Linux kernel supports the NX bit on x86-64 and IA-32 processors that support it, such as modern 64-bit processors made by AMD, Intel, Transmeta and VIA. The support for this feature in the 64-bit mode on x86-64 CPUs was added in 2004 by Andi Kleen, and later the same year, Ingo Molnár added support for it in 32-bit mode on 64-bit CPUs. These features have been part of the Linux kernel mainline since the release of kernel version 2.6.8 in August 2004.^[4]

The availability of the NX bit on 32-bit x86 kernels, which may run on both 32-bit x86 CPUs and 64-bit IA-32-compatible CPUs, is significant because a 32-bit x86 kernel would not normally expect the NX bit that an AMD64 or IA-64 supplies; the NX enabler patch assures that these kernels will attempt to use the NX bit if present.

Some desktop Linux distributions, such as Fedora, Ubuntu and openSUSE, do not enable the HIGHMEM64 option by default in their default kernels, which is required to gain access to the NX bit in 32-bit mode, because the PAE mode that is required to use the NX bit causes boot failures on pre-Pentium Pro (including Pentium MMX) and Celeron M and Pentium M processors without NX support. Other processors that do not support PAE are AMD K6 and earlier, Transmeta Crusoe, VIA C3 and earlier, and Geode GX and LX. VMware Workstation versions older than 4.0, Parallels Workstation versions older than 4.0, and Microsoft Virtual PC and Virtual Server do not support PAE on the guest. Fedora Core 6 and Ubuntu 9.10 and later provide a kernel-PAE package which supports PAE and NX.

NX memory protection has always been available in Ubuntu for any systems that had the hardware to support it and ran the 64-bit kernel or the 32-bit server kernel. The 32-bit PAE desktop kernel (linux-image-generic-pae) in Ubuntu 9.10 and later, also provides the PAE mode needed for hardware with the NX CPU feature. For systems that lack NX hardware, the 32-bit kernels now provide an approximation of the NX CPU feature via software emulation that can help block many exploits an attacker might run from stack or heap memory.

Non-execute functionality has also been present for other non-x86 processors supporting this functionality for many releases.

Red Hat kernel developer Ingo Molnár released a Linux kernel patch named Exec Shield to approximate and utilize NX functionality on 32-bit x86 CPUs. The Exec Shield patch was released to the Linux kernel mailing list on May 2, 2003, but was rejected for merging with the base kernel because it involved some intrusive changes to core code in order to handle the complex parts of the emulation. Exec Shield's legacy CPU support approximates NX emulation by tracking the upper code segment limit. This imposes only a few cycles of overhead during context switches, which is for all intents and purposes immeasurable. For legacy CPUs without an NX bit, Exec Shield fails to protect pages below the code segment limit; an mprotect() call to mark higher memory, such as the stack, executable will mark all memory below that limit executable as well. Thus, in these situations, Exec Shield's schemes fails. This is the cost of Exec Shield's low overhead. Exec Shield checks for two ELF header markings, which dictate whether the stack or heap needs to be executable. These are called PT_GNU_STACK and PT_GNU_HEAP respectively. Exec Shield allows these controls to be set for both binary executables and for libraries; if an executable loads a library requiring a given restriction relaxed, the executable will inherit that marking and have that restriction relaxed.

• Hardware Supported Processors: All that Linux supports NX on
• Emulation: NX approximation using the code segment limit on IA-32 (x86) and compatible
• Other Supported: None
• Standard Distribution: Fedora Core and Red Hat Enterprise Linux
• Release Date: May 2, 2003

The PaX NX technology can emulate NX functionality, or use a hardware NX bit. PaX works on x86 CPUs that do not have the NX bit, such as 32-bit x86. The Linux kernel still does not ship with PaX (as of May, 2007); the patch must be merged manually.

PaX provides two methods of NX bit emulation, called SEGMEXEC and PAGEEXEC. The SEGMEXEC method imposes a measurable but low overhead, typically less than 1%, which is a constant scalar incurred due to the virtual memory mirroring used for the separation between execution and data accesses.^[5] SEGMEXEC also has the effect of halving the task's virtual address space, allowing the task to access less memory than it normally could. This is not a problem until the task requires access to more than half the normal address space, which is rare. SEGMEXEC does not cause programs to use more system memory (i.e. RAM), it only restricts how much they can access. On 32-bit CPUs, this becomes 1.5 GB rather than 3 GB.

PaX supplies a method similar to Exec Shield's approximation in the PAGEEXEC as a speedup; however, when higher memory is marked executable, this method loses its protections. In these cases, PaX falls back to the older, variable-overhead method used by PAGEEXEC to protect pages below the CS limit, which may become quite a high-overhead operation in certain memory access patterns. When the PAGEEXEC method is used on a CPU supplying a hardware NX bit, the hardware NX bit is used, thus no significant overhead is incurred.

PaX supplies mprotect() restrictions to prevent programs from marking memory in ways that produce memory useful for a potential exploit. This policy causes certain applications to cease to function, but it can be disabled for affected programs.

PaX allows individual control over the following functions of the technology for each binary executable:

• PAGEEXEC
• SEGMEXEC
• mprotect() restrictions
• Trampoline emulation
• Randomized executable base
• Randomized mmap() base

PaX ignores both PT_GNU_STACK and PT_GNU_HEAP. In the past, PaX had a configuration option to honor these settings but that option has been removed for security reasons, as it was deemed not useful. The same results of PT_GNU_STACK can normally be attained by disabling mprotect() restrictions, as the program will normally mprotect() the stack on load. This may not always be true; for situations where this fails, simply disabling both PAGEEXEC and SEGMEXEC will effectively remove all executable space restrictions, giving the task the same protections on its executable space as a non-PaX system.

• Hardware Supported Processors: Alpha, AMD64, IA-64, MIPS (32 and 64 bit), PA-RISC, PowerPC, SPARC
• Emulation: IA-32 (x86)
• Other Supported: PowerPC (32 and 64 bit), SPARC (32 and 64 bit)
• Standard Distribution: Alpine Linux
• Release Date: October 1, 2000

macOS for Intel supports the NX bit on all CPUs supported by Apple (from Mac OS X 10.4.4 – the first Intel release – onwards). Mac OS X 10.4 only supported NX stack protection. In Mac OS X 10.5, all 64-bit executables have NX stack and heap; W^X protection. This includes x86-64 (Core 2 or later) and 64-bit PowerPC on the G5 Macs.

As of NetBSD 2.0 and later (December 9, 2004), architectures which support it have non-executable stack and heap.^[6]

Architectures that have per-page granularity consist of: alpha, amd64, hppa, i386 (with PAE), powerpc (ibm4xx), sh5, sparc (sun4m, sun4d), sparc64.

Architectures that can only support these with region granularity are: i386 (without PAE), other powerpc (such as macppc).

Other architectures do not benefit from non-executable stack or heap; NetBSD does not by default use any software emulation to offer these features on those architectures.

A technology in the OpenBSD operating system, known as W^X, marks writable pages by default as non-executable on processors that support that. On 32-bit x86 processors, the code segment is set to include only part of the address space, to provide some level of executable space protection.

OpenBSD 3.3 shipped May 1, 2003, and was the first to include W^X.

• Hardware Supported Processors: Alpha, AMD64, HPPA, SPARC
• Emulation: IA-32 (x86)
• Other Supported: None
• Standard Distribution: Yes
• Release Date: May 1, 2003

Solaris has supported globally disabling stack execution on SPARC processors since Solaris 2.6 (1997); in Solaris 9 (2002), support for disabling stack execution on a per-executable basis was added.

The first implementation of a non-executable stack for Windows (NT 4.0, 2000 and XP) was published by SecureWave via their SecureStack product in 2001, based on the work of PaX.^[7][8]

Starting with Windows XP Service Pack 2 (2004) and Windows Server 2003 Service Pack 1 (2005), the NX features were implemented for the first time on the x86 architecture. Executable space protection on Windows is called "Data Execution Prevention" (DEP).

Under Windows XP or Server 2003 NX protection was used on critical Windows services exclusively by default. If the x86 processor supported this feature in hardware, then the NX features were turned on automatically in Windows XP/Server 2003 by default. If the feature was not supported by the x86 processor, then no protection was given.

Early implementations of DEP provided no address space layout randomization (ASLR), which allowed potential return-to-libc attacks that could have been feasibly used to disable DEP during an attack.^[9] The PaX documentation elaborates on why ASLR is necessary;^[10] a proof-of-concept was produced detailing a method by which DEP could be circumvented in the absence of ASLR.^[11] It may be possible to develop a successful attack if the address of prepared data such as corrupted images or MP3s can be known by the attacker.

Microsoft added ASLR functionality in Windows Vista and Windows Server 2008. On this platform, DEP is implemented through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable.^[12] In Windows, from version Vista, whether DEP is enabled or disabled for a particular process can be viewed on the Processes/Details tab in the Windows Task Manager.

Windows implements software DEP (without the use of the NX bit) through Microsoft's "Safe Structured Exception Handling" (SafeSEH). For properly compiled applications, SafeSEH checks that, when an exception is raised during program execution, the exception's handler is one defined by the application as it was originally compiled. The effect of this protection is that an attacker is not able to add his own exception handler which he has stored in a data page through unchecked program input.^[12][13]

When NX is supported, it is enabled by default. Windows allows programs to control which pages disallow execution through its API as well as through the section headers in a PE file. In the API, runtime access to the NX bit is exposed through the Win32 API calls VirtualAlloc[Ex] and VirtualProtect[Ex]. Each page may be individually flagged as executable or non-executable. Despite the lack of previous x86 hardware support, both executable and non-executable page settings have been provided since the beginning. On pre-NX CPUs, the presence of the 'executable' attribute has no effect. It was documented as if it did function, and, as a result, most programmers used it properly. In the PE file format, each section can specify its executability. The execution flag has existed since the beginning of the format and standard linkers have always used this flag correctly, even long before the NX bit. Because of this, Windows is able to enforce the NX bit on old programs. Assuming the programmer complied with "best practices", applications should work correctly now that NX is actually enforced. Only in a few cases have there been problems; Microsoft's own .NET Runtime had problems with the NX bit and was updated.

• Hardware Supported Processors: x86-64 (AMD64 and Intel 64), IA-64, Efficeon, Pentium M (later revisions), AMD Sempron (later revisions)
• Emulation: Yes
• Other Supported: None
• Standard Distribution: Post Windows XP
• Release Date: August 6, 2004

In Microsoft's Xbox, although the CPU does not have the NX bit, newer versions of the XDK set the code segment limit to the beginning of the kernel's .data section (no code should be after this point in normal circumstances). Starting with version 51xx, this change was also implemented into the kernel of new Xboxes. This broke the techniques old exploits used to become a terminate-and-stay-resident program. However, new exploits were quickly released supporting this new kernel version because the fundamental vulnerability in the Xbox kernel was unaffected.

Where code is written and executed at runtime—a JIT compiler is a prominent example—the compiler can potentially be used to produce exploit code (e.g. using JIT Spray) that has been flagged for execution and therefore would not be trapped.^[14][15]

Return-oriented programming can allow an attacker to execute arbitrary code even when executable space protection is enforced.

• Buffer overflow
• Buffer overflow protection
• Heap overflow
• Stack-smashing protection
• Uncontrolled format string