Security vulnerabilities - From data analysis to protection mechanisms

This paper combines an analysis of data on security vulnerabilities (published in the Bugtraq database) and a focused source-code examination to (i) develop a finite state machine (FSM) model to depict and reason about security vulnerabilities and (ii) to extract characteristics shared by a large class of commonly seen vulnerabilities (e.g., integer overflow, heap overflow, format string, and stack buffer overflow). This information is used to devise Transparent Runtime Randomization (TRR), a generic, randomization-based technique for protecting against a wide range of security attacks. TRR dynamically and randomly relocates a programs stack, heap, shared libraries, and parts of its runtime control data structures inside the application memory address space. Making a programs memory layout different each time it runs foils the attackers assumptions about the memory layout of the vulnerable program and defeats potential attacks. TRR is implemented by changing the Linux dynamic program loader; hence, it is transparent to applications. TRR incurs less than 9% program startup overhead and no runtime overhead.