Type
Text
Type
Dissertation
Advisor
Ferdman, Mike | Sekar, R. | Polychronakis, Michalis | Lin, Zhiqiang.
Date
2015-05-01
Keywords
Computer science | Binary Instrumentation, Code Injection, Control Flow Integrity, Return Oriented Programming, System Security
Department
Department of Computer Science.
Language
en_US
Source
This work is sponsored by the Stony Brook University Graduate School in compliance with the requirements for completion of degree.
Identifier
http://hdl.handle.net/11401/77264
Publisher
The Graduate School, Stony Brook University: Stony Brook, NY.
Format
application/pdf
Abstract
Binary instrumentation has assumed an important role in software security, as well as related areas such as debugging and monitoring. Binary instrumentation can be performed statically or dynamically. Static binary instrumentation (SBI) is attractive because of its simplicity and efficiency. However, none of the previous SBI systems support secure instrumentation of COTS binaries. This is because of several challenges including: (a) static binary code disassembly errors, (b) difficulty of handling indirect control flow transfers, (c) ensuring completeness of instrumentation, i.e. | instrumenting all of the code, including code contained in system libraries and compiler-generated stubs, and (d) maintaining compatibility with complex code, i.e. | ensuring that the instrumentation does not break any existing code. We have developed a new static binary instrumentation approach, and present a software platform called PSI that implements this approach. PSI integrates a coarse grained control flow integrity (CFI) property as the basis of secure, non-bypassable instrumentation. PSI scales to large and complex stripped binaries, including low-level system libraries. It provides a powerful API that simplifies the development of custom instrumentations. We describe our approach, present several interesting security instrumentations, and analyze the performance of PSI. Our experiments on several real-world applications demonstrate that PSI’s runtime overheads are about an order of magnitude smaller than that of the most popular platforms available today, such as DynamoRIO and Pin. (Both these platforms rely on dynamic instrumentation.) PSI has been tested on over 300 MB of binaries. In addition to our platform PSI, we describe two novel security applications developed using PSI. First, we present a comprehensive defense against injected code attacks that ensures code integrity at all times, even against very powerful adversaries. Second, we present a defense against code reuse attacks such as return-oriented programming (ROP) that is effective against adversaries possessing a wide range of capabilities. | Binary instrumentation has assumed an important role in software security, as well as related areas such as debugging and monitoring. Binary instrumentation can be performed statically or dynamically. Static binary instrumentation (SBI) is attractive because of its simplicity and efficiency. However, none of the previous SBI systems support secure instrumentation of COTS binaries. This is because of several challenges including: (a) static binary code disassembly errors, (b) difficulty of handling indirect control flow transfers, (c) ensuring completeness of instrumentation, i.e. | instrumenting all of the code, including code contained in system libraries and compiler-generated stubs, and (d) maintaining compatibility with complex code, i.e. | ensuring that the instrumentation does not break any existing code. We have developed a new static binary instrumentation approach, and present a software platform called PSI that implements this approach. PSI integrates a coarse grained control flow integrity (CFI) property as the basis of secure, non-bypassable instrumentation. PSI scales to large and complex stripped binaries, including low-level system libraries. It provides a powerful API that simplifies the development of custom instrumentations. We describe our approach, present several interesting security instrumentations, and analyze the performance of PSI. Our experiments on several real-world applications demonstrate that PSI’s runtime overheads are about an order of magnitude smaller than that of the most popular platforms available today, such as DynamoRIO and Pin. (Both these platforms rely on dynamic instrumentation.) PSI has been tested on over 300 MB of binaries. In addition to our platform PSI, we describe two novel security applications developed using PSI. First, we present a comprehensive defense against injected code attacks that ensures code integrity at all times, even against very powerful adversaries. Second, we present a defense against code reuse attacks such as return-oriented programming (ROP) that is effective against adversaries possessing a wide range of capabilities. | 157 pages
Recommended Citation
Zhang, Mingwei, "Static Binary Instrumentation with Applications to COTS Software Security" (2015). Stony Brook Theses and Dissertations Collection, 2006-2020 (closed to submissions). 3085.
https://commons.library.stonybrook.edu/stony-brook-theses-and-dissertations-collection/3085