Parallelizing security checks on commodity hardware

www.eecs.umich.edu/~dpeek/asplos.pdf

by Edmund B. Nightingale et al. (ASPLOS’08)

These guys decouple security check from the normal run (which continues speculatively). Doing this, they accelerate taint analysis 1.6x-2x. Nice work.

Digging for datastructures (to identify malware)

Anthony Cozzie has built a system for detecting datastructures in applications’ heap memory. In this paper he presents the method and applies it to malware detection. His premise is that data layout is much harder to vary automatically than instructions (polymorphism). Therefore, data offers a better signature than instructions. He verifies this intuition by comparing multiple versions of Kraken and Storm (among others). A great example of generic system’s work applied to security. The full paper at OSDI 2008 can be found here.

Ether: Malware Analysis via Hardware Virtualization Extensions

By A. Dinaburg, P. Royal, M. Sharif and W. Lee

Proceedings of the 15th ACM Conference on Computer and Communications Security 2008 (CCS’08)

Ether is a malware analysis framework which leverages hardware virtualization extensions (specifically Intel VT) to remain transparent to malicious software. It supports both fine- (single instruction) and coarse- (system call) granularity tracing. (I’m curious what the performance penalty for the fine-grain tracing is. The authors only say that it’s “significant”.)

Both the GPL’ed source code for Ether and the paper are available for download at http://ether.gtisc.gatech.edu.

Dataflow Anomaly Detection

By S. Bhatkar, A. Chaturvedi and R. Sekar

Proceedings of the 2006 IEEE Symposium on Security and Privacy

Usually intrusion detection system based on modeling behaviours of programs in terms of system call sequences focus on control flows, with little emphasis on data flow involving system call arguments. In contrast, this paper presents an intrusion detection technique that is based on learning temporal properties involving arguments of different system calls, thus capturing the flow of security-sensitive data through the program. Basically, the approach hypothesizes the flows that may be present, based on relationships observed between the parameters of different system calls. Dataflow properties are categorized into unary relations that involve properties of a single system call argument, and binary relations that involve arguments of two different system calls.

The paper can be found here.

Improving software security via runtime instruction-level taint checking

Back from 2006, published in ASPLOS.

An extension to dynamic taint-analysis to capture non-control flow data attacks.

Somewhat limited because it’s based on doing some binary analysis beforehand, but still interesting.

http://www.cs.ucf.edu/~jfkong/ASID06.pdf

Exploring Multiple Execution Paths for Malware Analysis

By Andreas Moser and Christopher Kruegel and Engin Kirda

Proceedings of the 2007 IEEE Symposium on Security and Privacy

Authors of the paper address the problem of malware which doesn’t display malicious behaviour unless certain trigger conditions are present. Dynamic taint-tracking is used to discover conditionals in the program that are dependent on tainted inputs. When one of the two branches of such a conditional is about to be taken, they create a checkpoint and a snapshot of the analyzed process, and keeps exploring one of the branch. Subsequently, when the exploration of the taken branch ends or after a timeout threshold is reached, they force the execution of the unexplored branch.

The paper can be found here.

Securing software by enforcing data-flow integrity

A somewhat older paper. DFI goes beyond detecting control flow attacks to also handle criticial variables being overwritten (look at the SSH exploit). This solution requires source code and recompilation, but despite some limitations it’s very impressive.

http://research.microsoft.com/~manuelc/MS/dfiOSDI.pdf

Device Driver Safety Through a Reference Validation Mechanism

Paper can be found here

By Dan Williams, Patrick Reynolds, Kevin Walsh, Emin Gün Sirer, and Fred B. Schneider
OSDI 2008

This paper describes how the Nexus OS can be protected by malicious drivers by running drivers in userspace, and use of reference monitor. The system supports a software iommu, rate limiting for interrupts, and various other mechanisms.

Real-World Buffer Overflow Protection for Userspace & Kernelspace

(paper can be found here)

by Michael Dalton, Hari Kannan, Christos Kozyraki (Stanford)

Proceedings of USENIX Security’08.

This paper is a follow-up of the Raksha paper at ISCA’07 and I think it represents a huge improvement. The most interesting aspect of it is that they use a technique that detects the injection of pointers (by attackers), by marking all legitimate pointers of the system and all pointers derived from these pointers. Any dereference of a pointer that is not marked as a legitimate pointer triggers an alert. They are not the first to propose detection of ptr injection, but they are the first to come up with a practical way of doing so. Although it requires a lot of hard work (scanning ELF binaries, tracking dynamic allocations, etc.), the runtime overhead can be kept small. False positives are unlikely (although false negatives may still occur). Their method does appear to be somewhat tied to an architecture like SPARC and an open source OS like Linux (I would expect the number of FNs to  be quite large on x86/Windows). Still, it is one of the few applications of pointer tainting that look practical.

Panorama: capturing system-wide information flow for malware detection and analysis

http://bitblaze.cs.berkeley.edu/papers/panorama.pdf

by Heng Yin and Dawn Song and Manuel Egele and Christopher Kruegel and Engin Kirda (Proceedings of CCS’07).

The paper describes how potential malware can be monitored to see if it misbehaves, using a variety of clever tricks (one of which is pointer tainting). Note that non-control data attacks have the potential to  become  bigger problems than attacks that divert control, because they are harder to detect. As they use full pointer tainting, I wonder how they contain the propagation of taint.