Detecting intrusions in operating systems has been a subject of academic and industrial research for quite some time. Such intrusion detection systems fall into two broad categories: network-based and host-based. Host-based intrusion detection systems typically function by analyzing the system calls generated by “normal” processes. Such normal system call sequences are compared with those generated by the processes on a deployed system. Any deviations from the expected normal behavior can be then flagged as a possible intrusion.

One of the major problems with detecting intrusions in this manner is that the host-based intrusion detection agent software (from which such anomaly data is collected) can itself have been altered as a result of the intrusion. For example, the intruder could have manipulated the detection agent software to always report a normal sequence of system calls. Hence, such data cannot be entirely relied upon.

This paper presents an interesting approach, whereby platform virtualization technology is used to provide a fair degree of isolation between the detection agent and the system under possible intrusion. The essence of the approach lies in encapsulating the monitored system inside a “guest” virtual machine environment, and keeping the intrusion detection agent software isolated outside the virtual machine where the monitored system executes. The authors used a type two virtual machine monitor-based virtualization technology for this purpose (“type two” refers to a virtualization technology where the monitor executes as a normal process inside the host machine, which contains software to create more guest virtual machines).

One question remains unanswered: How do we ensure that the host machine is not running intruder software? Such an intrusion on the host-based monitor can have drastic consequences on its ability to report intrusion problems in the guest virtual machines.