This very readable paper approaches its topic by providing a dense but informative introduction, describing a novel testbed for evaluating intrusion detection (ID) software, and presenting the results of evaluating several intrusion detection software systems. The authors worked at the Air Force Research Laboratory, and their work addresses issues associated with computer security at military installations.

The introduction describes threats and attack techniques and includes a table of 19 attacks used to evaluate ID software. It also discusses ID basics: host-based, network-based, and router-based detection. The most interesting part is the description of a test environment developed by the authors. This environment is an immersive testbed simulating a typical military metropolitan area network (MAN). The virtual network comprises one firewall providing access to the Internet and many routers with attached hosts and subnets. This real-time network was simulated by a much smaller and simpler architecture: one firewall, two border routers, several internal routers, and several host computers, some of which were on a subnet. A separate computer generated outside traffic (coming from the Internet and passing through the firewall). Another computer, also not part of the network, generated inside traffic, which was inserted into the network through the two border routers. The traffic generators could run ten simultaneous network sessions (telnet, ftp, and so on).

The results of an evaluation of four ID systems--three advanced DARPA prototypes and a government off-the-shelf product--were predictable. All four missed most kinds of attacks and had far too many false alarms.