The performance and scalability of the open-source network simulator ns-3 in simulating mobile ad hoc networks (MANETs) are evaluated in this paper. Studies on the critical importance of accuracy in network model simulations are valuable, especially those exploring topics such as radio frequency (RF) propagation, mobility, network protocols, and issues related to large networks.
This paper does not compare the evaluation results of ns-3 to those of other network simulators. The authors note that some commercial simulation tools such as QualNet or OpNet have also been evaluated on high-performance computing platforms, but did not show sufficient scalability. The key property of ns-3 compared to those simulators is the significant parallelism in its distributed scheduler, which was released in 2010.
The distributed scheduler of ns-3 decomposes the entire network into a number of federates that communicate with each other via the message passing interface (MPI). One key point is that the performance of the simulation strongly depends on the timing among these federates. Each federate is responsible for processing its local events, which can be done in perfect parallelism. Once in awhile (in some time steps), federates may need to send packets to other federates, with the result that the parallelism could not be maintained and simulator timing management would be required. By reducing the communication between federates, the time period between consecutive simulation synchronizations can be extended. This improvement in parallel computations among the federates yields a higher overall simulation performance.
To evaluate the performance of ns-3 in this sense, a scenario was developed with federates consisting of one or more teams formed by a network using the optimized link state routing (OLSR) protocol. Routing outside of network teams is static. Team leaders report the updated positions of the nodes in their own group to other team leaders periodically. Mesh nodes are used for MPI interconnections between federates.
The results show that the compute time increases exponentially when the number of nodes in a team increases. Also, when the configuration area is made larger (which means fewer nodes are in each other’s radio range), the compute time reduces due to the OLSR packets that failed to be delivered. This means that a very large part of the computational load is related to OLSR inside federates, because each federation has enough computational load to handle in each period before a simulation synchronization moment. This demonstrates good simulation performance by ns-3.
The next performance test investigated the effect of the number of central processing unit (CPU) cores available in the system. With the same scenario as above, the results show a linear performance improvement by adding the number of cores. The rest of the test shows that this improvement tops out when the number of cores increases to the point where the computational load on each core due to synchronization exceeds the load due to routing.
In another scenario with four levels of hierarchy and static nodes, the scalability of the simulator was evaluated to determine the effect of system memory available to cores when simulating different numbers of nodes. It is concluded that memory was more constraining than numbers of cores. That is, it is more efficient to dedicate a single core with all available memory to each compute node rather than using several cores that have to share available memory. With 176 compute nodes, each using one core and a memory of 17.96 GB per node, a network of 360,448,000 nodes was simulated successfully.
This paper gives a quite interesting view of the performance and scalability of ns-3 in simulating ad hoc networks. However, the authors could have shown the results more clearly if they had included graphs of the scalability results in more detail. The paper is also missing a section on related work, which could show other similar research that has been done in the evaluation of network simulators.
