Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
SDN flow entry management using reinforcement learning
Mu T., Al-Fuqaha A., Shuaib K., Sallabi F., Qadir J. ACM Transactions on Autonomous and Adaptive Systems13 (2):1-23,2018.Type:Article
Date Reviewed: Apr 24 2019

Software-defined networking (SDN) technology makes for smoother network management and configuration. It allows network resources to be efficiently assigned to both prolonged and short-term traffic flows. However, substantial network reconfiguration can create weighty overhead in traffic flows and processing due to the inadequate size of ternary content-addressable memory (TCAM) in OpenFlow network switches. How should the rules for forwarding network streams of traffic be maintained in a traffic flow table? In the event of failure to locate packets in table entries on an SDN switch, how should rules be processed by the SDN controller? Mu et al. propose machine learning algorithms for controlling traffic flow table entries to minimize packet delivery delays and long-term control plane overhead between network controllers and switches.

The authors briefly discuss and critique the OpenFlow protocol, the features and confines of OpenFlow switches, the existing methods for ameliorating and making the best use of TCAM size, and the alternative utilization of reinforcement learning (RL) in networking research. Certainly, network configuration rules should be derived from traffic flow occurrences and the time measurement of each forwarding rule in the memory of a switch.

The state-space is “all possible combinations of values of the flow match frequency and the flow recentness parameters.” The action-space states that either no action should be performed or the value of a parameter, such as the traffic flow frequency, should increase or decrease. In the reward function, “a configuration with less overhead returns a positive reward 1 to the learning agent; otherwise, the agent is given a negative value -1 as the reward”; the reward is zero when there’s no additional overhead. A tuple (state-space, action-space, state transition probability, action reward) is used to create a Markov decision process (MDP), and then MDP is used to model an RL algorithm for generating a network configuration with minimized long-term control plane overhead.

Two RL algorithms are proposed for managing the SDN traffic flow entries. With the traditional reiterative RL conducted in discrete time steps, agents choose actions at specified states and make decisions using policies to select actions that will capitalize on the probable rewards. The deep RL algorithm uses directed graph links of interrelated neurons, assigns weights with connections, and establishes layers of neurons to create feedforward neural networks.

The search for optimum parameter sets that diminish the control plane overhead is formulated as an integer linear programming (ILP) problem. To gauge the performance of traditional RL and deep RL against the reputable multiple Bloom filters (MBF) method, the authors conduct emulation experiments with a SDN controller and an OpenFlow switch that connect host nodes to transmit and receive replicate network traffic. The emulation results indicate that both the traditional and deep RL algorithms significantly curtailed the network control plane overheads. However, the deep RL agent outpaced the traditional RL agent in locating the set of induced rates of network control plane overhead when forwarding different sizes of rules from the SDN controller or switch. Moreover, the deep RL agent is significantly more accurate than both the RL agent and the MBF approach in locating rules traffic from the flow table of the SDN switch. The authors clearly explain how to effectively manage SDN flow traffic entries. However, can the investigative results from the simplified simulated network topology used in this research be applied to complex real-world networks? I call on network designers and administrators to help answer this question.

Reviewer:  Amos Olagunju Review #: CR146544 (1907-0277)
Bookmark and Share
  Reviewer Selected
Featured Reviewer
 
 
Performance of Systems (C.4 )
 
 
Model Development (I.6.5 )
 
Would you recommend this review?
yes
no
Other reviews under "Performance of Systems": Date
A computer and communications network performance analysis primer
Stuck B., Arthurs E., Prentice-Hall, Inc., Upper Saddle River, NJ, 1985. Type: Book (9789780131639812)
Jun 1 1985
A mean value performance model for locking in databases
Tay Y., Suri R. (ed), Goodman N. Journal of the ACM 32(3): 618-651, 1985. Type: Article
Mar 1 1986
The relationship between benchmark tests and microcomputer price
Sircar S., Dave D. Communications of the ACM 29(3): 212-217, 1986. Type: Article
Nov 1 1986
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy