基于深度增強學習的軟件定義網絡路由優(yōu)化機制
doi: 10.11999/JEIT180870
-
國家數字交換系統(tǒng)工程技術研究中心 ??鄭州 ??450002
基金項目: 國家自然科學基金群體創(chuàng)新項目(61521003),國家自然科學基金(61502530)
A SDN Routing Optimization Mechanism Based on Deep Reinforcement Learning
-
National Digital Switching System Engineering & Technological Research Center, Zhengzhou 450002, China
Funds: The National Natural Science Foundation of China for Innovative Research Groups (61521003), The National Natural Science Foundation of China (61502530)
-
摘要: 為優(yōu)化軟件定義網絡(SDN)的路由選路,該文將深度增強學習原理引入到軟件定義網絡的選路過程,提出一種基于深度增強學習的路由優(yōu)化選路機制,用以削減網絡運行時延、提高吞吐量等網絡性能,實現連續(xù)時間上的黑盒優(yōu)化,減少網絡運維成本。此外,該文通過實驗對所提出的路由優(yōu)化機制進行評估,實驗結果表明,路由優(yōu)化機制具有良好的收斂性與有效性,較傳統(tǒng)路由協(xié)議可提供更優(yōu)的路由方案與實現更穩(wěn)定的性能。Abstract: In order to achieve routing optimization in the Software Defined Network (SDN) environment, deep reinforcement learning is imposed to the SDN routing process and a mechanism based on deep reinforcement learning is proposed to optimize routing. This mechanism can improve network performance such as delay, throughput, and realize black-box optimization in continuous time, which surely reduces network operation and maintenance costs. Besides, the proposed routing optimization mechanism is evaluated through a series of experiments. The experimental results show that the proposed SDN routing optimization mechanism has good convergence and effectiveness, and can provide better routing configurations and performance stability than traditional routing protocols.
-
BOUTABA R, SALAHUDDIN M A, LIMAM N, et al. A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities[J]. Journal of Internet Services and Applications, 2018, 9(1): 16. doi: 10.1186/s13174-018-0087-2 FADLULLAH Z M, TANG Fengxiao, MAO Bomin, et al. State-of-the-art deep learning: Evolving machine intelligence toward tomorrow’s intelligent network traffic control systems[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2432–2455. doi: 10.1109/COMST.2017.2707140 LI Wei, LI Guojun, and YU Xiufen. A fast traffic classification method based on SDN network[C]. The 4th International Conference on Electronics, Communications and Networks, Beijing, China, 2015: 223–229. WANG Fu, LIU Bo, ZHANG Lijia, et al. Dynamic routing and spectrum assignment based on multilayer virtual topology and ant colony optimization in elastic software-defined optical networks[J]. Optical Engineering, 2017, 56(7): 076111. doi: 10.1117/1.OE.56.7.076111 PARSAEI M R, MOHAMMADI R, and JAVIDAN R. A new adaptive traffic engineering method for telesurgery using ACO algorithm over Software Defined Networks[J]. European Research in Telemedicine, 2017, 6(3/4): 173–180. doi: 10.1016/j.eurtel.2017.10.003 WANG Junchao, DE LAAT C, and ZHAO Zhiming. QoS-aware virtual SDN network planning[C]. 2017 IFIP/IEEE Symposium on Integrated Network and Service Management, Lisbon, Portugal, 2017: 644–647. doi: 10.23919/INM.2017.7987350. LIN S C, AKYILDIZ I F, WANG Pu, et al. QoS-aware adaptive routing in multi-layer hierarchical software defined networks: a reinforcement learning approach[C]. 2016 IEEE International Conference on Services Computing, San Francisco, USA, 2016: 25–33. doi: 10.1109/SCC.2016.12. JIANG Jingyan, HU Liang, HAO Pingting, et al. Q-FDBA: Improving QoE fairness for video streaming[J]. Multimedia Tools and Applications, 2018, 77(9): 10787–10806. doi: 10.1007/s11042-017-4917-1 SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. Cambridge, MA: The MIT Press, 1988. SENDRA S, REGO A, LLORET J, et al. Including artificial intelligence in a routing protocol using Software Defined Networks[C]. 2017 IEEE International Conference on Communications Workshops, Paris, France, 2017: 670–674. doi: 10.1109/ICCW.2017.7962735. MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236 LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[P]. USA, Patent, WO2017019555, 2017. MESTRES A, RODRIGUEZ-NATAL A, CARNER J, et al. Knowledge-defined networking[J]. ACM SIGCOMM Computer Communication Review, 2017, 47(3): 2–10. doi: 10.1145/3138808.3138810 SILVER D, LEVER G, HEESS N, et al. Deterministic policy gradient algorithms[C]. International Conference on Machine Learning, Beijing, China, 2014: I-387–I-395. VARGA A and HORNIG R. An overview of the OMNeT++ simulation environment[C]. The 1st International Conference on Simulation Tools and Techniques for Communications, Networks and Systems & Workshops, Marseille, France, 2008: 60. ROUGHAN M. Simplifying the synthesis of internet traffic matrices[J]. ACM SIGCOMM Computer Communication Review, 2005, 35(5): 93–96. doi: 10.1145/1096536.1096551 PAN S J and YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359. doi: 10.1109/TKDE.2009.191 -