基于深度強化學(xué)習(xí)的服務(wù)功能鏈可靠部署算法
doi: 10.11999/JEIT190969
-
1.
重慶郵電大學(xué)通信與信息工程學(xué)院 重慶 400065
-
2.
重慶郵電大學(xué)移動通信技術(shù)重點實驗室 重慶 400065
Reliable Deployment Algorithm of Service Function Chain Based on Deep Reinforcement Learning
-
1.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
-
2.
Key Laboratory of Mobile Communication Technology, Chongqing University of Post and Telecommunications, Chongqing 400065, China
-
摘要:
針對當(dāng)前關(guān)于服務(wù)功能鏈(SFC)的部署問題都未考慮到虛擬網(wǎng)絡(luò)功能(VNF)的失效重要度，該文提出了基于深度強化學(xué)習(xí)的SFC可靠部署算法。首先建立VNF和虛擬鏈路可靠映射模型，為重要的VNF設(shè)置高可靠性需求，并通過鏈路部署長度限制盡可能保證虛擬鏈路可靠性需求。其次，以負載均衡為資源協(xié)調(diào)原則，與VNF可靠性聯(lián)合優(yōu)化，最終使用深度強化學(xué)習(xí)得到服務(wù)功能鏈部署策略。另外，提出了基于重要度的節(jié)點備份和鏈路備份策略，用于應(yīng)對部署過程中VNF/鏈路可靠性難以滿足的情況。仿真結(jié)果表明，該文的可靠部署算法在保證可靠性需求的基礎(chǔ)上能夠有效減少SFC失效損失，同時使虛擬網(wǎng)絡(luò)更加穩(wěn)定可靠。
-
關(guān)鍵詞:
- 網(wǎng)絡(luò)虛擬化 /
- 服務(wù)功能鏈 /
- 可靠性 /
- 深度強化學(xué)習(xí)
Abstract:In view of the current deployment of the Service Function Chain (SFC), the failure importance of the Virtual Network Function (VNF) is not considered,an SFC reliable deployment algorithm based on deep reinforcement learning is proposed. Firstly, a reliable mapping model of VNF and virtual links is establised, high reliability requirements is set for important VNFs, and the reliability requirements of virtual links is ensured as much as possible through link deployment length restrictions. Secondly, taking load balancing as the resource coordination principle, joint optimization the VNF reliability is jointly optimized. Finally, the deep reinforcement learning is used to get the service function chain deployment strategy. In addition, node backup and link backup strategies based on importance are proposed to deal with situations where VNF/link reliability is difficult to meet during deployment. Simulation results show that the reliable deployment algorithm in this paper can effectively reduce the failure SFC loss on the basis of ensuring the reliability requirements, and at the same time make the virtual network more stable and reliable.
-
表 1 基于深度強化學(xué)習(xí)的服務(wù)功能鏈可靠部署算法
算法1 基于深度強化學(xué)習(xí)的服務(wù)功能鏈可靠部署算法 經(jīng)驗回放池初始化為空，隨機初始化主網(wǎng)絡(luò)的參數(shù)，并復(fù)制給目標(biāo)網(wǎng)絡(luò)，探尋環(huán)境信息基于式(6)～式(9)得到${w_I}$,${u'_k}$ for episode=1, 2, ···, E do 初始化狀態(tài)為${{{s}}_1} = \{ {{{B}}_{{\rm{left}}}}{{,}}{{{C}}_{{\rm{left}}}}{{,}}{{}_{{\rm{map}}}}{{,}}{{{c}}_{{\rm{map}}}}\} $, ${{}_{{\rm{map}}}}{{,}}{{{c}}_{{\rm{map}}}}$為空 for t=1,2…D do 確定可選動作集${N'_{k{n_i}}}$, ${N''_{k{n_i}}}$隨機產(chǎn)生0到1的數(shù)$\tau $ if $\tau < \varepsilon $：先從${ {{N'} }_{k{n_i} } }$中選擇動作${a_t}$，若為空則根據(jù)算法2在${ {{N''} }_{k{n_i} } }$中選擇${a_t}$，都無動作選擇則${a_t}$為空 else：依據(jù)主神經(jīng)網(wǎng)絡(luò)估值${\pi ^*}(s) = {{\arg }}\max\limits_{a'} Q({{{s}}_t},{{{a'}}_t})$選取動作。end if 執(zhí)行動作${{{a}}_t}$，獲得獎勵${r_t}$、下一狀態(tài)${{{s}}_{t + 1}}$，將向量$[{{{s}}_t},{{{a}}_t},{r_t},{{{s}}_{t + 1}}]$放入經(jīng)驗回放池。 隨機取出小批量樣本向量$[{{{s}}_t},{{{a}}_t},{r_t},{{{s}}_{t + 1}}]$組成樣本向量集${\upsilon _t}$ for $[{{{s}}_t},{{{a}}_t},{r_t},{{{s}}_{t + 1}}]$ in ${\upsilon _t}$ do $L{{(}}\theta {{)}} = {(\tilde Q({{{s}}_t},{{{a}}_t}|{\theta ^ - }) - Q({{{s}}_t},{{{a}}_t}|\theta ))^2}$，$\theta \leftarrow \theta - {\lambda _{{\rm{SGD}}}}\nabla L(\theta )$ end for 在θ更新T次后將主網(wǎng)絡(luò)參數(shù)復(fù)制給目標(biāo)網(wǎng)絡(luò)。end for end for 通過主網(wǎng)絡(luò)得到每個VNF的策略${\pi ^*}(s)$ if累計獎勵$Q({s_t},{a_t}) < - 100$：拒絕該請求 else if 部署結(jié)果不滿足鏈路可靠性需求：執(zhí)行算法3 end if 
下載: 導(dǎo)出CSV
表 2 基于虛擬網(wǎng)絡(luò)功能重要度節(jié)點備份算法
算法2：基于虛擬網(wǎng)絡(luò)功能重要度的節(jié)點備份算法 初始化節(jié)點部署方案$g'(v_I^k)$為空 repeat： if ${\omega _I} < 1/D$：搜尋${N''_{k{n_i}}}$已作為備份節(jié)點的節(jié)點集$V_{I'}^k$ if $V_{I'}^k$非空：$g'(v_I^k)$←從$V_{I'}^k$中尋找備份節(jié)點，從${{{N''}}_{k{n_i}}}/V_{I'}^k$中尋
找部署節(jié)點else：$g'(v_I^k)$←從${N''_{k{n_i}}}$中選出2個節(jié)點?？煽啃宰罡叩臑椴渴鸸?jié)
點，另一備份。endifelse：$g'(v_I^k)$←從${N''_{k{n_i}}}$中選出2個節(jié)點?？煽啃宰罡叩臑椴渴鸸?jié)
點，另一備份。endifuntil $RN(g'(v_I^k)) > {(R_k^{{\rm{req}}})^{{w_I}}}$or 已遍歷所有組合 return 節(jié)點部署方案$g'(v_I^k)$
下載: 導(dǎo)出CSV
表 3 基于鏈路備份重要度的鏈路備份算法
算法3：基于鏈路備份重要度的鏈路備份算法 基于$\gamma _{I,J}^k$對虛擬鏈路鏈路$l_{I,J}^k \in L_v^k$排序 for 排過序的鏈路鏈路$l_{I,J}^k \in L_v^k$ if 鏈路可靠性不滿足： 確定$l_{I,J}^k$映射的段物理起始節(jié)點${n_i}$和終止節(jié)點${n_j}$:${p_{I,J}}$ $p{{ = \{ }}{\ell _{i,j}}|{L'_{k{l_{i,j}}}}/{p_{I,J}}\} $，部署方案增加鏈路P else: break end for return 部署方案
下載: 導(dǎo)出CSV
-
AYOUBI S, ZHANG Yanhong, and ASSI C. A reliable embedding framework for elastic virtualized services in the cloud[J]. IEEE Transactions on Network and Service Management, 2016, 13(3): 489–503. doi: 10.1109/TNSM.2016.2581484 湯紅波, 邱航, 游偉, 等. 基于聯(lián)合備份的服務(wù)功能鏈可靠性保障的部署方法[J]. 電子與信息學(xué)報, 2019, 41(12): 3006–3013. doi: 10.11999/JEIT190013TANG Hongbo, QIU Hang, YOU Wei, et al. A reliability-guarantee method for service function chain deployment based on joint backup[J]. Journal of Electronics &Information Technology, 2019, 41(12): 3006–3013. doi: 10.11999/JEIT190013 季新生, 趙碩, 艾健健, 等. 異構(gòu)備份式的虛擬網(wǎng)映射方法研究[J]. 電子與信息學(xué)報, 2018, 40(5): 1087–1093. doi: 10.11999/JEIT170730JI Xinsheng, ZHAO Shuo, AI Jianjian, et al. Research on heterogeneous-backup virtual network embedding[J]. Journal of Electronics &Information Technology, 2018, 40(5): 1087–1093. doi: 10.11999/JEIT170730 POURVALI M, BAI Hao, CRICHIGNO J, et al. Multicast virtual network services embedding for improved disaster recovery support[J]. IEEE Communications Letters, 2018, 22(7): 1362–1365. doi: 10.1109/LCOMM.2018.2822739 SUN Jian, ZHU Guangyang, SUN Gang, et al. A reliability-aware approach for resource efficient virtual network function deployment[J]. IEEE Access, 2018, 6: 18238–18250. doi: 10.1109/ACCESS.2018.2815614 DINH N T and KIM Y. An efficient reliability guaranteed deployment scheme for service function chains[J]. IEEE Access, 2019, 7: 46491–46505. doi: 10.1109/ACCESS.2019.2908185 劉光遠, 安秀芳, 蘇森. 基于節(jié)點可靠性感知和共享路徑保護的虛擬網(wǎng)映射算法研究[J]. 通信學(xué)報, 2016, 37(8): 51–57. doi: 10.11959/j.issn.1000-436x.2016155LIU Guangyuan, AN Xiufang, and SU Sen. Virtual network mapping algorithm with node reliability awareness and shared-path protection[J]. Journal on Communications, 2016, 37(8): 51–57. doi: 10.11959/j.issn.1000-436x.2016155 European Telecommunications Standards Institute. Network Functions Virtualisation (NFV); Reliability; Report on models and features for end-to-end reliability[R]. ETSI GS NFV-REL 003 V1.1.1, 2016. TANG Xiaoyong, LI Kenli, QIU Meikang, et al. A hierarchical reliability-driven scheduling algorithm in grid systems[J]. Journal of Parallel and Distributed Computing, 2012, 72(4): 525–535. doi: 10.1016/j.jpdc.2011.12.004 BIJWE S, MACHIDA F, ISHIDA S, et al. End-to-End reliability assurance of service chain embedding for network function virtualization[C]. 2017 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Berlin, Germany, 2017: 1–4. doi: 10.1109/NFV-SDN.2017.8169853. KUO Tungwei, LIOU B Hheng, LIN K C J, et al. Deploying Chains of virtual network functions: On the relation between link and server usage[J]. IEEE/ACM Transactions on Networking, 2018, 26(4): 1562–1576. doi: 10.1109/TNET.2018.2842798 魏亮, 黃韜, 張嬌, 等. 基于強化學(xué)習(xí)的服務(wù)鏈映射算法[J]. 通信學(xué)報, 2018, 39(1): 90–100. doi: 10.11959/j.issn.1000-436x.2018002WEI Liang, HUANG Tao, ZHANG Jiao, et al. Service Chain mapping algorithm based on reinforcement learning[J]. Journal on Communications, 2018, 39(1): 90–100. doi: 10.11959/j.issn.1000-436x.2018002 CAO Haotong, ZHU Yongxu, ZHENG Gan, et al. A novel optimal mapping algorithm with less computational complexity for virtual network embedding[J]. IEEE Transactions on Network and Service Management, 2018, 15(1): 356–371. doi: 10.1109/TNSM.2017.2778106 ZHANG Xin, QIAN Zhuzhong, ZHANG Sheng, et al. Pattern-aware reliable virtual network function chain deployment[C]. 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC), Guangzhou, China, 2017: 335–342. doi: 10.1109/ISPA/IUCC.2017.00055. SHAHRIAR N, AHMED R, CHOWDHURY S R, et al. Generalized recovery from node failure in virtual network embedding[J]. IEEE Transactions on Network and Service Management, 2017, 14(2): 261–274. doi: 10.1109/TNSM.2017.2693404 -
計量
- 文章訪問數(shù): 2781
- HTML全文瀏覽量: 1331
- PDF下載量: 187
- 被引次數(shù): 0
下載:
