一级黄色片免费播放|中国黄色视频播放片|日本三级a|可以直接考播黄片影视免费一级毛片

高級(jí)搜索

留言板

尊敬的讀者、作者、審稿人, 關(guān)于本刊的投稿、審稿、編輯和出版的任何問(wèn)題, 您可以本頁(yè)添加留言。我們將盡快給您答復(fù)。謝謝您的支持!

姓名
郵箱
手機(jī)號(hào)碼
標(biāo)題
留言內(nèi)容
驗(yàn)證碼

運(yùn)營(yíng)商網(wǎng)絡(luò)中基于深度強(qiáng)化學(xué)習(xí)的服務(wù)功能鏈遷移機(jī)制

陳卓 馮鋼 何穎 周楊

陳卓, 馮鋼, 何穎, 周楊. 運(yùn)營(yíng)商網(wǎng)絡(luò)中基于深度強(qiáng)化學(xué)習(xí)的服務(wù)功能鏈遷移機(jī)制[J]. 電子與信息學(xué)報(bào), 2020, 42(9): 2173-2179. doi: 10.11999/JEIT190545
引用本文: 陳卓, 馮鋼, 何穎, 周楊. 運(yùn)營(yíng)商網(wǎng)絡(luò)中基于深度強(qiáng)化學(xué)習(xí)的服務(wù)功能鏈遷移機(jī)制[J]. 電子與信息學(xué)報(bào), 2020, 42(9): 2173-2179. doi: 10.11999/JEIT190545
Zhuo CHEN, Gang FENG, Ying HE, Yang ZHOU. Deep Reinforcement Learning Based Migration Mechanism for Service Function Chain in Operator Networks[J]. Journal of Electronics & Information Technology, 2020, 42(9): 2173-2179. doi: 10.11999/JEIT190545
Citation: Zhuo CHEN, Gang FENG, Ying HE, Yang ZHOU. Deep Reinforcement Learning Based Migration Mechanism for Service Function Chain in Operator Networks[J]. Journal of Electronics & Information Technology, 2020, 42(9): 2173-2179. doi: 10.11999/JEIT190545

運(yùn)營(yíng)商網(wǎng)絡(luò)中基于深度強(qiáng)化學(xué)習(xí)的服務(wù)功能鏈遷移機(jī)制

doi: 10.11999/JEIT190545
基金項(xiàng)目: 國(guó)家自然科學(xué)基金(61471089, 61401076)
詳細(xì)信息
    作者簡(jiǎn)介:

    陳卓:男,1980年生,副教授,博士,碩士生導(dǎo)師,研究方向?yàn)榫W(wǎng)絡(luò)虛擬化和軟件定義網(wǎng)絡(luò)

    馮鋼:男,1964年生,教授,博士生導(dǎo)師,研究方向?yàn)橛?jì)算機(jī)網(wǎng)絡(luò)和無(wú)線接入網(wǎng)技術(shù)

    何穎:女,1993年生,碩士生,研究方向網(wǎng)絡(luò)虛擬化技術(shù)

    通訊作者:

    陳卓 chenzhuo@cqut.edu.cn

  • 中圖分類號(hào): TN915; TP393

Deep Reinforcement Learning Based Migration Mechanism for Service Function Chain in Operator Networks

Funds: The National Natural Science Foundation of China (61471089, 61401076)
  • 摘要: 為改善運(yùn)營(yíng)商網(wǎng)絡(luò)提供的移動(dòng)服務(wù)體驗(yàn),該文研究服務(wù)功能鏈(SFC)的在線遷移問(wèn)題。首先基于馬爾可夫決策過(guò)程(MDP)對(duì)服務(wù)功能鏈中的多個(gè)虛擬網(wǎng)絡(luò)功能(VNF)在運(yùn)營(yíng)商網(wǎng)絡(luò)中的駐留位置遷移進(jìn)行模型化分析。通過(guò)將強(qiáng)化學(xué)習(xí)和深度神經(jīng)網(wǎng)絡(luò)相結(jié)合提出一種基于雙深度Q網(wǎng)絡(luò)(double DQN)的服務(wù)功能鏈遷移機(jī)制,該遷移方法能在連續(xù)時(shí)間下進(jìn)行服務(wù)功能鏈的在線遷移決策并避免求解過(guò)程中的過(guò)度估計(jì)。實(shí)驗(yàn)結(jié)果表明,該文所提出的策略相比于固定部署算法和貪心算法在端到端時(shí)延和網(wǎng)絡(luò)系統(tǒng)收益等方面優(yōu)勢(shì)明顯,有助于運(yùn)營(yíng)商改善服務(wù)體驗(yàn)和資源的使用效率。
  • 圖  1  基于神經(jīng)網(wǎng)絡(luò)的強(qiáng)化學(xué)習(xí)決策

    圖  2  基于雙DQN的SFC遷移方法流程圖

    圖  3  移動(dòng)業(yè)務(wù)端到端時(shí)延對(duì)比

    圖  4  基于雙DQN方法在不同長(zhǎng)度VNF下的延遲對(duì)比

    圖  5  不同移動(dòng)次數(shù)下系統(tǒng)收益對(duì)比

    圖  6  系統(tǒng)累積收益對(duì)比

    表  1  基于雙DQN的SFC遷移算法的偽碼

     輸入:運(yùn)營(yíng)商網(wǎng)絡(luò)拓?fù)?G = \left( {N,E} \right)$,服務(wù)功能鏈集合C,網(wǎng)絡(luò)功
        能集合F;
     輸出:SFC遷移策略;
     步驟1:初始化隨機(jī)權(quán)重為$\psi $的神經(jīng)網(wǎng)絡(luò);
     步驟2:初始化動(dòng)作值函數(shù)Q;
     步驟3:初始化經(jīng)驗(yàn)池(experience replay)存儲(chǔ)器N;
     步驟4:for episode = 1, 2, ···, M do,
       觀察初始狀態(tài)${s^0}$,
        for t = 0, 1, ···, N–1 do,
         以概率為$\varepsilon $選擇一個(gè)隨機(jī)動(dòng)作${a^t}$,
         否則選擇動(dòng)作${a^t} = \arg \max Q({s^t},a;{\psi ^t})$;
         在仿真器中執(zhí)行動(dòng)作${a^t}$,并觀察回報(bào)${R_{t + 1}}$和新?tīng)顟B(tài)${s_{t + 1}}$,
         存儲(chǔ)中間量$ < {s^t},{a^t},{r^t},{s^{t + 1}} > $到經(jīng)驗(yàn)池存儲(chǔ)器N中,
         從經(jīng)驗(yàn)池存儲(chǔ)器N中獲取一組樣本,
          計(jì)算損失函數(shù)$L({\psi ^t})$,
          計(jì)算關(guān)于${\psi ^t}$的損失函數(shù)的梯度,
          更新${\psi ^t} \leftarrow {\psi ^t} - \phi {{\text{?}} _{ {\psi ^t} } }L({\psi ^t})$,其中$\phi $為學(xué)習(xí)率;
        end
       end
    下載: 導(dǎo)出CSV
  • CHATRAS B and OZOG F F. Network functions virtualization: The portability challenge[J]. IEEE Network, 2016, 30(4): 4–8. doi: 10.1109/MNET.2016.7513857
    ZHANG Qixia, LIU Fangming, and ZENG Chaobing. Adaptive interference-aware VNF placement for service-customized 5G network slices[C]. IEEE Conference on Computer Communications, Paris, France, 2019: 2449–2457. doi: 10.1109/INFOCOM.2019.8737660.
    AGARWAL S, MALANDRINO F, CHIASSERINI C F, et al. Joint VNF placement and CPU allocation in 5G[C]. IEEE Conference on Computer Communications, Honolulu, USA, 2018: 1943–1951. doi: 10.1109/INFOCOM.2018.8485943.
    KUO T W, LIOU B H, LIN K C J, et al. Deploying chains of virtual network functions: On the relation between link and server usage[C]. The 35th Annual IEEE International Conference on Computer Communications, San Francisco, USA, 2016: 1–9. doi: 10.1109/INFOCOM.2016.7524565.
    TALEB T, KSENTINI A, and FRANGOUDIS P A. Follow-me cloud: When cloud services follow mobile users[J]. IEEE Transactions on Cloud Computing, 2019, 7(2): 369–382. doi: 10.1109/TCC.2016.2525987
    ERAMO V, MIUCCI E, AMMAR M, et al. An approach for service function chain routing and virtual function network instance migration in network function virtualization architectures[J]. IEEE/ACM Transactions on Networking, 2017, 25(4): 2008–2025. doi: 10.1109/TNET.2017.2668470
    HOUIDI O, SOUALAH O, LOUATI W, et al. An efficient algorithm for virtual network function scaling[C]. 2017 IEEE Global Communications Conference, Singapore, 2017: 1–7. doi: 10.1109/GLOCOM.2017.8254727.
    CHO D, TAHERI J, ZOMAYA A Y, et al. Real-time Virtual Network Function (VNF) migration toward low network latency in cloud environments[C]. The 10th IEEE International Conference on Cloud Computing, Honolulu, USA, 2017: 798–801. doi: 10.1109/CLOUD.2017.118.
    蘭巨龍, 于倡和, 胡宇翔, 等. 基于深度增強(qiáng)學(xué)習(xí)的軟件定義網(wǎng)絡(luò)路由優(yōu)化機(jī)制[J]. 電子與信息學(xué)報(bào), 2019, 41(11): 2669–2674. doi: 10.11999/JEIT180870

    LAN Julong, YU Changhe, HU Yuxiang, et al. A SDN routing optimization mechanism based on deep reinforcement learning[J]. Journal of Electronics &Information Technology, 2019, 41(11): 2669–2674. doi: 10.11999/JEIT180870
    HUANG Xiaohong, YUAN Tingting, QIAO Guanhua, et al. Deep reinforcement learning for multimedia traffic control in software defined networking[J]. IEEE Network, 2018, 32(6): 35–41. doi: 10.1109/MNET.2018.1800097
    LEE J W, MAZUMDAR R R, and SHROFF N B. Non-Convex optimization and rate control for multi-class services in the Internet[J]. IEEE/ACM Transactions on Networking, 2005, 13(4): 827–840. doi: 10.1109/TNET.2005.852876
    李晨溪, 曹雷, 陳希亮, 等. 基于云推理模型的深度強(qiáng)化學(xué)習(xí)探索策略研究[J]. 電子與信息學(xué)報(bào), 2018, 40(1): 244–248. doi: 10.11999/JEIT170347

    LI Chenxi, CAO Lei, CHEN Xiliang, et al. Cloud reasoning model-based exploration for deep reinforcement learning[J]. Journal of Electronics &Information Technology, 2018, 40(1): 244–248. doi: 10.11999/JEIT170347
    MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236
    GHAZNAVI M, KHAN A, SHAHRIAR N, et al. Elastic virtual network function placement[C]. The 4th IEEE International Conference on Cloud Networking, Niagara Falls, Canada, 2015: 255–260. doi: 10.1109/CloudNet.2015.7335318.
    SUGISONO K, FUKUOKA A, and YAMAZAKI H. Migration for VNF instances forming service chain[C]. The 7th IEEE International Conference on Cloud Networking, Tokyo, Japan, 2018: 1–3. doi: 10.1109/CloudNet.2018.8549194.
    LIN Tachun, ZHOU Zhili, TORNATORE M, et al. Demand-aware network function placement[J]. Journal of Lightwave Technology, 2016, 34(11): 2590–2600. doi: 10.1109/JLT.2016.2535401
  • 加載中
圖(6) / 表(1)
計(jì)量
  • 文章訪問(wèn)數(shù):  1732
  • HTML全文瀏覽量:  829
  • PDF下載量:  107
  • 被引次數(shù): 0
出版歷程
  • 收稿日期:  2019-07-18
  • 修回日期:  2020-06-14
  • 網(wǎng)絡(luò)出版日期:  2020-07-14
  • 刊出日期:  2020-09-27

目錄

    /

    返回文章
    返回