一级黄色片免费播放|中国黄色视频播放片|日本三级a|可以直接考播黄片影视免费一级毛片

高級搜索

留言板

尊敬的讀者、作者、審稿人, 關(guān)于本刊的投稿、審稿、編輯和出版的任何問題, 您可以本頁添加留言。我們將盡快給您答復(fù)。謝謝您的支持!

姓名
郵箱
手機(jī)號碼
標(biāo)題
留言內(nèi)容
驗證碼

基于Q-Learning算法的毫微微小區(qū)功率控制算法

李云 唐英 劉涵霄

李云, 唐英, 劉涵霄. 基于Q-Learning算法的毫微微小區(qū)功率控制算法[J]. 電子與信息學(xué)報, 2019, 41(11): 2557-2564. doi: 10.11999/JEIT181191
引用本文: 李云, 唐英, 劉涵霄. 基于Q-Learning算法的毫微微小區(qū)功率控制算法[J]. 電子與信息學(xué)報, 2019, 41(11): 2557-2564. doi: 10.11999/JEIT181191
Yun LI, Ying TANG, Hanxiao LIU. Power Control Algorithm Based on Q-Learning in Femtocell[J]. Journal of Electronics & Information Technology, 2019, 41(11): 2557-2564. doi: 10.11999/JEIT181191
Citation: Yun LI, Ying TANG, Hanxiao LIU. Power Control Algorithm Based on Q-Learning in Femtocell[J]. Journal of Electronics & Information Technology, 2019, 41(11): 2557-2564. doi: 10.11999/JEIT181191

基于Q-Learning算法的毫微微小區(qū)功率控制算法

doi: 10.11999/JEIT181191
基金項目: 國家自然科學(xué)基金(61671096),重慶市研究生科研創(chuàng)新項目(CYS17220),重慶市“科技創(chuàng)新領(lǐng)軍人才支持計劃”(CSTCCXLJRC201710),重慶市基礎(chǔ)科學(xué)與前沿技術(shù)研究項目(cstc2017jcyjBX0005),重慶市留學(xué)人員創(chuàng)業(yè)創(chuàng)新支持計劃
詳細(xì)信息
    作者簡介:

    李云:男,1974年生,教授,博士生導(dǎo)師,主要研究領(lǐng)域為無線移動通信

    唐英:女,1993年生,碩士生,研究方向為異構(gòu)蜂窩無線網(wǎng)絡(luò)

    劉涵霄:男,1994年生,碩士生,研究方向為異構(gòu)蜂窩無線網(wǎng)絡(luò)

    通訊作者:

    唐英 17749963914@163.com

  • 中圖分類號: TN92

Power Control Algorithm Based on Q-Learning in Femtocell

Funds: The National Natural Science Foundation of China (61671096), The Chongqing Research and Innovation Program of Graduated Students (CYS17220), The Chongqing Science and Technology Innovation Leadership Talent Support Program (CSTCCXLJRC201710), The Chongqing Research Program of Basic Science and Frontier Technology (cstc2017jcyjBX0005), The Chongqing Overseas Students Entrepreneurship and Innovation Support Plan
  • 摘要: 該文研究macro-femto異構(gòu)蜂窩網(wǎng)絡(luò)中移動用戶的功率控制問題,首先建立了以最小接收信號信干噪比為約束條件,最大化毫微微小區(qū)的總能效為目標(biāo)的優(yōu)化模型;然后提出了基于Q-Learning算法的毫微微小區(qū)集中式功率控制(PCQL)算法,該算法基于強(qiáng)化學(xué)習(xí),能在沒有準(zhǔn)確信道狀態(tài)信息的情況下,實現(xiàn)對小區(qū)內(nèi)所有用戶終端的發(fā)射功率統(tǒng)一調(diào)整。仿真結(jié)果表明該算法能實現(xiàn)對用戶終端的功率有效控制,提升系統(tǒng)能效。
  • 圖  1  異構(gòu)蜂窩網(wǎng)絡(luò)模型

    圖  2  代理自主學(xué)習(xí)過程

    圖  3  小區(qū)用戶數(shù)為4時,系統(tǒng)能效對比

    圖  4  小區(qū)用戶數(shù)為4時,系統(tǒng)吞吐量對比

    圖  5  系統(tǒng)能效與用戶數(shù)的關(guān)系

    圖  6  系統(tǒng)吞吐量與用戶數(shù)的關(guān)系

    圖  7  信道狀態(tài)信息存在估計誤差時,系統(tǒng)能效與用戶數(shù)的關(guān)系

    圖  8  信道狀態(tài)信息存在估計誤差時,系統(tǒng)吞吐量與用戶數(shù)的關(guān)系

    圖  9  能效優(yōu)化的算法運(yùn)行時間對比

    圖  10  吞吐量優(yōu)化的算法運(yùn)行時間對比

    表  1  基于Q-Learning算法的毫微微小區(qū)功率控制算法(PCQL)

     輸入:W, ${n_0}$, $P_{b,\mu }^{\rm{c}} $, ${\rm{SINR}}_{b,\mu }^{\min }$, $p_{b,\mu }^{{\rm{max}}}$, $\gamma $, $\alpha $, $T\;$, $\varepsilon $,動作空間${A_b}$;
     輸出:${{\text{π}}^ * }$, $p_{b,\mu }^*$($\mu \in {U_b}$);
     定義:${\text{k}}$表示代理選取的動作;${\rm{SINR}}_{b,\mu }^{{\rm{real}}}$表示${u_{b,\mu }}$與基站$b$通信時 的實際信干噪比;
     $Q\left( {{{\text{s}}_b},{{\text{a}}_b}} \right) = 0$, ${\text{π}}\left( {{{\text{s}}_b},{{\text{a}}_b}} \right) = \frac{1}{{\left| {{A_b}\left( {{{\text{s}}_b}} \right)} \right|}}$, $\text{s}_b^t = \text{s}_b^0$;
     for $t = 0,1, ·\!·\!· ,T\;$ do
     若rand()<$\varepsilon $,從${A_b}$中隨機(jī)選動作${\text{k}}$;否則${\text{k}} \!=\! \mathop {\arg \max }\limits_{{\text{a}}_b^t} \!Q\left( {{\text{s}}_b^t,{\text{a}}_b^t} \right)$;
     根據(jù)式(1)確定${\rm{SINR}}_{b,\mu }^{{\rm{real}}}$;
     for $\mu = 1,2, ·\!·\!· ,{N_b}$ do
     若${\rm{SINR}}_{b,\mu }^{{\mathop{\rm real}\nolimits} } \ge {\rm{SINR}}_{b,\mu }^{\min }$,那么${\lambda _{b,\mu }} = 1$;否則${\lambda _{b,\mu }} = 0$;
     end for;
     根據(jù)式(7)計算采取動作${\text{a}}_b^t = {\text{k}}$所帶來的獎賞值${\Re _b}\left( {{\text{s}}_b^t,{\text{a}}_b^t} \right)$;
     ${\text{a}}_b^{t + 1} = {\text{π}}\left( {{\text{s}}_b^{t + 1}} \right)$;
     ${\rm Q}\left( { { {\text{s} } }_b^t,{ {\text{a} } }_b^t} \right) \leftarrow {\rm Q}\left( { { {\text{s} } }_b^t,{ {\text{a} } }_b^t} \right) + \alpha ( {\Re _b}\left( { { {\text{s} } }_b^t,{ {\text{a} } }_b^t} \right) \!+\! \gamma \mathop {\max}\limits_{ {\rm{a} }_b^{t + 1} } \left( { {\rm Q}\left( { { {\text{s} } }_b^{t + 1},{ {\text{a} } }_b^{t + 1} } \right)} \right)$  $\left.- {{\rm Q}\left( {{{\text{s}}}_b^t,{{\text{a}}}_b^t} \right)} \right)$;
     ${\text{s}}_b^t \leftarrow {\text{s}}_b^{t + 1}$;
     end for;
     ${{\text{π}}^ * }\left( {{{\text{s}}_b}} \right) = \mathop {\arg \max }\limits_{{{\text{a}}_b}} Q\left( {{{\text{s}}_b},{{\text{a}}_b}} \right),\forall {{\text{s}}_b} \in S$.
    下載: 導(dǎo)出CSV

    表  2  主要的仿真參數(shù)

    參數(shù)名稱參數(shù)值
    MBS/FBS1個/4個
    MUE/FUE最大的發(fā)射功率37 dBm/30 dBm
    MBS/FBS覆蓋范圍半徑250 m/50 m
    ${{\rm{SINR}} _{b,\mu }}^{\min }$–9 dB
    固定的電路功耗100 mW
    信道帶寬10 MHz
    高斯白噪聲的功率譜密度${10^{ - 11}}$ W/Hz
    下載: 導(dǎo)出CSV
  • LóPEZ-PéREZ D, DING M, CLAUSSEN H, et al. Towards 1 Gbps/UE in cellular systems: understanding ultra-dense small cell deployments[J]. IEEE Communications Surveys & Tutorials, 2015, 17(4): 2078–2101. doi: 10.1109/COMST.2015.2439636
    YUNAS S F, VALKAMA M, and NIEMEL? J. Spectral and energy efficiency of ultra-dense networks under different deployment strategies[J]. IEEE Communications Magazine, 2015, 53(1): 90–100. doi: 10.1109/MCOM.2015.7010521
    MARTOLIA D, SATHYA V, RANGISETTI A K, et al. Enhancing performance of victim macro users via joint ABSF and dynamic power control in LTE HetNets[C]. The Twenty-third National Conference on Communications, Chennai, India, 2017: 1–6.
    SHIN D and CHOI S. Dynamic power control for balanced data traffic with coverage in femtocell networks[C]. The 8th International Wireless Communications and Mobile Computing Conference, Limassol, Cyprus, 2012: 648–653.
    ZHANG Jinzhu, HONG Peilin, XUE Kaiping, et al. A novel power control scheme for femtocell in heterogeneous networks[C]. 2012 IEEE Consumer Communications and Networking Conference, Las Vegas, USA, 2012: 802–806.
    PAN Zhenni, MEGUMI, SAITOU, et al. Neuron control-based power adjustment scheme for sleep two-tier cellular networks[C]. 2014 IEEE Wireless Communications and Networking Conference, Istanbul, Turkey, 2014: 3201–3206.
    ZHOU Tianqing, LIU Zunxiong, ZHAO Junhui, et al. Joint user association and power control for load balancing in downlink heterogeneous cellular networks[J]. IEEE Transactions on Vehicular Technology, 2018, 67(3): 2582–2593. doi: 10.1109/TVT.2017.2768574
    MARTIN-VEGA F J, GOMEZ G, AGUAYO-TORRES M C, et al. Analytical modeling of interference aware power control for the uplink of heterogeneous cellular networks[J]. IEEE Transactions on Wireless Communications, 2016, 15(10): 6742–6757. doi: 10.1109/TWC.2016.2588469
    ZHANG Jing, LIAO Yan, and XIN Yili. Uplink power control for heterogeneous small cell networks[C]. 2016 IEEE 83rd Vehicular Technology Conference, Nanjing, China, 2016: 1–5.
    WANG Min, GAO Hui, and LV Tiejun. Energy-efficient user association and power control in the heterogeneous network[J]. IEEE Access, 2017, 5: 5059–5068. doi: 10.1109/ACCESS.2017.2690305
    ZHANG Jing, XIANG Lin, NG D W K, et al. Energy efficiency evaluation of multi-tier cellular uplink transmission under maximum power constraint[J]. IEEE Transactions on Wireless Communications, 2017, 16(11): 7092–7107. doi: 10.1109/TWC.2017.2739142
    PAN Zhenni and SHIMAMOTO S. Cell sizing based energy optimization in joint macro-femto deployments via sleep activation[C]. 2013 IEEE Wireless Communications and Networking Conference, Shanghai, China, 2013: 4765–4770.
    SHIFAT A S M Z, CHOWDHURY M Z, and JANG Y M. Game-based approach for QoS provisioning and interference management in heterogeneous networks[J]. IEEE Access, 2018, 6: 10208–10220. doi: 10.1109/ACCESS.2017.2704094
    MISHRA S and MURTHY C S R. Increasing energy efficiency via transmit power spreading in dense femto cell networks[J]. IEEE Systems Journal, 2018, 12(1): 971–980. doi: 10.1109/JSYST.2016.2573845
    GURUACHARYA S, NIYATO D, KIM D I, et al. Hierarchical competition for downlink power allocation in OFDMA femtocell networks[J]. IEEE Transactions on Wireless Communications, 2013, 12(4): 1543–1553. doi: 10.1109/TWC.2013.022213.120016
    WANG Haining, WANG Jiaheng, and DING Zhi. Distributed power control in a two-tier heterogeneous network[J]. IEEE Transactions on Wireless Communications, 2015, 14(12): 6509–6523. doi: 10.1109/TWC.2015.2456055
    MAO Tingli, FENG Gang, LIANG Liang, et al. Energy-efficient power control for macro-femto networks[C]. The 22nd Wireless and Optical Communication Conference, Chongqing, China, 2013: 122–125.
    MAO Tingli, FENG Gang, LIANG Liang, et al. Distributed energy-efficient power control for macro-femto networks[J]. IEEE Transactions on Vehicular Technology, 2016, 65(2): 718–731. doi: 10.1109/TVT.2015.2402618
    LAI Weisheng, CHANG T H, and LEE T S. Joint power and admission control for spectral and energy efficiency maximization in heterogeneous OFDMA networks[J]. IEEE Transactions on Wireless Communications, 2016, 15(5): 3531–3547. doi: 10.1109/TWC.2016.2522958
    LOODARICHEH R A, MALLICK S, BHARGAVA V K. Energy-efficient resource allocation for OFDMA cellular networks with user cooperation and QoS provisioning[J]. IEEE Transactions on Wireless Communications, 2014, 13(11): 6132–6146. doi: 10.1109/TWC.2014.2329877
    GHADIMI E, CALABRESE F D, PETERS G, et al. A reinforcement learning approach to power control and rate adaptation in cellular networks[C]. 2017 IEEE International Conference on Communications, Paris, France, 2017: 1–7.
    周志華. 機(jī)器學(xué)習(xí)[M]. 北京: 清華大學(xué)出版社, 2016: 372–390.

    ZHOU Zhihua. Machine Learning[M]. Beijing: Tsinghua University Press, 2016: 372–390.
  • 加載中
圖(10) / 表(2)
計量
  • 文章訪問數(shù):  3345
  • HTML全文瀏覽量:  1486
  • PDF下載量:  105
  • 被引次數(shù): 0
出版歷程
  • 收稿日期:  2018-12-28
  • 修回日期:  2019-04-10
  • 網(wǎng)絡(luò)出版日期:  2019-05-21
  • 刊出日期:  2019-11-01

目錄

    /

    返回文章
    返回