基于有序編碼的核極限學習順序回歸模型

李佩佳; 石勇; 汪華東; 牛凌峰

doi:10.11999/JEIT170765

基于有序編碼的核極限學習順序回歸模型

doi: 10.11999/JEIT170765

基金項目:

國家自然科學基金(71110107026, 71331005, 91546201, 11671379, 111331012)，中國科學院大學資助項目(Y55202LY00)

計量
- 文章訪問數(shù): 1584
- HTML全文瀏覽量: 167
- PDF下載量: 173
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2017-07-28
- 修回日期: 2018-01-22
- 刊出日期: 2018-06-19

Ordered Code-based Kernel Extreme Learning Machine for Ordinal Regression

Funds:

The National Natural Science Foundation of China (71110107026, 71331005, 91546201, 11671379, 111331012), The Grant of University of Chinese Academy of Sciences (Y55202LY00)

摘要

摘要: 順序回歸是機器學習領域中介于分類和回歸之間的有監(jiān)督問題。在實際中，許多帶有序關系標簽的問題都可以被建模成順序回歸問題，因此順序回歸受到眾多學者的關注?；跇O限學習機(ELM)的算法能有效避免因迭代過程陷入的局部最優(yōu)解，減少訓練時間，但基于極限學習機的算法在順序回歸問題上的研究較少。該文將核極限學習機與糾錯輸出編碼相結合，提出了一種基于有序編碼的核極限學習順序回歸模型。該模型有效解決了如何在順序回歸中取得良好的特征映射以及如何避免傳統(tǒng)極限學習機中隱層節(jié)點個數(shù)依賴于人工設置的問題。為驗證提出模型的有效性，該文在多個順序回歸數(shù)據(jù)集上進行了測試，測試結果表明，相比于傳統(tǒng)ELM模型，該文提出的模型在準確率上平均提升了10.8%，在數(shù)據(jù)集上預測表現(xiàn)最優(yōu)，而且獲得了最短的訓練時間，從而驗證了模型的有效性。
- 糾錯輸出編碼 /
- 順序回歸 /
- 極限學習機 /
- 核函數(shù)
Abstract: Ordinal regression is one of the supervised learning issues, which resides between classification and regression in machine learning fields. There exist many real problems in practice, which can be modeled as ordinal regression problems due to the ordering information between labels. Therefore ordinal regression has received increasing interest by many researchers recently. The Extreme Learning Machine (ELM)-based algorithms are easy to train without iterative algorithm and they can avoid the local optimal solution; meanwhile they reduce the training time compared with other learning algorithms. However, the ELM-based algorithms which are applied to ordinal regression have not been exploited much. This paper proposes a new ordered code-based kernel extreme learning ordinal regression machine to fill this gap, which combines the kernel ELM and error correcting output codes effectively. The model overcomes the problems of how to get high quality feature mappings in ordinal regression and how to avoid setting the number of hidden nodes by manual. To validate the effectiveness of this model, numerous experiments are conducted on a lot of datasets. The experimental results show that the model can improve the accuracy by 10.8% on average compared with traditional ELM-based algorithms and achieve the state- of-the-art performance with the least time.
- Error correct output code /
- Ordinal regression /
- Extreme learning machine /
- Kernel function

HTML全文

參考文獻(30)

NAKOV P, RITTER A, ROSENTHAL S, et al. SemEval- 2016 task 4: Sentiment analysis in Twitter[C]. International Workshop on Semantic Evaluation, San Diego, USA, 2016: 1-18. doi: 10.18653/v1/S16-1028.

TIAN Q, CHEN S, and TAN X. Comparative study among three strategies of incorporating spatial structures to ordinal image regression[J]. Neurocomputing, 2014, 136: 152-161. doi: 10.1016/j.neucom.2014.01.017.

CORRENTE S, DOUMPOS M, GRECO S, et al. Multiple criteria hierarchy process for sorting problems based on ordinal regression with additive value functions[J]. Annals of Operations Research, 2017, 251(1/2): 117-139. doi: 10.1007/ s10479-015-1898-1.

GUTIRREZ P A, PREZ-ORTIZ M, SANCHEZ- MONEDERO J, et al. Ordinal regression methods: Survey and experimental study[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(1): 127-146. doi: 10.1109/ TKDE.2015.2457911.

HUANG G B, ZHU Q Y, and SIEW C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1): 489-501. doi: 10.1016/j.neucom.2005.12.126.

RAJASEKARAN S and PAI G A V. Neural Networks, Fuzzy Systems and Evolutionary Algorithms: Synthesis and Applications[M]. Haryana, India: Rajkamal Electric Press, 2017: 151-168.

CHU W and KEERTHI S S. Support vector ordinal regression[J]. Neural Computation, 2007, 19(3): 792-815. doi: 10.1162/neco.2007.19.3.792.

HUANG G B, ZHOU H, DING X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2): 513-529. doi: 10.1109/TSMCB. 2011.2168604.

UCAR A, DEMIR Y, and GZELI C. A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering[J]. Neural Computing and Applications, 2016, 27(1): 131-142. doi: 10.1007/s00521-014-1569-1.

徐濤, 郭威, 呂宗磊. 基于快速極限學習機和差分進化的機場噪聲預測模型[J]. 電子與信息學報, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT150986.

XU Tao, GUO Wei, and L Zonglei. Prediction model of airport noise based on fast extreme learning machine and differential evolution[J]. Journal of Electronics Information Technology, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT 150986.

GOODFELLOW I, BENGIO Y, and COURVILLE A. Deep Learning[M]. Massachusetts, USA, MIT Press, 2016: 165-480. doi: 10.1038/nature14539.

DENG W Y, ZHENG Q H, LIAN S, et al. Ordinal extreme learning machine[J]. Neurocomputing, 2010, 74(1): 447-456. doi: 10.1016/j.neucom.2010.08.022.

RICCARDI A, FERNNDEZ-NAVARRO F, and CARLONI S. Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine[J]. IEEE Transactions on Cybernetics, 2014, 44(10): 1898-1909. doi: 10.1109/TCYB. 2014.2299291.

HORNIK K, STINCHCOMBE M, and WHITE H. Multilayer feedforward networks are universal approximators[J]. Neural Networks, 1989, 2(5): 359-366. doi: 10.1016/0893-6080(89) 90020-8.

HUANG G B and BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE Transactions on Neural Networks, 1998, 9(1): 224-229. doi: 10.1109/72.655045.

HUANG G B, CHEN L, and SIEW C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Transactions on Neural Networks, 2006, 17(4): 879-892. doi: 10.1109/TNN. 2006.875977.

HUANG G B. Learning capability and storage capacity of two-hidden-layer feedforward networks[J]. IEEE Transactions on Neural Networks, 2003, 14(2): 274-281. doi: 10.1109/TNN.2003.809401.

BARTLETT P L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network[J]. IEEE Transactions on Information Theory, 1998, 44(2): 525-536. doi: 10.1109/18.661502.

TANG J, DENG C, and HUANG G B. Extreme learning machine for multilayer perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821. doi: 10.1109/TNNLS.2015.2424995.

HOERL A E and KENNARD R W. Ridge regression: Biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1): 55-67. doi: 10.1080/00401706.1970.10488634.

ALLWEIN E L, SCHAPIRE R E, and SINGER Y. Reducing multiclass to binary: A unifying approach for margin classifiers[J]. Journal of Machine Learning Research, 2000, 1(12): 113-141. doi: 10.1162/15324430152733133.

雷蕾, 王曉丹, 羅璽, 等. ECOC多類分類研究綜述[J]. 電子學報, 2014, 42(9): 1794-1800. doi: 10.3969/j.issn.0372-2112. 2014.09.020.

LEI Lei, WANG Xiaodan, LUO Xi, et al. An overview of multi-classification based on error-correcting output codes[J]. Acta Electronica Sinica, 2014, 42(9): 1794-1800. doi: 10.3969 /j.issn.0372-2112.2014.09.020.

HUANG G, HUANG G B, SONG S, et al. Trends in extreme learning machines: A review[J]. Neural Networks, 2015, 61: 32-48. doi: 10.1016/j.neunet.2014.10.001.

LIU Q, HE Q, and SHI Z. Extreme support vector machine classifier[C]. 12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Osaka, Japan, 2008: 222-233. doi: 10.1007/978-3-540-68125-0_21.

FRNAY B and VERLEYSEN M. Using SVMs with randomised feature spaces: an extreme learning approach[C]. European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, 2010: 315-320.

HUANG G B, DING X, and ZHOU H. Optimization method based extreme learning machine for classification[J]. Neurocomputing, 2010, 74(1): 155-163. doi: 10.1016/j.neucom. 2010.02.019.

CHU W and GHAHRAMANI Z. Gaussian processes for ordinal regression[J]. Journal of Machine Learning Research, 2005, 6(7): 1019-1041.

BACCIANELLA S, ESULI A, and SEBASTIANI F. Evaluation measures for ordinal regression[C]. The Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 2009: 283-287. doi: 10.1109/ISDA. 2009.230.

施引文獻

資源附件(0)

訪問統(tǒng)計