基于遷移權(quán)重的條件對(duì)抗領(lǐng)域適應(yīng)

王進(jìn); 王科; 閔子劍; 孫開(kāi)偉; 鄧欣

doi:10.11999/JEIT190115

基于遷移權(quán)重的條件對(duì)抗領(lǐng)域適應(yīng)

doi: 10.11999/JEIT190115

重慶郵電大學(xué)數(shù)據(jù)工程與可視計(jì)算重點(diǎn)實(shí)驗(yàn)室 ??重慶 ??400065

基金項(xiàng)目: 國(guó)家自然科學(xué)基金(61806033), 國(guó)家社會(huì)科學(xué)基金西部項(xiàng)目(18XGL013)

詳細(xì)信息

作者簡(jiǎn)介:
王進(jìn)：男，1979年生，教授，研究方向?yàn)闄C(jī)器學(xué)習(xí)、數(shù)據(jù)挖掘

王科：男，1993年生，碩士生，研究方向?yàn)闄C(jī)器學(xué)習(xí)

閔子劍：男，1995年生，碩士生，研究方向?yàn)闄C(jī)器學(xué)習(xí)

孫開(kāi)偉：男，1987年生，講師，研究方向?yàn)闄C(jī)器學(xué)習(xí)、數(shù)據(jù)挖掘

鄧欣：男，1981年生，副教授，研究方向?yàn)闄C(jī)器學(xué)習(xí)、認(rèn)知計(jì)算

通訊作者:
王進(jìn)　wangjin@cqupt.edu.cn

中圖分類(lèi)號(hào): TP391.41
計(jì)量
- 文章訪(fǎng)問(wèn)數(shù): 3770
- HTML全文瀏覽量: 1424
- PDF下載量: 93
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2019-02-27
- 修回日期: 2019-06-11
- 網(wǎng)絡(luò)出版日期: 2019-06-24
- 刊出日期: 2019-11-01

Transfer Weight Based Conditional Adversarial Domain Adaptation

Key Laboratory of Data Engineering and Visual Computing, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Nature Science Foundation of China(61806033), The National Social Science Foundation of China(18XGL013)

摘要

摘要: 針對(duì)條件對(duì)抗領(lǐng)域適應(yīng)(CDAN)方法未能充分挖掘樣本的可遷移性，仍然存在部分難以遷移的源域樣本擾亂目標(biāo)域數(shù)據(jù)分布的問(wèn)題，該文提出一種基于遷移權(quán)重的條件對(duì)抗領(lǐng)域適應(yīng)(TW-CDAN)方法。首先利用領(lǐng)域判別模型的判別結(jié)果作為衡量樣本遷移性能的主要度量指標(biāo)，使不同的樣本具有不同的遷移性能；其次將樣本的可遷移性作為權(quán)重應(yīng)用在分類(lèi)損失和最小熵?fù)p失上，旨在消除條件對(duì)抗領(lǐng)域適應(yīng)中難以遷移樣本對(duì)模型造成的影響；最后使用Office-31數(shù)據(jù)集的6個(gè)遷移任務(wù)和Office-Home數(shù)據(jù)集的12個(gè)遷移任務(wù)進(jìn)行了實(shí)驗(yàn)，該方法在14個(gè)遷移任務(wù)上取得了提升，在平均精度上分別提升1.4%和3.1%。
- 遷移學(xué)習(xí) /
- 領(lǐng)域適應(yīng) /
- 對(duì)抗學(xué)習(xí) /
- 遷移權(quán)重
Abstract: Considering the failure of the Conditional adversarial Domain AdaptatioN(CDAN) to fully utilize the sample transferability, which still struggle with some hard-to-transfer source samples disturbed the distribution of the target domain samples, a Transfer Weight based Conditional adversarial Domain AdaptatioN(TW-CDAN) is proposed. Firstly, the discriminant results in the domain discriminant model as the main factor are employed to measure the transfer performance. Then the weight is applied to class loss and minimum entropy loss. It is for eliminating the influence of hard-to-transfer samples of the model. Finally, experiments are carried out using the six domain adaptation tasks of the Office-31 dataset and the 12 domain adaptation tasks of the Office-Home dataset. The proposed method improves the 14 domain adaptation tasks and increases the average accuracy by 1.4% and 3.1% respectively.
- Transfer learning /
- Domain adaptation /
- Adversarial learning /
- Transfer Weight(TW)

HTML全文

圖 1 TW-CDAN模型結(jié)構(gòu)圖

下載: 全尺寸圖片幻燈片

圖 2 Office-Home數(shù)據(jù)集

下載: 全尺寸圖片幻燈片

圖 3 算法收斂性對(duì)比實(shí)驗(yàn)

下載: 全尺寸圖片幻燈片

圖 4 T-SNE特征可視化

下載: 全尺寸圖片幻燈片

表 1 Office-31數(shù)據(jù)集結(jié)果(使用平均精度進(jìn)行評(píng)價(jià))

方法	A→W	D→W	W→D	A→D	D→A	W→A	平均
ResNet-50^[17]	68.4	96.7	99.3	68.9	62.5	60.7	76.1
DAN^[7]	80.5	97.1	99.6	78.6	63.6	62.8	80.4
RTN^[8]	84.5	96.8	99.4	77.5	66.2	64.8	81.6
DANN^[10]	82.0	96.9	99.1	79.7	68.2	67.4	82.2
ADDA^[11]	86.2	96.2	98.4	77.8	69.5	68.9	82.9
JAN^[18]	85.4	97.4	99.8	84.7	68.6	70.0	84.3
GTA^[19]	89.5	97.9	99.8	87.7	72.8	71.4	86.5
CDAN^[13]	93.1	98.6	100.0	92.9	71.0	69.3	87.5
TW-CDAN	94.9	99.2	100.0	94.0	72.7	72.5	88.9

下載: 導(dǎo)出CSV

表 2 Office-Home數(shù)據(jù)集結(jié)果(使用平均精度進(jìn)行評(píng)價(jià))

方法	Ar→Cl	Ar→Pr	Ar→Rw	Cl→Ar	Cl→Pr	Cl→Rw	Pr→Ar	Pr→Cl	Pr→Rw	Rw→Ar	Rw→Cl	Rw→Pr	平均
ResNet-50^[17]	34.9	50.0	58.0	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
DAN^[7]	43.6	57.0	67.9	45.8	56.5	60.4	44.0	43.6	67.7	63.1	51.5	74.3	56.3
DANN^[10]	45.6	59.3	70.1	47.0	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
JAN^[18]	45.9	61.2	68.9	50.4	59.7	61.0	45.8	43.4	70.3	63.9	52.4	76.8	58.3
CDAN^[13]	50.6	65.9	73.4	55.7	62.7	64.2	51.8	49.1	74.5	68.2	56.9	80.7	62.8
TW-CDAN	48.8	71.1	76.7	61.6	68.9	70.2	60.4	46.6	77.9	71.3	55.4	81.9	65.9

下載: 導(dǎo)出CSV

表 3 不同遷移權(quán)重設(shè)置在Office-31數(shù)據(jù)集結(jié)果(使用平均精度進(jìn)行評(píng)價(jià))

方法	A→W	D→W	W→D	A→D	D→A	W→A	平均
CDAN^[13]	93.1	98.6	100.0	92.9	71.0	69.3	87.5
CDAN(S)	93.0	98.7	100.0	92.7	71.0	69.1	87.4
TW-CDAN(E)	93.7	98.8	100.0	93.4	71.5	71.3	88.1
TW-CDAN(C)	94.2	98.9	100.0	93.1	72.1	71.8	88.4
TW-CDAN	94.9	99.2	100.0	94.0	72.7	72.5	88.9

下載: 導(dǎo)出CSV

參考文獻(xiàn)(19)

YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks?[C]. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, 2014: 3320-3328.

PAN S J and YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359. doi: 10.1109/TKDE.2009.191

GEBRU T, HOFFMAN J, LI Feifei, et al. Fine-grained recognition in the wild: A multi-task domain adaptation approach[C]. Proceedings of IEEE International Conference on Computer Vision, Venice, Italy, 2017: 1358–1367.

GLOROT X, BORDES A, and BENGIO Y. Domain adaptation for large-scale sentiment classification: A deep learning approach[C]. Proceedings of the 28th International Conference on Machine Learning, Bellevue, USA, 2011: 513–520.

WANG Mei and DENG Weihong. Deep visual domain adaptation: A survey[J]. Neurocomputing, 2018, 312: 135–153. doi: 10.1016/j.neucom.2018.05.083

GRETTON A, BORGWARDT K, RASCH M, et al. A kernel method for the two-sample-problem[C]. Proceedings of the 19th Conference on Neural Information Processing Systems, Vancouver, Canada, 2007: 513–520.

LONG Mingsheng, CAO Yue, WANG Jianmin, et al. Learning transferable features with deep adaptation networks[C]. Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 2015: 97–105.

LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Deep transfer learning with joint adaptation networks[C]. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 2017: 2208–2217.

GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, 2014: 2672–2680.

GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The Journal of Machine Learning Research, 2016, 17(1): 2096–2030.

TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial discriminative domain adaptation[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2962–2971.

MIRZA M and OSINDERO S. Conditional generative adversarial nets[EB/OL]. https://arxiv.org/abs/1411.1784, 2014.

LONG Mingsheng, CAO Zhangjie, WANG Jianmin, et al. Conditional adversarial domain adaptation[C]. Proceedings of the 32nd Conference on Neural Information Processing Systems, Montréal, Canada, 2018: 1647–1657.

GRANDVALET Y and BENGIO Y. Semi-supervised learning by entropy minimization[C]. Proceedings of the 17th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2004: 529–536.

SAENKO K, KULIS B, FRITZ M, et al. Adapting visual category models to new domains[C]. Proceedings of the 11th European Conference on Computer Vision, Heraklion, Greece, 2010: 213–226.

VENKATESWARA H, EUSEBIO J, CHAKRABORTY S, et al. Deep hashing network for unsupervised domain adaptation[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5385–5394.

HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.

LONG Mingsheng, ZHU Han, WANG Jianmin, et al. Unsupervised domain adaptation with residual transfer networks[C]. Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, Spain, 2016: 136–144.

SANKARANARAYANAN S, BALAJI Y, CASTILLO C D, et al. Generate to adapt: Aligning domains using generative adversarial networks[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 8503–8512.

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪(fǎng)問(wèn)統(tǒng)計(jì)