迁移学习及其在固体地球科学中的应用

林秋怡; 左仁广

doi:10.19509/j.cnki.dzkq.tb20230429

迁移学习及其在固体地球科学中的应用

doi: 10.19509/j.cnki.dzkq.tb20230429

林秋怡^a,,
左仁广^b, ,

a.
中国地质大学（武汉）资源学院
b.
中国地质大学（武汉）地质过程与矿产资源国家重点实验室，武汉 430074

基金项目: 湖北省创新群体项目（2023AFA001）；国家自然科学基金项目（42102332）

详细信息

作者简介:
林秋怡：E-mail：linqiuyi@cug.edu.cn

通讯作者:
E-mail：zrguang@cug.edu.cn

中图分类号: P628
计量
- 文章访问数: 228
- PDF下载量: 42
- 被引次数: 0
出版历程
- 收稿日期: 2023-07-22
- 录用日期: 2023-12-06
- 修回日期: 2023-11-20
- 网络出版日期: 2023-12-17

Transfer learning and its application in solid Earth geoscience

LIN Qiuyi^a
,,
ZUO Renguang^{b
, ,}

a.
China University of Geosciences (Wuhan) School of Earth Resources
b.
China University of Geosciences (Wuhan) State Key Laboratory of Geological Processes and Mineral Resources, China University of Geosciences (Wuhan), Wuhan 430074, China

More Information

Author Bio:
E-mail：linqiuyi@cug.edu.cn

Corresponding author: E-mail：zrguang@cug.edu.cn

摘要

摘要:
随着地球科学进入大数据时代，机器学习成为可发现数据复杂结构与模式的新兴工具。作为机器学习的一个重要子领域，深度学习通过构建多层隐含层的方式，层层递进地学习海量数据内在特征，可达到提高分类或预测效果等目的。然而机器学习模型往往需要海量数据作为支撑，从而限制了其在固体地球科学领域的广泛应用，迁移学习算法的引入为解决这一问题提供了新的方案。迁移学习可通过利用预先学习类似任务的知识来提高新任务的性能，将源域学习到的知识迁移到目标域，可以在一定程度上克服训练数据不足的问题。迁移学习算法为机器学习在固体地球科学领域的应用提供了新的思路。本文简要综述了迁移学习的基本概念和类别，通过分析迁移学习在固体地球科学中的典型应用案例，讨论了现有迁移学习方法在固体地球科学领域中面临的挑战。当前，迁移学习方法已经在岩石矿物自动识别与分类、地球化学异常识别等方面表现出较大潜力，其具备提高模型泛化性能、避免过拟合的能力，在固体地球科学领域具有广阔的应用前景。但目前迁移学习方法应用于固体地球科学领域的研究还相对较少，未来将持续针对源域数据集选择、迁移模型构建、负迁移评估及可解释性不足等问题开展更为深入的研究。
- 迁移学习 /
- 深度学习 /
- 固体地球科学 /
- 机器学习
Abstract: Significance
With the advent of big data in geoscience, machine learning has emerged as a powerful tool that are able to characterize intricate structures and patterns of data, thus rapidly gaining attention in solid Earth geoscience. As a crucial branch of machine learning domain, deep learning leverages large amounts of datasets to construct multilayer hidden layers, enhancing the classification or prediction performances. Nevertheless, one of the significant difficulties for machine learning models in geoscience is the scarcity of available data, which is limited in solid Earth studies. The advent of transfer learning has introduced a novel approach to address this challenge by using limited training data for effective applications.
Progress
As a typical machine learning technique, transfer learning enhances the performance of new tasks within limited data by utilizing preexisting knowledge from similar tasks through pretraining. By transferring knowledge from a source domain to a target domain, it can partially mitigate insufficient data availability so that prediction accuracy can be improved. This study provides an overview of transfer learning's basic concepts and categories, discussing challenges in current geoscience applications, and analyzing typical cases in solid Earth geosciences. Currently, deep transfer learning shows promising potential in automatic identification and of rocks-minerals classification and geochemical anomalies identification.
Conclusions and Prospects
Transfer learning holds considerable promise for enhancing model generalization performance and mitigating overfitting in solid Earth geosciences. However, some challenges still remain, such as identifying suitable source domains to supply relevant knowledge for target domains. Future research should be explored in terms of source domain dataset selection, transfer model construction, negative transfer assessment, and interpretability of transfer learning.
- transfer learning /
- deep learning /
- solid Earth geoscience /
- machine learning

HTML全文

图 1 传统机器学习(a)与迁移学习(b)比较^[14]

Figure 1. Comparison between traditional machine learning (a) and transfer learning (b)

下载: 全尺寸图片幻灯片

图 2 迁移学习的分类^[14]

Figure 2. Classification of transfer learning

下载: 全尺寸图片幻灯片

图 3 预训练模型作为特征提取器的策略

Figure 3. Strategy for using pretraining model as the feature extractor

下载: 全尺寸图片幻灯片

表 1 迁移学习方法的适用场景^[14]

Table 1. Application scenarios for transfer learning methods

方法	适用场景
基于实例	在域间分布差异较小时适用效果比较好
基于特征	在域间分布差异较大时，寻找域间可共享的特征
基于模型	在域间差异较小时使用效果较好；当域间差异较大时，需结合特征方法，将通用特征学习到的模型参数进行迁移，并微调其余部分
基于关系	侧重于样本间关系

下载: 导出CSV

表 2 深度迁移学习方法分类及基本思想^[23]

Table 2. Classification and basic idea for deep transfer learning methods

分类	基本思想
基于网络	利用大规模源域训练数据集训练网络以生成预训练网络，将基于源域预训练的网络迁移至针对目标域设计的新网络，最后对新网络进行调整以更新参数信息
基于实例	从源域中选择一些与目标域相似或相关的实例样本进行加权权重适应，并将其加入目标域训练集中来训练深度神经网络
基于映射	将源域和目标域的实例样本同时映射到一个新数据空间，在这个新的数据空间中2个域的实例样本具有更高的相似性，可被用于训练深度神经网络
基于对抗	在迁移过程中引入对抗的思想，将从源域和目标域中提取的特征同时输送至对抗网络，通过对抗网络的不断学习来选择可供迁移的特征

下载: 导出CSV

参考文献(71)

[1]	DUNJKO V,BRIEGEL H J. Machine learning & artificial intelligence in the quantum domain:A review of recent progress[J]. Reports on Progress in Physics,2018,81(7):74001. doi: 10.1088/1361-6633/aab406
[2]	BERGEN K J,JOHNSON P A,DE HOOP M V,et al. Machine learning for data-driven discovery in solid Earth geoscience[J]. Science,2019,363:1299.
[3]	SHI F,WANG J,SHI J,et al. Review of artificial intelligence techniques in imaging data acquisition,segmentation,and diagnosis for COVID-19[J]. IEEE Reviews in Biomedical Engineering,2021,14:4-15. doi: 10.1109/RBME.2020.2987975
[4]	杨文博,盖永浩,张文祥,等. 基于分数阶最优控制网络的复杂勘探随机噪声消减方法[J]. 地质科技通报,2023,42(2):214-222. YANG W B,GAI Y H,ZHANG W X,et al. Random noise reduction method of complex exploration based on a fractional optimal control network[J]. Bulletin of Geological Science and Technology,2023,42(2):214-222.
[5]	汤化明,郎增瑞,王宏铭,等. 基于深度学习的金属矿物镜下嵌布粒度检测方法[J]. 地质科技通报,2024,43(3):351-358. TANG H M,LANG Z R,WANG H M,et al. A method for detecting the dissemination size of metal minerals under the microscope based on deep learning[J]. Bulletin of Geological Science and Technology,2024,43(3):351-358. (in Chinese with English abstract
[6]	MIGNAN A,BROCCARDO M. One neuron versus deep learning in aftershock prediction[J]. Nature,2019,574:E1-E3. doi: 10.1038/s41586-019-1582-8
[7]	YANG X F,YE Y M,LI X T,et al. Hyperspectral image classification with deep learning models[J]. IEEE Transactions on Geoscience and Remote Sensing,2018,56(9):5408-5423. doi: 10.1109/TGRS.2018.2815613
[8]	ZHONG Z L,LI J,LUO Z M,et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing,2018,56(2):847-858.
[9]	XIONG Y H,ZUO R G,CARRANZA E. Mapping mineral prospectivity through big data analytics and a deep learning algorithm[J]. Ore Geology Reviews,2018,102:811-817. doi: 10.1016/j.oregeorev.2018.10.006
[10]	XIONG Y H,ZUO R G. Recognizing multivariate geochemical anomalies for mineral exploration by combining deep learning and one-class support vector machine[J]. Computers & Geosciences,2020,140:104484.
[11]	LI T,ZUO R G,ZHAO X F,et al. Mapping prospectivity for regolith-hosted REE deposits via convolutional neural network with generative adversarial network augmented data[J]. Ore Geology Reviews,2022,142:104693. doi: 10.1016/j.oregeorev.2022.104693
[12]	SERMANET P,EIGEN D,ZHANG X,et al. OverFeat:Integrated recognition,localization and detection using convolutional networks[J]. Eprint Arxiv,2013,4(3):34-45.
[13]	龙明盛. 迁移学习问题与方法研究[D]. 北京:清华大学,2014. LONG M S. Transfer learning:Problems and methods[D]. Beijing:Tsinghua University,2014 . (in Chinese with English abstract
[14]	PAN S J,YANG Q A. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359. doi: 10.1109/TKDE.2009.191
[15]	WEISS K,KHOSHGOFTAAR T M,WANG D D. A survey of transfer learning[J]. Journal of Big Data,2016,3(1):1-40. doi: 10.1186/s40537-016-0043-6
[16]	ZHUANG F Z,QI Z Y,DUAN K Y,et al. A comprehensive survey on transfer learning[J]. Proceedings of the IEEE,2021,109(1):43-76. doi: 10.1109/JPROC.2020.3004555
[17]	DAI W Y,YANG Q,XUE G R,et al. Boosting for transfer learning:Machine learning[C]//Anon. Proceedings of the Twenty-Fourth International Conference (ICML 2007). [S. l. ]:[S. n. ],2007.
[18]	SUGIYAMA M,NAKAJIMA S,KASHIMA H,et al. Direct importance estimation with model selection and its application to covariate shift adaptation[C]//Anon. Part of Advances in Neural Information Processing Systems 20 (NIPS 2007). [S. l. ]:[S. n. ] ,2007.
[19]	KULLBACK S,LEIBLER R A. On information and sufficiency[J]. The Annals of Mathematical Statistics,1951,22(1):79-86. doi: 10.1214/aoms/1177729694
[20]	BORGWARDT K M,GRETTON A,RASCH M J,et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics,2006,22(14):49-57. doi: 10.1093/bioinformatics/btl242
[21]	KONIDARIS G,SCHEIDWASSER I,BARTO A G. Transfer in reinforcement learning via shared features[J]. Journal of Machine Learning Research,2012,13:1333-1371.
[22]	ZUO H,ZHANG G Q,BEHBOOD V,et al. Feature spaces-based transfer learning[C]//Anon. 16th World Congress of the International-Fuzzy-Systems-Association (IFSA) / 9th Conference of the European-Society-for-Fuzzy-Logic-and-Technology (EUSFLAT). [S. l. ]:[S. n. ] ,2015.
[23]	TAN C Q,SUN F C,KONG T,et al. A survey on deep transfer learning[C]//Anon. 27th International Conference on Artificial Neural Networks (ICANN). Rhodes,Greece:[S. n. ] ,2018.
[24]	FRIEDMAN N,GETOOR L,KOLLER D,et al. Learning probabilistic relational models[M]. Berlin,Heidelberg:Springer,2000:1300-1309.
[25]	RICHARDSON M,DOMINGOS P. Markov logic networks[J]. Machine Learning,2006,62(1/2):107-136.
[26]	KUMARASWAMY R,RAMANAN N,ODOM P,et al. Interactive transfer learning in relational domains[J]. Künstliche Intelligenz,2020,34(2):181-192.
[27]	MIHALKOVA L,HUYNH T,MOONEY R J. Mapping and revising Markov logic networks for transfer learning[C]//Anon. National conference on Artificial Intelligence . [S. l. ]:AAAI Press,2007.
[28]	DAVIS J,DOMINGOS P. Deep transfer via second-order Markov logic[C]//Anon. International Conference on Machine Learning. [S. l. ]:ACM,2009.
[29]	DE FIGUEIREDO L F,PAES A,ZAVERUCHA G. Transfer learning for boosted relational dependency networks through genetic algorithm[C]//Anon. 30th International Conference on Inductive Logic Programming (ILP) Held as Part of the 1st International Joint Conference on Learning and Reasoning (IJCLR). [S. l. ]:[S. n. ] ,2022.
[30]	YOSINSKI J,CLUNE J,BENGIO Y,et al. How transferable are features in deep neural networks ?[C]//Anon. 28th Conference on Neural Information Processing Systems (NIPS). Montreal,Canada:[S. n. ] ,2014.
[31]	LONG M S,CAO Y,WANG J M,et al. Learning transferable features with deep adaptation networks[C]//Anon. 32nd International Conference on Machine Learning. [S. l. ]:[S. n. ] ,2015.
[32]	CAI Z C,PENG C L. A study on training fine-tuning of convolutional neural networks[C]//Anon. 13th International Conference on Knowledge and Smart Technology (KST). [S. l.]:Burapha Univ.,Fac Informat,Electr Network,2021.
[33]	RUSU A A,RABINOWITZ N C,DESJARDINS G,et al. Progressive neural networks[J]. arXiv Preprint arXiv, 2016:1606.04671.
[34]	XIA B J,WANG K,XU A D,et al. Intelligent fault diagnosis for bearings of industrial Robot joints under varying working conditions based on deep adversarial domain adaptation[J]. IEEE Transactions on Instrumentation and Measurement,2022,71:3508313.
[35]	HAN L,ZHAO Y Y,CHEN H N,et al. Advancing radar nowcasting through deep transfer learning[J]. IEEE Transactions on Geoscience and Remote Sensing,2022,60:4100609.
[36]	GANIN Y,USTINOVA E,AJAKAN H,et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research,2016,17(1):2030-2096.
[37]	YU C H,WANG J D,CHEN Y Q,et al. Transfer learning with dynamic adversarial adaptation network[C]//Anon. IEEE International Conference on Data Mining (ICDM). Beijing:[S. n. ] ,2019.
[38]	HOUSHMAND N,GOODFELLOW S,ESMAEILI K,et al. Rock type classification based on petrophysical,geochemical,and core imaging data using machine and deep learning techniques[J]. Applied Computing and Geosciences,2022,16:100104. doi: 10.1016/j.acags.2022.100104
[39]	XU Z H,MA W,LIN P,et al. Deep learning of rock images for intelligent lithology identification[J]. Computers & Geosciences,2021,154:104799.
[40]	DE LIMA R P,DUARTE D,NICHOLSON C,et al. Petrographic microfacies classification with deep convolutional neural networks[J]. Computers & Geosciences,2020,142:104481.
[41]	SINGH N,SINGH T N,TIWARY A,et al. Textural identification of basaltic rock mass using image processing and neural network[J]. Computational Geosciences,2010,14(2):301-310. doi: 10.1007/s10596-009-9154-x
[42]	张野,李明超,韩帅. 基于岩石图像深度学习的岩性自动识别与分类方法[J]. 岩石学报,2018,34(2):333-342. ZHANG Y,LI M C,HAN S. Automatic identification and classification in lithology based on deep learning in rock images[J]. Acta Petrologica Sinica,2018,34(2):333-342. (in Chinese with English abstract
[43]	POLAT Ö,POLAT A,EKICI T. Automatic classification of volcanic rocks from thin section images using transfer learning networks[J]. Neural Computing and Applications,2021,33(18):11531-11540. doi: 10.1007/s00521-021-05849-3
[44]	DE LIMA R P,DUARTE D. Pretraining convolutional neural networks for mudstone petrographic thin-section image classification[J]. Geosciences,2021,11(8):336. doi: 10.3390/geosciences11080336
[45]	FREIRE W M,AMARAL A M M M,COSTA Y M G. Gemstone classification using ConvNet with transfer learning and fine-tuning[C]//Anon. 29th International Conference on Systems,Signals and Image Processing (IWSSIP). [S. l. ]:[S. n. ] ,2022.
[46]	ZHANG Y,LI M C,HAN S,et al. Intelligent identification for rock-mineral microscopic images using ensemble machine learning algorithms[J]. Sensors (Basel),2019,19(18):3914. doi: 10.3390/s19183914
[47]	LIU C Z,LI M C,ZHANG Y,et al. An enhanced rock mineral recognition method integrating a deep learning model and clustering algorithm[J]. Minerals,2019,9(9):516. doi: 10.3390/min9090516
[48]	LI N,HAO H Z,GU Q,et al. A transfer learning method for automatic identification of sandstone microscopic images[J]. Computers & Geosciences,2017,103:111-121.
[49]	CHEN W H,SU L M,CHEN X Q,et al. Rock image classification using deep residual neural network with transfer learning[J]. Frontiers in Earth Science,2023,10:1079447. doi: 10.3389/feart.2022.1079447
[50]	翟明国,胡波. 矿产资源国家安全、国际争夺与国家战略之思考[J]. 地球科学与环境学报,2021,43(1):1-11. ZHAI M G,HU B. Thinking to state security,international competition and national strategy of mineral resource[J]. Journal of Earth Sciences and Environment,2021,43(1):1-11. (in Chinese with English abstract
[51]	左仁广. 基于深度学习的深层次矿化信息挖掘与集成[J]. 矿物岩石地球化学通报,2019,38(1):53-60. ZUO R G. Deep learning-based mining and integration of deep-level mineralization information[J]. Bulletin of Minerlogy,Petrology and Geochemistry,2019,38(1):53-60. (in Chinese with English abstract
[52]	ZUO R G. Mineral exploration using subtle or negative geochemical anomalies[J]. Journal of Earth Science,2021,32(2):439-454. doi: 10.1007/s12583-020-1079-2
[53]	ZUO R G,WANG J,XIONG Y H,et al. The processing methods of geochemical exploration data:Past,present,and future[J]. Applied Geochemistry,2021,132:105072. doi: 10.1016/j.apgeochem.2021.105072
[54]	ZUO R G,XIONG Y H,WANG J,et al. Deep learning and its application in geochemical mapping[J]. Earth-Science Reviews,2019,192:1-14. doi: 10.1016/j.earscirev.2019.02.023
[55]	ZUO R G. Machine learning of mineralization-related geochemical anomalies:A review of potential methods[J]. Natural Resources Research,2017,26(4):457-464. doi: 10.1007/s11053-017-9345-4
[56]	LI H,LI X H,YUAN F,et al. Convolutional neural network and transfer learning based mineral prospectivity modeling for geochemical exploration of Au mineralization within the Guandian-Zhangbaling area,Anhui Province,China[J]. Applied Geochemistry,2020,122:104747. doi: 10.1016/j.apgeochem.2020.104747
[57]	WU B C,LI X H,YUAN F,et al. Transfer learning and siamese neural network based identification of geochemical anomalies for mineral exploration:A case study from the Cu-Au deposit in the NW Junggar area of northern Xinjiang,China[J]. Journal of Geochemical Exploration,2022,232:106904. doi: 10.1016/j.gexplo.2021.106904
[58]	HUANG Y J,FENG Q,ZHANG W T,et al. Prediction of prospecting target based on selective transfer network[J]. Minerals,2022,12(9):1112.
[59]	LI S,LIU C,CHEN J P. Mineral prospecting prediction via transfer learning based on geological big data:A case study of Huayuan,Hunan,China[J]. Minerals,2023,13(4):504.
[60]	CHENG Q M. Mapping singularities with stream sediment geochemical data for prediction of undiscovered mineral deposits in Gejiu,Yunnan Province,China[J]. Ore Geology Reviews,2007,32(1):314-324.
[61]	ZHOU K,SUN T,LIU Y,et al. Prospectivity mapping of tungsten mineralization in southern Jiangxi Province using few-shot learning[J]. Minerals,2023,13(5):669. doi: 10.3390/min13050669
[62]	MUHAMMAD A U,ABBA S I. Transfer learning for streamflow forecasting using unguaged MOPEX basins data set[J]. Earth Science Informatics,2023,16:1241-1254. doi: 10.1007/s12145-023-00952-6
[63]	左仁广,彭勇,李童,等. 基于深度学习的地质找矿大数据挖掘与集成的挑战[J]. 地球科学,2021,46(1):350-358. ZUO R G,PENG Y,LI T,et al. Challenges of geological prospecting big data mining and integration using deep learning algorithms[J]. Earth Science,2021,46(1):350-358. (in Chinese with English abstract
[64]	CUNHA A,POCHET A,LOPES H,et al. Seismic fault detection in real data using transfer learning from a convolutional neural network pre-trained with synthetic seismic data[J]. Computers and Geosciences,2020,135:104344. doi: 10.1016/j.cageo.2019.104344
[65]	SOEKHOE D,VAN DER PUTTEN P,PLAAT A. On the impact of data set size in transfer learning using deep neural networks[C]//Anon. 15th International Symposium on Intelligent Data Analysis (IDA). [S. l. ]:[S. n. ] ,2016.
[66]	SHI B Z,ZHOU X L,QIN Z K,et al. Corn ear quality recognition based on DCGAN data enhancement and transfer learning[C]//Anon. 4th International Conference on Electronics,Communications and Control Engineering (ICECC). [S. l. ]:Electr Network,2021.
[67]	LIU S P,TIAN G H,XU Y. A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter[J]. Neurocomputing,2019,338:191-206. doi: 10.1016/j.neucom.2019.01.090
[68]	张倩,杨颖,刘刚,等. 融合数据增强与改进ResNet34的奶牛热红外图像乳腺炎检测[J]. 光谱学与光谱分析,2023,43(1):280-288. ZHANG Q,YANG Y,LIU G,et al. Detection of dairy cow mastitis from thermal images by data enhancement and improved ResNet34[J]. Spectroscopy and Spectral Analysis,2023,43(1):280-288. (in Chinese with English abstract
[69]	LI T,ZUO R G,XIONG Y H,et al. Random-drop data augmentation of deep convolutional neural network for mineral prospectivity mapping[J]. Nature Resources Research,2021,30(1):27-38. doi: 10.1007/s11053-020-09742-z
[70]	KOESHIDAYATULLAH A. Optimizing image-based deep learning for energy geoscience via an effortless end-to-end approach[J]. Journal of Petroleum Science and Engineering,2022,215:110681. doi: 10.1016/j.petrol.2022.110681
[71]	YIN B J,ZUO R G,XIONG Y H. Mineral prospectivity mapping via gated recurrent unit model[J]. Natural Resources Research,2022,31(4):2065-2079. doi: 10.1007/s11053-021-09979-2