| Citation: | LIN Qiuyi，ZUO Renguang. Transfer learning and its application in solid Earth geoscience[J]. Bulletin of Geological Science and Technology，2025，44（1）：346-356 doi: 10.19509/j.cnki.dzkq.tb20230429
|
With the advent of big data in geoscience, machine learning has emerged as a powerful tool that are able to characterize intricate structures and patterns of data, thus rapidly gaining attention in solid Earth geoscience. As a crucial branch of machine learning domain, deep learning leverages large amounts of datasets to construct multilayer hidden layers, enhancing the classification or prediction performances. Nevertheless, one of the significant difficulties for machine learning models in geoscience is the scarcity of available data, which is limited in solid Earth studies. The advent of transfer learning has introduced a novel approach to address this challenge by using limited training data for effective applications.
As a typical machine learning technique, transfer learning enhances the performance of new tasks within limited data by utilizing preexisting knowledge from similar tasks through pretraining. By transferring knowledge from a source domain to a target domain, it can partially mitigate insufficient data availability so that prediction accuracy can be improved. This study provides an overview of transfer learning's basic concepts and categories, discussing challenges in current geoscience applications, and analyzing typical cases in solid Earth geosciences. Currently, deep transfer learning shows promising potential in automatic identification and of rocks-minerals classification and geochemical anomalies identification.
Transfer learning holds considerable promise for enhancing model generalization performance and mitigating overfitting in solid Earth geosciences. However, some challenges still remain, such as identifying suitable source domains to supply relevant knowledge for target domains. Future research should be explored in terms of source domain dataset selection, transfer model construction, negative transfer assessment, and interpretability of transfer learning.
| [1] |
DUNJKO V,BRIEGEL H J. Machine learning & artificial intelligence in the quantum domain:A review of recent progress[J]. Reports on Progress in Physics,2018,81(7):74001. doi: 10.1088/1361-6633/aab406
|
| [2] |
BERGEN K J,JOHNSON P A,DE HOOP M V,et al. Machine learning for data-driven discovery in solid Earth geoscience[J]. Science,2019,363:1299.
|
| [3] |
SHI F,WANG J,SHI J,et al. Review of artificial intelligence techniques in imaging data acquisition,segmentation,and diagnosis for COVID-19[J]. IEEE Reviews in Biomedical Engineering,2021,14:4-15. doi: 10.1109/RBME.2020.2987975
|
| [4] |
杨文博,盖永浩,张文祥,等. 基于分数阶最优控制网络的复杂勘探随机噪声消减方法[J]. 地质科技通报,2023,42(2):214-222.
YANG W B,GAI Y H,ZHANG W X,et al. Random noise reduction method of complex exploration based on a fractional optimal control network[J]. Bulletin of Geological Science and Technology,2023,42(2):214-222.
|
| [5] |
汤化明,郎增瑞,王宏铭,等. 基于深度学习的金属矿物镜下嵌布粒度检测方法[J]. 地质科技通报,2024,43(3):351-358.
TANG H M,LANG Z R,WANG H M,et al. A method for detecting the dissemination size of metal minerals under the microscope based on deep learning[J]. Bulletin of Geological Science and Technology,2024,43(3):351-358. (in Chinese with English abstract
|
| [6] |
MIGNAN A,BROCCARDO M. One neuron versus deep learning in aftershock prediction[J]. Nature,2019,574:E1-E3. doi: 10.1038/s41586-019-1582-8
|
| [7] |
YANG X F,YE Y M,LI X T,et al. Hyperspectral image classification with deep learning models[J]. IEEE Transactions on Geoscience and Remote Sensing,2018,56(9):5408-5423. doi: 10.1109/TGRS.2018.2815613
|
| [8] |
ZHONG Z L,LI J,LUO Z M,et al. Spectral-spatial residual network for hyperspectral image classification:A 3-D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing,2018,56(2):847-858.
|
| [9] |
XIONG Y H,ZUO R G,CARRANZA E. Mapping mineral prospectivity through big data analytics and a deep learning algorithm[J]. Ore Geology Reviews,2018,102:811-817. doi: 10.1016/j.oregeorev.2018.10.006
|
| [10] |
XIONG Y H,ZUO R G. Recognizing multivariate geochemical anomalies for mineral exploration by combining deep learning and one-class support vector machine[J]. Computers & Geosciences,2020,140:104484.
|
| [11] |
LI T,ZUO R G,ZHAO X F,et al. Mapping prospectivity for regolith-hosted REE deposits via convolutional neural network with generative adversarial network augmented data[J]. Ore Geology Reviews,2022,142:104693. doi: 10.1016/j.oregeorev.2022.104693
|
| [12] |
SERMANET P,EIGEN D,ZHANG X,et al. OverFeat:Integrated recognition,localization and detection using convolutional networks[J]. Eprint Arxiv,2013,4(3):34-45.
|
| [13] |
龙明盛. 迁移学习问题与方法研究[D]. 北京:清华大学,2014.
LONG M S. Transfer learning:Problems and methods[D]. Beijing:Tsinghua University,2014 . (in Chinese with English abstract
|
| [14] |
PAN S J,YANG Q A. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359. doi: 10.1109/TKDE.2009.191
|
| [15] |
WEISS K,KHOSHGOFTAAR T M,WANG D D. A survey of transfer learning[J]. Journal of Big Data,2016,3(1):1-40. doi: 10.1186/s40537-016-0043-6
|
| [16] |
ZHUANG F Z,QI Z Y,DUAN K Y,et al. A comprehensive survey on transfer learning[J]. Proceedings of the IEEE,2021,109(1):43-76. doi: 10.1109/JPROC.2020.3004555
|
| [17] |
DAI W Y,YANG Q,XUE G R,et al. Boosting for transfer learning:Machine learning[C]//Anon. Proceedings of the Twenty-Fourth International Conference (ICML 2007). [S. l. ]:[S. n. ],2007.
|
| [18] |
SUGIYAMA M,NAKAJIMA S,KASHIMA H,et al. Direct importance estimation with model selection and its application to covariate shift adaptation[C]//Anon. Part of Advances in Neural Information Processing Systems 20 (NIPS 2007). [S. l. ]:[S. n. ] ,2007.
|
| [19] |
KULLBACK S,LEIBLER R A. On information and sufficiency[J]. The Annals of Mathematical Statistics,1951,22(1):79-86. doi: 10.1214/aoms/1177729694
|
| [20] |
BORGWARDT K M,GRETTON A,RASCH M J,et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics,2006,22(14):49-57. doi: 10.1093/bioinformatics/btl242
|
| [21] |
KONIDARIS G,SCHEIDWASSER I,BARTO A G. Transfer in reinforcement learning via shared features[J]. Journal of Machine Learning Research,2012,13:1333-1371.
|
| [22] |
ZUO H,ZHANG G Q,BEHBOOD V,et al. Feature spaces-based transfer learning[C]//Anon. 16th World Congress of the International-Fuzzy-Systems-Association (IFSA) / 9th Conference of the European-Society-for-Fuzzy-Logic-and-Technology (EUSFLAT). [S. l. ]:[S. n. ] ,2015.
|
| [23] |
TAN C Q,SUN F C,KONG T,et al. A survey on deep transfer learning[C]//Anon. 27th International Conference on Artificial Neural Networks (ICANN). Rhodes,Greece:[S. n. ] ,2018.
|
| [24] |
FRIEDMAN N,GETOOR L,KOLLER D,et al. Learning probabilistic relational models[M]. Berlin,Heidelberg:Springer,2000:1300-1309.
|
| [25] |
RICHARDSON M,DOMINGOS P. Markov logic networks[J]. Machine Learning,2006,62(1/2):107-136.
|
| [26] |
KUMARASWAMY R,RAMANAN N,ODOM P,et al. Interactive transfer learning in relational domains[J]. Künstliche Intelligenz,2020,34(2):181-192.
|
| [27] |
MIHALKOVA L,HUYNH T,MOONEY R J. Mapping and revising Markov logic networks for transfer learning[C]//Anon. National conference on Artificial Intelligence . [S. l. ]:AAAI Press,2007.
|
| [28] |
DAVIS J,DOMINGOS P. Deep transfer via second-order Markov logic[C]//Anon. International Conference on Machine Learning. [S. l. ]:ACM,2009.
|
| [29] |
DE FIGUEIREDO L F,PAES A,ZAVERUCHA G. Transfer learning for boosted relational dependency networks through genetic algorithm[C]//Anon. 30th International Conference on Inductive Logic Programming (ILP) Held as Part of the 1st International Joint Conference on Learning and Reasoning (IJCLR). [S. l. ]:[S. n. ] ,2022.
|
| [30] |
YOSINSKI J,CLUNE J,BENGIO Y,et al. How transferable are features in deep neural networks ?[C]//Anon. 28th Conference on Neural Information Processing Systems (NIPS). Montreal,Canada:[S. n. ] ,2014.
|
| [31] |
LONG M S,CAO Y,WANG J M,et al. Learning transferable features with deep adaptation networks[C]//Anon. 32nd International Conference on Machine Learning. [S. l. ]:[S. n. ] ,2015.
|
| [32] |
CAI Z C,PENG C L. A study on training fine-tuning of convolutional neural networks[C]//Anon. 13th International Conference on Knowledge and Smart Technology (KST). [S. l.]:Burapha Univ.,Fac Informat,Electr Network,2021.
|
| [33] |
RUSU A A,RABINOWITZ N C,DESJARDINS G,et al. Progressive neural networks[J]. arXiv Preprint arXiv, 2016:1606.04671.
|
| [34] |
XIA B J,WANG K,XU A D,et al. Intelligent fault diagnosis for bearings of industrial Robot joints under varying working conditions based on deep adversarial domain adaptation[J]. IEEE Transactions on Instrumentation and Measurement,2022,71:3508313.
|
| [35] |
HAN L,ZHAO Y Y,CHEN H N,et al. Advancing radar nowcasting through deep transfer learning[J]. IEEE Transactions on Geoscience and Remote Sensing,2022,60:4100609.
|
| [36] |
GANIN Y,USTINOVA E,AJAKAN H,et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research,2016,17(1):2030-2096.
|
| [37] |
YU C H,WANG J D,CHEN Y Q,et al. Transfer learning with dynamic adversarial adaptation network[C]//Anon. IEEE International Conference on Data Mining (ICDM). Beijing:[S. n. ] ,2019.
|
| [38] |
HOUSHMAND N,GOODFELLOW S,ESMAEILI K,et al. Rock type classification based on petrophysical,geochemical,and core imaging data using machine and deep learning techniques[J]. Applied Computing and Geosciences,2022,16:100104. doi: 10.1016/j.acags.2022.100104
|
| [39] |
XU Z H,MA W,LIN P,et al. Deep learning of rock images for intelligent lithology identification[J]. Computers & Geosciences,2021,154:104799.
|
| [40] |
DE LIMA R P,DUARTE D,NICHOLSON C,et al. Petrographic microfacies classification with deep convolutional neural networks[J]. Computers & Geosciences,2020,142:104481.
|
| [41] |
SINGH N,SINGH T N,TIWARY A,et al. Textural identification of basaltic rock mass using image processing and neural network[J]. Computational Geosciences,2010,14(2):301-310. doi: 10.1007/s10596-009-9154-x
|
| [42] |
张野,李明超,韩帅. 基于岩石图像深度学习的岩性自动识别与分类方法[J]. 岩石学报,2018,34(2):333-342.
ZHANG Y,LI M C,HAN S. Automatic identification and classification in lithology based on deep learning in rock images[J]. Acta Petrologica Sinica,2018,34(2):333-342. (in Chinese with English abstract
|
| [43] |
POLAT Ö,POLAT A,EKICI T. Automatic classification of volcanic rocks from thin section images using transfer learning networks[J]. Neural Computing and Applications,2021,33(18):11531-11540. doi: 10.1007/s00521-021-05849-3
|
| [44] |
DE LIMA R P,DUARTE D. Pretraining convolutional neural networks for mudstone petrographic thin-section image classification[J]. Geosciences,2021,11(8):336. doi: 10.3390/geosciences11080336
|
| [45] |
FREIRE W M,AMARAL A M M M,COSTA Y M G. Gemstone classification using ConvNet with transfer learning and fine-tuning[C]//Anon. 29th International Conference on Systems,Signals and Image Processing (IWSSIP). [S. l. ]:[S. n. ] ,2022.
|
| [46] |
ZHANG Y,LI M C,HAN S,et al. Intelligent identification for rock-mineral microscopic images using ensemble machine learning algorithms[J]. Sensors (Basel),2019,19(18):3914. doi: 10.3390/s19183914
|
| [47] |
LIU C Z,LI M C,ZHANG Y,et al. An enhanced rock mineral recognition method integrating a deep learning model and clustering algorithm[J]. Minerals,2019,9(9):516. doi: 10.3390/min9090516
|
| [48] |
LI N,HAO H Z,GU Q,et al. A transfer learning method for automatic identification of sandstone microscopic images[J]. Computers & Geosciences,2017,103:111-121.
|
| [49] |
CHEN W H,SU L M,CHEN X Q,et al. Rock image classification using deep residual neural network with transfer learning[J]. Frontiers in Earth Science,2023,10:1079447. doi: 10.3389/feart.2022.1079447
|
| [50] |
翟明国,胡波. 矿产资源国家安全、国际争夺与国家战略之思考[J]. 地球科学与环境学报,2021,43(1):1-11.
ZHAI M G,HU B. Thinking to state security,international competition and national strategy of mineral resource[J]. Journal of Earth Sciences and Environment,2021,43(1):1-11. (in Chinese with English abstract
|
| [51] |
左仁广. 基于深度学习的深层次矿化信息挖掘与集成[J]. 矿物岩石地球化学通报,2019,38(1):53-60.
ZUO R G. Deep learning-based mining and integration of deep-level mineralization information[J]. Bulletin of Minerlogy,Petrology and Geochemistry,2019,38(1):53-60. (in Chinese with English abstract
|
| [52] |
ZUO R G. Mineral exploration using subtle or negative geochemical anomalies[J]. Journal of Earth Science,2021,32(2):439-454. doi: 10.1007/s12583-020-1079-2
|
| [53] |
ZUO R G,WANG J,XIONG Y H,et al. The processing methods of geochemical exploration data:Past,present,and future[J]. Applied Geochemistry,2021,132:105072. doi: 10.1016/j.apgeochem.2021.105072
|
| [54] |
ZUO R G,XIONG Y H,WANG J,et al. Deep learning and its application in geochemical mapping[J]. Earth-Science Reviews,2019,192:1-14. doi: 10.1016/j.earscirev.2019.02.023
|
| [55] |
ZUO R G. Machine learning of mineralization-related geochemical anomalies:A review of potential methods[J]. Natural Resources Research,2017,26(4):457-464. doi: 10.1007/s11053-017-9345-4
|
| [56] |
LI H,LI X H,YUAN F,et al. Convolutional neural network and transfer learning based mineral prospectivity modeling for geochemical exploration of Au mineralization within the Guandian-Zhangbaling area,Anhui Province,China[J]. Applied Geochemistry,2020,122:104747. doi: 10.1016/j.apgeochem.2020.104747
|
| [57] |
WU B C,LI X H,YUAN F,et al. Transfer learning and siamese neural network based identification of geochemical anomalies for mineral exploration:A case study from the Cu-Au deposit in the NW Junggar area of northern Xinjiang,China[J]. Journal of Geochemical Exploration,2022,232:106904. doi: 10.1016/j.gexplo.2021.106904
|
| [58] |
HUANG Y J,FENG Q,ZHANG W T,et al. Prediction of prospecting target based on selective transfer network[J]. Minerals,2022,12(9):1112.
|
| [59] |
LI S,LIU C,CHEN J P. Mineral prospecting prediction via transfer learning based on geological big data:A case study of Huayuan,Hunan,China[J]. Minerals,2023,13(4):504.
|
| [60] |
CHENG Q M. Mapping singularities with stream sediment geochemical data for prediction of undiscovered mineral deposits in Gejiu,Yunnan Province,China[J]. Ore Geology Reviews,2007,32(1):314-324.
|
| [61] |
ZHOU K,SUN T,LIU Y,et al. Prospectivity mapping of tungsten mineralization in southern Jiangxi Province using few-shot learning[J]. Minerals,2023,13(5):669. doi: 10.3390/min13050669
|
| [62] |
MUHAMMAD A U,ABBA S I. Transfer learning for streamflow forecasting using unguaged MOPEX basins data set[J]. Earth Science Informatics,2023,16:1241-1254. doi: 10.1007/s12145-023-00952-6
|
| [63] |
左仁广,彭勇,李童,等. 基于深度学习的地质找矿大数据挖掘与集成的挑战[J]. 地球科学,2021,46(1):350-358.
ZUO R G,PENG Y,LI T,et al. Challenges of geological prospecting big data mining and integration using deep learning algorithms[J]. Earth Science,2021,46(1):350-358. (in Chinese with English abstract
|
| [64] |
CUNHA A,POCHET A,LOPES H,et al. Seismic fault detection in real data using transfer learning from a convolutional neural network pre-trained with synthetic seismic data[J]. Computers and Geosciences,2020,135:104344. doi: 10.1016/j.cageo.2019.104344
|
| [65] |
SOEKHOE D,VAN DER PUTTEN P,PLAAT A. On the impact of data set size in transfer learning using deep neural networks[C]//Anon. 15th International Symposium on Intelligent Data Analysis (IDA). [S. l. ]:[S. n. ] ,2016.
|
| [66] |
SHI B Z,ZHOU X L,QIN Z K,et al. Corn ear quality recognition based on DCGAN data enhancement and transfer learning[C]//Anon. 4th International Conference on Electronics,Communications and Control Engineering (ICECC). [S. l. ]:Electr Network,2021.
|
| [67] |
LIU S P,TIAN G H,XU Y. A novel scene classification model combining ResNet based transfer learning and data augmentation with a filter[J]. Neurocomputing,2019,338:191-206. doi: 10.1016/j.neucom.2019.01.090
|
| [68] |
张倩,杨颖,刘刚,等. 融合数据增强与改进ResNet34的奶牛热红外图像乳腺炎检测[J]. 光谱学与光谱分析,2023,43(1):280-288.
ZHANG Q,YANG Y,LIU G,et al. Detection of dairy cow mastitis from thermal images by data enhancement and improved ResNet34[J]. Spectroscopy and Spectral Analysis,2023,43(1):280-288. (in Chinese with English abstract
|
| [69] |
LI T,ZUO R G,XIONG Y H,et al. Random-drop data augmentation of deep convolutional neural network for mineral prospectivity mapping[J]. Nature Resources Research,2021,30(1):27-38. doi: 10.1007/s11053-020-09742-z
|
| [70] |
KOESHIDAYATULLAH A. Optimizing image-based deep learning for energy geoscience via an effortless end-to-end approach[J]. Journal of Petroleum Science and Engineering,2022,215:110681. doi: 10.1016/j.petrol.2022.110681
|
| [71] |
YIN B J,ZUO R G,XIONG Y H. Mineral prospectivity mapping via gated recurrent unit model[J]. Natural Resources Research,2022,31(4):2065-2079. doi: 10.1007/s11053-021-09979-2
|
Copyright © 2010Editorial Department of Bulletin of Geological Science and Technology
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
