储备池算法与动力系统分析研究进展

高健; 颜子翔; 肖井华

doi:10.12202/j.0476-0301.2023197

储备池算法与动力系统分析研究进展

doi: 10.12202/j.0476-0301.2023197

高健¹,
颜子翔¹,
肖井华^{1, 2, ,}

1.
北京邮电大学理学院，100876，北京
2.
北京邮电大学信息光子学与光通信国家重点实验室，100876，北京

基金项目: 国家自然科学基金资助项目(11775034)；中央高校基本科研业务费专项资助(2022RC26,2019XD-A10)

详细信息

通讯作者:
肖井华（1965—），男，博士，教授. 研究方向：复杂系统与统计物理. E-mail：jhxiao@bupt.edu.cn

中图分类号: O415.6
计量
- 文章访问数: 63
- HTML全文浏览量: 28
- PDF下载量: 13
- 被引次数: 0
出版历程
- 收稿日期: 2023-08-14
- 网络出版日期: 2023-11-14

Reservoir computing and Dynamical systems：A Survey

GAO Jian¹,
YAN Zixiang¹,
XIAO Jinghua^{1, 2
, ,}

1.
School of Science，Beijing University of Posts and Telecommunications，100876，Beijing，China
2.
State Key Lab of Information Photonics and Optical Communications，Beijing University of Posts and Telecommunications，100876，Beijing，China

摘要

摘要: 储备池算法因为其简约的算法结构，灵活多变的算法实现方式，以及同时具有非线性和记忆性等特点，在时间序列和动力系统相关问题的研究中被广泛应用．同时，经过训练的储备池本身形成了一个复杂动力系统．以储备池为桥梁，基于动力系统理论的分析对于探索学习过程的智能现象起到了重要作用．本文对于储备池算法的最新研究进展做了回顾与展望，梳理出逐渐形成的“用于动力系统的储备池算法”与“储备池算法的动力系统基础”两个相辅相成的研究方向．本文将有助于研究人员加深对以储备池为代表的人工智能算法的理解，促进人工智能、复杂系统及统计物理相关学科跨学科研究和交流．
- 人工神经网络 /
- 储备池算法 /
- 动力系统 /
- 非线性动力学
Abstract: With the development of artificial intelligence, science exploration based on artificial intelligence has gradually become the forefront of technological advancement. Simultaneously, the mathematical foundation of intelligent phenomena has also emerged as a crucial scientific issue requiring urgent resolution behind the rapid development of artificial intelligence. Among numerous artificial intelligence algorithms, the reservoir computing method due to its simple structure, flexible and diverse implementation methods, as well as characteristics such as nonlinearity and memory, has been widely applied in the research of time series and dynamical system-related problems. Besides, the trained reservoir itself forms a complex dynamical system. Serving as a bridge, the analysis based on dynamical system theory through the reservoir plays an important role in exploring the intelligent phenomena of the learning process. This paper reviews and prospects the latest research progress of the reservoir computing algorithm, outlining two complementary research directions as the reservoir computing for dynamical systems, and dynamical System Foundations of reservoir computing. This paper will contribute to deepening researchers' understanding of artificial intelligence algorithms, promoting interdisciplinary research and communication among artificial intelligence, complex systems, and statistical physics-related disciplines.
- artificial neural network /
- reservoir computing /
- dynamical system /
- nonlinear dynamics.

HTML全文

图 1 储备池算法结构示意图

下载: 全尺寸图片幻灯片

参考文献(51)

[1]	WANG H C,FU T F,DU Y Q,et al. Scientific discovery in the age of artificial intelligence[J]. Nature,2023,620(7972):47 doi: 10.1038/s41586-023-06221-2
[2]	LEIJNEN S,VAN VEEN F. The neural network zoo[J]. Proceedings,2020,47(1):9 doi: 10.3390/proceedings2020047009
[3]	LUKOŠEVIČIUS M,JAEGER H. Reservoir computing approaches to recurrent neural network training[J]. Computer Science Review,2009,3(3):127 doi: 10.1016/j.cosrev.2009.03.005
[4]	ZHANG H,VARGAS D V. A survey on reservoir computing and its interdisciplinary applications beyond traditional machine learning[J]. IEEE Access,2023,11:81033 doi: 10.1109/ACCESS.2023.3299296
[5]	MAASS W,NATSCHLÄGER T,MARKRAM H. Real-time computing without stable states:a new framework for neural computation based on perturbations[J]. Neural Computation,2002,14(11):2531 doi: 10.1162/089976602760407955
[6]	JAEGER H. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note[J]. Bonn,Germany:German National Research Center for Information Technology GMD Technical Report,2001,148(34):13.
[7]	VERSTRAETEN D,SCHRAUWEN B,D’HAENE M,et al. An experimental unification of reservoir computing methods[J]. Neural Networks,2007,20(3):391 doi: 10.1016/j.neunet.2007.04.003
[8]	SCHRAUWEN B,VERSTRAETEN D,CAMPENHOUT J V. An overview of reservoir computing:theory,applications and implementations[C]//In Proceedings of the 15th european symposium on artificial neural networks,Bruges,Belgium:Université catholique de Louvain 2007
[9]	CHATTOPADHYAY A,HASSANZADEH P,SUBRAMANIAN D. Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods:reservoir computing,artificial neural network,and long short-term memory network[J]. Nonlinear Processes in Geophysics,2020,27(3):373 doi: 10.5194/npg-27-373-2020
[10]	TANAKA G,YAMANE T,HÉROUX J B,et al. Recent advances in physical reservoir computing:a review[J]. Neural Networks,2019,115:100 doi: 10.1016/j.neunet.2019.03.005
[11]	MANTE V,SUSSILLO D,SHENOY K V,et al. Context-dependent computation by recurrent dynamics in prefrontal cortex[J]. Nature,2013,503(7474):78 doi: 10.1038/nature12742
[12]	BARONE P,JOSEPH J P. Prefrontal cortex and spatial sequencing in macaque monkey[J]. Experimental Brain Research,1989,78(3):447
[13]	RIGOTTI M,BARAK O,WARDEN M R,et al. The importance of mixed selectivity in complex cognitive tasks[J]. Nature,2013,497(7451):585 doi: 10.1038/nature12160
[14]	ENEL P,PROCYK E,QUILODRAN R,et al. Reservoir computing properties of neural dynamics in prefrontal cortex[J]. PLoS Computational Biology,2016,12(6):e1004967 doi: 10.1371/journal.pcbi.1004967
[15]	SUSSILLO D,ABBOTT L F. Generating coherent patterns of activity from chaotic neural networks[J]. Neuron,2009,63(4):544 doi: 10.1016/j.neuron.2009.07.018
[16]	ZHONG S S,XIE X L,LIN L,et al. Genetic algorithm optimized double-reservoir echo state network for multi-regime time series prediction[J]. Neurocomputing,2017,238:191 doi: 10.1016/j.neucom.2017.01.053
[17]	THIEDE L A,PARLITZ U. Gradient based hyperparameter optimization in echo state networks[J]. Neural Networks,2019,115:23 doi: 10.1016/j.neunet.2019.02.001
[18]	WANG H S,YAN X F. Optimizing the echo state network with a binary particle swarm optimization algorithm[J]. Knowledge-Based Systems,2015,86:182 doi: 10.1016/j.knosys.2015.06.003
[19]	GRIGORYEVA L,ORTEGA J P. Echo state networks are universal[J]. Neural Networks,2018(108):495
[20]	JAEGER H. Tutorial on training recurrent neural networks,covering BPPT,RTRL,EKF and the" echo state network" approach[EB/OL]. 2021 [2023-08-11]. https://api.semanticscholar.org/CorpusID:192593367
[21]	LUKOŠEVIČIUS M. A practical guide to applying echo state networks[M]//Lecture Notes in Computer Science. Berlin,Heidelberg:Springer Berlin Heidelberg,2012:659
[22]	GALLICCHIO C. Chasing the echo state property[EB/OL]. 2018[2023-08-11] https://arxiv.org/abs/1811.10892.pdf
[23]	LIU S Y,XIAO J H,YAN Z X,et al. Noise resistance of next-generation reservoir computing:a comparative study with high-order correlation computation[J]. Nonlinear Dynamics,2023,111(15):14295 doi: 10.1007/s11071-023-08592-7
[24]	SCHRAUWEN B,DEFOUR J,VERSTRAETEN D,et al. The introduction of time-scales in reservoir computing,applied to isolated digits recognition[M]//Lecture Notes in Computer Science. Berlin,Heidelberg:Springer Berlin Heidelberg,2007:471
[25]	LUKOŠEVIČIUS M,JAEGER H,SCHRAUWEN B. Reservoir computing trends[J]. KI-Künstliche Intelligenz,2012,26(4):365
[26]	GRIFFITH A,POMERANCE A,GAUTHIER D J. Forecasting chaotic systems with very low connectivity reservoir computers[J]. Chaos:an Interdisciplinary Journal of Nonlinear Science,2019,29(12):123108. doi: 10.1063/1.5120710
[27]	HALUSZCZYNSKI A,RÄTH C. Good and bad predictions:assessing and improving the replication of chaotic attractors by means of reservoir computing[J]. Chaos:an Interdisciplinary Journal of Nonlinear Science,2019,29(10):103143. doi: 10.1063/1.5118725
[28]	CHITSAZAN M A,SAMI FADALI M,TRZYNADLOWSKI A M. Wind speed and wind direction forecasting using echo state network with nonlinear functions[J]. Renewable Energy,2019,131:879 doi: 10.1016/j.renene.2018.07.060
[29]	WANG S,YANG X J,WEI C J. Harnessing non-linearity by sigmoid-wavelet hybrid echo state networks (SWHESN)[C]//2006 6th World Congress on Intelligent Control and Automation. June 21-23,2006,Dalian,China. IEEE,2006:3014
[30]	DALE M,MILLER J,STEPNEY S,et al. A substrate-independent framework to characterize reservoir computers[J]. Proceedings of the Royal Society A,2019,475(2226):20180723 doi: 10.1098/rspa.2018.0723
[31]	BALA A,ISMAIL I,IBRAHIM R,et al. Applications of metaheuristics in reservoir computing techniques:a review[J]. IEEE Access,2018,6:58012 doi: 10.1109/ACCESS.2018.2873770
[32]	LAN X,CHEN W,GAO J,et al. Dynamical Analysis of Reservoir Computing:From Single to Few Nodes[EB/OL],(2023-03-28) [2023-08-11],中国科技论文在线, http://www.paper.edu.cn/releasepaper/content/202303-327
[33]	JAIDEEP P,BRIAN H,MICHELLE G,et al. Model-free prediction of large spatiotemporally chaotic systems from data:a reservoir computing approach[J]. Physical Review Letters,2018,120(2):024102 doi: 10.1103/PhysRevLett.120.024102
[34]	ZIMMERMANN R S,PARLITZ U. Observing spatio-temporal dynamics of excitable media using reservoir computing[J]. Chaos:an Interdisciplinary Journal of Nonlinear Science,2018,28(4):043118 doi: 10.1063/1.5022276
[35]	FAN H W,JIANG J J,ZHANG C,et al. Long-term prediction of chaotic systems with machine learning[J]. Physical Review Research,2020,2:012080 doi: 10.1103/PhysRevResearch.2.012080
[36]	GAUTHIER D J,BOLLT E,GRIFFITH A,et al. Next generation reservoir computing[J]. Nature Communications,2021,12:5564 doi: 10.1038/s41467-021-25801-2
[37]	PATHAK J,LU Z X,HUNT B R,et al. Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data[J]. Chaos,2017,27(12):121102 doi: 10.1063/1.5010300
[38]	FOLLMANN R,ROSA E. Predicting slow and fast neuronal dynamics with machine learning[J]. Chaos,2019,29(11):113119. doi: 10.1063/1.5119723
[39]	KONG L W,FAN H W,GREBOGI C,et al. Machine learning prediction of critical transition and system collapse[J]. Physical Review Research,2021,3:013090 doi: 10.1103/PhysRevResearch.3.013090
[40]	FAN H W,KONG L W,LAI Y C,et al. Anticipating synchronization with machine learning[J]. Physical Review Research,2021,3(2):023237 doi: 10.1103/PhysRevResearch.3.023237
[41]	ZHANG H,FAN H W,WANG L,et al. Learning Hamiltonian dynamics with reservoir computing[J]. Physical Review E,2021,104:024205
[42]	KIM J Z,LU Z X,NOZARI E,et al. Teaching recurrent neural networks to infer global temporal structure from local examples[J]. Nature Machine Intelligence,2021,3(4):316 doi: 10.1038/s42256-021-00321-2
[43]	PIOTR A,MARVYN G,JAËL P,et al. Using a reservoir computer to learn chaotic attractors,with applications to chaos synchronization and cryptography[J]. Physical Review E,2018,98:012215 doi: 10.1103/PhysRevE.98.012215
[44]	WENG T F,YANG H J,GU C G,et al. Synchronization of chaotic systems and their machine-learning models[J]. Physical Review E,2019,99:042203 doi: 10.1103/PhysRevE.99.042203
[45]	GUO Y L,ZHANG H,WANG L A,et al. Transfer learning of chaotic systems[J]. Chaos:an Interdisciplinary Journal of Nonlinear Science,2021,31(1):011104. doi: 10.1063/5.0033870
[46]	KONG L,WENG Y,GLAZ B,et al. Reservoir computing as digital twins for nonlinear dynamical systems[J]. Chaos,2023,33(3):033111. doi: 10.1063/5.0138661
[47]	LU Z X,HUNT B R,OTT E. Attractor reconstruction by machine learning[J]. Chaos:an Interdisciplinary Journal of Nonlinear Science,2018,28(6):061104. doi: 10.1063/1.5039508
[48]	CHEN W,GAO J A,YAN Z X,et al. Proper choice of hyperparameters in reservoir computing of chaotic maps[J]. Journal of Physics A:Mathematical and Theoretical,2023,56(41):415702 doi: 10.1088/1751-8121/acfb54
[49]	LEGENSTEIN R,MAASS W. Edge of chaos and prediction of computational performance for neural circuit models[J]. Neural Networks,2007,20(3):323 doi: 10.1016/j.neunet.2007.04.017
[50]	YILDIZ I B,JAEGER H,KIEBEL S J. Re-visiting the echo state property[J]. Neural Networks,2012,35:1 doi: 10.1016/j.neunet.2012.07.005
[51]	WANG L A,FAN H W,XIAO J H,et al. Criticality in reservoir computer of coupled phase oscillators[J]. Physical Review E,2022,105(5):L052201 doi: 10.1103/PhysRevE.105.L052201