Impact of learning rate and momentum factor in the performance of back-propagation neural network to identify internal dynamics of chaotic motion
Keywords:
Back-propagation, learning rate, momentum factor, neural network.Abstract
The utilization of back-propagation neural network in identification of internal dynamics of chaotic motion is found appropriate. However, during its training through Rumelhart algorithm, it is found that, a high learning rate ( ) leads to rapid learning but the weights may oscillate, while a lower value of ` ' leads to slower learning process in weight updating formula Momentum factor ( ) is to accelerate the convergence of error during the training in the equation and while transfer function sigmoid . It is the most complicated and experimental task to identify optimum value of ` ' and ` ' during the training. To identify optimum value of ` ' and ` ' , firstly the network is trained with 103 epochs under different values of ` ' in the close interval and At the convergence of initial weights and minimization of error (i.e., mean square error) process is found appropriate. Afterwards to find optimum value of , the network was trained again with = 0.3 (fixed) and with different values of in the close interval for 103 epochs. It was observed that the convergence of initial weights and minimization of error was appropriate with = 0.3 and = 0.9. On this optimum value of and the network was trained successfully from local minima of error = 1.67029292416874E-03 at 103 epochs to global minima of error = 4.99180426869658E-04 at 15 105 epochs. At the global minima, the network has exhibited excellent performance in identification of internal dynamics of chaotic motion and in prediction of future values by past recorded data series. These essentials are presented through this research paper.
References
Basu, S. Andharia, H. I. 1992. The Chaotic time series of Indian Monsoon rainfall and its prediction, Proceedings of the Indian Academy of Science 101: 27-34.
Chih-Hong, K., Chun-Fei, H. Hon-Son, D. 2011. Design of an adaptive self-organizing fuzzy neural network controller for uncertain nonlinear chaotic systems. Neural Computing and Applications 21: 1243-1253.
Enireddy, V., Varma, K.V.S.R.P., Sankara Rao, P. Satapati, R. 2010. Prediction of rainfall using backpropagation neural network model. International Journal on Computer Science and Engineering 2: 1119-1121.
Guhathakurta, P. 1998. A hybrid neural network model for long range prediction of all india summer monsoon rainfall. Proceedings of WMO international workshop on dynamical extended range forecasting, Toulouse, France.
Guhathakurta, P., Rajeevan, M. Thapliyal, V. 1999. Long Range Forecasting Indian Summer Monsoon Rainfall by Hybrid Principal Component Neural Network Model. Meteorology and Atmospheric Physics 71: 255-266.
Guhathakurta, P. 2000. New models for long range forecasts of summer monsoon rainfall over north west and peninsular India. Meteorology and Atmospheric Physics 73: 211-255.
Guhathakurta, P. 2006. Long-Range Monsoon Rainfall Prediction of 2005 for the Districts and Sub-Division Kerala with Artificial Neural Network. Current Science 90: 773-779.
Karmakar, S., Kowar, M. K. Guhathakurta, P. 2009. Artificial neural network skeleton in deterministic forecast to recognize pattern of TMRF Ambikapur District, CSVTU Research Journal 2: 41-45.
Karmakar, S., Kowar, M. K. Guhathakurta, P. 2012. Application of neural network in long range weather forecasting: In the context of smaller geographical region (i.e.Chhattisgarh State, India). Lambert Academic Publishing, Germany. pp. 57-87.
Kowar, M. K., Karmakar, S. Guhathakurta, P. 2013. Development of backpropagation neural network for prediction of chaotic data time series. Logos Verlag Berlin, Germany.
Krishnamurthy, V. Kinter, J. L. 2003. The Indian monsoon and its relation to global climate variability. Global Climate - Current Research and Uncertainties in the Climate System (eds Rodo, X. and Comin, F. A.). pp. 186-236.
Krishnamurthy, V. Kirtman, B. P. 2003. Variability of the Indian Ocean: Relation to Monsoon and ENSO. Quarterly Journal of the Royal Meteorological Society 129: 1623-1646.
Kumar, S. 2007. Neural network computer engineering series. The McGraw-Hill, New Delhi, pp. 104-152.
Lorenz, E. 1996. The essence of chaos. University of Washington Press.
Lorenz, E. 1972. Predictability: Does the flap of a butterfly's wings in Brazil set off a Tornado in Texas. American Association for the Advancement of Science, Washington D.C.
Mohammed, A. 2010. Chaotic time series prediction using wavelet neural network. Journal of Artificial Intelligence 1:73-80.
Patil, C. Y. Ghatol, A. A. 2010. Rainfall forecasting using local parameters over a meteorological station: An Artificial Neural Network Approach. Journal of Engineering Research and Industrial Applications 3: 341-356.
Phillip, H. S. 2003. DTREG Predictive modeling software, available online on http://www.dtreg.com. pp. 175-181.
Rajeevan, M. 2001. Prediction of Indian summer monsoon: Status, problems and prospects. Current Science 81: 1451-1457.
Rajeevan, M., Pai, D. S., Dikshit, S. K. Kelkar, R. R. 2004. IMD's new operational models for long-range forecast of southwest monsoon rainfall over India and Their verification for 2003. Current Science 86: 422-431.
Rumelhart, D., Hinton, G. E. Williams, R. J. 1986. Learning internal representation by error propagation. r j parallel distributed processing: Exploration in the Microstructure of Cognition, MIT Press, Cambridge. pp. 318-362
Sahai, A. K., Grimm, A. M., Satyan. V. Pant, G. B. 2002. Prospects of prediction of indian summer monsoon rainfall using global sst anomalies. IITM Research Report No. RR-093.
Sawaitul, S.D., Wagh, K. P. Chatur, P. N. 2012. Classification and prediction of future weather by using back propagation algorithm-an approach, International Journal of Emerging Technology and Advanced Engineering 2: 111-115.
Sivanandam, S. N., Sumathi, S. Deepa, S. N. 2006. Introduction to neural networks using MATLAB 6.0. The McGraw Hill Companies, Delhi. pp. 189-192
Thapliyal, V. Kulshrestha, S. M. 1992. Recent models for long range forecasting of southwest monsoon rainfall over India. Mausam 43: 239-248.
Thapliyal, V. 1997. Preliminary and final long range forecasts for seasonal monsoon rainfall over India. Journal of Arid Environment 36: 385-403.
Thapliyal, V. Rajeevan, M. 2003. Updated operational models for long-range forecasts of Indian summer monsoon rainfall. Mausam 54: 495-504.
