Keywords
Least squares minimization, RTRL, Recurrent neural networks
Abstract
The thesis discusses a novel off-line and on-line learning approach for Fully Recurrent Neural Networks (FRNNs). The most popular algorithm for training FRNNs, the Real Time Recurrent Learning (RTRL) algorithm, employs the gradient descent technique for finding the optimum weight vectors in the recurrent neural network. Within the framework of the research presented, a new off-line and on-line variation of RTRL is presented, that is based on the Gauss-Newton method. The method itself is an approximate Newton's method tailored to the specific optimization problem, (non-linear least squares), which aims to speed up the process of FRNN training. The new approach stands as a robust and effective compromise between the original gradient-based RTRL (low computational complexity, slow convergence) and Newton-based variants of RTRL (high computational complexity, fast convergence). By gathering information over time in order to form Gauss-Newton search vectors, the new learning algorithm, GN-RTRL, is capable of converging faster to a better quality solution than the original algorithm. Experimental results reflect these qualities of GN-RTRL, as well as the fact that GN-RTRL may have in practice lower computational cost in comparison, again, to the original RTRL.
Notes
If this is your thesis or dissertation, and want to learn how to access it or for more information about readership statistics, contact us at STARS@ucf.edu
Graduation Date
2004
Semester
Summer
Advisor
Georgiopoulos, Michael
Degree
Master of Science (M.S.)
College
College of Engineering and Computer Science
Department
Electrical and Computer Engineering
Degree Program
Electrical and Computer Engineering
Format
application/pdf
Identifier
CFE0000091
URL
http://purl.fcla.edu/fcla/etd/CFE0000091
Language
English
Release Date
August 2004
Length of Campus-only Access
None
Access Status
Masters Thesis (Open Access)
Subjects
Dissertations, Academic -- Engineering and Computer Science; Engineering and Computer Science -- Dissertations, Academic
STARS Citation
Vartak, Aniket Arun, "Gauss-newton Based Learning For Fully Recurrent Neural Networks" (2004). Electronic Theses and Dissertations. 154.
https://stars.library.ucf.edu/etd/154