On-line Gauss-Newton-based learning for fully recurrent neural networks

    Authors

    A. A. Vartak; M. Georgiopoulos;G. C. Anagnostopoulos

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    Nonlinear Anal.-Theory Methods Appl.

    Keywords

    Mathematics, Applied; Mathematics

    Abstract

    In this paper we propose a novel, Gauss-Newton-based variant of the Real Time Recurrent Learning (RTRL) algorithm by Williams and Zipser (Neural Comput. 1 (1989) 270-280) for on-line training of Fully Recurrent Neural Networks. The new approach stands as a robust and effective compromise between the original, gradient-based RTRL (low computational complexity, slow convergence) and Newton-based variants of RTRL (high computational complexity, fast convergence). By gathering information over time in order to form Gauss-Newton search vectors, the new learning algorithm, GN-RTRL, is capable of converging faster to a better quality solution than the original algorithm. Experimental results reflect these qualities of GN-RTRL, as well as the fact that GN-RTRL may have in practice lower computational cost in comparison, again, to the original RTRL. (C) 2005 Elsevier Ltd. All rights reserved.

    Journal Title

    Nonlinear Analysis-Theory Methods & Applications

    Volume

    63

    Issue/Number

    5-7

    Publication Date

    1-1-2005

    Document Type

    Article

    Language

    English

    First Page

    E867

    Last Page

    E876

    WOS Identifier

    WOS:000208147800087

    ISSN

    0362-546X

    Share

    COinS