Constructive Learning Of Recurrent Neural Networks - Limitations Of Recurrent Casade Correlation And A Simple Solution

    Authors

    C. L. Giles; D. Chen; G. Z. Sun; H. H. Chen; Y. C. Lee;M. W. Goudreau

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    IEEE Trans. Neural Netw.

    Keywords

    FINITE; INDUCTION; AUTOMATA; NETS; Computer Science, Artificial Intelligence; Computer Science, Hardware &; Architecture; Computer Science, Theory & Methods; Engineering, ; Electrical & Electronic

    Abstract

    It is often difficult to predict the optimal neural network size for a particular application, Constructive or destructive methods that add or subtract neurons, layers, connections, etc, might offer a solution to this problem, We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities, It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata, We give a ''preliminary'' approach on how to get ground these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure, We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn.

    Journal Title

    Ieee Transactions on Neural Networks

    Volume

    6

    Issue/Number

    4

    Publication Date

    1-1-1995

    Document Type

    Article

    Language

    English

    First Page

    829

    Last Page

    836

    WOS Identifier

    WOS:A1995RF58200003

    ISSN

    1045-9227

    Share

    COinS