Title

Constructive Learning Of Recurrent Neural Networks - Limitations Of Recurrent Casade Correlation And A Simple Solution

Authors

Authors

C. L. Giles; D. Chen; G. Z. Sun; H. H. Chen; Y. C. Lee;M. W. Goudreau

Comments

Authors: contact us about adding a copy of your work at STARS@ucf.edu

Abbreviated Journal Title

IEEE Trans. Neural Netw.

Keywords

FINITE; INDUCTION; AUTOMATA; NETS; Computer Science, Artificial Intelligence; Computer Science, Hardware &; Architecture; Computer Science, Theory & Methods; Engineering, ; Electrical & Electronic

Abstract

It is often difficult to predict the optimal neural network size for a particular application, Constructive or destructive methods that add or subtract neurons, layers, connections, etc, might offer a solution to this problem, We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities, It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata, We give a ''preliminary'' approach on how to get ground these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure, We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn.

Journal Title

Ieee Transactions on Neural Networks

Volume

6

Issue/Number

4

Publication Date

1-1-1995

Document Type

Article

Language

English

First Page

829

Last Page

836

WOS Identifier

WOS:A1995RF58200003

ISSN

1045-9227

Share

COinS