Title
Constructive Learning Of Recurrent Neural Networks - Limitations Of Recurrent Casade Correlation And A Simple Solution
Abbreviated Journal Title
IEEE Trans. Neural Netw.
Keywords
FINITE; INDUCTION; AUTOMATA; NETS; Computer Science, Artificial Intelligence; Computer Science, Hardware &; Architecture; Computer Science, Theory & Methods; Engineering, ; Electrical & Electronic
Abstract
It is often difficult to predict the optimal neural network size for a particular application, Constructive or destructive methods that add or subtract neurons, layers, connections, etc, might offer a solution to this problem, We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities, It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata, We give a ''preliminary'' approach on how to get ground these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure, We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn.
Journal Title
Ieee Transactions on Neural Networks
Volume
6
Issue/Number
4
Publication Date
1-1-1995
Document Type
Article
DOI Link
Language
English
First Page
829
Last Page
836
WOS Identifier
ISSN
1045-9227
Recommended Citation
"Constructive Learning Of Recurrent Neural Networks - Limitations Of Recurrent Casade Correlation And A Simple Solution" (1995). Faculty Bibliography 1990s. 2995.
https://stars.library.ucf.edu/facultybib1990/2995
Comments
Authors: contact us about adding a copy of your work at STARS@ucf.edu