Title

Constructive Learning Of Recurrent Neural Networks: Limitations Of Recurrent Casade Correlation And A Simple Solution

Abstract

It is often difficult to predict the optimal neural network size for a particular application. Constructive or destructive methods that add or subtract neurons, layers, connections, etc. might offer a solution to this problem. We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities. It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata. We give a “preliminary" approach on how to get around these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure. We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn. © 1995 IEEE

Publication Date

1-1-1995

Publication Title

IEEE Transactions on Neural Networks

Volume

6

Issue

4

Number of Pages

829-836

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/72.392247

Socpus ID

0029341578 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/0029341578

This document is currently not available here.

Share

COinS