Generative Neural Networks for Multi-task Life-Long Learning

    Authors

    J. Reeder;M. Georgiopoulos

    Comments

    Authors: contact us about adding a copy of your work at STARS@ucf.edu

    Abbreviated Journal Title

    Comput. J.

    Keywords

    multi-task learning; neural networks; generative network topology; life-long learning; RECOGNITION; CLASSIFICATION; KNOWLEDGE; ARTMAP; MEMORY; Computer Science, Hardware & Architecture; Computer Science, Information; Systems; Computer Science, Software Engineering; Computer Science, ; Theory & Methods

    Abstract

    Multi-task learning (MTL) is an established method of inducing bias in neural network learning. It has been used as a method of knowledge transfer in sequential learning tasks, and as a method of overcoming the problem of impoverished training data. A standard problem in neural networks is the establishment of the proper network structure for a given problem. This problem is compounded in the MTL case since the structure of the network can significantly impact the effectiveness of the MTL method. This problem is exacerbated further in the sequential or life-long learning case, as choosing structure a priori for an unknown number of tasks is not possible. In this paper, we explore the use of cascade correlation (CC) as a generative neural network algorithm to overcome these limitations in MTL. We show that generative neural networks using the CC algorithm can display the same MTL advantages while alleviating the need to choose the network structure a priori. We then incorporate the functional transfer from the MTL techniques with the generative structure of CC networks to build a dual network life-long learning system that does not require the network topology or the number of tasks to be predefined.

    Journal Title

    Computer Journal

    Volume

    57

    Issue/Number

    3

    Publication Date

    1-1-2014

    Document Type

    Article

    Language

    English

    First Page

    427

    Last Page

    450

    WOS Identifier

    WOS:000332331700006

    ISSN

    0010-4620

    Share

    COinS