Title

Generative Neural Networks For Multi-Task Life-Long Learning

Keywords

generative network topology; life-long learning; multi-task learning; neural networks

Abstract

Multi-task learning (MTL) is an established method of inducing bias in neural network learning. It has been used as a method of knowledge transfer in sequential learning tasks, and as a method of overcoming the problem of impoverished training data. A standard problem in neural networks is the establishment of the proper network structure for a given problem. This problem is compounded in the MTL case since the structure of the network can significantly impact the effectiveness of the MTL method. This problem is exacerbated further in the sequential or life-long learning case, as choosing structure a priori for an unknown number of tasks is not possible. In this paper, we explore the use of cascade correlation (CC) as a generative neural network algorithm to overcome these limitations in MTL. We show that generative neural networks using the CC algorithm can display the same MTL advantages while alleviating the need to choose the network structure a priori. We then incorporate the functional transfer from the MTL techniques with the generative structure of CC networks to build a dual network life-long learning system that does not require the network topology or the number of tasks to be predefined. © The British Computer Society 2013.

Publication Date

3-1-2014

Publication Title

Computer Journal

Volume

57

Issue

3

Number of Pages

427-450

Document Type

Article

Personal Identifier

scopus

DOI Link

https://doi.org/10.1093/comjnl/bxt120

Socpus ID

84897677545 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/84897677545

This document is currently not available here.

Share

COinS