Title

How Novelty Search Escapes The Deceptive Trap Of Learning To Learn

Keywords

Adaptation; Learning; Neat; Neural networks; Neuroevolution; Neuromodulation; Novelty search

Abstract

A major goal for researchers in neuroevolution is to evolve artificial neural networks (ANNs) that can learn during their lifetime. Such networks can adapt to changes in their environment that evolution on its own cannot anticipate. However, a profound problem with evolving adaptive systems is that if the impact of learning on the fitness of the agent is only marginal, then evolution is likely to produce individuals that do not exhibit the desired adaptive behavior. Instead, because it is easier at first to improve fitness without evolving the ability to learn, they are likely to exploit domain-dependent static (i.e. non-adaptive) heuristics. This paper proposes a way to escape the deceptive trap of static policies based on the novelty search algorithm, which opens up a new avenue in the evolution of adaptive systems because it can exploit the behavioral difference between learning and non-learning individuals. The main idea in novelty search is to abandon objective-based fitness and instead simply search only for novel behavior, which avoids deception entirely and has shown prior promising results in other domains. This paper shows that novelty search significantly outperforms fitness-based search in a tunably deceptive T-Maze navigation domain because it fosters the emergence of adaptive behavior. Copyright 2009 ACM.

Publication Date

12-31-2009

Publication Title

Proceedings of the 11th Annual Genetic and Evolutionary Computation Conference, GECCO-2009

Number of Pages

153-160

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1145/1569901.1569923

Socpus ID

72749096047 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/72749096047

This document is currently not available here.

Share

COinS