Thoughtviz: Visualizing Human Thoughts Using Generative Adversarial Network

Keywords

EEG; Generative Adversarial Networks; Image Generation

Abstract

Studying human brain signals has always gathered great attention from the scientific community. In Brain Computer Interface (BCI) research, for example, changes of brain signals in relation to specific tasks (e.g., thinking something) are detected and used to control machines. While extracting spatio-temporal cues from brain signals for classifying state of human mind is an explored path, decoding and visualizing brain states is new and futuristic. Following this latter direction, in this paper, we propose an approach that is able not only to read the mind, but also to decode and visualize human thoughts. More specifically, we analyze brain activity, recorded by an ElectroEncephaloGram (EEG), of a subject while thinking about a digit, character or an object and synthesize visually the thought item. To accomplish this, we leverage the recent progress of adversarial learning by devising a conditional Generative Adversarial Network (GAN), which takes, as input, encoded EEG signals and generates corresponding images. In addition, since collecting large EEG signals in not trivial, our GAN model allows for learning distributions with limited training data. Performance analysis carried out on three different datasets - brain signals of multiple subjects thinking digits, characters, and objects - show that our approach is able to effectively generate images from thoughts of a person. They also demonstrate that EEG signals encode explicitly cues from thoughts which can be effectively used for generating semantically relevant visualizations.

Publication Date

10-15-2018

Publication Title

MM 2018 - Proceedings of the 2018 ACM Multimedia Conference

Number of Pages

950-958

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1145/3240508.3240641

Socpus ID

85058207237 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85058207237

This document is currently not available here.

Share

COinS