Exploiting Symmetries Of Distributions In Cnns And Folded Coding

Keywords

Activation Functions; Coding; Convolutional Neural Networks; Distribution Estimation; Symmetry

Abstract

We introduce the concept of Folded Coding' for continuous univariate distributions estimating the distribution and coding the samples simultaneously. Folded Coding assumes symmetries in the distribution and requires significantly fewer parameters compared to conventional models when the symmetry assumption is satisfied. We incorporate the mechanics of Folded Coding into Convolutional Neural Networks (CNN) in the form of layers referred to as Binary Expanded ReLU (BEReLU) Shared Convolutions and Instance Fully Connected (I-FC). BEReLU and I-FC force the network to have symmetric functionality in the space of samples. Therefore similar patterns of prediction are applied to sections of the space where the model does not have observed samples. We experimented with BEReLU on generic networks using different parameter sizes on CIFAR-10 and CIFAR-100. Our experiments show increased accuracy of the models equipped with the BEReLU layer when there are fewer parameters. The performance of the models with BEReLU layer remains similar to original network with the increase of parameter number. The experiments provide further evidence that estimation of distribution symmetry is part of CNNs' functionality.

Publication Date

12-13-2018

Publication Title

Proceedings - 2018 15th Conference on Computer and Robot Vision, CRV 2018

Number of Pages

47-54

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1109/CRV.2018.00017

Socpus ID

85060526937 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85060526937

This document is currently not available here.

Share

COinS