Abstract
Research in the field of neural networks has shown advancement in the device technology and machine learning application platforms of use. Some of the major applications of neural network prominent in recent scenarios include image recognition, machine translation, text classification and object categorization. With these advancements, there is a need for more energy-efficient and low area overhead circuits in the hardware implementations. Previous works have concentrated primarily on CMOS technology-based implementations which can face challenges of high energy consumption, memory wall, and volatility complications for standby modes. We herein developed a low-power and area-efficient hardware implementation for Long Short-Term Memory (LSTM) networks as a type of Recurrent Neural Network (RNN). To achieve energy efficiency while maintaining comparable accuracy commensurate with the ideal case, the LSTM network herein uses Resistive Random-Access Memory (ReRAM) based synapses along with spin-based non-binary neurons. The proposed neuron has a novel activation mechanism that mimics the ideal hyperbolic tangent (tanh) and sigmoid activation functions with five levels of output accuracy. Using ideal, binary, and the proposed non-binary neurons, we investigated the performance of an LSTM network for name prediction dataset. The comparison of the results shows that our proposed neuron can achieve up to 85% accuracy and perplexity of 1.56, which attains performance similar to algorithmic expectations of near-ideal neurons. The simulations show that our proposed neuron achieves up to 34-fold improvement in energy efficiency and 2-fold area reduction compared to the CMOS-based non-binary designs.
Notes
If this is your thesis or dissertation, and want to learn how to access it or for more information about readership statistics, contact us at STARS@ucf.edu
Graduation Date
2021
Semester
Summer
Advisor
DeMara, Ronald
Degree
Master of Science in Computer Engineering (M.S.Cp.E.)
College
College of Engineering and Computer Science
Department
Electrical and Computer Engineering
Degree Program
Computer Engineering
Format
application/pdf
Identifier
CFE0008750;DP0025481
URL
https://purls.library.ucf.edu/go/DP0025481
Language
English
Release Date
August 2021
Length of Campus-only Access
None
Access Status
Masters Thesis (Open Access)
STARS Citation
Vangala, Meghana Reddy, "Long Short-Term Memory with Spin-Based Binary and Non-Binary Neurons" (2021). Electronic Theses and Dissertations, 2020-2023. 779.
https://stars.library.ucf.edu/etd2020/779