Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy
Neuromorphic computing embraces the “device history” offered by many analog non-volatile memory (NVM) devices to implement the small weight changes computed by a gradient-descent learning algorithm such as backpropagation. Deterministic and stochastic imperfections in the condu...
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
IEEE
2018
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/36efdf8a930c4875b9b68848f34662b6 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:36efdf8a930c4875b9b68848f34662b6 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:36efdf8a930c4875b9b68848f34662b62021-11-19T00:00:36ZBidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy2168-673410.1109/JEDS.2017.2782184https://doaj.org/article/36efdf8a930c4875b9b68848f34662b62018-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/8171732/https://doaj.org/toc/2168-6734Neuromorphic computing embraces the “device history” offered by many analog non-volatile memory (NVM) devices to implement the small weight changes computed by a gradient-descent learning algorithm such as backpropagation. Deterministic and stochastic imperfections in the conductance response of real NVM devices can be encapsulated for modeling within a pair of “jump-tables.” Such jump-tables describe the full cumulative distribution function of conductance-change at each device conductance value, for both weight potentiation (SET) and depression (RESET). First, using several types of artificially constructed jump-tables, we revisit the relative importance of deviations from an ideal NVM with perfectly linear conductance response. Then, using jump-tables measured on improved non-filamentary resistive RAM devices based on Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub>[see companion paper], we simulate the effects of their nonlinear conductance response on the training of a three-layer fully connected neural network. We find that, despite the relatively large conductance changes exhibited by any Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> device when either potentiating from its lowest conductance state or depressing from its highest conductance states, neural network training accuracies of >90% can be achieved. Highest accuracies are achieved by programming both conductances on each timestep (“fully bidirectional”), with the improved conductance on/off ratio of Al/Mo/PCMO resulting in marked improvements in training and test accuracy. Further accuracy improvements can be obtained by tuning the relative learning rate for potentiation (SET) by a factor of <inline-formula> <tex-math notation="LaTeX">$1.66\times $ </tex-math></inline-formula> with respect to depression (RESET), to offset the slight asymmetry between the average size of the associated SET and RESET conductance changes. Finally, we show that the bidirectional programming of Al/Mo/PCMO can be used to implement high-density neuromorphic systems with a single conductance per synapse, at only a slight degradation to accuracy.Alessandro FumarolaSeverin SidlerKibong MoonJunwoo JangRobert M. ShelbyPritish NarayananYusuf LeblebiciHyunsang HwangGeoffrey W. BurrIEEEarticleMulti-layer neural networkneural network hardwarenonvolatile memoryElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Journal of the Electron Devices Society, Vol 6, Pp 169-178 (2018) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Multi-layer neural network neural network hardware nonvolatile memory Electrical engineering. Electronics. Nuclear engineering TK1-9971 |
| spellingShingle |
Multi-layer neural network neural network hardware nonvolatile memory Electrical engineering. Electronics. Nuclear engineering TK1-9971 Alessandro Fumarola Severin Sidler Kibong Moon Junwoo Jang Robert M. Shelby Pritish Narayanan Yusuf Leblebici Hyunsang Hwang Geoffrey W. Burr Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| description |
Neuromorphic computing embraces the “device history” offered by many analog non-volatile memory (NVM) devices to implement the small weight changes computed by a gradient-descent learning algorithm such as backpropagation. Deterministic and stochastic imperfections in the conductance response of real NVM devices can be encapsulated for modeling within a pair of “jump-tables.” Such jump-tables describe the full cumulative distribution function of conductance-change at each device conductance value, for both weight potentiation (SET) and depression (RESET). First, using several types of artificially constructed jump-tables, we revisit the relative importance of deviations from an ideal NVM with perfectly linear conductance response. Then, using jump-tables measured on improved non-filamentary resistive RAM devices based on Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub>[see companion paper], we simulate the effects of their nonlinear conductance response on the training of a three-layer fully connected neural network. We find that, despite the relatively large conductance changes exhibited by any Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> device when either potentiating from its lowest conductance state or depressing from its highest conductance states, neural network training accuracies of >90% can be achieved. Highest accuracies are achieved by programming both conductances on each timestep (“fully bidirectional”), with the improved conductance on/off ratio of Al/Mo/PCMO resulting in marked improvements in training and test accuracy. Further accuracy improvements can be obtained by tuning the relative learning rate for potentiation (SET) by a factor of <inline-formula> <tex-math notation="LaTeX">$1.66\times $ </tex-math></inline-formula> with respect to depression (RESET), to offset the slight asymmetry between the average size of the associated SET and RESET conductance changes. Finally, we show that the bidirectional programming of Al/Mo/PCMO can be used to implement high-density neuromorphic systems with a single conductance per synapse, at only a slight degradation to accuracy. |
| format |
article |
| author |
Alessandro Fumarola Severin Sidler Kibong Moon Junwoo Jang Robert M. Shelby Pritish Narayanan Yusuf Leblebici Hyunsang Hwang Geoffrey W. Burr |
| author_facet |
Alessandro Fumarola Severin Sidler Kibong Moon Junwoo Jang Robert M. Shelby Pritish Narayanan Yusuf Leblebici Hyunsang Hwang Geoffrey W. Burr |
| author_sort |
Alessandro Fumarola |
| title |
Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| title_short |
Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| title_full |
Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| title_fullStr |
Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| title_full_unstemmed |
Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part II: Impact of Al/Mo/Pr<sub>0.7</sub>Ca<sub>0.3</sub>MnO<sub>3</sub> Device Characteristics on Neural Network Training Accuracy |
| title_sort |
bidirectional non-filamentary rram as an analog neuromorphic synapse, part ii: impact of al/mo/pr<sub>0.7</sub>ca<sub>0.3</sub>mno<sub>3</sub> device characteristics on neural network training accuracy |
| publisher |
IEEE |
| publishDate |
2018 |
| url |
https://doaj.org/article/36efdf8a930c4875b9b68848f34662b6 |
| work_keys_str_mv |
AT alessandrofumarola bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT severinsidler bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT kibongmoon bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT junwoojang bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT robertmshelby bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT pritishnarayanan bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT yusufleblebici bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT hyunsanghwang bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy AT geoffreywburr bidirectionalnonfilamentaryrramasananalogneuromorphicsynapsepartiiimpactofalmoprsub07subcasub03submnosub3subdevicecharacteristicsonneuralnetworktrainingaccuracy |
| _version_ |
1718420662484205568 |