LSTM Cell Implementation on FPGAs
Abstract
This paper presents and discusses the implementation of an LSTM cell on an FPGA with an activation function inspired by the CORDIC algorithm. The realization is performed using both IEEE754 standard and 32-bit integer numbers. The case with floating-point arithmetic is analyzed with and without DSP blocks provided by the Xilinx design suite. The alternative implementation including the integer arithmetic was optimized for a minimal number of clock cycles. Presented implementation uses xc6slx150t-2fgg900 and achieves high calculations accuracy for both cases.
Communicated by Andrew Adamatzky
References
- 1. , Reconfigurable FPGA implementation of neural networks, Neurocomputing 308 (2018) 227–234. Crossref, ISI, Google Scholar
- 2. , Deep Learning with Python (Manning, November 2017). Google Scholar
- 3. , Implementing the on-chip backpropagation learning algorithm on FPGA architecture, in 2017 International Conference on System Science and Engineering (ICSSE),
2017 , pp. 538–541. Google Scholar - 4. , Implementation of activation functions for ELM based classifiers, in 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET),
2017 , pp. 1038–1042. Google Scholar - 5. , FPGA realization of activation function for neural network, in 2018 7th International Symposium on Next Generation Electronics (ISNE),
2018 , pp. 1–2. Google Scholar - 6. , A pipelined architecture for user-defined floating-point complex division on FPGA, in 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE),
2017 , pp. 1–4. Google Scholar - 7. , Long short-term memory, Neural Comput. 9 (November 1997) 1735–1780. Crossref, ISI, Google Scholar
- 8. , An FPGA implementation of a long short-term memory neural network, in 2016 International Conference on ReConFigurable Computing and FPGAs (ReConFig),
2016 , pp. 1–8. Google Scholar - 9. , Activation function architectures for FPGAs, in 2018 28th International Conference on Field Programmable Logic and Applications (FPL),
2018 , pp. 43–437. Google Scholar - 10. , Hardware implementation of sigmoid activation functions using FPGA, in 2019 IEEE 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM),
2019 , pp. 34–38. Google Scholar - 11. , Implementation of log and exponential function in FPGA, International Journal of Engineering Research & Technology 3(11) (2014). Google Scholar
- 12. , High throughput floating point exponential function implemented in FPGA, in 2015 IEEE Computer Society Annual Symposium on VLSI,
2015 , pp. 97–100. Google Scholar - 13. , E-LSTM: An efficient hardware architecture for long short-term memory, IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9(2) (2019) 280–291. Crossref, ISI, Google Scholar
- 14. , CORDIC II: A new improved CORDIC algorithm, IEEE Transactions on Circuits and Systems II: Express Briefs 63(2) (2016) 186–190. Crossref, ISI, Google Scholar


