A 307-fps 351.7-GOPs/W Deep Learning FPGA Accelerator for Real-Time Scene Text Recognition

FPGA-based deep learning accelerator has become important for high throughput and low power inference at edges. In this paper, we have developed a computing-in-memory (CIM) accelerator using the binary SegNet (BSEG) for real-time scene text recognition (STR) at edges. The accelerator can perform highly efficient pixel-wise character classification under CIM architecture with massive bit-level parallelism as well as optimized pipeline for low latency at critical path. The BSEG is obtained during training with a small model size of 2.1MB as well as a high classification accuracy over 90% on ICDAR-03 and ICDAR-13 datasets. The RTL-level realized FPGA-accelerator can process the STR with an energy-efficiency of 351.7 GOPs/W and a throughput of 307 fps for processing one frame of 128×32 pixels in latency of 3.875 ms.