RISCV-FNT: A Fast FNT-based RISC-V Processor for CNN Acceleration

Convolution forms the basis of computation in neural network applications. Many different approaches have been proposed in the past years to optimize the convolution operation. In this paper, we propose to use Fermat Number Transform (FNT) technique to accelerate the computation of convolution in neural networks. Calculations in FNT are all based on real numbers, which significantly reduce the complexity as compared to complex-number-based FFT calculations. Furthermore, by using diminished-1 encoding, multiplication and modulo operations can also be simplified into bit manipulations. In this paper, we have constructed a RISC-V based processor, called RISCV-FNT, which incorporates an FNT-based convolution acceleration unit, along with custom instruction sets. FPGA implementation of RISCV-FNT demonstrated an 8.5× speedup compared to other RISC-V processors without FNT acceleration when performing inference tasks on Lenet-5. Synthesized results from Synopsys^® DC achieved area energy efficiency of 93.9 GOPs/W/mm²