基于氧化铪忆阻器的多精度计算

近年来，人工智能技术飞速发展，诸如神经网络等技术的应用场景需 要大量的模型参数，然而在传统冯·诺依曼架构中，存储器和运算单元的 分离，大量参数的传输会造成大功耗和长延时，导致在硬件上计算和存储 的成本急剧上升，当前急需一种拥有高计算能效的硬件系统架构。忆阻器 具有非易失性、响应快速、和超高集成度等优点，是存内运算架构中重要 的候选器件，基于氧化铪型忆阻器的存内运算架构由于其高能效的特点引 起了广泛关注，它可以很好地解决冯·诺依曼架构面临的内存墙和数据存 取功耗两大瓶颈问题。 在这项研究中，我们提出了一种用于多精度计算的忆阻器存内运算方 案。在这套方案中，我们可以利用 3 位的氧化铪型忆阻器器件来实现较高 精度（4 位精度或 8 位精度）的有符号乘法和累加操作。我们的研究分析了 在使用忆阻器的非零电导态表示数字“0”引起的系统误差，提出了三种用 忆阻器实现的乘法器方案，通过仿真分析了三种方案下的系统误差，并选 取了其中误差最小的方案用于实现乘法单元，在这基础上提出了一种通过 加上偏置值的校正方法用于减小系统误差的影响。我们还分析了由于忆阻 器器件的本征随机性导致的电导变化而引起的随机误差，该误差是无法消 除的，但是我们提出的方案可以尽可能地抵抗随机误差带来的影响。并且 我们使用提出的存内运算方案在小型神经网络 LeNet-5 和大型神经网络 ResNet18 上进行了仿真，得到了较为可观的识别准确率。最后，评估了整 个系统的硬件效率，结果表明它是一种有希望在实际应用中使用的方案。

[1] 岳珂娟. 冯· 诺依曼体系结构在计算机组成原理教学中的重要性[J]. 计算机教育, 2013 (24): 99-101.
[2] MCKEE S A. Reflections on the memory wall[C]//Proceedings of the 1st Conference on Computing Frontiers. 2004: 162.
[3] ZOU X, XU S, CHEN X, et al. Breaking the von Neumann bottleneck: architecture -level processing-in-memory technology[J]. Science China Information Sciences, 2021,64(6): 160404.
[4] Sze V, CHEN Y H, YANG T J, et al. Efficient processing of deep neural networks: Atutorial and survey[J]. Proceedings of the IEEE, 2017, 105(12): 2295-2329.
[5] WILKES M V. The memory wall and the CMOS end-point[J]. ACM SIGARCHComputer Architecture News, 1995, 23(4): 4-6.
[6] SOMAVAT P, JADHAV S, NAMBOODIRI V. Accounting for the energy consumptionof personal computing including portable devices[C]//Proceedings of the 1stInternational Conference on Energy-Efficient Computing and Networking. 2010: 141-149.
[7] YU S, JIANG H, HUANG S, et al. Compute-in-memory chips for deep learning:Recent trends and prospects[J]. IEEE Circuits and Systems Magazine, 2021, 21(3):31-56.
[8] JASIONOWSKI B J, LAY M K, MARGALA M. A processor-in-memory architecturefor multimedia compression[J]. IEEE Transactions on Very Large Scale Integration(VLSI) Systems, 2007, 15(4): 478-483.
[9] WULF W A, MCKEE S A. Hitting the memory wall: Implications of the obvious[J].ACM SIGARCH Computer Architecture News, 1995, 23(1): 20-24.
[10] NOWATZYK A, Pong F, SAULSBURY A. Missing the memory wall: The case forprocessor/memory integration[C]//23rd Annual International Symposium onComputer Architecture (ISCA'96). IEEE, 1996: 90-90.
[11] KOGGE P M. EXECUBE-a new architecture for scaleable MPPs[C]//1994International Conference on Parallel Processing Vol. 1. IEEE, 1994, 1: 77 -84.
[12] PATTERSON D, ANDERSON T, CARDWELL N, et al. A case for intelligent RAM[J].IEEE Micro, 1997, 17(2): 34-44.
[13] KANG Y, HUANG W, YOO S M, et al. FlexRAM: Toward an advanced intelligentmemory system[C]//Proceedings 1999 IEEE International Conference on ComputerDesign: VLSI in Computers and Processors (Cat. No. 99CB37040). IEEE, 1999: 192-201.
[14] AUSTIN T M. DIVA: A reliable substrate for deep submicron microarchitecturedesign[C]//MICRO-32. Proceedings of the 32nd Annual ACM/IEEE InternationalSymposium on Microarchitecture. IEEE, 1999: 196-207.
[15] KEITEL-SCHULZ D, WEHN N. Issues in embedded DRAM development andapplications[C]//Proceedings. 11th International Symposium on System Synthesis(Cat. No. 98EX210). IEEE, 1998: 23-28.
[16] KEITEL-SCHULZ D, WEHN N. Embedded DRAM development: Technology,physical design, and application issues[J]. IEEE Design & Test of Computers, 2001,18(3): 7-15.
[17] ZIDAN M A, STRACHAN J P, LU W D. The future of electronics based on memristive systems[J]. Nature Electronics, 2018, 1(1): 22-29.
[18] HANYU T, ENDOH T, ANDO Y, et al. memory (STT-MRAM) technology[J].Advances in Non-volatile Memory and Storage Technology, 2019: 237.
[19] TOMINAGA J. The Design and Application on Interfacial Phase‐Change Memory[J].Physica Status Solidi (RRL)–Rapid Research Letters, 2019, 13(4): 1800539.
[20] YANG Z, WEI L. Logic Circuit and Memory Design for In-Memory ComputingApplications using Bipolar RRAMs[C]. 2019 IEEE International Symposium onCircuits and Systems (ISCAS). IEEE, 2019: 1-5.
[21] MITTAL S. A Survey of ReRAM-based Architectures for Processing-in-memory and Neural Networks[J]. Machine Learning and Knowledge Extraction, 2019, 1(1): 75 -114.
[22] SEBASTIAN A, LE GALLO M, KHADDAM-ALJAMEH R, et al. Memory devicesand applications for in-memory computing[J]. Nature Nanotechnology, 2020, 15(7):529-544.
[23] JELOKA S, AKESH N B, SYLVESTER D, et al. A 28 nm configurable memory(TCAM/BCAM/SRAM) using push-rule 6T bit cell enabling logic-in-memory[J].IEEE Journal of Solid-State Circuits, 2016, 51(4): 1009-1021.
[24] AGA S, JELOKA S, SUBRAMANIYAN A, et al. Compute caches[C]//2017 IEEEInternational Symposium on High Performance Computer Architecture (HPCA). IEEE,2017: 481-492.
[25] CHIH Y D, LEE P H, FUJIWARA H, et al. 16.4 An 89TOPS/W and 16.3 TOPS/mm 2all-digital SRAM-based full-precision compute-in memory macro in 22nm formachine-learning edge applications[C]//2021 IEEE International Solid-State CircuitsConference (ISSCC). IEEE, 2021, 64: 252-254.
[26] MERRIKH-BAYAT F, GUO X, KLACHKO M, et al. High-performance mixed-signalneurocomputing with nanoscale floating-gate memory cell arrays[J]. IEEETransactions on Neural Networks and Learning Systems, 2017, 29(10): 4782 -4790.
[27] KIM K, JEONG G. Memory technologies in the nano-era: challenges andopportunities[C]//2005 International Conference on Integrated Circuit Design andTechnology, 2005. ICICDT 2005. IEEE, 2005: 63-67.
[28] STRUKOV D B, SNIDER G S, STEWART D R, et al. The missing memristor found[J].Nature, 2008, 453(7191): 80-83.
[29] KIM Y B, LEE S R, LEE D, et al. Bi-layered RRAM with unlimited endurance andextremely uniform switching[C]//2011 Symposium on VLSI Technology-Digest ofTechnical Papers. IEEE, 2011: 52-53.
[30] FU Y, ZHOU Y, HUANG X, et al. Forming-free and Annealing-free V/VO x/HfWOx/Pt Device Exhibiting Reconfigurable Threshold and Resistive switching with highspeed (< 30ns) and high endurance (> 10 12/> 10 10)[C]//2021 IEEE InternationalElectron Devices Meeting (IEDM). IEEE, 2021: 12.6. 1-12.6. 4.
[31] KIM S, PARK J, KIM T H, et al. 4‐bit Multilevel Operation in Overshoot SuppressedAl2O3/TiOx Resistive Random ‐ Access Memory Crossbar Array[J]. AdvancedIntelligent Systems, 2022: 2100273.
[32] QIN S, TUNG M, BELLIVEAU E, et al. 8-Layer 3D Vertical Ru/AlO x N y/TiNRRAM with Mega- Ω Level LRS for Low Power and Ultrahigh-densityMemory[C]//2022 IEEE Symposium on VLSI Technology and Circuits (VLSITechnology and Circuits). IEEE, 2022: 314-315.
[33] PREZIOSO M, MERRIKH-BAYAT F, HOSKINS B D, et al. Training and operationof an integrated neuromorphic network based on metal-oxide memristors[J]. Nature,2015, 521(7550): 61-64.
[34] YAO P, WU H, GAO B, et al. Fully hardware-implemented memristor convolutional neural network[J]. Nature, 2020, 577(7792): 641-646.
[35] HUNG J M, HUANG Y H, HUANG S P, et al. An 8-Mb DC-Current-Free Binary-to8b Precision ReRAM Nonvolatile Computing-in-Memory Macro using Time-SpaceReadout with 1286.4-21.6 TOPS/W for Edge-AI Devices[C]//2022 IEEE International Solid-State Circuits Conference (ISSCC). IEEE, 2022, 65: 1-3.
[36] CORRELL J M, JIE L, SONG S, et al. An 8-bit 20.7 TOPS/W multi-level cell ReRAMbased compute engine[C]//2022 IEEE Symposium on VLSI Technology and Circuits(VLSI Technology and Circuits). IEEE, 2022: 264-265.
[37] CHUA L. Memristor-the missing circuit element[J]. IEEE Transactions on CircuitTheory, 1971, 18(5): 507-519.
[38] Strukov D B, Alibart F, Stanley Williams R. Thermophoresis/diffusion as a plausible mechanism for unipolar resistive switching in metal–oxide–metal memristors[J].Applied Physics A, 2012, 107: 509-518.
[39] Gao B, Kang J F, Chen Y S, et al. Oxide-based RRAM: Unified microscopic principle for both unipolar and bipolar switching[C]//2011 International Electron DevicesMeeting. IEEE, 2011: 417-420.
[40] MULAY A. Sustaining Moore's Law: Uncertainty Leading to a Certainty of IoTRevolution[J]. Synthesis Lectures on Emerging Engineering Technologies, 2015, 1(1):1-109.
[41] WEI S, GAO B, Wu D, et al. Trends and challenges in the circuit and macro of RRAMbased computing-in-memory systems[J]. Chip, 2022: 100004.
[42] HU M, STRACHAN J P, LI Z, et al. Dot-product engineas computing memory toaccelerate machine learning algorithms[C]// ISQED, 2016: 374-379.
[43] KUZUM D, YU S, WONG H P. Synaptic electronics: materials, devices andapplications[J]. Nanotechnology, 2013, 24(38): 382001.
[44] PÉREZ-BOSCH QUESADA E, ROMERO-ZALIZ R, PEREZ E, et al. Toward reliablecompact modeling of multilevel 1T-1R RRAM devices for neuromorphic systems[J].Electronics, 2021, 10(6): 645.
[45] 方伊琳. 面向人工神经网络应用的忆阻器器件及集成技术研究[D]. 合肥: 中国科学技术大学, 2020.
[46] LIN Z, SI M, LYU X, et al. High-Performance In2O3-Based 1T1R FET for BEOLMemory Application[J]. IEEE Transactions on Electron Devices, 2021, 68(8): 3775-3779.
[47] UPADHYAY N K, SUN W, LIN P, et al. A memristor with low switching current andvoltage for 1S1R integration and array operation[J]. Advanced Electronic Materials,2020, 6(5): 1901411.
[48] 郑远攀,李广阳,李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用,2019,55(12):20-36.
[49] HECHT-NIELSEN R. Theory of the backpropagation neural network[M]//NeuralNetworks for Perception. Academic Press, 1992: 65-93.
[50] 周飞燕,金林 鹏,董军 .卷 积神经网络研究 综述 [J].计算机学 报,2017,40(06):1229-1251.
[51] 季长清 ,高 志 勇 ,秦 静 等 .基于卷 积 神 经 网 络 的 图 像 分 类 算 法 综 述 [J].计算机应用,2022,42(04):1044-1049.
[52] MOTAMED M. A multi-fidelity neural network surrogate sampling method foruncertainty quantification[J]. International Journal for Uncertainty Quantification,2020, 10(4).
[53] IM S , SHIN D . ComboFTL: Improving performance and lifespan of MLC flashmemory using SLC flash buffer[J]. Journal of Systems Architecture, 2010, 56(12):641 -653.
[54] HONG X L, LOY D J J, DANANJAYA P A, et al. Oxide-based RRAM materials forneuromorphic computing[J]. Journal of Materials Science, 2018, 53: 8720-8746.
[55] 胡小方. 基于忆阻器的非易失性存储器研究[D]. 重庆: 西南大学, 2012.
[56] 伍法才. 氧化物阻变存储器多值存储与界面改性[D]. 武汉: 武汉大学, 2018.
[57] Zhang W, Pan L, Yan X, et al. Hardware friendly stochastic and adaptive learning in memristor convolutional neural networks[J]. Advanced Intelligent Systems, 2021, 3(9):2100041
[58] GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning[M]. MIT Press, 2016.
[59] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deepconvolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84 -90.
[60] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large -scaleimage recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[61] QU Z, ZHOU Z, CHENG Y, et al. Adaptive loss-aware quantization for multi-bitnetworks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision andPattern Recognition. 2020: 7988-7997.
[62] MENG X, KARNIADAKIS G E. A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems[J].Journal of Computational Physics, 2020, 401: 109020.
[63] HUANG M, LIU Y, MAN C, et al. A High Performance Multi-Bit-Width Booth VectorSystolic Accelerator for NAS Optimized Deep Learning Neural Networks[J]. IEEETransactions on Circuits and Systems I: Regular Papers, 2022, 69(9): 3619 -3631.
[64] CHAKRABORTY I, ALI M, ANKIT A, et al. Resistive crossbars as approximatehardware building blocks for machine learning: Opportunities and challenges[J].Proceedings of the IEEE, 2020, 108(12): 2276-2310.
[65] NI L, WANG Y, YU H, et al. An energy-efficient matrix multiplication accelerator bydistributed in-memory computing on binary RRAM crossbar[C]//2016 21st Asia andSouth Pacific Design Automation Conference (ASP-DAC). IEEE, 2016: 280-285.
[66] LIU D, ZHOU H, MAO W, et al. An Energy-Efficient Mixed-Bit CNN AcceleratorWith Column Parallel Readout for ReRAM-Based In-Memory Computing[J]. IEEEJournal on Emerging and Selected Topics in Circuits and Systems, 2022, 12(4): 821 -834.
[67] ZHONG G, ZI M, REN C, et al. Flexible electronic synapse enabled by ferroelectricfield effect transistor for robust neuromorphic computing[J]. Applied Physics Letters,2020, 117(9): 092903.
[68] HAO Z, GAO B, XU M, et al. Cryogenic HfOₓ-Based Resistive Memory With aThermal Enhancement Capping Layer[J]. IEEE Electron Device Letters, 2021, 42(9):1276-1279.
[69] HUANG X D, LI Y, LI H Y, et al. Forming-free, fast, uniform, and high enduranceresistive switching from cryogenic to high temperatures in W/AlO x/Al 2 O 3/Ptbilayer memristor[J]. IEEE Electron Device Letters, 2020, 41(4): 549 -552.
[70] BAI N, TIAN B, MAO G Q, et al. Homo-layer hafnia-based memristor with largeanalog switching window[J]. Applied Physics Letters, 2021, 118(4): 043502.
[71] DIWARE S, GEBREGIORGIS A, JOSHI R V, et al. Unbalanced bit-slicing schemefor accurate memristor-based neural network architecture[C]//2021 IEEE 3rdInternational Conference on Artificial Intelligence Circuits and Systems (AICAS).IEEE, 2021: 1-4.
[72] BAO H, ZHOU H, LI J, et al. Toward memristive in-memory computing: principlesand applications[J]. Frontiers of Optoelectronics, 2022, 15(1): 23.
[73] DIWARE S, SINGH A, GEBREGIORGIS A, et al. Accurate and Energy-Efficient BitSlicing for RRAM-Based Neural Networks[J]. IEEE Transactions on Emerging Topicsin Computational Intelligence, 2022.
[74] 杨玖, 王丽丹, 段书凯. 一种反向串联忆阻突触电路的设计及应用 [J]. 中国科学:信息科学, 2016, 46 (3): 391-403.
[75] WU H, YAO P, GAO B, et al. Device and circuit optimization of RRAM forneuromorphic computing[C]//2017 IEEE International Electron Devices Meeting(IEDM). IEEE, 2017: 11.5. 1-11.5. 4.
[76] TANG K T, WEI W C, YEH Z W, et al. Considerations of integrating computing -inmemory and processing-in-sensor into convolutional neural network accelerators forlow-power edge devices[C]//2019 Symposium on VLSI Circuits. IEEE, 2019: T166 -T167.
[77] YOON J H, CHANG M, KHWA W S, et al. A 40-nm, 64-Kb, 56.67 TOPS/W voltagesensing computing-in-memory/digital RRAM macro supporting iterative write withverification and online read-disturb detection[J]. IEEE Journal of Solid-State Circuits,2021, 57(1): 68-79.
[78] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied todocument recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278 -2324.
[79] DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical imagedatabase[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2009: 248-255.
[80] HE K, ZHANG X, REN S, et al. Deep residual learning for imagerecognition[C]//Proceedings of the IEEE Conference on Computer Vision and PatternRecognition. 2016: 770-778.
[81] WANG N, CHOI J, BRAND D, et al. Training deep neural networks with 8-bit floatingpoint numbers[J]. Advances in Neural Information Processing Systems, 2018, 31.
[82] PAN L, XUE P, LI H, et al. A General-Purpose and Configurable Planar DataProcessor for Energy-Efficient Pooling Computation[C]//2022 IEEE 4th InternationalConference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, 2022: 33 -36.
[83] KULL L, TOIFL T, SCHMATZ M, et al. A 3.1 mW 8b 1.2 GS/s single -channelasynchronous SAR ADC with alternate comparators for enhanc ed speed in 32 nmdigital SOI CMOS[J]. IEEE Journal of Solid-State Circuits, 2013, 48(12): 3049-3058.