Arm终端设备的GPU可信执行环境的研究与实现

GPU已经广泛应用在Arm终端设备中，用以提升计算密集型和数据密集型应用的性能，其中包括一些带有敏感数据的应用（如人脸识别和指纹识别）。然而，通过利用操作系统内核中的漏洞，攻击者不仅可以取得对GPU的控制权，也可以直接访问GPU内存中的敏感数据，这严重威胁了GPU计算的安全性。为了解决GPU计算所面临的威胁，现有的工作使用了可信执行环境技术来保护GPU计算。然而，目前大部分工作致力于为Intel平台的服务器GPU构建可信执行环境，在架构上与Arm终端设备的GPU存在着巨大差异，这为方案的设计引入了新的技术性挑战。除此之外，这些工作也存在着可用性差、可信计算基大以及兼容性差的问题。
为了解决上述问题，本文针对Arm终端设备GPU的架构特性，提出了一种同时具备安全性、高可用性、小可信计算基以及高兼容性的GPU可信执行环境设计方案。设计方案复用富执行环境中的GPU软件栈进行功能性操作，并在安全世界中部署一个轻量级的安全运行时来为机密GPU计算提供隔离执行环境、动态细粒度的内存保护以及安全性检查，在保证安全性的同时缩小了可信计算基的大小。并且，通过利用通用的Arm硬件特性和ArmTrustZone技术，本文在无需修改硬件或架构的前提下可以保护不同规模的敏感GPU计算，保证了高可用性和高兼容性。除此之外，本文进一步优化保护过程中冗余的加解密操作和内存控制权限转换操作，最终减少了保护操作所带来的开销。本文利用ArmMaliGPU实现设计方案的原型，并进行了广泛的性能评估和安全性分析。实验结果表明，本设计方案不仅能够成功确保GPU计算的安全性，并在具有代表性的基准测试中仅引入了较低的性能开销（4.70%-15.26%）。

[1] VAIBHAV J, DINESH P. A GPU Based Implementation of Robust Face Detection System[J].Procedia Computer Science, 2016, 87: 156-163.
[2] MIGUEL L, JESÚS C, DAVID G P, et al. Fast fingerprint identification using GPUs[J]. InformationSciences, 2015, 301: 195-214.
[3] PODBUCKI K, MARCINIAK T. Aspects of autonomous drive control using NVIDIA JetsonNano microcomputer[C]//Annals of Computer Science and Information Systems: Vol. 30Proceedings of the 17th Conference on Computer Science and Intelligence Systems, FedCSIS2022, Sofia, Bulgaria, September 4-7, 2022. 2022: 117-120.
[4] CVE. CVE-2023-20937[EB/OL]. 2023
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-20937.
[5] CVE. CVE-2022-4378[EB/OL]. 2022
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-4378.
[6] CVE. CVE-2022-40133[EB/OL]. 2022
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-40133.
[7] COSTAN V, DEVADAS S. Intel SGX Explained[J]. IACR Cryptol. ePrint Arch., 2016: 86.
[8] CVE. CVE-2022-21815[EB/OL]. 2022
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-21815.
[9] CVE. CVE-2021-1121[EB/OL]. 2021
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-1121.
[10] CVE. CVE-2021-1093[EB/OL]. 2021
[2023-03-22]. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-1093.
[11] ARM. TrustZone for Cortex-A[EB/OL]. 2023
[2023-03-22]. https://www.arm.com/technologies/trustzone-for-cortex-a.
[12] ARM. Arm Architecture Reference Manual Armv8, for Armv8-A architecture profile[EB/OL].2022
[2023-03-22]. https://developer.arm.com/documentation/ddi0487/latest/.
[13] QUALCOMM. Mobile Security Solutions[EB/OL]. 2023
[2023-03-22]. https://www.qualcomm.com/products/features/mobile-security-solutions.
[14] LINARO. Open Portable Trusted Execution Environment[EB/OL]. 2023
[2023-03-22]. https://github.com/OP-TEE/optee_os.
[15] HUA Z C, GU J Y, XIA Y B, et al. vTZ: Virtualizing ARM TrustZone[C]//26th USENIXSecurity Symposium, USENIX Security 2017, Vancouver, BC, Canada, August 16-18, 2017.USENIX Association, 2017: 541-556.
[16] BRASSER F, GENS D, JAUERNIG P, et al. SANCTUARY: ARMing TrustZone with UserspaceEnclaves[C]//26th Annual Network and Distributed System Security Symposium, NDSS2019, San Diego, California, USA, February 24-27, 2019. The Internet Society, 2019.
[17] SUN H, SUN K, WANG Y W, et al. TrustICE: Hardware-Assisted Isolated Computing Environmentson Mobile Devices[C]//45th Annual IEEE/IFIP International Conference on DependableSystems and Networks, DSN 2015, Rio de Janeiro, Brazil, June 22-25, 2015. IEEE ComputerSociety, 2015: 367-378.
[18] JANG J S, CHOI C, LEE J, et al. PrivateZone: Providing a Private Execution EnvironmentUsing ARM TrustZone[J]. IEEE Trans. Dependable Secur. Comput., 2018, 15(5): 797-810.
[19] LI D J, MI Z Y, XIA Y B, et al. TwinVisor: Hardware-isolated Confidential Virtual Machinesfor ARM[C]//SOSP ’21: ACM SIGOPS 28th Symposium on Operating Systems Principles,Virtual Event / Koblenz, Germany, October 26-29, 2021. ACM, 2021: 638-654.
[20] SUN L Z, WANG S C, WU H, et al. LEAP: TrustZone Based Developer-Friendly TEE forIntelligent Mobile Apps[J]. IEEE Trans. Mob. Comput., 2022.
[21] VASILIADIS G, ATHANASOPOULOS E, POLYCHRONAKIS M, et al. PixelVault: UsingGPUs for Securing Cryptographic Operations[C]//Proceedings of the 2014 ACM SIGSACConference on Computer and Communications Security, Scottsdale, AZ, USA, November 3-7,2014. ACM, 2014: 1131-1142.
[22] KWON O, KIM Y, HUH J, et al. ZeroKernel: Secure Context-Isolated Execution on CommodityGPUs[J]. IEEE Trans. Dependable Secur. Comput., 2021, 18(4): 1974-1988.
[23] JANG I, TANG A, KIM T, et al. Heterogeneous Isolated Execution for Commodity GPUs[C]//Proceedings of the Twenty-Fourth International Conference on Architectural Support forProgramming Languages and Operating Systems, ASPLOS 2019, Providence, RI, USA, April13-17, 2019. ACM, 2019: 455-468.
[24] LIU R J, GARCIA L, LIU Z X, et al. SecDeep: Secure and Performant On-device Deep LearningInference Framework for Mobile and IoT Devices[C]//IoTDI ’21: International Conferenceon Internet-of-Things Design and Implementation, Virtual Event / Charlottesville, VA, USA,May 18-21, 2021. ACM, 2021: 67-79.
[25] JIANG J Y, QI J, SHEN T X, et al. CRONUS: Fault-isolated, Secure and High-performanceHeterogeneous Computing for Trusted Execution Environment[C]//55th IEEE/ACM InternationalSymposium on Microarchitecture, MICRO 2022, Chicago, IL, USA, October 1-5, 2022.IEEE, 2022: 124-143.
[26] ARM. Secure virtualization[EB/OL]. 2023
[2023-03-22]. https://developer.arm.com/documentation/102142/0100/Secure-virtualization.
[27] VOLOS S, VASWANI K, BRUNO R. Graviton: Trusted Execution Environments on GPUs[C]//13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2018,Carlsbad, CA, USA, October 8-10, 2018. USENIX Association, 2018: 681-696.
[28] ZHU J P, HOU R, WANG X F, et al. Enabling Rack-scale Confidential Computing usingHeterogeneous Trusted Execution Environment[C]//2020 IEEE Symposium on Security andPrivacy, SP 2020, San Francisco, CA, USA, May 18-21, 2020. IEEE, 2020: 1450-1465.
[29] NVIDIA. NVIDIA Confidential Computing[EB/OL]. 2023
[2023-03-22]. https://www.nvidia.com/en-us/data-center/solutions/confidential-computing/.
[30] ARM. Arm-Trusted-Firmware[EB/OL]. 2023
[2023-03-22]. https://github.com/ARM-software/arm-trusted-firmware.
[31] CERDEIRA D, MARTINS J, SANTOS N, et al. ReZone: Disarming TrustZone with TEEPrivilege Reduction[C]//31st USENIX Security Symposium, USENIX Security 2022, Boston,MA, USA, August 10-12, 2022. USENIX Association, 2022: 2261-2279.
[32] ARM. Arm Generic Interrupt Controller Architecture Specification version 2.0[EB/OL]. 2013
[2023-03-22]. https://developer.arm.com/documentation/ihi0048/latest.
[33] NING Z Y, WANG C X, CHEN Y H, et al. Revisiting ARM Debugging Features: Nailgun andits Defense[J]. IEEE Trans. Dependable Secur. Comput., 2023, 20(1): 574-589.
[34] ARM. Open Source Mali Midgard GPU Kernel Drivers[EB/OL]. 2023
[2023-03-22]. https://developer.arm.com/tools-and-software/graphics-and-gaming/mali-drivers/midgard-kernel.
[35] ARM. OpenCL[EB/OL]. 2023
[2023-03-22]. https://developer.arm.com/tools-and-software/graphics-and-gaming/mali-drivers/user-space.
[36] ARM. ComputeLibrary[EB/OL]. 2023
[2023-03-22]. https://github.com/ARM-software/ComputeLibrary.
[37] LEE J, LIU Y X, LEE Y. ParallelFusion: Towards Maximum Utilization of Mobile GPU forDNN Inference[C]//EMDL@MobiSys 2021: Proceedings of the 5th International Workshop onEmbedded and Mobile Deep Learning, Virtual Event, Wisconsin, USA, June 25, 2021. ACM,2021: 25-30.
[38] JEONG J S, LEE J, KIM D, et al. Band: coordinated multi-DNN inference on heterogeneousmobile processors[C]//MobiSys ’22: The 20th Annual International Conference on Mobile Systems,Applications and Services, Portland, Oregon, 27 June 2022 - 1 July 2022. ACM, 2022:235-247.
[39] SANDLER M, HOWARD A G, ZHU M L, et al. MobileNetV2: Inverted Residuals and LinearBottlenecks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR2018, Salt Lake City, UT, USA, June 18-22, 2018. Computer Vision Foundation / IEEE ComputerSociety, 2018: 4510-4520.
[40] WANG J W, LI A, LI H R, et al. RT-TEE: Real-time System Availability for Cyber-physicalSystems using ARM TrustZone[C]//43rd IEEE Symposium on Security and Privacy, SP 2022,San Francisco, CA, USA, May 22-26, 2022. IEEE, 2022: 352-369.
[41] ALI N A, ABBASSI A E, CHERRADI B. The performances of iterative type-2 fuzzy C-meanon GPU for image segmentation[J]. J. Supercomput., 2022, 78(2): 1583-1601.
[42] MISTRY P, GREGG C, RUBIN N, et al. Analyzing program flow within a many-kernelOpenCL application[C]//Proceedings of 4th Workshop on General Purpose Processing onGraphics Processing Units, GPGPU 2011, Newport Beach, CA, USA, March 5, 2011. ACM,2011: 10.
[43] TORRES J F, HADJOUT D, SEBAA A, et al. Deep Learning for Time Series Forecasting: ASurvey[J]. Big Data, 2021, 9(1): 3-21.
[44] GANDHI V, WANG H, BOURD A. Optimization of Fast Fourier Transform (FFT) on QualcommAdreno GPU[C]//Proceedings of the International Workshop on OpenCL, IWOCL 2019,Boston, MA, USA, May 13-15, 2019. ACM, 2019: 21:1-21:2.
[45] MANAVSKI S A. CUDA compatible GPU as an efficient hardware accelerator for AES cryptography[C]//2007 IEEE International Conference on Signal Processing and Communications.IEEE, 2007: 65-68.
[46] ARM. Juno r2 Arm Development Platform SoC[EB/OL]. 2016
[2023-03-22]. https://developer.arm.com/documentation/ddi0515/latest.
[47] JANG J S, KONG S, KIM M, et al. SeCReT: Secure Channel between Rich Execution Environmentand Trusted Execution Environment[C]//22nd Annual Network and Distributed SystemSecurity Symposium, NDSS 2015, San Diego, California, USA, February 8-11, 2015. TheInternet Society, 2015.
[48] SUN H, SUN K, WANG Y W, et al. TrustOTP: Transforming Smartphones into Secure One-Time Password Tokens[C]//Proceedings of the 22nd ACM SIGSAC Conference on Computerand Communications Security, Denver, CO, USA, October 12-16, 2015. ACM, 2015: 976-988.
[49] EKLEKTIX. A deep dive into cma.[EB/OL]. 2012
[2023-03-22]. https://lwn.net/Articles/486301/.
[50] ARM. Midgard Architecture[EB/OL]. 2018
[2023-03-22]. https://gitlab.freedesktop.org/panfrost/mali-isa-docs/-/blob/master/Midgard.md.
[51] ARM. Mali-G78 GPUs Valhall instruction set documentation released after reverse-engineeringwork[EB/OL]. 2021
[2023-03-22]. https://www.cnx-software.com/2021/07/23/mali-g78-gpu-valhall-instruction-set-documentation-reverse-engineering/.
[52] ALDANIAL. cloc[EB/OL]. 2023
[2023-03-22]. https://github.com/AlDanial/cloc.
[53] CHE S, BOYER M, MENG J Y, et al. Rodinia: A benchmark suite for heterogeneous computing[C]//Proceedings of the 2009 IEEE International Symposium on Workload Characterization,IISWC 2009, October 4-6, 2009, Austin, TX, USA. IEEE Computer Society, 2009: 44-54.
[54] LEE H, KIM H, KIM C, et al. Idempotence-Based Preemptive GPU Kernel Scheduling forEmbedded Systems[J]. IEEE Trans. Computers, 2021, 70(3): 332-346.
[55] BAGHDADI R, BEAUGNON U, COHEN A, et al. PENCIL: A Platform-Neutral Compute IntermediateLanguage for Accelerator Programming[C]//2015 International Conference on ParallelArchitectures and Compilation, PACT 2015, San Francisco, CA, USA, October 18-21,2015. IEEE Computer Society, 2015: 138-149.
[56] LEE H, ROH J, SEO E. A GPU Kernel Transactionization Scheme for Preemptive PriorityScheduling[C]//IEEE Real-Time and Embedded Technology and Applications Symposium,RTAS 2018, 11-13 April 2018, Porto, Portugal. IEEE Computer Society, 2018: 202-213.
[57] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proc. IEEE, 1998, 86(11): 2278-2324.
[58] IANDOLA F N, MOSKEWICZ M W, ASHRAF K, et al. SqueezeNet: AlexNet-level accuracywith 50x fewer parameters and <1MB model size[J]. CoRR, 2016, abs/1602.07360.
[59] HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: Efficient Convolutional Neural Networksfor Mobile Vision Applications[J]. CoRR, 2017, abs/1704.04861.
[60] KATO S, MCTHROW M, MALTZAHN C, et al. Gdev: First-Class GPU Resource Managementin the Operating System[C]//2012 USENIX Annual Technical Conference, Boston, MA,USA, June 13-15, 2012. USENIX Association, 2012: 401-412.
[61] CHECKOWAY S, SHACHAM H. Iago attacks: why the system call API is a bad untrustedRPC interface[M]//Architectural Support for Programming Languages and Operating Systems,ASPLOS 2013, Houston, TX, USA, March 16-20, 2013. ACM, 2013: 253-264.
[62] ELECTRONICS F R. Rockchip RK3288 Technical Reference Manual Part1[EB/OL]. 2017
[2023-03-22]. http://opensource.rock-chips.com/images/8/8f/Rockchip_RK3288_TRM_V1.2_Part1-20170321.pdf.
[63] AMLOGIC. S905 Datasheet[EB/OL]. 2016
[2023-03-22]. https://dn.odroid.com/S905/DataSheet/S905_Public_Datasheet_V1.1.4.pdf.
[64] STMICROELECTRONICS. GPU device tree configuration[EB/OL]. 2023
[2023-03-22]. https://wiki.st.com/stm32mpu/wiki/GPU_device_tree_configuration.
[65] ARM. Arm Architecture Reference Manual Supplement Armv9, for Armv9-A architectureprofile[EB/OL]. 2022
[2023-03-22]. https://developer.arm.com/documentation/ddi0608/latest.
[66] ARM. Granule Protection Tables in TF-A[EB/OL]. 2021
[2023-03-22]. https://www.trustedfirmware.org/docs/tfa_tech_forum_2021_10_21_gpt.pdf.