基于视频的非接触重症患者监测

持续的生命体征监测使临床医生能够及时评估重症监护室（Intensive Care Unit, ICU）患者的生理状况，且心率变异性（Heart Rate Variability, HRV）的监 测还能够提供有关自主神经系统的重要信息，可以作为检测 ICU 不良事件的潜在 指标。此外，实时检测患者警觉状态变化也为医生衡量其康复程度提供重要参考。

在本文中，我们针对 ICU 中的监护和预警两个功能搭建相关远程监测系统。 首先我们提出的远程患者生理信息监测系统将摄像机的功能从监视扩展到看护。 提出的监测系统采用了最新的基于摄像机的光电容积描记术（Camera-based Photoplethysmography, Camera-PPG），用于测量心率和呼吸频率。其次，我们使用 CameraPPG 测量 HRV 特征，以构建能够将 ICU 患者划分为不同的恶化等级的早期预警 系统。另外，基于 ICU 的视频进行对眼睛状态估计的任务。对于基于手工设计特 征的方法，几何、纹理和 RGB 特征被结合起来作为支持向量机分类器的输入，以 将眼睛状态分类为睁眼和闭眼。对于基于深度学习的方法，眼睛和面部图像被用 作分类的联合输入。

ICU 的临床试验结果表明，与心电图的参考结果相比，该系统测量心率的总 体平均绝对值误差（Mean Absolute Error, MAE）为 1.3 bpm, 呼吸率的 MAE 为 0.7 brpm，均在临床可接受的范围内。而且，该系统能够实现对接触式监测设备的补充 监测以及有效的长时间监测。Camera-PPG 测量 HRV 特征的准确性与接触式 PPG 相当，与心电图测量结果显示出很强的一致性。结合机器学习的 HRV 在健康人和 ICU 患者的分类中表现出色，准确率为 86.02%，F1 评分为 84.27%。它在区分重症 和轻度患者方面也显示出了可行性，准确率为 57.22%，F1 评分为 54.46%。临床结 果表明，基于相机的 HRV 监测系统有潜力成为心电图的替代工具，用于跟踪患者 的生理状态并及时的提供预警信号。对 ICU 患者的眼部状态估计临床结果显示，基 于 HOG-RGB 的方法准确率为 91.39%，而基于深度学习的方法准确率为 89.54%。 该系统表明相机对医院护理病房中所需要的持续患者监测非常有用，并且可以通 过已有的医疗物联网硬件和基础设施快速集成到现有的医院信息系统中进行大规 模部署。

The continuous vital signs monitoring allows clinicians to timely assess the physiological conditions of the patient in Intensive Care Unit (ICU). Monitoring a patient's heart rate variability (HRV) can provide vital information regarding the physiological status and autonomic nervous system, serving as a potential metric for detecting adverse events in ICU.  In addition, real-time detection of changes in a patient's alertness status provides an indicative of recovery for physicians.

In this paper, we build a remote detection system for the two functions of monitoring and early warning in ICU. Firstly, we proposed a remote patient physiological information monitoring system that extends the function of the camera from surveillance to warding. The proposed monitoring system implemented the latest Camera-based Photoplethysmography (Camera-PPG) algorithms for heart rate (HR) and breathing rate (BR) measurement. Subsequently, Camera-PPG was used to measure HRV features for constructing an early warning system that could classify ICU patients into different deterioration levels. In addition, ICU-based videos were performed for the task of eye state estimation. For handcrafted feature-based methods, geometric, HOG and RGB features are combined as the input of the SVM classifier to classify the eye state as open and closed. For deep learning-based methods, the eye and face images were used as joint input for classification.

The clinical results show that our system achieves a Mean Absolute Error (MAE) of 1.3 bpm for HR and 0.7 brpm for BR in the far-focus mode, which are in the range of clinical acceptance. Moreover, the system enables supplementary monitoring of contact monitoring equipment and effective monitoring over long periods of time.  The accuracy of Camera-PPG in measuring HRV features is comparable to that of contact-PPG, showing strong agreement with electrocardiogram (ECG).  HRV combined with machine learning has reasonable performance in classifying healthy individuals and ICU patients, with 86.02% accuracy and 84.27% F1-score. It also shows feasibility in differentiating between severe and mildly ill patients, with 57.22% accuracy and 54.46% F1-score.  The clinical results show that camera-HRV has potential to be an alternative tool of ECG for tracking patient's physiological states and providing timely warning signs. Clinical results of eye state estimation for ICU patients show that the HOG-RGB based method has an accuracy of 91.39% while the deep learning based method has an accuracy of 89.54%. The prototypes show that cameras are useful for the ubiquitous patient monitoring required in hospital care wards. It can be quickly integrated into current hospital information systems for large-scale deployment, by leveraging the existing hardware and infrastructures of the IoT. 

[

[1] KYRIACOS U, JELSMA J, JORDAN S. Monitoring vital signs using early warning scoring systems: a review of the literature[J]. Journal of Nursing Management, 2011, 19(3): 311-330.

[2] BEAUMONT K, LUETTEL D, THOMSON R. Deterioration in hospital patients: early signs and appropriate actions[J]. Nursing Standard (through 2013), 2008, 23(1): 43.

[3] BERWICK D M, CALKINS D R, MCCANNON C J, et al. The 100 000 lives campaign: setting a goal and a deadline for improving health care quality[J]. Jama, 2006, 295(3): 324-327.

[4] HILLMAN K, BRISTOW P, CHEY T, et al. Antecedents to hospital deaths[J]. Internal Medicine Journal, 2001, 31(6): 343-348.

[5] MCQUILLAN P, PILKINGTON S, ALLAN A, et al. Confidential inquiry into quality of care before admission to intensive care[J]. British Medical Journal, 1998, 316(7148): 1853-1858.

[6] GOLDHILL D, MCNARRY A, MANDERSLOOT G, et al. A physiologically-based early warning score for ward patients: the association between score and outcome[J]. Anaesthesia, 2005, 60(6): 547-553.

[7] KNAUS W A, DRAPER E A, WAGNER D P, et al. APACHE II: a severity of disease classification system.[J]. Critical Care Medicine, 1985, 13(10): 818-829.

[8] ELLIOTT M, COVENTRY A. Critical care: the eight vital signs of patient monitoring[J]. British Journal of Nursing, 2012, 21(10): 621-625.

[9] JOHNSTON B W, BARRETT-JOLLEY R, KRIGE A, et al. Heart rate variability: Measurement and emerging use in critical care medicine[J]. Journal of the Intensive Care Society, 2020, 21 (2): 148-157.

[10] BODENES L, N’GUYEN Q T, LE MAO R, et al. Early heart rate variability evaluation enables to predict ICU patients’ outcome[J]. Scientific Reports, 2022, 12(1): 2498.

[11] HILLMAN K M, BRISTOW P J, CHEY T, et al. Duration of life-threatening antecedents prior to intensive care admission[J]. Intensive Care Medicine, 2002, 28(11): 1629-1634.

[12] BUIST M D, BURTON P R, BERNARD S A, et al. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care: A pilot study in a tertiary-care hospital[J]. Medical Journal of Australia, 1999, 171(1): 22-25.

[13] TASK FORCE OF THE EUROPEAN SOCIETY OF CARDIOLOGY THE NORTH AMERI CAN SOCIETY OF PACING E. Heart rate variability: standards of measurement, physiological interpretation, and clinical use[J]. Circulation, 1996, 93(5): 1043-1065.

[14] RAJENDRA ACHARYA U, PAUL JOSEPH K, KANNATHAL N, et al. Heart rate variability: a review[J]. Medical and Biological Engineering and Computing, 2006, 44: 1031-1051.

[15] FUJIWARA K, MIYAJIMA M, YAMAKAWA T, et al. Epileptic seizure prediction based on multivariate statistical process control of heart rate variability features[J]. IEEE Transactions on Biomedical Engineering, 2015, 63(6): 1321-1332.

[16] MEJÍA-MEJÍA E, MAY J M, ELGENDI M, et al. Differential effects of the blood pressure state on pulse rate variability and heart rate variability in critically ill patients[J]. NPJ Digital Medicine, 2021, 4(1): 82.

[17] KARMALI S N, SCIUSCO A, MAY S M, et al. Heart rate variability in critical care medicine: a systematic review[J]. Intensive Care Medicine Experimental, 2017, 5(1): 1-15.

[18] SZTAJZEL J, et al. Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system[J]. Swiss Medical Weekly, 2004, 134(35-36): 514-522.

[19] LA ROVERE M T, PINNA G D, MAESTRI R, et al. Short-term heart rate variability strongly predicts sudden cardiac death in chronic heart failure patients[J]. Circulation, 2003, 107(4): 565-570.

[20] WATKINSON P, BARBER V, PRICE J, et al. A randomised controlled trial of the effect of continuous electronic physiological monitoring on the adverse event rate in high risk medical and surgical patients[J]. Anaesthesia, 2006, 61(11): 1031-1039.

[21] KHANAM F T Z, AL-NAJI A, CHAHL J. Remote monitoring of vital signs in diverse nonclinical and clinical scenarios using computer vision systems: A review[J]. Applied Sciences, 2019, 9(20): 4474.

[22] SATHYANARAYANA S, SATZODA R K, SATHYANARAYANA S, et al. Vision-based patient monitoring: a comprehensive review of algorithms and technologies[J]. Journal of Ambient Intelligence and Humanized Computing, 2018, 9(2): 225-251.

[23] KHANAM F T Z, PERERA A G, AL-NAJI A, et al. Non-contact automatic vital signs monitoring of infants in a neonatal intensive care unit based on neural networks[J]. Journal of Imaging, 2021, 7(8): 122.

[24] ALIĆ B, ZAUBER T, WIEDE C, et al. Current methods for contactless optical patient diagnosis: a systematic review[J]. BioMedical Engineering OnLine, 2023, 22(1): 1-12.

[25] HAQUE A, GUO M, ALAHI A, et al. Towards vision-based smart hospitals: a system for tracking and monitoring hand hygiene compliance[C]//Machine Learning for Healthcare Conference. PMLR, 2017: 75-87.

[26] ESTEVA A, CHOU K, YEUNG S, et al. Deep learning-enabled medical computer vision[J]. NPJ Digital Medicine, 2021, 4(1): 1-9.

[27] JORGE J, VILLARROEL M, TOMLINSON H, et al. Non-contact physiological monitoring of post-operative patients in the intensive care unit[J]. NPJ Digital Medicine, 2022, 5(1): 1-11.

[28] VILLARROEL M, CHAICHULEE S, JORGE J, et al. Non-contact physiological monitoring of preterm infants in the neonatal intensive care unit[J]. NPJ Digital Medicine, 2019, 2(1): 1-18.

[29] WANG H, HUANG J, WANG G, et al. Surveillance camera-based cardio-respiratory monitoring for critical Patients in ICU[C]//2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE, 2022: 1-4.

[30] CHEN X, CHENG J, SONG R, et al. Video-based heart rate measurement: Recent advances and future prospects[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 68(10): 3600-3615.

[31] WANG W, DEN BRINKER A C, DE HAAN G. Discriminative signatures for remote-PPG[J]. IEEE Transactions on Biomedical Engineering, 2019, 67(5): 1462-1473.

[32] YU X, LAURENTIUS T, BOLLHEIMER C, et al. Noncontact monitoring of heart rate and heart rate variability in geriatric patients using photoplethysmography imaging[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 25(5): 1781-1792.

[33] WANG W, DEN BRINKER A C. Algorithmic insights of camera-based respiratory motion extraction[J]. Physiological Measurement, 2022, 43(7): 075004.

[34] BERGASA L M, NUEVO J, SOTELO M A, et al. Real-time system for monitoring driver vigilance[J]. IEEE Transactions on Intelligent Transportation Systems, 2006, 7(1): 63-77.

[35] GRAUMAN K, BETKE M, LOMBARDI J, et al. Communication via eye blinks and eyebrow raises: Video-based human-computer interfaces[J]. Universal Access in the Information Society, 2003, 2: 359-373.

[36] SZWOCH M, PIENIĄŻEK P. Eye blink based detection of liveness in biometric authentication systems using conditional random fields[C]//Computer Vision and Graphics: International Conference, ICCVG 2012, Warsaw, Poland, September 24-26, 2012. Proceedings. Springer, 2012: 669-676.

[37] BHUIYAN M N, RAHMAN M M, BILLAH M M, et al. Internet of things (IoT): a review of its enabling technologies in healthcare applications, standards protocols, security, and market opportunities[J]. IEEE Internet of Things Journal, 2021, 8(13): 10474-10498.

[38] RODRIGUES V F, RIGHI R R, COSTA C A, et al. HealthStack: providing an IoT middleware for malleable QoS service stacking for hospital 4.0 operating rooms[J]. IEEE Internet of Things Journal, 2022, 9(19): 18406-18430.

[39] HABIBZADEH H, DINESH K, SHISHVAN O R, et al. A survey of healthcare Internet of Things (HIoT): A clinical perspective[J]. IEEE Internet of Things Journal, 2019, 7(1): 53-71.

[40] GAHLOT S, REDDY S, KUMAR D. Review of smart health monitoring approaches with survey analysis and proposed framework[J]. IEEE Internet of Things Journal, 2018, 6(2): 2116- 2127.

[41] WANG J, HUANG D, FAN S, et al. PSDCE: Physiological signal-based double chaotic encryption for instantaneous E-healthcare services[J]. Future Generation Computer Systems, 2023, 141: 116-128.

[42] OLIER I, ORTEGA-MARTORELL S, PIERONI M, et al. How machine learning is impacting research in atrial fibrillation: implications for risk prediction and future management[J]. Cardiovascular Research, 2021, 117(7): 1700-1717.

[43] HUANG D M, HUANG J, QIAO K, et al. Deep learning-based lung sound analysis for intelligent stethoscope[J]. Military Medical Research, 2023, 10(1): 44.

[44] OH J, CHO D, PARK J, et al. Prediction and early detection of delirium in the intensive care unit by using heart rate variability and machine learning[J]. Physiological Measurement, 2018, 39(3): 035004.

[45] FAUST O, HONG W, LOH H W, et al. Heart rate variability for medical decision support systems: A review[J]. Computers in Biology and Medicine, 2022: 105407.

[46] LEAL A, PINTO M F, LOPES F, et al. Heart rate variability analysis for the identification of the preictal interval in patients with drug-resistant epilepsy[J]. Scientific Reports, 2021, 11(1): 1-11.

[47] DEGIORGIO C M, MILLER P, MEYMANDI S, et al. RMSSD, a measure of vagus-mediated heart rate variability, is associated with risk factors for SUDEP: the SUDEP-7 Inventory[J]. Epilepsy & Behavior, 2010, 19(1): 78-81.

[48] ACHARYA U R, FAUST O, KADRI N A, et al. Automated identification of normal and diabetes heart rate signals using nonlinear measures[J]. Computers in Biology and Medicine, 2013, 43 (10): 1523-1529.

[49] LAN K C, RAKNIM P, KAO W F, et al. Toward hypertension prediction based on PPG-derived HRV signals: A feasibility study[J]. Journal of Medical Systems, 2018, 42: 1-7.

[50] KORACH M, SHARSHAR T, JARRIN I, et al. Cardiac variability in critically ill adults: influence of sepsis[J]. Critical Care Medicine, 2001, 29(7): 1380-1385.

[51] CHEN W L, CHEN J H, HUANG C C, et al. Heart rate variability measures as predictors of inhospital mortality in ED patients with sepsis[J]. The American Journal of Emergency Medicine, 2008, 26(4): 395-401.

[52] 许彦坤, 石萍, 喻洪流. 基于成像式光电容积描记技术的人体生理参数检测研究进展[J]. 北京生物医学工程, 2017, 36(06): 648-654.

[53] SWINEHART D F. The beer-lambert law[J]. Journal of Chemical Education, 1962, 39(7): 333.

[54] 皮慧. 基于人脸图像的非接触式心率测量方法研究[D]. 东南大学, 2018.

[55] ABDULKAREEM K H, MOHAMMED M A, SALIM A, et al. Realizing an effective COVID19 diagnosis system based on machine learning and IOT in smart hospital environment[J]. IEEE Internet of Things Journal, 2021, 8(21): 15919-15928.

[56] ISLAM S R, KWAK D, KABIR M H, et al. The internet of things for health care: a comprehensive survey[J]. IEEE Access, 2015, 3: 678-708.

[57] ZHANG H, LI J, WEN B, et al. Connecting intelligent things in smart hospitals using NB-IoT [J]. IEEE Internet of Things Journal, 2018, 5(3): 1550-1560.

[58] YANG G, HE S, SHI Z, et al. Promoting cooperation by the social incentive mechanism in mobile crowdsensing[J]. IEEE Communications Magazine, 2017, 55(3): 86-92.

[59] JALEEL A, MAHMOOD T, HASSAN M A, et al. Towards medical data interoperability through collaboration of healthcare devices[J]. IEEE Access, 2020, 8: 132302-132319.

[60] LIVSHIZ-RIVEN I, KOYFMAN L, NATIV R, et al. Efficacy of covert closed-circuit television monitoring of the hand hygiene compliance of health care workers caring for patients infected with multidrug-resistant organisms in an intensive care unit[J]. American Journal of Infection Control, 2020, 48(5): 517-521.

[61] 杨益民, 李旭雯, 罗志昌, 等. 应用光电容积脉搏波法研制新型血流参数监护系统[J/OL]. 中国医疗器械信息, 2001(05): 6-8. DOI: 10.15971/j.cnki.cmdi.2001.05.003.

[62] 刘祎, 欧阳健飞. 基于人脸视频的非接触式心率测量方法[J/OL]. 纳米技术与精密工程, 2016, 14(01): 76-79. DOI: 10.13494/j.npe.20140108.

[63] VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light.[J]. Optics Express, 2008, 16(26): 21434-21445.

[64] POH M Z, MCDUFF D J, PICARD R W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.[J]. Optics Express, 2010, 18(10): 10762- 10774.

[65] LEWANDOWSKA M, RUMIŃSKI J, KOCEJKO T, et al. Measuring pulse rate with a webcam—a non-contact method for evaluating cardiac activity[C]//2011 Federated Conference on Computer Science and Information Systems (FedCSIS). IEEE, 2011: 405-410.

[66] WANG W, DEN BRINKER A C, STUIJK S, et al. Algorithmic principles of remote PPG[J]. IEEE Transactions on Biomedical Engineering, 2016, 64(7): 1479-1491.

[67] DE HAAN G, JEANNE V. Robust pulse rate from chrominance-based rPPG[J]. IEEE Transactions on Biomedical Engineering, 2013, 60(10): 2878-2886.

[68] TOMINAGA S. Dichromatic reflection models for a variety of materials[J]. Color Research & Application, 1994, 19(4): 277-285.

[69] DE HAAN G, VAN LEEST A. Improved motion robustness of remote-PPG by using the blood volume pulse signature[J]. Physiological Measurement, 2014, 35(9): 1913.

[70] WANG W, STUIJK S, DE HAAN G. A novel algorithm for remote photoplethysmography: Spatial subspace rotation[J]. IEEE Transactions on Biomedical Engineering, 2015, 63(9): 1974- 1984.

[71] ŠPETLÍK R, FRANC V, MATAS J. Visual heart rate estimation with convolutional neural network[C]//Proceedings of the British Machine Vision Conference, Newcastle, UK. 2018: 3-6.

[72] CHEN W, MCDUFF D. Deepphys: Video-based physiological measurement using convolutional attention networks[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 349-365.

[73] YU Z, PENG W, LI X, et al. Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 151-160.

[74] NIU X, SHAN S, HAN H, et al. Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation[J]. IEEE Transactions on Image Processing, 2019, 29: 2409- 2423.

[75] SUN Z, LI X. Contrast-phys: Unsupervised video-based remote physiological measurement via spatiotemporal contrast[C]//European Conference on Computer Vision. Springer, 2022: 492-510.

[76] 杨雯. 基于人脸视频的非接触式心测量算法的研究与实现[D]. 北京邮电大学, 2019.

[77] BARTULA M, TIGGES T, MUEHLSTEFF J. Camera-based system for contactless monitoring of respiration[C]//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2013: 2672-2675.

[78] JANSSEN R, WANG W, MOÇO A, et al. Video-based respiration monitoring with automatic region of interest detection[J]. Physiological Measurement, 2015, 37(1): 100.

[79] ROCQUE M. Fully automated contactless respiration monitoring using a camera[C]//2016 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2016: 478-479.

[80] PEREIRA C B, YU X, BLAZEK V, et al. Robust remote monitoring of breathing function by using infrared thermography[C]//2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2015: 4250-4253.

[81] PEREIRA C B, YU X, GOOS T, et al. Noncontact monitoring of respiratory rate in newborn infants using thermal imaging[J]. IEEE Transactions on Biomedical Engineering, 2018, 66(4): 1105-1114.

[82] JAGADEV P, GIRI L I. Non-contact monitoring of human respiration using infrared thermography and machine learning[J]. Infrared Physics & Technology, 2020, 104: 103117.

[83] MIRMOHAMADSADEGHI L, FALLET S, MOSER V, et al. Real-time respiratory rate estimation using imaging photoplethysmography inter-beat intervals[C]//2016 Computing in Cardiology Conference (CinC). IEEE, 2016: 861-864.

[84] IOZZA L, LÁZARO J, CERINA L, et al. Monitoring breathing rate by fusing the physiological impact of respiration on video-photoplethysmogram with head movements[J]. Physiological Measurement, 2019, 40(9): 094002.

[85] LUGUERN D, MACWAN R, BENEZETH Y, et al. Wavelet variance maximization: a contactless respiration rate estimation method based on remote photoplethysmography[J]. Biomedical Signal Processing and Control, 2021, 63: 102263.

[86] MASSARONI C, NICOLO A, SACCHETTI M, et al. Contactless methods for measuring respiratory rate: A review[J]. IEEE Sensors Journal, 2020, 21(11): 12821-12839.

[87] LÁZARO J, GIL E, BAILÓN R, et al. Deriving respiration from the pulse photoplethysmographic signal[C]//2011 Computing in Cardiology. IEEE, 2011: 713-716.

[88] LÁZARO J, NAM Y, GIL E, et al. Respiratory rate derived from smartphone-camera-acquired pulse photoplethysmographic signals[J]. Physiological Measurement, 2015, 36(11): 2317.

[89] MOODY G B, MARK R G. A database to support development and evaluation of intelligent intensive care monitoring[C]//Computers in Cardiology 1996. IEEE, 1996: 657-660.

[90] 陈真诚, 牛春望, 朱健铭, 等. 一种利用经验模态分解算法的光电容积脉搏波信号中提取 呼吸波的方法研究[J/OL]. 生物医学工程研究, 2019, 38(02): 134-139. DOI: 10.19529/j.cnk i.1672-6278.2019.02.02.

[91] BENNETT S L, GOUBRAN R, KNOEFEL F. Comparison of motion-based analysis to thermalbased analysis of thermal video in the extraction of respiration patterns[C]//2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2017: 3835-3839.

[92] LORATO I, STUIJK S, MEFTAH M, et al. Multi-camera infrared thermography for infant respiration monitoring[J]. Biomedical Optics Express, 2020, 11(9): 4848-4861.

[93] JAKKAEW P, ONOYE T. Non-contact respiration monitoring and body movements detection for sleep using thermal imaging[J]. Sensors, 2020, 20(21): 6307.

[94] BRIEVA J, PONCE H, MOYA-ALBOR E. A contactless respiratory rate estimation method using a hermite magnification technique and convolutional neural networks[J]. Applied Sciences, 2020, 10(2): 607.

[95] ZHAN Q, HU J, YU Z, et al. Revisiting motion-based respiration measurement from videos[C]// 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020: 5909-5912.

[96] FÖLDESY P, ZARÁNDY Á, SZABÓ M. Reference free incremental deep learning model applied for camera-based respiration monitoring[J]. IEEE Sensors Journal, 2020, 21(2): 2346- 2352.

[97] VILLARROEL M, JORGE J, MEREDITH D, et al. Non-contact vital-sign monitoring of patients undergoing haemodialysis treatment[J]. Scientific Reports, 2020, 10(1): 1-21.

[98] KHANAM F T Z, AL-NAJI A, PERERA A G, et al. Remote vital signs monitoring in neonatal intensive care unit using a digital camera[J]. International Journal of Biomedical and Biological Engineering, 2022, 16(10): 138-144.

[99] 于清. 基于光电容积脉搏波成像技术的移动端心率分析引擎的设计与实现[D]. 北京邮电 大学, 2018.

[100] AARTS L A, JEANNE V, CLEARY J P, et al. Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—A pilot study[J]. Early Human Development, 2013, 89(12): 943-948.

[101] MESTHA L K, KYAL S, XU B, et al. Towards continuous monitoring of pulse rate in neonatal intensive care unit with a webcam[C]//2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2014: 3817-3820.

[102] VAN GASTEL M, BALMAEKERS B, OETOMO S B, et al. Near-continuous non-contact cardiac pulse monitoring in a neonatal intensive care unit in near darkness[C]//Optical Diagnostics and Sensing XVIII: Toward Point-of-care Diagnostics: Vol. 10501. SPIE, 2018: 230-238.

[103] CHEN Q, JIANG X, LIU X, et al. Non-contact heart rate monitoring in neonatal intensive care unit using RGB camera[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2020: 5822-5825.

[104] SAHOO N N, MURUGESAN B, DAS A, et al. Deep learning based non-contact physiological monitoring in neonatal intensive care Unit[C]//2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2022: 1327-1330.

[105] MAURYA L, ZWIGGELAAR R, CHAWLA D, et al. Non-contact respiratory rate monitoring using thermal and visible imaging: a pilot study on neonates[J]. Journal of Clinical Monitoring and Computing, 2023, 37(3): 815-828.

[106] ZENG Y, SONG X, CHEN H, et al. A multi-modal clinical dataset for critically-ill and premature infant monitoring: EEG and videos[C]//2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI). IEEE, 2022: 1-5.

[107] RASCHE S, TRUMPP A, WALDOW T, et al. Camera-based photoplethysmography in critical care patients[J]. Clinical Hemorheology and Microcirculation, 2016, 64(1): 77-90.

[108] RASCHE S, TRUMPP A, SCHMIDT M, et al. Remote photoplethysmographic assessment of the peripheral circulation in critical care patients recovering from cardiac surgery[J]. Shock, 2019, 52(2): 174-182.

[109] KUBLANOV V, PURTOV K, KONTOROVICH M. Video-based vital sign monitoring system of patients in intensive care unit[C]//2017 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON). IEEE, 2017: 556-560.

[110] KUBLANOV V, PURTOV K, BELKOV D. Remote photoplethysmography for the neuroelectrostimulation procedures monitoring the possibilities of remote photoplethysmography application for the analysis of high frequency parameters of heart rate variability[M]//International Conference on Bio-inspired Systems and Signal Processing. SciTePress, 2017.

[111] LIU Z, HUANG B, LIN C L, et al. Contactless respiratory rate monitoring for iCU patients based on unsupervised learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 6004-6013.

[112] TAN X, HU M, ZHAI G, et al. Unobtrusive respiratory monitoring system for intensive care[C]// ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023: 1-5.

[113] MORIDANI M K, SETAREHDAN S K, NASRABADI A M, et al. Analysis of heart rate variability as a predictor of mortality in cardiovascular patients of intensive care unit[J]. Biocybernetics and Biomedical Engineering, 2015, 35(4): 217-226.

[114] SHAFFER F, GINSBERG J P. An overview of heart rate variability metrics and norms[J]. Frontiers in Public Health, 2017: 258.

[115] MAYAMPURATH A, VOLCHENBOUM S L, SANCHEZ-PINTO L N. Using photoplethysmography data to estimate heart rate variability and its association with organ dysfunction in pediatric oncology patients[J]. NPJ Digital Medicine, 2018, 1(1): 29.

[116] KHANDOKER A H, KARMAKAR C K, PALANISWAMI M. Comparison of pulse rate variability with heart rate variability during obstructive sleep apnea[J]. Medical Engineering & Physics, 2011, 33(2): 204-209.

[117] WANG H, HUANG J, WANG G, et al. Contactless patient care using hospital IoT: CCTV Camera-Based physiological monitoring in ICU[J]. IEEE Internet of Things Journal, 2024, 11 (4): 5781-5797.

[118] DUNAEVA A, KONOVALOVA D, KOSTOUSOV V. Video analysis methods for remote measurement of respiration characteristics and heart rate variability[C]//2020 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT). IEEE, 2020: 0171-0174.

[119] JORGE J, VILLARROEL M, CHAICHULEE S, et al. Non-contact monitoring of respiration in the neonatal intensive care unit[C]//2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). IEEE, 2017: 286-293.

[120] SUN G, NAKAYAMA Y, DAGDANPUREV S, et al. Remote sensing of multiple vital signs using a CMOS camera-equipped infrared thermography system and its clinical application in rapidly screening patients with suspected infectious diseases[J]. International Journal of Infectious Diseases, 2017, 55: 113-117.

[121] CASALINO G, CASTELLANO G, PASQUADIBISCEGLIE V, et al. Contact-less real-time monitoring of cardiovascular risk using video imaging and fuzzy inference rules[J]. Information, 2018, 10(1): 9.

[122] NEGISHI T, ABE S, MATSUI T, et al. Contactless vital signs measurement system using RGB-thermal image sensors and its clinical screening test on patients with seasonal influenza [J]. Sensors, 2020, 20(8): 2171.

[123] EBRAHIMZADEH E, KALANTARI M, JOULANI M, et al. Prediction of paroxysmal Atrial Fibrillation: A machine learning based approach using combined feature vector and mixture of expert classification on HRV signal[J]. Computer Methods and Programs in Biomedicine, 2018, 165: 53-67.

[124] VIGIER M, VIGIER B, ANDRITSCH E, et al. Cancer classification using machine learning and HRV analysis: preliminary evidence from a pilot study[J]. Scientific Reports, 2021, 11(1): 22292.

[125] UNURSAIKHAN B, TANAKA N, SUN G, et al. Development of a novel web camera-based contact-free major depressive disorder screening system using autonomic nervous responses induced by a mental task and its clinical application[J]. Frontiers in Physiology, 2021, 12: 642986.

[126] SAURAV S, GIDDE P, SAINI R, et al. Real-time eye state recognition using dual convolutional neural network ensemble[J]. Journal of Real-Time Image Processing, 2022, 19(3): 607-622.

[127] BACIVAROV I, IONITA M, CORCORAN P. Statistical models of appearance for eye tracking and eye-blink detection and measurement[J]. IEEE Transactions on Consumer Electronics, 2008, 54(3): 1312-1320.

[128] CECH J, SOUKUPOVA T. Real-time eye blink detection using facial landmarks[M]//21th Computer Vision Winter Workshop (CVWW’16). 2016: 1-8.

[129] YANG H Y, JIANG X H, WANG L, et al. Eye statement recognition for driver fatigue detection based on gabor wavelet and hmm[C]//Applied Mechanics and Materials: Vol. 128. Trans Tech Publ, 2012: 123-129.

[130] ZHOU L, WANG H. Open/closed eye recognition by local binary increasing intensity patterns[C]//2011 IEEE 5th International Conference on Robotics, Automation and Mechatronics (RAM). IEEE, 2011: 7-11.

[131] SONG F, TAN X, LIU X, et al. Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients[J]. Pattern Recognition, 2014, 47(9): 2825-2838.

[132] DONG Y, ZHANG Y, YUE J, et al. Comparison of random forest, random ferns and support vector machine for eye state classification[J]. Multimedia Tools and Applications, 2016, 75: 11763-11783.

[133] LIU X, TAN X, CHEN S. Eyes closeness detection using appearance based methods[C]// Intelligent Information Processing VI: 7th IFIP TC 12 International Conference, IIP 2012, Guilin, China, October 12-15, 2012. Proceedings 7. Springer, 2012: 398-408.

[134] KIM K W, HONG H G, NAM G P, et al. A study of deep CNN-based classification of open and closed eyes using a visible light camera sensor[J]. Sensors, 2017, 17(7): 1534.

[135] PAN G, SUN L, WU Z, et al. Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C]//2007 IEEE 11th International Conference on Computer Vision. IEEE, 2007: 1-8.

[136] GOU C, WU Y, WANG K, et al. A joint cascaded framework for simultaneous eye detection and eye state estimation[J]. Pattern Recognition, 2017, 67: 23-31.

[137] ZHAO L, WANG Z, ZHANG G, et al. Eye state recognition based on deep integrated neural network and transfer learning[J]. Multimedia Tools and Applications, 2018, 77: 19415-19438.

[138] CHOWDHURY A I, NILOY A R, SHARMIN N, et al. A deep learning based approach for real-time driver drowsiness detection[C]//2021 5th International Conference on Electrical Engineering and Information Communication Technology (ICEEICT). IEEE, 2021: 1-5.

[139] MIDDLETON P M. Practical use of the Glasgow Coma Scale; a comprehensive narrative review of GCS methodology[J]. Australasian Emergency Nursing Journal, 2012, 15(3): 170- 183.

[140] NAVED S A, SIDDIQUI S, KHAN F H. APACHE-II score correlation with mortality and length of stay in an intensive care unit[J]. Journal of the College of Physicians and Surgeons Pakistan, 2011, 21(1): 4.

[141] TIAN Y, YAO Y, ZHOU J, et al. Dynamic APACHE II score to predict the outcome of intensive care unit patients[J]. Frontiers in Medicine, 2022, 8: 744907.

[142] TARASSENKO L, VILLARROEL M, GUAZZI A, et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models[J]. Physiological Measurement, 2014, 35(5): 807.

[143] LUCAS B, KANADE T. An iterative image registration technique with an application to stereo vision[C]//International Joint Conference on Artificial Intelligence: Vol. 81. 1981.

[144] WANG W, STUIJK S, DE HAAN G. Exploiting spatial redundancy of image sensor for motion robust rPPG[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(2): 415-425.

[145] WANG W, STUIJK S, DE HAAN G. Living-skin classification via remote-PPG[J]. IEEE Transactions on Biomedical Engineering, 2017, 64(12): 2781-2792.

[146] BAZAREVSKY V, KARTYNNIK Y, VAKUNOV A, et al. Blazeface: Sub-millisecond neural face detection on mobile gpus[A]. 2019.

[147] TAN K S, SAATCHI R, ELPHICK H, et al. Real-time vision based respiration monitoring system[C]//2010 7th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP 2010). IEEE, 2010: 770-774.

[148] KLEIGER R E, MILLER J P, BIGGER JR J T, et al. Decreased heart rate variability and its association with increased mortality after acute myocardial infarction[J]. The American Journal of Cardiology, 1987, 59(4): 256-262.

[149] MALIK M, BIGGER J T, CAMM A J, et al. Heart rate variability: Standards of measurement, physiological interpretation, and clinical use[J]. European Heart Journal, 1996, 17(3): 354-381.

[150] CLIFFORD G D, TARASSENKO L. Quantifying errors in spectral estimates of HRV due to beat replacement and resampling[J]. IEEE Transactions on Biomedical Engineering, 2005, 52 (4): 630-638.

[151] GROSSMAN P, TAYLOR E W. Toward understanding respiratory sinus arrhythmia: Relations to cardiac vagal tone, evolution and biobehavioral functions[J]. Biological Psychology, 2007, 74(2): 263-285.

[152] BURR R L. Interpretation of normalized spectral heart rate variability indices in sleep research: a critical review[J]. Sleep, 2007, 30(7): 913-919.

[153] TULPPO M P, MAKIKALLIO T H, TAKALA T, et al. Quantitative beat-to-beat analysis of heart rate dynamics during exercise[J]. American Journal of Physiology-heart and Circulatory Physiology, 1996, 271(1): H244-H252.

[154] LIPPMAN N, STEIN K M, LERMAN B B. Comparison of methods for removal of ectopy in measurement of heart rate variability[J]. American Journal of Physiology-Heart and Circulatory Physiology, 1994, 267(1): H411-H418.

[155] FOR THE ADVANCEMENT OF MEDICAL INSTRUMENTATION A, et al. Cardiac monitors, heart rate meters, and alarms[J]. American National Standard (ANSI/AAMI EC13: 2002) Arlington, VA, 2002: 1-87.

[156] BERGESE S D, MESTEK M L, KELLEY S D, et al. Multicenter study validating accuracy of a continuous respiratory rate measurement derived from pulse oximetry: a comparison with capnography[J]. Anesthesia and Analgesia, 2017, 124(4): 1153.

[157] BLEYER A J, VIDYA S, RUSSELL G B, et al. Longitudinal analysis of one million vital signs in patients in an academic medical center[J]. Resuscitation, 2011, 82(11): 1387-1392.

[158] SMITH G B, PRYTHERCH D R, MEREDITH P, et al. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death[J]. Resuscitation, 2013, 84(4): 465-470.

[159] SCHÄFER A, VAGEDES J. How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram[J]. International Journal of Cardiology, 2013, 166(1): 15-29.

[160] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05): Vol. 1. IEEE, 2005: 886-893.

[161] ZHENG Z, YANG J, YANG L. A robust method for eye features extraction on color image[J]. Pattern Recognition Letters, 2005, 26(14): 2252-2261.

[162] TAN C W, KUMAR A. Automated segmentation of iris images using visible wavelength face images[C]//Cvpr 2011 Workshops. IEEE, 2011: 9-14.

[163] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.

[164] FRIESE R S. Sleep and recovery from critical illness and injury: a review of theory, current practice, and future directions[J]. Critical Care Medicine, 2008, 36(3): 697-705.