Detecting Driver Drowsiness by Analyzing HRV and Facial Expression with Hierarchical Processing
Because of my love of national parks, I often drive long distances, and I will often feel mind numb if my companion isn't talking to me the entire time. Even though I've driven all roads safely so far, every now and then when I pass an accident car on the highway, I think about just how tired a driver must be to take a break. Luckily, our smartwatch allows for HRV monitoring, and I use it all the time to keep track of anxiety and stress levels. So, I started to gather information to research if it's really possible to use HRV and driver face learning to tell the driver: it's time for you to take a break.
I Background
A. Heart Rate Variability
HRV is defined as a physiological metric, which is widely applied in clinical diagnosis for illnesses like post-diabetic depression and survival prognosis in premature infants. It is easliy obtained from the ECG, making HRV is used widely. Currently, wearable sensor technology leads to monitoring physiological factors like HRV to be commonly used. In the research of Ajakwe et al. [1], some key characteristics in wearables for healthcare delivery highlight the potential of the application of this technology in the context of driving safety.
B. Data Analyzation
Previous studies indicate multi-feature fusion, for instance, combining CNN and Inception V3-LSTM outperforms single parameter type [2]. This project builds on these findings by employing HRV data and facial expressions sequentially, which provides more accurate results.
C. Driving Safety Detection Technology
Numerous studies have examined how physiological signals and outward manifestations interact to determine a driver's state. An actual use of HRV monitoring for vehicle safety is presented by Lu et al. [4], who show how consumer wearable devices can be used to detect driver tiredness in real-road driving situations. Likewise, the research of Kerautret et al. [5] addresses the importance of real-time cardiac measurements in the context of driver stress and anticipated hazards, which offers insights into the use of physiological data in real-world driving scenarios.
II Implementation
The implementation of this research includes several parts that consist of the main research steps and work together to form a dependable and efficient system.
A. Hardware and Software
In this case, we select Fitbit or Apple Watches for their accuracy in recording HRV data. These devices will be in the sync with facial expression analysis device, that is in-vehicle cameras, to gather and transmit data instantly. To guarantee the precision and dependability of HRV measurement and its analysis afterward, SDNN and RMSSD are employed to test.
A. Data Analysis
Example of a Python script for HRV data training
First, the script reads a list of heart beats and their patterns from a file and separates the answers it's supposed to learn (like whether a heartbeat is normal or not). Then, it splits this list into two parts - one part (70%) for learning and the other (30%) to test how well it has learned.
This project will heavily rely on visualization tools, especially those created with R. Both the HRV data and the outcomes of the facial expression analysis will be visualized using scripts.
However, due to no access to the real data file (.csv), the desired ouput shows with assumed data.
Output
III Method
We speculate that some HRV measurements, especially RMSSD and SDNN, are predictive of the shift from wakefulness to sleepiness. Furthermore, we propose that facial expressions offer essential additional information that improves detection precision.
A. Paticipant Selection
To take part in our real-world test, we intend to enlist 100 drivers from a varied population. To provide a wide representation of driving circumstances and behaviors, this sample size was selected. A variety of age groups and driving backgrounds, including both professional and recreational drivers, will be represented in the participant selection process.
B. Parameters
HRV Metrics (SDNN and RMSSD): These metrics are the primary physiological indicators that will be measured using wearable devices. Facial expression is a visual indicator of drowsiness captured by the camera, focusing on cues such as yawning, closed eyes, and nodding. Driver drowsiness was measured by HRV and facial expression variability. The study obtained the degree of drowsiness by collecting and analyzing HRV data and facial expression data.
Data will be collected using devices such as Fitbits or Apple Watches. In order to detect whether a driver is driving fatigued, these devices will continuously record the driver's HRV measurements, specifically SDNN and RMSSD. This study applies a convolutional neural network (CNN) for analyzing facial expressions and a random forest algorithm for HRV data analysis. This system uses statistical techniques to evaluate important performance metrics including precision, accuracy, and F1 scores to determine its success.
IV Conclusion
In summary, this research proposal presents a comprehensive approach to recognizing sleepy driving, which is a key issue for road safety. It hopes to combine smart wearable devices and image analysis technology to create a system that combines heart rate variability data and facial expressions to accurately detect indicators of driver sleepiness, thereby reducing sleep-related accidents. We hypothesize that a high accuracy rate for identifying sleepy driving will result from the combined analysis of facial expression and HRV data. The accuracy is anticipated to be between 80% and 90%, which is a considerable improvement over systems that only use facial expression analysis or HRV. A robust negative association is expected between the drowsiness level and HRV measures, particularly SDNN. The chance of feeling sleepy rises when SDNN falls. This correlation is anticipated to be statistically significant, proving the validity of HRV as a gauge of driver weariness.
Along with making a substantial contribution to the fields of intelligent transportation systems and driver safety technologies, this research intends to develop the technology for detecting drowsy driving. Our goal is to decrease the number of accidents caused by intoxicated drivers by developing better knowledge and more precise means of identifying driver drowsiness. This will increase road safety for all users.
REFERENCES
[1] S. O. Ajakwe, C. I. Nwakanma, D. Kim, and J. Lee, "Key wearable device technologies parameters for innovative healthcare delivery in B5G network: A review," IEEE Access, vol. 10, pp. 49956-49974, 2022.
[2] Y. Zhao, K. Xie, Z. Zou, and J. He, "Intelligent recognition of fatigue and sleepiness based on InceptionV3-LSTM via multi-feature fusion," IEEE Access, vol. 8, pp. 144205-144217, 2020.
[3] M. Amin, K. Ullah, M. Asif, H. Shah, A. Mehmood, and M. A. Khan, "Real-world driver stress recognition and diagnosis based on multimodal deep learning and fuzzy EDAS approaches," MDPI AG, 2023.
[4] K. Lu, J. Karlsson, A. S. Dahlman, B. A. Sjoqvist, and S. Candefjord, "Detecting driver sleepiness using consumer wearable devices in manual and partial automated real-road driving," IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 5, pp. 4801-4810, 2022.
[5] L. Kerautret, S. Dabic, and J. Navarro, "Detecting driver stress and hazard anticipation using real‐time cardiac measurement: A simulator study," Wiley, 2022.
Comments
Post a Comment