ISSN: 0970-938X (Print) | 0976-1683 (Electronic)
An International Journal of Medical Sciences
- Biomedical Research (2014) Volume 25, Issue 3
1Department of CSE, Vels University, Pallavaram, Chennai 600 117, India
2AI-Rehab Research Group, Universiti Malaysia Perlis (UniMAP), Kampus Pauh Putra, 02600 Arau, Perlis, Malaysia
Accepted February 19 2014
In recent years, driver drowsiness and driver inattention are the major causes for road accidents leading to severe traumas such as physical injuries, deaths, and economic losses. This necessitates the need for a system that can alert the driver on time, whenever he is drowsy or inattentive. Previous research works report the detection of either drowsiness or inattention only. In this work, we aim to develop a system that can detect hypovigilance, which includes both drowsiness and inattention, using Electromyogram (EMG) signals. Fifteen male volunteers participated in the data collection experiment where they were asked to drive for two hours at 3 different times of the day (00:00 – 02:00 hrs, 03:00 – 05:00 hrs and 15:00 – 17:00 hrs) when their circadian rhythm is low. The results indicate that the standard deviation feature of EMG is efficient to detect hypovigilance with a maximum classification accuracy of 89%.
Hypovigilance, inattention, fatigue, sEMG, driver safety
According to the statistics released by the World Health Organisation more than 1.2 million people die each year on the world’s roads, and between 20 and 50 million suffer non-fatal injuries due to road accidents [1]. The National Highway Traffic Safety Administration (NHTSA), USA conservatively estimated 100000 police reports on vehicle crashes each year which were the direct results of driver drowsiness. Such accidents also result in approximately 1550 deaths, 71000 injuries and $12.5 billion in monetary losses [2]. The National Sleep Foundation (NSF) reported that in 2009, 54% of adult drivers had driven a vehicle while feeling drowsy and 28% had actually fallen asleep [3]. Driver inattention includes concentrating on secondary tasks like using cell phone, music player, etc while driving. In the year 2008, NHTSA estimated 5870 deaths, 350,000 injuries and 745,000 property damages due to driver distraction (NHTSA’s National Centre for Statistics and Analysis, America, 2009 report). In US alone, damages of $43 billion per year has been estimated due to cell phone related crashes [4]. A naturalistic driving study found that 78% of crashes and 65% of near-crashes included inattention as a contributing factor [5]. According to the United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP), around 1 million deaths, 23 million injuries and 10 million vehicles are exposed to the road accidents in their region every year. They also conclude that more than 85% of the causalities due to road accidents are from the developing countries [6]. All these statistics convey that driver hypovigilance, which includes both driver drowsiness and driver inattention is one of the main factors for road accidents throughout the world. Most of these accidents can be avoided, if the fatigue or distracted driver is alerted on time. This requires an efficient hypovigilance detection system that can detect both drowsiness and inattention to be developed.
The term ‘Hypovigilance’ is derived from two words ‘Hypo’ & ‘Vigilance’. ‘Hypo’ originates from a Greek word meaning ‘diminished’ and ‘vigilance’ means ‘alertness’. So, ‘hypovigilance’ together means ‘diminished alertness,’ and can be defined as anything that causes a decrease in paying a close and continuous attention. Impairment of alertness in a driver may be due to prolonged sleepiness or short term inattention. It may lead the driver to lose control of the vehicle which in turn can lead to accidents like crashing of the vehicle onto other vehicles or stationary surroundings. In order to prevent these devastating incidents, the state of the driver should be continuously monitored.
Driver fatigue is synonymously used with driver drowsiness. Driver drowsiness mainly depends on (i) the quality of the last sleep, (ii) the circadian rhythm (time of day) and (iii) the increase in the duration of the driving task [7-9]. Recent statistics from countries such as the United Kingdom, the United States, Israel, Finland, and France indicate that an increased number of vehicle accidents caused by driver drowsiness occurred during the peak drowsiness periods sleeping hours of 02:00 to 06:00 hours and 14:00 to 16:00 hours. During these time frames, the circadian rhythm shows higher chance of getting drowsy and drivers are 3 times more likely to fall asleep at these times than at 10:00 hours or at 19:00 hours [10]. So researchers have manipulated drowsiness by asking the driver to drive for longer duration in a monotonous environment during times of day when their circadian rhythm is low. The term inattention and distraction has been used synonymously. According to Hedlund et al. “Distraction involves a diversion of attention from driving, because the driver is temporarily focusing on an object, person, task, or event not related to driving, which in turn reduces the awareness, decision-making, and/or performance of the driver, leading to an increased risk of corrective actions, near-crashes, or crashes” [11]. Since distractions may not produce immediate consequences, it would be better if a driver who is distracted is alerted on time. Researchers have mainly dealt with two types of distraction namely: cognitive distraction (e.g., talking in the cell phone) and visual distraction (eg. Texting a sms) [12-15]
In general any one of the following measures has been used for measuring either drowsiness or distraction. They are 1) Behavioral measures 2) Vehicle based measures and 3) Physiological measures.
The behavior of the driver, including yawning, eye closure, eye blinking, head pose, etc., is monitored through a camera and the driver is alerted if any of the drowsiness or inattention symptoms are detected [16-18]. Visionbased measures are an efficient way to detect hypovigilance and some real-time products have been developed such as Seeing Machines [19] and Lexus [20]. However, when evaluating the available real-time detection systems, Lawrence et al. observed that different illumination conditions affect the reliability and accuracy of the measurements [21]. Vehicle based measures are useful to measure hypovigilance when the driver’s lack of vigilance has an effect on vehicle control or deviation. However, in some cases there was no impact on vehicle based parameters even if the driver was drowsy [7]. This makes vehicle based drowsiness detection system unreliable. Though intrusive, physiological signal based measures are reliable and accurate as they provide the true internal state of the driver. Electrooculogram (EoG) signal measures the electric potential difference between the cornea and the retina by generating an electrical field in context to the orientation of the eyes [22-25]. Researchers have used EoG to track the eye movement, which is then used to detect drowsiness or inattention. Electroencephalogram (EEG) signal has various frequency bands such as the delta band (0.5–4 Hz) corresponding to the sleep activity, the theta band (4–8 Hz) related to drowsiness, the alpha band (8–13 Hz) corresponding to relaxation and creativity, and the beta band (13–25 Hz) corresponding to activity and alertness [26,27]. Inattention is related to Beta band and drowsiness is related to theta band. Many researchers have extracted features from these bands and have classified drowsiness or inattention [26-29]. Heart rate and Heart Rate Variability (HRV) signals that are derived from Electrocardiogram (ECG) signals, is also found to vary significantly during the different states of the driver such as alertness, drowsiness and inattentiveness [8,14,30,31]. Researchers have worked on HRV signals that are derived from ECG and analyzed features such as RR Interval (RRI) to detect driver drowsiness or inattention [29,30,32-34]. Also by extracting and analysing the Low-Frequency (LF) (0.04Hz - 0.15Hz) and High- Frequency (HF) (0.15Hz - 0.45Hz) components, they have found the ratio of the LF to HF to decrease progressively as driver moves from alert to drowsy state [35-37]. Researchers have also observed the heart rate (HR) to vary significantly during the different states such as inattention and drowsiness [8,14,38,39].
The surface electromyography (sEMG) is a non-invasive index of the level of muscle activation [40]. In recent years, more attention is focused on the analysis of EMG signals during sleep because of known aberrations during REM. Researchers have also attempted to quantify muscle activity from sEMG during sleep [41,42]. However, very few works have been done in the context of driver fatigue [43]. The advantages and limitations of various measures are given in Table 1.
Among the physiological parameters such as EEG, EOG, ECG and EMG, ECG and EMG can be measured in a less intrusive manner. The EEG signals require a large number of electrodes to be placed on the scalp of the driver and the electrodes for measuring EoG signal have to be placed near the eye which can hinder vision when driving. Researchers have used non-intrusive means to measure ECG by placing electrodes in the steering wheel [48] or on the driver’s seat [49] (Yu X, University of Minnesota, Duluth report 2009, Personal Communications). By considering the advantages of physiological measures over the other measures and the availability of non-intrusive measurement modalities, ECG and EMG signals are chosen to detect hypovigilance in this work. In this experiment, we tend to find if alert, fatigue and distraction states can been traced from sEMG signals.
Protocol
Driver fatigue mainly depends on the: (i) the quality of the last sleep, (ii) the circadian rhythm (time of day) and (iii) the increase in the duration of driving task [7-9].
Hence, the protocol was designed to take data during three different times of the day (00:00 – 02:00 hrs, 03:00 – 05:00 hrs and 15:00 – 17:00 hrs). The lighting conditions in laboratory were also simulated accordingly. A simulator game, TORCS, was used to enable driving and the maximum speed was set to 70 km/hr in order to create a monotonous environment [50].
The protocol used to obtain the data for normal, fatigue and distraction states in the driver is as shown in Figure 1. In the first 15 minutes, normal signals were obtained from the subjects, as they were only driving without any distraction. Then in order to stimulate visual distraction, the subjects were asked to reply back for four text messages that were sent during the next five minutes, with questions related to their hobbies and general interest such as food, sports, etc. Cognitive distraction was stimulated by making a phone call to the subject at the 25th minute; the subjects responded to the arithmetic questions that were asked. The level of difficulty was based on the speed of answering by the subject. This was done so that the subject could think for a while, rather than spontaneously answer. In order to stimulate drowsiness, the subjects drove for an hour and half without any disturbance.
Subjects and Experimental setup
The experimental set up is shown in Figure 2. A fixed table top driving simulator was used to enable driving. In the data collection experiment, two physiological signals (sEMG and ECG) were obtained simultaneously. The experimental results obtained from ECG signal has been published in [39]. The sEMG data was collected by placing corresponding electrodes in the trapezius muscle and PHYWE (Germany) was used for the sEMG data collection with a sampling frequency of 256 Hz. The video of the subject’s face while driving was also recorded for the entire 2 hour session using an IR camera (30 fps). Fifteen healthy university male students (mean age ± SD = 25.6 ± 2.5 years, range 23-32) with valid driving license participated in the experiment after providing written consent. All the experiments were conducted during the times of the day when fatigue is much prevalent (00:00 – 02:00 hrs, 03:00 – 05:00 hrs and 15:00 – 17:00 hrs). All the fifteen subjects returned on different days to finish the three sessions.
Data Analysis
Preprocessing
The EMG signals obtained during driving are highly prone to movement artifacts and noise. Removing these artifacts is one of the biggest challenges in data analysis. In this experiment, the high frequency noises were removed using a 4th order Chebyshev low pass filter with a cutoff frequency of 30 Hz. In case of Chebyshev filter the total number of maxima and minima in the pass band region is equal to the filter order. This filter is specified by the maximum pass band gain, the cutoff frequency and the filter order. The raw signal and the filtered signal are shown in Figure 3.
Discrete Fourier Transform
DFT is an efficient algorithm that transforms the signal from time domain to frequency domain. For any signal X(t), the DFT is calculated as follows,
where is the input signal, X is the signal in frequency domain and N is the total number of samples. Discrete Fourier Transform provides information about the frequency content available in the entire signal. After applying DFT to the preprocessed signal, the various statistical features were extracted.
Classification
The extracted features were trained and classified using the Linear Disciminant Analysis (LDA), Quadratic Discriminant Analysis (QDA) [51-53] and k-nearest neighbor (kNN) [51,54,55] classifiers.
Under LDA we assume that the density for X, given for every class k is following a Gaussian distribution. The density formula for a multivariate Gaussian distribution is:
where p represents the dimension and Σk is the covariance matrix. This involves the square root of the determinant of this matrix. In this case matrix multiplication is done. The vector x and the mean vector μk are both column vectors.
For Linear discriminant analysis (LDA): Σk = Σ, ∀k.
In LDA, an assumption that the covariance matrix is identical for different k is made and thereby the classifier becomes linear. Linear Discriminant Analysis (LDA) determines parameters in such a way that, there is the best discrimination between the various classes. It does this by preserving most of the class discriminatory information.
The only difference of LDA from QDA is that in case of QDA no assumption that the covariance matrix is identical for different classes is made, but instead the decision boundary is determined by a quadratic function.
KNN is a simple data driven lazy learning algorithm, where an unlabelled point is attributed to the predominant class within the k nearest labeled points belonging to the training class. In all cases, 70% of data is used for training, while the remaining 30% for testing.
The distribution of various statistical features for normal, fatigue and cognitive distraction states is shown in Table 2. The statistically significant standard deviation feature, obtained during the various states of the driver were used to classify the fatigue and inattentiveness using LDA, QDA and KNN classifiers.
Table 3 shows the results obtained by classifying the inattentive, fatigue and normal state of the driver. The classification of fatigue is 80.45 % in case of LDA, wheareas in the case of knn it is 71.26 %. However, the classification accuracy is 100% for normal state in QDA and knn but very less in LDA classifier. In the case of inattention, QDA and KNN performed best giving an accuracy of 97.29 %. In general, QDA and KNN classifiers produced improved results. This indicates that a hypovigilance detection system can be developed by extracting the conventional standard deviation feature from EMG signal and this can be used with other reliable physiological signals for reliable results.
Most of the published work on drowsiness or inattention has focused on either EEG or ECG. However, Balasubramanian et al. observed that the muscle activity during simulated driving showed significant difference between 1st minute and 15th minute of driving [43]. sEMG was captured from trapezius and deltoid muscles during monotonous car driving and muscle fatigue was analyzed by [56]. Researchers have investigated and proved that the sEMG variables of left biceps (LB), right biceps (RB), left forearm flexor (LW), right forearm flexor (RW) and frontal (L) muscles, are valid and reliable indicators of muscular fatigue [57]. They recommended that analysis has to be made on different groups of muscles (i.e. latissimus dorsi, gluteus maximus, deltoid, trapezius) to further verify whether the proposed metrics provide useful information about the fatigue level. Though sEMG has not been studied in the context of drowsiness, it would be significant to analyze the pattern of muscle fatigue during drowsiness. Akin et al. observed the success rate to be higher when using a combination of EEG and EMG signals compared to using either one of signal in detecting drowsiness [26]. This also indicates that sEMG can be used along with other physiological signals to enhance the performance of the system [22,25]. Researchers have also found that thinking or cognitive work activates sEMG[58,59]. However, so far no research work has been done to correlate inattention and driving in the context of sEMG signals. In our work, it was observed that sEMG can be a viable tool to detect fatigue and inattention.
Monitoring driver behavior is a much needed factor for safe driving as driver drowsiness and driver inattention are the major causes for road accidents. Though researchers have probed into either drowsiness or inattention, not one of them has worked on a universal system to detect both drowsiness and inattention. In this work, hypovigilance has been detected using EMG signals. The raw signals being prone to noise were preprocessed using efficient filtering methods. The standard deviation feature of the preprocessed EMG signal has been classified into the various hypovigilance states such as fatigue, normal and inattention successfully with accuracy of 89.52%. Based on the outcome of this research study, a system to detect the hypovigilance and alert the driver can be worked out. The EMG measure can also be combined with other nonintrusive physiological measures like ECG for reliable results. In the future, behavioral measures and vehicle based measures can also be fused with physiological measures for better and robust detection.