CG MiniProject Report
CG MiniProject Report
CHAPTER 1
INTRODUCTION
Driver drowsiness detection is a car safety technology which helps prevent accidents caused by the driver
getting drowsy. Various studies have suggested that around 20% of all road accidents are fatigue-related,
up to 50% on certain roads. The drowsiness detection system is capable of detecting drowsiness quickly.
The driver behaviors are noticed in many conditions such as wearing spectacles and also in the dark
condition inside the vehicle.
The system is capable of detecting the drowsiness condition within the duration of more than two seconds.
After the detection of abnormal behaviors, it is alerted to the driver through alarms and the parking lights
will be on that will stop the vehicle which reduces the accidents due to drowsiness of the driver. A deep
learning Architecture detects the face and eyes, based on the status of the eyes. If the eyes are closed more
than usual time, it generates an alarm, intimating the driver. Neglecting our duties towards safer travel has
enabled hundreds of thousands of tragedies to get associated with this wonderful invention every year.
In order to monitor and prevent a destructive outcome from such negligence, many researchers have
written research papers on driver drowsiness detection systems. But at times, some of the points and
observations made by the system are not accurate enough. Hence, to provide data and another perspective
on the problem at hand, in order to improve their implementations and to further optimize the solution, this
project has been done.
Road safety is a critical issue, with driver fatigue being a significant contributing factor to traffic accidents
worldwide. Drowsy driving impairs reaction times, attention, and decision-making abilities, posing serious
risks to both the driver and others on the road. Despite awareness campaigns and technological
advancements in vehicle safety, incidents related to driver fatigue remain alarmingly high. The primary
challenge is to develop an effective and reliable system that can detect early signs of drowsiness in drivers
and alert them promptly, thus preventing potential accidents. Existing solutions often lack accuracy, have
delayed response times, or are not accessible for all drivers.
1.2 Scope
The project aims to develop a drowsy driver detection system that utilizes a webcam to monitor facial
expressions and eye movements, identifying signs of drowsiness such as yawning and prolonged eye
closure. This system will leverage computer vision techniques and machine learning algorithms to analyze
these visual cues in real time, triggering an alert sound to wake the driver when drowsiness is detected. The
scope includes the design and implementation of the system, testing under various conditions, and
documentation of the process. However, the project does not cover the development of specialized
hardware beyond standard webcams, integration with vehicle control systems, or advanced monitoring
features like heart rate analysis. This prototype is intended as a proof of concept and will not extend to
commercial deployment or mass production.
The prevalence of road accidents caused by drowsy driving highlights a critical need for effective solutions
to enhance driver safety. Our project addresses this issue by developing a drowsy driver detection system
that uses a standard webcam to monitor and analyze a driver's facial expressions and eye movements. The
system leverages computer vision and machine learning techniques to identify signs of drowsiness, such as
yawning and prolonged eye closure.
Once signs of drowsiness are detected, the system triggers an alert sound to wake the driver, thereby
preventing potential accidents. The design and implementation process includes selecting appropriate
algorithms, developing a user-friendly interface for calibration, and conducting comprehensive testing
under various environmental conditions. The project focuses on creating an accessible and affordable
solution that can be easily integrated into existing vehicle setups, enhancing overall road safety. This
1.4 Objectives
The project focuses on these objectives, which are:
To study on eyes and mouth from the video images of participants in the experiment of driving
simulation conducted by MIROS that can be used as an indicator of fatigue and drowsiness.
To investigate the physical changes of fatigue and drowsiness.
To develop a system that use eyes closure and yawning as a way to detect fatigue and drowsiness.
This project is an active topic that is still being enhanced and improved by researches and can be applied in
many areas such as detecting the attention-level of students in classrooms and lectures. This is also relevant
to the three author’s field of study since it requires the author to apply and combine the knowledge of
electronics, programming and algorithms.
CHAPTER 2
LITERATURE SURVEY
There are many previous researches regarding driver drowsiness detection system that can be used as a
reference to develop a real-time system on detecting drowsiness for drivers. There is also several method
which use different approaches to detect the drowsiness signs. According to MIROS (Malaysia Institute of
Road Safety), from the year of 2007 until 2010, they were 439 cases of road accidents have been
investigated by the MIROS crash team [1].
Antoine Picot et al, [2] stated that drowsiness is where a person is in the middle of awake and sleepy state.
This situation leads the driver to not giving full attention to their driving. Therefore, the vehicle can no
longer be controlled due to the driver being in a semi-conscious state. According to Gianluca Borghini et
al, [3] mental fatigue is a factor of drowsiness and it caused the person who experiences to not be able to
perform because it decreases the efficiency of the brain to respond towards sudden events.
The disadvantages of this method are, it is very sensitive to noise around the sensors. For example, when
the person is doing the EEG experiment, the surrounding area must be completely silent. The noise will
interfere with the sensors that detect the brain activity. Another disadvantage of this method is that even if
the result might be accurate, it is not suitable to use for real driving application [10]. Imagine when a
person is driving and he is wearing something on his head with full of wires and when the driver moves
their head, the wire may strip off from their place. Even though it is not convenient to be used for real-
time driving but for experiment purposes and data collection, it is one of the best methods so far [2].
Drowsiness can be detected by using face area detection [5], [6] and [14]. The methods to detect
drowsiness within face area are vary due to drowsiness sign are more visible and clear to be detected at
face area. From the face area, we can detect the eyes location. From eyes detection, author in [5] stated
that there are four types of eyelid movement that can be used for drowsiness detection. They are complete
open, complete close, and in the middle where the eyes are from open to close and vice versa [5]. Fig. 2.2
is an example of the image taken for detecting eyelid movement.
The problem occurring with this method is that the size area of eye might vary from one person to another.
Someone may have small eyes and looks like it is sleepy but some are not. Other than that, if the person is
wearing glasses, there is obstacle to detect eye region. The images that being captured must be in certain
range from the camera because when the distance is far from the camera, the images are blurred [6].
Drowsiness can be captured by detecting the eye blinks [5] and percentage of eye closure (PERCLOS).
For eye blink detection, [5] propose a method which learned the pattern of duration of eyelid closed.
According to [10], ‘this proposed method measures the time for a person closed their eyes and if they are
closed longer than the normal eye blink time, it is possible that the person is falling asleep’. In [10], the
author mentioned that ‘nearly 310.3ms are the average of normal person eye blink’.
PERCLOS method proposes that drowsiness is measured by calculating the percentage of the eyelid
‘droops’ [11]. Sets of eye open and eye closed have been stored in the software library to be used as a
parameter to differentiate either the eyes is fully open or fully closed. For eyelid to droops, it happened in
much slower time as the person is slowly falling asleep. Hence, the transition of the driver’s drowsy can
be recorded. Thus, PERCLOS method put a proportional value where when the eyes is 80% closed, which
it is nearly to fully close, it assumed that the driver is drowsy [2], [10], and [12].
This method is not convenient to be used in real-time driving as it needs fix threshold value of eye
opening for the PERCLOS method to perform accurately. Both methods to detect drowsiness using eye
blink pattern and PERCLOS have the same problem where the camera need to be placed at a specific
angle in order to get a good image of video with no disturbance of eyebrow and shadow that cover the
eyes.
According to [15], drowsiness of a person can be observed by looking at their face and behavior. The
Dept of CSE, JVIT Page 6
Detection of Drowsy Driver 2023-2024
author propose a method where drowsiness can be detected by mouth positioning and the images were
process by using cascade of classifier that has been proposed by Viola-Jones for faces. Some people will
close their mouth by their hand while yawning. It is an obstacle to get good images if a person is closing
their mouth while yawning but yawning is definitely a sign of a person having drowsiness and fatigue.
Fig. 2.3 are the examples of yawning detection method used in the [15] research.
After gone through the research papers and the existing methods, this project proposed that eyes and
yawning detection method will be used. Eye blink duration gives the data that the longer the person’s
close their eyes, the drowsier it will be considered. It is because when a person is in drowsy state; its eyes
will be closed longer than the normal eye blink. Other than that, yawning is one of the symptoms of
drowsiness where it is a normal human response when yawning is the sign that they feel drowsy or
fatigue.
CHAPTER 3
PROJECT DESCRIPTION
NO EYE DETECTION – most critical sign of drowsiness Yawning and nodding are not always
practical. Varies from person to person – some may not yawn when they are sleepy sometimes.
Physiological sensors:
CHAPTER 4
SYSTEM REQUIREMENTS
The development of a drowsy driver detection system involves a comprehensive analysis of the
requirements needed to ensure the system's functionality, performance, and integration capabilities. This
analysis covers functional requirements, which define the core features and behaviors of the system;
performance requirements, which specify the expected operational efficiency and responsiveness; power
requirements, which outline the energy consumption and management considerations; and both software
and hardware requirements, which detail the necessary technological components and tools.
Drowsiness Detection: The system must accurately detect signs of drowsiness, such as yawning,
prolonged eye closure, and other relevant facial cues, using real-time video analysis.
Alert System: Upon detecting drowsiness, the system should promptly trigger an audible alert to
wake the driver and prevent potential accidents.
User Interface: A user-friendly interface must be provided for setup, calibration, and monitoring,
allowing users to adjust sensitivity and other settings.
Data Privacy: The system should ensure that all video data is processed locally to maintain user
privacy and comply with data protection regulations.
Energy Efficiency: The system should be designed to minimize power consumption, particularly
when integrated into vehicles, to avoid draining the vehicle's battery.
Power Supply: The system should be capable of operating using standard vehicle power sources or
through an independent power supply, ensuring consistent operation.
Operating System-Windows 11
Code Editor- Jupyter Notebook
Libraries used- OpenCV, pygame, numpy
CHAPTER 5
MODULES DESCRIPTION
5.1 Module 1: Web Capturing
With a webcam, we will take images as input. So to access the webcam, we made an infinite loop that will
capture each frame. We use the method provided by OpenCV, cv2.VideoCapture(0) to access the camera
and set the capture object(cap). cap.read() will read each frame and we store the image in a frame variable.
To detect the face in the image, we need to first convert the image into grayscale as the OpenCV
algorithm for object detection takes gray images in the input. We don’t need color information to
detect the objects. We will be using haarcascade classifier to detect faces. This line is used to set
our classifier .
The cascades themselves are just a bunch of XML files that contain OpenCV data used to detect
objects. You initialize your code with the cascade you want, and then it does the work for you.
It loads face cascade into memory and read the mage and convert into grayscale(many operations
are done in grayscale as we need not define Green , Red and Blue colours here)
The value we will use to determine how long the person has closed his eyes is threshold value. So if
both eyes are closed, we will keep on increasing score and when eyes are open, we decrease the
score. We are drawing the result on the screen using cv2.putText() function which will display real
time status of the person.
A threshold is defined for example if score becomes greater than 15 that means the person’s eyes
are closed for a long period of time. This is when we beep the alarm using sound.play().
CHAPTER 6
IMPLEMENTATION
First we have used a camera which is setup at desirable position in a car that looks for faces stream. If face
gets detected, the facial landmark detection task is applied and region of eyes is extracted. Once we get the
eye region, we calculate the Eye Aspect Ratio to find out if the eye-lids are down for a substantial amount
of time.
On the off chance that the Eye Aspect Ratio demonstrates that the eyes are shut for a considerably long
6.2 Methodology
Fig. 6.1: Flow chart showing entire process of drowsiness detection system
The proposed technique is primarily based on eye blinking and yawning of driver which can be
behavioural measures. The aim of this project is to detect closed eyes or opened mouth, this is yawning
and to alert the driver. This is done by placing a camera or recording device in front of the driver and
capturing real time video continuously using OpenCV. The application is executed in python and
processing is done in laptop’s camera.
Each eye is characterized by 6 coordinates as in figure 6.2. An equation called Eye Aspect Ratio (EAR)
which reflects the relation between width and height of coordinators can be derived. The distance
between vertical eye landmarks is computed in numerator and those of horizontal eye landmarks are
calculated in the denominator using the formula for Euclidean distance.
This ratio of eye landmark distances can be used to determine whether a person is blinking or not.
Following optimized tools and image processing libraries are used by author for implementation of
presented algorithm.
Open CV: OpenCV (Open-source Computer Vision) is the Swiss Army knife of computer vision. It has a
wide range of modules that can help us with a lot of computer vision problems. But perhaps the most
useful part of OpenCV is its architecture and memory management. It provides you with a framework in
which you can work with images and video in any way you want, using OpenCV’s algorithms or your
own, without worrying about allocating and reallocating memory for your images. Open CV libraries and
functions are highly optimized and can be used for real time image and video processing. OPENCV’s
highly optimized image processing function are used by author for real time image processing of live video
feed from camera.
Jupyter Notebook: Open-source community version of Jupyter Notebook platform is used as main coding
editor.
CHAPTER 7
SOURCE CODE
import cv2
import numpy as np
from scipy.spatial import distance as dist
import pygame
# Load webcam
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
# Detect faces
faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
# Release resources
cap.release()
cv2.destroyAllWindows()
CHAPTER 8
SNAPSHOTS
CHAPTER 9
CONCLUSION
This project looks at how to detect tiredness in a driver in real time by looking at eye closure and
yawning. This technology has the advantage of detecting tiredness early on and sounding an alarm before
an accident occurs. The use of OpenCV is considered to be more suitable for this application based on
the design of the proposed work because it meets the relevant requirements such as cost, power, and size.
Face, eye, and mouth are easily detected by this technology, and these are captured using a webcam. The
technology can detect whether the eyes and mouth were open or closed during monitoring. A warning
signal will be issued if the eyes have been closed for an extended period of time or if yawning is
detected.
REFERENCES
[1] Z. Ahmad Noor Syukri, M. Siti Atiqah, L. Fauziana, and A. Abdul Rahmat, "MIROS crash
investigation and reconstruction: annual statistical 2007-2010," 2012.
[2] A. Picot, S. Charbonnier, and A. Caplier, "On-line automatic detection of driver drowsiness using a
single electroencephalographic channel," in Engineering in Medicine and Biology Society, 2008.
EMBS 2008. 30th Annual International Conference of the IEEE, 2008, pp. 3864-3867.
[4] B. T. Jap, S. Lal, P. Fischer, and E. Bekiaris, "Using EEG spectral components to assess
algorithms for detecting fatigue," Expert Systems with Applications, vol. 36, pp. 23522359, 2009.
[5] D. Liu, P. Sun, Y. Xiao, and Y. Yin, "Drowsiness Detection Based on Eyelid Movement," in
Education [1]Technology and Computer Science (ETCS), 2010 Second International Workshop
on, 2010, pp. 49-52.
[8] T. Welsh, M. Ashikhmin, and K. Mueller, "Transferring color to greyscale images," ACM
Transactions on Graphics, vol. 21, pp. 277-280, 2002.
[10] T. Danisman, I. M. Bilasco, C. Djeraba, and N. Ihaddadene, "Drowsy driver detection system
using eye blink patterns," in Machine and Web Intelligence (ICMWI), 2010 International
Conference on, 2010, pp. 230-233.
[12] S. T. Lin, Y. Y. Tan, P. Y. Chua, L. K. Tey, and C. H. Ang, "PERCLOS Threshold for Drowsiness
Detection during Real Driving," Journal of Vision, vol. 12, pp. 546-546, 2012.
[13] R. Grace, V. E. Byrne, D. M. Bierman, J.-M. Legrand, D. Gricourt, B. Davis, et al., "A drowsy
driver detection system for heavy vehicles," in Digital Avionics Systems Conference, 1998.
Proceedings., 17th DASC. The AIAA/IEEE/SAE, 1998, pp. I36/1-I36/8 vol. 2.
[14] Q. Wang, J. Yang, M. Ren, and Y. Zheng, "Driver fatigue detection: a survey," in Intelligent
Control and Automation, 2006. WCICA 2006. The Sixth World Congress on, 2006, pp. 8587-8591.
[15] M. Saradadevi and P. Bajaj, "Driver fatigue detection using mouth and yawning analysis," IJCSNS
International Journal of Computer Science and Network Security, vol. 8, pp. 183-188, 2008.
[16] Bereshpolova Y, Stoelzel CR, Zhuang J, Amitai Y, Alonso JM, Swadlow HA. (2011) "Getting
drowsy? Alert/nonalert transitions and visual thalamocortical network dynamics". J Neurosci.
2011, 48: 17480-17487
[17] McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlan, E. M. (1984).
Clinical diagnosis of Alzheimer's disease Report of the NINCDS ADRDA Work Group* under
the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease.
Neurology, 34(7), 939-939.
[18] Qing, W., BingXi, S., Bin, X., & Junjie, Z. (2010, October). A perclos-based driver fatigue
recognition application for smart vehicle space. In Information Processing (ISIP), 2010 Third
International Symposium on (pp. 437-441). IEEE.