Project Presentation

IDENTIFY DAMAGED CAR PARTS

WITH VERTEX AI and AUTOML
VISION
UNDER THE GUIDENCE OF: PRESENTED BY:
Mr. Deepak P Impana A - 4NN20CS023
Assistant Professor Kusum Sharma - 4NN20CS025
Dept of CSE Likith MV - 4NN20CS026
NIEIT,Mysuru Mahadevaswamy JG - 4NN20CS027
LIST OF CONTENTS
Introduction

Literature Survey

Problem statement

Objective

System requirements

Design and Methodology

Implementation

Existing sytem

Test cases

Expected outcome

References
ABSTRACT
Cars can suffer damage from sharp objects, road accidents, or other incidents that impact their
original shape and form. Vertex AI combines Google Cloud services for building machine learning
models under a unified UI and API. Users can train and compare models using AutoML or custom
code training, with models stored in a central repository. AutoML Vision helps those with limited ML
expertise train high-quality image classification models. A hands-on lab teaches how to create
custom ML models for damaged car parts.
INTRODUCTION
Vertex AI brings together the Google Cloud services for building ML under one, unified UI
and API.
In Vertex AI, we can easily train and compare models using AutoML or custom code training
and all your models are stored in one central model repository.
AutoML Vision helps anyone with limited Machine Learning (ML) expertise train high quality
image classification models. It produce a custom ML model that automatically recognizes
damaged car parts.
Once you’ve produced your ML model, it’ll be immediately available for use. You can use
the UI or the REST API to start generating predictions directly from the Google Cloud
Console.
LITERATURE SURVEY
1. Damage Identification of Selected Car Parts Using Image Classification and
Deep Learning
Authors: Abraham C. Chua, Christian Rei B. Mercado, John Phillip R. Pin, Angelo Kyle T. Tan, Jose Benito L.
Tinhay, Elmer P. Dadios, and Robert Kerwin C. Billones
Year: 2021
Publisher: IEEE
This study uses image classification and deep learning to identify and assess damaged
vehicle parts in insurance claims. A two-level machine learning-based system classifies car parts
and detects damages. The image dataset used was a Google image. The model achieved training
accuracy of 94.84% and validation accuracy of 81.25% for car parts classification, and 97.16% and
validation accuracy of 49.28% for damage identification.
2.DAMAGED CAR DETECTION USING MULTIPLE CONVOLUTIONAL
NEURAL NETWORKS WITH FLASK WEBAPP

Authors:Mrs. N.Senthamilarasi , M.Tech, Aldrin Jefferson R, Hari Haran N,4Mohammed

Mabas Deen A
Year: MAY 2023
Publisher: JETIR
This project aims to create a damaged car detection system using multiple
convolutional neural networks (CNN) and deploy it as a Flask web application. The
system helps in detecting vehicle damage after an accident, aiding insurance
assessors, auto repair shops, and car buyers in making informed decisions. The
system involves training the model using ensemble learning on a GPU-enabled
machine, and creating a Flask web application that uses an image to predict
damage severity.
3. A SURVEY ON AUTOML METHODS AND SYSTEMS FOR
CLUSTERING

Authors: Yannis Poulakis, Christos Doulkeridis, Dimosthenis Kyriazis

Year: 27 FEB 2024
Publisher: Association for Computing Machinery New York, NY, United States

This survey focuses on unsupervised learning and AutoML methods for

clustering. It presents a systematic review of recent research on automated
clustering, provides a taxonomy for classification, and performs a qualitative
comparison. The survey provides a comprehensive overview of the field and
identifies open challenges for future research in this area, highlighting the time-
consuming and computationally expensive task of finding the best model for a
specific problem.
4. SMART INTELLIGENT SYSTEM FOR CERVIX CANCER IMAGE
CLASSIFICATION USING GOOGLE CLOUD PLATFORM

Authors: P. Subashini, T. T. Dhivyaprabha, M. Krishnaveni & M. B. Jennyfer Susan

Year: 26 FEB 2023
Publisher Name: Springer, Cham

The system uses real-time images from EVA digital colposcopy, categorizes stages of
cervical cancer, and evaluates its accuracy. The system uses AutoML framework to
categorize cervical cancer stages, and the model is uploaded, stored, and deployed on a
vertex AI platform. The results show that the system helps physicians diagnose cervical
cancer stages with minimal effort and time, enabling early detection of precancerous
lesions and reducing the need for biopsy. This technology supports sustainable smart city
development.
5. ARTIFICAL INTELLIGENT SYSTEM FOR VEHICLE CLASSIFICATION

Authors: Shi Hao Tan, Joon Huang Chuah, Jeevan Kanesan, Hung Yang Leong
Year: 2023

The paper explores that vehicle classification in intelligent transportation

systems for smart cities, categorized into sensor-based and image-based methods
which operate using information collected from sensing devices and images,
respectively. The collected information is utilized by machine learning or deep
learning algorithms to perform vehicle classification task.
6. PATTERNS OF VEHICLE LIGHTS: ADDRESSING COMPLEXITIES OF
CAMERA BASED VEHICLE LIGHT DATASETS AND METRICS

Authors: Ross Greer, Akshay Gopalkrishnan, Maitrayee Keskar, Mohan M. Trivedi

Year: 26 JULY 2023

This explore that vehicle light representation in computer vision for

autonomous driving, introducing the LISA Vehicle lights datasets and light visibility
neural networks to address occlusion challenges.
7. AUTOMATIC OUTER CONTOUR DETECTION AND QUANTIFICATION
OF VEHICLES USING MONOCULAR VISION

Authors: Dayong Han, Chaodong Zhang, Liang Wang, Xianglong Xu and Yingkai
Liu
Year: 23 Jan 2024

The study introduces a framework using computer vision and view geometry to
detect and quantify vehicle dimensions in traffic scenes, preventing infrastructure
collisions. It uses a modified Mask R-CNN, segmented vehicle instances, and a view
geometry-based algorithm for accurate identification.
8. AUTOMATED DATA PROCESSING AND FEATURE ENGINEERING FOR
DEEP LEARNING AND BIG DATA APPLICATIONS

Authors: Alhassan Mumuni, Fuseini Mumuni

Year: 1 JAN 2024
Published by: Elsevier B.V

Modern AI leans on data driven algorithms, especially in deep learning, yielding

impressive outcomes. Today, end-to-end automated data processing systems based
on automated machine learning (AutoML) techniques are capable of taking raw
data and transforming them into useful features for Big Data tasks by automating
all intermediate processing stages
PROBLEM STATEMENT
Car accidents are a common occurrence on roads, leading to considerable damage to
vehicles. Analyzing the extent of damage accurately and efficiently is crucial for insurance
claims, auto repairs, and vehicle purchases.
Manual inspection is a time consuming and expensive process that may lead to human
errors.
The car damage detection process is crucial to the motor industry and the related
stakeholders.
It identifies damages and estimates the repair cost. It is often impossible to manually
inspect every part of a car, but vehicle inspections are efficient and feasible using machine
learning algorithms.
OBJECTIVE

Upload a labeled dataset to Cloud Storage using a CSV file and connect it to
Vertex AI as a Managed Dataset

Inspect the uploaded images to ensure there are no errors in our dataset

Kick off an AutoML Vision model training job

Vertex AI handles this automatically, without requiring you to write any of the
model code.
SYSTEM REQUIREMENTS

8 GB RAM
100GB free disk space
Browser
Dual core Processor
Windows 10 or 11
Stable Internet connection
Google Cloud Platform
DESIGN & METHODOLOGY
Create Bucket

Prepare the
A datasets

B Upload to cloud Cloud storage Google cloud

storage platform

C
Create Datasets
in VertexAI

D Inspect Images Train AutoML E

vision Model

Create artifact
Deploy the Model F
registry

Cloud run Create AutoML

proxy service

Run the Request the G

Application predication

Predication result H
DESIGN & METHODOLOGY

Creation of cloud storage (Bucket A) on Google Cloud Platform (GCP) for storing
datasets.
·Preparation and upload of datasets containing labeled images of damaged car parts
to Bucket A.
·AutoML Model Training and Deployment.
Dataset Management and Inspection.
Artifact Registry and Model Display.
Application Execution and Prediction.
Gather the data and annotated data

Upload data to google cloud platform

Train the model with vertex AutoML

IMPLEMENTATION
Evaluate the performance

Deploy Model

Integrate the model into system

User Image Input

Predict Car Damage

Display the prediction results

Take the Action

Feedback Loop
IMPLEMENTATION
Train the Model with Vertex AutoML and Annotated Data: Utilize Google's Vertex AutoML
platform to train a machine learning model for car part damage prediction and train the model using
annotated data.
Upload Data to Google Cloud Storage: Upload the annotated data, including labeled images, to
Google Cloud Storage.
Gather Data and Annotate Images: Gather relevant data, such as images of damaged car parts,
from various sources and annotate the gathered images to label and categorize them based on the
type of car part damage.
Integrate Model into System: Integrate the trained machine learning model into the existing system
architecture,.
Deploy the Model: Deploy the trained model on a cloud platform, such as GCP.
User Image Input and Prediction Display:·Allow users to input images of damaged car parts
through a user-friendly interface, such as a web application or mobile app.
Feedback Loop and Action Based on Prediction: Implement a feedback loop mechanism where
users can provide feedback on the prediction results, helping improve the model's accuracy and
performance over time
IMPLEMENTATION

Figure 1: Creating a datasets for image classification

Figure 2: Importing data file from cloud storage bucket:

Figure 3: Training a new model for data set we created

TEST CASES
Test Actual
Test Cases Test Purpose Test Condition Expected Results Pass/Fail
No Results

The image damaged

Identify exact damage part and Show the
To detect damaged parts To detect part will be detected
1 label image Pass
of car from Vertex AI and accuracy will be
it accuracy
shown

The bumper damaged

Identify exact damage part and Show the
To detect damaged parts To detect part will be detected
2 label image Pass
of car from Vertex AI and accuracy will be
bumper accuracy
shown

The Engine
Identify exact damage part and Compartment Show the
To detect damaged parts To detect
3 label damaged part will be image Pass
of car from Vertex AI
Engine Compartment detected and accuracy accuracy
will be shown
Test No Test Cases Test Purpose Test Condition Expected Results Actual Results Pass/Fail

The Windshield damaged

To detect damaged parts Identify exact damage part and label To detect Show the image
4 part will be detected and Pass
of car Windshield from Vertex AI accuracy
accuracy will be shown

The Lateral damaged

To detect damaged parts Identify exact damage part and label To detect Show the image
5 part will be detected and Pass
of car Lateral from Vertex AI accuracy
accuracy will be shown

The Hood damaged

To detect damaged parts Identify exact damage part and label To detect Show the image
6 part will be detected and Pass
of car Hood from Vertex AI accuracy
accuracy will be shown

Show the
incorrect
The image damaged
accuracy/ No
To detect damaged parts If image is not a damaged car parts To detect part will not be detected
7 label will be Fail
of car it should not provide a proper output to it. from Vertex AI and incorrect accuracy will
displayed for the
be shown
given image.
EXISTING SYSTEM
Damage Identification of Selected Car Parts Using Image Classification and Deep Learning:
The image dataset was collected through a local automotive dealer. The car parts classification will include
three parts, namely front bumper, car wheel, and rear bumper
This study presents the use of image classification and deep learning in the field of insurance claims and
management for the identification and assessment of damaged vehicle parts

PROPOSED SYSTEM
Identifying Damaged Car Parts with Vertex AutoML Vision
It identifies damages and estimates the repair cost. It is often impossible to manually inspect every part of a car,
but vehicle inspections are efficient and feasible using machine learning algorithms. The car parts classification
will include, Windshield, hood, bumper, lateral.
The proposed system for identifying damaged car parts leveraging the Google Cloud Platform (GCP) offers a
comprehensive and efficient solution. Utilizing a user-friendly web or mobile interface, users can upload images
of damaged vehicles, which are stored securely in Google Cloud Storage (GCS)
EXPECTED OUTCOME

Request a prediction from a hosted model:

To request predictions from the model, you will send predictions to an endpoint inside of
your project that will forward the request to the hosted model and return back the output.
Sending a prediction to the AutoML Proxy is very similar to the way that you would
interact with your model you just created, so you can use this as practice.
EXPERIMENTAL RESULT
The project yielded highly accurate results in identifying and assessing damages on selected car parts using
deep learning and image classification techniques. Trained CNN models like VGG16, ResNet50, and InceptionV3
showcased exceptional performance, accurately categorizing scratches, dents, cracks, and other damages. This
precision aids insurance assessments, auto repair shops, and car buyers in making informed decisions. Overall, the
project demonstrated the effectiveness of deep learning in automating and improving car damage assessment
processes

Figure 1: Confusion matrix of trained dataset Figure 2: Model output prediction

REFERENCES
https://partner.cloudskillsboost.google/course_sessions/11783244/labs/452135
https://inspektlabs.com/blog/car-damage-detection-using-ai-methodology-and-approach-
for-high-accuracy/
https://www.coursera.org/projects/googlecloud-identify-damaged-car-parts-with-vertex-
automl-vision-bugvm
https://www.researchgate.net/publication/359271153_Damage_Identification_of_Selecte
d_Car_Parts_Using_Image_Classification_and_Deep_Learning
https://www.jetir.org/papers/JETIR2305437.pdf
THANK YOU