Our Client:

A leading company in the automotive industry.

Situation:

Quality assurance for high-end products, such as car lights, often involves rigorous visual inspections performed by human operators. This process can be time-consuming and inefficient. Our client sought to streamline their operations by automating this quality assurance process using AI. To protect our client's proprietary data, we used the openly available "Visual LED Status” dataset to demonstrate our approach.

Task:

DSL was tasked with developing a machine learning model using cutting-edge object detection architecture. The system was designed to take an image of a car and output a bounding box around its lights, classifying whether they are on or off. Despite the power of deep learning models, tailoring and fine-tuning these models to specific challenges, such as detecting car lights, was necessary to achieve high performance suitable for QA automation.

The quality of the dataset was a critical component. We faced challenges such as a small dataset size, data ambiguity, and inconsistency with standard annotation formats. To address these, we used tools like the FiftyOne library to visualize our dataset and ensure its quality.

Result:

We established a baseline model using YOLOX-s and improved it incrementally, verifying that each change enhanced overall performance. To counteract the small dataset issue, we implemented augmentation techniques.

We also identified a challenge with data ambiguity and reviewing the client’s requirements and their environment that this technology would reside, we advised on a solution that would provide a more holistic approach by looking at a sequence of images, instead of still snapshots. During a PoC, it is crucial to identify such issues in the approach, so they can be effectively mitigated on a smaller scale before the solution enters the productionization phase where it’s going to be a lot harder to fix.

The final model achieved a high mAP score of 66.2, compared to the benchmark score on MS COCO of 40.5. All car lights present in the images were precisely identified by the model, and no car headlights received the wrong class classification label.

However, the model was sensitive to certain edge cases, such as detecting the car's logo as a car headlight and reflections of other car headlights. These can be mitigated by catering to the client's specific environment, angle and positioning of the cameras, and the light conditions present.

This proof of concept demonstrated the feasibility of the proposed approach, which will result in significant efficiency gains in the client's production facilities.