GreenQuest vision models

About the project

Vision for a robot that cleans industrial sites: find the waste bays, and see stairs, walls and obstacles at floor level so it doesn't drive off an edge.

GreenQuest was building a robot to clean industrial sites. A robot like that needs eyes: it has to find where the waste is, and see stairs, walls and obstacles at floor level so it doesn't fall or crash.

As AI Vision Engineer in 2023, I trained models for both, soon after YOLOv8 came out. The floor-hazard model, trained on about 970 labelled photos, scored 97% mAP@50 on its validation set. The waste-bay model had only 13 photos to learn from, and stayed an early prototype.

How it works

1. 1

   Collect

   Photos from the site, including floor-level shots.

2. 2

   Label

   Draw boxes in Roboflow: waste bays, or obstacles, stairs and walls.

3. 3

   Train

   Fine-tune YOLOv8 on the labelled sets, and train the floor-hazard model on Roboflow 3.0 from a COCO starting point.

4. 4

   Detect

   Run the model over new photos and crop each waste bay it finds.

Under the hood

Waste bays

A one-class detector, plus a cropping script.

Floor hazards

A three-class detector on about 970 images, trained on Roboflow 3.0: 97% mAP@50, 89.3% precision, 91% recall on the validation set.

Checker page

Upload a photo and get "Cleaned" or "Not cleaned" from my garbage-detection model, hosted on Roboflow.

What I did

• Labelled a waste-bay dataset in Roboflow and fine-tuned YOLOv8
• A script that finds each bay in a folder of photos and saves it as a crop
• A 3-class floor-level dataset (obstacle, stairs, wall) with about 970 images, in two versions
• Trained the floor-hazard model on Roboflow 3.0 (Fast, from a COCO checkpoint) in December 2023: 97.0% mAP@50, 89.3% precision, 91.0% recall and 90.1% F1 on the validation set
• Also fine-tuned YOLOv8 locally on both dataset versions
• A garbage-detection model on Roboflow, and a one-page checker that uses it: upload a photo and it says "Cleaned" or "Not cleaned"

What was new

• My first object detection: labelling, augmentation, training, and reading the metrics
• Seeing what data does: 13 photos gave a model that missed new bays, 970 gave one that scored 97% mAP@50

Problems I hit, and how I fixed them

1. 01

   Only 13 photos of waste bays.

   Fix Augmentation. Still too few: the model missed bays in photos it hadn't seen.

2. 02

   The first floor dataset was exported in grayscale.

   Fix A second version in colour.

3. 03

   Tight crops cut off the top of the bay.

   Fix Extend each crop upward.

Screenshots

To be clear

The waste-bay model learned from only 13 photos, so it's an early prototype. The floor-hazard scores come from Roboflow's validation set, not from the robot working on site. The site photos show an industrial plant and its workers, so they aren't shown here.