What Is Object Detection? How It Works and How It Is Scored

LLM foundationsModels and inferencePublished By Simon Budziak

Object detection finds each object in an image or video frame and marks it with a labeled bounding box. It is a core task in computer vision. Unlike image classification, which gives one label for the whole picture, it answers both what is there and where.

How does object detection work?

A deep learning model, usually a convolutional or transformer neural network, looks at the image and outputs candidate boxes. Each box carries a class, such as pallet or person, and a confidence between 0 and 1. Many candidates overlap on the same object, so a step called non-maximum suppression keeps the highest-confidence box and discards close duplicates. A threshold then drops weak guesses. The model is trained on images where people have drawn the boxes, so quality depends on data annotation.

Two families dominate. Two-stage detectors first propose regions and then classify them, and tend to be more accurate. Single-stage detectors such as YOLO predict boxes and classes in one pass and tend to be faster, which matters for live video.

How is object detection scored?

A predicted box counts as correct when its intersection over union (IoU) with the true box passes a cutoff. IoU is the overlap area divided by the combined area of both boxes.

A worked example: a true box covers 100 pixels, the predicted box covers 100 pixels, and 60 of them overlap. The union is 140 pixels, so IoU is 60 / 140, about 0.43, a miss at a typical 0.5 cutoff. Shift the prediction so 80 pixels overlap, and IoU is 80 / 120, about 0.67, a hit.

Across all classes, teams report mean average precision (mAP), the average over classes of the area under each class’s precision and recall curve. Always ask at which IoU cutoff a reported mAP was measured, because a figure at 0.5 is easier to reach than one averaged over stricter cutoffs.

How does object detection differ from classification and segmentation?

Task Output Example question
Classification One label per image Is this a defective part?
Object detection A box and label per object Where are the defects, and how many?
Segmentation A pixel mask per object What is the exact outline of the defect?

Choose detection when you need counts or locations but not exact shapes. It is cheaper to label than segmentation and does more than classification.

Where is object detection used in business?

Counting stock on shelves, spotting missing safety gear on a site, reading parts on a conveyor and flagging damage in photos from a claim are typical jobs. It is the core of many computer vision systems. Accuracy on your own images matters more than a benchmark score: lighting, camera angle and object variety change results, so test on a sample from your site before committing.

This entry was drafted with AI assistance.

Frequently asked questions

How does object detection work?

A neural network proposes boxes and, for each, predicts a class and a confidence score. Overlapping duplicates are removed, usually by non-maximum suppression, and boxes below a confidence threshold are dropped.

Is YOLO object detection?

YOLO, short for You Only Look Once, is a family of object detection models that predict all boxes in a single pass, which makes them fast enough for live video. It is one of several approaches, not the name of the task.

Summarize this page with

Train your team to build this