Object Detection Explained: From Model to Production
A current-generation detector scores 53.1 mAP on COCO and runs inference in 4.7 milliseconds on a T4 GPU with TensorRT. The same model on a CPU takes roughly 220 milliseconds — about 47 times slower, on hardware plenty of deployments actually use.
Researchers have a name for this: the export gap, the discrepancy between performance observed during GPU training and latency realised on deployed hardware. Training a detector has never been easier. Closing that gap is where most projects stall, and it shapes every decision below. This piece sits within the broader computer vision for business picture.
Hire Edge Computer Vision
Expertly deploy AI at the edge, schedule a consultation
Model benchmarks are published under conditions that rarely match deployment: a server-grade GPU, full floating-point precision, batched inputs, and a curated dataset. Production is usually the opposite, an edge device, reduced precision, one frame at a time, and imagery from your camera rather than COCO.
The gap this opens is not a rounding error. Benchmarking across edge platforms found CPU-only execution of large detectors on a Raspberry Pi 5 produces multi-second per-frame latency, which is simply not viable for real-time work, while the same architecture on a Jetson Orin NX with TensorRT runs comfortably. The model didn't change. The runtime did.
The practical consequence is that choosing a model on mAP alone is choosing on the least deployment-relevant number available.
Three Things That Break on the Way to Production
Latency becomes unpredictable, not just slower
Traditional detectors generate many candidate boxes and then run non-maximum suppression to remove duplicates. NMS cost scales with how many objects are in frame, so a crowded scene is slower than an empty one. For a system with a fixed frame budget, that variance matters more than the average. Newer NMS-free architectures predict a fixed set of objects directly, which makes latency deterministic regardless of scene density.
Hire Edge Computer Vision
Expertly deploy AI at the edge, schedule a consultation
Quantization loses accuracy unevenly
Edge deployment usually means converting the model to lower precision. Some operations survive this cleanly; others don't. Softmax layers, used in the distribution-based box regression that boosted benchmark scores in several recent model generations, are difficult to quantize and become a primary latency bottleneck on integer-arithmetic hardware like mobile NPUs and drone DSPs. An architecture optimised for benchmark mAP can be actively hostile to the hardware you need to run it on.
Benchmark accuracy doesn't transfer to your camera
COCO contains everyday objects photographed in good conditions. Your deployment may involve a fixed overhead camera, unusual lighting, motion blur, and object classes that don't exist in any public dataset. A model's published mAP describes its performance on someone else's problem.
The Path From Trained Model to Running System
A working deployment moves through four steps, and the middle two are where most of the engineering time actually goes.
Step
What happens
1. Train or fine-tune
Adapt a pre-trained detector to your classes and imagery
2. Export
Convert to a deployment format: ONNX, TensorRT, CoreML, or TFLite
3. Optimise
Quantize and compile for the target hardware; TensorRT commonly cuts latency 2–6x
4. Integrate and monitor
Wire into the camera feed and downstream action; track drift over time
Export cleanliness is worth weighting heavily at model selection time. An architecture that converts without custom operators or unsupported layers reaches production in days; one that needs bespoke kernels for every target runtime can consume weeks. This is precisely why recent detector designs have traded small amounts of benchmark accuracy for simpler graphs that export cleanly across runtimes.
Hire Edge Computer Vision
Expertly deploy AI at the edge, schedule a consultation
What to Measure Instead of mAP
Four numbers predict deployment success better than benchmark accuracy does.
Latency at the 95th percentile on your target hardware, not the average on a T4. Accuracy retention after export and quantization, measured on your own held-out images rather than COCO. Precision and recall at your chosen confidence threshold, since the threshold is a business decision about the relative cost of a miss versus a false alarm. And power draw, if the device is battery-powered or thermally constrained.
A vendor or contractor who reports only mAP is describing the easy part of the problem. vetting a CV consultant covers the wider set of questions worth asking before an engagement starts.
