Edge CV for Outdoor Inspection Under Domain Shift

The problem

Build a robust object detection system for outdoor inspection where targets are small, partially occluded, and safety-critical. The system must perform consistently across real-world conditions (weather/lighting/motion blur) and run on an edge device (NVIDIA Jetson).

The constraint: dataset reality (what makes it hard)

In inspection problems, the hardest part is usually not "model selection". It’s the gap between a clean benchmark and the real dataset:

• Domain shift: camera changes, seasonal effects, different backgrounds, exposure, and motion blur change the input distribution.
• Small objects: targets occupy few pixels; mAP can look okay while false negatives remain unacceptable.
• Label noise: inconsistent bounding boxes and missed positives inflate training loss and distort metrics.
• Imbalance: defects are rare; naive training optimizes for the majority class and misses the critical cases.

The hardest part is not model selection. It’s the gap between a clean benchmark and the real dataset.

Approach: model choice (YOLO vs RT-DETR), why these

YOLO and RT-DETR make sense as a practical comparison:

• YOLO family is typically strong for real-time deployment and tooling (export, TensorRT, batching, etc.).
• RT-DETR (a real-time detection transformer) is attractive when you want transformer-style matching/architecture with an emphasis on speed.

Decision framing (what matters)

• If your main constraint is edge throughput + stable latency, YOLO variants are often the fastest path to a shippable system.
• If your main issue is hard scenes + crowded/occluded objects, transformer-based detectors can be worth testing, especially if you have enough data and careful training.

Results: metrics (what actually matters in inspection)

For inspection, the business metric is usually "don’t miss the defect". That means you should emphasize:

• Recall / false negative rate at an operating point.
• Per-condition performance (night vs day, rain vs dry, high blur vs low blur).
• Small-object slices (e.g., area < 32×32 px) to avoid being fooled by aggregate mAP.
• Latency / FPS on the target device, not on a desktop GPU.

Deployment constraints (Jetson)

Edge deployment introduces constraints that often change the “best” model:

• Latency budget: a fixed end-to-end budget per frame (preprocess + inference + postprocess).
• Memory limits: GPU memory and RAM are smaller; bigger backbones can thrash.
• Precision tradeoffs: FP16/INT8 quantization can improve speed but may reduce small-object recall unless calibrated carefully.
• Operational stability: thermal throttling and long-running performance matter.

What I would do differently: one failure + one lesson (literature-backed)

Failure (common)

A model that looks strong on a validation set fails in the field due to domain shift (different camera, season, or weather). Typical symptom: mAP is okay offline, but live deployment shows missed detections on blur/low-light.

Lesson

Don’t treat “test set” as a single bucket. Evaluate and train for the real distribution of conditions:

• Stratify validation by condition (night/rain/blur/occlusion) and track recall per slice.
• Use targeted augmentation (motion blur, low-light, weather) and/or synthetic data, but validate synthetic→real transfer explicitly.
• Prefer deployment-driven model selection: the best paper metric is not the best field metric.

Reproducible demo (public/synthetic)

1. 01Public dataset (or synthetic generator) that mimics: small objects + blur + low-light.
2. 02Two training runs: YOLO vs RT-DETR baseline.
3. 03Export to ONNX/TensorRT and benchmark on Jetson (FPS, latency, memory).
4. 04Publish a concise README + one diagram showing the pipeline.

Built so far: step 01, scoped to dataset shift specifically rather than object detection. A small CPU-only demo that trains on synthetic shapes, measures how badly accuracy breaks under noise/blur/lighting shift, then shows shift-aware training recovering most of it. Public/synthetic data only, no client data, runs with one command: demos/dataset-shift-and-robustness. Steps 02–04 (YOLO vs RT-DETR, Jetson/TensorRT export and benchmarking) remain roadmap, not built.

References (starting points)

• Ultralytics YOLOv8 documentation + export/deployment guides (for ONNX/TensorRT).
• RT-DETR: “Real-Time DEtection TRansformer” (paper + official implementation).
• General literature on domain shift, dataset bias, and robustness evaluation in CV.