27 Jan 2026 · Notes

Edge CV Under Domain Shift

What actually breaks in the real world (and how to measure it)

Context: In outdoor/industrial inspection, models rarely fail because the architecture is “wrong”. They fail because the data distribution changes and we were measuring the wrong thing.

Domain shift is not a research buzzword: it’s the default

If you train a detector on “mostly sunny, stable camera, clean labels” and deploy it on “rain + motion blur + partial occlusion”, you are not doing the same task anymore. Your input distribution changed (sometimes subtly), and the failure mode usually appears as false negatives.

The trap: a single validation score

A single overall metric (like mAP) can hide the exact cases you care about. In inspection, the worst failures cluster in:

• Small objects (few pixels, easy to miss)
• Motion blur (fast movement, vibration, rolling shutter)
• Low light / harsh contrast (night, tunnels, glare)
• Occlusion (partial visibility, clutter)
• Rare classes (defects)

The fix: evaluate in slices

The best practical upgrade you can make is to stop treating the dataset as one bucket. Create “slices” that mirror reality:

• Day vs night
• Rain/wet vs dry
• High blur vs low blur
• Occluded vs unoccluded
• Small-object subset (area threshold)

Then track recall (or false-negative rate) per slice. This makes model improvements measurable and prevents you from over-optimizing for the easy conditions.

for slice_name, subset in slices.items():
    m = evaluate(model, subset)
    print(f"{slice_name:>12}  recall={m.recall:.3f}  fn={m.false_negatives}")
# aggregate mAP hides the slice that actually fails

Augmentations: useful, but only when targeted

Generic augmentation can help, but inspection problems benefit more from targeted, realistic transforms:

• Motion blur (directional, variable kernel)
• Low-light + noise
• Exposure/contrast shifts
• Weather overlays (careful: don’t create “synthetic artifacts” the model overfits to)

Synthetic-to-real: treat it like a hypothesis, not a guarantee

Synthetic data is powerful for rare defect coverage, but it’s easy to fool yourself. The correct question is: does synthetic data improve real-world recall on the slices that matter? If not, you may need better rendering realism, domain randomization, or fine-tuning on a small real set.

A single overall metric can hide the exact cases you care about.

Edge deployment changes what “best model” means

On Jetson (or any edge target), you don’t optimize for benchmark leaderboards. You optimize for:

• Stable latency (not just average FPS)
• Memory footprint
• Precision mode (FP16/INT8) impact on small objects
• Long-running stability (thermals, throttling)

A simple weekly routine that compounds

• Pick one slice (e.g., blur + small objects)
• Measure recall + FN count
• Make one change (augmentation, data cleaning, thresholding, training schedule)
• Re-measure and write a 5–10 line note

Do this weekly and you’ll build a proof portfolio that reads like real engineering, not generic AI hype.