Most camera-trap photos show nothing. I let a 2B model decide which ones a vision model sees
Strands Decider 2B checks every camera-trap frame before a vision model does. On cameras it had never seen, it cleared nearly half the empty frames for free and still caught 96.7% of animals.
A ranger who pulls the card from a camera trap gets a night of photos, and about half of them show nothing. In the Caltech Camera Traps dataset, 125,745 of 243,100 images carry the label empty, which is 51.7%. I built Trailgate so a 2B decision model looks at every frame before anything expensive does. On 600 frames from cameras it had never seen, it cleared 48.4% of the empty frames without calling a vision model and still caught 262 of 271 animal frames (96.7%).
A camera trap fires on wind and grass as readily as on a bobcat. Send every frame to a vision model and you pay for every empty one. Send them all to the cheap model and you get poor labels on grainy infrared shots taken at 3am. So Strands Decider 2B goes first, one call per frame. The router skips empty frames, Amazon Nova Lite labels the animals Decider is confident about, Amazon Nova Pro gets the frames it's unsure about, and a person or vehicle goes to the ranger's phone.
The code, raw Decider answers and charts are on GitHub . The live demo replays real Decider
answers in the ranger console.
What runs on AWS, and why
- Strands Decider 2B makes the routing decision on every frame. It runs locally in 472 ms at the median on my laptop, for $0 per call. Because it returns probabilities, the policy is a threshold I can defend with a chart.
- Amazon Nova Lite on Bedrock labels the species when Decider is confident. It costs $0.06 per million input tokens.
- Amazon Nova Pro on Bedrock gets only the frames Decider is unsure about. At $0.80 per million input tokens it costs 13 times as much as Nova Lite, so it never sees the easy ones.
- The Bedrock Converse API uses one call shape for both models, so moving a frame between tiers changes one model id.
- Strands Agents puts the decision in the tool path. An
InterventionHandlerdenies the tool call for an empty frame before any model runs, and picks the tier for everything else.
- Amazon SNS can push the ranger alert to a phone. Alerts also land in a local outbox.
Going by Bedrock's published prices, Trailgate comes to an estimated $5.55 per 10,000 frames. Nova Pro on every frame would cost $16.03, and Nova Lite on every frame $1.20. All three are estimates, because my account's Nova quota was 0. I explain the method further down.
The decision I handed to Decider
Decider is a 2B decision model. It runs one forward pass, returns probabilities and never writes text. With
--vision it reads images zero-shot. I ask it three typed questions about each frame in one call, so it encodes the image once. Two are noul questions, Decider's yes/no type, and each returns P(true). The third is a choice with two options, clear and hard .Python
{"present": {"type": "noul","instructions": "Is there an animal anywhere in this photo, even partly hidden or at the edge?","criteria": {"true": "an animal or part of one is visible","false": "only vegetation, ground, sky, or camera artefacts",},},"escalate": {"type": "noul","instructions": "Is there a person or a vehicle in this photo?","criteria": {"true": "a person, car, truck, motorbike or other vehicle, or part of one, is visible","false": "only animals, vegetation, ground, sky, or camera artefacts",},},"difficulty": {"type": "choice","instructions": "How easy is the subject to see?","criteria": {"clear": "the subject is large, lit and in focus","hard": "the subject is small, dark, blurred, infrared or partly hidden",},},}
Plain Python turns those probabilities into a route. I set these v2 thresholds on a separate tuning set.
Route | Rule | Bedrock call |
|---|---|---|
Alert | P(person or vehicle) ≥ 0.54 in any frame of the burst | Nova Lite writes the alert, Nova Pro labels the frame |
Skip | the highest P(animal) in the burst is below 0.41 | none |
Nova Lite · Decider confident | P(animal) ≥ 0.72 | Nova Lite |
Nova Pro · Decider unsure | everything in between | Nova Pro |
The alert and skip rows look at the whole camera burst. That rule fixed Decider's worst mistake, which I'll get to.
I tagged this
#model-routing because that's the question Decider answers. Does this frame need the big model, the small one, or nothing? Take Decider out and the agent can't skip a frame or pick a tier, and it has no idea when to wake someone.
Where it sits in the Strands agent
The ranger's agent has five tools,
list_frames , archive_empty , label_species , alert_ranger and daily_digest . A Strands InterventionHandler I called PresenceRouter sits in front of label_species . Before the tool runs, the handler asks Decider about the image. For an empty frame it returns Deny , and Bedrock never sees the frame. Otherwise it returns Transform , which writes model_tier into the tool's arguments. For a person or vehicle it also sets escalate=True , and the agent calls alert_ranger .The handler fails open. If Decider throws, the handler catches the error and sends the frame to the large tier, and
on_error = "proceed" covers anything else. I'd rather a broken router cost money than lose a photo.A plain pipeline,
python -m trailgate run <folder> , does the same routing with no LLM in the loop. It's also the only path that applies the burst rule. The agent sees one tool call at a time, so it routes frame by frame, and that's the weaker policy in my ablation.How I measured it
I split Caltech Camera Traps by camera location, so no test frame comes from a camera I tuned on.
- Tuning set. 80 frames from 13 locations. I picked the wording and every threshold here.
- Held-out set. 600 frames from 71 other locations in their natural mix of 310 empty, 271 animal and 19 vehicle. I froze v1's thresholds before running it. v2 came after, and I say below what that means.
- Vehicle set. 40 more
carframes from held-out locations, because 19 vehicles is too few to judge alerts.
Camera traps fire in bursts, mostly one or three frames long, so I fetched the rest of every held-out burst too, 1,079 frames in 599 bursts, each with its own Decider call. The raw answers are in the repo, and
python scripts/evaluate.py --no-ask rebuilds every table without a GPU.Decider ran on an Apple M5 laptop with 16 GB, on PyTorch's MPS backend. Over 1,119 calls it took 472 ms per frame at the median and 580 ms at p90. One thing caught me out.
--vision refuses MLX.Results
Held-out | Empty frames skipped | Animal frames missed | Vehicles caught |
|---|---|---|---|
All | 150/310 (48.4%) | 9/271 (3.3%) | 57/59 |
Day | 118/228 (51.8%) | 5/112 (4.5%) | |
Night (infrared) | 32/82 (39.0%) | 4/159 (2.5%) | |
159 of the 600 frames, or 26.5%, went to the archive without touching a model. That counts one sampled frame per burst, the unit the dataset labels. Counted over all 1,079 frames in those bursts, 17.4% reach no model and the router skips 23 of 706 animal frames, still 3.3%. Across the held-out and vehicle sets, 57 of 59 vehicle frames raised an alert.
I opened all 9 missed animal frames at full size. One plainly shows a bird's tail in a corner, at P(animal) 0.395. Four show something faint, like eye shine. In the other four I can't find an animal at all. Caltech labels whole bursts, so a frame can say "coyote" after the coyote has left. I still count those four as misses.
What happens when Decider is unsure
When Decider puts P(animal) at 0.9 or higher, 171 of 172 frames really hold an animal. Between 0.2 and 0.7 it isn't calibrated at all. Frames scored 0.6 to 0.7 held an animal 11% of the time, against 21% for frames scored 0.4 to 0.5. So Trailgate trusts the top of the scale and skips the bottom. It pays for Nova Pro only in the middle band, where Decider's number means little. On the held-out set that band held 165 of 600 frames.
The skip threshold is the safety knob. This chart shows what each setting costs.
With the skip threshold at 0.20, v2 misses no held-out animal but skips only 5.2% of empty frames. At 0.50 it skips 63.5% of empty frames and misses 17 of 271 animal frames, or 6.3%. Where to set it is the ranger's call.
Cost and latency against Nova on every frame
System | $ per 10,000 frames (estimated) |
|---|---|
Nova Pro on every frame | $16.03 |
Trailgate v2 | $5.55 |
Nova Lite on every frame | $1.20 |
The Bedrock numbers are estimates. My AWS account had a Bedrock daily token quota of 0 for every Nova model, and the quota wasn't adjustable. So I multiplied the published on-demand prices by Nova's documented image token counts, about 1,944 input tokens per frame.
scripts/run_nova_baseline.py measures cost, latency and labels for both Nova models on any account with Nova access. I don't have a measured Nova latency to put next to Decider's yet.What I did measure is the decision itself, 472 ms and $0 per frame. For 26.5% of frames that's the whole bill. By the estimate, Trailgate costs 34.6% of Nova Pro on every frame. It costs more than Nova Lite on every frame, because the 165 unsure frames go to Nova Pro. If the baseline shows Lite is good enough on those frames, setting
TRAILGATE_LARGE_MODEL moves that tier to Lite and the estimate drops to $0.93, below Lite on every frame.Where Decider was wrong, and what I changed
My first policy, v1, sent every frame Decider called "hard" to Nova Pro. On the held-out set it skipped 41.9% of empty frames, missed 16 of 271 animal frames and cost an estimated $10.27 per 10,000 frames. That's more than eight times Nova Lite on every frame. Two things in the data explained it.
The difficulty question barely works. Decider called 504 of 600 held-out frames "hard", including 192 of 271 animal frames. As a night-versus-day signal, P(hard) scores an AUC of 0.67, where 0.5 is a coin flip. Routing on it sent 356 of 600 frames to Nova Pro. v2 still asks the question, since one more question in the batched call costs almost nothing, but it ignores the answer. The middle band of P(animal) turned out to be a better sign of doubt than asking Decider whether it was in doubt.
Most misses came from bursts. Of v1's 16 misses, 13 were frame 2 or later of a burst, often taken after the animal had left. v2 skips a frame only if no frame in its burst reaches P(animal) 0.41, and a frame that a sibling keeps alive goes to Nova Pro. At the same thresholds, that rule cut missed animal frames from 23 to 9.
Policy | Empty frames skipped | Animal frames missed | $ per 10,000 (estimated) |
|---|---|---|---|
v1: Nova Pro when Decider says "hard" | 41.9% | 16/271 | $10.27 |
v2 without the burst rule | 49.0% | 23/271 | $5.10 |
v2: confidence band + burst rule | 48.4% | 9/271 | $5.55 |
v2's threshold values came from the tuning set only. Its shape is another matter. I chose that shape after seeing where v1 failed on these same held-out frames, so v2's numbers are a second look at those frames and weaker evidence than v1's first run. The fair next test is a fresh set of cameras. Both policies and the ablation are in the repo.
Other places Decider got it wrong or surprised me:
- Night infrared is harder. It skipped 39.0% of empty frames at night against 51.8% by day.
- The tuning threshold didn't carry over cleanly. The presence question scored AUC 0.954 on tuning cameras and 0.900 on held-out ones, and the v1 threshold that missed no tuning animals missed 16 held-out ones.
- It found people the dataset calls empty. Caltech Camera Traps has no person class, so its labels mark staff walking past as
empty. Of the 4 held-out alerts on frames without acarlabel, 3 show people. The fourth is a real false alert, a deer under a street lamp that scored 0.77 for "person or vehicle".
- Two vehicles got through. One is a dark dawn frame where I can't see a vehicle either. The other is a motion-blurred shape under a bridge deck.
What moved the numbers
Following Mike Chambers' method, I scored five wordings of the presence question on the tuning set, all in one batched call per frame. Adding explicit true/false criteria barely moved AUC, from 0.951 to 0.954. At a 0.35 threshold, though, it lifted recall from 88.4% to 100%. Flipping the question to "Is this photo empty?" did worse. Batching didn't cost accuracy either. The same wording asked in a 9-question call and in the 3-question production call differed by at most 0.004.
I also tried one call per burst with up to four frames attached. It agreed with max-over-frames on all 29 multi-frame tuning bursts and was faster, 621 ms against 1,473 ms per burst. Only one of those bursts was empty, though, which is too thin to trust, so I kept per-frame calls.
PresenceRouter, the part worth copying
patterns/presence_router/ is the code meant to be copied, and nothing in it is specific to wildlife. questions.py builds the typed questions from two plain strings, target and escalate . router.py holds the confidence band and the burst rule and imports no framework. handler.py is the Strands InterventionHandler , and tune_questions.py scores wordings on your own labelled images.Here it guards a delivery-photo agent and skips doorstep photos with no parcel in them.
Python
router = PresenceRouter(PresenceRouterCore(target="a parcel or package", escalate=None, thresholds=Thresholds(skip_below=0.3)),tools={"describe_image"},)agent = Agent(tools=[describe_image], interventions=[router])
examples/parcel_photos.py is that agent, ready to run. To set thresholds, label about 50 of your own images and run python -m patterns.presence_router.tune_questions . It prints the highest skip threshold that misses nothing. Pick a value a little below it, then check it on images you didn't tune on. I watched my own threshold fail to transfer, so that last step is the one I'd least want anyone to skip.What I learned
Decider is best at the extremes. Above 0.9 it was almost never wrong, and every held-out frame it scored below 0.2 was empty. The middle is where it's telling you it doesn't know, and that's where a bigger model earns its price.
The question I expected to matter most, how hard the subject is to see, did the least. Decider is much better at saying what's in a picture than at judging its own difficulty. I now read uncertainty off the presence probability instead of asking for it.
Most of my gains came from opening frames and looking at them. Bursts explained the misses, and missing person labels explained the "false" alerts. Testing on unseen cameras cost me the clean zero-miss result I had on tuning, and I'd rather report that than hide it.
Next I want the measured Nova baseline on the unsure band, then a second dataset such as Snapshot Serengeti to see how far the thresholds move.
Links
- Dataset: Caltech Camera Traps , from Beery et al., ECCV 2018, under the Community Data License Agreement, permissive variant
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article