
I Taught My Phone to Recognize My Own Suitcase
Skycase uses Amazon Bedrock and Rekognition so a phone camera can spot your bag on the carousel and catch a missing one early.
Baggage claim hasn't really changed in thirty years. You still squint at a moving line of nearly identical black suitcases, half-grab the wrong one, and eventually start wondering whether yours is coming at all. None of that is fixable from the outside — I have no access to an airline's loading logs or a conveyor's real sensor data — but a lot of it turns out to be solvable with nothing more than the camera already in your pocket. So I spent a weekend building exactly that.
Scoping it to the passenger's side only
The first real decision was what not to build. No airline integration, no airport data feed, nothing that assumes access I don't have. Just two things a phone camera can do on its own:
Photograph your bag once before you fly, then have the phone highlight it on the belt so you don't grab a look-alike. And if the belt keeps producing bags that clearly aren't yours for a while, flag that your bag might not have arrived and draft a lost-baggage report on the spot.
Both features watch the same camera feed, so it became one app instead of two.

Checking Bedrock and Rekognition actually worked before writing anything
Before designing anything, I confirmed the models were actually callable in this AWS account — a model showing up in
list-foundation-models doesn't mean InvokeModel will succeed.aws bedrock-runtime invoke-model \ --region us-east-1 \ --model-id amazon.titan-embed-image-v1 \ --body '{"inputText":"test suitcase"}' \ --cli-binary-format raw-in-base64-out \ /tmp/titan_test_out.jsonThat returned a real embedding. Claude Sonnet needed one more step:
An error occurred (ValidationException) when calling the InvokeModel operation:Invocation of model ID anthropic.claude-sonnet-4-5-20250929-v1:0 with on-demand throughput isn't supported.Retry your request with the ID or ARN of an inference profile that contains this model.Swapping in the inference profile ID (
us.anthropic.claude-sonnet-4-6 ) fixed it. Rekognition's DetectLabels checked out too. Only after all three actually responded did I start writing app code.Designing around multiple bags in one frame
A real carousel has several bags in view at once, so the pipeline has to handle that instead of assuming one clean shot. Rekognition's
DetectLabels returns Instances[].BoundingBox for Suitcase/Luggage labels, which lets the client crop each candidate out of the frame, embed each crop separately with Titan Multimodal Embeddings, and compare it against the registered bag's vector.I used two similarity thresholds instead of one:
export const MATCH_THRESHOLD = 0.9;export const AMBIGUOUS_THRESHOLD = 0.8;A score at or above 0.9 gets a confident green highlight. If two or more crops land at or above 0.8 in the same tick, the UI switches to a yellow "similar bags detected — check the tag" warning instead of silently picking a winner. Titan's embedding space isn't calibrated to some universal "0.9 means identical" scale, so these numbers are a starting point I expect to retune against real photos, not a finished constant.
The "belt looks idle" signal is a similarly rough heuristic: keep a rolling 60-second window of recently seen bag embeddings, and if a newly detected bag doesn't match any of them above 0.9, count it as a distinct new arrival. If 45 seconds pass with no new arrival and the user's own bag still hasn't matched, the app suggests drafting a report. No object tracking across frames, no belt-speed modeling — a workable shortcut for a personal project, not a rigorous algorithm.
Reusing an OIDC pattern I'd already built for a different app
The app deploys to Vercel, and I didn't want a single static AWS access key anywhere in it. I reused a Vercel-to-AWS OIDC Federation setup from an earlier project almost unchanged:
function credentials() { const roleArn = process.env.AWS_ROLE_ARN; if (roleArn) { return awsCredentialsProvider({ roleArn, roleSessionName: "skycase-app-web" }); } return fromNodeProviderChain();}The IAM role's trust policy is scoped on both the
aud and sub claims:Condition: StringEquals: "oidc.vercel.com/yuuki-yamashitas-projects:aud": "https://vercel.com/yuuki-yamashitas-projects" "oidc.vercel.com/yuuki-yamashitas-projects:sub": "owner:yuuki-yamashitas-projects:project:skycase-app:environment:production"One detail I only caught by actually checking: Claude's inference profile routes requests across three regions, so the IAM policy needs
bedrock:InvokeModel on the foundation-model ARNs in us-east-1 , us-east-2 , and us-west-2 individually — not just the inference-profile ARN itself. aws bedrock get-inference-profile is what surfaced that.A stale closure made the camera error show up in two languages at once
While testing in the browser, a camera-permission failure briefly rendered its error message in English and Japanese simultaneously. The cause was a stale closure inside a
useEffect :useEffect(() => { // ... } catch { setError(true); onError?.(); // called the onError captured back at mount time } // ...}, []); // deliberately empty — restarting the camera on every render would be worseThe
getUserMedia startup effect runs once by design, so it doesn't restart the camera stream on unrelated re-renders. But the onError callback passed in as a prop is a fresh function on every parent render, so by the time the async rejection actually fired, it called the original mount-time closure — which happened to still be holding the pre-language-switch text, while the component's own live-rendered fallback showed the new language. The fix was routing the callback through a ref that's updated every render, so the effect always calls the latest version instead of the one it closed over at mount.Running the newer
eslint-config-next (16.3.0) also flagged several places where I was reading ref.current during render or seeding a useRef with Date.now() directly — the newer react-hooks/refs and react-hooks/purity rules catching real correctness issues, not just style nits.Confirming OIDC actually worked in production, with curl
Locally, missing
AWS_ROLE_ARN just falls back to the default AWS CLI credential chain, so local testing alone can't prove OIDC is wired correctly. After deploying, I hit the production URL directly:curl -s -X POST https://skycase-app.vercel.app/api/embed \ -H "Content-Type: application/json" -d @test_payload.json{"embedding":[-0.0012723518,0.018475832,...A 200 with a real embedding back is the actual proof that
AssumeRoleWithWebIdentity exchanged the Vercel OIDC token for temporary STS credentials. A 403 at that specific point — as opposed to locally — almost always means either a sub claim mismatch or a missing IAM action, which is exactly the failure mode local testing can't catch.Renaming the project tripped a Vercel protection setting I hadn't touched
Partway through, the app got renamed from an earlier working title to Skycase, which meant renaming the Vercel project too. After re-aliasing the production domain to the new name, that specific URL started returning:
{"error":{"code":"401","message":"Protected deployment"} ...The project's
ssoProtection setting hadn't changed — it was all_except_custom_domains the whole time — but manually re-aliasing the domain apparently made that protection actually apply to it, where it hadn't before. Disabling it fixed the public access immediately. That specific command got blocked by my own environment's permission classifier as a protection-setting change, so I had to get explicit sign-off before running it — a small reminder that even a routine rename can wander into something that looks, on paper, like a security-relevant action.
What's actually running
Take one photo of your bag before you fly — that gets embedded with Titan Multimodal Embeddings and stored client-side only, no server database at all. At the carousel, the phone samples the camera every 1.8 seconds, has Rekognition find candidate bags in frame, crops and embeds each one, and compares it against the registered bag. A match gets a green highlight, ambiguous matches get a yellow warning, and a stalled belt with no match yet offers to draft a lost-baggage report using Claude Sonnet 4.6. Vercel talks to AWS purely through OIDC Federation — no static access keys anywhere in the deployment.



What I haven't verified yet
The browser environment I used for development is sandboxed and has no real camera hardware, so the actual multi-bag detection, the ambiguous-match warning, and the belt-idle heuristic haven't been tested against a real physical scene yet — that's the obvious next step. The 0.8/0.9 similarity thresholds are placeholders pending real photos. What I have confirmed, end to end and in production, is that the Bedrock, Rekognition, and OIDC wiring actually works. Everything past that is honestly still "probably works" until I try it at an actual carousel.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article