Amazon Kiro and Bedrock Agents Lessons
An article on the challenges faced when building a personal web app using Amazon Kiro
The Day My AI Job Scraper Started Lying to Me
I built e-lise to solve a problem I’d lived through myself: hunting for jobs and scholarships across a dozen tabs, rewriting the same cover letter eight different ways, and losing track of which application needed what. The pitch was simple scrape real opportunities from the open web, use AI to tailor documents to each one, and stop doing the boring part by hand.
Then, a few weeks in, I opened my own opportunities page and saw a “scholarship” from a provider that, as far as I could tell, didn’t run that program. The link went to a generic homepage. The deadline looked suspiciously round. I’d built a system that was confidently making things up, and it had been doing it long enough that I’d stopped double-checking.
That’s the story I actually want to tell, because it’s not really a scraper bug. It’s a systems-design problem: what do you do when a piece of your pipeline can fail in a way that looks exactly like success?
Imagine you asked a friend to check if the toy shop had your favorite toy in stock, and instead of saying “I couldn’t get through, I don’t know,” they just guessed “yeah probably” and handed you an empty box wrapped like a present. It looks like an answer. It’s not one.
Where the lie was hiding
Going source by source through my scrapers, I found three flavors of the same mistake. One scraper (
fastweb.py ) never called a real endpoint at all it just generated eight plausible-looking scholarships every single run. Another ( boldorg.py ) tried real API guesses first, and only fell back to three hardcoded scholarships when those guesses failed, which they always did, since Bold.org doesn't publish a documented API. A third case was subtler: the UK job scraper required a paid API key, and when that key was missing (the default state), it silently substituted ten invented job listings instead of telling anyone the key was missing.None of these crashed. None of them logged anything a human would notice. The failure mode was indistinguishable from the success mode unless you clicked through and manually verified a link.
Here’s the actual before/after, trimmed down, from the Reed job scraper:
# beforeasync def scrape(self, max_results=50): if not settings.REED_API_KEY: listings = self._generate_reed_samples() # 10 invented UK jobs else: listings = await self._call_reed_api() return listings
# afterasync def scrape(self, max_results=50): if not settings.REED_API_KEY: logger.warning("[Reed] No API key configured, skipping (no fabricated fallback)") return [] return await self._call_reed_api()The fix isn’t clever, and that’s the point it’s just deleting the “helpful” fallback. An empty list is honest. A fabricated list is a landmine that goes off downstream, in a cover letter, pointed at a scholarship that may not exist.
The Bold.org case is more interesting because the fallback logic there felt reasonable when I wrote it “try three endpoint guesses, and if none of them work, at least show something.” The removal looked like this:
# The guessing part (kept, because it's honest it's a real attempt# against real endpoints, it just doesn't always succeed)for endpoint in ["/api/scholarships", "/api/v1/scholarships"]: try: data = await self._fetch_json(self.base_url + endpoint, ...) ... except Exception: continue# The part that got deleted entirelyif not opportunities: opportunities = self._generate_bold_samples() # 3 hardcoded scholarshipsI also tightened the URL handling in the ATS scrapers (Greenhouse, Lever). They used to fall back to a generic company careers homepage when a specific posting URL wasn’t available. That felt harmless “at least point them at the company”until I realized it means the app is telling a user “here’s a real posting” while actually handing them “here’s a company that maybe has some posting, somewhere.” Now a listing without a real, specific link just gets skipped.
It’s the difference between “no cookies today” and handing you a rock painted like a cookie. The empty jar is sad but true. The painted rock is worse, because you bite into it expecting a cookie.
The broader lesson for anyone wiring AI or scraping into a pipeline: fallback logic is where silent lies live. If your fallback is “make something up that looks plausible,” you’ve built a system that fails successfully and those are the hardest bugs to catch, because nothing tells you to look. Once I started treating “no data” as an acceptable, visible outcome instead of something to paper over, about a third of my scrapers turned out to be guesses wearing a real endpoint’s clothes.
The infrastructure decisions, explained
Once the data layer was honest, the next problem was speed. Document generation calls Claude on Bedrock, and LLM calls are slow and occasionally flaky not something you want blocking an HTTP request while a user stares at a spinner. So the API and the actual work got split into two processes talking through a queue:
┌─────────────┐ enqueue ┌──────────────┐│ API service │ ─────────────────▶ │ Task queue ││ (FastAPI) │ │ (SQS/memory) │└─────────────┘ └──────┬───────┘ ▲ │ │ status polling ▼ │ ┌──────────────┐ └────────────────────────────│ Celery worker │ │ (documents, │ │ scraping, │ │ auto-apply) │ └──────────────┘The broker selection is a single
if , and it's the actual infrastructure decision in miniature:if settings.SQS_QUEUE_URL and settings.ENV != "dev": # Production: SQS — a worker crash or restart doesn't lose the task, # it just sits in the queue until a worker picks it back up. broker_url = f"sqs://{settings.AWS_ACCESS_KEY_ID}:{settings.AWS_SECRET_ACCESS_KEY}@"else: # Local dev: in-memory — no external dependency to stand up # just to test a task locally. broker_url = "memory://"At home you can just yell “do this!” across the room and trust your sibling heard you. At school, you write it on a sticky note and hand it over, because if they walk away for a second, the yelled instruction is gone forever but the sticky note is still stuck to their desk. SQS is the sticky note.
The real decision here isn’t “queue vs. no queue,” it’s matching the durability of your infrastructure to the cost of losing a task not to what looks impressive in an architecture diagram. Locally, losing a task means re-running a command. In production, losing a task might mean a user’s application silently never got submitted, so it’s worth the extra moving part.
I made a similar call on rate limiting, in the opposite direction. The app runs as a single Uvicorn process (no multi-worker, no load balancer yet), so I used an in-memory sliding-window limiter instead of a Redis-backed one:
# in-memory, per-process — correct for a single-process deployment.# If this ever scales to multiple instances, each process would track# its own independent counts, and this should move to shared state._ai_generation_limiter = RateLimiter(max_requests=10, window_seconds=60)_scraping_limiter = RateLimiter(max_requests=3, window_seconds=60)Same principle as the queue, applied to a smaller problem: don’t reach for distributed infrastructure before you actually have multiple processes to coordinate. I left a comment saying exactly when that assumption breaks, so future-me doesn’t have to rediscover it the hard way.
There’s a bigger infranstructure pivot in the project’s history worth being honest about: I originally designed for ECS Fargate small autoscaling containers, CloudFront, the whole AWS-native shape and my GitHub Actions pipeline still builds and pushes Docker images to ECR and rolls ECS services. But the guide I actually followed to put this in front of two users runs on an EC2 instance instead. The fancy autoscaling design isn’t wrong, it’s just infrastructure sized for a scale I don’t have yet. Bedrock stays managed (pay-per-call, no server to run), but everything else API, worker, database connections runs on one small box with Nginx in front of it, because a queue-and-container architecture for near-zero traffic is a cost decision that fails the “does this make sense” test.
The model decisions, explained (with real numbers)
Somewhere in the build I had to answer: cheap-and-fast model, or slow-and-expensive model, for a given document? I didn’t want to guess, so I priced it out using AWS’s current published Bedrock rates. Claude Haiku 4.5 runs $1 per million input tokens and $5 per million output tokens ; Claude Sonnet 4.5 is exactly 3x that on both counts, $3 and $15 .
For a lean call just a prompt, no attached resume or job description that’s roughly half a cent on Haiku versus about a cent and a half on Sonnet for the same output length. A real document-generation call carries a lot more context (full resume, full job posting), which pushes both numbers up together, but the ratio holds: Sonnet always costs 3x what Haiku costs for the same job. At low volume, that’s nothing. At scale, it’s the line item that actually shows up on the bill.
So the routing in
content_gen.py isn't "always use the best model" or "always use the cheapest" it's tiered by how much a single output is worth to the person reading it:if model_tier == "high": model_id = settings.BEDROCK_MODEL_ID_SONNET # cover letters, essays — submitted once, matterselif model_tier == "light": model_id = settings.BEDROCK_MODEL_ID # Haiku — fast, cheap, reviewed casuallyelse: # medium (default) model_id = settings.BEDROCK_MODEL_IDThe chat feature does the same tiering by intent rather than a user toggle a status check (“where’s my Google application?”) doesn’t need the same model as “help me plan a career pivot”:
INTENT_MODELS = { "status_query": _HAIKU, "complex_task": _SONNET, "research": _SONNET, "interview_prep": _SONNET, "general": _HAIKU,}It’s like using a crayon for a doodle and the good paint set for the picture that’s going on the fridge. Both are “drawing,” but you don’t use the expensive paint for a doodle, and you don’t use a crayon for the fridge picture.
The CI/CD decisions, explained
The pipeline is boring on purpose. Three jobs, and the third can’t run unless the first two pass:
jobs: test-backend: steps: - run: pytest --tb=short -q
test-frontend: steps: - run: npx tsc --noEmit - run: npm run lint - run: npm run build deploy: needs: [test-backend, test-frontend] if: github.ref == 'refs/heads/main' && github.event_name == 'push' steps: - uses: aws-actions/amazon-ecr-login@v2 - run: docker build -t <registry>/api:${{ github.sha }} . && docker push ... - run: aws ecs update-service --force-new-deployment ...Three decisions worth naming:
-
needs: [test-backend, test-frontend]— the deploy job is structurally incapable of running before both test jobs succeed. Not a convention, a graph constraint.
- The
ifon deploy — it only fires on a push tomain, not on pull requests. PRs get the same test gate (so reviewers see red/green before merging) without ever touching production credentials.
-
--tbf-short, type-check, lint, build — no exotic checks. The gate caught real things: a missing environment variable that only broke in production config, a type mismatch that would've shipped a broken filter to users. None of it is exciting. That's the point. The exciting bugs are the ones that get through anyway; the boring gate is what stops the ones that shouldn't.
It’s like your mom checking you have shoes on and your backpack packed before you’re allowed out the doornot because either check is impressive, but because skipping either one means you show up somewhere and realize halfway there.
What I’d tell someone starting this build
Treat every AI-adjacent boundary a scraper, a fallback, a “just in case” default as a place a confident lie can hide. Test whether your failure paths are honest, not just whether your happy path works. When you’re deciding between infrastructure that looks correct and infrastructure that’s sized for your actual traffic, write down what a failure actually costs before you pick a lost background task costs differently than a lost HTTP request. And when you’re choosing between “fast and cheap” and “slow and good” for an AI call, look up the real per-token prices before you decide; it takes ten minutes and it’s usually not what your gut said.
The scraper is honest now. If a source is down, the page shows fewer results, not fake ones. That felt like a downgrade for about a day, and then it felt like the only version of this I should have shipped in the first place.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article