Code - Sherpa (TEAM MERIDIAN)
Code Sherpa takes a GitHub issue link and explains the bug in plain language: what's going wrong, which file to open, and how to fix it. This post covers how we built it with FastAPI, Streamlit, Gemini and Amazon S3, how we pick the right files from a large repo, and why we moved from Bedrock to Gemini after running into authentication problems.
Every beginner has had this moment. You want to contribute to a project, so you open the Issues tab and pick one that looks doable. Then you read it. The description talks about race conditions and middleware, the repo has four hundred files, and you close the tab.
We built Code Sherpa for that moment, as part of FIrst Commit by WeMakeDevs x AWS. You paste a GitHub issue link, and it tells you what's going wrong, which file to open, how to fix it, and why it matters. It uses plain language that a first-year student can follow. This post covers how we built it, what we used, and the part where our original plan didn't work out.
What it does
Code Sherpa has two parts. A Streamlit app takes the issue URL. A FastAPI backend does the actual work. It reads the issue, downloads the repository, picks the files that look relevant, sends them to a language model, and returns a short explanation.
Every answer has the same four parts: what's going wrong (with one everyday analogy), where the bug is, how to fix it, and what happens if nobody does.
What we used
- Frontend: Streamlit with custom CSS
- Backend: FastAPI and Uvicorn, hosted on Render's free tier
- Source data: the GitHub REST API
- AI: Google Gemini through the google-genai SDK
- Storage: Amazon S3, through boto3, for scan receipts
- Access control: a dedicated IAM user for the S3 writes
Picking the right files
A repository can hold thousands of files, and a model can't read all of them, so we don't send everything. The backend downloads the repo as a zip straight into memory and scores every file against the issue.
A file named in the issue goes to the top. After that, words from the issue are matched against file paths and file contents, and the title counts double. Test files and docs get pushed down. Folders like node_modules and build output, along with lockfiles and minified files, are skipped entirely. The best-scoring files fill a budget of about 60,000 characters, and that's all the model sees.
It's plain scoring, nothing fancy, but it keeps every request small and focused on the issue. Without a GitHub token we download from the github.com archive links, which don't count against the API rate limit.
Getting the tone right
The prompt tells the model to act like a patient mentor talking to someone who has never seen the codebase. Sentences stay short, jargon gets explained in brackets, and there's exactly one analogy. It also has hard rules. The model can only mention files and line numbers that appear in the code we send. If it isn't sure, it has to say so and label its answer as a guess. Fixes stay minimal, and the whole thing stays under about 300 words.
The issue text and the code go into the prompt wrapped in tags, and the model is told to treat them as material to read, never as instructions. We also strip lookalike tags out of that text, so an issue can't close our tag and slip in commands of its own.
Where Bedrock didn't work out
Our plan was to run the explanation step on Amazon Bedrock, which would have kept the whole project inside AWS. We couldn't get verification for our aws account. We spent a good chunk of time on it, and at some point you have to ship something that works. So we moved that one step to Google Gemini.
The one good thing about the detour is that the model call lives in a single function. GitHub fetching, file scoring, the prompt, the S3 receipts and the UI all stayed exactly as they were. Going back to Bedrock later means rewriting that one function, and we plan to do that once we've sorted out the authentication problem.
Once we were on Gemini, we tried to make it as good as we could. That meant working on the prompt, the retries and the API keys.
Keeping Gemini's limits alive
Free-tier limits run out fast when several people are testing at once. So the backend accepts a list of Gemini API keys, comma separated in one environment variable, and each request uses one picked at random. It's simple random choice, not smart failover, but it stops one key from taking all the traffic.
When Gemini answers with a 429 or a 503, the backend waits 4 seconds, then 8, and tries up to three times. If it still fails, the user gets a clear message instead of a stack trace.
S3 receipts
Every successful scan drops a small JSON file into an S3 bucket under a
receipts/ prefix. It holds the issue URL, the verdict and a timestamp. It gives us a simple record of what people scanned and what answer they got.The IAM user behind it can only do
PutObject on that prefix, and the bucket keeps public access blocked. If the S3 write fails, the user still gets their answer.The frontend
The Streamlit app runs like a pipeline. It validates the link, pulls the issue from GitHub for a quick preview, connects to the server, runs the analysis, and prepares the verdict. Each stage shows its status, and there's a live timer.
Render's free tier goes to sleep when idle, so the first request after a pause can take close to a minute. The app notices this and shows "waking server" instead of an error. It also rejects pull request links with a friendly message, lets you download the verdict as a Markdown file, and includes a short guide on finding an issue URL for people who have never done it.
Guardrails
- Only github.com issue URLs are accepted, so the backend can't be pointed at random sites.
- Repositories over 100 MB are refused, files over 200 KB are skipped, and at most 5,000 files are read.
- Repository code is never written to disk.
What's still rough
The explanation can be wrong, so treat any suggested fix as a starting point and check it before merging. Very large bugs that spread across many files may only get a best guess, because we send a limited amount of code.
What's next
- Move the analysis back to Bedrock
- Skip a key when it's rate limited, instead of picking purely at random
- Explain pull requests, not just issues
- Cache results per issue to save quota
Try it
Code: K-Advika/Code-Sherpa If you're new to open source, pick a small issue and paste it in. If the explanation is wrong or confusing, we'd like to hear about it.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article