Designing RAG for an Embeddable Support Agent, and How I'd Move It to AWS

I built an embeddable AI customer-support agent as a portfolio project: a business uploads its documents, and a chat widget on its site answers questions from them. It runs on Next.js, FastAPI and Supabase with pgvector. It is deployed, but it has no real users yet, so this is a design write-up, not a production report. Below are the decisions that shaped it and how I would map each one to AWS.

The frontend is Next.js and the backend is FastAPI. The backend owns everything that touches a model: ingestion, retrieval and generation. That seam matters because the model provider and the vector store are the two things you are most likely to swap. On AWS, the FastAPI service could run on App Runner or ECS Fargate, and only the model client and the retrieval code would change.

Documents are split into overlapping chunks, embedded, and stored with their source and position. Re-running ingestion is how the agent learns a new document, with no retraining. On AWS, the raw files would sit in Amazon S3 and the chunks would be embedded with Titan Text Embeddings on Amazon Bedrock. The vector column width has to match the embedding model, so switching models means re-embedding everything.

I used Postgres with pgvector, so vectors live next to the tenant, document and conversation tables. That allows an ordinary SQL filter in the same query as the similarity search. The AWS equivalent is Aurora PostgreSQL with pgvector. If you would rather not manage chunking and sync yourself, Amazon Bedrock Knowledge Bases is the managed alternative, at the cost of some control.

A support platform serves many businesses, so every retrieval query must be filtered by tenant. That filter should be enforced in one place, such as row-level security or a mandatory parameter in the data-access layer, so no code path can forget it. A leak between tenants is the worst bug this kind of product can have.

The model should answer only from the retrieved passages and say so when it does not know. Returning the source with each answer lets the business verify it. On AWS, Amazon Bedrock Guardrails can add input and output checks on top of that.

In a RAG system, a bad answer is more often a retrieval problem than a model problem. Chunk size, overlap and the number of passages passed to the model are the first things to tune. Hybrid search (keyword plus vector) and reranking are the usual next steps.

- Build a fixed set of test questions with the passages that should be retrieved, and score retrieval before changing anything else.
- Add hybrid search and compare it against vector-only retrieval on that test set.
- Port the model and embedding calls to Amazon Bedrock and compare cost and latency.

If you have run pgvector on Aurora for multi-tenant RAG, how do you handle per-tenant filtering as the corpus grows: one table with a tenant column and an index, or separate schemas?