Service-Side Infra - RAG, Vector, and a Local Service API

This piece starts the second project. It stacks on the first, so you need the node from Part 2 standing before any of this makes sense. The code lives in ai-stack, and like the node it is a one-shot deploy into its own capped resource group.

So far the system can act. Now we give it knowledge. Three pieces, all in one resource group.

Retrieval (RAG)

RAG means "look it up first, then answer." Instead of hoping the model memorized your facts, the system fetches the relevant ones at question time and hands them to the model along with the question. The model's answer is grounded in your material, not its training.

A vector store

To "fetch the relevant ones" you need a way to find text by meaning, not exact words. A vector store turns each piece of text into a list of numbers that captures its meaning, so "close in meaning" becomes "close in space" (more on the idea in embeddings). We use Postgres with the pgvector extension, so the documents, their metadata, and their vectors all live in one database you can actually open and inspect.

A local service API

The rest of the system should not talk to the database directly. A small service API is the one internal door everything goes through, wired for both internal use and scoped outside access. It keeps the data plane clean and swappable.

Cost note

This layer costs a bit more than the node. Container pieces scale to zero, but Postgres has an hourly floor, so the cap sits higher than the node's roughly ten dollars. Still small, still capped, still yours.