Skip to content

Repository files navigation

FastAPI RAG

A RAG service built with FastAPI, LangChain, PostgreSQL with pgvector, and Qwen models served with vLLM.

Architecture

                     ┌─────────────┐
                     │   FastAPI   │
                     └──────┬──────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
        ┌───────────┐ ┌───────────┐ ┌──────────┐
        │  Postgres │ │ Embedding │ │ Reranker │
        │  pgvector │ └───────────┘ └──────────┘
        │chat_hist. │
        └───────────┘       ┌─────────────┐
                            │ Qwen3-4B-AWQ│
                            └─────────────┘

Postgres stores two things: the document chunks with their embeddings in the embedding pgvector table (PGVectorStore, HNSW index, cosine distance), and the state of every chat session in the checkpoint* tables (LangGraph AsyncPostgresSaver, session_id is the thread id), so requests sharing a session_id keep their history. All tables are created by Alembic migrations (apps/migrations/), which the one-shot migrations compose service applies with alembic upgrade head before the API starts; the embedding table name comes from POSTGRES__EMBEDDING_TABLE_NAME. The API checks on startup that the checkpoint schema matches the installed langgraph-checkpoint-postgres, so its upgrade with new migrations needs a new Alembic revision.

Stack

  • Python 3.13+
  • FastAPI
  • LangChain
  • langchain-postgres (PGVectorStore, v2 engine)
  • PostgreSQL with pgvector
  • psycopg (async driver)
  • vLLM
  • Docker Compose
  • Qwen3-4B-AWQ
  • Qwen3-Embedding-0.6B
  • Qwen3-Reranker-0.6B

RAG Pipeline

  1. Documents are split into chunks.
  2. Embeddings are generated and stored in PostgreSQL with pgvector.
  3. The query retrieves relevant chunks.
  4. Retrieved chunks are reranked.
  5. The most relevant chunks are passed to the generative model.
  6. The model generates the final answer.

Run

Create a .env file and start the services:

docker compose up -d

Check the service status:

docker compose ps

Stop the services:

docker compose down

Conversations

Start a conversation with its first message; the response contains the new session_id:

curl -X POST http://localhost/sessions \
  -H 'Content-Type: application/json' \
  -d '{"message": "How do I deploy the service?"}'

Continue it by sending messages to that session:

curl -X POST http://localhost/sessions/<session_id>/messages \
  -H 'Content-Type: application/json' \
  -d '{"message": "And in debug mode?"}'

The second call sees the previous messages: the user question and the final answer of every turn are stored in Postgres through an async psycopg connection pool and passed to the agent on the next request.

Other session endpoints:

Method Path Description
GET /sessions?limit=&offset= Sessions, most recently active first
GET /sessions/{id}?include_sources= History; with sources, the passages of every answer
PATCH /sessions/{id} Rename: {"title": "..."}
POST /sessions/{id}/regenerate Ask the last question again, replacing its answer
DELETE /sessions/{id}/messages/last Remove the last question and its answer
DELETE /sessions/{id} Delete the session

A turn that fails leaves the history as it was before it.

Development

The repository is a uv workspace with one lockfile and two members, each with its own dependencies and Dockerfile:

  • apps/api — the FastAPI service
  • apps/migrations — the Alembic migrations

Install the dependencies of both:

uv sync --all-packages

Run linting and formatting:

uv run ruff check .
uv run ruff format .

To drop the stored vectors and conversations, roll the migrations back and apply them again (locally the host comes from apps/migrations/.env, since .env points at the compose service):

uv run alembic downgrade base
uv run alembic upgrade head

Note: the psycopg async driver (used by both the vector store and the checkpointer) cannot run on the Windows proactor event loop that the plain fastapi run entry point forces, so start the API locally with fastapi dev (or uvicorn --reload).

Contributors

Languages