A RAG service built with FastAPI, LangChain, PostgreSQL with pgvector, and Qwen models served with vLLM.
┌─────────────┐
│ FastAPI │
└──────┬──────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌──────────┐
│ Postgres │ │ Embedding │ │ Reranker │
│ pgvector │ └───────────┘ └──────────┘
│chat_hist. │
└───────────┘ ┌─────────────┐
│ Qwen3-4B-AWQ│
└─────────────┘
Postgres stores two things: the document chunks with their embeddings in the
embedding pgvector table (PGVectorStore, HNSW index, cosine distance), and
the state of every chat session in the checkpoint* tables (LangGraph
AsyncPostgresSaver, session_id is the thread id), so requests sharing a
session_id keep their history. All tables are created by Alembic migrations
(apps/migrations/), which the one-shot migrations compose service
applies with alembic upgrade head before the API starts; the embedding table
name comes from POSTGRES__EMBEDDING_TABLE_NAME. The API checks on startup
that the checkpoint schema matches the installed langgraph-checkpoint-postgres,
so its upgrade with new migrations needs a new Alembic revision.
- Python 3.13+
- FastAPI
- LangChain
- langchain-postgres (
PGVectorStore, v2 engine) - PostgreSQL with pgvector
- psycopg (async driver)
- vLLM
- Docker Compose
- Qwen3-4B-AWQ
- Qwen3-Embedding-0.6B
- Qwen3-Reranker-0.6B
- Documents are split into chunks.
- Embeddings are generated and stored in PostgreSQL with pgvector.
- The query retrieves relevant chunks.
- Retrieved chunks are reranked.
- The most relevant chunks are passed to the generative model.
- The model generates the final answer.
Create a .env file and start the services:
docker compose up -dCheck the service status:
docker compose psStop the services:
docker compose downStart a conversation with its first message; the response contains the new
session_id:
curl -X POST http://localhost/sessions \
-H 'Content-Type: application/json' \
-d '{"message": "How do I deploy the service?"}'Continue it by sending messages to that session:
curl -X POST http://localhost/sessions/<session_id>/messages \
-H 'Content-Type: application/json' \
-d '{"message": "And in debug mode?"}'The second call sees the previous messages: the user question and the final answer of every turn are stored in Postgres through an async psycopg connection pool and passed to the agent on the next request.
Other session endpoints:
| Method | Path | Description |
|---|---|---|
GET |
/sessions?limit=&offset= |
Sessions, most recently active first |
GET |
/sessions/{id}?include_sources= |
History; with sources, the passages of every answer |
PATCH |
/sessions/{id} |
Rename: {"title": "..."} |
POST |
/sessions/{id}/regenerate |
Ask the last question again, replacing its answer |
DELETE |
/sessions/{id}/messages/last |
Remove the last question and its answer |
DELETE |
/sessions/{id} |
Delete the session |
A turn that fails leaves the history as it was before it.
The repository is a uv workspace with one lockfile and two members, each with its own dependencies and Dockerfile:
apps/api— the FastAPI serviceapps/migrations— the Alembic migrations
Install the dependencies of both:
uv sync --all-packagesRun linting and formatting:
uv run ruff check .
uv run ruff format .To drop the stored vectors and conversations, roll the migrations back and
apply them again (locally the host comes from apps/migrations/.env, since
.env points at the compose service):
uv run alembic downgrade base
uv run alembic upgrade headNote: the psycopg async driver (used by both the vector store and the
checkpointer) cannot run on the Windows proactor event loop that the plain
fastapi run entry point forces, so start the API locally with fastapi dev
(or uvicorn --reload).