Chat with your own documents — PDFs, scans, Word files — entirely on your machine, no API keys.

---

Overview

Stage	Behavior
Ingest	PDFs (PyMuPDF + page OCR when needed), images (EasyOCR), DOCX, TXT
Index	Recursive split → Ollama nomic-embed-text → ChromaDB
Query	Top-k similarity search → context + session history → Ollama chat model
Output	Server-Sent Events (SSE) token stream; assistant UI renders GFM Markdown

Session memory: last 10 user/assistant turns per session_id (in-process).

---

Architecture

┌─────────────┐     REST / SSE      ┌──────────────┐     HTTP      ┌─────────┐
│  Next.js    │ ◄────────────────► │   FastAPI    │ ◄───────────► │ Ollama  │
│  :3000      │                     │   :8000      │               │ chat +  │
└─────────────┘                     └──────┬───────┘               │ embed   │
                                           │                       └─────────┘
                                           ├── ChromaDB (persistent)
                                           └── uploads/

---

Stack

Component	Technology
LLM	Ollama (llama3.2 default; configurable)
Embeddings	nomic-embed-text (Ollama)
Vector store	ChromaDB (local persistence)
RAG orchestration	LangChain LCEL
Backend	Python 3.11, FastAPI
Document / OCR	PyMuPDF, EasyOCR, python-docx
Frontend	Next.js 16, TypeScript, Tailwind CSS v4
Chat rendering	react-markdown, remark-gfm

---

Prerequisites

• Python 3.11 (recommended for EasyOCR / lockfile compatibility)
• Node.js 18+
• Ollama installed and running
• ~8 GB RAM (16 GB preferred for larger models)
• ~6 GB disk (venv, models, Chroma, uploads)

---

Setup

git clone https://github.com/roydonsequeira/RagChatbot.git
cd RagChatbot

Models

ollama pull llama3.2
ollama pull nomic-embed-text

Backend

cd backend
python -m venv .venv
# Windows:  .venv\Scripts\activate
# Unix:     source .venv/bin/activate
pip install -r requirements.txt
cp ../.env.example .env
uvicorn main:app --reload --port 8000

OpenAPI: http://localhost:8000/api/docs

Frontend

cd frontend
npm install
npm run dev

App: http://localhost:3000

Windows (PATH / cwd): from repo root, .\scripts\start-backend.ps1 and .\scripts\start-frontend.ps1. Run uvicorn only from backend/ (not repo root).

---

Configuration

Copy .env.example → backend/.env. Key variables:

Variable	Role
OLLAMA_MODEL	Chat model name
OLLAMA_EMBEDDING_MODEL	Embedding model
CHROMA_PERSIST_DIR	Chroma persistence path
CHUNK_SIZE / CHUNK_OVERLAP	Text splitting
IMAGE_CHUNK_SIZE / IMAGE_CHUNK_OVERLAP	OCR-heavy documents
RETRIEVAL_K	Chunks retrieved per query
ALLOWED_ORIGINS	CORS (default http://localhost:3000)

---

HTTP API

Method	Path	Description
POST	/api/chat	Body: { session_id, message }. Response: SSE (token, done, error).
POST	/api/upload	multipart/form-data, field file.
GET	/api/knowledge	Lists indexed documents.
DELETE	/api/knowledge/{doc_id}	Removes document vectors.

Rate limits: slowapi (e.g. chat 30/min, upload 10/min — see routers).

---

Notes

• Stale index after pipeline changes: delete backend/chroma_db/ or remove entries in the UI and re-upload sources.
• First OCR run: EasyOCR downloads model weights once (~200 MB class).
• Custom README screenshot: save PNG as docs/screenshot.png and point the top <img> src to it.

---

Security (summary)

Environment secrets via .env (gitignored). CORS allowlist, upload type/size checks, UUID storage paths, bleach on user messages to the model, standard security headers on API responses.

---

License

MIT