LLM Chat

Personal LLM chat interface with multi-model support, streaming responses, and conversation history.

Stack: Next.js 14 + FastAPI + Ollama

Setup

Prerequisites

• Python 3.10+
• Node.js 18+
• Ollama (local or remote)

Backend

cd backend; python -m venv venv; .\venv\Scripts\activate; pip install -r requirements.txt

Frontend

cd frontend; npm install

Environment

Create .env in the project root:

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_DEFAULT_MODEL=qwen2.5:7b

For a remote GPU server, replace with your server's IP:

OLLAMA_BASE_URL=http://your-gpu-ip:11434

Run

Terminal 1 — Backend:

cd backend; .\venv\Scripts\activate; uvicorn main:app --reload --port 8000

Terminal 2 — Frontend:

cd frontend; npm run dev

Open http://localhost:3000

Ollama Setup

ollama serve
ollama pull qwen2.5:7b
ollama pull qwen3:30b        # optional, needs ~20GB VRAM
ollama pull deepseek-r1:70b  # optional, needs ~40GB VRAM

For remote servers, start Ollama on all interfaces:

OLLAMA_HOST=0.0.0.0 ollama serve

Docker

docker compose up --build

Set OLLAMA_BASE_URL in .env before building. Defaults to http://host.docker.internal:11434 (host machine's Ollama).

API

Endpoint	Method	Description
/chat	POST	SSE streaming chat
/conversations	GET	List conversations
/conversations	POST	Create conversation
/conversations/{id}	GET	Get conversation + messages
/conversations/{id}	PATCH	Update title/model
/conversations/{id}	DELETE	Delete conversation
/models	GET	List models + health status
/control/health	GET	Ollama + model health
/control/load/{model}	POST	Load model into VRAM
/control/unload/{model}	POST	Unload model from VRAM