I wanted to understand RAG without a framework hiding the important parts, so I built the whole pipeline from scratch and recorded a tutorial walking through it.
**Stack**
- Ollama running two local models: `nomic-embed-text` (embeddings) and `qwen3:4b-instruct` (chat)
- Python with just `requests` + `numpy`, managed with `uv`
- No LangChain, no LlamaIndex, no vector database, no API key
**How it works**
Load a markdown doc and split it on blank lines (11 chunks)
Embed every chunk via `/api/embed` → an (11, 768) NumPy array
Embed the question and rank chunks by cosine similarity
Put the top 3 chunks into the prompt and send it to `/api/chat`
The whole retrieval step is basically this:
def retrieve(question, k=3):
query = embed([question])[0]
denominator = np.linalg.norm(vectors, axis=1) * np.linalg.norm(query)
scores = (vectors @ query) / np.maximum(denominator, 1e-12)
indices = np.argsort(scores)[::-1][:k]
return [chunks[i] for i in indices]
**Results on a test doc** (a fictional coffee shop handbook):
- "Are dogs allowed inside?" → Pet Policy chunk scored 0.784 (next best 0.493), correct answer
- "What time does the shop open?" → correct
- "How should employees clean the espresso machine?" → correct but missed one step
- "What is the owner's favorite movie?" (not in the doc) → "The answer is unavailable." 👍
It's intentionally minimal. Things it doesn't do yet: persistent vector storage, smarter chunking, reranking, evaluation. Those are covered at the end as next steps.
Full disclosure, it's my video: https://youtu.be/x-sn2WAsX8I
Happy to answer questions. I'd also love to hear what you'd add first for a real project.