Uploaded August 2026 | Updated November 2026, 1 day ago
Build a production-grade RAG system for finance, live, end to end, and fully local with NVIDIA Nemotron 3.5 Lightning on Ollama. No OpenAI key, no cloud, everything runs on your own machine.
We take real SEC 10-K filings from Amazon, Alphabet and Meta and build a pipeline that answers questions with page-level citations, refuses what the documents do not contain, and streams every step of an agent as it works.
What we build in this session:
✅ PDF to markdown extraction and page-based chunking
✅ Local embeddings with Ollama (nomic-embed-text) and Qdrant in Docker
✅ Hybrid search (dense + BM25) with metadata filters: company, year, document type
✅ LLM-extracted metadata at ingest, per file
✅ Grounded answers with citations, and refusals when the answer is not in the corpus
✅ Model choice matters: why Nemotron 3.5 Lightning at ~180 tokens/sec beats Qwen3.6 27B for this pipeline
✅ Agentic RAG: the agent inspects the collection and decides the filters itself
✅ Streaming the agent loop: watch every tool call and result live
✅ Conversation memory with SQLite checkpointing
✅ A Chainlit chat UI over the same agent
✅ Exposing the whole pipeline as an MCP server for Claude
Stack: Python, LangChain, Ollama with NVIDIA Nemotron 3.5 Lightning (benchmarked against Qwen3.6 27B), Qdrant, RAGWire, Chainlit, MCP
RAGWire (open source): github.com/laxmimerit/ragwire
🔔 Subscribe for more hands-on AI engineering: youtube.com/@KGPTalkie
🌐 Website: kgptalkie.com
💼 LinkedIn: linkedin.com/in/laxmimerit
#RAG #Nemotron #LangChain #Ollama #Qdrant #LocalLLM #AIAgents #GenAI
Build a production-grade RAG system for finance, live, end to end, and fully local with NVIDIA Nemotron 3.5 Lightning on Ollama. No OpenAI key, no cloud, everything runs on your own machine.
We take real SEC 10-K filings from Amazon, Alphabet and Meta and build a pipeline that answers questions with page-level citations, refuses what the documents do not contain, and streams every step of an agent as it works.
What we build in this session:
✅ PDF to markdown extraction and page-based chunking
✅ Local embeddings with Ollama (nomic-embed-text) and Qdrant in Docker
✅ Hybrid search (dense + BM25) with metadata filters: company, year, document type
✅ LLM-extracted metadata at ingest, per file
✅ Grounded answers with citations, and refusals when the answer is not in the corpus
✅ Model choice matters: why Nemotron 3.5 Lightning at ~180 tokens/sec beats Qwen3.6 27B for this pipeline
✅ Agentic RAG: the agent inspects the collection and decides the filters itself
✅ Streaming the agent loop: watch every tool call and result live
✅ Conversation memory with SQLite checkpointing
✅ A Chainlit chat UI over the same agent
✅ Exposing the whole pipeline as an MCP server for Claude
Stack: Python, LangChain, Ollama with NVIDIA Nemotron 3.5 Lightning (benchmarked against Qwen3.6 27B), Qdrant, RAGWire, Chainlit, MCP
RAGWire (open source): github.com/laxmimerit/ragwire
🔔 Subscribe for more hands-on AI engineering: youtube.com/@KGPTalkie
🌐 Website: kgptalkie.com
💼 LinkedIn: linkedin.com/in/laxmimerit
#RAG #Nemotron #LangChain #Ollama #Qdrant #LocalLLM #AIAgents #GenAI










