From b494e1fa3ffd998b1ab3c3e96e50aa81db9edc62 Mon Sep 17 00:00:00 2001 From: Jake Pullen Date: Sat, 16 May 2026 11:27:28 +0100 Subject: [PATCH] =?UTF-8?q?refactor:=20=F0=9F=94=A8=20Ollama=20migration?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 94 ++++++++++++++++++------------------------ load_ingestion_llms.sh | 5 --- 2 files changed, 40 insertions(+), 59 deletions(-) delete mode 100755 load_ingestion_llms.sh diff --git a/README.md b/README.md index 13f64b8..5ce0bdb 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,22 @@ -# πŸ‰ Dungeon Masters Vault: Local RAG Assistant +# Dungeon Masters Vault: Local RAG Assistant -An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world’s lore using **DSPy** and **Local LLMs**. +An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world's lore using **DSPy** and **Local LLMs**. -## βš”οΈ Key Features +## Key Features -* **Parallel Enrichment:** Utilizes a configurable multithreading to process multiple document chunks simultaneously across local LLM slots for high-speed ingestion. -* **Deep Context Retrieval:** Unlike standard RAG, this system retrieves relevant chunks and then "peeks" at the full source file to provide the LLM with broader narrative context. -* **Local-First:** Designed to run entirely on your hardware using **LM Studio**, keeping your campaign secrets private. +* **Parallel Enrichment:** Configurable multithreading processes multiple document chunks simultaneously across local LLM slots for high-speed ingestion. +* **Deep Context Retrieval:** Retrieves relevant chunks and "peeks" at the full source file to provide the LLM with broader narrative context. +* **Local-First:** Runs entirely on your hardware using **Ollama**, keeping your campaign secrets private. --- -## πŸ—οΈ Architecture - -1. **Ingestion:** Scans `DATA_DIR` for `.md` files. -2. **Chunking:** Splits documents into 800-character segments with overlap. -3. **Enrichment:** A DSPy `IngestionAgent` analyzes each chunk to extract: - * **Synopsis:** A one-sentence summary. - * **Tags:** Plot points, item names, or themes. - * **Entities:** Specific NPCs, Locations, or Factions. -4. **Vector Store:** Chunks and metadata are embedded using `text-embedding-qwen3` and stored in a local **Turso** database. -5. **Interactive RAG:** A terminal loop that uses **ReAct (Reasoning and Acting)** to answer queries based on retrieved context. - ---- - -## πŸ› οΈ Setup +## Setup ### Prerequisites -* **UV [Link to install here](https://docs.astral.sh/uv/)** -* **LM Studio:** Running a local server at `localhost:1234` (or your specific IP). -* **Models:** * Inference & Embedding: Configurable for your preference. grab your model in LMStudio and update the conifg - +* **[UV](https://docs.astral.sh/uv/)** β€” Python package manager +* **Ollama** β€” Running a local server (default `localhost:11434`) +* **Local Models** β€” Pull your inference and embedding models with `ollama pull` ### Installation @@ -40,52 +26,52 @@ uv sync --- -## πŸš€ Usage +## Usage -### 1. Ingest & Enrich +### Ingest & Enrich -Run the ingestion script to process your markdown files and build the vector database. +Process your markdown campaign files and build the vector database: ```bash uv run src/ingest.py ``` -### 2. Query the LLM +### Query the LLM -Launch the interactive session to ask questions about your campaign. +Launch the interactive session to ask questions about your campaign: ```bash uv run src/retrieve.py ``` -**Example Query:** +**Example interaction:** -> `πŸ“ Query: Why did the party get free bread at the Golden Grain Inn?` -> `πŸ“œ AI RESPONSE: Based on the session notes from 'Session_12.md', the party received free bread because the Rogue successfully intimidated the baker's assistant, and the Cleric later performed a minor miracle (Thaumaturgy) that impressed the owner.` +> Query: Why did the party get free bread at the Golden Grain Inn? +> +> Based on the session notes from 'Session_12.md', the party received free bread because the Rogue intimidated the baker's assistant and the Cleric performed Thaumaturgy to impress the owner. --- -## πŸ“‚ File Structure +## File Structure ``` . -β”œβ”€β”€ config.yaml # Configuration for the app -β”œβ”€β”€ load_ingestion_llms.sh # script to load multiple LLMs (Run before ingest) -β”œβ”€β”€ README.md +β”œβ”€β”€ config.yaml # App configuration +β”œβ”€β”€ load_ingestion_llms.sh # Script to load multiple LLMs (run before ingest) +β”œβ”€β”€ README.md β”œβ”€β”€ ROADMAP.md -β”œβ”€β”€ src -β”‚Β Β  β”œβ”€β”€ config_loader.py # Loads the config yaml file -β”‚Β Β  β”œβ”€β”€ embedding.py # Class to talk to LMStudio Embedding Model Server -β”‚Β Β  β”œβ”€β”€ experts -β”‚Β Β  β”‚Β Β  β”œβ”€β”€ ingestion_agent.py # Agent Class for ingestion enrichment -β”‚Β Β  β”‚Β Β  └── retrieval_agent.py # Agent Class for retrieval, with tools and database calls -β”‚Β Β  β”œβ”€β”€ ingest.py # Ingestion script to load your DnD Campaign Notes -β”‚Β Β  └── retrieve.py # main Q&A for your notes -β”œβ”€β”€ data # GitIgnored Folder for Notes Database -β”‚Β Β  β”œβ”€β”€ dmv.db -β”‚Β Β  β”œβ”€β”€ dmv.db-wal -β”‚Β Β  β”œβ”€β”€ dmv.log -β”‚Β Β  └── time_file.txt +β”œβ”€β”€ src/ +β”‚ β”œβ”€β”€ config_loader.py # Loads config.yaml +β”‚ β”œβ”€β”€ embedding.py # Ollama embedding model client +β”‚ β”œβ”€β”€ experts/ +β”‚ β”‚ β”œβ”€β”€ ingestion_agent.py # AI agent for document enrichment +β”‚ β”‚ └── retrieval_agent.py # AI agent for queries, with tools and DB calls +β”‚ β”œβ”€β”€ ingest.py # Campaign notes ingestion script +β”‚ └── retrieve.py # Interactive Q&A interface +β”œβ”€β”€ data/ # Campaign database (gitignored) +β”‚ β”œβ”€β”€ dmv.db +β”‚ β”œβ”€β”€ dmv.log +β”‚ └── time_file.txt β”œβ”€β”€ pyproject.toml β”œβ”€β”€ LICENSE └── uv.lock @@ -93,12 +79,12 @@ uv run src/retrieve.py --- -## βš™οΈ Configuration +## Configuration -In `config.yaml`, you can adjust multiple things: +Edit `config.yaml` to customize: -* Enrichment / embedding & Retrieval Mdels -* DnD Notes Location (data_dir) -* System Prompts for Ingestion & Retrieval Agents +* Inference and embedding models +* Campaign notes location (`data_dir`) +* System prompts for ingestion and retrieval agents --- diff --git a/load_ingestion_llms.sh b/load_ingestion_llms.sh deleted file mode 100755 index b62b842..0000000 --- a/load_ingestion_llms.sh +++ /dev/null @@ -1,5 +0,0 @@ -# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-0" --ttl 1800 -# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-1" --ttl 1800 -# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-2" --ttl 1800 -# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-3" --ttl 1800 -# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-4" --ttl 1800