Merge pull request 'refactor: 🔨 working Ollama migration' (#1) from migrate_to_ollama into main

Reviewed-on: #1

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "toon-python"]
+	path = toon-python
+	url = git@github.com:toon-format/toon-python.git

README.md

@@ -1,36 +1,22 @@
-# 🐉 Dungeon Masters Vault: Local RAG Assistant
+# Dungeon Masters Vault: Local RAG Assistant
 
-An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world’s lore using **DSPy** and **Local LLMs**.
+An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world's lore using **DSPy** and **Local LLMs**.
 
-## ⚔️ Key Features
+## Key Features
 
-* **Parallel Enrichment:** Utilizes a configurable multithreading to process multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
+* **Parallel Enrichment:** Configurable multithreading processes multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
-* **Deep Context Retrieval:** Unlike standard RAG, this system retrieves relevant chunks and then "peeks" at the full source file to provide the LLM with broader narrative context.
+* **Deep Context Retrieval:** Retrieves relevant chunks and "peeks" at the full source file to provide the LLM with broader narrative context.
-* **Local-First:** Designed to run entirely on your hardware using **LM Studio**, keeping your campaign secrets private.
+* **Local-First:** Runs entirely on your hardware using **Ollama**, keeping your campaign secrets private.
 
 ---
 
-## 🏗️ Architecture
+## Setup
-
-1. **Ingestion:** Scans `DATA_DIR` for `.md` files.
-2. **Chunking:** Splits documents into 800-character segments with overlap.
-3. **Enrichment:** A DSPy `IngestionAgent` analyzes each chunk to extract:
-    * **Synopsis:** A one-sentence summary.
-    * **Tags:** Plot points, item names, or themes.
-    * **Entities:** Specific NPCs, Locations, or Factions.
-4. **Vector Store:** Chunks and metadata are embedded using `text-embedding-qwen3` and stored in a local **Turso** database.
-5. **Interactive RAG:** A terminal loop that uses **ReAct (Reasoning and Acting)** to answer queries based on retrieved context.
-
----
-
-## 🛠️ Setup
 
 ### Prerequisites
 
-* **UV [Link to install here](https://docs.astral.sh/uv/)**
+* **[UV](https://docs.astral.sh/uv/)** — Python package manager
-* **LM Studio:** Running a local server at `localhost:1234` (or your specific IP).
+* **Ollama** — Running a local server (default `localhost:11434`)
-* **Models:** * Inference & Embedding: Configurable for your preference. grab your model in LMStudio and update the conifg
+* **Local Models** — Pull your inference and embedding models with `ollama pull`
-
 
 ### Installation
 
@@ -40,52 +26,52 @@ uv sync
 
 ---
 
-## 🚀 Usage
+## Usage
 
-### 1. Ingest & Enrich
+### Ingest & Enrich
 
-Run the ingestion script to process your markdown files and build the vector database.
+Process your markdown campaign files and build the vector database:
 
 ```bash
 uv run src/ingest.py
 ```
 
-### 2. Query the LLM
+### Query the LLM
 
-Launch the interactive session to ask questions about your campaign.
+Launch the interactive session to ask questions about your campaign:
 
 ```bash
 uv run src/retrieve.py
 ```
 
-**Example Query:**
+**Example interaction:**
 
-> `📝 Query: Why did the party get free bread at the Golden Grain Inn?`  
+> Query: Why did the party get free bread at the Golden Grain Inn?
-> `📜 AI RESPONSE: Based on the session notes from 'Session_12.md', the party received free bread because the Rogue successfully intimidated the baker's assistant, and the Cleric later performed a minor miracle (Thaumaturgy) that impressed the owner.`
+>
+> Based on the session notes from 'Session_12.md', the party received free bread because the Rogue intimidated the baker's assistant and the Cleric performed Thaumaturgy to impress the owner.
 
 ---
 
-## 📂 File Structure
+## File Structure
 
 ```
 .
-├── config.yaml # Configuration for the app
+├── config.yaml                   # App configuration
-├── load_ingestion_llms.sh  # script to load multiple LLMs (Run before ingest)
+├── load_ingestion_llms.sh        # Script to load multiple LLMs (run before ingest)
-├── README.md 
+├── README.md
 ├── ROADMAP.md
-├── src
+├── src/
-│   ├── config_loader.py # Loads the config yaml file
+│   ├── config_loader.py          # Loads config.yaml
-│   ├── embedding.py  # Class to talk to LMStudio Embedding Model Server
+│   ├── embedding.py              # Ollama embedding model client
-│   ├── experts
+│   ├── experts/
-│   │   ├── ingestion_agent.py # Agent Class for ingestion enrichment
+│   │   ├── ingestion_agent.py    # AI agent for document enrichment
-│   │   └── retrieval_agent.py # Agent Class for retrieval, with tools and database calls
+│   │   └── retrieval_agent.py    # AI agent for queries, with tools and DB calls
-│   ├── ingest.py # Ingestion script to load your DnD Campaign Notes
+│   ├── ingest.py                 # Campaign notes ingestion script
-│   └── retrieve.py # main Q&A for your notes
+│   └── retrieve.py               # Interactive Q&A interface
-├── data # GitIgnored Folder for Notes Database
+├── data/                         # Campaign database (gitignored)
-│   ├── dmv.db
+│   ├── dmv.db
-│   ├── dmv.db-wal
+│   ├── dmv.log
-│   ├── dmv.log
+│   └── time_file.txt
-│   └── time_file.txt
 ├── pyproject.toml
 ├── LICENSE
 └── uv.lock
@@ -93,12 +79,12 @@ uv run src/retrieve.py
 
 ---
 
-## ⚙️ Configuration
+## Configuration
 
-In `config.yaml`, you can adjust multiple things:
+Edit `config.yaml` to customize:
 
-* Enrichment / embedding & Retrieval Mdels
+* Inference and embedding models
-* DnD Notes Location (data_dir)
+* Campaign notes location (`data_dir`)
-* System Prompts for Ingestion & Retrieval Agents
+* System prompts for ingestion and retrieval agents
 
 ---

ROADMAP.md

@@ -16,4 +16,6 @@
 ## Planned Later
 
 * entity chunking & re-ranking
-* Logging in Ingestion
+* Logging in Ingestion
+* database retrieve for tag or entity
+*

config.yaml

@@ -1,33 +1,35 @@
 # --- Connection Settings ---
 api:
-  base_url: "http://framework.tawny-bellatrix.ts.net:1234"
+  base_url: "http://100.110.238.94:11434"
   api_version: "/v1/"
 
 # --- Model Settings ---
 models:
-  enrich: "lm_studio/qwen-" # will have an identifier, based on amount of active LLMs see ./load_ingestion_llms.sh
+  enrich: "ollama/granite4.1:3b"
-  embedding: "text-embedding-qwen3-embedding-8b"
+  embedding: "qwen3-embedding:4b"
-  retrieval: "lm_studio/qwen/qwen3-30b-a3b-2507"
+  retrieval: "ollama/qwen3.6:latest"
+  expansion: "ollama/granite4.1:3b"
 
 # --- Ingestion Settings ---
 ingestion:
-  data_dir: "/home/cosmic/DnD"
+  data_dir: "/home/jake/DnD"
-  db_path: "./data/dmv.db"
+  db_path: "./data/"
-  active_llms: 2
+  db_name: "dmv.db"
-  parallel_requests_per_llm: 2
+  active_llms: 1
-  chunk_size: 800
+  parallel_requests_per_llm: 6
-  chunk_overlap: 100
+  chunk_size: 1200
+  chunk_overlap: 200
   embedding_batch_size: 32
   time_file_location: "./data/time_file.txt"
 
 # ---- Agent Settings ----
-ingestion_agent: 
+ingestion_agent:
   ingestion_signature: |
     You are an expert Dungeon Master's assistant.
     Analyze the provided notes and extract a concise synopsis and relevant metadata.
     synopsis = A one-sentence summary of the document.
     tags = Relevant tags (NPCs, Locations, Items, Plot Points).
-    entities = a list of Key names of people, places, or factions.
+    entities = a list of names for people, places, or factions.
     "note -> synopsis:str, tags: list[str], entities: list[str]"
 
 retrieval_agent:
@@ -36,3 +38,10 @@ retrieval_agent:
     Given the context and the question, answer the question.
     Do not make things up, base all of your answers on the context.
     Always site the file location of your source of information.
+
+expansion_agent:
+  expansion_signature: |
+    You are an expert Dungeon Master's assistant.
+    Given a user's question, generate 3-5 similar but enhanced search queries that would help find more relevant information in DnD notes.
+    Each expanded query should be distinct and add different perspective to the original question.
+    Return only the queries as a JSON list with key "queries"."""

load_ingestion_llms.sh

@@ -1,5 +0,0 @@
-lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-0" --ttl 1800
-lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-1" --ttl 1800
-# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-2" --ttl 1800
-# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-3" --ttl 1800
-# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-4" --ttl 1800

src/embedding.py

@@ -1,5 +1,6 @@
 import requests
 from langchain_core.embeddings import Embeddings
+
 from config_loader import load_config
 
 CFG = load_config()
@@ -9,7 +10,7 @@ API_VERSION = CFG["api"]["api_version"]
 
 class LocalLMEmbeddings(Embeddings):
     def __init__(self, model: str, base_url: str = API_BASE, batch_size: int = 32):
-        self.url = f"{base_url}/{API_VERSION}embeddings"
+        self.url = f"{base_url}/api/embed"
         self.model = model
         self.batch_size = batch_size
 
@@ -21,10 +22,11 @@ class LocalLMEmbeddings(Embeddings):
             response = requests.post(
                 self.url, json=payload, timeout=120
             )  # Longer timeout for batches
+            # print(response)
             response.raise_for_status()
             data = response.json()
             # print(data)
-            return [item["embedding"] for item in data["data"]]
+            return data["embeddings"]
         except Exception as e:
             print(f"❌ Batch request failed: {e}")
             # Returning empty lists to maintain index integrity if needed,
@@ -37,7 +39,7 @@ class LocalLMEmbeddings(Embeddings):
 
         for i in range(0, len(texts), self.batch_size):
             batch = texts[i : i + self.batch_size]
-            print(f"🚀 Processing batch {(i // self.batch_size) + 1} (Size: {len(batch)})...")
+            # print(f"🚀 Processing batch {(i // self.batch_size) + 1} (Size: {len(batch)})...")
 
             batch_vectors = self._post_request(batch)
             all_embeddings.extend(batch_vectors)

src/experts/retrieval_agent.py

@@ -1,7 +1,7 @@
 import os
-import turso
-import dspy
 
+import dspy
+import turso
 
 from config_loader import load_config
 from embedding import LocalLMEmbeddings
@@ -9,27 +9,28 @@ from embedding import LocalLMEmbeddings
 CFG = load_config()
 
 DATABASE_PATH = CFG["ingestion"]["db_path"]
+DATABASE_NAME = CFG["ingestion"]["db_name"]
 EMBEDDING_MODEL = CFG["models"]["embedding"]
 API_BASE = CFG["api"]["base_url"]
 RETRIEVAL_CONFIG = CFG["retrieval_agent"]
+EXPANSION_CONFIG = CFG["expansion_agent"]
 
 
 def retrieve_from_turso(embedded_question, k=5):
     query = f"""
-    SELECT file_path, synopsis, tags, entities, chunk_data,
+    SELECT file_path, synopsis, tags, chunk_data,
-    vector_distance_cos(embedding, vector32('{embedded_question[0]}')) AS distance
+    vector_distance_cos(embedding, vector32('{embedded_question}')) AS distance
     FROM notes
     ORDER BY distance ASC
     LIMIT {k};
     """
-    con = turso.connect(DATABASE_PATH)
+    con = turso.connect(DATABASE_PATH + DATABASE_NAME)
     cur = con.cursor()
     cur.execute(query)
     rows = cur.fetchall()
     return rows
 
 
-# --- DSPy Signature ---
 class DnDContextQA(dspy.Signature):
     f"{RETRIEVAL_CONFIG['retrieval_signature']}"
 
@@ -38,46 +39,68 @@ class DnDContextQA(dspy.Signature):
     answer = dspy.OutputField(desc="A detailed answer based on the notes, citing the source file.")
 
 
+class ExpansionSignature(dspy.Signature):
+    f"{EXPANSION_CONFIG['expansion_signature']}"
+    question = dspy.InputField()
+    answer = dspy.OutputField(
+        desc="A list of questions that will be used to vector search the database."
+    )
+
+
 class DnDRAG(dspy.Module):
     def __init__(self):
         super().__init__()
         self.embeddings_model = LocalLMEmbeddings(
             model=EMBEDDING_MODEL,
             base_url=API_BASE,
-            batch_size=1,  # we only send 1 question at a time.
+            # batch_size=1,
         )
-        # Tools exposed to the ReAct loop
+        self.retrieval_lm = dspy.LM(model=CFG["models"]["retrieval"], api_base=API_BASE)
+        with dspy.context(lm=self.retrieval_lm, signature=ExpansionSignature):
+            self.query_expander = dspy.Predict("question -> queries:list[str]")
+
         self.tools = [self.load_file]
         self.generate_answer = dspy.ReAct(signature=DnDContextQA, tools=self.tools)
 
     def forward(self, question):
-        # TODO: Add step here to LLM Expand
+        print("Enhancing Question")
-        # given the current question, generate 3-5 distinct search queries.
+        with dspy.context(lm=self.retrieval_lm):
-        # embed all the questions
+            expanded_queries = self.query_expander(question=question).queries
-        embedded_question = self.embeddings_model._post_request(question)
+        # print("Enhanced Queries:")
-        # store the 5 from all 3-5 questions (15 - 25 results)
+        # for q in expanded_queries:
-        results = retrieve_from_turso(embedded_question, k=5)  # k is limit to return
+        #     print("    ", q)
+        all_embeddings = self.embeddings_model.embed_documents([question] + expanded_queries)
+        # print(all_embeddings)
+        all_results = []
+        for embedded_question in all_embeddings:
+            results = retrieve_from_turso(embedded_question, k=5)
+            all_results.extend(results)
+
+        seen = set()
+        unique_results = []
+        for row in all_results:
+            key = (row[0], row[3])
+            if key not in seen:
+                seen.add(key)
+                unique_results.append(row)
 
-        # Format context as before
         context_parts = []
-        for i, row in enumerate(results):
+        for i, row in enumerate(unique_results):
-            source = row[0]  # file_path
+            source = row[0]
-            synopsis = row[1]  # synopsis
+            synopsis = row[1]
-            tags = row[2]  # tags
+            tags = row[2]
-            entities = row[3]  # entities
+            # entities = row[3]
-            content = row[4]  # chunk_data
+            content = row[3]
+            closeness = row[4]
 
             context_parts.append(f"""
 --- Chunk {i + 1} from {source} ---
 synopsis: {synopsis},
 tags: {tags},
-entities: {entities}
+closeness: {closeness},
 {content}
 """)
-
+        # entities: {entities},
-        # print('Closest embedding hits')
-        # for part in context_parts:
-        #     print(part)
 
         context = "\n\n".join(context_parts)
 

src/ingest.py

@@ -16,6 +16,7 @@ from experts.ingestion_agent import IngestionAgent
 CFG = load_config()
 DATA_DIR = CFG["ingestion"]["data_dir"]
 DATABASE_PATH = CFG["ingestion"]["db_path"]
+DATABASE_NAME = CFG["ingestion"]["db_name"]
 MODEL_BASE = CFG["models"]["enrich"]
 EMBEDDING_MODEL = CFG["models"]["embedding"]
 API_BASE = CFG["api"]["base_url"]
@@ -82,7 +83,7 @@ def enrich_chunks(chunks: list) -> list:
 
         try:
             with dspy.context(
-                lm=dspy.LM(model=f"{MODEL_BASE}{lm_index}", api_base=API_BASE + API_VERSION),
+                lm=dspy.LM(model=f"{MODEL_BASE}", api_base=API_BASE),
                 chat_template_kwargs={"enable_thinking": False},
             ):
                 response = IngestionAgent().ingest(note=chunk.page_content)
@@ -139,13 +140,11 @@ def embed_chunks(chunks: List[Any], batch_size: int = EMBEDDING_BATCH_SIZE) -> L
     # Process chunks in batches
     for i in tqdm(range(0, total_chunks, batch_size), desc="Embedding batches"):
         batch = chunks[i : i + batch_size]
+        # print(f"🚀 Processing batch {(i // batch_size) + 1} (Size: {len(batch)})...")
         batch_content = [chunk.page_content for chunk in batch]
-
         try:
-            # Use model's batched embedding method
-            # batch_embeddings = embeddings_model.embed_query(batch_content)
             batch_embeddings = embeddings_model.embed_documents(batch_content)
-
+            # print(len(batch_embeddings[0]))
             # Process each chunk in the batch
             for j, (chunk, embedding) in enumerate(zip(batch, batch_embeddings)):
                 # Extract metadata
@@ -228,15 +227,22 @@ def save_to_db(chunk_dicts):
     Each dict maps to a row in the 'notes' table.
     """
     print("connecting to db")
-    con = turso.connect(DATABASE_PATH)
+    con = turso.connect(DATABASE_PATH + DATABASE_NAME)
     print("opening cursor")
     cur = con.cursor()
 
     # SQL with named placeholders for clarity and safety
     insert_sql = """
     INSERT INTO notes (
-        file_path, file_name, chunk_data, synopsis, tags, entities, embedding, timestamp
+        file_path,
-    ) VALUES (?, ?, ?, ?, ?, ?, vector32(?), ?)
+        file_name,
+        chunk_data,
+        synopsis,
+        tags,
+       -- entities,
+        embedding,
+        timestamp
+    ) VALUES (?, ?, ?, ?, ?, vector32(?), ?)
     """
 
     # Prepare batch data: convert each dict to a tuple in correct order
@@ -252,7 +258,7 @@ def save_to_db(chunk_dicts):
                 entry["chunk_data"],
                 entry["synopsis"],
                 ",".join(entry["tags"]),  # Store as comma-separated string
-                ",".join(entry["entities"]),  # Store as comma-separated string
+                # ",".join(entry["entities"]),  # Store as comma-separated string
                 embedding_str,
                 entry["timestamp"],
             )
@@ -267,7 +273,8 @@ def save_to_db(chunk_dicts):
 
 
 def create_db():
-    con = turso.connect(DATABASE_PATH)
+    Path(DATABASE_PATH).mkdir(exist_ok=True)
+    con = turso.connect(DATABASE_PATH + DATABASE_NAME)
     cur = con.cursor()
 
     cur.execute("""
@@ -278,8 +285,8 @@ def create_db():
         chunk_data TEXT NOT NULL,
         synopsis TEXT,
         tags TEXT,  -- comma-separated
-        entities TEXT,  -- comma-separated
+        -- entities TEXT,  -- comma-separated
-        embedding F32_BLOB(4096),
+        embedding F32_BLOB(2560),
         timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
     )
     """)
@@ -334,7 +341,7 @@ def delete_from_db(embedded_chunks):
 
     print(f"Deleting existing rows for {len(file_paths)} file(s)")
 
-    con = turso.connect(DATABASE_PATH)
+    con = turso.connect(DATABASE_PATH + DATABASE_NAME)
     cur = con.cursor()
 
     # Use a single DELETE statement with IN clause for efficiency

src/retrieve.py

@@ -87,8 +87,7 @@ def main():
     dspy.configure(verbose_errors=True)
     dspy.configure(callbacks=[CallbackHandler(logger)])
     # 1. Setup the LLM
-    print("🚀 Initializing Qwen-8B via LM Studio...")
+    lm = dspy.LM(RETRIEVE_MODEL, api_base=API_BASE)
-    lm = dspy.LM(RETRIEVE_MODEL, api_base=API_BASE + API_VERSION)
     dspy.configure(lm=lm)
 
     # 2. Load the RAG System (only happens once!)