1.0 Release

README.md

@@ -1,17 +1,12 @@
-This `README.md` is designed to reflect the sophisticated local RAG pipeline you've built, highlighting the multi-threaded enrichment and the DSPy-powered "Smart Retrieval" system.
-
----
-
-# 🐉 DnD Campaign Oracle: Local RAG Assistant
+# 🐉 Dungeon Masters Vault: Local RAG Assistant
 
 An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world’s lore using **DSPy** and **Local LLMs**.
 
 ## ⚔️ Key Features
 
-* **Parallel Enrichment:** Utilizes a `ThreadPoolExecutor` to process multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
-* **Structured Metadata:** Uses **DSPy TypedPredictors** and **Pydantic** to force LLMs to output valid JSON synopses, tags, and entity lists.
+* **Parallel Enrichment:** Utilizes a configurable multithreading to process multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
 * **Deep Context Retrieval:** Unlike standard RAG, this system retrieves relevant chunks and then "peeks" at the full source file to provide the LLM with broader narrative context.
-* **Local-First:** Designed to run entirely on your hardware using **LM Studio** and **FAISS**, keeping your campaign secrets private.
+* **Local-First:** Designed to run entirely on your hardware using **LM Studio**, keeping your campaign secrets private.
 
 ---
 
@@ -20,13 +15,11 @@ An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Mas
 1. **Ingestion:** Scans `DATA_DIR` for `.md` files.
 2. **Chunking:** Splits documents into 800-character segments with overlap.
 3. **Enrichment:** A DSPy `IngestionAgent` analyzes each chunk to extract:
-* **Synopsis:** A one-sentence summary.
-* **Tags:** Plot points, item names, or themes.
-* **Entities:** Specific NPCs, Locations, or Factions.
-
-
-4. **Vector Store:** Chunks and metadata are embedded using `text-embedding-qwen3` and stored in a local **FAISS** index.
-5. **Interactive RAG:** A terminal loop that uses **Chain of Thought (CoT)** reasoning to answer queries based on retrieved context.
+    * **Synopsis:** A one-sentence summary.
+    * **Tags:** Plot points, item names, or themes.
+    * **Entities:** Specific NPCs, Locations, or Factions.
+4. **Vector Store:** Chunks and metadata are embedded using `text-embedding-qwen3` and stored in a local **Turso** database.
+5. **Interactive RAG:** A terminal loop that uses **ReAct (Reasoning and Acting)** to answer queries based on retrieved context.
 
 ---
 
@@ -34,18 +27,15 @@ An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Mas
 
 ### Prerequisites
 
-* **Python 3.10+**
-* **LM Studio:** Running a local server at `http://192.168.0.49:1234` (or your specific IP).
-* **Models:** * Inference: `qwen3-8b` (or similar).
-* Embedding: `text-embedding-qwen3-embedding-8b`.
-
+* **UV [Link to install here](https://docs.astral.sh/uv/)**
+* **LM Studio:** Running a local server at `localhost:1234` (or your specific IP).
+* **Models:** * Inference & Embedding: Configurable for your preference. grab your model in LMStudio and update the conifg
 
 
 ### Installation
 
 ```bash
 uv sync
-
 ```
 
 ---
@@ -58,40 +48,57 @@ Run the ingestion script to process your markdown files and build the vector dat
 
 ```bash
 uv run src/ingest.py
-
 ```
 
-### 2. Query the Oracle
+### 2. Query the LLM
 
 Launch the interactive session to ask questions about your campaign.
 
 ```bash
 uv run src/retrieve.py
-
 ```
 
 **Example Query:**
 
-> `📝 Query: Why did the party get free bread at the Golden Grain Inn?`
+> `📝 Query: Why did the party get free bread at the Golden Grain Inn?`  
 > `📜 AI RESPONSE: Based on the session notes from 'Session_12.md', the party received free bread because the Rogue successfully intimidated the baker's assistant, and the Cleric later performed a minor miracle (Thaumaturgy) that impressed the owner.`
 
 ---
 
 ## 📂 File Structure
 
-* `ingest.py`: Handles file loading, multi-threaded enrichment, and FAISS storage.
-* `retrieve.py`: The interactive terminal-based retrieval loop.
-* `experts/ingestion_agent.py`: Contains the `IngestionAgent` and Pydantic schemas.
-* `embedding.py`: Custom wrapper for `LocalLMEmbeddings` with batch processing support.
-* `local_faiss_db/`: Directory where the vector index and metadata are persisted.
+```
+.
+├── config.yaml # Configuration for the app
+├── load_ingestion_llms.sh  # script to load multiple LLMs (Run before ingest)
+├── README.md 
+├── ROADMAP.md
+├── src
+│   ├── config_loader.py # Loads the config yaml file
+│   ├── embedding.py  # Class to talk to LMStudio Embedding Model Server
+│   ├── experts
+│   │   ├── ingestion_agent.py # Agent Class for ingestion enrichment
+│   │   └── retrieval_agent.py # Agent Class for retrieval, with tools and database calls
+│   ├── ingest.py # Ingestion script to load your DnD Campaign Notes
+│   └── retrieve.py # main Q&A for your notes
+├── data # GitIgnored Folder for Notes Database
+│   ├── dmv.db
+│   ├── dmv.db-wal
+│   ├── dmv.log
+│   └── time_file.txt
+├── pyproject.toml
+├── LICENSE
+└── uv.lock
+```
 
 ---
 
 ## ⚙️ Configuration
 
-In `ingest_notes.py`, you can tune the processing speed:
+In `config.yaml`, you can adjust multiple things:
 
-* `max_workers=8`: Adjust based on your GPU/CPU capability to handle concurrent LLM requests.
-* `chunk_size=800`: Increase for more context per chunk, decrease for more granular searching.
+* Enrichment / embedding & Retrieval Mdels
+* DnD Notes Location (data_dir)
+* System Prompts for Ingestion & Retrieval Agents
 
 ---