refactor: 🔨 Ollama migration
This commit is contained in:
@@ -1,36 +1,22 @@
|
|||||||
# 🐉 Dungeon Masters Vault: Local RAG Assistant
|
# Dungeon Masters Vault: Local RAG Assistant
|
||||||
|
|
||||||
An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world’s lore using **DSPy** and **Local LLMs**.
|
An advanced Retrieval-Augmented Generation (RAG) system designed for Dungeon Masters. This tool ingests markdown-based campaign notes, enriches them with AI-generated metadata, and provides an interactive terminal interface to query your world's lore using **DSPy** and **Local LLMs**.
|
||||||
|
|
||||||
## ⚔️ Key Features
|
## Key Features
|
||||||
|
|
||||||
* **Parallel Enrichment:** Utilizes a configurable multithreading to process multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
|
* **Parallel Enrichment:** Configurable multithreading processes multiple document chunks simultaneously across local LLM slots for high-speed ingestion.
|
||||||
* **Deep Context Retrieval:** Unlike standard RAG, this system retrieves relevant chunks and then "peeks" at the full source file to provide the LLM with broader narrative context.
|
* **Deep Context Retrieval:** Retrieves relevant chunks and "peeks" at the full source file to provide the LLM with broader narrative context.
|
||||||
* **Local-First:** Designed to run entirely on your hardware using **LM Studio**, keeping your campaign secrets private.
|
* **Local-First:** Runs entirely on your hardware using **Ollama**, keeping your campaign secrets private.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🏗️ Architecture
|
## Setup
|
||||||
|
|
||||||
1. **Ingestion:** Scans `DATA_DIR` for `.md` files.
|
|
||||||
2. **Chunking:** Splits documents into 800-character segments with overlap.
|
|
||||||
3. **Enrichment:** A DSPy `IngestionAgent` analyzes each chunk to extract:
|
|
||||||
* **Synopsis:** A one-sentence summary.
|
|
||||||
* **Tags:** Plot points, item names, or themes.
|
|
||||||
* **Entities:** Specific NPCs, Locations, or Factions.
|
|
||||||
4. **Vector Store:** Chunks and metadata are embedded using `text-embedding-qwen3` and stored in a local **Turso** database.
|
|
||||||
5. **Interactive RAG:** A terminal loop that uses **ReAct (Reasoning and Acting)** to answer queries based on retrieved context.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🛠️ Setup
|
|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
* **UV [Link to install here](https://docs.astral.sh/uv/)**
|
* **[UV](https://docs.astral.sh/uv/)** — Python package manager
|
||||||
* **LM Studio:** Running a local server at `localhost:1234` (or your specific IP).
|
* **Ollama** — Running a local server (default `localhost:11434`)
|
||||||
* **Models:** * Inference & Embedding: Configurable for your preference. grab your model in LMStudio and update the conifg
|
* **Local Models** — Pull your inference and embedding models with `ollama pull`
|
||||||
|
|
||||||
|
|
||||||
### Installation
|
### Installation
|
||||||
|
|
||||||
@@ -40,52 +26,52 @@ uv sync
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Usage
|
## Usage
|
||||||
|
|
||||||
### 1. Ingest & Enrich
|
### Ingest & Enrich
|
||||||
|
|
||||||
Run the ingestion script to process your markdown files and build the vector database.
|
Process your markdown campaign files and build the vector database:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run src/ingest.py
|
uv run src/ingest.py
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Query the LLM
|
### Query the LLM
|
||||||
|
|
||||||
Launch the interactive session to ask questions about your campaign.
|
Launch the interactive session to ask questions about your campaign:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run src/retrieve.py
|
uv run src/retrieve.py
|
||||||
```
|
```
|
||||||
|
|
||||||
**Example Query:**
|
**Example interaction:**
|
||||||
|
|
||||||
> `📝 Query: Why did the party get free bread at the Golden Grain Inn?`
|
> Query: Why did the party get free bread at the Golden Grain Inn?
|
||||||
> `📜 AI RESPONSE: Based on the session notes from 'Session_12.md', the party received free bread because the Rogue successfully intimidated the baker's assistant, and the Cleric later performed a minor miracle (Thaumaturgy) that impressed the owner.`
|
>
|
||||||
|
> Based on the session notes from 'Session_12.md', the party received free bread because the Rogue intimidated the baker's assistant and the Cleric performed Thaumaturgy to impress the owner.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📂 File Structure
|
## File Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
.
|
.
|
||||||
├── config.yaml # Configuration for the app
|
├── config.yaml # App configuration
|
||||||
├── load_ingestion_llms.sh # script to load multiple LLMs (Run before ingest)
|
├── load_ingestion_llms.sh # Script to load multiple LLMs (run before ingest)
|
||||||
├── README.md
|
├── README.md
|
||||||
├── ROADMAP.md
|
├── ROADMAP.md
|
||||||
├── src
|
├── src/
|
||||||
│ ├── config_loader.py # Loads the config yaml file
|
│ ├── config_loader.py # Loads config.yaml
|
||||||
│ ├── embedding.py # Class to talk to LMStudio Embedding Model Server
|
│ ├── embedding.py # Ollama embedding model client
|
||||||
│ ├── experts
|
│ ├── experts/
|
||||||
│ │ ├── ingestion_agent.py # Agent Class for ingestion enrichment
|
│ │ ├── ingestion_agent.py # AI agent for document enrichment
|
||||||
│ │ └── retrieval_agent.py # Agent Class for retrieval, with tools and database calls
|
│ │ └── retrieval_agent.py # AI agent for queries, with tools and DB calls
|
||||||
│ ├── ingest.py # Ingestion script to load your DnD Campaign Notes
|
│ ├── ingest.py # Campaign notes ingestion script
|
||||||
│ └── retrieve.py # main Q&A for your notes
|
│ └── retrieve.py # Interactive Q&A interface
|
||||||
├── data # GitIgnored Folder for Notes Database
|
├── data/ # Campaign database (gitignored)
|
||||||
│ ├── dmv.db
|
│ ├── dmv.db
|
||||||
│ ├── dmv.db-wal
|
│ ├── dmv.log
|
||||||
│ ├── dmv.log
|
│ └── time_file.txt
|
||||||
│ └── time_file.txt
|
|
||||||
├── pyproject.toml
|
├── pyproject.toml
|
||||||
├── LICENSE
|
├── LICENSE
|
||||||
└── uv.lock
|
└── uv.lock
|
||||||
@@ -93,12 +79,12 @@ uv run src/retrieve.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚙️ Configuration
|
## Configuration
|
||||||
|
|
||||||
In `config.yaml`, you can adjust multiple things:
|
Edit `config.yaml` to customize:
|
||||||
|
|
||||||
* Enrichment / embedding & Retrieval Mdels
|
* Inference and embedding models
|
||||||
* DnD Notes Location (data_dir)
|
* Campaign notes location (`data_dir`)
|
||||||
* System Prompts for Ingestion & Retrieval Agents
|
* System prompts for ingestion and retrieval agents
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,5 +0,0 @@
|
|||||||
# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-0" --ttl 1800
|
|
||||||
# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-1" --ttl 1800
|
|
||||||
# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-2" --ttl 1800
|
|
||||||
# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-3" --ttl 1800
|
|
||||||
# lms load qwen-4b-instruct-2507 --parallel 2 --identifier "qwen-4" --ttl 1800
|
|
||||||
Reference in New Issue
Block a user