ready for open source

This commit is contained in:
Jake Pullen
2024-08-10 09:50:37 +01:00
parent c9b01e85d5
commit b29b738a7a
6 changed files with 81 additions and 48 deletions
+3 -7
View File
@@ -15,11 +15,6 @@ erDiagram
boolean deleted
}
ACCOUNT_TYPES {
int account_type_id
string account_type_name
}
CATEGORIES {
int category_id
string category_name
@@ -69,7 +64,6 @@ erDiagram
boolean deleted
}
ACCOUNTS ||--o{ ACCOUNT_TYPES : "has type"
TRANSACTIONS ||--o{ ACCOUNTS : "belongs to"
TRANSACTIONS ||--o{ CATEGORIES : "belongs to"
TRANSACTIONS ||--o{ PAYEES : "belongs to"
@@ -77,4 +71,6 @@ erDiagram
SCHEDULED_TRANSACTIONS ||--o{ ACCOUNTS : "belongs to"
SCHEDULED_TRANSACTIONS ||--o{ CATEGORIES : "belongs to"
SCHEDULED_TRANSACTIONS ||--o{ PAYEES : "belongs to"
SCHEDULED_TRANSACTIONS ||--o{ DATES : "scheduled on"
SCHEDULED_TRANSACTIONS ||--o{ DATES : "scheduled on"
```
+41
View File
@@ -0,0 +1,41 @@
# How to get started
This document will guide you through the process of setting up the project on your local machine. It will cover the following topics:
- [Prerequisites](#prerequisites)
- [Setting up the project](#setting-up-the-project)
- [Running the project](#running-the-project)
## Prerequisites
### .env file
Create a `.env` file in the root of the project with the following content:
```bash
API_TOKEN=your_api_token_here
BUDGET_ID=your_budget_id_here
```
You can follow [This Link](https://api.ynab.com/#access-token-usage:~:text=ynab.com.-,Quick%20Start,-If%20you%27re%20the) for a guide on how to get your API token
For the `BUDGET_ID`, you can get it from the URL of your budget page on the YNAB website. It is in between the `app.ynab.com/` and the `/budget/` in the URL. For example, if your URL is `https://app.ynab.com.com/your_budget_id/budget`, then your `BUDGET_ID` is `your_budget_id`.
## setting up the project
### Clone the repository
```bash
git clone #link tbc
```
### Install dependencies
```bash
pip install -r requirements.txt
```
## Running the project
```bash
python3 main.py
```
+28
View File
@@ -1,2 +1,30 @@
# Flow of data from source to gold
```mermaid
graph TD
A[Source Data] --> B[Raw Data/Bronze]
B --> C[Base Data/Silver]
C --> D[Data Warehouse/Gold]
B --> G[Processed Archive]
```
## Source
The Source Data is hosted in a web application called [You Need A Budget](https://www.youneedabudget.com/). We pull the data from the [YNAB API](https://api.ynab.com/), using the access token method of authentication.
The data is in JSON format.
## Raw Data/Bronze
The Raw Data is the data as it is pulled from the YNAB API. It is stored as JSON files in the `data/raw/` directory with a folder for each entity.
## Base Data/Silver
The Base Data is the data after it has been cleaned and transformed. It is stored as parquet files in the `data/base/` directory with a file for each entity.
## Data Warehouse/Gold
The Data Warehouse is the data after it has been aggregated and transformed. It is stored as parquet files in the `data/warehouse/` directory with a file for each entity.
## Processed Archive
The Processed Archive is the data after it has been processed and stored in the base tables. It is the raw json files in the `data/processed/` directory with a folder for each entity and file for each load that has been processed.