Commit Graph

43 Commits

Author SHA1 Message Date
Jake Pullen 91d67896d1 Refactor join conditions in dash_app
fix is weekday issue, making fridays a weekend
update ERD
2024-08-29 11:02:15 +01:00
Jake Pullen bd0ebd38e9 tidying up facts and dims
removing duplicated columns
2024-08-29 10:39:55 +01:00
Jake Pullen 9e7ff808a5 tidied up dimensions and removed duplicated columns 2024-08-29 09:23:30 +01:00
Jake-Pullen d999a8175c Merge pull request #8 from Jake-Pullen/feature/add_visuals
Feature/add visuals
2024-08-28 15:26:01 +01:00
Jake Pullen 845f6a28cc separation of areas 2024-08-28 12:54:01 +01:00
Jake Pullen 7b80b52998 changes to make dash app work 2024-08-27 15:12:44 +01:00
Jake Pullen 173c0594a8 Merge branch 'main' into feature/add_visuals 2024-08-27 08:47:01 +01:00
Jake-Pullen d69b1ac493 Merge pull request #4 from Jake-Pullen/bugfix/merge_not_merging_maybe
Bugfix/merge not merging maybe
2024-08-12 22:18:46 +01:00
Jake Pullen a966ad5ebd better handling of merging the data in the combine, lead to not needing to handle duplicates 2024-08-12 16:51:06 +01:00
Jake Pullen e54d48de5d added a log for start and end of base processing 2024-08-11 10:44:30 +01:00
Jake Pullen 201f8eb2c9 more dash workings 2024-08-11 10:42:05 +01:00
Jake Pullen 3504641643 understanding dash 2024-08-10 21:47:08 +01:00
Jake-Pullen f06ce9999f Merge pull request #2 from Jake-Pullen/Develop
1.1 Release, Better Error Handling and Better Logging
2024-08-10 20:17:14 +01:00
Jake-Pullen b8681f010f Merge pull request #1 from Jake-Pullen/feature/better_error_handling
Feature/better error handling
2024-08-10 19:48:22 +01:00
Jake Pullen 19b633eb40 added error handling to facts and dimensions transformations 2024-08-10 19:45:17 +01:00
Jake Pullen 861ccd0dc4 bug fix, a sucessful pipeline completion resulted in error message...
bug fix, a successful dimension save resulted in failed save and skip the rest of processing
2024-08-10 18:47:06 +01:00
Jake Pullen 812e107013 Added more error handling in for raw to base 2024-08-10 18:36:39 +01:00
Jake Pullen 87c6bf8718 added log for start and end of pipeline 2024-08-10 17:34:50 +01:00
Jake Pullen c5cdb16a8a JP is stupid 2024-08-10 17:13:51 +01:00
Jake Pullen 24cc006ef5 Added more error handling and logic to ingest 2024-08-10 17:11:57 +01:00
Jake Pullen d25136fbb0 Added some more error handling and request limit handling 2024-08-10 15:54:31 +01:00
Jake Pullen 4fea87030e added init file to config 2024-08-10 14:21:15 +01:00
Jake Pullen 296787d7b0 added better logging and moved config to its own folder 2024-08-10 14:10:16 +01:00
Jake Pullen ad633edefd added check for present api toen and budget id 2024-08-10 13:26:02 +01:00
Jake Pullen fa6332e18a removed pycache 2024-08-10 10:09:26 +01:00
Jake Pullen c7a5b5f092 name update 2024-08-10 09:57:08 +01:00
Jake Pullen b29b738a7a ready for open source 2024-08-10 09:50:37 +01:00
Jake Pullen c9b01e85d5 fixed scheduled transactions 2024-08-10 08:29:50 +01:00
Jake Pullen 696b72988d added facts 2024-08-10 08:07:14 +01:00
Jake Pullen 609ea38c43 slashes might be an issue? 2024-08-09 18:16:38 +01:00
Jake Pullen b7b6f8887c dimensions done, still need to fully test 2024-08-09 16:27:36 +01:00
Jake Pullen da5cf943d6 accounts into dimension 2024-08-09 15:14:04 +01:00
Jake Pullen 165464820b starting to build the warehouse 2024-08-09 14:23:52 +01:00
Jake Pullen 33c9496eb0 . 2024-08-09 13:47:48 +01:00
Jake Pullen cea3b94b8b base working fine i think 2024-08-09 08:53:02 +01:00
Jake Pullen 41046c0da1 base almost done, just throwing categories out the pram rn 2024-08-07 08:30:44 +01:00
Jake Pullen 22d7df2224 almost into base nicely 2024-08-06 22:38:18 +01:00
Jake Pullen eb99746bdb added requirements.txt 2024-08-06 18:54:26 +01:00
Jake Pullen 69d0d29eaa added tbc pseudo code 2024-07-29 16:38:43 +01:00
Jake Pullen 6bc992e011 first pass at base layer 2024-07-29 15:12:26 +01:00
Jake Pullen 23458af133 start of raw_to_base 2024-07-29 07:40:45 +01:00
Jake Pullen 31b82dc1d0 fine tuning injestion 2024-07-28 10:45:59 +01:00
Jake Pullen fe99ee4469 initial commit, injest POC done 2024-07-28 10:02:18 +01:00