Commit Graph

32 Commits

Author SHA1 Message Date
Jake Pullen 3504641643 understanding dash 2024-08-10 21:47:08 +01:00
Jake-Pullen f06ce9999f Merge pull request #2 from Jake-Pullen/Develop
1.1 Release, Better Error Handling and Better Logging
2024-08-10 20:17:14 +01:00
Jake-Pullen b8681f010f Merge pull request #1 from Jake-Pullen/feature/better_error_handling
Feature/better error handling
2024-08-10 19:48:22 +01:00
Jake Pullen 19b633eb40 added error handling to facts and dimensions transformations 2024-08-10 19:45:17 +01:00
Jake Pullen 861ccd0dc4 bug fix, a sucessful pipeline completion resulted in error message...
bug fix, a successful dimension save resulted in failed save and skip the rest of processing
2024-08-10 18:47:06 +01:00
Jake Pullen 812e107013 Added more error handling in for raw to base 2024-08-10 18:36:39 +01:00
Jake Pullen 87c6bf8718 added log for start and end of pipeline 2024-08-10 17:34:50 +01:00
Jake Pullen c5cdb16a8a JP is stupid 2024-08-10 17:13:51 +01:00
Jake Pullen 24cc006ef5 Added more error handling and logic to ingest 2024-08-10 17:11:57 +01:00
Jake Pullen d25136fbb0 Added some more error handling and request limit handling 2024-08-10 15:54:31 +01:00
Jake Pullen 4fea87030e added init file to config 2024-08-10 14:21:15 +01:00
Jake Pullen 296787d7b0 added better logging and moved config to its own folder 2024-08-10 14:10:16 +01:00
Jake Pullen ad633edefd added check for present api toen and budget id 2024-08-10 13:26:02 +01:00
Jake Pullen fa6332e18a removed pycache 2024-08-10 10:09:26 +01:00
Jake Pullen c7a5b5f092 name update 2024-08-10 09:57:08 +01:00
Jake Pullen b29b738a7a ready for open source 2024-08-10 09:50:37 +01:00
Jake Pullen c9b01e85d5 fixed scheduled transactions 2024-08-10 08:29:50 +01:00
Jake Pullen 696b72988d added facts 2024-08-10 08:07:14 +01:00
Jake Pullen 609ea38c43 slashes might be an issue? 2024-08-09 18:16:38 +01:00
Jake Pullen b7b6f8887c dimensions done, still need to fully test 2024-08-09 16:27:36 +01:00
Jake Pullen da5cf943d6 accounts into dimension 2024-08-09 15:14:04 +01:00
Jake Pullen 165464820b starting to build the warehouse 2024-08-09 14:23:52 +01:00
Jake Pullen 33c9496eb0 . 2024-08-09 13:47:48 +01:00
Jake Pullen cea3b94b8b base working fine i think 2024-08-09 08:53:02 +01:00
Jake Pullen 41046c0da1 base almost done, just throwing categories out the pram rn 2024-08-07 08:30:44 +01:00
Jake Pullen 22d7df2224 almost into base nicely 2024-08-06 22:38:18 +01:00
Jake Pullen eb99746bdb added requirements.txt 2024-08-06 18:54:26 +01:00
Jake Pullen 69d0d29eaa added tbc pseudo code 2024-07-29 16:38:43 +01:00
Jake Pullen 6bc992e011 first pass at base layer 2024-07-29 15:12:26 +01:00
Jake Pullen 23458af133 start of raw_to_base 2024-07-29 07:40:45 +01:00
Jake Pullen 31b82dc1d0 fine tuning injestion 2024-07-28 10:45:59 +01:00
Jake Pullen fe99ee4469 initial commit, injest POC done 2024-07-28 10:02:18 +01:00