diff --git a/README.MD b/README.MD index 910aa6c..164081f 100644 --- a/README.MD +++ b/README.MD @@ -5,6 +5,7 @@ This project provides tools for processing UK Met Office Rain Radar NIMROD image ## Overview The project consists of a main pipeline workflow that processes multiple modules in sequence: + - `main.py`: Main pipeline orchestrator that calls on the modules as needed - `batch_nimrod.py`: Module for batch processing multiple NIMROD files with configurable bounding boxes - `generate_timeseries.py`: Module for extracting cropped rain data and creating rainfall timeseries @@ -13,22 +14,26 @@ The project consists of a main pipeline workflow that processes multiple modules ## Features ### main.py + - Orchestrates the entire workflow pipeline - Processes DAT files to ASC format - Generates timeseries data for specified locations - Combines grouped CSV files into consolidated datasets ### batch_nimrod.py + - Process multiple NIMROD dat files - Automatically extract datetime from file data - Export clipped raster data to ASC format ### generate_timeseries.py + - Extract cropped rain data based on specified locations - Create rainfall timeseries CSVs for each location - Parse datetime from filename and create proper datetime index ### combine_timeseries.py + - Combine multiple timeseries CSV files into grouped datasets - Group locations by specified output groups - Create consolidated CSV files for each group @@ -49,29 +54,32 @@ It is recommended to use UV for environment and package handling. 1. RunMain Pipeline `uv run main.py 1. find the output in the COMBINED_FOLDER (as per config location) - The main pipeline will: + 1. Process DAT files to ASC format if needed -2. Generate timeseries data for specified locations -3. Combine grouped CSV files into consolidated datasets +1. Generate timeseries data for specified locations +1. Combine grouped CSV files into consolidated datasets ## Configuration The `config.py` file defines folder paths: + - DAT_TOP_FOLDER: "./dat_files" - ASC_TOP_FOLDER: "./asc_files" - CSV_TOP_FOLDER: "./csv_files" - COMBINED_FOLDER: "./combined_files" Example of how the zone csv files should look: -``` -filler, zone_name, easting, northing, other_filler, last_filler, zone_number -aa, TM0816, 608500, 216500, a, a, 1 -aa, TF6842, 568500, 342500, a, a, 1 + +```csv +1K Grid, easting, northing, zone_number +TM0816, 608500, 216500, 1 +TF6842, 568500, 342500, 1 ``` ## Acknowledgments Thank you to the following projects for their inspiration and code: -* [Richard Thomas - Original Nimrod dat to asc file conversion](https://github.com/richard-thomas/MetOffice_NIMROD) -* [Declan Valters - building the timeseries from the asc files](https://github.com/dvalters/NIMROD-toolbox) + +- [Richard Thomas - Original Nimrod dat to asc file conversion](https://github.com/richard-thomas/MetOffice_NIMROD) +- [Declan Valters - building the timeseries from the asc files](https://github.com/dvalters/NIMROD-toolbox) diff --git a/main.py b/main.py index 617122c..1b0e509 100644 --- a/main.py +++ b/main.py @@ -24,11 +24,11 @@ if __name__ == "__main__": reader = csv.reader(csvfile) header = next(reader) # Skip header row for row in reader: - # Extract the relevant fields: Ossheet (location ID), Easting, Northing, Zone - zone_id = row[1] # Ossheet column - easting = int(row[2]) # Easting column - northing = int(row[3]) # Northing column - zone = int(row[6]) # ZoneID column + # Extract the relevant fields: 1K Grid, Easting, Northing, Zone + zone_id = row[0] # Ossheet column + easting = int(row[1]) # Easting column + northing = int(row[2]) # Northing column + zone = int(row[3]) # ZoneID column locations.append([zone_id, easting, northing, zone]) batch = BatchNimrod(Config)