Table of Contents
This tool was developed as a trigger mechanism for the typhoon Early action protocol of the Philippines Red Cross FbF project. The model will predict the potential damage of a typhoon before landfall, and the prediction will be percentage of completely damaged houses per municipality. The tool is available under the GPL license
To run the pipeline, you need access to an Data.zip, and credentiials for 510 Datalake and FTP server. If you or your organization is interested in using the pipeline,
please contact 510 Global to obtain the credentials. You will receive a file either called secrets or .env, which you need to place in the top-level directory.
The main use of this data pipeline is to update the status of Typhoon IBF portal based on the latest ECMWF forecast. The status of Triggered/Not Triggred is defined based on EAP trigger value. Currently this is based on Average impact. This value can be updated the setting.py file at this line Defult value is Average, other possible values are 50,70 and 90 , which is the percentage of ensamble members passing the treshold.
- Clone the repo
- Fill in necessary credentials:
- If using
secret.py, change/IBF-Typhoon-model/src/typhoonmodel/utility_fun/secrets.py.templatetosecrets.pyand fill in credentials. - If using
.env, change/IBF-Typhoon-model/.env.templateto.envand fill in credentials.
- If using
- Install Docker-compose follow this link
Create a new conda environment named tyworkflow with all required packages by running:
conda create -n tyworkflow python=3.11 ipykernel pandas numpy geopandas scikit-learn xgboost rasterstats xarray rasterio fiona shapely cartopy dask earthpy pycountry contextily imageio h5py numba pathos cftime cfgrib netCDF4 eccodes statsmodels requests pybufrkit pint feedparser lxml beautifulsoup4 tqdm azure-storage-file azure-identity azure-storage-blob azure-storage-file azure-storage-file-datalake python-dotenv -c conda-forgeMake sure you have VS Code, JupyterLab, or another IDE that supports Jupyter notebooks.
- Open the project folder in Visual Studio Code.
- Open and run the
pipeline.ipynbnotebook within thetyworkflowenvironment.
You will need to have docker and docker-compose installed.
To build and run the image, ensure you are in the top-level directory and execute:
docker-compose up --build
When you are finished the run
docker-compose down
to remove any docker container(s).
Follow the instraction here there is also a workflow in github action to update docker image in azure registery, which will be the image used in logic app.