reflective-error

project_title

project_description
Explore the docs »

Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

Reflective Error: A Metric for Assessing Predictive Performance at Extremes

This repo contains the files necessary to reproduce the machine learning model for the paper entitled 'Reflective Error: A Metric for Assessing Predictive Performance at Extremes'; instructions, in terms of prerequisites and accessing the required data, are included below. There are two notebooks: the first, Part I, goes through the fictitious experiments and highlights the motivation in a more straight forward context; the second, Part II, introduces the application of Reflective Error as a loss function for neural networks whilst also highlighting the error metric's usage in a real world setting. Part I requires negligible storage whilst Part II requires no more than 2GB of data; this can be reduced by taking a subset of the ERA5 data stipulated in the notebook (we took data from across the UK as part of a broader hydrological research project). The neural model used was trained on a 24GB M2 Macbook Air (8‑core CPU, 10‑core GPU, 16‑core Neural Engine) in less than a minute, so the computational demand is also low.

In order to download ERA5 data, an ECMWF CDS account will be required along with a CDS API key. Note that the CDS servers were upgraded in the summer of 2024, so users of the system prior to the upgrade will require a new account. Instructions on account creation and downloading data can be found here.

(back to top)

Built With

The following libraries/tools/etc. are required for this project.

• python3
  • pandas
  • numpy
  • scipy
  • matplotlib
• cdsapi
• xarray
• pytorch
• scikit-learn

We recommend using conda for package management, for which instructions on downloading and installing it can can be found here.

(back to top)

Installation

1. This repository can be installed from the command line via:

   git clone https://github.com/robert-edwin-rouse/reflective-error.git

2. The most up to date version of apollo can be installed via:

   git clone https://github.com/robert-edwin-rouse/apollo.git

(back to top)

Usage

As discussed in the brief, the journey through the paper is documented and can be followed through the two Jupyter notebooks. The first notebook covers the fictitious experiments using synthetic, simplified datasets, whilst the second covers the real world application to a hydrological problem. Alternatively, provided the catchment database csv file has been populated with data for the target catchments, the initial combined meteorology and streamflow files can be produced by running the assembly.py script. The artificial neural network can then be trained and the results on the test set, as partitioned in the paper, produced by running reflective_streamflow.py. We have also provided the results we obtained from a grid search through alpha and beta parameters for the reflective loss function and a script that generates the corresponding plots in reflective_streamflow.py.

(back to top)

Roadmap

• Add script to create additional plots.

See the open issues for a full list of proposed features (and known issues).

(back to top)

Contributing

Contributions are welcome.

(back to top)

License

Distributed under the MIT License. See LICENSE.txt for more information.

(back to top)

Contact

Project Link: https://github.com/robert-edwin-rouse/reflective-error

(back to top)

Acknowledgments

• The reviewers who recommended substantial changes to the codebase in order to improve its accessibility and interpretability.

(back to top)