HiTEA - High Throughput Environmental Assessment Pipeline

The High Throughput Environmental Assessment Pipeline (HiTEA) project will build an end-user friendly tool for sophisticated sensitivity and uncertainty analysis towards reaching the Sustainable Development Goals (SDGs).

HiTEA will make it simple, fast and cost-effective for researchers to perform thousands of model runs when they would previously only undertake one. The main impact will be to change the standard practice of point estimates in environmental assessments to exploration of solution spaces. By integrating life-cycle assessments and supply-chain analysis into parameterized scenarios, the likelihood of “problems” and “opportunities” can be realized, with direct benefits for better information on long-term sustainability policy design.

The HiTEA architecture combines emerging digital container technologies, in memory column storage and interactive notebooks. HiTEA is planned to run on various e-infrastructure systems ranging from local multi-core servers, to HPC infrastructure at NTNU (OpenStack), Sigma2 (Nird Toolbox) and to commercial Kubernetes based cloud services like Amazon AWS and MS Azure. After reaching a mature software product, we aim to include HiTEA into the service catalogue of the EOSCHub.

Case studies connected to SDGs will be conducted during the project. These run in parallel to the software development, providing the feedback needed to ensure software which thoroughly meets scientific user requirements. To do so, the project adopts industry standard Agile Software Development practices for the use in scientific software development.

HiTEA embraces Open Source development. Besides maximizing outreach and exploitation, this will provide feedback on the usability and required capabilities already during the development phase. In addition, it actively encourages code contributions and usage from outside the project team to deliver a tool fully owned by the sustainability science community.

HiTEA is a HPC (High Performance Computing) focused software project that combines emerging digital container technologies, in memory column storage (Apache Arrow) and interactive (Jupyter) notebooks to build the standard tool for quantitative sustainability research for conducting high throughput environmental assessments. With HiTEA we aim to enable researchers to routinely conduct sophisticated (HPC-dependent) solution-space estimates rather than single–point estimates of results and policy options in Sustainable Development Goals (SDG) relevant analysis. In its core, HiTEA consists of a data pipeline which accepts data from various quantitative sustainability accounting frameworks. Several already existing parsers will be bundled in HiTEA to provide a generic entry point into the HiTEA work flow. After passing the data through the distributed/cloud-based calculation engine the result data will be gathered in a designated module which will allow to export the results in various format.