Over the past two decades, independent state-of-the-art international atmospheric reanalysis programs have been conducted by ECMWF (European Centre for Medium-Range Weather Forecasts) and JMA (Japan Meteorological Agency). The most recent reanalyses released by these programs are ERA5 (from ECMWF, spanning 1950 to the current time), and JRA-55 (from JMA, spanning 1958 to the current time). In addition, ECMWF produces near real-time ultra high resolution operational atmospheric forecasts that are an important complement to ERA5.
In a collaborative effort within CISL ISD, DECS (the Data Engineering and Curation Section), downloads, processes, archives, and provides web-based access and services for ERA5, JRA-55, and ECMWF operational forecast data to a wide cross-section of users and organizations located worldwide. In an ongoing effort to improve the generality and usability of these datasets, processing by DECS personnel using CISL HPC resources include quality control, regridding to standardized lat-lon grids, restructuring incoming data into single-parameter time series, and in the case of ECMWF data, conversion of GRIB format to CF-conforming compressed netCDF4-HDF5 files. In addition, extensive logistical support and scientific consulting is provided via several platforms, including the NCAR Research Data Archive (RDA) Helpdesk
These efforts support a broad range of climate diagnostics, model development and validation, and intercomparison projects to name a few. The ERA5, JRA-55, and ECMWF operational forecast data is organized into 17 datasets within the Research Data Archive which is managed by DECS. The total volume of these datasets is 2 PB, all of which is stored on the CISL GLADE shared file system. Metrics for web-access alone over the last 15 months indicate that 3600 users have downloaded 2.7 PB of RDA ERA5, JRA-55, and ECMWF operational forecast data. We encourage CISL HPC users to explore this archive via the RDA website, and especially to take advantage of the availability on GLADE. Three RDA datasets in particular are good entry points to begin your investigations:
- ERA5 Reanalysis (0.25 Degree Latitude-Longitude Grid
- GLADE root: /gpfs/fs1/collections/rda/data/ds633.0
- Japanese 55-year Reanalysis, Daily 3-Hourly and 6-Hourly Data
- GLADE root: /gpfs/fs1/collections/rda/data/ds628.0
- ECMWF IFS CY41r2 High-Resolution Operational Forecasts
- GLADE root: /gpfs/fs1/collections/rda/data/ds113.1
HPC users will find that the above links provide more detailed listings of GRIB and netCDF4-HDF5 files located on GLADE, supporting metadata, documentation, and cross references to related RDA ERA5 and JRA-55 datasets including monthly means and variances, back extensions of ERA5 data to 1950, JRA-55 near real-time data, and supplementary and derived products. We endeavor that this serves the HPC community well and in many capacities.
