GRIB1, GRIB2, NetCDF: What do I use?

[Note: An older version of this information is available under the Documentation Tab of both ds083.2 FNL and ds084.1 GFS as README_Formats.pdf.  This page is maintained and should be considered the definitive version.]

We get this question often from users setting up their data workflow, particularly for datasets for which the data translation option, "Get Converted Files" is available.

Data Access Matrix for ds083.2, which is available in its native GRIB1 and GRIB2 formats or translated to NetCDF.  If you want data translation to NetCDF, click on the options circled in red.

The answer (or non answer) is it depends upon which dataset you want to use (and dates within the series) and which tool(s) you will use to analyze the data.



GRIB is a World Meteorological Organization (WMO) international standard for exchanging GRidded BInary or General Regularly-distributed Information in Binary form data.

GRIB1 is the older format.

GRIB2 improves upon the standard with the same compression software commonly used for images to gain a roughly 50% reduction in file size over GRIB1.  Recently, version 2 of GRIB2 was released.  It uses a slightly less aggressive form of compression, which leads to about 10% larger files than the first version of GRIB2, but decompresses about 10x faster.

NetCDF is another binary format in common use in the weather and climate community.

All three claim to be self-describing in the sense that the files tell you whats in that particular file.  GRIB1 descriptors are short and can be very cryptic.  GRIB2 is slightly better, with longer self-descriptions.  However, GRIB files in general depend on external tables referred to as GRIB Tables to explain the details of the file contents.

NetCDF file metadata tends to be more complete and descriptive.

You can perform many data processing tasks on GRIB files with wgrib (for GRIB1) and wgrib2 (for GRIB2). You can process and visualize GRIB files with NCL (NCAR Command Language), CDO (Climate Data Operators), GRADS, IDL (Interactive Data Language) GRIB Routines, MATLAB (with nctoolbox or MATLAB-CDI), Panoply, and R (using the rNOMADS package)

You can use the extensive NetCDF software libraries if you convert the files to NetCDF. However, be mindful that NetCDF files are not as compact as GRIB2, even when compressed.  A 15 MB GRIB2 FNL ds083.2 file from 2014 can be 30 MB if translated to GRIB1 (an option we no longer support) or 86 MB in NetCDF format.  Even with gzip compression, the GRIB1 and NetCDF files are 24 and 31 MB respectively.  Read README_Formats.pdf for details.

Select your files, then click "View Selected Files/Convert to NetCDF."   When the pop-up window appears, click on "Request converting format to NetCDF for selected files."

In summary, try to store and use the data in it's native format.  This is not an endorsement of any given data format because there are shortcomings and strengths to all of them (but not with the same ratio).  If you want a partisan but fair critique, I recommend John Caron's discussion, On the suitability of BUFR and GRIB for archiving data.

But, if you are already familiar with data tools for a different data format such as NetCDF, and you have the bandwidth to handle the larger file size, then go ahead and translate it. You decide the tradeoff between download time and your learning time.

If you are going to work with weather and climate data, then you will need to learn how to play with GRIB.  You will probably also need to learn to play with NetCDF and HDF5.  Stay tuned for our series about how to get started with GRIB and then NetCDF4/HDF5.