How to work around operational changes to GFS/GDAS/FNL

Operational models are moving targets.

Any changes in software for an operational analysis can result in spurious signals or shifts. Operational systems' software is frequently changed as they uncover bugs or biases and fix them; code segments are improved to better represent atmospheric phenomena. The changes are usually not announced ahead of time. The change log may be difficult for the non-expert user to decipher. Thus, operational analyses are not appropriate for compiling a long time series to study changes over time, e.g. to look for climate signals.

A handful of users in the last week have contacted rdahelp@ucar.edu about problems initializing WRF with FNL data spanning 2016-05-11. Why does WPS, the WRF Preprocessing System, crash right at this juncture?

The answer lies in the operational changes made to the GFS/GDAS system at NOAA NCEP. You can view the full list of changes to GFS/GDAS since 1991 including the announcement of the changes effective 2016-05-11 at 12 UTC.

- Addition of five layers in the upper stratosphere in gridded output

This change affects ds084.1 NCEP GFS 0.25 Degree, ds083.2 NCEP FNL 1.0 Degree, ds083.3 NCEP FNL 0.25 Degree and ds335.0 Historical Unidata Internet Data Distribution (IDD) Gridded Model Data.

The first thing to do when things break is to examine your data. You can use wgrib2, or take a close look at the metadata for your input data set, as I've shown earlier, in WRF Vtable Carpentry.

Scroll down to the "Vertical Levels" section of the data set information page and click on "detailed metadata."

You will see that six parameters (HGT, TMP, RH, UGRD, VGRD, 03MR), many used by WRF, received five additional levels (1, 2, 3, 5, 7 mb) beginning 2016-05-11 at 12 UTC.

These changes do not affect the performance of ungrib.exe, but they may impact real.exe and wrf.exe.

The RDA team (rdahelp@ucar.edu) helps our data users with WRF WPS up to providing an accurate Vtable that describes the data obtained from us. The WRF team (wrfhelp@ucar.edu) picks up support from there. This problem spans both support groups.

I contacted WRF to coordinate our help and I am posting information beyond RDA's normal scope here as a service to our users so that all the information to get unstuck resides in one place.

First, you can work around the problem by removing the new vertical levels in the files after the GFS/GDAS change.  Then all of your WPS input will be consistent.  You can do this yourself with wgrib2, or our "Get a Subset" tool.

UPDATE Nov 18, 2016
The following method, provided by wrfhelp, does not work. You need to use the first method above. For details on how to achieve this using wgrib2, read GRIB2 file carpentry.

If you want to use all of the available data in a WRF run that spans a discontinuity of data:

1. Run ungrib.exe as usual
2. Using a namelist that accurately describes the number of vertical levels found in the input GRIB2 files before the change (27 in this case), run metgrid.exe and real.exe for the times before the change.
3. Edit your namelist to the new number of vertical levels (32) and then run metgrid.exe and real.exe on the remainder of the GRIB2 input files for the period after the change.
4. This generates a complete set of wrfbdy and wrfinput files for your entire WRF run.
5. Run wrf.exe up until the discontinuity with 27 in the namelist.  
6. Then edit the namelist for 32 vertical levels and restart wrf.exe for the remainder of the run.

Pat yourself on the back. You've just completed numerical integration of a complex system of differential equations around a discontinuity. ;-)