2016-08-30

How to access a restricted data set

I alluded earlier to a few* exceptions to open access.  They are open to varying degrees due to restrictions from the data providers (not from RDA.)

To see what is available to you,
  • Click on "dashboard" (at the top of the RDA web portal, right next to "sign out"
  • Then click on "Edit/Change Profile"
  • The bottom half of the screen for an unaffiliated user will look something like this:
Restricted data sets at the RDA.
  • ECMWF Operational data is only accessible to users at UCAR member institutions in the U.S. and Canada, as well as users at all other U.S. universities and government institutions. Eligible users must agree to the ECMWF Terms of Use (TOU).
  • JRA products are available to all affiliated users.
  • COSMIC data access is granted only to affiliated users who have also registered at CDAAC.
  • ECMWF Data (Other Than Operational Data) is available to all registered users who agree to their TOU.
Suppose you want to compare how JRA-55 (ds628.0) and NCEP/NCAR Global Reanalysis (ds090.0) differ over your region of interest.  Read their data set Description pages.  Note that ds628.0 has sections labeled "Access Restrictions" and "Usage Restrictions" while ds090.0 does not.

JRA-55 Access and Usage Restriction information.
Click the "Data Access" tab for JRA-55 and you will see the screen below if you are nonaffiliated. If you are eligible (affiliated), you will be taken to a page with the Terms of Use. Read and accept the Terms of Use and you are then able to access the data.

Nonaffiliated users see this.
If you do have a valid and current affiliation, edit your profile.  Reply to the verification email we send, and then you will be able to read and accept the Terms of Use.
Edit your profile here.  We WILL confirm your email to be sure that it is current.
Some have been reluctant to update their email/affiliations when they leave their universities. Losing access to something you once had is a bummer. I still miss the Olympic-sized swimming pool at Berkeley.

But, users of restricted datasets need to reverify their status by replying to verification email notices sent out every six months. Thus, you don't lose anything by keeping your email and affiliation status up-to-date. In fact, you lose access to email notices that we send out about outages and important updates about data you have downloaded from us in the past.

* Only 78 of our 600+ data sets belong to one of the restricted classes of data sets.

2016-08-26

Yes, Ireland does have droughts: Notes from the 9th ACRE Workshop

The RDA hosts many climate reanalysis collections, yet only two in our archive are unique in that they extend beyond the modern (post-1948) radiosonde era: the NOAA-CIRES 20th Century Reanalysis (1851–2014; 20CR) and the ERA-20C (1900–2010) produced under ECMWF's ERA-CLIM project.  In order to produce a climate reconstruction dating back over a century, both of these reanalyses rely on assimilating very old surface and marine weather observations, which have been assembled into long-term observational datasets such as the International Surface Pressure Databank (ISPD) and the International Comprehensive Ocean-Atmosphere Dataset (ICOADS).  How do these historic observations find their way into a modern climate reconstruction such as 20CR and ERA-20C?

It all starts with the Atmospheric Circulation Reconstructions over the Earth (ACRE) initiative, which is an international collaboration between meteorological organizations, atmospheric scientists, librarians, and volunteers who are dedicated to the recovery and digitization of historic global weather observations.  By facilitating the process of rescuing, digitizing, and performing quality control of data from old weather log books, ACRE ensures that these important observations provide a better understanding of past climate, which in turn aids in modern climate applications and studies of climate impacts.

The 9th ACRE Workshop


ACRE holds annual workshops to discuss all things related to data rescue, and I attended the 9th ACRE Workshop and Historical Weather and Climate Data Forum, held June 20-24 at Maynooth University in Ireland.  The meeting agenda was populated with talks illustrating the full cycle of how historic data observations make their way from being discovered and digitized all the way to being assimilated into climate reconstructions and applications.

Most notably, the annual ACRE workshop is held in different locations around the world, and the local ACRE partners which host the workshop are featured prominently in the meeting agenda.  This not only brings attention to data rescue efforts in the region which hosts the workshop and encourages local groups to become more involved, but it also highlights the importance of how data rescue facilitated by ACRE leads to a better understanding of the past local climate and subsequent future impacts.  At the most recent meeting in June, climate scientists from Maynooth University and Met √Čireann, the Irish national weather service, gave illuminating talks on how they're using ACRE supported observations and reanalyses to understand long-term Irish precipitation trends, the Irish droughts of the 19th and 20th centuries, and how weather played a role in the Easter Rising of April 1916.


St. Patrick's College at Maynooth University provided a beautiful setting for the 9th ACRE Workshop

Other highlights included presentations on the following topics:
  • Dona Cuppett and Rick Crouthamel from the International Environmental Data Rescue Organization (IEDRO) volunteer their time to travel to developing countries around the world to assist in setting up a data rescue and digitization operation.  Their presentation described the logistical challenges involved in undertaking this effort, and underscored the importance of rescuing historic weather observations in developing countries, which are most vulnerable to climate impacts.
  • Reports on efforts to scan and digitize pages from historic weather logs.  These included talks on citizen science projects such as OldWeather.orgWeatherDetective, and the Sir Charles Todd folios project led by Mac Benoy.
  • Stefan Br√∂nnimann presented on the Mount Tambora eruption of 1815 and the subsequent period of global cooling, commonly known as the "Year Without a Summer".
  • Dispatches from ACRE regional chapters leading data rescue efforts around the world.
  • Updates from various groups leading climate reanalysis initiatives, including the 20CR, UERRA, and ERA-CLIM2 projects.


Wanted: Southern Hemisphere data


The benchmark of a good climate reanalysis is performing better than climatology, yet the uncertainty in a reanalysis is controlled by the amount of underlying observations assimilated into the analysis.  More observations correspond to a higher confidence in the analysis, whereas a data sparse region returns an analysis with low confidence.  For long-term reanalyses such as the 20CR and ERA-20C, the analysis in the 19th and early 20th centuries poses a challenge due to the overall scarcity of input data.  In particular, there is a great need for data in the Southern Hemisphere, not only for earlier periods, but for more recent years as well.  Philip Brohan's "Fog of Ignorance" visualizations have become a well-known highlight at ACRE meetings, and illustrate how ground-truth observations improve the certainty in historical climate reconstructions.

Antarctic weather 1909-10 from Philip Brohan illustrates the "Fog of Ignorance" in the 20th Century Reanalysis due to a lack of observations. Note the grey fog disappears in the vicinity of observations (marked by yellow dots) made available by ACRE facilitated data rescue.

ACRE is therefore making a strong push to recover historical data from south of the Equator, with efforts being led in regional chapters such as Chile, Southeast Asia, Southwest Pacific, and Antarctica, the latter of which is recovering observations from Antarctic whaling expeditions in the late 19th and early 20th centuries.

Learn more about ACRE at http://met-acre.net/.

2016-08-19

NCAR RDA outage August 22-24, 2016

UPDATE Thu Aug 25, 2016:
All systems are back up and operating normally.  Thank-you for your patience.

NCAR CISL supercomputers and central file disk systems will be down for system maintenance from 0:01 MDT on Monday, August 22, 2016 until late Wednesday, August 24.  The HPSS systems will begin to shut down at 21:00 MDT on Sunday, August 21.

When we come back, we'll have more storage and be able to provide more data for you on disks.

From the Daily B:
The Yellowstone, Geyser, Caldera, and GLADE systems will be unavailable beginning at 12:01 a.m. Monday, and the HPSS system will be unavailable starting at 9 p.m. Sunday, earlier than previously announced. All of these systems are expected to be back in production by late Wednesday afternoon, August 24.

Facilities upgrades during the downtime are related to expansion of the GLADE file systems and preparation for the new Cheyenne HPC system at the NCAR-Wyoming Supercomputing Center. Power will be unavailable for much of that period.

When the systems are returned to service following the downtime, users will have larger GLADE home space quotas. The current 10-GB quota will be increased to 25 GB. The larger GLADE file systems will serve both the Yellowstone and Cheyenne supercomputers. Users will be informed via the CISL Notifier service when each system is returned to production.

2016-08-01

NCAR RDA Partial Outage Aug 2, 2016

NCAR CISL supercomputers and central file disk system will be down for system maintenance 7:00 to 17:00 MDT on Tuesday Aug 2, 2016.

The RDA web server will remain in operation, but users will not be able to transfer data while the central file disk system is unavailable.  Additionally, data subset processing and translation services will be unavailable during this outage.