"What is GRIB?" is a nontrivial question. There isn't even agreement on whether it stands for GRIdded Binary or General Regularly-distributed Information in Binary form. This is an idiosyncratic and non-official take on GRIB by an autodidact of met data.
GRIB and BUFR are World Meteorological Organization (WMO) standards for data exchange. WMO calls them data exchange formats, but many refer to them simply as data formats. The loss of that one middle word is significant.
Met data is continually being exchanged around the world between nations and the emphasis is on timely data that can be consumed or ingested by data processing systems at the national weather centers. Transmission efficiency and robustness (to network outages) takes priority over extensive metadata.
Binary data is more compact that human-readable ASCII, but is often not portable between machines. GRIB is designed to be a portable and endian-independent binary format.
A full-fledged data format should ideally contain all the information required to understand the data in the file in a self-contained manner. However, GRIB files need an independent Parameter Table to map GRIB 'codes' correctly to the physical parameters they represent. Apply the wrong parameter table and you could produce garbage instead of insightful analysis.
[I personally agree with (though it is not official RDA policy) John Caron's plea that GRIB and BUFR are not suitable data formats for long-term archival purposes on their own. However, we archive data as close to their 'raw' form as possible, but try to provide information and metadata to help future users read and interpret the data correctly.]
World-wide, more weather data is exchanged in GRIB than any other format. Whether you like GRIB or not; if you want to work with gridded weather data, you need to learn to work with GRIB. You may even need to learn how to work with more than one flavor of GRIB. (Yes, there is more than one flavor of GRIB.)
GRIB works very well as a data exchange or data transmission format for use in operational weather centers, who are not concerned with long-term data preservation. Data archivists and curators need to work a bit harder to ensure that the correct metadata (e.g. GRIB parameter tables) are kept with the corresponding data files. We work with the national centers to keep abreast of the changes and document them for RDA users.
For instance, go to ds084.1 NCEP GFS 0.25 Degree Global Forecast Grids Historical Archive and read its GRIB2 parameter table in HTML. Try that with any of our GRIB data holdings.
Note, these codes exist in the parameter table files, but not in the GRIB files themselves. You have to apply the correct table so that the appropriate codes are applied. Fortunately, many modern GRIB readers and viewers are smart enough to infer the correct table to apply based upon the header information.
The gridded fields of values within GRIB files are sometimes referred to as a 'message', a nod to its heritage as a transmission format. Each message is 'atomic' in the sense that it is the smallest unit of data that makes any sense. They contain a starting point, an end point (in bytes), GRIB codes describing the message contents, and the data values themselves.