Why did we decide to propose FIDUCEO? The basic idea behind the project arose from discussions over what was needed to improve the broad band infrared data used to produce certain climate data records (CDRs). Over the past few years it had become apparent (at least to us) that the radiances provided from many sensors could be significantly in error when compared to the accuracies required for climate change studies. So we decided to try and fix some of the well-known sensors and do it in a way which would also make the data as trustworthy as possible.
It also turns out that many of the sensors which we knew had problems were among the most used and/or most long lived available and are crucial for climate change studies. The sensors we were thinking of included the Advanced Very High Resolution Radiometer (AVHRR), microwave sensors to look at the critical upper tropospheric humidity and the High resolution Infrared Radiation Sounder (HIRS), all of which have versions dating back more than 30 years. It is also true that some of the more modern sensors have also shown significant biases relative to top-of-atmosphere reference sensors. So it may be that a different approach to the problem of accurate instrument calibration is needed – which is where FIDUCEO comes in.
Perhaps the first question to ask is why does the operational calibration for so many sensors introduce biases and errors? There are a number of reasons for this. Sometimes there were issues with the design of the sensor. It should be remembered that many of the early operational sensors were designed with weather forecasting in mind and so what was important were things like ‘where are the clouds?’ rather than providing very high levels of accuracy. As an example, the design specification for the original AVHRR sensors was (in the IR) an accuracy of 1K only, a long way from <0.1 (or 0.05)K requirement from the climate community. So problems such as stray light issues (caused by a very open design) or the presence of strong thermal gradients across calibration targets limiting their usability were not really an issue for the original designers. Also, often problems were caused by limited and/or poor pre-launch characterization either related to lack of facilities and/or time pressures or simply not understanding that the orbital conditions can be significantly different from that of the test environment. But from an operational point of view instruments such as the AVHRR still met their design specification so there was no strong requirement to push the calibration to higher levels of accuracy.
The presence of such biases in the radiance record make it very difficult (if not impossible) to produce CDRs at the level of accuracy needed for climate studies. It turns out that all is not lost, however, in that for many sensors the problems listed above can be removed or at least ameliorated by understanding that often there is a good stable sensor “under the hood” which was being hindered by a poor (or non-existent) calibration system. The reason for this is simple as at the core of many sensors is a detector which is (mostly) kept at a fixed operating temperature so its characteristics should be quite stable. Then the problem devolves to whether or not the problems with the calibration system (if present) can be modelled in such a way as to improve the accuracy of the data. It turns out that in certain circumstances you can as long as the problem is approached in a physically based manner. By physically based we mean approaching the instrument from a physicists point of view and try and incorporate physically based models as much as possible rather than use ad-hoc algorithms. Previous work by members of the FIDUCEO team had already shown an order of magnitude improvement in accuracy for the AVHRRs flown on the MetOp series of satellites so aspects of the approach had already been validated. The FIDUCEO plan is to broaden the scope of such work and provide a framework for satellite recalibration that could be applied to a range of sensors.
Of course just improving the calibration of a series of sensors does not provide all the information needed. From our point of view representative uncertainties are just as important as accurate radiances. This is why we brought in the concepts of Metrology (the science of measurement) which is defined as “the science of measurement, embracing both experimental and theoretical determinations at any level of uncertainty in any field of science and technology”. From our point of view perhaps the most important part of Metrology is the concept of traceability defined as "the property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty". This statement is actually more complex than it may seem because in order to get sensible estimates of the uncertainties you first have to remove known sources of bias and error from the data. It also forces you to think about every single process however small, and also think about any assumptions that have been made. The issue of understanding assumptions is critical to getting a good Fundamental Climate Data Record (FCDR). One of the major issues with the AVHRR, for example, was an assumption that the calibration parameters derived from pre-launch testing would be applicable for the in-orbit conditions even though the satellite thermal environments were completely different between the two and this difference lead to large (up to 0.5K) biases in the data.
Finally we realized that we couldn’t just leave it at creating new FCDRs with traceable uncertainties - most people who study climate change look at geophysical variables and/or essential climate variables such as surface temperature, not raw satellite radiances. So we also included climate data record generation as part of FIDUCEO with the idea of both using the data with proper uncertainties developed for the FIDUCEO FCDRs as well as applying the same metrological rigor to the CDR generation process itself. Then, by having metrological sound CDRs which will have traceable uncertainties traced back to a known reference and/or physical processes we (and the rest of the climate change community) will be in a position to know to what extent the data can be trusted and the limits to which the data can be pushed in detecting the sometimes small changes introduced by climate change.