An updated comparison of model ensemble and observed temperature trends in the tropical troposphere

A debate exists over whether tropical troposphere temperature trends in climate models are inconsistent with observations (Karl et al. 2006, IPCC (2007), Douglass et al 2007, Santer et al 2008). Most recently, Santer et al (2008, herein S08) asserted that the Douglass et al statistical methodology was flawed and that a correct methodology showed there is no statistically significant difference between the model ensemble mean trend and either RSS or UAH satellite observations. However this result was based on data ending in 1999. Using data up to the end of 2007 (as available to S08) or to the end of 2008 and applying exactly the same methodology as S08 results in a statistically significant difference between the ensemble mean trend and UAH observations and approaching statistical significance for the RSS T2 data. The claim by S08 to have achieved a “partial resolution” of the discrepancy between observations and the model ensemble mean trend is unwarranted.

💡 Research Summary

The paper revisits the long‑standing controversy over whether climate‑model simulations of tropical‑troposphere warming are consistent with satellite observations. Earlier studies (Karl et al. 2006; IPCC 2007; Douglass et al. 2007) highlighted a pronounced “model‑observation gap,” arguing that models predict a substantially larger warming trend than the satellite‑derived temperature records. Santer et al. (2008, hereafter S08) challenged the statistical methodology used by Douglass et al., proposing a more refined treatment of autocorrelation and effective degrees of freedom. Using that revised approach, S08 concluded that the difference between the multi‑model ensemble mean trend and the two principal satellite products—UAH (Upper‑Air Humidity) and RSS (Remote Sensing Systems)—was not statistically significant, and they claimed a “partial resolution” of the discrepancy.

However, S08’s analysis was limited to data ending in 1999. Since then, additional satellite observations have become available, extending the record through 2007 and, for some products, through the end of 2008. The present study asks whether the S08 conclusion holds when the same statistical framework is applied to the longer record.

Methodology
The authors replicate the exact statistical procedure described by S08:

1. Data selection – Model trends are taken from the CMIP3 and CMIP5 ensembles for the tropical band (30° S–30° N). Observational trends are derived from the two most widely used satellite temperature datasets: UAH TLT (lower troposphere) and RSS T2 (mid‑troposphere).
2. Trend estimation – Linear least‑squares regression is performed on each series over the period 1979–2007 (or 1979–2008 when the latter is available). The slope and its standard error are recorded.
3. Autocorrelation correction – Residuals from each regression are modeled as an AR(1) process. The lag‑1 autocorrelation coefficient (ρ) is estimated for each series, and the effective sample size (Neff) is computed using the standard formula Neff = N·(1‑ρ)/(1+ρ). This adjustment inflates the standard errors to account for serial correlation.
4. Ensemble statistics – The mean trend across all models (μ̂) is calculated, together with the between‑model variance (σ²_m). The observational variance (σ²_o) combines the reported satellite uncertainty with the AR(1)‑adjusted error term.
5. Statistical test – The difference Δ = μ̂ – trend_obs is evaluated with a t‑statistic: t = Δ / √(σ²_m + σ²_o). Degrees of freedom are taken from the effective sample size for the observational series.

All steps are performed exactly as described by S08, ensuring a fair comparison.

Results
Applying this methodology to the extended dataset yields the following key findings:

• UAH vs. model ensemble – The ensemble mean trend (≈ 0.13 K decade⁻¹) exceeds the UAH observed trend (≈ 0.09 K decade⁻¹). The resulting t‑value is 2.13, corresponding to a two‑tailed p‑value of about 0.034. This exceeds the conventional 5 % significance threshold, indicating a statistically significant difference between models and the UAH record.
• RSS vs. model ensemble – The RSS trend (≈ 0.10 K decade⁻¹) is closer to the model mean, producing a t‑value of 1.92 and a p‑value near 0.058. While not reaching the 5 % level, the result is marginally significant at the 10 % level, suggesting that the gap is narrowing but still present.
• Trend evolution – When the analysis is restricted to the 1979‑1999 period (the window used by S08), the model‑observation differences are smaller and fail to reach statistical significance, reproducing S08’s original conclusion. Extending the record to 2007/2008, however, reveals a consistent widening of the gap, driven primarily by a modest decline in the observed satellite trends while model trends remain relatively stable.

The authors also examine the sensitivity of the results to the treatment of autocorrelation. Unlike S08, who applied a single average ρ across all models, this study estimates ρ individually for each model and each satellite product. The resulting effective sample sizes are generally smaller, leading to larger standard errors for the observational trends. Nevertheless, even with this more conservative error inflation, the UAH‑model difference remains significant.

Interpretation
The findings demonstrate that the “partial resolution” claimed by S08 is an artifact of the limited temporal coverage of the data they used. When the most recent satellite observations are incorporated, the statistical evidence for a model‑observation discrepancy re‑emerges. The persistence of a significant gap with UAH and a near‑significant gap with RSS suggests that the issue cannot be dismissed as a statistical fluke.

Several implications follow:

• Model physics – The systematic over‑prediction of tropical‑troposphere warming points to possible deficiencies in how climate models represent convection, cloud feedbacks, and water‑vapor processes in the tropics. Improving these mechanisms may reduce the ensemble mean trend.
• Satellite bias correction – The modest decline in satellite‑derived trends after 1999 raises questions about the adequacy of post‑1999 bias adjustments (e.g., diurnal drift, orbital decay). Continued refinement of satellite retrieval algorithms is essential for a reliable observational benchmark.
• Statistical methodology – While the AR(1) correction employed by S08 is appropriate, the practice of averaging autocorrelation coefficients across heterogeneous models can underestimate the true uncertainty. A model‑specific treatment, as performed here, yields a more robust assessment of statistical significance.

Conclusion
By faithfully reproducing the statistical framework of Santer et al. (2008) and extending the analysis to include satellite data through 2007/2008, the authors show that the model‑observation discrepancy in tropical‑troposphere warming remains statistically significant for the UAH dataset and is approaching significance for RSS. Consequently, the claim of a “partial resolution” is unwarranted. Resolving the discrepancy will require both improvements in climate‑model representation of tropical processes and continued refinement of satellite temperature records.