Resources for uncertainty quantification for physical ocean variables

Guides, websites, and handbooks for best practices in quantifying uncertainty

Publications and reports

UQ in (physical) oceanography

  • Abraham, J. P., Baringer, M., Bindoff, N. L., Boyer, T., Cheng, L. J., Church, J. A., et al. (2013). A review of global ocean temperature observations: Implications for ocean heat content estimates and climate change. Reviews of Geophysics, 52(3), 450–483.
  • Bos, M. S., Williams, S. D. P., Araujo, I. B., & Bastos, L. (2014). The effect of temporal correlated noise on the sea level rate and acceleration uncertainty. Geophysical Journal International, 196(3), 1423–1430.
  • Chelton, D. B. (1994). Physical Oceanography: A Brief Overview for Statisticians. Statistical Science, 9(2), 150–166.
  • Chelton, D. B. (1983). Effects of sampling errors in statistical estimation. Deep Sea Research Part a. Oceanographic Research Papers, 30(10), 1083–1103.
  • Forget, G., & Wunsch, C. (2007). Estimated Global Hydrographic Variability. Journal of Physical Oceanography, 37(8), 1997–2008.
  • Kennedy, J. J. (2014). A review of uncertainty in in situ measurements and data sets of sea surface temperature. Reviews of Geophysics, 52(1), 1–32.
  • Kuusela, M. and M. L. Stein. Locally stationary spatio-temporal interpolation of Argo profiling float data. Proceedings of the Royal Society A 474:20180400, 2018.
  • Matthews, J. L., Mannshardt, E., & Gremaud, P. (2013). Uncertainty Quantification for Climate Observations. Bulletin of the American Meteorological Society, 94(3), ES21–ES25.
  • Merchant, C.J., Paul, F., Popp, T., Ablain, M., Bontemps, S., Defourny, P., Hollmann, R., Lavergne, T., Laeng, A., De Leeuw, G. and Mittaz, J., 2017. Uncertainty information in climate data records from Earth observation. Earth System Science Data, 9(2), pp.511-527.
  • Moroni, David F.; Ramapriyan, Hampapuram; Peng, Ge; Hobbs, Jonathan; Goldstein, Justin; Downs, Robert; et al. (2019): Understanding the Various Perspectives of Earth Science Observational Data Uncertainty. ESIP. Report.
  • Liu, C., X. Liang, D. P. Chambers, and R. M. Ponte, Global Patterns of Spatial and Temporal Variability in Salinity from Multiple Gridded Argo Products. J. Climate, doi:
  • National Research Council. 1993. Statistics and Physical Oceanography. Washington, DC: The National Academies Press.
  • Ramapriyan, H K, Peng G, Moroni D, Shie C-L, Ensuring and Improving Information Quality for Earth Science Data and Products. D-Lib Magazine, 23 (7/8), July/August 2017, DOI:
  • Sutton, A.J., Sabine, C.L., Maenner-Jones, S., Lawrence-Slavas, N., Meinig, C., Feely, R.A., Mathis, J.T., Musielewicz, S., Bott, R., McLain, P.D. and Fought, H.J., 2014. A high-frequency atmospheric and seawater pCO2 data set from 14 open-ocean sites using a moored autonomous system. Earth Syst. Sci. Data, 6(2), pp.353-366.
  • Timms GP, de Souza PA Jr, Reznik L, Smith DV. Automated data quality assessment of marine sensors. Sensors (Basel). 2011;11(10):9589-9602. doi:10.3390/s111009589
  • UNESCO/IOC. 2020. Quality Control of in situ Sea Level Observations: A Review and Progress towards Automated Quality Control, Vol. 1. Paris, UNESCO. IOC Manuals and Guides No. 83. (IOC/2020/MG/83Vol.1),
  • Wunsch, C. (2018). Towards determining uncertainties in global oceanic mean values of heat, salt, and surface elevation. Tellus a: Dynamic Meteorology and Oceanography, 70(1), 1–14.

UQ in science & UQ communication

  • Buchanan, M. (2020). The certainty of uncertainty. Nature Physics, 16(2), 120–120.
  • Carslaw, K., Lee, L., Regayre, L., & Johnson, J. (2018). Climate Models Are Uncertain, but We Can Do Something About It. Eos, Transactions American Geophysical Union, 99, 1–6.
  • Gibbs, P. (2013). Making sense of uncertainty (pp. 1–28). London: Sense about Science.
  • Ho, E. H., & Budescu, D. V. (2019). Climate uncertainty communication. Nature Climate Change, 9(11), 802–803.
  • Hogan Carr, R., Montz, B., Maxfield, K., Hoekstra, S., Semmens, K., & Goldman, E. (2016). Effectively Communicating Risk and Uncertainty to the Public: Assessing the National Weather Service's Flood Forecast and Warning Tools. Bulletin of the American Meteorological Society, 97(9), 1649–1665.
  • Howe, L. C., MacInnis, B., Krosnick, J. A., Markowitz, E. M., & Socolow, R. (2019). Acknowledging uncertainty impacts public acceptance of climate scientists' predictions. Nature Climate Change, 9(11), 863–867.
  • National Research Council. (2013). Environmental Decisions in the Face of Uncertainty. (F. A. Sloan, Ed.) (Vol. 12568, pp. 1–228). National Academies Press. Retrieved from
  • Oppenheimer, M., Little, C. M., & Cooke, R. M. (2016). Expert judgement and uncertainty quantification for climate change. Nature Geoscience, 6(5), 445–451.
  • Oreskes, N. (2015). The fact of uncertainty, the uncertainty of facts and the cultural resonance of doubt. Philosophical Transactions of the Royal Society of London. Series a: Physical and Engineering Sciences, 373(2055), 20140455–21.
  • Palmer, T. N., & Hardaker, P. J. (2011). Handling uncertainty in science. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 369(1956), 4681–4684.
  • Spiegelhalter, D. (2017). Risk and Uncertainty Communication. Annual Review of Statistics and Its Application, 4(1), 31–60.
  • Spiegelhalter, D. J. (2014). The future lies in uncertainty. Science, 345(6194), 264–265.

Statistical & stochastic representation of UQ

  • Berger, J. O., & Smith, L. A. (2019). On the Statistical Formalism of Uncertainty Quantification. Annual Review of Statistics and Its Application, 6(1), 433–460.
  • Chelton, D. B., Eddy, W. F., & DeVeaux, R. (1994). Report on Statistics and Physical Oceanography. Statistical Science, 9(2), 167–221.
  • Coveney, P. V., Dougherty, E. R., & Highfield, R. R. (2016). Big data need big theory too. Philosophical Transactions of the Royal Society of London. Series a: Physical and Engineering Sciences, 374(2080), 20160153–11.
  • Pitman, B. E. (2020, March 2). Model Uncertainty: Mathematical and Statistical. Retrieved March 13, 2020, from
  • Williamson, D. B., & Sansom, P. G. (2019). How Are Emergent Constraints Quantifying Uncertainty and What Do They Leave Behind? Bulletin of the American Meteorological Society, 100(12), 2571–2588.
  • Arnold, H. M., Moroz, I. M., & Palmer, T. N. (2013). Stochastic parametrizations and model uncertainty in the Lorenz '96 system. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 371(1991), 20110479–20110479.
  • Subramanian, A., Juricke, S., Dueben, P., & Palmer, T. (2019). A Stochastic Representation of Subgrid Uncertainty for Dynamical Core Development. Bulletin of the American Meteorological Society, 100(6), 1091–1101.

UQ in (Bayesian) inverse modeling & data assimilation

  • Bui-Thanh, T., Burstedde, C., Ghattas, O., Martin, J., Stadler, G., & Wilcox, L. C. (2012). Extreme-scale UQ for Bayesian inverse problems governed by PDEs. IEEE Computer Society Press.
  • Challenor, P. (2012). Using Emulators to Estimate Uncertainty in Complex Models. (A. Dienstfrey & R. F. Boisvert, Eds.) (pp. 1–14). IFIP International Federation for Information Processing.
  • Chen, P., Villa, U., & Ghattas, O. (2019). Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty. Journal of Computational Physics, 385, 163–186.
  • Flath, H. P., Wilcox, L. C., Akçelik, V., Hill, J., van Bloemen Waanders, B., & Ghattas, O. (2011). Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations. SIAM Journal on Scientific Computing, 33(1), 407–432.
  • Gregg, W. W., Friedrichs, M. A. M., Robinson, A. R., Rose, K. A., Schlitzer, R., Thompson, K. R., & Doney, S. C. (2009). Skill assessment in ocean biological data assimilation. Journal of Marine Systems, 76(1-2), 16–33.
  • Isaac, T., Petra, N., Stadler, G., & Ghattas, O. (2015). Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet. Journal of Computational Physics, 296(C), 348–368.
  • Janjić, T., Bormann, N., Bocquet, M., Carton, J. A., Cohn, S. E., Dance, S. L., et al. (2017). On the representation error in data assimilation. Quarterly Journal of the Royal Meteorological Society, 144(713), 1257–1278.
  • Kalmikov, A. G., & Heimbach, P. (2014). A Hessian-Based Method for Uncertainty Quantification in Global Ocean State Estimation. SIAM Journal on Scientific Computing, 36(5), S267–S295.
  • Kaminski, T., Kauker, F., Toudal Pedersen, L., Vossbeck, M., Haak, H., Niederdrenk, L., et al. (2018). Arctic Mission Benefit Analysis: impact of sea ice thickness, freeboard, and snow depth products on sea ice forecast performance. The Cryosphere, 12(8), 2569–2594.
  • Mohammadi, B. (2015). Backward uncertainty propagation in shape optimization. International Journal for Numerical Methods in Fluids, 80(5), 285–305.
  • Moser, R. D., & Oliver, T. A. (2015). Validation of Physical Models in the Presence of Uncertainty. Handbook of Uncertainty Quantification (pp. 1–28). Cham: Springer International Publishing.
  • Oden, T., Moser, R., & Ghattas, O. (2010). Computer predictions with quantified uncertainty, Part I. SIAM News, 43(9), 1–3.
  • Rougier, J. (2013). "Intractable and unsolved": some thoughts on statistical data assimilation with uncertain static parameters. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 371(1991), 20120297–20120297.
  • Roy, C. J., & Oberkampf, W. L. (2011). A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Computer Methods in Applied Mechanics and Engineering, 200(25-28), 2131–2144.
  • Saibaba, A. K., & Kitanidis, P. K. (2015). Fast computation of uncertainty quantification measures in the geostatistical approach to solve inverse problems. Advances in Water Resources, 82(C), 124–138.
  • Salter, J. M., B, W. D., John, S., & Viatcheslav, K. (2019). Uncertainty Quantification for Computer Models With Spatial Output Using Calibration-Optimal Bases. Journal of the American Statistical Association, 0(0), 1–24.

UQ in climate science/prediction

  • Curry, J. A., & Webster, P. J. (2011). Climate Science and the Uncertainty Monster. Bulletin of the American Meteorological Society, 92(12), 1667–1682.
  • Hawkins, E., Smith, R. S., Gregory, J. M., & Stainforth, D. A. (2015). Irreducible uncertainty in near-term climate projections. Climate Dynamics, 1–13.
  • Katz, R. W., Craigmile, P. F., Guttorp, P., Haran, M., Sansó, B., & Stein, M. L. (2013). Uncertainty analysis in climate change assessments. Nature Climate Change, 3(9), 769–771.
  • Marotzke, J. (2018). Quantifying the irreducible uncertainty in near‐term climate projections. Wiley Interdisciplinary Reviews: Climate Change, 10(1), e563.
  • Parker, W. S. (2013). Ensemble modeling, uncertainty and robust predictions. Wiley Interdisciplinary Reviews: Climate Change, 4(3), 213–223.
  • Qian, Y., Jackson, C., Giorgi, F., Booth, B., Duan, Q., Forest, C., et al. (2016). Uncertainty Quantification in Climate Modeling and Projection. Bulletin of the American Meteorological Society, 97(5), 821–824.
  • Slingo, J., & Palmer, T. (2011). Uncertainty in weather and climate prediction. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 369(1956), 4751–4767.
  • Wunsch, C. (1999). The Interpretation of Short Climate Records, with Comments on the North Atlantic and Southern Oscillations. Bulletin of the American Meteorological Society, 80(2), 245–255.
  • Zanna, L., Brankart, J. M., Huber, M., Leroux, S., Penduff, T., & Williams, P. D. (2019). Uncertainty and scale interactions in ocean ensembles: From seasonal forecasts to multidecadal climate predictions. Quarterly Journal of the Royal Meteorological Society, 145(S1), 160–175.