Resources for uncertainty quantification for physical ocean variables

Guides, websites, and handbooks for best practices in quantifying uncertainty

Publications and reports

UQ in (physical) oceanography

  • Abraham, J. P., Baringer, M., Bindoff, N. L., Boyer, T., Cheng, L. J., Church, J. A., et al. (2013). A review of global ocean temperature observations: Implications for ocean heat content estimates and climate change. Reviews of Geophysics, 52(3), 450–483. http://doi.org/10.1002/rog.20022
  • Bos, M. S., Williams, S. D. P., Araujo, I. B., & Bastos, L. (2014). The effect of temporal correlated noise on the sea level rate and acceleration uncertainty. Geophysical Journal International, 196(3), 1423–1430. http://doi.org/10.1093/gji/ggt481
  • Chelton, D. B. (1994). Physical Oceanography: A Brief Overview for Statisticians. Statistical Science, 9(2), 150–166.
  • Chelton, D. B. (1983). Effects of sampling errors in statistical estimation. Deep Sea Research Part a. Oceanographic Research Papers, 30(10), 1083–1103.
  • Forget, G., & Wunsch, C. (2007). Estimated Global Hydrographic Variability. Journal of Physical Oceanography, 37(8), 1997–2008. http://doi.org/10.1175/JPO3072.1
  • Kennedy, J. J. (2014). A review of uncertainty in in situ measurements and data sets of sea surface temperature. Reviews of Geophysics, 52(1), 1–32. http://doi.org/10.1002/2013RG000434
  • Kuusela, M. and M. L. Stein. Locally stationary spatio-temporal interpolation of Argo profiling float data. Proceedings of the Royal Society A 474:20180400, 2018.
  • Matthews, J. L., Mannshardt, E., & Gremaud, P. (2013). Uncertainty Quantification for Climate Observations. Bulletin of the American Meteorological Society, 94(3), ES21–ES25. http://doi.org/10.1175/BAMS-D-12-00042.1
  • Merchant, C.J., Paul, F., Popp, T., Ablain, M., Bontemps, S., Defourny, P., Hollmann, R., Lavergne, T., Laeng, A., De Leeuw, G. and Mittaz, J., 2017. Uncertainty information in climate data records from Earth observation. Earth System Science Data, 9(2), pp.511-527.
  • Moroni, David F.; Ramapriyan, Hampapuram; Peng, Ge; Hobbs, Jonathan; Goldstein, Justin; Downs, Robert; et al. (2019): Understanding the Various Perspectives of Earth Science Observational Data Uncertainty. ESIP. Report. https://doi.org/10.6084/m9.figshare.10271450.v1
  • Liu, C., X. Liang, D. P. Chambers, and R. M. Ponte, Global Patterns of Spatial and Temporal Variability in Salinity from Multiple Gridded Argo Products. J. Climate, doi: https://doi.org/10.1175/JCLI-D-20-0053.1.
  • National Research Council. 1993. Statistics and Physical Oceanography. Washington, DC: The National Academies Press. https://doi.org/10.17226/9028.
  • Ramapriyan, H K, Peng G, Moroni D, Shie C-L, Ensuring and Improving Information Quality for Earth Science Data and Products. D-Lib Magazine, 23 (7/8), July/August 2017, DOI:https://doi.org/10.1045/july2017-ramapriyan
  • Sutton, A.J., Sabine, C.L., Maenner-Jones, S., Lawrence-Slavas, N., Meinig, C., Feely, R.A., Mathis, J.T., Musielewicz, S., Bott, R., McLain, P.D. and Fought, H.J., 2014. A high-frequency atmospheric and seawater pCO2 data set from 14 open-ocean sites using a moored autonomous system. Earth Syst. Sci. Data, 6(2), pp.353-366.
  • Timms GP, de Souza PA Jr, Reznik L, Smith DV. Automated data quality assessment of marine sensors. Sensors (Basel). 2011;11(10):9589-9602. doi:10.3390/s111009589
  • UNESCO/IOC. 2020. Quality Control of in situ Sea Level Observations: A Review and Progress towards Automated Quality Control, Vol. 1. Paris, UNESCO. IOC Manuals and Guides No. 83. (IOC/2020/MG/83Vol.1), https://unesdoc.unesco.org/ark:/48223/pf0000373566
  • Wunsch, C. (2018). Towards determining uncertainties in global oceanic mean values of heat, salt, and surface elevation. Tellus a: Dynamic Meteorology and Oceanography, 70(1), 1–14. http://doi.org/10.1080/16000870.2018.1471911

UQ in science & UQ communication

  • Buchanan, M. (2020). The certainty of uncertainty. Nature Physics, 16(2), 120–120. http://doi.org/10.1038/s41567-020-0786-z
  • Carslaw, K., Lee, L., Regayre, L., & Johnson, J. (2018). Climate Models Are Uncertain, but We Can Do Something About It. Eos, Transactions American Geophysical Union, 99, 1–6. http://doi.org/10.1029/2018EO093757
  • Gibbs, P. (2013). Making sense of uncertainty (pp. 1–28). London: Sense about Science.
  • Ho, E. H., & Budescu, D. V. (2019). Climate uncertainty communication. Nature Climate Change, 9(11), 802–803. http://doi.org/10.1038/s41558-019-0606-6
  • Hogan Carr, R., Montz, B., Maxfield, K., Hoekstra, S., Semmens, K., & Goldman, E. (2016). Effectively Communicating Risk and Uncertainty to the Public: Assessing the National Weather Service's Flood Forecast and Warning Tools. Bulletin of the American Meteorological Society, 97(9), 1649–1665. http://doi.org/10.1175/BAMS-D-14-00248.1
  • Howe, L. C., MacInnis, B., Krosnick, J. A., Markowitz, E. M., & Socolow, R. (2019). Acknowledging uncertainty impacts public acceptance of climate scientists' predictions. Nature Climate Change, 9(11), 863–867. http://doi.org/10.1038/s41558-019-0587-5
  • National Research Council. (2013). Environmental Decisions in the Face of Uncertainty. (F. A. Sloan, Ed.) (Vol. 12568, pp. 1–228). National Academies Press. Retrieved from http://www.nap.edu/catalog.php?record_id=12568
  • Oppenheimer, M., Little, C. M., & Cooke, R. M. (2016). Expert judgement and uncertainty quantification for climate change. Nature Geoscience, 6(5), 445–451. http://doi.org/10.1038/nclimate2959
  • Oreskes, N. (2015). The fact of uncertainty, the uncertainty of facts and the cultural resonance of doubt. Philosophical Transactions of the Royal Society of London. Series a: Physical and Engineering Sciences, 373(2055), 20140455–21. http://doi.org/10.1098/rsta.2014.0455
  • Palmer, T. N., & Hardaker, P. J. (2011). Handling uncertainty in science. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 369(1956), 4681–4684. http://doi.org/10.1098/rsta.2011.0280
  • Spiegelhalter, D. (2017). Risk and Uncertainty Communication. Annual Review of Statistics and Its Application, 4(1), 31–60. http://doi.org/10.1146/annurev-statistics-010814-020148
  • Spiegelhalter, D. J. (2014). The future lies in uncertainty. Science, 345(6194), 264–265. http://doi.org/10.1126/science.1251122

Statistical & stochastic representation of UQ

  • Berger, J. O., & Smith, L. A. (2019). On the Statistical Formalism of Uncertainty Quantification. Annual Review of Statistics and Its Application, 6(1), 433–460. http://doi.org/10.1146/annurev-statistics-030718-105232
  • Chelton, D. B., Eddy, W. F., & DeVeaux, R. (1994). Report on Statistics and Physical Oceanography. Statistical Science, 9(2), 167–221.
  • Coveney, P. V., Dougherty, E. R., & Highfield, R. R. (2016). Big data need big theory too. Philosophical Transactions of the Royal Society of London. Series a: Physical and Engineering Sciences, 374(2080), 20160153–11. http://doi.org/10.1098/rsta.2016.0153
  • Pitman, B. E. (2020, March 2). Model Uncertainty: Mathematical and Statistical. Retrieved March 13, 2020, from https://sinews.siam.org/Details-Page/model-uncertainty-mathematical-and-statistical
  • Williamson, D. B., & Sansom, P. G. (2019). How Are Emergent Constraints Quantifying Uncertainty and What Do They Leave Behind? Bulletin of the American Meteorological Society, 100(12), 2571–2588. http://doi.org/10.1175/BAMS-D-19-0131.1
  • Arnold, H. M., Moroz, I. M., & Palmer, T. N. (2013). Stochastic parametrizations and model uncertainty in the Lorenz '96 system. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 371(1991), 20110479–20110479. http://doi.org/10.1038/nature03301
  • Subramanian, A., Juricke, S., Dueben, P., & Palmer, T. (2019). A Stochastic Representation of Subgrid Uncertainty for Dynamical Core Development. Bulletin of the American Meteorological Society, 100(6), 1091–1101. http://doi.org/10.1175/BAMS-D-17-0040.1

UQ in (Bayesian) inverse modeling & data assimilation

  • Bui-Thanh, T., Burstedde, C., Ghattas, O., Martin, J., Stadler, G., & Wilcox, L. C. (2012). Extreme-scale UQ for Bayesian inverse problems governed by PDEs. IEEE Computer Society Press.
  • Challenor, P. (2012). Using Emulators to Estimate Uncertainty in Complex Models. (A. Dienstfrey & R. F. Boisvert, Eds.) (pp. 1–14). IFIP International Federation for Information Processing.
  • Chen, P., Villa, U., & Ghattas, O. (2019). Taylor approximation and variance reduction for PDE-constrained optimal control under uncertainty. Journal of Computational Physics, 385, 163–186. http://doi.org/10.1016/j.jcp.2019.01.047
  • Flath, H. P., Wilcox, L. C., Akçelik, V., Hill, J., van Bloemen Waanders, B., & Ghattas, O. (2011). Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations. SIAM Journal on Scientific Computing, 33(1), 407–432. http://doi.org/10.1137/090780717
  • Gregg, W. W., Friedrichs, M. A. M., Robinson, A. R., Rose, K. A., Schlitzer, R., Thompson, K. R., & Doney, S. C. (2009). Skill assessment in ocean biological data assimilation. Journal of Marine Systems, 76(1-2), 16–33. http://doi.org/10.1016/j.jmarsys.2008.05.006
  • Isaac, T., Petra, N., Stadler, G., & Ghattas, O. (2015). Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet. Journal of Computational Physics, 296(C), 348–368. http://doi.org/10.1016/j.jcp.2015.04.047
  • Janjić, T., Bormann, N., Bocquet, M., Carton, J. A., Cohn, S. E., Dance, S. L., et al. (2017). On the representation error in data assimilation. Quarterly Journal of the Royal Meteorological Society, 144(713), 1257–1278. http://doi.org/10.1002/qj.3130
  • Kalmikov, A. G., & Heimbach, P. (2014). A Hessian-Based Method for Uncertainty Quantification in Global Ocean State Estimation. SIAM Journal on Scientific Computing, 36(5), S267–S295. http://doi.org/10.1137/130925311
  • Kaminski, T., Kauker, F., Toudal Pedersen, L., Vossbeck, M., Haak, H., Niederdrenk, L., et al. (2018). Arctic Mission Benefit Analysis: impact of sea ice thickness, freeboard, and snow depth products on sea ice forecast performance. The Cryosphere, 12(8), 2569–2594. http://doi.org/10.5194/tc-12-2569-2018
  • Mohammadi, B. (2015). Backward uncertainty propagation in shape optimization. International Journal for Numerical Methods in Fluids, 80(5), 285–305. http://doi.org/10.1002/fld.4077
  • Moser, R. D., & Oliver, T. A. (2015). Validation of Physical Models in the Presence of Uncertainty. Handbook of Uncertainty Quantification (pp. 1–28). Cham: Springer International Publishing. http://doi.org/10.1007/978-3-319-11259-6_2-1
  • Oden, T., Moser, R., & Ghattas, O. (2010). Computer predictions with quantified uncertainty, Part I. SIAM News, 43(9), 1–3.
  • Rougier, J. (2013). "Intractable and unsolved": some thoughts on statistical data assimilation with uncertain static parameters. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 371(1991), 20120297–20120297. http://doi.org/10.1098/rsif.2008.0172
  • Roy, C. J., & Oberkampf, W. L. (2011). A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Computer Methods in Applied Mechanics and Engineering, 200(25-28), 2131–2144. http://doi.org/10.1016/j.cma.2011.03.016
  • Saibaba, A. K., & Kitanidis, P. K. (2015). Fast computation of uncertainty quantification measures in the geostatistical approach to solve inverse problems. Advances in Water Resources, 82(C), 124–138. http://doi.org/10.1016/j.advwatres.2015.04.012
  • Salter, J. M., B, W. D., John, S., & Viatcheslav, K. (2019). Uncertainty Quantification for Computer Models With Spatial Output Using Calibration-Optimal Bases. Journal of the American Statistical Association, 0(0), 1–24. http://doi.org/10.1080/01621459.2018.1514306

UQ in climate science/prediction

  • Curry, J. A., & Webster, P. J. (2011). Climate Science and the Uncertainty Monster. Bulletin of the American Meteorological Society, 92(12), 1667–1682. http://doi.org/10.1175/2011BAMS3139.1
  • Hawkins, E., Smith, R. S., Gregory, J. M., & Stainforth, D. A. (2015). Irreducible uncertainty in near-term climate projections. Climate Dynamics, 1–13. http://doi.org/10.1007/s00382-015-2806-8
  • Katz, R. W., Craigmile, P. F., Guttorp, P., Haran, M., Sansó, B., & Stein, M. L. (2013). Uncertainty analysis in climate change assessments. Nature Climate Change, 3(9), 769–771. http://doi.org/10.1038/nclimate1980
  • Marotzke, J. (2018). Quantifying the irreducible uncertainty in near‐term climate projections. Wiley Interdisciplinary Reviews: Climate Change, 10(1), e563. http://doi.org/10.1002/wcc.563
  • Parker, W. S. (2013). Ensemble modeling, uncertainty and robust predictions. Wiley Interdisciplinary Reviews: Climate Change, 4(3), 213–223. http://doi.org/10.1002/wcc.220
  • Qian, Y., Jackson, C., Giorgi, F., Booth, B., Duan, Q., Forest, C., et al. (2016). Uncertainty Quantification in Climate Modeling and Projection. Bulletin of the American Meteorological Society, 97(5), 821–824. http://doi.org/10.1175/BAMS-D-15-00297.1
  • Slingo, J., & Palmer, T. (2011). Uncertainty in weather and climate prediction. Philosophical Transactions of the Royal Society a: Mathematical, Physical and Engineering Sciences, 369(1956), 4751–4767. http://doi.org/10.1038/nature03301
  • Wunsch, C. (1999). The Interpretation of Short Climate Records, with Comments on the North Atlantic and Southern Oscillations. Bulletin of the American Meteorological Society, 80(2), 245–255.
  • Zanna, L., Brankart, J. M., Huber, M., Leroux, S., Penduff, T., & Williams, P. D. (2019). Uncertainty and scale interactions in ocean ensembles: From seasonal forecasts to multidecadal climate predictions. Quarterly Journal of the Royal Meteorological Society, 145(S1), 160–175. http://doi.org/10.1002/qj.3397