No Access

Images of Gaussian and other stochastic processes under closed, densely-defined, unbounded linear operators

Tadashi Matsumoto

Mathematics Institute, University of Warwick, Coventry, CV4 7AL, United Kingdom

E-mail Address: [email protected]

Search for more papers by this author

and

T. J. Sullivan

https://orcid.org/0000-0003-4081-8301

Mathematics Institute and School of Engineering, University of Warwick, Coventry, CV4 7AL United Kingdom

Alan Turing Institute, 96 Euston Road, London, NW1 2DB, United Kingdom

E-mail Address: [email protected]

Search for more papers by this author

https://doi.org/10.1142/S0219530524400025Cited by:1 (Source: Crossref)

This article is part of the issue:

Special Issue on Interactions between Harmonic Analysis and Data Science
Editors: Hrushikesh N. Mharskar, Nicole Mücke and Ding-Xuan Zhou

Abstract

Gaussian Processes (GPs) are widely-used tools in spatial statistics and machine learning and the formulae for the mean function and covariance kernel of a GP $T u$ that is the image of another GP $u$ under a linear transformation $T$ acting on the sample paths of $u$ are well known, almost to the point of being folklore. However, these formulae are often used without rigorous attention to technical details, particularly when $T$ is an unbounded operator such as a differential operator, which is common in many modern applications. This note provides a self-contained proof of the claimed formulae for the case of a closed, densely-defined operator $T$ acting on the sample paths of a square-integrable (not necessarily Gaussian) stochastic process. Our proof technique relies upon Hille’s theorem for the Bochner integral of a Banach-valued random variable.

Keywords:

AMSC: 60G12, 60G15, 46G10, 47B01

References

1. C. E. Rasmussen and C. K. I. Williams , Gaussian Processes for Machine Learning (MIT Press, 2006). Google Scholar
2. A. Papoulis and S. U. Pillai , Probability, Random Variables and Stochastic Processes, 4th edn. (McGraw-Hill Education, 2002). Google Scholar
3. C. Jidling, N. Wahlström, A. Wills and T. B. Schön , Linearly constrained Gaussian processes, in Advances in Neural Information Processing Systems 30, eds. I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett (Curran Associates, Inc., 2017), pp. 1215–1224. Google Scholar
4. M. Lange-Hegermann , Algorithmic linearly constrained Gaussian processes, in Advances in Neural Information Processing Systems 31, eds. S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (Curran Associates, Inc., 2018), pp. 2137–2148. Google Scholar
5. M. Lange-Hegermann , Linearly constrained Gaussian processes with boundary conditions, in Proc. 24th Int. Conf. Artificial Intelligence and Statistics 2021, Proceedings of Machine Learning Research, Vol. 130, San Diego, California, 2021. Google Scholar
6. A. Besginow and M. Lange-Hegermann , Constraining Gaussian processes to systems of linear ordinary differential equations, in Advances in Neural Information Processing Systems 35, eds. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho and A. Oh , (Curran Associates, Inc., 2022), pp. 29386–29399. Google Scholar
7. G. Pang and G. Karniadakis , Physics-informed learning machines for partial differential equations: Gaussian processes versus neural networks, in Emerging Frontiers in Nonlinear Science, Nonlinear Systems and Complexity, Vol. 32 (Springer, 2020), pp. 323–343. Crossref, Google Scholar
8. M. Raissi, P. Perdikaris and G. Karniadakis , Numerical Gaussian processes for time-dependent and nonlinear partial differential equations, SIAM J. Sci. Comput. 40(1) (2018) A172–A198. Crossref, Web of Science, Google Scholar
9. J. Cockayne, C. Oates, T. J. Sullivan and M. Girolami , Probabilistic numerical methods for PDE-constrained Bayesian inverse problems, in Proc. 36th Int.Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering ed. G. Verdoolaege , AIP Conference Proceedings, Vol. 1853 (AIP Publishing, 2017), pp. 1–8. Crossref, Google Scholar
10. P. Hennig, M. A. Osborne and H. P. Kersting , Probabilistic Numerics: Computation as Machine Learning (Cambridge University Press, Cambridge, 2022). Crossref, Google Scholar
11. C. J. Oates, J. Cockayne, R. G. Aykroyd and M. Girolami , Bayesian probabilistic numerical methods in time-dependent state estimation for industrial hydrocyclone equipment, J. Amer. Statist. Assoc. 114(528) (2019) 1518–1531. Crossref, Web of Science, Google Scholar
12. M. Pförtner, I. Steinwart, P. Hennig and J. Wenger, Physics-informed Gaussian process regression generalizes linear PDE solvers, preprint (2022), 2212.12474. Google Scholar
13. S. Särkkä , Linear operators and stochastic partial differential equations in Gaussian process regression, in Artificial Neural Networks and Machine Learning, eds. T. Honkela, W. Duch, M. Girolami and S. Kaski , Lecture Notes in Computer Science, Vol. 6792 (Springer, Berlin, Heidelberg, 2011), pp. 151–159. Crossref, Google Scholar
14. M. Härkönen, M. Lange-Hegermann and B. Rajţă , Gaussian process priors for systems of linear partial differential equations with constant coefficients, in Proc. 40th Int. Conf. Machine Learning, Proceedings of Machine Learning Research, Vol. 202, Honolulu, Hawaii, USA, 2023, pp. 12587–12615. Google Scholar
15. R. R. Lin, H. Z. Zhang and J. Zhang , On reproducing kernel Banach spaces: Generic definitions and unified framework of constructions, Acta Math. Sin. 38(8) (2022) 1459–1483. Crossref, Google Scholar
16. R. A. Ryan , Introduction to Tensor Products of Banach Spaces, Springer Monographs in Mathematics (Springer-Verlag, London, 2002). Crossref, Google Scholar
17. V. I. Bogachev , Gaussian Measures, Mathematical Surveys and Monographs, Vol. 62 (American Mathematical Society, Providence, RI, 1998). Crossref, Google Scholar
18. T. Kato , Perturbation Theory for Linear Operators, Classics in Mathematics (Springer-Verlag, Berlin, 1995). Crossref, Google Scholar
19. M. Reed and B. Simon , Methods of Modern Mathematical Physics. I. Functional Analysis, 2nd edn. (Academic Press, New York, 1980). Google Scholar
20. J. Diestel and J. J. Uhl , Vector Measures, Mathematical Surveys, Vol. 15 (American Mathematical Society, Providence, 1977). Crossref, Google Scholar
21. E. Talvila , Some divergent trigonometric integrals, Amer. Math. Mon. 108(5) (2001) 432–436. Crossref, Web of Science, Google Scholar
22. E. Talvila , Necessary and sufficient conditions for differentiating under the integral sign, Amer. Math. Mon. 108(6) (2001) 544–548. Crossref, Web of Science, Google Scholar
23. J. Marcinkiewicz , Sur une propriété de la loi de Gauß, Math. Z. 44(1) (1939) 612–618. Crossref, Google Scholar
24. X. Fernique , Intégrabilité des vecteurs gaussiens, C. R. Acad. Sci. Paris Sér. A–B 270 (1970) A1698–A1699. Google Scholar
25. D. G. Krige, A statistical approach to some mine valuations and allied problems at the Witwatersrand, Master’s thesis, University of the Witwatersrand (1951). Google Scholar
26. G. Matheron , Principles of geostatistics, Econ. Geo. 58(8) (1963) 1246–1266. Crossref, Google Scholar
27. I. Cialenco, G. E. Fasshauer and Q. Ye , Approximation of stochastic partial differential equations by a kernel-based collocation method, Int. J. Comput. Math. 89(18) (2012) 2543–2561. Crossref, Web of Science, Google Scholar
28. G. E. Fasshauer , Solving differential equations with radial basis functions: Multilevel methods and smoothing, Adv. Comput. Math. 11(2–3) (1999) 139–159. Crossref, Web of Science, Google Scholar
29. J. Behrndt and M. Langer , Boundary value problems for elliptic partial differential operators on bounded domains, J. Funct. Anal. 243(2) (2007) 536–565. Crossref, Web of Science, Google Scholar
30. M. Scheuerer , Regularity of the sample paths of a general second order random field, Stoch. Process. Appl. 120(10) (2010) 1879–1897. Crossref, Web of Science, Google Scholar
31. S. Banach , Théorie des Opérations Linéaires (Monografje Matematyczne, Warszawa, 1932). Google Scholar
32. M. I. Ostrovskii , Weak $^{*}$ sequential closures in Banach space theory and their applications, in General Topology in Banach Spaces (Nova Science Publication, Huntington, New York, 2001), pp. 21–34. Google Scholar