From Euclidean to Hilbert Spaces
Introduction to Functional Analysis and its Applications
1. Edition September 2021
368 Pages, Hardcover
Wiley & Sons Ltd
ISBN:
978-1-78630-682-1
John Wiley & Sons
Further versions
From Euclidian to Hilbert Spaces analyzes the transition from finite dimensional Euclidian spaces to infinite-dimensional Hilbert spaces, a notion that can sometimes be difficult for non-specialists to grasp. The focus is on the parallels and differences between the properties of the finite and infinite dimensions, noting the fundamental importance of coherence between the algebraic and topological structure, which makes Hilbert spaces the infinite-dimensional objects most closely related to Euclidian spaces.
The common thread of this book is the Fourier transform, which is examined starting from the discrete Fourier transform (DFT), along with its applications in signal and image processing, passing through the Fourier series and finishing with the use of the Fourier transform to solve differential equations.
The geometric structure of Hilbert spaces and the most significant properties of bounded linear operators in these spaces are also covered extensively. The theorems are presented with detailed proofs as well as meticulously explained exercises and solutions, with the aim of illustrating the variety of applications of the theoretical results.
Preface xi
Chapter 1. Inner Product Spaces (Pre-Hilbert) 1
1.1. Real and complex inner products 1
1.2. The norm associated with an inner product and normed vector spaces 6
1.2.1. The parallelogram law and the polarization formula 9
1.3. Orthogonal and orthonormal families in inner product spaces 11
1.4. Generalized Pythagorean theorem 11
1.5. Orthogonality and linear independence 13
1.6. Orthogonal projection in inner product spaces 15
1.7. Existence of an orthonormal basis: the Gram-Schmidt process 19
1.8. Fundamental properties of orthonormal and orthogonal bases 20
1.9. Summary 28
Chapter 2. The Discrete Fourier Transform and its Applications to Signal and Image Processing 31
2.1. The space l²(ZN) and its canonical basis 31
2.1.1. The orthogonal basis of complex exponentials in l²(ZN) 34
2.2. The orthonormal Fourier basis of l²(ZN) 38
2.3. The orthogonal Fourier basis of l²(ZN) 40
2.4. Fourier coefficients and the discrete Fourier transform 41
2.4.1. The inverse discrete Fourier transform 44
2.4.2. Definition of the DFT and the IDFT with the orthonormal Fourier basis 46
2.4.3. The real (orthonormal) Fourier basis 47
2.5. Matrix interpretation of the DFT and the IDFT 48
2.5.1. The fast Fourier transform 51
2.6. The Fourier transform in signal processing 51
2.6.1. Synthesis formula for 1D signals: decomposition on the harmonic basis 51
2.6.2. Signification of Fourier coefficients and spectrums of a 1D signal 53
2.6.3. The synthesis formula and Fourier coefficients of the unit pulse 54
2.6.4. High and low frequencies in the synthesis formula 55
2.6.5. Signal filtering in frequency representation 59
2.6.6. The multiplication operator and its diagonal matrix representation 60
2.6.7. The Fourier multiplier operator 60
2.7. Properties of the DFT 61
2.7.1. Periodicity of Z and ? 62
2.7.2. DFT and shift 63
2.7.3. DFT and conjugation 67
2.7.4. DFT and convolution 68
2.8. The DFT and stationary operators 73
2.8.1. The DFT and the diagonalization of stationary operators 75
2.8.2. Circulant matrices 77
2.8.3. Exhaustive characterization of stationary operators 78
2.8.4. High-pass, low-pass and band-pass filters 82
2.8.5. Characterizing stationary operators using shift operators 83
2.8.6. Frequency analysis of first and second derivation operators (discrete case) 84
2.9. The two-dimensional discrete Fourier transform (2D DFT) 88
2.9.1. Matrix representation of the 2D DFT: Kronecker product versus iteration of two 1D DFTs 91
2.9.2. Properties of the 2D DFT 93
2.9.3. 2D DFT and stationary operators 95
2.9.4. Gradient and Laplace operators and their action on digital images 97
2.9.5. Visualization of the amplitude spectrum in 2D 97
2.9.6. Filtering: an example of digital image filtering in a Fourier space 100
2.10. Summary 102
Chapter 3. Lebesgue's Measure and Integration Theory 105
3.1. Riemann versus Lebesgue 105
3.2. sigma-algebra, measurable space, measures and measured spaces 106
3.3. Measurable functions and almost-everywhere properties (a.e) 108
3.4. Integrable functions and Lebesgue integrals 109
3.5. Characterization of the Lebesgue measure on R and sets with a null Lebesgue measure 111
3.6. Three theorems for limit operations in integration theory 113
3.7. Summary 114
Chapter 4. Banach Spaces and Hilbert Spaces 115
4.1. Metric topology of inner product spaces 116
4.2. Continuity of fundamental operations in inner product spaces 120
4.2.1. Equivalence of separated topologies in finite-dimension vector spaces 128
4.3. Cauchy sequences and completeness: Banach and Hilbert 129
4.3.1. Completeness of vector spaces 133
4.3.2. Characterizing the completeness of normed vector spaces using series 135
4.3.3. Banach fixed-point theorem 139
4.4. Remarkable examples of Banach and Hilbert spaces 145
4.4.2. L¯oo and l¯oo spaces 156
4.4.3. Inclusion relationships between l¯p spaces 161
4.4.4. Inclusion relationships between L¯p spaces 163
4.4.5. Density theorems in L¯p(X,A,mu) 165
4.5. Summary 169
Chapter 5. The Geometric Structure of Hilbert Spaces 171
5.1. The orthogonal complement in a Hilbert space and its properties 171
5.2. Projection onto closed convex sets: theorem and consequences 174
5.2.1. Characterization of closed vector subspaces in Hilbert spaces 180
5.3. Polar and bipolar subsets of a Hilbert space 182
5.4. The (orthogonal) projection theorem in a Hilbert space 185
5.5. Orthonormal systems and Hilbert bases 188
5.5.1. Bessel's inequality and Fourier coefficients 189
5.5.2. The Fischer-Riesz theorem 192
5.5.3. Characterizations of a Hilbert basis (or complete orthonormal system) 194
5.5.4. Isomorphisms between Hilbert spaces 199
5.5.5. l²(N,K) as the prototype of separable Hilbert spaces of infinite dimension 201
5.6. The Fourier Hilbert basis in L¯2 202
5.6.1. L²[-pi, pi] or L²[0, 2pi] 202
5.6.2. L²(T) 204
5.6.3. L²[a, b] 205
5.6.4. Real Fourier series 206
5.6.5. Pointwise convergence of the real Fourier series: Dirichlet's theorem 212
5.6.6. The Gibbs phenomenon and Cesàro summation 214
5.6.7. Speed of convergence to 0 of Fourier coefficients 214
5.6.8. Fourier transform in L²(T) and shift 218
5.7. Summary 219
Chapter 6. Bounded Linear Operators in Hilbert Spaces 221
6.1. Fundamental properties of bounded linear operators between normed vector spaces 223
6.1.1. Continuity of linear operators defined on a finite-dimensional normed vector space 226
6.2. The operator norm, convergence of operator sequences and Banach algebras 227
6.2.1. A classical example of a non-bounded linear operator on a vector space of infinite dimension 238
6.3. Invertibility of linear operators 239
6.4. The dual of a Hilbert space and the Riesz representation theorem 244
6.4.1. The scalar product induced on the dual of a Hilbert space 249
6.5. Bilinear forms, sesquilinear forms and associated quadratic forms 249
6.5.1. The Lax-Milgram theorem and its consequences 257
6.6. The adjoint operator: presentation and properties 261
6.7. Orthogonal projection operators in a Hilbert space 269
6.7.1. Bounded multiplication operators and their relation to orthogonal projectors 278
6.7.2. Geometric realization of orthogonal projection operators via orthonormal systems 280
6.8. Isometric and unitary operators 286
6.8.1. Characterizations of isometric and unitary operators 288
6.8.2. Relationship between isometric and unitary operators and orthonormal systems 293
6.9. The Fourier transform on S(R¯n), L¹(R¯n) and L²(R¯n) 296
6.9.1. The invariance of the Schwartz space with respect to the Fourier transform 296
6.9.2. Extension of the Fourier transform of S(R¯n) to L¹(R¯n): the Riemann-Lebesgue theorem 301
6.9.3. Extension of the Fourier transform to a unitary operator on L²(R¯n): the Fourier-Plancherel transform 302
6.9.4. Relationship between the Fourier-Plancherel transform and the Hermitian Hilbert basis 305
6.9.5. The Fourier transform and convolution 306
6.9.6. Convolution and Fourier transforms in L²: localization of the Fourier transform 309
6.10. The Nyquist-Shannon sampling theorem 310
6.10.1. The Nyquist frequency: aliasing and oversampling 312
6.11. Application of the Fourier transform to solve ordinary and partial differential equations 313
6.11.1. Solving an ordinary differential equation using the Fourier transform 313
6.11.2. The Fourier transform and partial differential equations 315
6.11.3. Solving the partial differential equation for heat propagation using the Fourier transform 316
6.12. Summary 319
Appendix 1: Quotient Space 323
Appendix 2: The Transpose (or Dual) of a Linear Operator 329
Appendix 3: Uniform, Strong and Weak Convergence 331
References 335
Index 337
Chapter 1. Inner Product Spaces (Pre-Hilbert) 1
1.1. Real and complex inner products 1
1.2. The norm associated with an inner product and normed vector spaces 6
1.2.1. The parallelogram law and the polarization formula 9
1.3. Orthogonal and orthonormal families in inner product spaces 11
1.4. Generalized Pythagorean theorem 11
1.5. Orthogonality and linear independence 13
1.6. Orthogonal projection in inner product spaces 15
1.7. Existence of an orthonormal basis: the Gram-Schmidt process 19
1.8. Fundamental properties of orthonormal and orthogonal bases 20
1.9. Summary 28
Chapter 2. The Discrete Fourier Transform and its Applications to Signal and Image Processing 31
2.1. The space l²(ZN) and its canonical basis 31
2.1.1. The orthogonal basis of complex exponentials in l²(ZN) 34
2.2. The orthonormal Fourier basis of l²(ZN) 38
2.3. The orthogonal Fourier basis of l²(ZN) 40
2.4. Fourier coefficients and the discrete Fourier transform 41
2.4.1. The inverse discrete Fourier transform 44
2.4.2. Definition of the DFT and the IDFT with the orthonormal Fourier basis 46
2.4.3. The real (orthonormal) Fourier basis 47
2.5. Matrix interpretation of the DFT and the IDFT 48
2.5.1. The fast Fourier transform 51
2.6. The Fourier transform in signal processing 51
2.6.1. Synthesis formula for 1D signals: decomposition on the harmonic basis 51
2.6.2. Signification of Fourier coefficients and spectrums of a 1D signal 53
2.6.3. The synthesis formula and Fourier coefficients of the unit pulse 54
2.6.4. High and low frequencies in the synthesis formula 55
2.6.5. Signal filtering in frequency representation 59
2.6.6. The multiplication operator and its diagonal matrix representation 60
2.6.7. The Fourier multiplier operator 60
2.7. Properties of the DFT 61
2.7.1. Periodicity of Z and ? 62
2.7.2. DFT and shift 63
2.7.3. DFT and conjugation 67
2.7.4. DFT and convolution 68
2.8. The DFT and stationary operators 73
2.8.1. The DFT and the diagonalization of stationary operators 75
2.8.2. Circulant matrices 77
2.8.3. Exhaustive characterization of stationary operators 78
2.8.4. High-pass, low-pass and band-pass filters 82
2.8.5. Characterizing stationary operators using shift operators 83
2.8.6. Frequency analysis of first and second derivation operators (discrete case) 84
2.9. The two-dimensional discrete Fourier transform (2D DFT) 88
2.9.1. Matrix representation of the 2D DFT: Kronecker product versus iteration of two 1D DFTs 91
2.9.2. Properties of the 2D DFT 93
2.9.3. 2D DFT and stationary operators 95
2.9.4. Gradient and Laplace operators and their action on digital images 97
2.9.5. Visualization of the amplitude spectrum in 2D 97
2.9.6. Filtering: an example of digital image filtering in a Fourier space 100
2.10. Summary 102
Chapter 3. Lebesgue's Measure and Integration Theory 105
3.1. Riemann versus Lebesgue 105
3.2. sigma-algebra, measurable space, measures and measured spaces 106
3.3. Measurable functions and almost-everywhere properties (a.e) 108
3.4. Integrable functions and Lebesgue integrals 109
3.5. Characterization of the Lebesgue measure on R and sets with a null Lebesgue measure 111
3.6. Three theorems for limit operations in integration theory 113
3.7. Summary 114
Chapter 4. Banach Spaces and Hilbert Spaces 115
4.1. Metric topology of inner product spaces 116
4.2. Continuity of fundamental operations in inner product spaces 120
4.2.1. Equivalence of separated topologies in finite-dimension vector spaces 128
4.3. Cauchy sequences and completeness: Banach and Hilbert 129
4.3.1. Completeness of vector spaces 133
4.3.2. Characterizing the completeness of normed vector spaces using series 135
4.3.3. Banach fixed-point theorem 139
4.4. Remarkable examples of Banach and Hilbert spaces 145
4.4.2. L¯oo and l¯oo spaces 156
4.4.3. Inclusion relationships between l¯p spaces 161
4.4.4. Inclusion relationships between L¯p spaces 163
4.4.5. Density theorems in L¯p(X,A,mu) 165
4.5. Summary 169
Chapter 5. The Geometric Structure of Hilbert Spaces 171
5.1. The orthogonal complement in a Hilbert space and its properties 171
5.2. Projection onto closed convex sets: theorem and consequences 174
5.2.1. Characterization of closed vector subspaces in Hilbert spaces 180
5.3. Polar and bipolar subsets of a Hilbert space 182
5.4. The (orthogonal) projection theorem in a Hilbert space 185
5.5. Orthonormal systems and Hilbert bases 188
5.5.1. Bessel's inequality and Fourier coefficients 189
5.5.2. The Fischer-Riesz theorem 192
5.5.3. Characterizations of a Hilbert basis (or complete orthonormal system) 194
5.5.4. Isomorphisms between Hilbert spaces 199
5.5.5. l²(N,K) as the prototype of separable Hilbert spaces of infinite dimension 201
5.6. The Fourier Hilbert basis in L¯2 202
5.6.1. L²[-pi, pi] or L²[0, 2pi] 202
5.6.2. L²(T) 204
5.6.3. L²[a, b] 205
5.6.4. Real Fourier series 206
5.6.5. Pointwise convergence of the real Fourier series: Dirichlet's theorem 212
5.6.6. The Gibbs phenomenon and Cesàro summation 214
5.6.7. Speed of convergence to 0 of Fourier coefficients 214
5.6.8. Fourier transform in L²(T) and shift 218
5.7. Summary 219
Chapter 6. Bounded Linear Operators in Hilbert Spaces 221
6.1. Fundamental properties of bounded linear operators between normed vector spaces 223
6.1.1. Continuity of linear operators defined on a finite-dimensional normed vector space 226
6.2. The operator norm, convergence of operator sequences and Banach algebras 227
6.2.1. A classical example of a non-bounded linear operator on a vector space of infinite dimension 238
6.3. Invertibility of linear operators 239
6.4. The dual of a Hilbert space and the Riesz representation theorem 244
6.4.1. The scalar product induced on the dual of a Hilbert space 249
6.5. Bilinear forms, sesquilinear forms and associated quadratic forms 249
6.5.1. The Lax-Milgram theorem and its consequences 257
6.6. The adjoint operator: presentation and properties 261
6.7. Orthogonal projection operators in a Hilbert space 269
6.7.1. Bounded multiplication operators and their relation to orthogonal projectors 278
6.7.2. Geometric realization of orthogonal projection operators via orthonormal systems 280
6.8. Isometric and unitary operators 286
6.8.1. Characterizations of isometric and unitary operators 288
6.8.2. Relationship between isometric and unitary operators and orthonormal systems 293
6.9. The Fourier transform on S(R¯n), L¹(R¯n) and L²(R¯n) 296
6.9.1. The invariance of the Schwartz space with respect to the Fourier transform 296
6.9.2. Extension of the Fourier transform of S(R¯n) to L¹(R¯n): the Riemann-Lebesgue theorem 301
6.9.3. Extension of the Fourier transform to a unitary operator on L²(R¯n): the Fourier-Plancherel transform 302
6.9.4. Relationship between the Fourier-Plancherel transform and the Hermitian Hilbert basis 305
6.9.5. The Fourier transform and convolution 306
6.9.6. Convolution and Fourier transforms in L²: localization of the Fourier transform 309
6.10. The Nyquist-Shannon sampling theorem 310
6.10.1. The Nyquist frequency: aliasing and oversampling 312
6.11. Application of the Fourier transform to solve ordinary and partial differential equations 313
6.11.1. Solving an ordinary differential equation using the Fourier transform 313
6.11.2. The Fourier transform and partial differential equations 315
6.11.3. Solving the partial differential equation for heat propagation using the Fourier transform 316
6.12. Summary 319
Appendix 1: Quotient Space 323
Appendix 2: The Transpose (or Dual) of a Linear Operator 329
Appendix 3: Uniform, Strong and Weak Convergence 331
References 335
Index 337
Edoardo Provenzi is Professor of Mathematics at the University of Bordeaux, France. He studies visual phenomena and their applications in image processing and computer vision, employing tools from differential geometry, harmonic analysis and mathematical physics.
