Schur product theorem

In mathematics, particularly in linear algebra, the Schur product theorem states that the Hadamard product of two positive definite matrices is also a positive definite matrix. The result is named after Issai Schur^[1] (Schur 1911, p. 14, Theorem VII) (note that Schur signed as J. Schur in Journal für die reine und angewandte Mathematik.^[2]^[3])

Proof

Proof using the trace formula

For any matrices $M$ and $N$ , the Hadamard product $M\circ N$ considered as a bilinear form acts on vectors $a,b$ as

a^{*}(M\circ N)b=\operatorname {tr} (M^{T}\operatorname {diag} (a^{*})N\operatorname {diag} (b))

where $\operatorname {tr}$ is the matrix trace and $\operatorname {diag}(a)$ is the diagonal matrix having as diagonal entries the elements of $a$ .

Suppose $M$ and $N$ are positive definite, and so Hermitian. We can consider their square-roots $M^{{1/2}}$ and $N^{{1/2}}$ , which are also Hermitian, and write

\operatorname {tr} (M^{T}\operatorname {diag} (a^{*})N\operatorname {diag} (b))=\operatorname {tr} ({\overline {M}}^{1/2}{\overline {M}}^{1/2}\operatorname {diag} (a^{*})N^{1/2}N^{1/2}\operatorname {diag} (b))=\operatorname {tr} ({\overline {M}}^{1/2}\operatorname {diag} (a^{*})N^{1/2}N^{1/2}\operatorname {diag} (b){\overline {M}}^{1/2})

Then, for $a=b$ , this is written as $\operatorname {tr} (A^{*}A)$ for $A=N^{1/2}\operatorname {diag} (a){\overline {M}}^{1/2}$ and thus is strictly positive for $A\neq 0$ , which occurs if and only if $a\neq 0$ . This shows that $(M\circ N)$ is a positive definite matrix.

Proof using Gaussian integration

Case of M = N

Let $X$ be an $n$ -dimensional centered Gaussian random variable with covariance $\langle X_{i}X_{j}\rangle =M_{{ij}}$ . Then the covariance matrix of $X_{i}^{2}$ and $X_{j}^{2}$ is

\operatorname {Cov}(X_{i}^{2},X_{j}^{2})=\langle X_{i}^{2}X_{j}^{2}\rangle -\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle

Using Wick's theorem to develop $\langle X_{i}^{2}X_{j}^{2}\rangle =2\langle X_{i}X_{j}\rangle ^{2}+\langle X_{i}^{2}\rangle \langle X_{j}^{2}\rangle$ we have

\operatorname {Cov}(X_{i}^{2},X_{j}^{2})=2\langle X_{i}X_{j}\rangle ^{2}=2M_{{ij}}^{2}

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{{ij}}^{2}$ is a positive definite matrix.

General case

Let $X$ and $Y$ be $n$ -dimensional centered Gaussian random variables with covariances $\langle X_{i}X_{j}\rangle =M_{{ij}}$ , $\langle Y_{i}Y_{j}\rangle =N_{{ij}}$ and independent from each other so that we have

\langle X_{i}Y_{j}\rangle =0

for any

i,j

Then the covariance matrix of $X_{i}Y_{i}$ and $X_{j}Y_{j}$ is

\operatorname {Cov}(X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}Y_{i}X_{j}Y_{j}\rangle -\langle X_{i}Y_{i}\rangle \langle X_{j}Y_{j}\rangle

Using Wick's theorem to develop

\langle X_{i}Y_{i}X_{j}Y_{j}\rangle =\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle +\langle X_{i}Y_{i}\rangle \langle X_{j}Y_{j}\rangle +\langle X_{i}Y_{j}\rangle \langle X_{j}Y_{i}\rangle

and also using the independence of $X$ and $Y$ , we have

\operatorname {Cov}(X_{i}Y_{i},X_{j}Y_{j})=\langle X_{i}X_{j}\rangle \langle Y_{i}Y_{j}\rangle =M_{{ij}}N_{{ij}}

Since a covariance matrix is positive definite, this proves that the matrix with elements $M_{{ij}}N_{{ij}}$ is a positive definite matrix.

Proof using eigendecomposition

Proof of positive semidefiniteness

Let $M=\sum \mu _{i}m_{i}m_{i}^{T}$ and $N=\sum \nu _{i}n_{i}n_{i}^{T}$ . Then

M\circ N=\sum _{{ij}}\mu _{i}\nu _{j}(m_{i}m_{i}^{T})\circ (n_{j}n_{j}^{T})=\sum _{{ij}}\mu _{i}\nu _{j}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}

Each $(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}$ is positive semidefinite (but, except in the 1-dimensional case, not positive definite, since they are rank 1 matrices). Also, $\mu _{i}\nu _{j}>0$ thus the sum $M\circ N$ is also positive semidefinite.

Proof of definiteness

To show that the result is positive definite requires further proof. We shall show that for any vector $a\neq 0$ , we have $a^{T}(M\circ N)a>0$ . Continuing as above, each $a^{T}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}a\geq 0$ , so it remains to show that there exist $i$ and $j$ for which the inequality is strict. For this we observe that

a^{T}(m_{i}\circ n_{j})(m_{i}\circ n_{j})^{T}a=\left(\sum _{k}m_{{i,k}}n_{{j,k}}a_{k}\right)^{2}

Since $N$ is positive definite, there is a $j$ for which $n_{{j,k}}a_{k}$ is not 0 for all $k$ , and then, since $M$ is positive definite, there is an $i$ for which $m_{{i,k}}n_{{j,k}}a_{k}$ is not 0 for all $k$ . Then for this $i$ and $j$ we have $\left(\sum _{k}m_{{i,k}}n_{{j,k}}a_{k}\right)^{2}>0$ . This completes the proof.

References

↑ "Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen". Journal für die reine und angewandte Mathematik (Crelle's Journal). 1911 (140): 1–00. 1911. doi:10.1515/crll.1911.140.1.
↑ Zhang, Fuzhen, ed. (2005). "The Schur Complement and Its Applications". Numerical Methods and Algorithms. 4. doi:10.1007/b105056. ISBN 0-387-24271-6. , page 9, Ch. 0.6 Publication under J. Schur
↑ Ledermann, W. (1983). "Issai Schur and His School in Berlin". Bulletin of the London Mathematical Society. 15 (2): 97–106. doi:10.1112/blms/15.2.97.

External links

Bemerkungen zur Theorie der beschränkten Bilinearformen mit unendlich vielen Veränderlichen at EUDML

This article is issued from Wikipedia - version of the 8/8/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.