127x Filetype PDF File size 0.35 MB Source: www.cs.toronto.edu
The Matrix Cookbook Kaare Brandt Petersen Michael Syskind Pedersen Version: February 16, 2006 What is this? These pages are a collection of facts (identities, approxima- tions, inequalities, relations, ...) about matrices and matters relating to them. It is collected in this form for the convenience of anyone who wants a quick desktop reference . Disclaimer: Theidentities, approximations and relations presented here were obviously not invented but collected, borrowed and copied from a large amount of sources. These sources include similar but shorter notes found on the internet and appendices in books - see the references for a full list. Errors: Very likely there are errors, typos, and mistakes for which we apolo- gize and would be grateful to receive corrections at cookbook@2302.dk. Its ongoing: The project of keeping a large repository of relations involving matrices is naturally ongoing and the version will be apparent from the date in the header. Suggestions: Your suggestion for additional content or elaboration of some topics is most welcome at cookbook@2302.dk. Keywords: Matrix algebra, matrix relations, matrix identities, derivative of determinant, derivative of inverse matrix, differentiate a matrix. Acknowledgements: We would like to thank the following for contribu- tions and suggestions: Christian Rishøj, Douglas L. Theobald, Esben Hoegh- Rasmussen, Lars Christiansen, and Vasile Sima. We would also like thank The Oticon Foundation for funding our PhD studies. 1 CONTENTS CONTENTS Contents 1 Basics 5 1.1 Trace and Determinants . . . . . . . . . . . . . . . . . . . . . . . 5 1.2 The Special Case 2x2 . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Derivatives 7 2.1 Derivatives of a Determinant . . . . . . . . . . . . . . . . . . . . 7 2.2 Derivatives of an Inverse . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Derivatives of Matrices, Vectors and Scalar Forms . . . . . . . . 9 2.4 Derivatives of Traces . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.5 Derivatives of Structured Matrices . . . . . . . . . . . . . . . . . 12 3 Inverses 15 3.1 Basic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2 Exact Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.3 Implication on Inverses . . . . . . . . . . . . . . . . . . . . . . . . 17 3.4 Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.5 Generalized Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.6 Pseudo Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4 Complex Matrices 19 4.1 Complex Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . 19 5 Decompositions 22 5.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . 22 5.2 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . 22 5.3 Triangular Decomposition . . . . . . . . . . . . . . . . . . . . . . 24 6 Statistics and Probability 25 6.1 Definition of Moments . . . . . . . . . . . . . . . . . . . . . . . . 25 6.2 Expectation of Linear Combinations . . . . . . . . . . . . . . . . 26 6.3 Weighted Scalar Variable . . . . . . . . . . . . . . . . . . . . . . 27 7 Gaussians 28 7.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 7.2 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 7.3 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 7.4 Mixture of Gaussians . . . . . . . . . . . . . . . . . . . . . . . . . 33 8 Special Matrices 34 8.1 Units, Permutation and Shift . . . . . . . . . . . . . . . . . . . . 34 8.2 The Singleentry Matrix . . . . . . . . . . . . . . . . . . . . . . . 35 8.3 Symmetric and Antisymmetric . . . . . . . . . . . . . . . . . . . 37 8.4 Vandermonde Matrices . . . . . . . . . . . . . . . . . . . . . . . . 37 8.5 Toeplitz Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 8.6 The DFT Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Petersen & Pedersen, The Matrix Cookbook, Version: February 16, 2006, Page 2 CONTENTS CONTENTS 8.7 Positive Definite and Semi-definite Matrices . . . . . . . . . . . . 40 8.8 Block matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 9 Functions and Operators 43 9.1 Functions and Series . . . . . . . . . . . . . . . . . . . . . . . . . 43 9.2 Kronecker and Vec Operator . . . . . . . . . . . . . . . . . . . . 44 9.3 Solutions to Systems of Equations . . . . . . . . . . . . . . . . . 45 9.4 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 9.5 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 9.6 Integral Involving Dirac Delta Functions . . . . . . . . . . . . . . 48 9.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 A One-dimensional Results 50 A.1 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 A.2 One Dimensional Mixture of Gaussians . . . . . . . . . . . . . . . 51 B Proofs and Details 53 B.1 Misc Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Petersen & Pedersen, The Matrix Cookbook, Version: February 16, 2006, Page 3 CONTENTS CONTENTS Notation and Nomenclature A Matrix A Matrix indexed for some purpose ij A Matrix indexed for some purpose i ij A Matrix indexed for some purpose An Matrix indexed for some purpose or The n.th power of a square matrix −1 A The inverse matrix of the matrix A A+ The pseudo inverse matrix of the matrix A (see Sec. 3.6) A1/2 The square root of a matrix (if unique), not elementwise (A) The (i,j).th entry of the matrix A ij Aij The (i,j).th entry of the matrix A [A] The ij-submatrix, i.e. A with i.th row and j.th column deleted ij a Vector ai Vector indexed for some purpose ai The i.th element of the vector a a Scalar ℜz Real part of a scalar ℜz Real part of a vector ℜZ Real part of a matrix ℑz Imaginary part of a scalar ℑz Imaginary part of a vector ℑZ Imaginary part of a matrix det(A) Determinant of A Tr(A) Trace of the matrix A diag(A) Diagonal matrix of the matrix A, i.e. (diag(A)) =δ A ij ij ij vec(A) The vector-version of the matrix A (see Sec. 9.2.2) ||A|| Matrix norm (subscript if any denotes what norm) T A Transposed matrix ∗ A Complex conjugated matrix H A Transposed and complex conjugated matrix (Hermitian) A◦B Hadamard(elementwise) product A⊗B Kroneckerproduct 0 The null matrix. Zero in all entries. I The identity matrix ij J The single-entry matrix, 1 at (i,j) and zero elsewhere Σ Apositive definite matrix Λ Adiagonal matrix Petersen & Pedersen, The Matrix Cookbook, Version: February 16, 2006, Page 4
no reviews yet
Please Login to review.