1.1 Basic Notation and Terminology  14
1.2 Operations Relating Matrices  16
1.3 Inequality Relations Between Matrices 17
1.4 Operations Related to Individual Matrices 17
1.5 Some Special Matrices 22
1.6 Some Terms and Quantities Related to Matrices 24

2.1 Rules Related to Matrix Sums and Differences  27
2.2 Rules Related to Matrix Multiplication  28
2.3 Rules Related to Multiplication by a Scalar 30
2.4 Rules for the Kronecker Product 31
2.5 Rules for the Hadamard Product  32
2.6 Rules for Direct Sums 34

3.1 The Transpose 35
3.2 The Conjugate 36
3.3 The Conjugate Transpose 37
3.4 The Adjoint of a Square Matrix  39
3.5 The Inverse of a Square Matrix  39
3.5.1 General Results 39
3.5.2 Inverses Involving Sums and Differences 40
3.5.3 Partitioned Inverses  41
3.5.4 Inverses Involving Commutation, Duplication and 43
3.6 Generalized Inverses  44
3.6.1 General Results 44
3.6.2 The Moore-Penrose Inverse 46
3.7 Matrix Powers 49
3.8 The Absolute Value  51


4.1 The Trace 53
4.1.1 General Results 53
4.1.2 Inequalities Involving the Trace  55
4.1.3 Optimization of Functions Involving the Trace 57
4.2 The Determinant 59
4.2.1 General Results 59
4.2.2 Determinants of Partitioned Matrices  61
4.2.3 Determinants Involving Duplication Matrices 63
4.2.4 Determinants Involving Elimination Matrices 64
4.2.5 Determinants Involving Both Duplication and Elimination 65
4.2.6 Inequalities Related to Determinants  66
4.2.7 Optimization of Functions Involving a Determinant 68
4.3 The Rank of a Matrix  70
4.3.1 ?????  70
4.3.2 Matrix Decompositions Related to the Rank 72
4.3.3 Inequalities Related to the Rank  73


5.1 Definitions 74
5.2 Properties of Eigenvalues and Eigenvectors  75
5.2.1 General Results 75
5.2.2 Optimization Properties of Eigenvalues  78
5.2.3 Matrix Decompositions Involving Eigenvalues 80
5.3 Eigenvalue Inequalities 83
5.3.1 Inequalities for the Eigenvalues of a Single Matrix 83
5.3.2 Relations Between Eigenvalues of More Than One Matrix 85
5.4 Results for the Spectral Radius 87
5.5 Singular Values 89
5.5.1 General Results 89
5.5.2 Inequalities  91


6.1 Complex Matrix Decompositions 93
6.1.1 Jordan Type Decompositions  93
6.1.2 Diagonal Decompositions 95
6.1.3 Other Triangular Decompositions and Factorizations  96
6.1.4 Miscellaneous Decompositions  98
6.2 Real Matrix Decompositions  99
6.2.1 Jordan Decompositions 99
6.2.2 Other Real Block Diagonal and Diagonal Decompositions  100
6.2.3 Other Triangular and Miscellaneous Reductions  102

7.1 Definitions  104
7.2 Rules for the vec Operator 106
7.3 Rules for the vech Operator  108

8.1 General Definitions  110
8.2 Specific Norms and Inner Products  112
8.3 Results for General Norms and Inner Products 113
8.4 Results for Matrix Norms 115
8.4.1 General Matrix Norms 115
8.4.2 Induced Matrix Norms 117
8.5 Properties of Special Norms  118
8.5.1 General Results  118
8.5.2 Inequalities 120


9.1 Circulant Matrices 122
9.2 Commutation Matrices 124
9.2.1 General Properties 125
9.2.2 Kronecker Products 126
9.2.3 Relations With Duplication and Elimination Matrices  127
9.3 Convergent Matrices  128
9.4 Diagonal Matrices  129
9.5 Duplication Matrices 131
9.5.1 General Properties 131
9.5.2 Relations With Commutation and Elimination Matrices  132
9.5.3 Expressions With vec and vech Operators  132
9.5.4 Duplication Matrices and Kronecker Products  133
9.5.5 Duplication Matrices, Elimination Matrices and Kronecker 135
9.6 Elimination Matrices 136
9.6.1 General Properties 136
9.6.2 Relations With Commutation and Duplication Matrices  136
9.6.3 Expressions With vec and vech Operators  137
9.6.4 Elimination Matrices and Kronecker Products  137
9.6.5 Elimination Matrices, Duplication Matrices and Kronecker 139
9.7 Hermitian Matrices 140
9.7.1 General Results  140
9.7.2 Eigenvalues of Hermitian Matrices  142
9.7.3 Eigenvalue Inequalities  143
9.7.4 Decompositions of Hermitian Matrices 146
9.8 Idempotent Matrices  147
9.9 Nonnegative, Positive and Stochastic Matrices  148
9.9.1 Definitions  148
9.9.2 General Results  149
9.9.3 Results Related to the Spectral Radius 150
9.10 Orthogonal Matrices 151
9.10.1 General Results 152
9.10.2 Decompositions of Orthogonal Matrices 153
9.11 Partitioned Matrices  153
9.11.1 General Results 154
9.11.2 Determinants of Partitioned Matrices  155
9.11.3 Partitioned Inverses  156
9.11.4 Partitioned Generalized Inverses  157
9.11.5 Partitioned Matrices Related to Duplication Matrices  158
9.12 Positive Definite, Negative Definite and Semidefinite 159
9.12.1 General Properties  160
9.12.2 Eigenvalue Results  162
9.12.3 Decomposition Theorems for Definite Matrices  163
9.13 Symmetric Matrices  165
9.13.1 General Properties  165
9.13.2 Symmetry and Duplication Matrices 166
9.13.3 Eigenvalues of Symmetric Matrices 167
9.13.4 Eigenvalue Inequalities 168
9.13.5 Decompositions of Symmetric and Skew-Symmetric  172
9.14 Triangular Matrices 173
9.14.1 Properties of General Triangular Matrices 173
9.14.2 Triangularity, Elimination and Duplication Matrices 174
9.14.3 Properties of Strictly Triangular Matrices  176
9.15 Unitary Matrices  176

10.1 Notation  179
10.2 Gradients and Hessian Matrices of Real Valued 182
10.2.1 Gradients 182
10.2.2 Hessian Matrices  183
10.3 Derivatives of Real Valued Functions with Matrix  184
10.3.1 General and Miscellaneous Rules 184
10.3.2 Derivatives of the Trace  185
10.3.3 Derivatives of Determinants 189
10.4 Jacobian Matrices of  10.4.1  10.4.2  10.4.3  10.4.4  
10.5 Product Rules 197
10.5.1 Matrix Products 197
10.5.2 Kronecker and Hadamard Products 200
10.5.3 Functions with Symmetric Matrix Arguments 201
10.5.4 Functions with Lower Triangular Matrix Arguments  203
10.5.5 Products of Matrix Valued Functions with Vector 204
10.6 Jacobian Matrices of Functions Involving Inverse  206
10.6.1 Matrix Products 206
10.6.2 Kronecker and Hadamard Products 207
10.6.3 Matrix Valued Functions with Vector Arguments 208
10.7 Chain Rules and Miscellaneous Jacobian Matrices 210
10.8 Jacobian Determinants 212
10.8.1  10.8.2 Non 10.9 Matrix Valued Functions of a Scalar Variable 216


11.1 Definitions and Notations 218 
11.1.1 Definitions and Notation Related to Polynomials 218 
11.1.2 Matrices Related to Polynomials 220 
11.1.3 Polynomials and Power Series Related to Matrices  221 
11.2 Results Relating Polynomials and Matrices 222 
11.3 Polynomial Matrices 224 
11.3.1 Definitions 224 
11.3.2 Results for Polynomial Matrices 228