MATH 0047
Advanced Linear Algebra
Department of Mathematics
UCL
Lecture Notes
2021-2022
Isidoros Strouthos
September 30, 2021
Abstract
In this course, we will aim to develop aspects of the theory of matrices and give an introduction to
the theory of vector spaces and the theory of linear maps. During the course, we are also due to see
examples of techniques which illustrate the use of the relevant mathematical objects.
Some of the topics we will cover in the first part of the course, apart from a revision of the basic
theory of matrices, will be row reduction and Gaussian elimination, elementary matrices, and matrix
determinants. The material will allow us to describe techniques used to find matrix inverses and
solve (systems of) linear equations.
The second part of the course will be an introduction to vector spaces and linear maps. It will include
the notions of inner products, subspaces, bases, orthogonality, norm, eigenvalues and eigenvectors,
results such as the Cauchy-Schwarz inequality, and techniques such as the Gram-Schmidt process.
Contents
1 Matrices and linear equations 2
1
Chapter 1
Matrices and linear equations
The mathematical objects we will be working with for most of this course are matrices. So, before
we go on to study some of the tools and techniques we will see in this course, let us review some
of the basic notions related to matrices, and introduce the subscript notation, which allows us to
prove, in a concise way, that matrices have various properties, by letting us “go down” to the level
of matrix entries, or numbers.
Definition 1.1. Amatrix is a rectangular array of numbers.
In this course, our matrices will always contain complex numbers, so we may refer to them as
complex matrices, even though we shall often simply refer to them as matrices. So, unless otherwise
stated, below, a matrix is necessarily a complex matrix, i.e. it contains complex numbers as (array)
entries (though many of our examples will involve only real matrices, i.e. matrices with real numbers
as entries).
We will denote the set of complex numbers by Cand the set of real numbers by R. So, x∈Cmeans
that “xis a complex number”, while x∈Rmeans that “xis a real number”.
If a matrix Ahas mrows and ncolumns, we will say that Ahas size m×n, or, equivalently, that A
is an m×nmatrix
e.g.
A=
A11 A12 · · · A1n
A21 A22 · · · A2n
.
.
..
.
.....
.
.
Am1Am2· · · Amn
is an m×nmatrix
2
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 3
Note the way in which we label the entries in a matrix. For a matrix, Asay (we will almost always
use capital letters for matrices), Aij is the number in row iand column j. It is often useful to move
over to this subscript notation in order to show that matrices have certain properties.
Let us see some examples of matrices:
1. A=1 0 −2
4 1 7 is a 2×3matrix, where
A11 = 1, A12 = 0, A13 =−2, A21 = 4, A22 = 1, A23 = 7
2. 1 5 7is a 1×3matrix.
3. B=1
23
5−6is a 2×2matrix.
4. 5is a 1×1matrix.
5. b=
1+5i
0
3
−2
is a 4×1matrix.
6. x=
x1
x2
x3
is a 3×1matrix, if x1,x2,x3are complex numbers, i.e. if x1, x2, x3∈C.
Sometimes, as in example 6 above, we will not explicitly give the numbers in the matrix, but will
label them instead (for example, the entries may correspond to the unknowns or variables appearing
in a set of equations that we are trying to solve, using matrices).
Definition 1.2. A matrix Mis a square matrix if it has the same number of rows and columns, i.e.
if it is a matrix of size n×n, for some n.
In the examples given above, (only) the matrices given in examples 3 and 4 are square matrices.
As mentioned earlier, we shall almost always use capital letters to denote matrices, except, possibly,
when a matrix is a vector, i.e. when it consists of a single column (or row). In those cases, we may
use a small (lowercase) letter, as in the cases of the matrices given in examples 5 and 6 above.
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 4
We would like to refer to two matrices, Aand Bsay, as equal if they have the same size and contain
the same numbers in corresponding positions i.e. if all corresponding entries are equal.
Another way of saying that “all corresponding entries are equal” is to use the subscript notation,
which “picks out” the entries for us.
Definition 1.3. Two matrices, Aand Bsay, are equal if they are of the same size, and if, for all i,j:
Aij =Bij
e.g.
1 2
3 4=1 2
3 4
1 2 5
3 4 6̸=1 2
3 4
a0
1b=3 0
1−4precisely when a= 3 and b=−4
Let us now describe some of the basic operations related to matrices.
If two matrices are of the same size, then we can add them together simply by adding corresponding
entries together.
Another way of expressing the same idea is to say that, assuming that Aand Bhave the same size,
we can form a matrix A+B, such that, for all (relevant) iand j, the number in row i, column jof
A+Bis the number in row i, column jof Aplus the number in row i, column jof B. Once again,
the subscript ‘entrywise’ notation allows us to write this down in a concise way:
Definition 1.4. Suppose that two matrices, Aand Bsay, have the same size, m×nsay. Then, the
sum of Aand B, denoted by A+B, is the m×nmatrix defined by:
(A+B)ij =Aij +Bij for all i,j(i.e. for all 1≤i≤m,1≤j≤n)
A+B=
A11 · · · A1n
.
.
.....
.
.
Am1· · · Amn
+
B11 · · · B1n
.
.
.....
.
.
Bm1· · · Bmn
=
A11 +B11 · · · A1n+B1n
.
.
.....
.
.
Am1+Bm1· · · Amn +Bmn
For example, if A=1 8 −3
2 5 7 and B=0−5 1
123, then:
A+B=1 3 −2
3 7 10
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 5
We can also multiply a matrix by a constant, a fixed complex number, by simply multiplying each
entry in the matrix by the given number.
Definition 1.5. Suppose that Ais a complex matrix, of size m×nsay, and cis a fixed complex
number. Then, the scalar multiple cA is the matrix defined (entrywise) by
(cA)ij =c(Aij )for all i,j
cA =c
A11 · · · A1n
.
.
.....
.
.
Am1· · · Amn
=
cA11 · · · cA1n
.
.
.....
.
.
cAm1· · · cAmn
For example, if A=
1 2
8 5
−3 7
, then:
3A=
3 6
24 15
−9 21
,−A=
−1−2
−8−5
3−7
,−2A=
−2−4
−16 −10
6−14
,1
2A=
1
21
45
2
−3
2
7
2
It is also (sometimes) possible to multiply two given matrices together, as you might have seen
before.
If we wish to find the product of matrices Aand B, then the number of columns of Amust be equal
to the number of rows of B.
This is because of the way in which multiplication is defined. If we want to find the number which
should appear in row i, column jof the product AB, then we must “pair up” row iof Awith column
jof B, multiply corresponding entries together and then add the resulting products up. For this to
work “properly”, we must have the same number of entries in each row of Aas in each column of
B, i..e the number of columns of Amust be equal to the number of rows of B.
For example, suppose that Ais a matrix with 3columns and Bis a matrix with 3rows.
Let us write down what will appear in row 1, column 2of the product, i.e. what (AB)12 will be:
(AB)12 =A11B12 +A12B22 +A13B32
In other words, we are “multiplying” row 1of Aby column 2of B:
A11 A12 A13
.
.
..
.
..
.
.
.
.
..
.
..
.
.
· · · B12 · · ·
· · · B22 · · ·
· · · B32 · · ·
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 6
Observe that, in each of the “little entry products” above, the column of the entry from Amatches
the row of the entry from B, so that we may rewrite the above as:
(AB)12 =
3
X
k=1
A1kBk2
where ksimply denotes the ‘counter’ we are using.
The same idea applies when we wish to find the answer for an entry in any row and column of the
product matrix, not simply for the entry in row 1, column 2. So, we could write, for all i,j:
(AB)ij =
3
X
k=1
AikBkj
Notice also that k“goes” from 1to 3because, in this example Ahas 3columns (and Bhas 3rows).
We are, therefore, led to the following definition (or description) of (general) matrix multiplication:
Definition 1.6. Suppose that Ais an m×rmatrix and Bis an r×nmatrix, so that the number of
columns of Ais equal to the number of rows of B. Then, we can define the matrix product AB to be
the m×nmatrix satisfying:
(AB)ij =
r
X
k=1
AikBkj for all i,j(i.e. for all 1≤i≤m,1≤j≤n)
For example, if A=
1 2
8 5
−3 7
,B=0 2 1
1 0 −4,C=3 0
5−2, then:
AB =
2 2 −7
5 16 −12
7−6−31
, BA =13 17
13 −26, AC =
13 −4
49 −10
26 −14
, CB =0 6 3
−2 10 13
and C2=C C =9 0
5 4
whereas the products A2,B2,CA,BC are not defined.
Suppose that Ais a square matrix, of size n×n. Then, the sum A11 +A22 +· · · +Ann turns out to
be quite an important feature of the matrix, for some purposes. This number is known as the trace
of the matrix, and can otherwise be described as the sum of numbers on the main diagonal of the
matrix (the main diagonal of an n×nmatrix Ais the diagonal “running from the top left to the
bottom right” i.e. the one containing the set of entries {A11, A22,· · · , Ann}).
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 7
Definition 1.7. Let Abe a square matrix, of size n×n. Then the trace of A, denoted by tr(A), or
trace(A), is the sum of the entries along the main diagonal:
tr(A) = A11 +A22 +· · · +Ann or tr(A) =
n
X
i=1
Aii
For example, if A=
1 0 2
2 5 6
0 0 −4
, then:
tr(A) = 1 + 5 + (−4) = 2
As you might have seen before, we can also “flip” a matrix, in order to obtain its transpose. If we
are dealing with a square matrix, we can also imagine the “transpose action” as a “reflection” along
the main diagonal. However, the transpose is defined for a matrix of any size: it interchanges rows
and columns, and so contains in row i, column jwhat was “before” in row j, column i.
Definition 1.8. Suppose that Ais an m×nmatrix. Then, the transpose of A, denoted by AT, is
the n×mmatrix obtained by interchanging rows and columns, i.e. for all iand j(1≤i≤n,
1≤j≤m):
ATij =Aji
For example:
If A=
1 8 −3
2 5 7
0 0 4
,then AT=
1 2 0
8 5 0
−3 7 4
If B=1 8 −3
2 5 7 ,then BT=
1 2
8 5
−3 7
If C=
1 8
2 5
0 0
,then CT=1 2 0
8 5 0
Please see exercise 1on Sample Exercises 1and exercise 1on Exercise Set 1for some more practice
related to the ideas mentioned above.
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 8
Before we proceed to give a description of the inverse of a matrix, let us give some results on
matrices, which involve the properties and operations defined above. We will prove a few of these
results by moving over to the subscript ‘entrywise’ setting, and making use of the fact that, once
we are dealing with subscripts and actual numbers, rather than whole matrices, many properties and
operations “work in a simple way”.
At first glance, the proofs below might appear quite complicated. However, using subscripts turns
out to provide a concise way of proving results like the ones that follow, and a quite powerful way
too. In the proofs, most of the steps follow by simply using one of the definitions given above. So,
once you are comfortable with the main definitions given above, most of the steps in the proofs will
hopefully be understandable.
The following are some concepts and techniques that crop up quite often in proofs of results similar
to the ones we are about to see:
·Subscripts appear in most of the definitions above, so it is natural, in a way, to use them
below. In fact, most of the individual steps in the proofs below follow by simply using one of
the definitions above (one which is relevant to what we are dealing with at any step), and then
rearranging our expression in some way to get closer to the desired end.
·A useful tip when “rearranging” is that, once we are at the level of subscripts, we are working
with actual numbers in a matrix, rather than matrices themselves, so have more “freedom”.
For example, we can add numbers in any order we wish:
Aij +Bij =Bij +Aij
we can multiply numbers in any order we wish:
AikBkj =Bkj Aik
we can “open brackets”:
Aik(Bkj +Ckj ) = AikBkj +Aik Ckj
etc.
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 9
·Another useful “rearranging tip” is that, once we are at the level of subscripts and are therefore
dealing with numbers, if sums also happen to appear (e.g. if we are dealing with matrix
multiplication), we can “do sums” in any order we wish.
So, for example, if Cis an m×nmatrix:
m
X
i=1
n
X
j=1
Cij =
n
X
j=1
m
X
i=1
Cij
Furthermore, we can suitably rename the ‘counters’ appearing in a sum, e.g. for n×nmatrices
C,D:n
X
k=1
CikDkj =
n
X
l=1
CilDlj
which we may obtain by renaming kas l, or even
n
X
k=1
n
X
l=1
CklDlk =
n
X
l=1
n
X
k=1
ClkDkl
which we may obtain by renaming kas land las k, wherever they appear.
Proposition 1.9. Suppose that Aand Bare m×nmatrices. Then:
A+B=B+A
Proof. To show that A+Bis equal to B+A, we will show that their corresponding entries are
equal, i.e. that (A+B)ij = (B+A)ij for all iand j. Now, for all 1≤i≤m,1≤j≤n:
(A+B)ij =Aij +Bij by definition of matrix addition; Definition 1.4
=Bij +Aij since we are dealing with numbers
=(B+A)ij by definition of matrix addition; Definition 1.4
So, since for all i,j:(A+B)ij = (B+A)ij , we deduce that A+B=B+A, as required.
CHAPTER 1. MATRICES AND LINEAR EQUATIONS 10
We can prove the following in a similar way:
Proposition 1.10. Suppose that A,Band Care m×nmatrices. Then:
A+ (B+C) = (A+B) + C
We can use the same method to prove many more results related to matrices, e.g.:
Proposition 1.11. Suppose that Ais an m×rmatrix, and that Band Care r×nmatrices. Then:
A(B+C) = AB +AC
Proof. For all 1≤i≤m,1≤j≤n:
(A(B+C))ij =
r
X
k=1
Aik(B+C)kj by definition of matrix multiplication; Definition 1.6
=
r
X
Aik(Bkj +Ckj )by definition of matrix addition; Definition 1.4
