B.1 Motivation

Vector and matrix algebra provides us with a convenient language for expressing computations on sequential and tabular data.

Vector and matrix algebra operations are supported by every major programming language – either natively (e.g., R, Matlab, GNU Octave, Mathematica) or via an additional library/package (e.g, Python with numpy, tensorflow or pytorch; C++ with Eigen/Armadillo; C, C++ or Fortran with LAPACK).

By using matrix notation, we generate more concise and readable code.

For instance, given two vectors $$\boldsymbol{x}=(x_1,\dots,x_n)$$ and $$\boldsymbol{y}=(y_1,\dots,y_n)$$ like:

x <- c(1.5, 3.5, 2.3,-6.5)
y <- c(2.9, 8.2,-0.1, 0.8)

s <- 0
n <- length(x)
for (i in 1:n)
s <- s + (x[i]-y[i])^2
sqrt(s)
## [1] 9.11592

to mean:

$\sqrt{ (x_1-y_1)^2 + (x_2-y_2)^2 + \dots + (x_n-y_n)^2 } = \sqrt{\sum_{i=1}^n (x_i-y_i)^2},$

which denotes the (Euclidean) distance between the two vectors (the square root of the sum of squared differences between the corresponding elements in $$\boldsymbol{x}$$ and $$\boldsymbol{y}$$), we shall soon become used to writing:

sqrt(sum((x-y)^2))
## [1] 9.11592

or:

$\sqrt{(\boldsymbol{x}-\boldsymbol{y})^{T}(\boldsymbol{x}-\boldsymbol{y})}$

or even:

$\|\boldsymbol{x}-\boldsymbol{y}\|_2$

In order to be able to read this notation, we only have to get to know the most common “building blocks”. There are just a few of them, but it’ll take some time until we become comfortable with their use.

What’s more, we should note that vectorised code might be much faster than the for loop-based one (a.k.a. “iterative” style):

library("microbenchmark")
n <- 10000
x <- runif(n) # n random numbers in [0,1]
y <- runif(n)
print(microbenchmark(
t1={
# "iterative" style
s <- 0
n <- length(x)
for (i in 1:n)
s <- s + (x[i]-y[i])^2
sqrt(s)
},
t2={
# "vectorised" style
sqrt(sum((x-y)^2))
}
), signif=3, unit='relative')
## Unit: relative
##  expr min  lq mean median  uq  max neval
##    t1 128 125  106    121 103 93.3   100
##    t2   1   1    1      1   1  1.0   100