} END

Abbreviations

a.k.a. == also known as

w.r.t. == with respect to

s.t. == such that

iff == if and only if

e.g. == for example (Latin: exempli gratia)

i.e. == that is (Latin: id est)

etc. == and so forth (Latin: et cetera)

AI == artificial intelligence

API == application programming interface

GA == genetic algorithm

GD == gradient descent

GLM == generalised linear model

ML == machine learning

NN == neural network

SGD == stochastic gradient descent

IDE = integrated development environment

Notation Convention – Logic and Set Theory

\(\forall\) – for all

\(\exists\) – exists

By writing \(x \in \{a, b, c\}\) we mean that “\(x\) is in a set that consists of \(a\), \(b\) and \(c\)” or “\(x\) is either \(a\), \(b\) or \(c\)

\(A\subseteq B\) – set \(A\) is a subset of set \(B\) (every element in \(A\) belongs to \(B\), \(x\in A\) implies that \(x\in B\))

\(A\cup B\) – union (sum) of two sets, \(x\in A\cup B\) iff \(x\in A\) or \(x\in B\)

\(A\cap B\) – intersection (sum) of two sets, \(x\in A\cap B\) iff \(x\in A\) and \(x\in B\)

\(A\setminus B\) – difference of two sets, \(x\in A\setminus B\) iff \(x\in A\) and \(x\not\in B\)

\(A\times B\) – Cartesian product of two sets, \(A\times B = \{ (a,b): a\in A, b\in B \}\)

\(A^p = A\times A \times \dots\times A\) (\(p\) times) for any \(p\)

Notation Convention – Symbols

\(\mathbf{X,Y,A,I,C}\) – bold (I use it for denoting vectors and matrices)

\(\mathbb{X,Y,A,I,C}\) – blackboard bold (I sometimes use it for sets)

\(\mathcal{X,Y,A,I,C}\) – calligraphic (I use it for set families = sets of sets)

\(X, x, \mathbf{X}, \mathbf{x}\) – inputs (usually)

\(Y, y, \mathbf{Y}, \mathbf{y}\) – outputs

\(\hat{Y}, \hat{y}, \hat{\mathbf{Y}}, \hat{\mathbf{y}}\) – predicted outputs (usually)

  • \(X\) – independent/explanatory/predictor variable

  • \(Y\) – dependent/response/predicted variable

\(\mathbb{R}\) – the set of real numbers, \(\mathbb{R}=(-\infty, \infty)\)

\(\mathbb{N}\) – the set of natural numbers, \(\mathbb{N}=\{1,2,3,\dots\}\)

\(\mathbb{N}_0\) – the set of natural numbers including zero, \(\mathbb{N}_0=\mathbb{N}\cup\{0\}\)

\(\mathbb{Z}\) – the set of integer numbers, \(\mathbb{Z}=\{\dots,-2,-1,0,1,2,\dots\}\)

\([0,1]\) – the unit interval

\((a, b)\) – an open interval; \(x\in(a,b)\) iff \(a < x < b\) for some \(a< b\)

\([a, b]\) – a closed interval; \(x\in[a,b]\) iff \(a \le x \le b\) for some \(a\le b\)

Notation Convention – Vectors and Matrices

\(\boldsymbol{x}=(x_1,\dots,x_n)\) – a sequence of \(n\) elements (\(n\)-ary sequence/vector)

if it consists of real numbers, we write \(\boldsymbol{x}\in\mathbb{R}^n\)

\(\mathbf{x}=[x_1\ x_2\ \dots\ x_p]\) – a row vector, \(\mathbf{x}\in\mathbb{R}^{1\times p}\) (a matrix with 1 row)

\(\mathbf{x}=[x_1\ x_2\ \dots\ x_n]^T\) – a column vector, \(\mathbf{x}\in\mathbb{R}^{n\times 1}\) (a matrix with 1 column)

\(\mathbf{X}\in\mathbb{R}^{n\times p}\) – matrix with \(n\) rows and \(p\) columns

\[ \mathbf{X}= \left[ \begin{array}{cccc} x_{1,1} & x_{1,2} & \cdots & x_{1,p} \\ x_{2,1} & x_{2,2} & \cdots & x_{2,p} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n,1} & x_{n,2} & \cdots & x_{n,p} \\ \end{array} \right] \]

\(x_{i,j}\) – element in the \(i\)-th row, \(j\)-th column

\(\mathbf{x}_{i,\cdot}\) – the \(i\)-th row of \(\mathbf{X}\)

\(\mathbf{x}_{\cdot,j}\) – the \(j\)-th column of \(\mathbf{X}\)

\[ \mathbf{X}=\left[ \begin{array}{c} \mathbf{x}_{1,\cdot} \\ \mathbf{x}_{2,\cdot} \\ \vdots\\ \mathbf{x}_{n,\cdot} \\ \end{array} \right] = \left[ \begin{array}{cccc} \mathbf{x}_{\cdot,1} & \mathbf{x}_{\cdot,2} & \cdots & \mathbf{x}_{\cdot,p} \\ \end{array} \right]. \]

\[ \mathbf{x}_{i,\cdot} = \left[ \begin{array}{cccc} x_{i,1} & x_{i,2} & \cdots & x_{i,p} \\ \end{array} \right]. \]

\[ \mathbf{x}_{\cdot,j} = \left[ \begin{array}{cccc} x_{1,j} & x_{2,j} & \cdots & x_{n,j} \\ \end{array} \right]^T=\left[ \begin{array}{c} {x}_{1,j} \\ {x}_{2,j} \\ \vdots\\ {x}_{n,j} \\ \end{array} \right], \]

\({}^T\) denotes the matrix transpose; \(\mathbf{B}=\mathbf{A}^T\) is a matrix such that \(b_{i,j}=a_{j,i}\).

\(\|\boldsymbol{x}\| = \|\boldsymbol{x}\|_2 = \sqrt{ \sum_{i=1}^n x_i^2 }\) – the Euclidean norm

Notation Convention – Functions

\(f:X\to Y\) means that \(f\) is a function mapping inputs from set \(X\) (the domain of \(f\)) to elements of \(Y\) (the codomain)

\(x\mapsto x^2\) denotes a (inline) function mapping \(x\) to \(x^2\), equivalent to function(x) x^2 in R

\(\exp x = e^x\) – exponential function with base \(e\simeq 2.718\)

\(\log x\) – natural logarithm (base \(e\))

it holds \(e^x = y\) iff \(\log y = x\)

\(\log ab = \log a + \log b\)

\(\log a^c = c\log a\)

\(\log a/b = \log a - \log b\)

\(\log 1 = 0\)

\(\log e = 1\)

hence \(\log e^x = x\)

Notation Convention – Sums and Products

\(\sum_{i=1}^n x_i=x_1+x_2+\dots+x_n\)

\(\sum_{i=1,\dots,n} x_i\) – the same

\(\sum_{i\in\{1,\dots,n\}} x_i\) – the same

note display (stand-alone) \(\displaystyle\sum_{i=1}^n x_i\) vs text (in-line) \(\textstyle\sum_{i=1}^n x_i\) style

\(\prod_{i=1}^n x_i = x_1 x_2 \dots x_n\)