A Notation Convention
Abbreviations
a.k.a. == also known as
w.r.t. == with respect to
s.t. == such that
iff == if and only if
e.g. == for example (Latin: exempli gratia)
i.e. == that is (Latin: id est)
etc. == and so forth (Latin: et cetera)
AI == artificial intelligence
API == application programming interface
GA == genetic algorithm
GD == gradient descent
GLM == generalised linear model
ML == machine learning
NN == neural network
SGD == stochastic gradient descent
IDE = integrated development environment
Notation Convention – Logic and Set Theory
\(\forall\) – for all
\(\exists\) – exists
By writing \(x \in \{a, b, c\}\) we mean that “\(x\) is in a set that consists of \(a\), \(b\) and \(c\)” or “\(x\) is either \(a\), \(b\) or \(c\)”
\(A\subseteq B\) – set \(A\) is a subset of set \(B\) (every element in \(A\) belongs to \(B\), \(x\in A\) implies that \(x\in B\))
\(A\cup B\) – union (sum) of two sets, \(x\in A\cup B\) iff \(x\in A\) or \(x\in B\)
\(A\cap B\) – intersection (sum) of two sets, \(x\in A\cap B\) iff \(x\in A\) and \(x\in B\)
\(A\setminus B\) – difference of two sets, \(x\in A\setminus B\) iff \(x\in A\) and \(x\not\in B\)
\(A\times B\) – Cartesian product of two sets, \(A\times B = \{ (a,b): a\in A, b\in B \}\)
\(A^p = A\times A \times \dots\times A\) (\(p\) times) for any \(p\)
Notation Convention – Symbols
\(\mathbf{X,Y,A,I,C}\) – bold (I use it for denoting vectors and matrices)
\(\mathbb{X,Y,A,I,C}\) – blackboard bold (I sometimes use it for sets)
\(\mathcal{X,Y,A,I,C}\) – calligraphic (I use it for set families = sets of sets)
\(X, x, \mathbf{X}, \mathbf{x}\) – inputs (usually)
\(Y, y, \mathbf{Y}, \mathbf{y}\) – outputs
\(\hat{Y}, \hat{y}, \hat{\mathbf{Y}}, \hat{\mathbf{y}}\) – predicted outputs (usually)
\(X\) – independent/explanatory/predictor variable
\(Y\) – dependent/response/predicted variable
\(\mathbb{R}\) – the set of real numbers, \(\mathbb{R}=(-\infty, \infty)\)
\(\mathbb{N}\) – the set of natural numbers, \(\mathbb{N}=\{1,2,3,\dots\}\)
\(\mathbb{N}_0\) – the set of natural numbers including zero, \(\mathbb{N}_0=\mathbb{N}\cup\{0\}\)
\(\mathbb{Z}\) – the set of integer numbers, \(\mathbb{Z}=\{\dots,-2,-1,0,1,2,\dots\}\)
\([0,1]\) – the unit interval
\((a, b)\) – an open interval; \(x\in(a,b)\) iff \(a < x < b\) for some \(a< b\)
\([a, b]\) – a closed interval; \(x\in[a,b]\) iff \(a \le x \le b\) for some \(a\le b\)
Notation Convention – Vectors and Matrices
\(\boldsymbol{x}=(x_1,\dots,x_n)\) – a sequence of \(n\) elements (\(n\)-ary sequence/vector)
if it consists of real numbers, we write \(\boldsymbol{x}\in\mathbb{R}^n\)
\(\mathbf{x}=[x_1\ x_2\ \dots\ x_p]\) – a row vector, \(\mathbf{x}\in\mathbb{R}^{1\times p}\) (a matrix with 1 row)
\(\mathbf{x}=[x_1\ x_2\ \dots\ x_n]^T\) – a column vector, \(\mathbf{x}\in\mathbb{R}^{n\times 1}\) (a matrix with 1 column)
\(\mathbf{X}\in\mathbb{R}^{n\times p}\) – matrix with \(n\) rows and \(p\) columns
\[ \mathbf{X}= \left[ \begin{array}{cccc} x_{1,1} & x_{1,2} & \cdots & x_{1,p} \\ x_{2,1} & x_{2,2} & \cdots & x_{2,p} \\ \vdots & \vdots & \ddots & \vdots \\ x_{n,1} & x_{n,2} & \cdots & x_{n,p} \\ \end{array} \right] \]
\(x_{i,j}\) – element in the \(i\)-th row, \(j\)-th column
\(\mathbf{x}_{i,\cdot}\) – the \(i\)-th row of \(\mathbf{X}\)
\(\mathbf{x}_{\cdot,j}\) – the \(j\)-th column of \(\mathbf{X}\)
\[ \mathbf{X}=\left[ \begin{array}{c} \mathbf{x}_{1,\cdot} \\ \mathbf{x}_{2,\cdot} \\ \vdots\\ \mathbf{x}_{n,\cdot} \\ \end{array} \right] = \left[ \begin{array}{cccc} \mathbf{x}_{\cdot,1} & \mathbf{x}_{\cdot,2} & \cdots & \mathbf{x}_{\cdot,p} \\ \end{array} \right]. \]
\[ \mathbf{x}_{i,\cdot} = \left[ \begin{array}{cccc} x_{i,1} & x_{i,2} & \cdots & x_{i,p} \\ \end{array} \right]. \]
\[ \mathbf{x}_{\cdot,j} = \left[ \begin{array}{cccc} x_{1,j} & x_{2,j} & \cdots & x_{n,j} \\ \end{array} \right]^T=\left[ \begin{array}{c} {x}_{1,j} \\ {x}_{2,j} \\ \vdots\\ {x}_{n,j} \\ \end{array} \right], \]
\({}^T\) denotes the matrix transpose; \(\mathbf{B}=\mathbf{A}^T\) is a matrix such that \(b_{i,j}=a_{j,i}\).
\(\|\boldsymbol{x}\| = \|\boldsymbol{x}\|_2 = \sqrt{ \sum_{i=1}^n x_i^2 }\) – the Euclidean norm
Notation Convention – Functions
\(f:X\to Y\) means that \(f\) is a function mapping inputs from set \(X\) (the domain of \(f\)) to elements of \(Y\) (the codomain)
\(x\mapsto x^2\) denotes a (inline) function mapping \(x\) to \(x^2\),
equivalent to function(x) x^2 in R
\(\exp x = e^x\) – exponential function with base \(e\simeq 2.718\)
\(\log x\) – natural logarithm (base \(e\))
it holds \(e^x = y\) iff \(\log y = x\)
\(\log ab = \log a + \log b\)
\(\log a^c = c\log a\)
\(\log a/b = \log a - \log b\)
\(\log 1 = 0\)
\(\log e = 1\)
hence \(\log e^x = x\)
Notation Convention – Sums and Products
\(\sum_{i=1}^n x_i=x_1+x_2+\dots+x_n\)
\(\sum_{i=1,\dots,n} x_i\) – the same
\(\sum_{i\in\{1,\dots,n\}} x_i\) – the same
note display (stand-alone) \(\displaystyle\sum_{i=1}^n x_i\) vs text (in-line) \(\textstyle\sum_{i=1}^n x_i\) style
\(\prod_{i=1}^n x_i = x_1 x_2 \dots x_n\)