Chapter 1: Mathematics Foundations for AI: Sets, Functions, Notation & the Language of Machine Learning

Chapter 1: Mathematics Foundations for AI: Sets, Functions, Notation & the Language of Machine Learning

AI Reading

Quick summary of this article

Preparing the AI summary…

AI Mathematics Masterclass — Chapter 1 of 48

Welcome to Chapter 1 of the AI Mathematics Masterclass. Every neural network is a mathematical object, and every training step is a mathematical operation. This chapter builds the notation grammar that the remaining forty-seven chapters depend on: how sets define the space of possible inputs, how functions define the mapping from input to prediction, how summation notation compresses loops into single expressions, and how the scalar–vector–matrix–tensor ladder encodes the shape of every quantity in a model. By the end you will be able to read a model equation from a research paper aloud, symbol by symbol, and know exactly which array operation it implies.

1. Technical Audit & Core Learning Objectives

Most mathematics-for-machine-learning material fails in the same three places: it teaches calculus before establishing notation, it treats linear algebra as a detached topic rather than the language of shape, and it never connects a symbol on the page to an operation in an array library. This chapter front-loads notation deliberately, because notation is the interface between a paper and a running program.

Component Common Tutorial Deficiency Engineering Standard Applied Here
Notation Symbols introduced ad hoc, with no statement of type, rank, or shape. Every symbol carries a type declaration (x ∈ ℝⁿ) and an explicit rank.
Set Theory Omitted entirely, leaving subspaces, supports, and samples undefined. Sets introduced as the ground layer: domain, support, sample space, parameter space.
Summation Shown as a loop in code, never linked back to the Σ expression in papers. Every Σ expression is translated to its exact array reduction.
Shapes Rank and axis semantics deferred to a later “practical” chapter. The notation ladder is taught first, so shape errors become impossible to misdiagnose.

Core learning objectives. After this chapter you should be able to:

  1. Perform union, intersection, difference, and complement on concrete sets, and state the inclusion–exclusion identity from memory.
  2. Distinguish a function from a relation, and identify domain, codomain, image, and preimage in a given mapping.
  3. Compose two functions and explain why f(g(x)) is the algebraic shape of every multi-layer network.
  4. Read a nested summation such as Σᵢ Σⱼ A[i,j]·B[j,i] and name the corresponding array operation.
  5. Declare the rank and shape of any quantity in a model on sight, including batched forms.
  6. Translate five standard Greek symbols from deep-learning papers into their operative meaning.

2. Why Mathematics Is the Instruction Set of Machine Learning

A common misconception holds that mathematics is a prerequisite to be cleared before “real” machine learning begins. The opposite is true. Mathematics is not a gate in front of the pipeline — it is the instruction set that the pipeline executes at every iteration.

Four mathematical pillars behind every AI model: linear algebra, calculus, probability and information theory
Figure 1.1: Four mathematical pillars converge on a single trainable model. A parameter update is a simultaneous application of all four, not a sequence.

Consider the smallest possible training step. You hold a weight matrix W, you compute a prediction ŷ = f(x; W), you measure a loss L(ŷ, y), and you subtract a scaled gradient. Four distinct branches of mathematics appear in that one sentence:

Operation in the Loop Mathematical Branch Symbol You Will See
Building a prediction Linear algebra y = Wx + b, W ∈ ℝ^(d_out × d_in)
Measuring error Information theory / probability L = −Σ yᵢ log ŷᵢ
Attributing blame Calculus ∂L/∂W
Updating parameters Optimization (applied calculus) W ← W − η·∂L/∂W
Judging generalisation Probability / statistics E[L] over p(x, y)
Machine learning pipeline showing where each branch of mathematics enters: encoding, tensors, matrix multiplication, entropy, chain rule and optimization
Figure 1.2: Each stage of the machine-learning pipeline is owned by a specific branch of mathematics. The loop repeats until the gradient stops improving the loss.

3. Sets: The Ground Layer of Everything

A set is an unordered collection of distinct objects, called elements. If x is an element of A we write x ∈ A; if not, x ∉ A. A set with no elements is the empty set, written ∅.

Sets matter because every object in machine learning is defined as a member of one. The domain of a function is a set. The set of all possible parameter values is a set. The training dataset is a finite set of pairs. A class label space is a finite set. Even the output of a classifier is an element of a probability simplex — a specific geometric set.

3.1 Set specification methods

# Roster method: list the elements explicitly
labels = {"cat", "dog", "horse"}

# Set-builder (comprehension) method: state the rule
# B = { x ∈ ℝ | 0 ≤ x ≤ 1 }   "all reals between 0 and 1 inclusive"
unit_interval_rule = lambda x: 0.0 <= x <= 1.0

# A finite dataset is a set of (input, label) pairs
# D = { (x₁,y₁), (x₂,y₂), ..., (xₙ,yₙ) }
print(len(labels))          # 3 — sets drop duplicates

3.2 The four core operations

Venn diagram of set union, intersection, difference and subset with their machine-learning interpretations
Figure 1.3: Set operations and the machine-learning question each one answers.

The inclusion–exclusion identity governs how cardinalities combine. It is the reason a naive "count of A plus count of B" overcounts whenever the sets overlap:

|A ∪ B| = |A| + |B| − |A ∩ B|

# Extends to three sets:
|A ∪ B ∪ C| = |A| + |B| + |C|
             − |A ∩ B| − |A ∩ C| − |B ∩ C|
             + |A ∩ B ∩ C|

This identity is not academic. If you deduplicate a corpus by computing len(set_a | set_b), you are applying inclusion–exclusion in one line — and if you instead wrote len(set_a) + len(set_b), you would overcount precisely by |A ∩ B|, silently inflating your training set and biasing every downstream evaluation metric.

3.3 Sets that appear constantly in practice

  • Sample space Ω — every outcome a random experiment could produce. The denominator of all probability.
  • Parameter space Θ — every weight configuration the model could occupy. Training is a walk through this set.
  • Support — the set of inputs where a distribution is non-zero. A softmax over k classes has support of size k.
  • Subset A ⊆ B — a training split is a subset of the full dataset; the test split must be a disjoint subset.

4. Tuples, Relations and Cartesian Products

A tuple is an ordered collection. This is the crucial difference from a set: (1, 2) ≠ (2, 1), whereas {1, 2} = {2, 1}. Machine-learning data is almost always tuple-valued, because feature order carries meaning — swapping two columns of a feature matrix changes the model.

The Cartesian product A × B is the set of all ordered pairs with the first element from A and the second from B:

A = {1, 2},  B = {a, b}
A × B = {(1,a), (1,b), (2,a), (2,b)}       |A × B| = |A| · |B|

A relation from A to B is any subset of A × B. A function is a relation with an extra constraint: every element of A appears in exactly one pair. That single constraint is the entire distinction between a relation and a function, and it is what makes functions computable — given an input, there is never ambiguity about which output to return.

Concept Ordered? Duplicates? Function Constraint
Set No Dropped —
Tuple Yes Allowed —
Relation Yes Not applicable None
Function Yes Not applicable Exactly one output per input

5. Functions: Domain, Codomain, Image and Composition

A function f : X → Y assigns to every element of the domain X exactly one element of the codomain Y. The set of values actually produced is the image (or range), which may be a proper subset of the codomain.

Function as a mapping from domain X to codomain Y, showing multiple inputs may share an output
Figure 1.4: A function need not be one-to-one. Many inputs may map to the same output — which is exactly why softmax can assign the same probability to different logit gaps.

5.1 The three properties that matter

Property Meaning Consequence for AI
Injective (one-to-one) Distinct inputs give distinct outputs Information is preserved; an inverse can exist on the image.
Surjective (onto) Every codomain element is hit Decoder can reach every output pattern; generative coverage.
Bijective Both of the above Invertible — the basis of normalising flows and reversible layers.

5.2 Composition is the shape of a neural network

Given g : X → Y and f : Y → Z, the composition (f ∘ g)(x) = f(g(x)) applies g first. A four-layer network is nothing more than a four-fold composition:

# A network is function composition, written left-to-right in code
# but right-to-left in mathematical notation.
#
#   ŷ = f₄( f₃( f₂( f₁(x) ) ) )   ==   (f₄ ∘ f₃ ∘ f₂ ∘ f₁)(x)

import numpy as np

W1 = np.random.randn(784, 256) * 0.01; b1 = np.zeros(256)
W2 = np.random.randn(256, 128) * 0.01; b2 = np.zeros(128)
W3 = np.random.randn(128, 64)  * 0.01; b3 = np.zeros(64)
W4 = np.random.randn(64, 10)   * 0.01; b4 = np.zeros(10)
x  = np.random.randn(784)

def relu(h):
    return np.maximum(0.0, h)

def softmax(h):
    e = np.exp(h - h.max(axis=-1, keepdims=True))
    return e / e.sum(axis=-1, keepdims=True)

def layer1(x): return x @ W1 + b1       # affine
def layer2(h): return relu(h @ W2 + b2) # affine + nonlinearity
def layer3(h): return relu(h @ W3 + b3)
def layer4(h): return softmax(h @ W4 + b4)

y_hat = layer4(layer3(layer2(layer1(x))))
print(y_hat.shape, y_hat.sum().round(6))   # (10,) 1.0

This is the reason the chain rule (Chapter 21) is the single most important theorem in deep learning. Differentiating a composition requires differentiating each factor in order and multiplying. Backpropagation is the chain rule applied to the composition graph, computed from the output backwards.

5.3 Inverse functions and why most layers are not invertible

An inverse f⁻¹ exists only when f is bijective. ReLU — max(0, x) — is not injective, because every negative input maps to 0. That single fact is why a standard network cannot be run backwards to recover its input, and why architectures that do need reversibility (normalising flows, invertible residual networks) replace ReLU with strictly monotonic activations.

6. The Notation Ladder: Scalars, Vectors, Matrices, Tensors

Rank is the number of axes an array possesses. It is the single most useful fact you can attach to a symbol, because almost every runtime error in deep learning is a rank or axis-alignment error wearing a different costume.

Notation ladder from scalar to vector to matrix to tensor with ranks, shapes and example arrays
Figure 1.5: The notation ladder. Every symbol in a paper sits on exactly one rung, and its shape is implied by that rung.
Rung Typographic Convention Example in a Model
Rank 0 — scalar lowercase italic x learning rate η, a single loss value
Rank 1 — vector bold lowercase x one embedding, one feature row, one bias vector
Rank 2 — matrix bold uppercase W a weight matrix, a batch of feature rows
Rank 3+ — tensor calligraphic 𝒳 or bold uppercase attention scores, a batch of images

A shape declaration should always be read outward from the operation. In y = Wx with W ∈ ℝ^(4×3) and x ∈ ℝ³, the inner dimensions must agree (3 = 3) and the result inherits the outer ones, giving y ∈ ℝ⁴. In the batched form Y = XW^T with X ∈ ℝ^(B×3), the batch dimension B passes through untouched: Y ∈ ℝ^(B×4).

# Reading shapes outward from the operation
# y = W x      W:(4,3)  x:(3,)      -> y:(4,)        inner dims agree
# Y = X Wᵀ     X:(B,3)  W:(4,3)     -> Y:(B,4)       batch dim preserved
# scores = Q Kᵀ / √d    Q:(B,H,T,d) K:(B,H,T,d) -> (B,H,T,T)

import numpy as np

W = np.zeros((4, 3));  x = np.zeros((3,))
X = np.zeros((8, 3))

assert W.shape[1] == x.shape[0], "inner dimensions must agree"
Y = X @ W.T
assert Y.shape == (8, 4), "batch dimension must be preserved"
print("y:", (W @ x).shape, "Y:", Y.shape)

7. Summation, Product and Index Notation

Summation notation is the compression scheme that turns a loop into a single expression. Every Σ you meet in a paper has exactly one honest translation into array code, and learning that translation is worth more than memorising a hundred formulas.

Anatomy of summation notation with end index, start index, term body, and equivalent array reductions
Figure 1.6: Anatomy of a summation, and the array reduction each form corresponds to.

7.1 The three symbols you need

Symbol Name Meaning
Σ Sigma Add the terms: Σᵢ₌₁ⁿ xᵢ = x₁ + … + xₙ
Π Pi Multiply the terms: Πᵢ₌₁ⁿ pᵢ — the joint probability of independent events
∫ Integral Continuous summation — expectation over a density

7.2 Nested sums and their array equivalents

import numpy as np

A = np.array([[1., 2.], [3., 4.]])
B = np.array([[5., 6.], [7., 8.]])

# Σᵢ Σⱼ A[i,j]              -> one scalar: total of every element
total = A.sum()

# Σⱼ A[i,j]                 -> per-row sum, collapses axis 1, result shape (2,)
row_sums = A.sum(axis=1)

# Σᵢ A[i,j]                 -> per-column sum, collapses axis 0, result shape (2,)
col_sums = A.sum(axis=0)

# Σⱼ A[i,j] · B[j,k]        -> matrix product, shape (2,2)
matmul = A @ B

# Σᵢ A[i,j] · B[i,j]        -> elementwise product then total: the Frobenius inner product
frob = (A * B).sum()

print(total, row_sums, col_sums, frob)
# 10.0 [3. 7.] [4. 6.] 70.0

Two rules make nested sums decodable. First, each summation sign removes exactly one axis, so the number of Σs tells you the rank drop. Second, the order of summation does not matter for a finite sum — you may evaluate whichever index is cheapest first, which is precisely the freedom that lets a compiler reorder reductions and lets you choose between einsum, matmul, or a broadcasted multiply.

7.3 Einstein summation

Einstein notation drops the Σ entirely by the convention that any index appearing twice in a term is summed over. It is the native language of tensor libraries:

import numpy as np

A = np.random.rand(2, 3)
B = np.random.rand(3, 4)

# Σⱼ A[i,j] · B[j,k]   is written   A_ij B_jk   in Einstein notation
result = np.einsum('ij,jk->ik', A, B)   # identical to A @ B

# Batched attention scores: Σ_d Q[b,h,i,d] · K[b,h,j,d]
B, H, T, d = 2, 4, 8, 16
Q = np.random.rand(B, H, T, d)
K = np.random.rand(B, H, T, d)
scores = np.einsum('bhid,bhjd->bhij', Q, K)
print(scores.shape)   # (2, 4, 8, 8)

8. Greek Symbols and Paper Conventions

Research papers use Greek letters with remarkable consistency. Learning the conventional mappings removes most of the friction from reading them.

Symbol Name Standard Meaning
θ theta All trainable parameters, taken as one vector
η eta Learning rate
λ lambda Regularisation strength; also an eigenvalue
σ sigma Sigmoid; standard deviation; singular value
μ mu Mean of a distribution
∇ nabla Gradient operator — vector of partial derivatives
ε epsilon Small constant for numerical stability; also noise
∂ partial Partial derivative, holding other variables fixed

8.1 Decorative notation that carries real information

  • Hat ŷ — a prediction or estimate, as opposed to the observed truth y.
  • Bar x̄ — the arithmetic mean of a collection.
  • Subscript xᵢ — the i-th element of a collection; not "x times i".
  • Superscript x⁽ˡ⁾ — the value at layer l. Beware: this is not exponentiation. Context disambiguates, and layer superscripts are almost always parenthesised.
  • T Wᵀ — transpose: axes swapped. For W ∈ ℝ^(m×n), Wᵀ ∈ ℝ^(n×m).

9. Reading a Model Equation Aloud

The test of notation literacy is whether you can narrate an equation. Take the workhorse of the field:

ŷ = softmax(W x + b)

"y-hat equals softmax of W times x plus b"

ŷ        the predicted probability vector over classes
W        a weight matrix of shape (n_classes, n_features)
x        the input feature vector of shape (n_features,)
W x      a matrix-vector product: mixes every feature into every class score
b        a bias vector of shape (n_classes,), one offset per class
W x + b  the affine map — a linear transform followed by a translation
softmax  normalises the raw scores into a probability distribution

And its batched, deep counterpart — the form most papers actually write:

H⁽ˡ⁺¹⁾ = σ( H⁽ˡ⁾ Wᵀ⁽ˡ⁾ + b⁽ˡ⁾ )

H⁽ˡ⁾     activations at layer l, shape (B, d_l)   — B is the batch dimension
W⁽ˡ⁾     weight matrix of layer l, shape (d_{l+1}, d_l)
σ        an elementwise nonlinearity applied independently to each entry
H⁽ˡ⁺¹⁾   activations at layer l+1, shape (B, d_{l+1})

Note that σ is applied elementwise, so it never changes shape. Only the matrix product changes shape, and it changes exactly one axis — the feature axis. The batch axis passes through the entire network untouched. That single observation eliminates the majority of shape bugs in practice.

10. Worked Examples

10.1 Set cardinality with overlap

A training set has 4,800 reviews. 3,200 are positive, 2,100 are negative, and 500 are labelled both (multi-label sentiment). How many distinct reviews exist?

|P ∪ N| = |P| + |N| − |P ∩ N| = 3200 + 2100 − 500 = 4800 ✓

The identity is consistent with the stated total. If the union had come out below 4,800, the labels would contain duplicates; above, some reviews would be unlabelled.

10.2 Determining whether a relation is a function

Given R = {(1,2), (2,4), (3,6), (1,8)} on {1,2,3} → {2,4,6,8}: is R a function?

No. The input 1 appears twice, mapped to 2 and to 8. A function requires exactly one output per input. Removing either pair makes it a function; this is precisely the constraint a database primary key enforces.

10.3 Composing three layers and tracking shape

Let x ∈ ℝ^(784), W₁ ∈ ℝ^(256×784), W₂ ∈ ℝ^(10×256). Compute the output shape.

h  = σ(W₁x)     (256×784)·(784)  -> (256)
ŷ  = softmax(W₂h) (10×256)·(256) -> (10)
784 → 256 → 10        two matrix products, two axis changes, no others.

10.4 Evaluating a nested sum by hand

Compute Σᵢ₌₁² Σⱼ₌₁³ (i + j).

i=1:  (1+1)+(1+2)+(1+3) = 2+3+4 = 9
i=2:  (2+1)+(2+2)+(2+3) = 3+4+5 = 12
total = 9 + 12 = 21

Two summation signs, and the result is rank 0 — a scalar. Each Σ removed one axis.

10.5 Rescaling a loss by the summation formulation

Express mean squared error over n samples as a summation, then as an array reduction.

MSE = (1/n) Σᵢ₌₁ⁿ (ŷᵢ − yᵢ)²

Python:  ((y_hat - y) ** 2).mean()
NumPy:   np.mean((y_hat - y) ** 2)

The 1/n outside the sum is what converts a total into an average. Move it inside and you have a per-sample loss vector instead of a scalar — a distinction that matters the moment you introduce sample weighting.

11. Verify It in Python

Every claim in this chapter is checkable in a few lines. Run this to confirm the identities rather than trusting them.

import numpy as np

# --- 1. Inclusion–exclusion identity -------------------------------
A = {f"review_{i}" for i in range(3200)}          # 3200 elements
B = {f"review_{i}" for i in range(2700, 4800)}     # 2100 elements, 500 overlap
lhs = len(A | B)
rhs = len(A) + len(B) - len(A & B)
assert lhs == rhs, (lhs, rhs)
print(f"union={lhs}  identity={rhs}  overlap={len(A & B)}")

# --- 2. A function has exactly one output per input -----------------
R = [(1, 2), (2, 4), (3, 6)]
is_function = len({a for a, _ in R}) == len(R)
print("is_function:", is_function)          # True

# --- 3. Shape propagation through a stack --------------------------
x  = np.random.randn(784)
W1 = np.random.randn(256, 784)
W2 = np.random.randn(10, 256)

h = np.maximum(0.0, W1 @ x)                 # (256,)
y = W2 @ h                                  # (10,)
assert h.shape == (256,) and y.shape == (10,)
print("shapes:", h.shape, y.shape)

# --- 4. Nested summation, both signs -------------------------------
manual = sum((i + j) for i in range(1, 3) for j in range(1, 4))
assert manual == 21
print("nested sum:", manual)

# --- 5. Sigmoid numerics: why epsilon exists -----------------------
def naive_sigmoid(z):
    return 1.0 / (1.0 + np.exp(-z))

for z in (-800.0, 800.0):
    with np.errstate(over="raise"):
        try:
            print(z, naive_sigmoid(np.float64(z)))
        except FloatingPointError:
            print(z, "FloatingPointError: np.exp overflowed to inf")

The final block is the first practical reason for the stability constants met throughout this series. For sufficiently negative z, np.exp(-z) overflows to infinity and the division underflows to zero — silently, in most vectorised code. Chapter 47 returns to this problem with the numerically stable formulation.

12. Practice Questions

  1. Let A = {1, 2, 3, 4} and B = {3, 4, 5}. Compute A ∪ B, A ∩ B, A \ B, and A × B, and state the cardinality of each.
  2. Why is the relation {(a,1), (b,2), (c,2)} a valid function while {(a,1), (a,2)} is not, even though both are subsets of a Cartesian product?
  3. A layer maps ℝ^(512) → ℝ^(128). State the exact shape of its weight matrix and bias vector, then give the batched shapes for a batch of B = 32.
  4. Translate Σᵢ₌₁ⁿ Σⱼ₌₁ᵐ C[i,j] into NumPy, then translate Σₖ A[i,k] B[k,j] and name the operation.
  5. Explain why applying an activation function elementwise can never change a tensor's shape, while a matrix product always can.
  6. Given that ReLU is not injective, describe one architectural workaround that restores invertibility for a reversible network.
  7. Expand Πᵢ₌₁³ (1 − pᵢ) and interpret it probabilistically for three independent events.
  8. In H⁽ˡ⁺¹⁾ = σ(H⁽ˡ⁾Wᵀ⁽ˡ⁾ + b⁽ˡ⁾), which symbol encodes the layer index and how do you know it is not an exponent?
  9. You observe Y ∈ ℝ^(32×768×128) after a projection from X ∈ ℝ^(32×768×64). Which axis changed, and what shape was the weight matrix?
  10. Why does the inclusion–exclusion identity matter when deduplicating a training corpus, and what specific metric gets biased if it is ignored?

Leave a Reply

Your email address will not be published. Required fields are marked *