Vector spaces, bases, rank, and identifiability

A matrix equation can determine its output while leaving several input coordinates interchangeable. A model containing both Celsius and Fahrenheit measurements plus an intercept supplies a concrete example: those three columns cannot carry three independent directions of information. Different weights can agree on every physically valid temperature record. A different design can agree only on the observed training rows and disagree on the next record. Distinguishing these claims is the purpose of identifiability.

This lesson turns the pivot calculations from Module 10 into statements about spaces, coordinates and information. You will prove closure, separate spanning from independence, compute all four fundamental spaces, and derive rank–nullity rather than memorising a dimension equation. The final section connects exact algebra with numerical rank policies and parameter interpretation.

Retrieval check: explain a reversible augmented-row operation and the meaning of a free variable. Revisit Module 10 and proof structure if needed. Use NumPy preparation for the numerical labs. All spaces and scalars here are real; the labs deliberately distinguish exact rational arithmetic from floating-point decisions.

A vector space is a set V equipped with addition and scalar multiplication satisfying the familiar laws of coordinate algebra. Addition is associative and commutative, there is an additive identity 0, and every vector has an additive inverse. Scalars distribute over vector addition and their own addition; multiplying scalars successively agrees with their product, and multiplication by one changes nothing. Both operations must stay inside V. Specifying a set alone does not specify a vector space: the operations and scalar field matter too.

Real coordinate space satisfies these laws component by component. Vectors need not be arrows or arrays. The real polynomials of degree at most two form a vector space under ordinary polynomial addition and scaling: coefficients add and scale, and these operations do not introduce a cubic term. The zero polynomial belongs despite having no ordinary finite degree. Real functions on a fixed domain form another space under pointwise operations. Here the zero vector is the zero function, and equality means agreement at every domain point, not merely agreement at sampled inputs.

A subspace W of V uses the same operations as V and is itself a vector space. A convenient sufficient and necessary test is: W is nonempty and, for every u,v in W and every real a,b, the combination au+bv belongs to W. Necessity follows from closure under the two operations. For sufficiency, nonemptiness supplies some u; choosing both coefficients zero supplies 0, choosing a=−1 gives negatives, and appropriate choices give addition and scaling. The remaining algebraic laws are inherited from V, since the operations are unchanged. This argument explains why an empty set cannot pass the test by vacuous closure alone.

Many solution sets are subspaces because a homogeneous linear equation respects combinations. If L is linear, L(au+bv)=aL(u)+bL(v). Thus the set of vectors satisfying L(v)=0 contains zero and is closed. Intersections of such sets are subspaces: a combination of two members satisfies every shared homogeneous constraint. This also shows that the intersection of any collection of subspaces is a subspace. Their union need not be one. A vector on the horizontal axis plus a vector on the vertical axis can leave the union of the two axes.

The distinction between a homogeneous subspace and an affine translate is important for feasible sets and model constraints. Given one solution p of Av=b, every other solution differs from p by an element of the kernel: A(v−p)=0. Conversely p+n solves the system for every kernel element n. That statement describes the entire solution set, not merely a convenient way to produce a few examples. A consistent nonhomogeneous system may happen to contain zero only when b=0; otherwise its solution set cannot be a subspace.

Do not replace the subspace test with the phrase “looks flat.” A ray is straight but fails closure under negative scaling. A bounded line segment fails closure under arbitrary scaling. A sphere fails to contain zero. Two intersecting lines through zero fail closure under addition unless they are the same line. A proof must cover every pair and scalar, whereas one counterexample suffices to disprove closure. In feature engineering, the nonnegative orthant is closed under nonnegative combinations but is not a real vector subspace; that different structure is useful and should receive its own assumptions.

The span of vectors v₁,…,vₖ is the set of every finite combination a₁v₁+…+aₖvₖ with real coefficients. It is a subspace: zero coefficients supply zero, and adding or scaling combinations simply changes their coefficients. It is also the smallest subspace containing the listed vectors. Any subspace containing them must contain every combination by closure. This gives span a constructive meaning: it describes what directions the representation can express, while the coefficients describe how a particular vector is expressed.

A list is linearly independent when the only combination giving zero is the one with every coefficient zero. It is dependent when some nonzero coefficient vector gives zero. “Nonzero coefficient vector” means at least one coefficient differs from zero, not that every coefficient must do so. Rearranging a dependence relation around any nonzero coefficient expresses that listed vector as a combination of the others. Conversely such an expression yields a dependence relation. Hence dependence is exactly a redundancy in a nonempty list. A list containing the zero vector is dependent, and a repeated vector creates dependence even when that vector is nonzero.

The definition handles boundary cases consistently. The empty list is independent, since there is no nontrivial coefficient choice. Its span is the zero subspace, using the empty sum convention. A one-vector list containing a nonzero vector is independent: av=0 forces a=0 by a nonzero coordinate, or by multiplying by 1/a if a≠0. Two nonzero vectors can be dependent without being identical; one can be any scalar multiple of the other. Independence is a collective property, so testing pairs does not establish independence of three or more vectors.

For a matrix A with these vectors as columns, a coefficient vector c satisfies Ac=0 exactly when its coefficients give a dependence relation. This connects the definition directly to elimination: a nonzero solution of the homogeneous system certifies dependence. If every column has a pivot, there are no free coefficient variables and the columns are independent. If a free column exists, assigning its free variable one and the others zero creates a nonzero relation. The computation supplies a witness, whereas a bare numerical “rank” value can hide which columns are redundant.

To find a spanning independent subset, process a finite generating list in order. Keep a vector only if it is outside the span of those already kept. The kept list remains independent: a new nontrivial relation with a nonzero new coefficient would express that new vector using the old ones. Each rejected vector is already represented, so the final kept list spans every generator and therefore the original span. This finite greedy argument proves existence of a basis for a finitely generated space. It does not promise a uniquely preferred subset; ordering and the intended interpretation can influence the choice.

The span concerns exact representability, not automatically similarity, physical meaning or predictive quality. Two feature columns can be algebraically independent yet badly scaled or nearly redundant. Conversely a dependent feature can be a useful named quantity while providing no additional linear direction. Removing redundancy can clarify parameters, but a claim that a feature is unnecessary for every future task would require more than a dependence calculation for one chosen representation and sample.

A basis is a spanning independent list. Spanning gives an expression for every vector, and independence makes that expression unique: subtracting two expressions for the same vector produces a zero combination, so their coefficients agree term by term. Both conditions are needed. A redundant spanning list can have many coordinate descriptions of the same vector. An independent list that does not span cannot describe all vectors in the space. Basis coordinates therefore depend on both the space and the selected ordered basis.

If B=(b₁,…,b_d) is a basis, write [v]_B for the coefficient tuple in v=Σcᵢbᵢ. A vector and its coordinate tuple are related by this basis map, but they are not interchangeable without identifying B. With the coordinate unit basis in Rᵈ, the distinction is easily hidden because the tuple entries coincide. For a different basis the tuple changes while the represented vector stays the same. Reordering basis vectors changes the coordinate order. Scaling one basis vector requires an inverse scaling of its corresponding coordinate.

Why is the size of a basis well defined? We need a finite exchange argument, not a diagram. Let b₁,…,b_d span V and let u₁,…,u_k be independent. Express u₁ using the spanning list; because it is nonzero, at least one coefficient is nonzero. Solve for that corresponding b and replace it by u₁. The new d-vector list still spans. Suppose the first j−1 independent vectors have already been inserted. Express u_j in the current spanning list. At least one coefficient on a remaining original vector must be nonzero, since otherwise u_j would lie in the span of earlier u’s, contradicting independence. Replace that original vector using the same rearrangement. At each stage one remaining place is used, so the process cannot insert more than d independent vectors. Therefore k≤d.

Apply this bound to two bases in both directions. The first independent basis has size no greater than the second spanning basis, and vice versa. Their sizes are equal. This common finite size is the dimension of V. The zero space has dimension zero and the empty basis. A subspace of a d-dimensional space has dimension at most d, because any independent list in the subspace is also independent in the ambient space. An independent list with d vectors in a d-dimensional space must span: otherwise adding a vector outside its span would create d+1 independent vectors, contradicting the bound.

The same argument proves basis extension. Start with any independent list in a finite-dimensional space and inspect an existing finite spanning list. Add a generator whenever it is outside the current span. Independence is preserved, and after the generators have been processed the list spans. Thus a basis of a subspace can be extended to a basis of its ambient space. We will use this existence proof in rank–nullity. In computations, the extension can be found through pivots; in a theorem it supplies a logical bridge between local constraints and global dimensions.

For two bases B and C of the same d-dimensional space, the coordinate-change matrix T has columns [bᵢ]_C. Then [v]_C=T[v]_B by linearity of combinations, and T is invertible because both coordinate descriptions are unique. A map’s matrix also depends on the input and output bases. If input coordinates change by P and output coordinates by Q relative to the original coordinate systems, its new matrix is Q⁻¹AP. For an endomorphism using the same change on both sides this becomes P⁻¹AP. These are descriptions of one map; changing A without the accompanying convention can instead change the map itself.

For an m×n matrix A, the column space C(A) is the span of its columns in Rᵐ, equivalently the image {Ax:x∈Rⁿ}. The row space R(A) is the span of its rows, regarded as vectors in Rⁿ. The null space N(A)={x∈Rⁿ:Ax=0} lives in the input space. The left null space N(Aᵀ)={z∈Rᵐ:Aᵀz=0} lives in the output space. The word left refers to zᵀA=0; it does not mean a fifth type of matrix operation. Their ambient dimensions matter before any calculation, especially for a rectangular matrix.

Reversible row operations preserve the row space, since each new row is a combination of old rows and the inverse operations give the reverse inclusion. They preserve the null space because they preserve the homogeneous equation’s solutions. They usually change the column space as a subset of Rᵐ. If E is the invertible product of row-operation matrices, the reduced matrix is EA and its column space is E applied to the old column space. Equal dimensions do not make these two sets equal. This is why pivot columns selected in a reduced matrix must be taken from the original matrix when constructing a basis for C(A).

Here is the exact justification for that rule. In echelon form, the pivot columns are independent: inspecting pivot rows from the last to the first forces each coefficient of a zero combination to vanish. Every nonpivot column is their combination. One can solve its coefficients by back substitution on the pivot-row triangular system; rows below the pivots are zero. Thus the pivot columns form a basis for the reduced column space. Applying E⁻¹ to the relation and to the independence claim gives the same column indices as a basis for the original column space. The number of pivots is consequently the column-space dimension.

The nonzero echelon rows are independent, since their distinct leading positions force a zero combination’s coefficients to vanish from the first leading position onward. They span the echelon row space, which equals the original row space. Therefore the row-space dimension is the same pivot count. We call this common value the rank r. This proves equality of row rank and column rank rather than treating the two numbers as an unexplained coincidence. It also yields r≤min(m,n), including zero matrices with rank zero.

Each null vector is perpendicular to every row, because those dot products are the entries of Ax. Each left-null vector is perpendicular to every column, because those dot products are the entries of Aᵀz. These orthogonality statements follow directly from dot products. In particular, if Ax=b, any left-null z must satisfy zᵀb=zᵀAx=0. A nonzero value is an inconsistency certificate. For the example, b=(2,3,5) meets the condition and b=(2,3,6) fails it by one.

The left-null condition is also sufficient. Reduce [A|b] using E. If no solution exists, echelon form contains a row with zero coefficients and a nonzero constant. The corresponding row zᵀ of E satisfies zᵀA=0 and zᵀb≠0. Therefore if every left-null vector is perpendicular to b, no such contradiction exists, and b belongs to the column space. This proves C(A)=N(Aᵀ)⊥. Applying the argument to Aᵀ also proves R(A)=N(A)⊥. Projection formulas for these perpendicular spaces come in Module 12; this lesson already explains their algebraic role in solvability.

The nullity of a linear map is the dimension of its kernel. Rank–nullity states that for a linear map A from an n-dimensional input space, rank(A)+dimN(A)=n. It divides input degrees of freedom into directions that change the output and directions that disappear. The output space can have a different dimension m; left nullity is m−r, not n−r. Confusing these two counts can make a rectangular system look consistent or uniquely solvable when it is neither.

We can prove the theorem without counting free variables as an unexplained rule. Choose a kernel basis k₁,…,k_q and extend it to an input basis k₁,…,k_q,u₁,…,u_{n−q}, using the finite extension argument above. We claim that Au₁,…,Au_{n−q} form a basis of the image. For spanning, express any input x in the extended basis. Applying A removes its kernel terms, so Ax is a combination of the Au’s. For independence, suppose ΣcᵢAuᵢ=0. Then Σcᵢuᵢ lies in the kernel and has an expression Σdⱼkⱼ. Subtracting these two expressions gives a zero combination of the entire extended basis. Its independence forces every cᵢ and dⱼ to vanish. Hence the image list is independent and has n−q vectors. Its size is the rank, giving r+q=n. No assumption of a square or invertible matrix was needed.

Elimination supplies a computational version of this proof. With r pivot variables and n−r free variables in a homogeneous system, set one free coordinate to one and the others to zero and solve the pivot variables. Do this for each free coordinate. The resulting n−r vectors are independent because their free-coordinate positions contain distinct unit vectors. They span because any free-coordinate assignment is that same combination of these particular assignments, and the pivot equations determine all remaining coordinates. Thus they form a kernel basis. The method describes every null vector, unlike choosing a few arbitrary examples that merely happen to satisfy Ax=0.

These facts recover complete system classification. A target has a solution precisely when it lies in C(A). If it has one solution, the family is p+N(A); it is unique exactly when N(A)={0}, equivalently r=n. Every possible target in Rᵐ is attainable exactly when C(A)=Rᵐ, equivalently r=m. Both properties hold together only for a square full-rank matrix. A tall full-column-rank matrix can identify a unique input for a compatible target while leaving many targets incompatible. A wide full-row-rank matrix can reach every target but retains at least n−m null directions. “Full rank” alone must identify which dimension is limiting.

The orthogonality relations also separate a general input x into a row-space component and a kernel component. Their intersection contains only zero: a vector in both is perpendicular to the row space while belonging to it, so its dot product with itself is zero. The dimensions sum to n by rank–nullity. Joining bases of the two spaces gives n independent vectors, hence an input-space basis. Every x therefore has a unique decomposition x=u+k with u∈R(A),k∈N(A). A maps u and x to the same output. Its restriction to the row space is injective, and these row-space inputs reach the whole column space. The map loses exactly the kernel information.

In constraint solving, null vectors describe permissible changes after one solution has been found. In learning, they describe parameter changes that the design matrix cannot distinguish. In compression or sensing, they describe signals producing the same measurements. These are algebraically the same statement with different interpretations of x. A domain restriction can change identifiability: if inputs are required to belong to a special set, two admissible inputs must differ by a null vector for ambiguity, but not every null displacement remains admissible. This lesson’s uniqueness statements concern unrestricted real inputs unless another set is explicitly stated.

For a fixed design matrix X, linear predictions are Xw. Two weights w and w′ produce the same predictions exactly when X(w′−w)=0. Thus their difference lies in N(X), and all weights equivalent to one w form w+N(X). Parameter identifiability for this unconstrained linear representation means different weights give different predictions on the specified records; it holds exactly when X has full column rank. Unique fitted predictions can coexist with nonunique weights. An algorithm may choose one representative without proving that the data identified it.

An intercept plus all one-hot columns for a single categorical variable has the same issue. If exactly one of K category indicators equals one on each record, their sum is the intercept column. The null direction is (−1,1,…,1). Dropping one indicator creates a baseline-category representation. Its intercept is the baseline prediction and each retained coefficient is a difference from that baseline. Removing the intercept instead gives a separate category coefficient for every category. These parameterisations can represent the same prediction functions with different coefficient meanings. Full rank still requires that the observed records cover the retained categories; a missing category can create additional ambiguity.

Equivalent training predictions do not always mean equivalent functions on new records. For features (1,x,x²) observed only at x=0,1, the last two columns are equal. The weight change (0,−1,1) leaves those predictions unchanged but changes a prediction at x=2 by −2+4=2. By contrast, the Celsius–Fahrenheit relation is a structural feature identity on the stated domain of valid temperature conversions, so its null direction remains invisible there. A faulty sensor or a different feature-generation rule can violate even that domain assumption. State whether a dependence holds on the sample, on a declared input domain, or on all possible real feature tuples.

Exact rank asks whether a relation is precisely zero in the mathematical model. Numerical rank asks which directions are distinguishable at a stated resolution. The matrix [[1,1],[1,1+δ]] has determinant δ and exact rank two whenever δ≠0. With δ=10⁻¹², it is close to the dependent δ=0 case. A small input or measurement perturbation can alter whether that tiny difference is practically resolvable. Thresholding a floating-point computation can deliberately report one effective direction without proving that the exact determinant vanished.

NumPy’s pinned rank interface uses singular values, nonnegative direction-strength quantities developed in Module 14, and counts values above a tolerance. Its default scales with the largest value, matrix dimensions and machine precision. A user-specified absolute cutoff has different units and can change its conclusion when the entire matrix is scaled. In Lab 3, tolerances 10⁻¹⁴ and 10⁻¹⁰ give numerical ranks two and one for the same near-dependent input. Multiplying by a million with the same absolute cutoff changes the decision; scaling the cutoff proportionally preserves this example’s decision. Record the dtype, feature units, tolerance and purpose. Choosing a threshold should reflect computational or measurement resolution, rather than silently asserting an exact theorem.

Removing an exactly redundant feature or imposing a constraint can choose a unique representation, but the resulting coefficient meaning depends on that choice. A minimum-norm convention and regularisation introduce further criteria, developed in later lessons. They can stabilise or select parameters without recovering information that the observations never distinguished. Independence also establishes no causal effect, uncertainty bound or generalisation guarantee. Those questions require statistical and modelling assumptions beyond this algebra. The useful habit is to state precisely what the matrix determines, what it leaves free, and on which domain the interpretation is intended.

Use Python 3.11 and the NumPy environment from array preparation. The first lab uses standard-library Fractions. The displayed output is captured from the downloadable scripts at build time. Keep every shape, null witness and declared tolerance in your lab notes.

Lab A Exact spaces and certificates

Predict the pivot indices and all four space dimensions before running. Trace how each free variable generates one null basis vector. Check every basis vector by multiplication. Compare the original and reduced pivot columns. Explain why the left-null certificate rejects the second target but accepts the first in this particular example.

Download lab1_exact_spaces.py

"""Exact rational pivots and four spaces; no floating-point zero decisions."""
from fractions import Fraction

def rref(matrix):
    if not matrix or not matrix[0] or any(len(r) != len(matrix[0]) for r in matrix):
        raise ValueError("Need a nonempty rectangular matrix")
    rows = [[Fraction(v) for v in row] for row in matrix]
    pivots, active = [], 0
    for column in range(len(rows[0])):
        selected = next((i for i in range(active, len(rows)) if rows[i][column]), None)
        if selected is None:
            continue
        rows[active], rows[selected] = rows[selected], rows[active]
        pivot = rows[active][column]
        rows[active] = [v / pivot for v in rows[active]]
        for i in range(len(rows)):
            if i != active:
                factor = rows[i][column]
                rows[i] = [a - factor * b for a, b in zip(rows[i], rows[active])]
        pivots.append(column)
        active += 1
        if active == len(rows):
            break
    return rows, pivots

def null_basis(matrix):
    rows, pivots = rref(matrix)
    free = [j for j in range(len(rows[0])) if j not in pivots]
    basis = []
    for j in free:
        vector = [Fraction(0)] * len(rows[0])
        vector[j] = 1
        for i, p in enumerate(pivots):
            vector[p] = -rows[i][j]
        basis.append(vector)
    return basis

def matvec(matrix, vector):
    return [sum(a * b for a, b in zip(row, vector)) for row in matrix]

if __name__ == "__main__":
    A = [[1, 0, 1], [0, 1, 1], [1, 1, 2]]
    reduced, pivots = rref(A)
    column_basis = [[row[j] for row in A] for j in pivots]
    row_basis = reduced[:len(pivots)]
    kernel = null_basis(A)
    transpose = [list(col) for col in zip(*A)]
    left_kernel = null_basis(transpose)
    strings = lambda rows: [[str(v) for v in row] for row in rows]
    print("RREF:", strings(reduced))
    print("Original pivot columns:", column_basis)
    print("Row-space basis:", strings(row_basis))
    print("Null / left-null bases:", strings(kernel), strings(left_kernel))
    assert all(matvec(A, v) == [0, 0, 0] for v in kernel)
    assert all(matvec(transpose, v) == [0, 0, 0] for v in left_kernel)
    assert len(pivots) == 2 and len(kernel) == 1
    print("Rank / nullity / left nullity:", len(pivots), len(kernel), len(left_kernel))
    for target in [[2, 3, 5], [2, 3, 6]]:
        certificate = sum(a*b for a, b in zip(left_kernel[0], target))
        print("Target / left-null certificate:", target, str(certificate), "consistent" if certificate == 0 else "inconsistent")
    for matrix in [[[0, 0], [0, 0]], [[1, 0], [0, 1]], [[1, 2], [2, 4]]]:
        _, p = rref(matrix)
        assert len(p) + len(null_basis(matrix)) == len(matrix[0])
    print("Zero, identity, and dependent cases satisfy exact rank-nullity.")

RREF: [['1', '0', '1'], ['0', '1', '1'], ['0', '0', '0']]
Original pivot columns: [[1, 0, 1], [0, 1, 1]]
Row-space basis: [['1', '0', '1'], ['0', '1', '1']]
Null / left-null bases: [['-1', '-1', '1']] [['-1', '-1', '1']]
Rank / nullity / left nullity: 2 1 1
Target / left-null certificate: [2, 3, 5] 0 consistent
Target / left-null certificate: [2, 3, 6] 1 inconsistent
Zero, identity, and dependent cases satisfy exact rank-nullity.

Deliverable: a table of the four spaces, ambient dimensions, bases and dimensions, plus a proof that these bases span all the claimed spaces. The successful multiplications alone prove membership; the pivot/free-variable argument proves completeness.

Lab B Identical predictions different weights

Compute both weight vectors by hand. Check the unseen temperature within the valid feature domain, then compare the polynomial feature coincidence. Identify the null direction in the one-hot design and explain two different ways to remove that redundancy.

Download lab2_equivalent_features.py

"""Prediction equivalence can be structural or restricted to observed records."""
import numpy as np

np.set_printoptions(precision=6, suppress=True)
celsius = np.array([0., 10., 20.])
X = np.column_stack((np.ones(3), celsius, 1.8*celsius + 32))
w = np.array([5., 2., -1.])
n = np.array([-32., -1.8, 1.])
alternative = w + 0.5*n
assert np.allclose(X@n, 0) and np.allclose(X@w, X@alternative)
print("Temperature weights / alternative:", w, alternative)
print("Training predictions:", X@w, X@alternative)
new = np.array([1., 30., 86.])
print("New valid temperature prediction:", new@w, new@alternative)
assert np.allclose(new@w, new@alternative)

categorical = np.column_stack((np.ones(3), np.eye(3)))
categorical_null = np.array([-1., 1., 1., 1.])
assert np.allclose(categorical@categorical_null, 0)
print("Intercept plus all one-hot columns rank:", np.linalg.matrix_rank(categorical), "of", categorical.shape[1])
print("One-hot null direction:", categorical_null)
assert np.linalg.matrix_rank(categorical[:, :3]) == 3

training = np.array([[1., 0., 0.], [1., 1., 1.]])
training_null = np.array([0., -1., 1.])
assert np.allclose(training@training_null, 0)
unseen = np.array([1., 2., 4.])
print("x and x squared: training change / unseen x=2 change:", training@training_null, unseen@training_null)
assert unseen@training_null == 2
print("A structural temperature identity generalises; a training-only polynomial coincidence does not.")

Temperature weights / alternative: [ 5.  2. -1.] [-11.    1.1  -0.5]
Training predictions: [-27. -25. -23.] [-27. -25. -23.]
New valid temperature prediction: -21.0 -21.0
Intercept plus all one-hot columns rank: 3 of 4
One-hot null direction: [-1.  1.  1.  1.]
x and x squared: training change / unseen x=2 change: [0. 0.] 2.0
A structural temperature identity generalises; a training-only polynomial coincidence does not.

Deliverable: classify each equivalence as structural on a stated domain or limited to the observed rows. Give an unseen record that breaks the training-only claim, and avoid inferring causality from an identified coefficient.

Lab C Exact versus numerical rank

Use the exact determinant as a mathematical certificate, and the numerical library output as a policy-dependent measurement. Compare two absolute thresholds and one relative threshold under global scaling. Singular values are used as library diagnostics here; Module 14 will derive their decomposition and geometry.

Download lab3_rank_tolerances.py

"""An exact nonzero determinant and two numerical rank policies can coexist."""
from fractions import Fraction
import numpy as np

delta = Fraction("0.000000000001")
exact = [[Fraction(1), Fraction(1)], [Fraction(1), 1+delta]]
determinant = exact[0][0]*exact[1][1] - exact[0][1]*exact[1][0]
assert determinant == delta and determinant != 0
print("Exact determinant:", str(determinant), "exact rank: 2")
A = np.array([[1., 1.], [1., 1.+float(delta)]])
singular_values = np.linalg.svd(A, compute_uv=False)
print("Singular values:", [format(v, ".6e") for v in singular_values])
for tolerance in [1e-14, 1e-10]:
    rank = int(np.linalg.matrix_rank(A, tol=tolerance))
    print("Absolute tolerance / numerical rank:", tolerance, rank)
assert np.linalg.matrix_rank(A, tol=1e-14) == 2
assert np.linalg.matrix_rank(A, tol=1e-10) == 1
scaled = 1e6*A
print("Scaling by one million at absolute tolerance 1e-10:", np.linalg.matrix_rank(scaled, tol=1e-10))
assert np.linalg.matrix_rank(scaled, tol=1e-10) == 2
for label, matrix in [("A", A), ("scaled A", scaled)]:
    s = np.linalg.svd(matrix, compute_uv=False)
    relative_cutoff = 1e-10*s[0]
    print("Relative cutoff policy / rank:", label, int(np.linalg.matrix_rank(matrix, tol=relative_cutoff)))
    assert np.linalg.matrix_rank(matrix, tol=relative_cutoff) == 1
print("The cutoff is a numerical resolution policy, not an exact dependence proof.")

Exact determinant: 1/1000000000000 exact rank: 2
Singular values: ['2.000000e+00', '5.000744e-13']
Absolute tolerance / numerical rank: 1e-14 2
Absolute tolerance / numerical rank: 1e-10 1
Scaling by one million at absolute tolerance 1e-10: 2
Relative cutoff policy / rank: A 1
Relative cutoff policy / rank: scaled A 1
The cutoff is a numerical resolution policy, not an exact dependence proof.

Deliverable: report exact rank, dtype, scales, tolerances and numerical ranks. Explain what resolution assumption could justify dropping the weak direction. Do not turn a chosen threshold into an assertion that the exact integer/rational model has changed.

Exercises 1–12 are required. Exercises 13–14 add 35 optional minutes. Include the relevant space and field when writing a closure or independence claim.

Use the linear-algebra chapter’s vector spaces, bases and linear-map selections in the authors’ Mathematics for Machine Learning companion. Read the official NumPy 1.26 matrix-rank interface for the numerical contract. The original proofs and examples above stand independently of these supplementary readings.

Without notes, prove the subspace test, distinguish a generating list from a basis, and explain why dimensions are well defined. Draw the four spaces on the correct input/output sides and derive both nullity counts. Give a null-space prediction witness and delimit its future-record claim.

Exit task: A=[[1,2,3],[0,1,1]] has rank two. Its kernel is span{(−1,−1,1)}, since x₂=−x₃ and x₁=−x₃. Every b=(b₁,b₂) is attainable, with solutions (b₁−2b₂−t,b₂−t,t). The left null space is zero; the column space is R². Explain why reachability of every target does not make the parameters unique.

Ready to move on: you can give complete bases, dimensions, solution families and interpretation boundaries. Module 12 develops orthogonal projections and least squares, including how a unique fitted prediction can coexist with many minimising weights. Check the course overview for availability.