Generalisation, kernels, and mathematical learning theory

A learning algorithm receives observations and produces a predictor. To evaluate the predictor mathematically, specify an input space, an output space, a loss, and a distribution P over examples Z=(X,Y). A hypothesis class H is a set of allowed prediction rules. Population risk R_P(h)=E_P ℓ(h;Z) measures expected loss on a new draw from P. Empirical risk R_hat_S(h) averages loss on the observed sample S of size n. These are different quantities even when the same loss formula appears in both.

The sample model matters as much as the algorithm. In this chapter’s finite-class concentration argument, the n observations are independent draws from the same P, the candidate rules are fixed before those observations, and each loss lies in [0,1]. Zero-one classification error satisfies that range. Squared error and negative log-likelihood do not generally satisfy it. One cannot substitute an unbounded loss into the same Hoeffding formula without additional assumptions or a declared transformation of the evaluated quantity.

A predictor can be random because training data and optimiser randomness are random. Once it is fixed independently of a fresh test sample, condition on its training history and analyse test loss using the fixed-predictor law. The test estimate then describes its risk under the test-generating distribution. It does not automatically assess a future distribution with different labels, prevalence, inputs, or collection processes. Independence refers to the sampling units; copied images, shared patients, or repeated records can invalidate a row-count argument.

Training fits parameters and transformations. Validation selects such choices as penalties, widths, component counts and stopping times. Final test data estimate the risk of the frozen result. Calling a repeatedly inspected dataset “test” does not preserve its independence from model choice. Choosing a threshold, feature transformation or reporting rule using test outcomes is also selection. Establish a complete evaluation protocol, including the metric and threshold, before inspecting the final labels. A new test claim requires a fresh evaluation or a justified procedure accounting for adaptation.

Empirical risk minimisation chooses an element whose sample risk is smallest in H. It does not say that training risk equals population risk or that the class contains the best possible predictor. A finite class can impose useful inductive structure, while an excessively flexible class can match accidental training patterns. Distinguish approximation limitations of the class, sample variation of its estimates, and optimisation error when the algorithm does not find the empirical minimum. These limitations require different mathematical and experimental evidence.

The candidate rules in this example are predeclared, and their random validation scores are not new independently trained models with unspecified populations. If a collection of trained candidates was produced using separate training data, condition on that training data and apply validation bounds to its fixed candidate set. If candidates were constructed after seeing the very sample being bounded, merely counting the final shortlist is not enough. One must control the larger search or its data dependence; a printed M does not resolve that obligation.

Conditioning on a training history is useful because it makes a precise independence claim. The pool can depend arbitrarily on training labels while the fresh validation rows retain their IID distribution. The conditional failure probability is then bounded for every eligible training history; averaging over those histories preserves the bound. Conversely, if a candidate uses validation labels in preprocessing or target encoding, the pool is no longer fixed under this conditioning. A technical split name cannot override the information actually used by the algorithm. Trace dependencies in the full fitting procedure, including transformations and human selection decisions.

For a fixed h and IID bounded losses, the two-sided Hoeffding bound gives P(|R_hat(h)−R(h)|≥ε)≤2 exp(−2nε²). This named concentration result was introduced with its assumptions in Module 24. It controls one predeclared predictor, not an arbitrarily chosen winner after examining the sample. A union bound can turn it into a simultaneous statement over a finite predeclared class without assuming different models’ loss estimates are independent of one another.

Let H contain M rules. For each rule define the bad event that its empirical risk differs from population risk by at least ε. If any rule is bad, the union of these M events occurs. Subadditivity bounds its probability by the sum of their probabilities, at most 2M exp(−2nε²). Set this expression to δ and solve for ε. With probability at least 1−δ, every rule satisfies |R_hat(h)−R(h)|≤sqrt(log(2M/δ)/(2n)). Natural logarithms appear because the exponential bound uses e.

Now let h_hat minimise empirical risk and h_star minimise population risk in the finite class. On the simultaneous good event, R(h_hat)≤R_hat(h_hat)+ε≤R_hat(h_star)+ε≤R(h_star)+2ε. The middle step is exactly the ERM property. If optimisation is within τ of the empirical minimum, add τ to this upper bound. The guarantee compares with the best member of H, rather than with an arbitrary population-optimal rule outside it. A bound on estimation error does not remove approximation error.

The simultaneity is what permits the selected h_hat to depend on S: all members were controlled before the algorithm chose among them. It does not permit changing H after observing S without another argument. The sample size n must count independent examples from the declared population, and M must count the controlled class or independently constructed candidate pool. Correlation between model scores causes no difficulty for the union step; correlation between sample rows violates the fixed-model Hoeffding premise.

A radius exceeding one can be truncated to one for the difference of [0,1] risks, but this supplies no useful discrimination between models. A large loose bound is neither proof that learning fails nor evidence of a precise required practical dataset size. Report the theorem’s target event, loss range and confidence alongside its number. Confidence refers to repeated sample probability; it is not posterior probability assigned to a realised fixed unknown risk unless a Bayesian model is separately specified.

For bounded loss in [a,b], divide by its range to apply the unit-range theorem, then multiply the radius by b−a. A constant loss with b=a has no sampling error. If a probability model is restricted so every realised true-label probability is at least q_min>0, its NLL lies between zero and −log q_min, giving a different range factor. Enforcing that restriction or clipping scores changes the evaluated model or loss and must be disclosed. Without it, a confident wrong prediction can produce arbitrarily large NLL, so a classification-error guarantee cannot be reused as a log-loss guarantee.

For a countable class, one can assign predeclared positive weights π_h summing to one and give each rule failure budget δπ_h. Union bounding yields rule-specific radii involving log(2/(δπ_h)). This illustrates how a declared complexity preference affects a simultaneous guarantee. We do not derive general infinite-class results here. Simply replacing M by an informal number of parameters is unjustified, and discretising a continuous class requires an approximation argument connecting grid rules to off-grid risks.

Bias–variance decomposition is a different statement from a concentration bound. Fix an input x and let the fitted prediction f_hat_D(x) vary over random training data D, including optimiser randomness if specified. Let a fresh label satisfy Y=f_star(x)+ε, with conditional mean-zero noise and variance σ²(x), independent of the fitting history at this x. Define m(x)=E_D f_hat_D(x), bias m(x)−f_star(x), and variance E_D(f_hat_D(x)−m(x))². These expectations must exist.

Expand f_hat_D−Y=(f_hat_D−m)+(m−f_star)−ε. Taking the expectation of its square leaves the three squared terms. The cross term between f_hat_D−m and the fixed bias vanishes by centring. Noise cross terms vanish through conditional mean zero and independence from D. Therefore expected fresh squared error equals prediction variance plus squared bias plus σ²(x). Integrating over a specified input distribution gives a population decomposition if the required moments are integrable.

This is an exact identity under the stated model, not a theorem that model size monotonically increases variance or that a particular regulariser always improves total risk. A penalty changes the fitted prediction’s distribution and can reduce variance while increasing bias. Different sample sizes, noise levels, algorithms and classes can change both. A flexible interpolating model can behave well or badly depending on structure. Training error alone does not identify these population components, and one split cannot directly measure a training-distribution expectation.

Optimiser seed variation conditional on a fixed training dataset measures only the variability due to that algorithmic randomness. Across new training datasets, additional sampling variation appears. The law of total variance separates E_D Var_seed(f_hat|D) from Var_D E_seed(f_hat|D). Repeating seeds on one dataset cannot establish the second term, and repeating dependent splits does not create independent future observations. Distinguish uncertainty about the evaluation sample from variability of the fitting procedure.

The identity is specific to squared error and a squared-distance target. Classification error and log-loss have different geometry; do not relabel their observed errors as the same bias and variance components. For Bernoulli log-loss, Module 27 instead relates expected NLL to entropy plus a KL mismatch, where support and the conditional prediction distribution matter. Distinguish an identity of risk from a confidence statement about estimating it and from an optimisation statement about reducing training loss.

A polynomial interpolation example is a useful diagnostic. Nine equally spaced observations can determine a degree-eight polynomial with essentially zero training residual, while between those points the fit oscillates. Lab 3 compares it with degree two against a known smooth function on a dense deterministic grid. The higher degree has lower training error but larger grid error in this example. The grid calculation is approximation evidence for that function and domain, rather than an IID confidence estimate or a universal verdict on high-degree models.

The target conditional mean f_star minimises expected squared error pointwise: for any fixed prediction a, E[(a−Y)²|x]=(a−f_star(x))²+σ²(x). This follows by the same centred-noise expansion without training randomness. A hypothesis class may be unable to represent that mean, creating approximation error even with infinite data and perfect optimisation. Choosing a model is therefore not solely a contest between empirical fit and an estimation radius. State which target the class approximates and whether the loss expresses the application’s cost before interpreting a capacity trade-off.

Cardinality is a convenient capacity measure for finite H, but many useful classes have infinitely many parameter values. For binary predictions, a class shatters a finite input set if it can realise every possible binary labelling of those points. Its VC dimension is the largest size of a shattered set, or infinite if no finite maximum exists. This definition concerns representable patterns, not successful optimisation, observed training accuracy, or the raw number of numerical parameters.

Consider thresholds h_t(x)=1 if x≥t and zero otherwise, with fixed orientation. Any single real point can receive either label by placing t below or above it, so one point is shattered. For any two ordered points x1<x2, the labels (1,0) are impossible: assigning one to x1 requires t≤x1, which also assigns one to x2. Thus no two-point set is shattered and this class has VC dimension one, despite its infinitely many possible thresholds. This complete example explains why infinite cardinality need not prevent learnability.

VC-based uniform convergence results require more machinery and are named here rather than proved. Their statements include a sample model, a loss setting, and confidence/complexity terms. For modern large networks, a crude parameter-count bound can be extremely loose; useful analysis may involve norm constraints, margins, stability, compression or data-dependent complexity. This introductory module does not establish a universal explanation for neural-network generalisation. Do not convert a suggestive capacity narrative into a theorem outside the conditions actually proved.

Distribution shift changes the population being evaluated. Under source P and target Q, risks are R_P(h) and R_Q(h), which can differ even for a perfectly measured fixed predictor. Covariate shift refers to a changed input distribution with the label-given-input law preserved; label or concept changes alter other parts of the joint law. Such distinctions are modelling assumptions to investigate, not something a successful source test automatically confirms. Source confidence intervals quantify sampling variability under P, rather than bound every future Q.

If Q is absolutely continuous relative to P, importance weighting can express target risk as E_P[w(Z)ℓ(h;Z)] with density ratio w=dQ/dP. Missing source support prevents this direct representation. Even where it is possible, weights may be large, unknown or estimated; their range and variance change the concentration problem. Under covariate shift with a preserved conditional law, an input-density ratio suffices, but an incorrect shift assumption invalidates the correction. A formula is not evidence that weights can be estimated accurately from limited data.

Class prevalence can change the meaning of accuracy even without a dramatic feature example. An always-zero predictor has accuracy .95 when class-one prevalence is .05 and .05 when it is .95. Conditional sensitivity and specificity, probability calibration, log-loss, and the chosen decision cost answer different questions. Report the population and metric needed by the application. A metric can be calculated exactly on a finite dataset while still being a misleading summary of the intended deployment task.

A feature map φ sends an input to a vector in an inner-product space. Define k(x,z)=φ(x)ᵀφ(z). For any finite inputs x_i, their Gram matrix K_ij=k(x_i,x_j) is symmetric. For any coefficient vector c, cᵀKc=||Σ c_iφ(x_i)||²≥0, so it is positive semidefinite. A valid real kernel must have this property for every finite input set. A numerically PSD matrix on one dataset checks that set, not all possible inputs of an arbitrary proposed similarity function.

Take the two-dimensional map φ(t)=(1,t). Its kernel is 1+st. At inputs 0,1,−1 the Gram matrix has rows (1,1,1), (1,2,0), (1,0,2), with eigenvalues 0,2,3. A zero mode is allowed: PSD does not require positive definiteness. Symmetry and positive diagonal entries alone are insufficient. The matrix with diagonal one and off-diagonal two has eigenvalues three and minus one; c=(1,−1) gives cᵀKc=−2, an exact certificate of failure.

The one-dimensional RBF kernel exp(−(s−t)²/(2ℓ²)), ℓ>0, is also PSD. Factor it into exp(−s²/(2ℓ²))exp(−t²/(2ℓ²))exp(st/ℓ²), and expand the last exponential. Each term is a positive coefficient times a product of an s-only and a t-only feature. For finitely many inputs, the quadratic form is a convergent sum of squares, hence nonnegative. This proves the scalar kernel used in Lab 2; vector versions use analogous feature arguments. The length parameter declares an input scale and must be selected without test leakage.

For a finite-dimensional feature ridge problem, minimise mean half-squared error plus λ||w||²/2 with λ>0. Decompose w into the span of training feature vectors and its orthogonal complement. The complement cannot change training predictions and only adds squared penalty, so an optimum lies in the span: w=Σα_iφ(x_i). Substitution gives the coefficient objective ||Kα−y||²/(2n)+λαᵀKα/2. This span argument motivates kernel computation; general infinite-space representer results require an appropriate Hilbert-space setting.

Differentiate to obtain K(Kα−y)/n+λKα=0. A sufficient solution is (K+nλI)α=y. Since K is PSD and λ>0, this shifted matrix is positive definite and invertible, even when K is singular. Coefficients minimising the original coefficient objective can differ in K’s nullspace, but their represented feature vector and predictions coincide. Use a solve rather than explicit inverse. New prediction is k_xᵀα, where k_x contains similarities to the training inputs; all preprocessing and scale choices belong to the fitted model.

The displayed inverse is a mathematical formula, not an instruction to form it in code. Dense kernel methods store n² entries and a generic dense solve costs order n³; an explicit small feature map can be cheaper. Kernel centring and an unpenalised intercept require their own training-fitted transformation. Lab 2 instead declares a fixed training-mean offset and fits centred residual targets, without claiming this is joint optimisation of a free intercept. A kernel’s validity, numerical conditioning and population performance are three separate questions.

Kernel scaling also interacts with the penalty convention. If K is replaced by cK with c>0 and λ by cλ, the shifted system gives new coefficients α/c and predictions remain unchanged, since the new similarity vector is ck_x. Changing kernel amplitude without changing λ generally changes the fit. Likewise, changing input units without the corresponding RBF length changes similarities. Record amplitude, length, preprocessing and mean-versus-sum reduction as parts of the model. Comparing only a printed penalty number across different kernel scales does not compare equal regularisation strength.

For labels y_i∈{−1,1}, a linear score f(x)=wᵀφ(x)+b predicts by its sign with a declared tie convention. The signed geometric distance to its hyperplane is y_i f(x_i)/||w|| when w≠0. Multiplying w and b by a positive constant leaves the decision rule unchanged, so functional score scale alone is not a geometric margin. A hard-margin formulation chooses the scale y_i f(x_i)≥1 and minimises ||w||²/2; it requires separability in the chosen feature space.

To justify the distance, move a feature vector by a displacement d until its score becomes zero. Then wᵀd=−f(x). Cauchy–Schwarz requires ||d||≥|f(x)|/||w||, and equality is achieved by displacement −f(x)w/||w||². The margin is this perpendicular feature-space distance with the label sign. It is not necessarily a distance in the raw input domain after a nonlinear feature map. At w=0 the formula is undefined; feasibility and slack can still be discussed through scores without inventing a geometric distance.

Nonseparable observations, including opposite labels at the same feature vector, make hard-margin constraints infeasible. Soft margin introduces ξ_i≥0 and requires y_i f(x_i)≥1−ξ_i, minimising ||w||²/2+CΣξ_i for C>0. Fix w,b and minimise over ξ: each equals max(0,1−y_i f(x_i)), the hinge loss. This gives an unconstrained convex objective with a norm penalty and a loss convention. Multiplying by 1/(nC) connects it to mean hinge loss plus λ||w||²/2 with λ=1/(nC).

This constrained introduction is self-contained for the AI route without Module 20. The full duality extension uses Lagrange multipliers α_i≥0 for margin constraints and μ_i≥0 for ξ_i≥0. Stationarity gives w=Σα_i y_iφ(x_i), Σα_i y_i=0 from the free intercept, and C−α_i−μ_i=0, hence 0≤α_i≤C. Substitution yields the dual maximisation Σα_i−(1/2)Σ_i,j α_iα_j y_i y_j K_ij with that box constraint and intercept equality.

Convexity and a strictly feasible soft-margin point, such as w=0,b=0,ξ_i=2, support strong duality under the standard Slater result from Module 20. The PSD kernel makes the dual quadratic concave. Complementary slackness explains support vectors and margin activity, but this paragraph is not a derivation of an implemented SVM solver. Lab 2 implements kernel ridge. Kernelising the SVM dual changes the feature representation, while its training margin still does not by itself prove a finite-class guarantee for an unrestricted infinite model class.

A useful final report connects the predictor’s mathematical class, objective, optimisation, numerical implementation and evaluation. State whether a theorem is proved here, a named result used under assumptions, an exact finite calculation, or a diagnostic experiment. When a bound is loose, explain its sufficient nature. When a model overfits, identify the selection or approximation mechanism rather than infer a universal rule from one plot. The capstones next ask you to defend these distinctions in complete executable projects.

Comparative evaluation should preserve paired observations: two frozen predictors scored on the same independent test examples produce a per-example loss difference, whose variance depends on their covariance. Reporting two separate standard errors and assuming independence can misdescribe their difference. For bounded differences, rescale the actual difference range before concentration; for a bootstrap, resample independent example pairs together. This uses Module 26’s experimental-unit discipline. The same principle applies when comparing a kernel model with a baseline: the shared test inputs support a paired comparison, while repeated tuning on those outcomes would turn the comparison into another selection stage.

Lab A · selected optimism with known population risk

Use the pinned NumPy environment. Predict how the validation maximum changes with the predeclared candidate count. The exact binomial count simulation corresponds to independent fair-bit features and labels, so every rule still has population accuracy .5. Report repeat averages separately from the one-sample uniform radius.

Download lab1_selection_optimism.py

"""A predeclared finite null class; each true classification accuracy is .5.

X contains independent fair bits and Y is an independent fair bit; h_j(X)=X_j.
Validation error counts are independent Binomial(n,.5) under this model.
The count sampler implements that exact distribution without storing all rows.
"""
import math
import numpy as np

rng = np.random.default_rng(30030)
repeats, n, ntest = 1000, 100, 2000
for candidates in (1, 20, 200):
    validation = rng.binomial(n, .5, size=(repeats, candidates))/n
    winner = validation.max(axis=1)
    # Every selected candidate still has population accuracy .5; fresh test is independent.
    test = rng.binomial(ntest, .5, size=repeats)/ntest
    epsilon = math.sqrt(math.log(2*candidates/.05)/(2*n))
    print(f'M={candidates}; mean selected validation={winner.mean():.6f}; mean independent test={test.mean():.6f}; uniform radius={epsilon:.6f}')
print('The repeat average is an experiment summary; the bound is for one IID sample and all fixed candidates.')

M=1; mean selected validation=0.497960; mean independent test=0.500209; uniform radius=0.135810
M=20; mean selected validation=0.592690; mean independent test=0.499582; uniform radius=0.182820
M=200; mean selected validation=0.636930; mean independent test=0.500013; uniform radius=0.211981
The repeat average is an experiment summary; the bound is for one IID sample and all fixed candidates.

Lab B · a kernel-ridge solve and frozen comparison

Use 40/25/25 disjoint independent synthetic observations. Declare raw x units and the training-only fixed offset. Select the RBF length and mean-objective penalty on validation, then evaluate the selected model and constant baseline on test once. PSD numerical tolerances are diagnostics supporting an analytically valid kernel.

Download lab2_kernel_ridge.py

"""RBF kernel ridge, mean-loss convention; choose only on validation."""
import numpy as np

rng = np.random.default_rng(30031)
x = rng.uniform(-2, 2, 90)
y = np.sin(3*x)+rng.normal(0, .15, 90)
train, val, test = np.arange(40), np.arange(40,65), np.arange(65,90)
offset = y[train].mean()  # Fixed training-only offset, not a separately optimised intercept.
def kernel(a, b, length):
    return np.exp(-((a[:,None]-b[None,:])**2)/(2*length**2))
records = []
for length in (.2, .6, 1.5):
    K = kernel(x[train], x[train], length)
    assert np.linalg.eigvalsh(K).min() > -1e-10
    for penalty in (.0001, .01, .1):
        alpha = np.linalg.solve(K+len(train)*penalty*np.eye(len(train)), y[train]-offset)
        prediction = offset+kernel(x[val], x[train], length)@alpha
        mse = np.mean((prediction-y[val])**2)
        records.append((mse, length, penalty, alpha))
        print(f'length={length:.1f}; lambda={penalty:g}; validation MSE={mse:.6f}')
_, length, penalty, alpha = min(records, key=lambda row: row[0])
prediction = offset+kernel(x[test],x[train],length)@alpha
print(f'frozen choice: length={length}; lambda={penalty}; test MSE={np.mean((prediction-y[test])**2):.6f}')
print(f'training-mean baseline test MSE={np.mean((offset-y[test])**2):.6f}')
print('40/25/25 independent rows; raw x units; fixed training-only offset; no final-test selection.')

length=0.2; lambda=0.0001; validation MSE=0.052735
length=0.2; lambda=0.01; validation MSE=0.054247
length=0.2; lambda=0.1; validation MSE=0.258900
length=0.6; lambda=0.0001; validation MSE=0.016715
length=0.6; lambda=0.01; validation MSE=0.039747
length=0.6; lambda=0.1; validation MSE=0.246924
length=1.5; lambda=0.0001; validation MSE=0.119131
length=1.5; lambda=0.01; validation MSE=0.383195
length=1.5; lambda=0.1; validation MSE=0.526769
frozen choice: length=0.6; lambda=0.0001; test MSE=0.025248
training-mean baseline test MSE=0.658235
40/25/25 independent rows; raw x units; fixed training-only offset; no final-test selection.

Lab C · similarity, shift, and interpolation

Provide an exact indefinite quadratic witness, inspect a feature Gram’s tiny rounded eigenvalue, reverse a label mechanism, and compare polynomial interpolation on a dense deterministic grid. Explain why each result challenges a different assumption.

Download lab3_similarity_shift_and_interpolation.py

"""Indefinite similarity, changed labels, and zero-training-error diagnosis."""
import numpy as np

bad = np.array([[1.,2.],[2.,1.]])
c = np.array([1.,-1.])
print('symmetric positive-diagonal similarity eigenvalues:', np.linalg.eigvalsh(bad))
print('witness c.T K c:', c@bad@c)
assert c@bad@c < 0
phi = np.array([[1.,0.],[1.,1.],[1.,-1.]])
good = phi@phi.T
print('feature-map Gram eigenvalues:', np.linalg.eigvalsh(good))
assert np.linalg.eigvalsh(good).min() > -1e-12
x = np.array([-1.,1.])
prediction = x>0
source_y = x>0
target_y = x<0
print('same feature masses, changed label mechanism: source accuracy=',np.mean(prediction==source_y),'; target accuracy=',np.mean(prediction==target_y))
print('always-zero accuracy under class-1 prevalence .05/.95:', .95, .05)
t = np.linspace(-1,1,9)
y = 1/(1+25*t*t)
grid = np.linspace(-1,1,1001)
truth = 1/(1+25*grid*grid)
for degree in (2,8):
    design = np.vander(t, degree+1, increasing=True)
    beta = np.linalg.lstsq(design,y,rcond=None)[0]
    fitted = design@beta
    dense = np.vander(grid,degree+1,increasing=True)@beta
    print(f'degree={degree}; training MSE={np.mean((fitted-y)**2):.6e}; dense-grid MSE={np.mean((dense-truth)**2):.6e}')
print('Dense-grid error is a deterministic interpolation diagnostic, not a confidence estimate from an IID test sample.')

symmetric positive-diagonal similarity eigenvalues: [-1.  3.]
witness c.T K c: -2.0
feature-map Gram eigenvalues: [-4.4408921e-16  2.0000000e+00  3.0000000e+00]
same feature masses, changed label mechanism: source accuracy= 1.0 ; target accuracy= 0.0
always-zero accuracy under class-1 prevalence .05/.95: 0.95 0.05
degree=2; training MSE=4.503752e-02; dense-grid MSE=3.946358e-02
degree=8; training MSE=1.474889e-27; dense-grid MSE=1.356205e-01
Dense-grid error is a deterministic interpolation diagnostic, not a confidence estimate from an IID test sample.

Read the authors’ Understanding Machine Learning catalogue for Chapters 4, 6, 15 and 16, then compare the official Cornell ERM notes and SVM notes. The finite-class, threshold, feature-PSD and ridge arguments above are worked here; broader capacity results are identified as named extensions.

Exit task: derive the risk radius and ERM comparison, prove a PSD Gram and a threshold capacity claim, diagnose a selection and shift failure, and state a kernel-ridge/SVM objective convention.

Ready to move on: build Module 31’s CS capstone or Module 32’s AI capstone after its prerequisites. Return to the course overview.