AI 综合项目：具备数学检查的学习流程

按固定生成模型，从数据到诚实最终评估，实施并答辩小型二分类器。CPU NumPy 即可，无 GPU、付费 API 或下载数据。十六小时核心为四个四小时时段。参考脚本在固定 NumPy 环境独立运行、各自生成输入并展示实际结果。查看参考前，先写推导并预测故障。

提交数据拆分清单、训练变换说明、数学附录、可运行模型、导数证据、验证选择表、四故障诊断、冻结测试报告、三种实际诊断图和完整预测更新追踪。参考结果不等于学习者提交；若改数据、抽样顺序、优化器、惩罚或预算，执行自己的版本，不复制这些数字。

固定协议按均值 NLL 选择，预先指定阈值 .5 准确率为决策指标，同时报告 Brier、混淆计数和五固定概率校准箱。验证在预先网格选最终模型；PCA 比较和常概率基线也在测试前冻结。分数是声明抽样或混合解释下的估计，不是未来部署保证。

用 NumPy default_rng 数据种子 7 生成 1,000 独立例子。为精确复现，先按行顺序抽形状 (1000,5) 标准正态矩阵，列为 z1,z2,z3,e1,e2，再抽 1,000 均匀值生成 Bernoulli 标签。固定 NumPy 1.26.4 中生成器为 PCG64，仅整数种子不足，需生成器和抽样顺序。

X=(z1,100z2,z1+.02e1,z3,e2)，成功概率 p_star=σ(1.5z1−2z2+.5z3)，后续均匀值小于 p_star 时 y=1。生成模型各潜变量、例子独立，观测第一第三特征因构造近相关。第五特征不影响条件标签概率；近重复仍有非零噪声方向，非精确冗余列。

独立拆分种子 11，精确 600/200/200。正类按最大余数分配：三正类配额取下整，剩余给小数部分最大者，同值按训练验证测试；其余槽填负类。各类索引分别打乱并分配，再打乱每拆分。整数允许范围内尽量保比例且不重复行。参考共 503 正类，分配 302/101/100。

分层明确使用标签构造拆分，但不允许拿保留特征、标签拟合变换或选择参数。固定类别计数也影响不确定性；不能静默把不受限随机混合的 IID 二项公式用于此设计。说明每估计区间对应哪个总体或固定类混合。

只在训练拟合 μ_j、ddof=0 标准差 s_j，用 (x−μ)/s 变换所有拆分。零训练尺度需拒绝或预先常特征政策；参考尺度正。标准化后加截距。只对训练标准矩阵 SVD 拟合 PCA，验证选成分数。本参考不白化，不在测试重拟合 scaler 或 PCA。

数据、拆分与曲率参考

下载 lab1_dataset_splits_and_curvature.py

"""Exact capstone draw order and stratified split; training-only diagnostics."""
import hashlib
import sys
import numpy as np

def sigmoid(z):
    z = np.asarray(z,dtype=float)
    out = np.empty_like(z)
    positive = z >= 0
    out[positive] = 1/(1+np.exp(-z[positive]))
    e = np.exp(z[~positive])
    out[~positive] = e/(1+e)
    return out

def dataset():
    rng = np.random.default_rng(7)
    latent = rng.normal(size=(1000,5)) # Row-major z1,z2,z3,e1,e2, then uniform labels.
    z1,z2,z3,e1,e2 = latent.T
    X = np.column_stack([z1,100*z2,z1+.02*e1,z3,e2])
    probability = sigmoid(1.5*z1-2*z2+.5*z3)
    y = (rng.random(1000)<probability).astype(int)
    return X,y

def split(y):
    rng = np.random.default_rng(11)
    sizes = np.array([600,200,200])
    quotas = sizes*y.sum()/len(y)
    positive = np.floor(quotas).astype(int)
    remainder = int(y.sum()-positive.sum())
    priority = sorted(range(3),key=lambda j:(-(quotas[j]-positive[j]),j))
    for j in priority[:remainder]:
        positive[j] += 1
    negative = sizes-positive
    blocks = [[] for _ in sizes]
    for label,counts in [(0,negative),(1,positive)]:
        indices = np.flatnonzero(y==label)
        rng.shuffle(indices)
        start = 0
        for j,count in enumerate(counts):
            blocks[j].extend(indices[start:start+count].tolist())
            start += count
    result = []
    for block in blocks:
        indices = np.array(block,dtype=int)
        rng.shuffle(indices)
        result.append(indices)
    assert len(set(np.concatenate(result).tolist())) == len(y)
    assert [len(i) for i in result] == sizes.tolist()
    return result

X,y = dataset()
train,val,test = split(y)
mean,scale = X[train].mean(axis=0),X[train].std(axis=0,ddof=0)
assert np.all(scale>0)
standardised = (X[train]-mean)/scale
raw_s = np.linalg.svd(X[train]-mean,compute_uv=False)
scaled_s = np.linalg.svd(standardised,compute_uv=False)
print('Python',sys.version.split()[0],'; NumPy',np.__version__,'; default_rng PCG64; data seed 7; split seed 11')
print('data SHA256:',hashlib.sha256(X.astype('<f8').tobytes()+y.astype('u1').tobytes()).hexdigest())
print('shape:',X.shape,'; positive labels:',int(y.sum()))
print('split sizes / positives:',[(len(i),int(y[i].sum())) for i in (train,val,test)])
print('training mean:',mean)
print('training scale ddof=0:',scale)
print('centred raw singular values:',raw_s)
print('standardised singular values:',scaled_s,'; numerical rank:',np.linalg.matrix_rank(standardised))
for name,data in [('raw',X[train]),('scaled',standardised)]:
    design = np.column_stack([np.ones(len(data)),data])
    bound = np.linalg.norm(design,2)**2/(4*len(data))
    print(f'{name} unpenalised logistic Hessian upper bound={bound:.6f}; reciprocal={1/bound:.6f}')
print('near dependence is not exact rank deficiency; e1 contributes a small nonzero mode')

Python 3.11.8 ; NumPy 1.26.4 ; default_rng PCG64; data seed 7; split seed 11
data SHA256: 66153f6986d1b4c996690c54eb78b99c5b19c7cef9e47f6c8fda47e2516dcc72
shape: (1000, 5) ; positive labels: 503
split sizes / positives: [(600, 302), (200, 101), (200, 100)]
training mean: [-0.08151553 -1.46092851 -0.08189573 -0.02051704 -0.04430283]
training scale ddof=0: [ 0.9664932  99.56059306  0.96628383  0.97397309  0.97456022]
centred raw singular values: [2.43872925e+03 3.33444431e+01 2.42684764e+01 2.33544055e+01
 3.49774556e-01]
standardised singular values: [34.80735141 25.123916   24.89247068 23.18342157  0.36193962] ; numerical rank: 5
raw unpenalised logistic Hessian upper bound=2478.617241; reciprocal=0.000403
scaled unpenalised logistic Hessian upper bound=0.504813; reciprocal=1.980931
near dependence is not exact rank deficiency; e1 contributes a small nonzero mode

中心化原始设计最大奇异值约 2439，对应大 z2，最小约 .35。标准化最大降至 34.81，但近重复对比小模态仍约 .362，数值秩五。满秩仍可有强相关系数方向。最小解释方差比约 4.37×10^(−5)，不是算术零。

阶段门槛：在测试环境复现 hash、计数，证索引不重叠且完整，列变换拟合行，解释各奇异模态。SHA256 只是此生成字节的复现标识，区别模块 31 教学模校验和。

数学附录：似然、形状与梯度

A=[1,Z] 是训练缩放后的设计，n 行 q=d+1 列，θ=(b,w) 有 q 项。logits t=Aθ、y、p=σ(t) 均 n 项。固定标签 Bernoulli 似然为 Πp_i^(y_i)(1−p_i)^(1−y_i)，条件独立支持乘积。用 NLL 避免乘积下溢。

均值 F=mean[softplus(t)−yt]+λ||w||²/2，b 不罚，λ 属均值约定，改总和需对应改尺度。有限 logits 用 max(t,0)−yt+log1p(exp(−|t|))，sigmoid 分正负支路避免上溢。概率裁剪改变目标；有限精度概率可到端点，logit 损失仍可算。

softplus 导数 sigmoid，线性标签项导数 −y，链式给 Aᵀ(p−y)/n；惩罚为截距零、斜率 λw。形状 (n,q)ᵀ×(n,)→(q,)。标签 (n,1) 与概率 (n,) 广播可意外形成 (n,n) 误差矩阵，应断言预期一维形状。

Hessian=AᵀDA/n+λdiag(0,1,…,1)，D_ii=p_i(1−p_i)∈[0,1/4]，PSD 故凸。谱范数≤||A||2²/(4n)+λ，解释原尺度需很小安全步长。该界充分，不是每条轨迹必坏的阈值。λ>0 罚斜率，自由截距和数据仍决定全 Hessian；λ=0 可分性可能使有限 MLE 不存在，如模块 26。

本例原始无罚 Hessian 上界约 2478.62，标准化约 .504813。原坐标 η=.2 远超保守倒曲率尺度，实际训练 NLL 从 log2≈.693，一步升 76.18、十步 104.50。稳定算术准确计算坏更新；修复需改坐标和合理步长，而非裁剪遮住大损失。

证明与导数义务

写似然到损失、链式和全部维数。在八个受控训练行、θ=linspace(−.2,.3,6)、λ=.03、h=10^(−5)，独立中心差分检查同一平滑惩罚目标，参考范数误差约 1.55×10^(−11)。说明局部证据、为何应扫 h、遗漏 n 为代数约定错误。两检查都明确截距惩罚。

训练标准矩阵 Z_train=UΣVᵀ，保留 V_k 为 (d,k)，投影行 k 项、logistic 设计 k+1 列。方差比 σ_j²/Σσ²，公共样本协方差分母抵消。PCA 保输入方差，不直接保标签信号；验证可偏好多成分，小方差方向在其他问题未必无用。全五成分不白化只是正交旋转，非压缩。

阶段门槛：给有限损失、维数和导数证据，曲率预测原步长失败，区分近相关、精确秩亏和 PCA 截断。

完整参考流程

每候选从零做 800 次全批更新。完整特征网格 η=.05,.2,1，λ=0,.01,.1；PCA k=2,3,4,5，η=.2、λ=.01。训练拟合公共 scaler、PCA 一次；验证均值 NLL 选十三配置，平局取首网格项。常模型报训练正类比例约 .503333。最终指标前冻结全部、阈值 .5、五等宽概率箱。

更新数相同，步长改变该预算下优化进展。验证偏好慢步长可能体现有限时间正则，不是完全收敛问题的通用最优率。参考选完整模型 η=.05、λ=0；单独 PCA 比较选全五成分。所以不能声称降维有益，尽管减少成分的候选正确经验证评估。

脚本在测试前打印冻结配置，保留训练权重，不静默在训练加验证重拟合，后者是另一程序。若要重拟合，测试前说明 scaler、PCA、步数、惩罚怎样重设，不悄然替换参考约定。

下载 lab2_checked_learning_pipeline.py

"""Complete NumPy CPU capstone: fixed draw/split, selection, PCA, frozen test.

Mean Bernoulli NLL + lambda/2 * squared slopes; intercept unpenalised.
800 full-batch zero-initialised updates for each predeclared candidate.
Bootstrap intervals are approximate and conditional on observed class counts;
they freeze the fitted predictors rather than refitting the learning procedure.
"""
import argparse
import hashlib
import math
from pathlib import Path
import sys
import numpy as np

def sigmoid(z):
    z = np.asarray(z,dtype=float)
    out = np.empty_like(z)
    positive = z>=0
    out[positive] = 1/(1+np.exp(-z[positive]))
    e = np.exp(z[~positive])
    out[~positive] = e/(1+e)
    return out

def dataset():
    rng = np.random.default_rng(7)
    latent = rng.normal(size=(1000,5))
    z1,z2,z3,e1,e2 = latent.T
    X = np.column_stack([z1,100*z2,z1+.02*e1,z3,e2])
    y = (rng.random(1000)<sigmoid(1.5*z1-2*z2+.5*z3)).astype(int)
    return X,y

def split(y):
    rng = np.random.default_rng(11)
    sizes = np.array([600,200,200])
    quotas = sizes*y.sum()/len(y)
    positive = np.floor(quotas).astype(int)
    priority = sorted(range(3),key=lambda j:(-(quotas[j]-positive[j]),j))
    for j in priority[:int(y.sum()-positive.sum())]:
        positive[j] += 1
    blocks = [[] for _ in sizes]
    for label,counts in [(0,sizes-positive),(1,positive)]:
        indices = np.flatnonzero(y==label)
        rng.shuffle(indices)
        start = 0
        for j,count in enumerate(counts):
            blocks[j].extend(indices[start:start+count].tolist())
            start += count
    result = []
    for block in blocks:
        indices = np.array(block,dtype=int)
        rng.shuffle(indices)
        result.append(indices)
    assert len(set(np.concatenate(result).tolist())) == len(y)
    assert [len(i) for i in result] == [600,200,200]
    return result

def design(X):
    return np.column_stack([np.ones(len(X)),X])

def losses(A,y,w):
    assert A.ndim == 2 and y.shape == (len(A),) and w.shape == (A.shape[1],), 'design, labels and parameters must have the declared shapes'
    z = A@w
    return np.maximum(z,0)-y*z+np.log1p(np.exp(-np.abs(z)))

def objective(A,y,w,penalty):
    return losses(A,y,w).mean()+penalty*np.dot(w[1:],w[1:])/2

def gradient(A,y,w,penalty):
    assert A.ndim == 2 and y.shape == (len(A),) and w.shape == (A.shape[1],), 'avoid unintended label-column broadcasting'
    result = A.T@(sigmoid(A@w)-y)/len(y)
    result[1:] += penalty*w[1:]
    return result

def fit(A,y,validation,val_y,rate,penalty,steps=800):
    w = np.zeros(A.shape[1])
    trace = []
    for t in range(steps+1):
        if t%40 == 0:
            trace.append((t,losses(A,y,w).mean(),losses(validation,val_y,w).mean()))
        if t<steps:
            w -= rate*gradient(A,y,w,penalty)
            assert np.isfinite(w).all()
    return w,trace

X,y = dataset()
train,val,test = split(y)
mean,scale = X[train].mean(0),X[train].std(0,ddof=0)
assert np.all(scale>0)
Z = (X-mean)/scale
_,singular,Vt = np.linalg.svd(Z[train],full_matrices=False)
print('Python',sys.version.split()[0],'; NumPy',np.__version__,'; PCG64 data/split seeds 7/11')
print('data SHA256:',hashlib.sha256(X.astype('<f8').tobytes()+y.astype('u1').tobytes()).hexdigest())
print('split sizes/positives:',[(len(i),int(y[i].sum())) for i in (train,val,test)])
print('training-only standardised singular values:',singular)
print('training-only PCA explained variance ratios:',singular**2/np.sum(singular**2))
# Independent smooth derivative check of the full objective on a controlled subset.
small = design(Z[train[:8]])
small_y = y[train[:8]]
probe = np.linspace(-.2,.3,6)
penalty,step = .03,1e-5
analytic = gradient(small,small_y,probe,penalty)
numeric = np.array([(objective(small,small_y,probe+step*np.eye(6)[j],penalty)-objective(small,small_y,probe-step*np.eye(6)[j],penalty))/(2*step) for j in range(6)])
error = np.linalg.norm(numeric-analytic)
assert error<1e-8
print(f'mean NLL + unpenalised-intercept ridge gradient-check error={error:.6e}; h={step:g}')
# Deliberately unsafe raw-coordinate rate, diagnosed using training only.
raw = design(X[train])
bad = np.zeros(6)
raw_trace = []
for t in range(11):
    raw_trace.append(losses(raw,y[train],bad).mean())
    if t<10:
        bad -= .2*gradient(raw,y[train],bad,0)
print('raw eta=.2 train NLL at 0/1/10:',[raw_trace[j] for j in (0,1,10)])
assert raw_trace[10]>raw_trace[0]

records = []
for rate in (.05,.2,1.):
    for penalty in (0.,.01,.1):
        A,B = design(Z[train]),design(Z[val])
        w,trace = fit(A,y[train],B,y[val],rate,penalty)
        score = losses(B,y[val],w).mean()
        record = {'kind':'full','k':5,'rate':rate,'penalty':penalty,'score':score,'w':w,'trace':trace,'basis':None}
        records.append(record)
        print(f'full rate={rate:g}; lambda={penalty:g}; validation NLL={score:.6f}')
for k in (2,3,4,5):
    basis = Vt[:k].T # ONLY training SVD, no whitening.
    A,B = design(Z[train]@basis),design(Z[val]@basis)
    w,trace = fit(A,y[train],B,y[val],.2,.01)
    score = losses(B,y[val],w).mean()
    records.append({'kind':'PCA','k':k,'rate':.2,'penalty':.01,'score':score,'w':w,'trace':trace,'basis':basis})
    print(f'PCA k={k}; rate=.2; lambda=.01; validation NLL={score:.6f}')
chosen = min(records,key=lambda r:r['score']) # Stable first-grid-entry tie order.
pca_chosen = min((r for r in records if r['kind']=='PCA'),key=lambda r:r['score'])
def config(r):
    return {key:r[key] for key in ('kind','k','rate','penalty')}
print('FROZEN primary choice:',config(chosen))
print('FROZEN PCA comparison:',config(pca_chosen))
baseline_probability = y[train].mean()
print(f'FROZEN baseline probability={baseline_probability:.6f}; threshold=.5; five fixed calibration bins')

def model_design(r,indices):
    features = Z[indices] if r['basis'] is None else Z[indices]@r['basis']
    return design(features)

test_A = model_design(chosen,test)
test_loss = losses(test_A,y[test],chosen['w'])
test_probability = sigmoid(test_A@chosen['w'])
test_correct = (test_probability>=.5)==y[test]
base_loss = -(y[test]*np.log(baseline_probability)+(1-y[test])*np.log1p(-baseline_probability))
base_correct = (baseline_probability>=.5)==y[test]
pca_A = model_design(pca_chosen,test)
print(f'primary test NLL={test_loss.mean():.6f}; accuracy@.5={test_correct.mean():.6f}; Brier={np.mean((test_probability-y[test])**2):.6f}')
print(f'baseline test NLL={base_loss.mean():.6f}; accuracy@.5={base_correct.mean():.6f}')
print(f'frozen PCA test NLL={losses(pca_A,y[test],pca_chosen["w"]).mean():.6f}; no test-based reselection')
predicted = test_probability>=.5
TP = int(np.sum(predicted&(y[test]==1)))
FP = int(np.sum(predicted&(y[test]==0)))
FN = int(np.sum(~predicted&(y[test]==1)))
TN = int(np.sum(~predicted&(y[test]==0)))
print('test TP/FP/FN/TN:',TP,FP,FN,TN)
calibration = []
for j in range(5):
    selected = (test_probability>=j/5)&((test_probability<(j+1)/5) if j<4 else (test_probability<=1))
    count = int(selected.sum())
    if count:
        item = (j,count,float(test_probability[selected].mean()),float(y[test][selected].mean()))
        calibration.append(item)
        print(f'calibration bin {j}: n={count}; mean predicted={item[2]:.6f}; observed fraction={item[3]:.6f}')
    else:
        print(f'calibration bin {j}: empty; no fraction estimate')

rng = np.random.default_rng(32032)
strata = [np.flatnonzero(y[test]==label) for label in (0,1)]
bootstrap = []
for _ in range(1000):
    indices = np.concatenate([rng.choice(group,len(group),replace=True) for group in strata])
    bootstrap.append((test_loss[indices].mean(),test_correct[indices].mean(),(test_loss[indices]-base_loss[indices]).mean()))
intervals = np.quantile(np.array(bootstrap),[.025,.975],axis=0)
print('approximate conditional stratified percentile 95% intervals; 1000 repeats, PCG64 seed32032:')
for j,name in enumerate(('NLL','accuracy','paired NLL difference primary-minus-baseline')):
    print(name,intervals[:,j])
print('Frozen predictions; fixed observed class counts. Intervals omit refitting and prevalence uncertainty; no universal population guarantee.')

# One complete mean update and final prediction trace on the first training row.
first_A = model_design(chosen,train)
g0 = gradient(first_A,y[train],np.zeros(first_A.shape[1]),chosen['penalty'])
w1 = -chosen['rate']*g0
row = first_A[0]
print('trace first training ID:',int(train[0]),'; raw row:',X[train[0]],'; standardised row:',Z[train[0]],'; label:',int(y[train[0]]))
print('trace model row:',row,'; initial z=0,p=.5,loss=log2; row gradient:',(.5-y[train[0]])*row)
print('trace full mean g0:',g0,'; first updated w:',w1,'; first updated probability:',float(sigmoid(np.array([row@w1]))[0]))
print('trace final w:',chosen['w'],'; final row logit:',float(row@chosen['w']),'; probability:',float(sigmoid(np.array([row@chosen['w']]))[0]))

parser = argparse.ArgumentParser()
parser.add_argument('--output')
args = parser.parse_args()
if args.output:
    pieces = ['<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1010" role="img" aria-label="Executed training validation loss curves, fixed-bin test calibration and training singular values"><rect width="1000" height="1010" fill="white"/><g font-family="system-ui" fill="#1a2e4a">']
    def label(x,y,s,size=20,anchor='start'):
        pieces.append(f'<text x="{x}" y="{y}" font-size="{size}" text-anchor="{anchor}">{s}</text>')
    def line(points,colour):
        path = ' '.join(('M' if j==0 else 'L')+f'{a:.3f},{b:.3f}' for j,(a,b) in enumerate(points))
        pieces.append(f'<path d="{path}" fill="none" stroke="{colour}" stroke-width="3"/>')
    label(500,30,'Actual outputs / 实际输出',24,'middle')
    label(500,70,'Frozen selected model: training / validation NLL',22,'middle')
    pieces.append('<path d="M90,95V285H940" fill="none" stroke="#64748b"/>')
    trace = chosen['trace']
    lower = min(min(row[1:]) for row in trace)-.02
    upper = max(max(row[1:]) for row in trace)+.02
    for index,colour in [(1,'#0284c7'),(2,'#7e22ce')]:
        line([(90+850*t/800,285-190*(row[index]-lower)/(upper-lower)) for row in trace for t in [row[0]]],colour)
    for value in (lower,(lower+upper)/2,upper):
        label(80,290-190*(value-lower)/(upper-lower),f'{value:.3f}',17,'end')
    for t in (0,400,800):
        label(90+850*t/800,310,str(t),18,'middle')
    label(500,343,'Blue train / 蓝训练; purple validation / 紫验证; update index',18,'middle')
    label(500,392,'Frozen test calibration / 冻结测试校准; count per fixed bin',22,'middle')
    pieces.append('<path d="M90,420V650H940" fill="none" stroke="#64748b"/>')
    for j,count,pred,observed in calibration:
        x = 145+160*j
        for off,value,colour in [(0,pred,'#0284c7'),(45,observed,'#7e22ce')]:
            pieces.append(f'<rect x="{x+off}" y="{650-210*value}" width="35" height="{210*value}" fill="{colour}"/>')
        label(x+40,680,f'{j/5:.1f}–{(j+1)/5:.1f}: n={count}',17,'middle')
    for value in (0,.5,1):
        label(80,655-210*value,str(value),18,'end')
    label(500,714,'Blue mean probability; purple observed fraction / 蓝预测均值，紫实际比例',18,'middle')
    label(500,761,'Training-only standardised singular values / 训练标准化奇异值',22,'middle')
    pieces.append('<path d="M90,790V950H940" fill="none" stroke="#64748b"/>')
    logs = np.log10(singular)
    low,high = float(logs.min()-.2),float(logs.max()+.2)
    line([(150+170*j,950-160*(v-low)/(high-low)) for j,v in enumerate(logs)],'#15803d')
    for j,value in enumerate(singular):
        label(150+170*j,978,f'{j+1}: {value:.3f}',18,'middle')
    for value in (low,(low+high)/2,high):
        label(80,955-160*(value-low)/(high-low),f'{value:.2f}',17,'end')
    label(500,1005,'Vertical axis log10(singular value); all five modes retained as numerical evidence',17,'middle')
    pieces.append('</g></svg>')
    Path(args.output).write_text(''.join(pieces),encoding='utf-8')

Python 3.11.8 ; NumPy 1.26.4 ; PCG64 data/split seeds 7/11
data SHA256: 66153f6986d1b4c996690c54eb78b99c5b19c7cef9e47f6c8fda47e2516dcc72
split sizes/positives: [(600, 302), (200, 101), (200, 100)]
training-only standardised singular values: [34.80735141 25.123916   24.89247068 23.18342157  0.36193962]
training-only PCA explained variance ratios: [4.03850571e-01 2.10403718e-01 2.06545032e-01 1.79157012e-01
 4.36667623e-05]
mean NLL + unpenalised-intercept ridge gradient-check error=1.546096e-11; h=1e-05
raw eta=.2 train NLL at 0/1/10: [0.6931471805599453, 76.18494016307439, 104.49551978394732]
full rate=0.05; lambda=0; validation NLL=0.439358
full rate=0.05; lambda=0.01; validation NLL=0.439965
full rate=0.05; lambda=0.1; validation NLL=0.474164
full rate=0.2; lambda=0; validation NLL=0.440024
full rate=0.2; lambda=0.01; validation NLL=0.439577
full rate=0.2; lambda=0.1; validation NLL=0.474160
full rate=1; lambda=0; validation NLL=0.439975
full rate=1; lambda=0.01; validation NLL=0.439575
full rate=1; lambda=0.1; validation NLL=0.474160
PCA k=2; rate=.2; lambda=.01; validation NLL=0.507205
PCA k=3; rate=.2; lambda=.01; validation NLL=0.491433
PCA k=4; rate=.2; lambda=.01; validation NLL=0.439581
PCA k=5; rate=.2; lambda=.01; validation NLL=0.439577
FROZEN primary choice: {'kind': 'full', 'k': 5, 'rate': 0.05, 'penalty': 0.0}
FROZEN PCA comparison: {'kind': 'PCA', 'k': 5, 'rate': 0.2, 'penalty': 0.01}
FROZEN baseline probability=0.503333; threshold=.5; five fixed calibration bins
primary test NLL=0.401414; accuracy@.5=0.825000; Brier=0.129401
baseline test NLL=0.693169; accuracy@.5=0.500000
frozen PCA test NLL=0.406029; no test-based reselection
test TP/FP/FN/TN: 79 14 21 86
calibration bin 0: n=57; mean predicted=0.077600; observed fraction=0.087719
calibration bin 1: n=30; mean predicted=0.302816; observed fraction=0.300000
calibration bin 2: n=32; mean predicted=0.488911; observed fraction=0.437500
calibration bin 3: n=36; mean predicted=0.717846; observed fraction=0.777778
calibration bin 4: n=45; mean predicted=0.917400; observed fraction=0.977778
approximate conditional stratified percentile 95% intervals; 1000 repeats, PCG64 seed32032:
NLL [0.33920608 0.46799432]
accuracy [0.77  0.875]
paired NLL difference primary-minus-baseline [-0.35396333 -0.22517509]
Frozen predictions; fixed observed class counts. Intervals omit refitting and prevalence uncertainty; no universal population guarantee.
trace first training ID: 623 ; raw row: [-0.43464826  3.15570979 -0.44138343  0.02479421 -0.69902493] ; standardised row: [-0.36537528  0.04637014 -0.37203117  0.04652208 -0.67181286] ; label: 0
trace model row: [ 1.         -0.36537528  0.04637014 -0.37203117  0.04652208 -0.67181286] ; initial z=0,p=.5,loss=log2; row gradient: [ 0.5        -0.18268764  0.02318507 -0.18601558  0.02326104 -0.33590643]
trace full mean g0: [-0.00333333 -0.17202693  0.23010693 -0.17178737 -0.08341481  0.00668106] ; first updated w: [ 0.00016667  0.00860135 -0.01150535  0.00858937  0.00417074 -0.00033405] ; first updated probability: 0.49842835081599035
trace final w: [-0.00846219  0.66168633 -1.65527811  0.65585103  0.48320526  0.03208655] ; final row logit: -0.5700549555373022 ; probability: 0.36122414424442933

四项必需故障诊断

故障脚本逐项隔离，与最终模型比较分离，避免故意损坏成为隐藏候选。每项给预测、测得失败和数学修复。

下载 lab3_required_fault_diagnostics.py

"""Four required faults, each demonstrated and repaired on controlled inputs."""
import numpy as np

def sigmoid(z):
    z = np.asarray(z,dtype=float)
    out = np.empty_like(z)
    positive = z>=0
    out[positive] = 1/(1+np.exp(-z[positive]))
    e = np.exp(z[~positive])
    out[~positive] = e/(1+e)
    return out

rng = np.random.default_rng(7)
latent = rng.normal(size=(1000,5))
z1,z2,z3,e1,e2 = latent.T
X = np.column_stack([z1,100*z2,z1+.02*e1,z3,e2])
y = (rng.random(1000)<sigmoid(1.5*z1-2*z2+.5*z3)).astype(int)
split_rng = np.random.default_rng(11)
sizes = np.array([600,200,200])
quotas = sizes*y.sum()/1000
counts = np.floor(quotas).astype(int)
priority = sorted(range(3),key=lambda j:(-(quotas[j]-counts[j]),j))
for j in priority[:int(y.sum()-counts.sum())]:
    counts[j] += 1
blocks = [[] for _ in sizes]
for label,allocations in [(0,sizes-counts),(1,counts)]:
    ids = np.flatnonzero(y==label)
    split_rng.shuffle(ids)
    begin = 0
    for j,count in enumerate(allocations):
        blocks[j].extend(ids[begin:begin+count].tolist())
        begin += count
train,val,test = [np.array(block,dtype=int) for block in blocks]
for ids in (train,val,test):
    split_rng.shuffle(ids)
assert len(set(np.concatenate([train,val,test]).tolist())) == 1000

# 1. Preprocessing leakage: change a held-out value, never a training value.
mean = X[train].mean(0)
changed = X.copy()
changed[val[0],0] += 1e6
assert np.array_equal(changed[train].mean(0),mean)
leaked_change = np.linalg.norm(changed.mean(0)-X.mean(0))
assert leaked_change>999
print('leakage fault: held-out sentinel changes all-data mean by',leaked_change,'; train-only mean unchanged')
# 2. Overflow: preserving the exact finite-logit loss rather than clipping.
z = np.array([-1000.,1000.])
labels = np.array([1.,0.])
with np.errstate(over='ignore',divide='ignore',invalid='ignore'):
    naive_probability = 1/(1+np.exp(-z))
    naive_loss = -labels*np.log(naive_probability)-(1-labels)*np.log(1-naive_probability)
stable_loss = np.maximum(z,0)-labels*z+np.log1p(np.exp(-np.abs(z)))
assert np.isfinite(stable_loss).all()
print('overflow fault: naive losses',naive_loss,'; repaired logit losses',stable_loss)
# 3. Missing mean factor: a smooth numerical check independent of the gradient.
scale = X[train].std(0,ddof=0)
A = np.column_stack([np.ones(12),(X[train[:12]]-mean)/scale])
labels = y[train[:12]]
w = np.linspace(-.2,.3,6)
def objective(v):
    logits = A@v
    return np.mean(np.logaddexp(0,logits)-labels*logits)
g = A.T@(sigmoid(A@w)-labels)/len(labels)
h = 1e-5
numerical = np.array([(objective(w+h*np.eye(6)[j])-objective(w-h*np.eye(6)[j]))/(2*h) for j in range(6)])
correct_error = np.linalg.norm(numerical-g)
wrong_error = np.linalg.norm(numerical-len(labels)*g)
assert correct_error<1e-8 and wrong_error>.1
print(f'mean-factor fault: corrected gradient error={correct_error:.6e}; omitted-factor error={wrong_error:.6e}')
# 4. Prevalence: the same zero rule has an opposite accuracy summary.
for positive in (5,95):
    labels = np.r_[np.ones(positive,dtype=int),np.zeros(100-positive,dtype=int)]
    predicted = np.zeros(100,dtype=int)
    accuracy = np.mean(predicted==labels)
    recall = np.mean(predicted[labels==1]==1)
    print(f'prevalence fault: positives={positive}/100; always-zero accuracy={accuracy:.2f}; positive recall={recall:.2f}')
print('All four faults diagnosed. Seeds define data/generator/draw order; zero-initialised full-batch training itself has no optimiser-seed variation.')

leakage fault: held-out sentinel changes all-data mean by 999.9999999999997 ; train-only mean unchanged
overflow fault: naive losses [inf inf] ; repaired logit losses [1000. 1000.]
mean-factor fault: corrected gradient error=1.186567e-11; omitted-factor error=3.482474e+00
prevalence fault: positives=5/100; always-zero accuracy=0.95; positive recall=0.00
prevalence fault: positives=95/100; always-zero accuracy=0.05; positive recall=0.00
All four faults diagnosed. Seeds define data/generator/draw order; zero-initialised full-batch training itself has no optimiser-seed variation.

哨兵测信息依赖，不声称泄漏总提高分数。极端测试不说原生成数据实际含 1000 logits；它检查数值边界。比例例子改变混合，不是篡改原最终测试。明确这些区别。

阶段门槛：交完整验证表与四诊断，最终配置书面冻结再测试。遗漏泄漏或导数诊断则不通过，无论总分。

实际主模型测试 NLL≈.401414、阈值 .5 准确率 .825、Brier .129401。常基线 NLL .693169、准确率 .5。TP=79,FP=14,FN=21,TN=86，总和 200，正确 165。单独冻结 PCA NLL≈.406029。测试差异不触发重选，也不证明每随机数据都如此。

五预定箱比较预测均值和正类比例，应给各计数。小箱接近不是强证据，空箱无比例。输出箱计数 57,30,32,36,45。有限诊断不证每输入条件校准，也不证偏移后校准；预定箱避免按保留标签设计好看总结。

不确定性、种子与数学追踪

PCG64 种子 32032 做 1,000 分层 bootstrap，在观测标签类内有放回抽测试索引、保持类数，模型基线损失一起配对。百分位 95% 区间近似：NLL [.339206,.467994]，准确率 [.77,.875]，主减基线均值 NLL 差 [−.353963,−.225175]。负差有利于此冻结模型和条件混合估计。

假设类内评估例子代表声明类总体的独立条件抽样，并冻结拟合模型。区间条件于类计数，不含总体比例不确定、重拟合缩放选择变化，非精确有限保证或无条件 IID 二项区间。若要评整个学习算法在新数据的性能，应独立重复完整生成拟合，或用适合目标的有效方法。相关拆分、种子不成为独立测试人。

零初始化、全批次使固定数据与算术下参考确定，优化器不使用种子，改其种子无作用。数据种子改观测，拆分种子改分配，随机小批或初始化才增算法变化。记录生成器与顺序，区分模块 28–30 的来源。平台小数值差用合理容差，不保证所有设备位一致。

第一训练行 ID 623，标准特征约 (−.365375,.046370,−.372031,.046522,−.671813)，标签零。初 θ=0 得 logit0、概率 .5、损失 log2；单例梯度贡献为截距增广行乘 .5。全部 600 均值梯度约 (−.003333,−.172027,.230107,−.171787,−.083415,.006681)。选 η=.05，首更新是该向量乘 −.05，首行更新概率约 .498428。

最终权重给此行 logit≈−.570055、概率 .361224。追原输入→训练 scaler→设计→点积→sigmoid→损失决策，再区分单行贡献和全均值更新。零处即使 λ 非零惩罚梯度也零；后续只罚斜率。脚本打印实际数组和总结，读者可重现每步。

n 行 d 特征，每稠密全批梯度 O(nd)，设计存 O(nd)、权重 O(d)。十三候选各 800 步是预算，完整和 PCA 宽度不同。n≥d 薄 SVD 约 nd²，右基 d×d；各拆分投影约 ndk。线性模型无需核式 n² 存储。时间常数依库硬件，小参考不是部署基准。

经验目标用固定训练标签，最终指标估声明设计风险。固定输入下 Bernoulli 标签仍随机，有总体不可约不确定。小梯度、凸性、低训练 NLL 或导数通过都不单独建立泛化。偏移可改全部指标；标准化改变优化坐标及斜率惩罚几何，标准 λ 不自动等于原坐标先验。

报告建议英文约 1,500–2,500 词，中文提供相当信息量，加代码附录图。含抽样拆分 hash、维数、似然目标梯度、Hessian、秩 PCA、导数检查、原标准化行为、验证选择、故障、冻结测试基线校准、条件不确定限制和完整追踪。引用实际输出，注明改协议。它是数学整合项目，不是只看准确率演示。

推荐至少 80/100，正确诊断泄漏与导数故障。逐项评分并链接材料，高准确率不能抵缺失。

十六小时核心外，可做两层网络或三词元注意力，推反传并在平滑受控例检查，比较与凸线性参考的动力学、容量。保留训练变换、明确选择及冻结测试。

重访 14、18、26、27、28、29、30。本附录自行推似然、梯度曲率，无需库拟合。这是最后一模块，回课程概览 补缺或走互补 CS 路线。