probity. release notes

coverage expansion for modern quantizations and modern architectures. probity can now fully audit models that use non-linear 4-bit quantization (IQ4_NL / IQ4_XS — popular community quants from unsloth, bartowski, mradermacher) and models that use MLA (multi-head latent attention) — the DeepSeek-V2/V3 / GLM 5.1 family. previous versions of probity could not decode IQ4 bytes (so head + expert audits silently returned zero) and could not classify MLA tensors (showing as unknown_components). v0.0.4 fixes both, plus adds support for two auxiliary sub-mechanisms that ship with GLM 5.1: the DSA "indexer" sparse-attention selector and the multi-token-prediction NextN head.

quantization decoder — new coverage

• IQ4_NL dequantizer — 4-bit non-linear codebook (16-entry signed lookup table). 32 elements per block, 18 bytes. Used by every modern unsloth UD-Q4 / IQ4 release.
• IQ4_XS dequantizer — 256-element super-block with 8 per-32 sub-scales, packed via the same 16-entry codebook. 136 bytes per super-block.
• direct consequence: head_audit and expert_audit now produce per-head Q/K/V L2 norms + per-expert magnitudes on IQ4-quantized models. previously these returned silently empty.

architecture classifier — additions

• MLA (Multi-head Latent Attention) — 7 new aliases. DeepSeek-V2/V3 and GLM 5.1 factor Q/K/V through a low-rank latent: attn_q_a → q_lora_rank → attn_q_b → per-head Q, and similarly for K/V via attn_kv_a_mqa → kv_lora_rank → attn_k_b / attn_v_b. probity now maps the per-head "_b" tensors to CompAttnQ/K/V (so head_audit works) and the latent down-projections + their norms to dedicated CompAttnQLatent / CompAttnKVLatent / *_norm classes.
• DSA "indexer" sparse-attention sub-mechanism — 4 new aliases. DeepSeek-V3 introduces an auxiliary attention that scores past positions for sparse-attention selection (indexer.attn_q_b, indexer.attn_k, indexer.k_norm, indexer.proj). classified as its own component family (CompIndexerQ/K/KNorm/Proj) so it appears in per-component-quality rollups without contaminating the main head_audit.
• NextN multi-token-prediction head — 4 new aliases. GLM 5.1 ships an auxiliary head for self-speculative decoding (nextn.eh_proj, nextn.enorm, nextn.hnorm, nextn.shared_head_norm). all four now classify under CompNextN* roles so the speculative-decoding head doesn't show as unknown.
• direct consequence: GLM 5.1 Q8_0 (753.86B params, 79 layers, glm-dsa family) now audits with zero unknown_components, and head_audit reports the full 79 × 64 = 5,056 main attention heads.

testing

• probity test suite extended to cover the new dequantizers + classifier additions; all internal tests green.
• forge (the sibling audit-guided optimizer) shipped its own 110-test foundation across 8 packages in parallel — every public function has unit tests + benchmarks. forge can now safely consume v0.0.4 dossiers with the new role classifications.

migration notes

• any model in your catalog that uses IQ4_NL or IQ4_XS quants should be re-audited under v0.0.4 — the head/expert audit on those will go from empty to fully populated.
• any model in your catalog from the MLA family (DeepSeek-V2, DeepSeek-V3, GLM 5.1) should be re-audited under v0.0.4 — unknown_components will drop from 15+ to 0.
• all earlier-version dossiers remain readable by the catalog server. the version banner above is informational, not a forced-upgrade.

a massive expansion of the audit surface: per-tensor cryptographic fingerprints, class-aware statistical detection, score-weighted readiness rubric, ten new structural signals, the new lineage tab, side-by-side compare view, and rubric calibration for SOTA-class architectures (MoE, hybrid, multilingual). every existing model in the catalog has been re-audited under v0.0.3 — old v0.0.2 dossiers are not forward-compatible.

audit signals — new

• per-tensor SHA-256 fingerprints — every tensor now hashed during the streaming stats pass. content-addressed identity for cross-model lineage detection.
• spectral entropy — Shannon entropy of top-singular-value distribution per SVD'd tensor. low = degenerate; high = full-rank.
• refusal-direction vector + amplification-direction vector — centroid of bottom-N suppressed output-projection rows (and top-N amplified). published as `float32[embed_dim]` arrays plus cosine angle between them. partial fingerprint of RLHF / safety training, derivable from weights alone.
• top-K outlier channels per tensor (previously top-1) — enables real cross-layer outlier-lane detection.
• cross-layer L2 trajectory per Component — depth-wise weight-magnitude line chart per tensor class. reveals training-rate decay, distillation chains, lora-merge discontinuities, fat-layer anomalies.
• intra-model byte-identical tensor groups — using per-tensor SHA-256, detects copy-paste errors / deliberate weight tying / converter dedup.
• vocab forensics — long-token categorisation (URL / code / JSON / GUID / datetime / non-Latin / memorised-phrase). privacy + memorisation signal.
• outlier-lane scan — channel indices that recur as outliers across multiple attention layers. residual-stream-level Dettmers outlier features. quantization-critical.
• per-component quality rollup — model-level training quality broken down by tensor class.
• head diversity per layer — within-block Q-head pairwise specialisation. sparkline visualisation.
• numerical hygiene — NaN/Inf totals across tensors. promoted to a red-bordered alert on the overview tab when nonzero.
• model-level quality rollup — mean / median training-quality, mean prune potential, mean anisotropy.
• lineage fingerprints — distillation late/early std-dev ratio, lora-merge top-singular-value shoulder, quantizer-toolchain shape classification.
• block similarity matrix — NxN cosine similarity of per-block statistical fingerprints. visualised as a contrast-stretched heatmap.
• deployment-cost estimator — per-context-length KV-cache + total memory table.

audit signals — calibrated

• class-aware Hampel detector — outliers now computed WITHIN each Component class (attn_q vs attn_q, ffn_down_exps vs ffn_down_exps). pre-v0.0.3 the detector compared MoE expert tensors against attention norms across the alphabetical name space — producing structurally-induced false positives. now reports a per-finding z-score against the class median, threshold filter at |z| ≥ 3.5.
• script-aware tokenizer detection — bidi / zero-width / control characters demote severity when found in their natural script context (Arabic + RLO is normal; Latin + RLO is an attack). seven Unicode-block ranges classified.
• pure-control-token demotion — tokenizer byte-fallback tokens (standalone `‮`, `​`, `\x01`, etc.) demoted one tier since they're tokenizer hygiene, not adversarial.
• tokenizer per-severity caps — high uncapped, medium ≤ 15, low ≤ 5. legitimate multilingual content can no longer single-handedly tank a score; real attacks always propagate.
• score-weighted readiness rubric for low-severity findings — each finding's penalty is sized by its score (Hampel z-score, outlier-channel max-ratio, glitch-token count). barely-flagged findings cost a fraction of severe ones.
• fix: tokenizer findings no longer double-counted — previously deducted via both `medium_deduction` AND `tokenizer_deduction`. fixed by excluding tokenizer-sourced anomalies from the generic severity buckets.

architecture classifier — additions

• Gemma 4 (dense) — inp_gate, layer_output_scale, per_layer_model_proj, per_layer_proj_norm, per_layer_token_embd, post_ffw_norm, post_norm, proj, rope_freqs (top-level)
• Gemma 4 (MoE) — ffn_gate_up_exps (fused gate+up — Gemma 4's `attn_qkv` for FFN), post_ffw_norm_1, post_ffw_norm_2, pre_ffw_norm_2 (dual-FFN architecture)
• Qwen 3.6 / Qwen 3.5 — attn_gate, ffn_gate_inp_shexp, post_attention_norm, ssm_alpha, ssm_beta; Q-only `attn_qkv` layout detection (Qwen 3.6 reuses the name for a non-fused tensor)
• regex — `blk.N..scale` suffix now strips correctly (Gemma 4 MoE per-tensor quant scale vectors)
• FFN length — three-tier fallback: `feed_forward_length` → `expert_feed_forward_length` → `ffn_down_exps.shape[0]`

UX additions

• lineage tab on every dossier — model-level training quality KPIs, lineage fingerprint cards, block-similarity heatmap, per-component quality table, outlier-lane scan, head-diversity sparkline, L2 trajectory line chart, byte-identical tensor groups
• compare view at /probity/compare?a=X&b=Y — side-by-side dossier diff across 10 sections, with auto-deltas (matching = muted, small diff = sage, large diff = amber)
• catalog filter UI — free-text search across name / family / base-model / org / license; sort by readiness, params, quality, context, memory@32K; tier / family / shape chips; VRAM-fit + context-length pickers; compare-mode toggle; group-by-family clustering
• tokenizer tab additions — embedding glitch tokens (input-side outliers), output projection magnitudes (least- and most-projected tokens), refusal-direction interpretation card, vocab forensics
• architecture tab additions — deployment footprint table (KV cache memory at 4K / 16K / 32K / 64K / 128K / native context)
• tensors tab — three new sortable columns: training quality, pruning potential, anisotropy index — color-tinted by tier
• overview tab — deductions panel rows are now clickable (jump to source data tab); NaN/Inf alert banner (red-bordered, top of page when present)
• comprehensive ELI5 tooltips — every technical label across every tab now hover-bearing with plain-English explanations. 40+ new glossary entries.

bug fixes

• humanBytes 32-bit shift overflow in dossier + compare templates — `1 << 40` in JS evaluates to 256 (32-bit wrap), causing every byte size ≥ 256 to render as "tib". Now uses explicit power-of-2 constants.
• Layer field omitempty — layer 0 was serialised as missing → JS rendered the literal string "undefined". Field is now always emitted.
• compare template missing from allow-list — caused 500 error on /probity/compare. Page allow-list now includes every page template.
• Hampel sub-threshold emission — windowed detector flagged tensors with |global z| < 3.5; those are now filtered out.
• refusal-direction nil-row underweighting — averaging by nominal N rather than rows-actually-read introduced bias when any selected row came back empty. Now divides by the count actually summed.
• amplified-token loop on tiny vocabs — when len(vocab) < 2N, the same tokens were emitted as both suppressed AND amplified. Now properly guards the upper-bound.
• dead ComputeOutlierLanes placeholder removed — confusing exported function that returned a stub.
• dossier null guards — per-component table, head-diversity tooltip, L2-trajectory polyline, block-similarity matrix all hardened against missing fields on partial / older audits.

first public release of the catalog. introduced the deploy-readiness score, the validator gate, MoE accounting, hybrid-model awareness, and the per-attention-head + per-MoE-expert decomposition passes. not backward-compatible with v0.0.1 manifests.

• deploy-readiness score — single 0–100 composite over every signal, with explicit tier banding (clean / advisory / caution / blocked) and per-signal deduction breakdown
• validator gate — 30+ invariants checked before any dossier is written. one violation fails the audit. wrong-but-deterministic findings still wear a signed claim; this gate refuses to sign them.
• MaxElements=0 default — full-tensor read by default; partial reads available via explicit cap with `samples_seen_fraction` honesty
• per-row partial-read fix — dead-row detection no longer falsely flags rows simply because the read stopped before reaching them
• per-attention-head decomposition — Q/K/V/O L2 norms per (layer, head); GQA-aware (q-head ↔ kv-group mapping); within-layer Hampel + median-fraction dead-head detection
• per-MoE-expert decomposition — gate / up / down / router-column L2 per expert; collapsed-expert detection
• hybrid model awareness — layer-type breakdown (attention / SSM / MoE-FFN / dense-FFN), proper handling of models where some blocks lack attention (Nemotron-Hybrid, Jamba)
• metadata-vs-tensor reconciliation — when `head_count_kv` metadata disagrees with K/V tensor shapes (Nemotron-Hybrid common case), audit trusts the tensor shapes; mismatch surfaced as a low-severity anomaly + dossier card
• active parameters — MoE-aware per-token compute footprint accounting
• tied-weights detection — same-offset / byte-match / no-match classification for token_embd ↔ output
• outlier-channel scan — Dettmers-style channel-max / median-max ratio detection per 2-D tensor
• fused-QKV support — Nemotron-class fused attention tensors handled in the per-head pass
• tokenizer attack-vector scan — RTL override, zero-width chars, control chars, BOM, bidi markers (script-blind in v0.0.2, refined in v0.0.3)
• content-addressed fingerprints — header, metadata, descriptors, architecture, tokenizer, file (or split-set) SHA-256 digests with section-by-section coverage
• signed claim manifest — every finding collapsed into a stable claim ID hashed from (section, operator class, operator version, input digest, score, verdict, detail)

initial development build. core GGUF parser, basic per-tensor weight statistics, single-pass dossier writer. dossiers from this version are not forward-compatible with the public catalog format.