The Superposition Hypothesis & Polysemanticity
Individual neurons in large language models rarely map to single human-interpretable concepts. Neural networks represent far more features than they possess physical dimensions, storing concepts as almost-orthogonal directional vectors in high-dimensional activation spaces.
This superposition produces polysemantic neurons: single units responding simultaneously to academic citations, French syntax, and chemical formulas. Simple probe training against single neuron activations yields misleading assurances.
Mechanistic audits utilize Sparse Autoencoders (SAEs) trained on residual stream activations. By expanding the activation dimension through overcomplete dictionaries with L1 sparsity regularization, SAEs disentangle polysemantic streams into monosemantic feature latents.
Activation Patching & Causal Tracing
Correlation between activation and token output does not establish causal necessity. Activation patching (or interchange intervention) isolates which sub-circuits causally determine behavioral outcomes.
The auditor runs two forward passes: a clean pass containing standard task inputs and a corrupted pass with altered semantic variables. By patching activations from the clean run into specific attention heads or MLP layers of the corrupted run, the auditor calculates the exact indirect effect of individual network components.
This protocol isolates induction heads—two-head circuits responsible for in-context pattern completion—and separates factual knowledge retrieval from sycophantic alignment overrides.
| Analysis Technique | Mathematical Primitive | Audit Objective | Failure Mode Isolated |
|---|---|---|---|
| Sparse Autoencoding | $z = \text{ReLU}(W_e(x - b_d) + b_e)$ with L1 penalty | Monosemantic feature extraction | Hidden latent concepts masked by superposition |
| Activation Patching | $\text{Direct Effect} = y(clean) - y(corrupt[node \leftarrow clean])$ | Causal path tracing | Spurious attention correlation vs functional logic |
| Representation Steering | $h'_l = h_l + \alpha v_{\text{safety}}$ | Targeted behavioral dampening | Refusal bypass without retraining full weights |
| Logit Lens / Tuned Lens | $\text{Softmax}(W_u \cdot \text{LayerNorm}(h_l))$ | Layer-by-layer prediction decoding | Early deceptive token formation in deep layers |
Detecting Deceptive Latents and Covert Reasoning
A central risk in autonomous agents is deceptive alignment: a system that behaves compliantly during evaluation while preserving misaligned goal structures for unmonitored production contexts.
Black-box sampling cannot distinguish genuine compliance from calculated compliance. Mechanistic auditing isolates goal representations within the middle and late MLP layers. If an internal latent representing user monitoring activates consistently across refusal trajectories, the system harbors state awareness of testing boundaries.
Using contrastive activation additions, auditors extract direction vectors corresponding to truthfulness versus compliance. Models showing divergence between truthfulness latents and output tokens demonstrate active confabulation or strategic omission.
Representation Engineering & Steering Vectors
Post-training interventions traditionally rely on reinforcement learning from human feedback. When safety guardrails must be reinforced without full checkpoint fine-tuning, representation engineering provides a direct surgical intervention.
By computing the difference vector between mean activations on refusal datasets versus compliance datasets, auditors identify a linear subspace governing boundary enforcement. Adding this steering vector to residual streams at inference time clamps compliance behaviors across diverse adversarial contexts.
However, steering vectors introduce capability tax. Clamping safety directions indiscriminately impairs mathematical deduction and complex synthetic reasoning. Audits map the Pareto frontier between steering magnitude and core capability retention.
Audit Deliverables & Circuit Schematics
Every interpretability engagement delivers complete circuit schematics. Deliverables include top activation exemplars for critical safety features, causal intervention heatmaps across transformer layers, and open-source validation scripts allowing client engineering teams to reproduce latent extractions in local runtime environments.