Uploaded July 2026 | Updated November 2026, 16 hours ago
"Active Circuit Discovery: A Multi-Action POMDP Agent for Causal Feature Identification in Transformer Attribution Graphs"
Dr. Sharath Sathish (London, UK).
mdpi.com/2073-8994/18/6/1043
Code: github.com/SharathSPhD/ActiveCIrcuitDiscovery
Mechanistic interpretability seeks to reverse-engineer the computational circuits within large language models, but current methods rely on exhaustive or heuristic search over exponentially many feature interactions. This paper introduces Active Circuit Discovery (ACD), a framework that combines attribution-graph analysis with active inference to select interventions efficiently. ACD uses Anthropic’s circuit-tracer library as its attributiongraph backend, applying Edge Attribution Patching with transcoders to identify the active transcoder features for each prompt. A partially observable Markov decision process (POMDP) agent, implemented with pymdp, maintains a multi-factor generative model of feature importance, layer role, and causal influence. At each step, the agent selects both a target feature and an intervention type (ablation, activation patching, or feature steering) by minimising Expected Free Energy over the joint feature–action space, and it learns its observation model online through Dirichlet parameter updates. ACD is an interventionselection layer over existing attribution-graph tools; it is not a whole-circuit discovery method, and no claim of state-of-the-art circuit discovery is made. The framework is evaluated on Gemma-2-2B (26 layers) and Llama-3.2-1B (16 layers) across four settings: Indirect Object Identification (IOI), multi-step reasoning, feature steering, and a multidomain benchmark spanning geography, mathematics, science, logic, and history. With a budget of 20 interventions per prompt, an ablation-only agent scored by bounded oracle efficiency against the ablation oracle reaches 82.0% efficiency on Gemma IOI and 73.0% on Gemma multi-step. It exceeds random selection by 43.5% (relative) on Gemma IOI (paired permutation p = 0.031) and is competitive with greedy ranking, a heuristic UCB bandit, and a plain UCB baseline. A direct Edge-Attribution-Patching ranking is itself a strong baseline that the agent does not consistently surpass, and on Llama multi-step the agent reaches 9.3% efficiency (37.8% with finer layer-role bins). All comparisons report bootstrap 95% confidence intervals. The full multi-action agent is characterised separately by a Relative Cumulative KL, a steering-driven amplification factor reported apart from the bounded efficiency. Feature steering changes the top-1 prediction in a dose-dependent manner, but a matched random-feature control shows that circuit-selected features are only marginally, and not significantly, more steerable than random active features at large multipliers, indicating that part of the effect is generic activation scaling. Multi-domain analysis shows task-dependent circuit structure, with IOI circuits concentrated in late layers and reasoning and scientific knowledge recruiting early and middle layers. Code, notebooks (free T4), AMD64/aarch64 Docker images, and raw results are publicly available.
----
Active Inference Institute information:
Website: https://www.activeinference.institute/
Activities: https://activities.activeinference.institute/
Discord: https://discord.activeinference.institute/
Donate: http://donate.activeinference.institute/
YouTube: youtube.com/c/ActiveInference
X: https://x.com/InferenceActive
Active Inference Livestreams: https://video.activeinference.institute/
------
CHAPTERS
0:00 Welcome and introduction
1:30 Interpretability meets active inference
6:00 From thesis to paper
7:30 Which feature to probe next?
10:30 Interactive interface and code
12:00 Transformers as the foundation
13:30 Features and circuits
15:00 Example: indirect object identification
18:00 Tokens and the residual stream
22:30 Why interpretability matters
24:00 Superposition
27:00 Interlude (no speech)
42:00 Neuronpedia: features in Gemma
45:00 Interpretability as abstraction
46:30 MLPs and the residual stream
48:00 Features as vocabulary, circuits as explanations
49:30 Attention heads
51:00 Limits of manual analysis
52:30 Ablation and KL divergence
55:30 Edge attribution patching
57:00 Transcoders and attribution graphs
58:30 Walking through the example
1:03:00 The cost of exhaustive experiments
1:06:00 Summary of part one
1:09:00 Gaps in interpretability
1:10:30 Active inference for circuit discovery
1:12:00 Expected information gain
1:13:30 Analogy: a physician choosing tests
1:15:00 Mapping the terminology
1:18:00 Entropy and stopping
1:19:30 The pymdp model
1:22:30 States: importance, layer role, causal influence
1:24:00 Ablation, patching and steering
1:27:00 Widget demo
1:28:30 Epistemic vs. pragmatic phases
1:31:30 Results
1:33:00 Artifacts and side effects
1:34:30 Discussion
"Active Circuit Discovery: A Multi-Action POMDP Agent for Causal Feature Identification in Transformer Attribution Graphs"
Dr. Sharath Sathish (London, UK).
mdpi.com/2073-8994/18/6/1043
Code: github.com/SharathSPhD/ActiveCIrcuitDiscovery
Mechanistic interpretability seeks to reverse-engineer the computational circuits within large language models, but current methods rely on exhaustive or heuristic search over exponentially many feature interactions. This paper introduces Active Circuit Discovery (ACD), a framework that combines attribution-graph analysis with active inference to select interventions efficiently. ACD uses Anthropic’s circuit-tracer library as its attributiongraph backend, applying Edge Attribution Patching with transcoders to identify the active transcoder features for each prompt. A partially observable Markov decision process (POMDP) agent, implemented with pymdp, maintains a multi-factor generative model of feature importance, layer role, and causal influence. At each step, the agent selects both a target feature and an intervention type (ablation, activation patching, or feature steering) by minimising Expected Free Energy over the joint feature–action space, and it learns its observation model online through Dirichlet parameter updates. ACD is an interventionselection layer over existing attribution-graph tools; it is not a whole-circuit discovery method, and no claim of state-of-the-art circuit discovery is made. The framework is evaluated on Gemma-2-2B (26 layers) and Llama-3.2-1B (16 layers) across four settings: Indirect Object Identification (IOI), multi-step reasoning, feature steering, and a multidomain benchmark spanning geography, mathematics, science, logic, and history. With a budget of 20 interventions per prompt, an ablation-only agent scored by bounded oracle efficiency against the ablation oracle reaches 82.0% efficiency on Gemma IOI and 73.0% on Gemma multi-step. It exceeds random selection by 43.5% (relative) on Gemma IOI (paired permutation p = 0.031) and is competitive with greedy ranking, a heuristic UCB bandit, and a plain UCB baseline. A direct Edge-Attribution-Patching ranking is itself a strong baseline that the agent does not consistently surpass, and on Llama multi-step the agent reaches 9.3% efficiency (37.8% with finer layer-role bins). All comparisons report bootstrap 95% confidence intervals. The full multi-action agent is characterised separately by a Relative Cumulative KL, a steering-driven amplification factor reported apart from the bounded efficiency. Feature steering changes the top-1 prediction in a dose-dependent manner, but a matched random-feature control shows that circuit-selected features are only marginally, and not significantly, more steerable than random active features at large multipliers, indicating that part of the effect is generic activation scaling. Multi-domain analysis shows task-dependent circuit structure, with IOI circuits concentrated in late layers and reasoning and scientific knowledge recruiting early and middle layers. Code, notebooks (free T4), AMD64/aarch64 Docker images, and raw results are publicly available.
----
Active Inference Institute information:
Website: https://www.activeinference.institute/
Activities: https://activities.activeinference.institute/
Discord: https://discord.activeinference.institute/
Donate: http://donate.activeinference.institute/
YouTube: youtube.com/c/ActiveInference
X: https://x.com/InferenceActive
Active Inference Livestreams: https://video.activeinference.institute/
------
CHAPTERS
0:00 Welcome and introduction
1:30 Interpretability meets active inference
6:00 From thesis to paper
7:30 Which feature to probe next?
10:30 Interactive interface and code
12:00 Transformers as the foundation
13:30 Features and circuits
15:00 Example: indirect object identification
18:00 Tokens and the residual stream
22:30 Why interpretability matters
24:00 Superposition
27:00 Interlude (no speech)
42:00 Neuronpedia: features in Gemma
45:00 Interpretability as abstraction
46:30 MLPs and the residual stream
48:00 Features as vocabulary, circuits as explanations
49:30 Attention heads
51:00 Limits of manual analysis
52:30 Ablation and KL divergence
55:30 Edge attribution patching
57:00 Transcoders and attribution graphs
58:30 Walking through the example
1:03:00 The cost of exhaustive experiments
1:06:00 Summary of part one
1:09:00 Gaps in interpretability
1:10:30 Active inference for circuit discovery
1:12:00 Expected information gain
1:13:30 Analogy: a physician choosing tests
1:15:00 Mapping the terminology
1:18:00 Entropy and stopping
1:19:30 The pymdp model
1:22:30 States: importance, layer role, causal influence
1:24:00 Ablation, patching and steering
1:27:00 Widget demo
1:28:30 Epistemic vs. pragmatic phases
1:31:30 Results
1:33:00 Artifacts and side effects
1:34:30 Discussion










