Lyra C's Neuron Explorer Lets Researchers Steer Qwen3.5-4B's Hidden Channels
A new open-source tool pushes every MLP neuron in Qwen3-4B up and down, saves every rollout, then lets you browse the model as a 3D map.
Takeaways
- Lyra C released neuron-explorer, a steering-first interpretability tool for Qwen3-4B with a 3D map of every MLP neuron.
- Each neuron is pushed up and down on a few prompts, with every rollout stored and browsable in a local web UI.
- Steering doses are fractions of the median residual norm, keeping their meaning across layers and neurons.
- Sweeps run on a CUDA GPU pod using a steering fork of vLLM; shipped caches let you browse without rerunning.
- Includes an MCP integration that lets the Claude CLI drive the explorer, run the live model and move the map.
- Already ported to Gemma 4 E4B and Gemma 3 4B and cited as inspiration by multiple downstream interpretability projects.
Lyra C has released Neuron Explorer, an open-source interpretability tool that tests individual MLP channels by injecting their output directions into a model’s residual stream and recording the resulting text. The initial release targets Qwen3.5-4B and includes cached sweeps, a local visual interface, live inference tools, and an agent integration.
| Component | Details |
|---|---|
| Primary model | Qwen3.5-4B |
| Core method | Positive and negative steering of every MLP channel in a selected layer |
| Interface | Local web app at localhost:8791 |
| Fresh sweeps | CUDA GPU and the project’s steering fork of vLLM |
| Live inference | MPS, CUDA, or CPU |
| Reported ports | Gemma 4 E4B and Gemma 3 4B |
From observation to intervention
Transformer MLPs contain intermediate channels commonly described as neurons. Each channel contributes through a column of the MLP’s down-projection matrix, which writes information back into the residual stream, the shared state consumed by later layers. Neuron Explorer injects a scaled version of that write direction and measures how generation changes.
Intervention strength is expressed as a fraction of the median residual norm leaving the selected layer. A dose of +0.5 therefore adds a vector with roughly half that reference magnitude. This normalization makes doses more comparable across channels and layers, although equal doses can still produce different behavioral effects.
The default sweep crosses a small prompt set with positive and negative doses. It also includes an unsteered baseline and a free-form prompt at a strong positive dose. The pipeline retains each rollout, including generated text, token IDs, per-step log probabilities, and top-k token predictions. Per-channel records also include a generated name, lens results, connection data, and a three-column chat comparing negative, zero, and positive steering.