Skip to main content
Back

Lyra C's Neuron Explorer Lets Researchers Steer Qwen3.5-4B's Hidden Channels

A new open-source tool pushes every MLP neuron in Qwen3-4B up and down, saves every rollout, then lets you browse the model as a 3D map.

Repo, Full Access
  • Lyra C released neuron-explorer, a steering-first interpretability tool for Qwen3-4B with a 3D map of every MLP neuron.
  • Each neuron is pushed up and down on a few prompts, with every rollout stored and browsable in a local web UI.
  • Steering doses are fractions of the median residual norm, keeping their meaning across layers and neurons.
  • Sweeps run on a CUDA GPU pod using a steering fork of vLLM; shipped caches let you browse without rerunning.
  • Includes an MCP integration that lets the Claude CLI drive the explorer, run the live model and move the map.
  • Already ported to Gemma 4 E4B and Gemma 3 4B and cited as inspiration by multiple downstream interpretability projects.

Lyra C has released Neuron Explorer, an open-source interpretability tool that tests individual MLP channels by injecting their output directions into a model’s residual stream and recording the resulting text. The initial release targets Qwen3.5-4B and includes cached sweeps, a local visual interface, live inference tools, and an agent integration.

Project overview
Component Details
Primary model Qwen3.5-4B
Core method Positive and negative steering of every MLP channel in a selected layer
Interface Local web app at localhost:8791
Fresh sweeps CUDA GPU and the project’s steering fork of vLLM
Live inference MPS, CUDA, or CPU
Reported ports Gemma 4 E4B and Gemma 3 4B

From observation to intervention

Transformer MLPs contain intermediate channels commonly described as neurons. Each channel contributes through a column of the MLP’s down-projection matrix, which writes information back into the residual stream, the shared state consumed by later layers. Neuron Explorer injects a scaled version of that write direction and measures how generation changes.

Intervention strength is expressed as a fraction of the median residual norm leaving the selected layer. A dose of +0.5 therefore adds a vector with roughly half that reference magnitude. This normalization makes doses more comparable across channels and layers, although equal doses can still produce different behavioral effects.

The default sweep crosses a small prompt set with positive and negative doses. It also includes an unsteered baseline and a free-form prompt at a strong positive dose. The pipeline retains each rollout, including generated text, token IDs, per-step log probabilities, and top-k token predictions. Per-channel records also include a generated name, lens results, connection data, and a three-column chat comparing negative, zero, and positive steering.