If you've read "Attention Is All You Need" and still find yourself hand-waving through what actually happens between Q, K, and V, sometimes the fastest fix is watching the numbers move.
Open the attention visualizer →
The attention visualizer was built using Perplexity for the initial exploration and then Claude Fable for the final html, css and JavaScript code.
- Q, K, V matrices seeded with random integers 1–6 ( for simplification, they start identical, as if no learned projections Wq,Wk and Wv were applied)
- Raw scores — Q × Kᵀ, an N × N matrix
- Scaling by 1/√d
- Attention weights — row-wise softmax of the scaled scores
- Context vectors (Z) — attention weights × V
The Context vectors (Z) are then sent to the rest of the transformer pipeline, including projections, normallizations, MLP, softmax to get the final probability distribution.
Every matrix in the diagram shows its actual computed values, so you can trace a single number from input to output.
Two sliders control the shape of the computation: sequence length (N) from 2 to 8, and model dimension (d) from 2 to 8. The diagram re-lays itself out as the matrices grow and shrink, which makes the dimension bookkeeping (N×d · d×N → N×N) concrete in a way equations don't. A re-seed button rolls new random values so you can see how the softmax sharpens or flattens with different inputs.
The whole thing is one HTML file with the matrix math — transpose, matmul, row-wise softmax — written in about 25 lines of vanilla JavaScript. Here's the core of it:
const transpose = A => A[0].map((_, j) => A.map(row => row[j]));
function matmul(A, B) {
const r = A.length, k = A[0].length, c = B[0].length;
return Array.from({length: r}, (_, i) =>
Array.from({length: c}, (_, j) => {
let s = 0; for (let x = 0; x < k; x++) s += A[i][x] * B[x][j]; return s;
}));
}
const scaleM = (A, f) => A.map(row => row.map(v => v * f));
const softmaxRows = A => A.map(row => {
const mx = Math.max(...row); // subtract max for stability
const ex = row.map(v => Math.exp(v - mx));
const sum = ex.reduce((a, b) => a + b, 0);
return ex.map(v => v / sum);
});
// the attention pipeline itself
const KT = transpose(K); // d x N
const raw = matmul(Q, KT); // N x N raw scores
const attn = softmaxRows(scaleM(raw, 1 / Math.sqrt(d)));
const Z = matmul(attn, V); // N x d context vectors