LLM Visualizer

See how large language models process text, step by step.

Tokenization

Text is broken into smaller units called tokens. These are the building blocks that LLMs actually process, not words, not characters, but tokens.

Input
62/1000
Tokens9 tokens
Artificial·intelligence·is·transforming·how·we·build·software.
9Tokens
62Characters
6.9Chars / Token
8Words
tokens flow into embedding

Embedding

Each token is converted into a high-dimensional vector, a list of numbers that captures its meaning. Similar words end up close together in this space. Below is a 2D projection of these vectors.

0/0 tokens mapped
embeddings receive position signals

Positional Encoding

The model knows what each token means, but not where it appears. Positional encoding adds a unique signal to each position so the model can distinguish word order.

Sentence A
dog0bites1man2
≠
Sentence B
man0bites1dog2

Same tokens, different meaning. Without a position signal, the model treats both sentences identically. The subscript is each token's position, it changes the final vector.

Sinusoidal pattern8 positions × 50 dimensions · even dims = sin, odd = cos · click a row to inspect
−1
+1
Position signal: “Position 0”position 0
Embeddingmeaning of the word
+
Position Signalwhere it appears
=
Final Vectormeaning + order → attention
positioned vectors enter attention

Self-Attention

Type text in the tokenizer above to see attention in action.

attention output enters FFN block

FFN & Residuals

Type text in the tokenizer above to see the feed-forward network in action.

processed tokens predict the next word

Generation

Type text in the tokenizer above to see next-token prediction.