How Transformers Actually Work: One Sentence, Start to Finish

First appeared in 2017 in the “Attention is all you need” article by Google, the transformer architecture is at the heart of groundbreaking models like ChatGPT and Claude, sparking a new wave of excitement in the AI community.

A Transformer is surprisingly simple. It takes a sequence of tokens as input and produces a sequence of tokens as output. In other words, it is a tokens-in, tokens-out machine. Since tokens come from text and are converted back into text, we can also think of it as a text-in, text-out machine.

To put it simply:

A transformer is a type of artificial intelligence model that learns to understand and generate human-like text by analyzing patterns in large amounts of text data.

Alright, that's it for the definition, let's dive deep, but first I want to clarify:

We'll lean on one core example — a single sentence — for everything hands-on: tokenization, embeddings, the attention arithmetic, multi-head reasoning, and the full translation walkthrough at the end. A couple of short, separate sentences show up briefly along the way, purely to illustrate one specific point each, before we come back to the main thread. The vectors we'll use are also deliberately tiny numbers instead of big ass ones. So, the arithmetic stays doable by hand.
The operations are identical at real scale; only the matrices get bigger.

Pack your seat belt.

Why Transformers?

Before transformers, the standard approach was to read a sentence one word at a time, carrying a running "memory" forward from each word to the next, roughly what a recurrent neural network (RNN) does.

This had two big problems:

• It's Slow: Word two depends on word one's output, word three on word two's, and so on. The whole sequence has to be processed strictly in order — step 50 genuinely cannot begin before step 49 is done, and no amount of extra compute changes that.
• Forgetting: Even with gating designed to preserve it, information about word one has to pass through every intermediate state to reach word fifty — like a message passed down a long line of people, picking up a little noise at each handoff.

Here comes Transformer to the rescue:

• it processes all input tokens paralelly where each word's attention output ends up as a blend of information from every other word.
• As everything happens in parallel, long-range connections are no longer a major problem, training becomes faster.

The Encoder and the Decoder

The original transformer is built from two halves working together:

• The encoder reads the input and builds a rich understanding of it.
• The decoder takes that understanding and produces the output, one word at a time.

One simple way to hold onto this: the encoder does the reading, the decoder does the writing.

Step 1: Tokenization and Embeddings

Neural networks only do math. So before anything else happens, your text has to become numbers first. This happens in two stages.

Tokenization splits raw text into pieces called tokens. A token might be a whole word, a fragment of a word, or a punctuation mark — real tokenizers often split words into smaller sub-word pieces, but for this walkthrough we'll keep it simple and treat each word as one token.

Take our sentence, "Birds build nests." Tokenized, that's three tokens: Birds, build, nests.

Embedding is the next step: it turns each token into a vector, a list of numbers standing in for its meaning. Similar words end up with similar vectors.
For e.g., quick and speedy would sit close together, quick and umbrella wouldn't.

This starting vector isn't the model's full understanding of the word yet, though — it knows nothing about this sentence specifically. That comes next.

For our sentence, let's use small, easy-to-follow 4-number vectors:

Step 2: Positional Encoding

There's a wrinkle: the encoder looks at every token at the same time, not in sequence. That's what makes it fast, but it also means the model can't tell "Birds build nests" apart from "Nests build Birds" unless we give it that information some other way.

Positional encoding fixes this by adding a second vector to each token — one that encodes its position before anything else happens. (Real transformers generate these with fixed sine and cosine functions at different frequencies, rather than picking arbitrary numbers, but the effect is the same: give every position a distinct, consistent signature).

These get added directly onto the word embeddings:

From here on, each token's vector carries two things fused together: what the word means, and where it sits in the sentence.

Step 3: Self-Attention

Try reading this sentence: "The suitcase didn't fit in the car because it was too small."

Now read this one: "The suitcase didn't fit in the car because it was too big."

One word changed — "small" to "big" — and "it" now points somewhere completely different. In the first sentence, "it" is the car. In the second, it's the suitcase. Nothing about the grammar changed, only the meaning did — and a model has to get this right, or it hasn't really understood the sentence at all.

This is what self-attention does: for every word, it looks at every other word in the sentence and decides how much each one matters, based on meaning.

Query, Key, and Value

To do this, every token gets converted into three separate vectors:

• Query (Q) — what this word is trying to find out
• Key (K) — what this word has to offer, like a label
• Value (V) — the actual content this word contributes, if picked

A useful picture: imagine a library. You walk in with a question in mind — that's your Query. Every book has a label on its spine summarizing what it's about — that's each book's Key. Attention compares your Query against every book's Key to find what's relevant, then hands you the actual contents of the best matches — the Value.

Worth being precise about, though: this is only an intuition. A Query isn't literally a sentence-like question, and a Key isn't a readable label — both are just learned vectors, and it's their dot product that turns out to be useful for deciding how information should move between positions. Nothing in there is human-readable, the network just finds that this structure works.

The Math, Step by Step

Let's use the plain embeddings from Step 1:

Birds = [1, 1, 0, 1]
build = [1, 0, 1, 0]
nests = [0, 1, 1, 1]

X =    | 1 1 0 1  | ← Birds  
       | 1 0 1 0  | ← build
       | 0 1 1 1  | ← nests

We get Q, K, and V by multiplying X by three learned weight matrices —
W_Q, W_K, and W_V. Let's set d_k = 3, so each has shape 4×3 (in a real model these start random and are learned; here they're just chosen to keep the arithmetic clean):

W_Q =            W_K =          W_V =    
| 0 0 1 |        | 1 0 0 |      | 1 0 1 |
| 0 1 1 |        | 0 0 1 |      | 1 1 0 |
| 1 0 0 |        | 0 1 0 |      | 0 1 1 |
| 1 1 0 |        | 0 0 0 |      | 0 1 0 |

Multiplying X by each gives:

Q =          K =            V =      
| 1 2 2 |    | 1 0 1 |      | 2 2 1 |  -> Birds
| 1 0 1 |    | 1 1 0 |      | 1 1 2 |  -> build
| 2 2 1 |    | 0 1 1 |      | 1 3 1 |  -> nests

Birds gives a score of 4 to nests, versus 3 for itself and build — nests is already standing out as the most relevant word for Birds.

Scaling. Divide every value by √d_k. With d_k = 3, √3 ≈ 1.732:

Large raw scores would push softmax toward extreme, all-or-nothing outputs, which makes training harder — scaling keeps things in a range where softmax gives a more useful, graded distribution.

Softmax turns each row into percentages:

softmax(x_i) = e^(x_i) / (sum of e^(x_j) for every value in that row)

Working through the Birds row, [1.732, 1.732, 2.309]:

e^1.732 ≈ 5.652
e^1.732 ≈ 5.652
e^2.309 ≈ 10.068

Sum = 21.372

Birds → Birds: 5.652 / 21.372 ≈ 0.264
Birds → build: 5.652 / 21.372 ≈ 0.264
Birds → nests: 10.068 / 21.372 ≈ 0.471

Do this for every row and you get:

Each row sums to 1.0.
A clean pattern falls out: Birds pays the most attention to nests, build pays the most attention to Birds, nests pays the most attention to build — every word leaning on a different word, exactly the kind of relationship self-attention is built to find.

One honest note: I picked these specific matrices because they keep the arithmetic clean and the resulting pattern easy to read. In a real model, these start out random and get learned during training — an untrained head produces a far messier, less interpretable pattern than the tidy one we're about to compute.

Getting the final output — multiply the attention weights by V:

Birds:  0.264×[2,2,1] + 0.264×[1,1,2] + 0.471×[1,3,1]  =  [1.264, 2.207, 1.264]
build:  0.471×[2,2,1] + 0.264×[1,1,2] + 0.264×[1,3,1]  =  [1.471, 2.000, 1.264]
nests:  0.264×[2,2,1] + 0.471×[1,1,2] + 0.264×[1,3,1]  =  [1.264, 1.793, 1.471]

Birds's new vector leans heavily on nests's Value — Birds's representation is no longer just about Birds. It now carries a blend of the whole sentence, weighted by relevance. That blending, repeated for every word, is what attention does.

Step 4: Multi-Head Attention

Look again at "Birds build nests." A single sentence, but several relationships are tangled up inside it:

• Birds is the subject; build is the action; nests is the object
• Birds and build form a subject–verb pair
• build and nests form a verb–object pair
• Birds and nests are connected indirectly, through the act of building itself

Could one attention head, with just one Q/K/V projection, track all of these equally well? In practice, not really — a single head tends to specialize in one dominant pattern. So transformers run several heads in parallel, each with its own learned Q, K, and V, so each is free to specialize.

The Mechanics

Take BERT-base as a concrete example: d_model = 768 (every token is a 768-number vector), split across h = 12 heads.

Each head has its own learned weight matrices — its own W_Q, W_K, and W_V — which take the full 768-number vector and project it down into a smaller space: d_k = 768 / 12 = 64. Every head sees the whole token vector; it just looks at it through its own lens.d_k = d_model / h = 768 / 12 = 64

head_i = Attention(Q_i, K_i, V_i) = softmax(Q_i × K_i^T / √64) × V_i

where Q_i, K_i, and V_i all come from projecting the same full input vector through head i's own weights.

Twelve heads run this way side by side, each free to pick up on a different kind of pattern. One might lean toward subject-verb relationships, another toward nearby words, another toward something less nameable. Their outputs (64 numbers each) get lined up back into one 768-number vector, then passed through one more learned matrix, W_O, blending everything together.

You'll see this same multi-head mechanism in three spots as we go: inside the encoder (every input word attends to every other input word), inside the decoder (each output word attends only to itself and the words already generated), and between the two (the decoder checks back against the encoder's output).

Step 5: The Feed-Forward Network

Attention is a group conversation.
Every word checking in with every other word. The feed-forward network (FFN) is the opposite: a private step. Once a word has gathered context from its neighbors, the FFN takes that word's vector and processes it alone.
Every transformer layer repeats this rhythm: attention first, then the FFN.

Expand, Activate, Contract

1. Expand. The input vector is multiplied by a weight matrix and stretched into a much larger dimension — 768 numbers might expand to 3072.

hidden = input × W1 + b1

2. Activate. The expanded vector passes through an activation function, introducing the non-linearity the network needs.

activated = activation(hidden)

3. Contract. A second weight matrix shrinks it back to the original size.

output = activated × W2 + b2


Put together: FFN(x) = activation(x × W1 + b1) × W2 + b2

The bigger hidden layer is scratch space — room to combine features before compressing back down to a size the rest of the model can use.

Activation Functions

Why do you need step 2 at all? Without it, the FFN is just two matrix multiplications back to back and stacking linear operations, no matter how many, only ever produces another linear operation.
Picture drawing a curve with nothing but straight rulers laid end to end: no matter how many you chain together, you still only get a straight line. Activation functions are the bend that lets the network model the curved, context-dependent patterns real language is full of.

ReLU is the simplest one:

ReLU(x) = max(0, x)

Think of it as a strict bouncer at each neuron's door: positive values are waved straight through, unchanged. Negative values are turned away entirely — replaced with zero, no partial credit.

ReLU(3) = 3 (positive → let it through as-is)
ReLU(-2) = 0 (negative → blocked, becomes zero)

GELU, used in models like GPT and BERT, is a gentler version — a bouncer with a softer policy. Instead of a hard cutoff at zero, it lets small negative values through partially, only clamping down hard on strongly negative ones:

ReLU and GELU nearly agree at the extremes; they mostly disagree in the middle, where GELU's softer curve lets a little signal leak through that ReLU would discard outright. That smoother transition tends to make training easier.

Worth knowing too: SwiGLU, used in models like LLaMA, adds a learned gate — a second, parallel calculation that decides how much of each signal to let through, rather than applying one fixed rule to everyone. That flexibility costs one extra weight matrix (SwiGLU needs three total instead of two) but tends to noticeably improve performance, which is why several recent large language models have adopted it.

What Does the Feed-Forward Network Actually Learn?

Attention figures out which words matter to which; the feed-forward network takes each word's now-context-aware vector and refines it, using patterns picked up during training.

Say the model is partway through "The tallest mountain in the world is." Attention has already linked tallest, mountain, and world together. The feed-forward network takes that combined signal and pushes the model's next-word guess toward something it's seen match this pattern many times before: Everest.

Step 6: Residual Connections and Layer Normalization

Residual connections solve a simple problem: as data passes through many layers, useful information can get diluted. The fix is that after each sub-layer does its work, its output is added back to whatever went into it, rather than replacing it:

output = sub-layer(input) + input

Think of it like annotating a document instead of rewriting it. Every time you learn something new, you don't erase the original page and start over — you stick a note on top and keep both. The original signal never gets thrown away; new information just accumulates on top of it, layer after layer.

Layer normalization handles a different problem: keeping the numbers themselves from spiraling out of a manageable range as they pass through many stacked layers. After each residual connection, every vector is rescaled, row by row, so its values sit in a stable range.

Here's the arithmetic on a simple example row, [6, 9, 12]:

Mean = (6 + 9 + 12) / 3 = 9
Deviations from mean: [6-9, 9-9, 12-9] = [-3, 0, 3]
Variance = ((-3)² + 0² + 3²) / 3 = (9 + 0 + 9) / 3 = 6
Standard deviation = √6 ≈ 2.449

Normalized = [-3/2.449, 0/2.449, 3/2.449] ≈ [-1.225, 0, 1.225]

Every row gets this same treatment, independently.

Two details worth adding. First, real layer normalization doesn't stop at this raw normalized value, it also applies a learned scale and shift to each feature afterward, so the network isn't permanently forced to keep every representation centered at exactly zero with a standard deviation of exactly one.
It can stretch or shift that range back out if that turns out to be useful.
Second, the ordering shown here — normalize after adding the residual — is the original transformer's design (LayerNorm(x + Sublayer(x)), sometimes called post-norm).

Plenty of newer architectures normalize before the sub-layer instead (pre-norm), or swap LayerNorm for a variant like RMSNorm.
Different choice, same underlying goal: keep the numbers well-behaved as they pass through many layers.

Step 7: Stacking Layers

One multi-head attention sub-layer plus one feed-forward sub-layer (each wrapped in its own residual connection and layer normalization) makes up one transformer layer, sometimes called a transformer block. Real models don't use just one — they stack many, one feeding directly into the next.

Every layer has its own independently learned weights. Nothing is shared between layers. More layers generally means richer, more abstract representations, at the cost of more computation and parameters to train.

Inside the Encoder

Every encoder layer repeats the same rhythm: self-attention first (every word checks in with every other word), then the feed-forward network (each word reflects individually on what it just heard).

Stack enough of these and understanding deepens with each pass. Early on, "build" starts connecting directly to "Birds" and "nests". A few layers in, those connections combine — "build" isn't just linked to two separate words anymore, it starts representing the whole idea of birds building nests. By the top of the stack, every word's vector is shaped by the entire sentence around it: "nests" isn't just a generic noun anymore, it specifically encodes the thing birds are building, in this sentence.

Once the encoder finishes, you're left with one context-rich vector per input word, which gets handed off to the decoder next.

Inside the Decoder

The decoder generates the output one token at a time, using two sources of information: what it's already written, and what the encoder understood about the input. Each decoder layer has three components.

1. Masked self-attention. Like the encoder's self-attention, except each word can only look at itself and the words before it — never ones that haven't been generated yet. If you're translating "Birds build nests" and the decoder has produced "Los pájaros" so far, it can look back at those two words, but it has no way to peek ahead at "construyen" or "nidos" before they've been written. This restriction is called causal masking.

2. Cross-attention. The bridge between the two halves. The Query comes from the decoder ("what am I trying to say next?"), while the Key and Value come from the encoder's finished output ("what did the input actually say?"). This keeps the decoder grounded in the source instead of drifting off-topic.

3. Feed-forward network. Same refinement step as in the encoder.

Training and Generation Aren't Quite the Same Process

Everything above describes generation, but what happens when the model is actually producing a translation, one new token at a time, with no access to future tokens because they don't exist yet.

Training works a little differently. During training, the correct target sentence is already known in full. So instead of looping one token at a time, the model is shown the entire shifted target sequence at once — and the causal mask (the same "no peeking ahead" restriction from masked self-attention) simply blocks each position from using information from positions after it. That means every target position can be trained in parallel, even though generation, later, will still happen one token at a time. The mask is what keeps that training-time parallelism from letting the model cheat by looking at the answer.

So: training is parallel, with masking to keep it honest. Inference — actually generating new text — is the genuinely sequential, one-token-at-a-time loop you see when a model streams its answer out.

From Output Vector to Predicted Word

After the final decoder layer, the output is still just a vector of numbers. A final linear layer converts it into a raw score for every word in the vocabulary, and softmax turns those into probabilities that add up to 1.

Say the decoder has generated "Los pájaros construyen" and needs the next word:

"nidos":    0.79
"casas":    0.08
"árboles":  0.02
...(50,000 total, almost all with tiny probabilities)

Picking the highest probability at each step like this is called greedy decoding and it's the easiest version to reason about.

Real translation systems more often use beam search instead, keeping several promising partial translations alive at once and picking the best complete sequence at the end, rather than committing irreversibly to one word at a time (the original transformer paper itself used beam search). Sampling methods like temperature, top-k, and top-p show up more in open-ended generation — chatbots, creative writing, where variety matters more than finding the single best sequence.

Putting It All Together

Let's trace the whole thing, translating "Birds build nests" into Spanish.

Training happens first. The model has already trained on millions of English–Spanish sentence pairs, so by the time it's asked to translate anything, it's recalling what it already knows, not learning Spanish on the fly.

The encoder phase:

1. Tokenize: "Birds build nests" → Birds, build, nests
2. Embed: each token becomes a vector
3. Add position: Birds is 1st, build is 2nd, nests is 3rd
4. Pass through every encoder layer: self-attention connects the words; the FFN refines each one; residual connections and layer normalization keep everything numerically stable

The decoder phase, one word at a time:

• "Los": starting from a start token, masked self-attention has nothing to look back at yet. Cross-attention checks the encoder's output. The model predicts "Los."
• "pájaros": decoder now has [start, Los]. Same process. Predicts "pájaros."
• "construyen": decoder has [start, Los, pájaros]. Predicts "construyen."
• "nidos": decoder has [start, Los, pájaros, construyen]. Predicts "nidos."
• End token: the decoder signals the translation is complete.

Final output: "Los pájaros construyen nidos" — "Birds build nests," in Spanish.

Final Thoughts

A transformer layer, underneath the vocabulary, is a small set of operations, repeated and stacked: project vectors, compare them, blend information across positions, transform each position individually, keep the residual stream intact, normalize the result. None of it is especially mysterious on its own.

The model's capability comes from learning what goes into those matrices, and from repeating this enough times, across enough layers, for rich representations to emerge.

Our sentence used tiny, hand-picked vectors to keep the arithmetic readable, but the path it took is the same path a production model follows — just at a scale too large to compute by hand.

This is it, that's how the Transformer works from start to finish.

Thanks for Reading!

If this blog helped you learn something new, please consider:

• following me on X and LinkedIn
• check out my other blogs

Have a great day :)