The Fine-Tuning Workflow

Fine-tuning a classifier is a loop around five ideas:

1. Model: a pretrained body plus a new, randomly initialised classification head.
2. Batch: token IDs, an attention mask and labels for a handful of examples.
3. Loss: how wrong the model's predictions are on that batch (cross-entropy).
4. Update: gradients of the loss for every weight, then a small step against them, with a carefully scheduled learning rate.
5. Selection: regularly check validation accuracy, and keep the best version (checkpoint) of the model.

Then test once, and save.

Figure 8.4.1. Only training batches drive gradients and optimizer updates. Validation selects checkpoints; a declared final test reports the selected setup.

The same fixed split and the same 1,000 training reviews as Lesson 8.3, checked by fingerprint, with head+tail truncation to 128 tokens (Lesson 8.2). We keep each review as a list of token IDs; a data collator pads every batch to its longest member and adds the attention mask.

AutoModelForSequenceClassification loads DistilBERT's pretrained body and adds a classification head (Lesson 7.1, Section 4). We also give it readable label names, which are saved with the model. The LOAD REPORT that appears is expected. The UNEXPECTED weights are the base model's word-guessing head, which a classifier doesn't use. The MISSING weights are our new head, which starts random.

The head is two layers: pre_classifier, 768 → 768, and classifier, 768 → 2, one score per label. All 66,955,010 parameters are trainable. That's what makes this full fine-tuning: every pretrained weight, not just the head, will be updated.

Key terms

• Classification head: the new layers mapping the body's output to one score per class; here pre_classifier and classifier.
• num_labels, id2label, label2id: configuration settings for the number of classes and their names, saved with the model.

Here is one batch of 4 training reviews, as the model receives it:

Each review becomes 128 token IDs, starting with 101 ([CLS]). The attention mask marks real tokens with 1 and padding with 0. The batch is as long as its longest review; any shorter review would be padded with 0s. The labels are plain integers, 1 for positive and 0 for negative. When the batch includes labels, the model computes the loss itself.

The objective is the familiar cross-entropy loss (Module 2): for each review, take the probability the model gives to the correct class, and take minus its logarithm. Averaging over the batch gives the loss. Let's compute it by hand and compare it with the model's own:

The new head is random, so its scores (logits) for the two classes are tiny and almost equal: the model has no opinion yet. Our hand-computed loss, 0.6874, matches the model's exactly, and both are close to ln 2 = 0.6931. That's the cross-entropy of a model that gives each of two classes a probability of one half. Checking the starting loss against this value is a quick way to catch bugs before training (Lesson 6.5 did the same for a language model).

One training step is exactly the Module 2 loop: forward pass, loss, backward() to compute gradients, then an optimiser step. Two details matter at this scale. We look at the gradient sizes in each part of the model, and we clip the overall gradient size at 1 (Module 3, Lesson 6.4), so one unusual batch can't produce a huge update.

The gradients are largest at the head (1.424) and shrink steadily towards the input, from 0.451 in the last Transformer layer to 0.134 at the embeddings. The new, random head has the most to learn; the pretrained layers near the input are already useful and get gentler signals. The overall gradient norm, 1.639, is above 1, so clipping scaled it down for this step. After one step, the loss on the same 4 reviews falls from 0.6874 to 0.6379. That's the whole loop in miniature, repeated 63 times below.

Key terms

• Gradient norm: the overall size of the gradients, as a single number.
• Gradient clipping: scaling the gradients down when their norm exceeds a limit (here 1.0), to prevent unstable updates.
• AdamW: the Adam optimiser with decoupled weight decay; the standard choice for fine-tuning Transformers.

Visual walkthrough: Separate learning from selection.

1. Training forward pass. Training batch → logits → loss. The labels of this training batch define the optimization objective.

2. Backward and update. loss.backward() → optimizer.step(). Trainable parameters change; clear gradients as required before the next accumulation cycle.

3. Validation checkpoint. No gradients · score saved checkpoint. Validation selects settings and checkpoints. It does not directly update weights in this training loop.

4. Final test reporting. Selected setup → held-out test. Repeatedly tuning after inspecting test results requires a fresh final test for an unbiased report.

The app adds a paused animation of this same supplied example.

When you trained small networks from scratch in Modules 2 and 3, learning rates around 1e-3 were typical. Fine-tuning a pretrained Transformer usually uses something like 1e-5 to 5e-5, twenty or more times smaller. The reason is that the weights are already good: we want to nudge them, not overwrite them. Here's how far a single AdamW step moves one of DistilBERT's feed-forward weight matrices at each learning rate:

At 5e-5, one step changes the matrix by 0.120% of its size; at 1e-3, by 2.402%, twenty times more. Over 63 steps, a large rate would move the pretrained weights far from where pretraining left them, destroying much of what made them valuable (and making forgetting, Lesson 8.1, much worse). We use 5e-5.

• Batch size 16: large enough for stable gradients, small enough to be fast on a CPU.
• One epoch: one pass over the 1,000 reviews, 63 steps. With a pretrained model, one to three epochs is typical; Lesson 8.1 showed a small dataset overfitting by the third.
• Warmup, then linear decay: the learning rate starts near zero, rises to its peak over the first 10% of steps, then falls steadily to zero. Warmup protects the pretrained body in the first steps, while the random head is still producing large, meaningless gradients.

With 63 steps, warmup lasts 7 steps, the peak learning rate is 5e-05, and the rate decays to zero by the end.

Key terms

• Learning rate: the step size of each update; fine-tuning uses small values (here 5e-5).
• Warmup: gradually raising the learning rate from near zero at the start of training.
• Linear decay: lowering the learning rate steadily to zero by the end of training.
• Learning-rate schedule: the plan for how the learning rate changes over training.

The model after the last step isn't necessarily the best one. Training can overshoot or start to overfit, and on small data the validation score wobbles. So we evaluate during training and keep the best checkpoint: a copy of the weights at that moment.

To keep evaluation cheap during training, we check every 16 steps on the first 200 validation reviews, and keep the checkpoint with the best accuracy. That is a choice made on validation, never on test. After training we restore it and measure it on all 1,000 validation reviews. The helper below returns P(positive) for a list of reviews.

Here is the complete loop: AdamW, the warmup-then-decay schedule, gradient clipping, and a checkpoint check every 16 steps. We start from a freshly loaded model, with the same seed for the new head.

At step 16, the model still knows nothing useful: validation accuracy is 0.475, below chance. Then it learns fast: 0.840 at step 32 and 0.865 at step 48, the best checkpoint, before slipping to 0.845 at the last step. The plot shows the two views:

The training loss jumps around from step to step, because each batch has only 16 reviews, but its moving average falls steadily: the printed 16-step averages go from 0.671 to 0.425. Validation accuracy rises quickly and then levels off, and the last checkpoint is slightly worse than the one at step 48. That's why we select rather than simply taking the end. We restore the best checkpoint and measure it on the full validation set:

The chosen checkpoint scores 0.860 on the 1,000 validation reviews.

Hugging Face's Trainer runs this same loop for you. You describe the recipe in TrainingArguments (batch size, epochs, learning rate, warmup, schedule, clipping), and Trainer handles batching, the optimiser, the schedule, logging, devices and, if asked, saving and evaluation. Two details for this library version:

• warmup_steps accepts a fraction, so 0.1 means 10% of all steps.
• use_cpu must be set explicitly on a CPU-only run. On a Mac, Trainer would otherwise pick the Apple GPU by itself (Lesson 7.4 met the same behaviour in pipelines).

The Trainer ran on the CPU, as intended. Its logged training loss follows the same path as our loop's (0.674, 0.557, 0.396 at steps 16, 32 and 48), and its final model scores 0.870 on validation, against 0.860 for our chosen checkpoint. The two models agree on 90.4% of validation reviews. They aren't identical, because the batches are shuffled differently and dropout draws different random numbers, but the recipe gives the same quality. A difference of 0.010 on 1,000 reviews is ten reviews in these runs. Repeated runs and a paired uncertainty analysis are needed before claiming the recipes have equivalent quality.

So which should you use? Write the loop yourself when you're learning or need unusual control (custom losses, special sampling). Use Trainer for standard jobs: it is less code and fewer chances for bugs, and it brings logging, checkpointing, mixed precision and multi-GPU training for free.

Key terms

• Checkpoint selection: keeping the weights from the training moment with the best validation score.
• Trainer / TrainingArguments: Hugging Face's ready-made training loop and its settings.

Common mistake: letting Trainer (or a pipeline) choose the device on its own. On a Mac it may silently train on the Apple GPU, and on a machine with a GPU it will use it even if you intended a CPU comparison. Set the device explicitly when it matters.

We save the chosen checkpoint and its tokeniser together (Lesson 7.4), reload them, and check that nothing changed:

The folder holds the configuration, the weights (268.5 MB in total) and the tokeniser. The reloaded model gives identical probabilities, and the label names travelled with it: {0: 'NEGATIVE', 1: 'POSITIVE'}. We then delete the copy, since every task fine-tuned this way needs its own full-size copy, a cost Lesson 8.5 avoids.

All choices are made, so we use the test set once:

The fine-tuned model scores 0.834 on the 1,000 test reviews, with recall 0.8742 on negative reviews and 0.7961 on positive ones.

How does that compare with the baselines? Lesson 8.3 saved each baseline's prediction for every test review, so we can compare on exactly the same reviews with paired sign tests:

This is the honest result of the workflow.

• Fine-tuning clearly learned the task: it beats the majority-class floor overwhelmingly.
• It is statistically not significantly different in this paired comparison from the two cheaper 1,000-label baselines: TF-IDF (105 against 99, p = 0.73) and the frozen encoder with a linear head (78 against 81, p = 0.87).
• It is significantly worse than the off-the-shelf SST-2 model (50 against 89, p = 0.0012), and than TF-IDF with all labels.

With 1,000 labels, one epoch and 128 tokens, updating all 66,955,010 weights bought nothing over training only a linear head on frozen features (Lesson 8.3), and less than a model someone else had already adapted on a large sentiment dataset. That's not a failure of the workflow, which ran correctly, but a measured answer to "is fine-tuning worth it here?" Lesson 8.5 asks the same question of a much cheaper method, and Lesson 8.7 makes the decision.

Finally, Lesson 8.1's forgetting measure: the word-guessing loss of the original masked-language head on 200 held-out reviews, with the pretrained body and with our fine-tuned body.

The loss rises from 3.381 to 4.07. Full fine-tuning on 1,000 reviews has measurably degraded the pretrained skill, even with a small learning rate and a single epoch.

Everything is saved for Lesson 8.7: settings, costs (including a memory estimate of weights, gradients and two Adam states in float32, which Lesson 8.5 explains), validation and test results, test predictions, the forgetting measure, the loss curve and the checkpoint scores.

Interpret the evidence carefully. The retention comparison uses the same tied-output setup explained in 8.1. A loss increase here is evidence of reduced performance on this specific probe, not a comprehensive measure of catastrophic forgetting.

Common mistake: reporting the best of several checkpoints by their test scores, or tuning until the test number looks good. Choose everything on validation, test once, and report what you get, even when, as here, it's not the result you hoped for.

Let's reassemble the workflow.

We loaded DistilBERT with a new classification head (the LOAD REPORT listed the new head as MISSING and the unused word-guessing head as UNEXPECTED), making all 66,955,010 parameters trainable. A batch carried 128-token head+tail inputs, an attention mask and labels. The loss at the start, 0.6874 by hand and by the model, matched ln 2, the sign of a correct, untrained setup. One step showed gradients largest at the new head (1.424) and smallest at the embeddings (0.134), with clipping limiting the overall size. A small learning rate matters: one step at 1e-3 moved a weight matrix twenty times further than at 5e-5. Over 63 steps with warmup and linear decay, validation accuracy on 200 reviews rose from 0.475 to 0.865 at step 48, the checkpoint we kept. It scored 0.860 on all 1,000 validation reviews. The Trainer ran the same recipe in a few lines and reached 0.870, agreeing with our model on 90.4% of reviews. The saved model reloaded with identical outputs.

The single test run gave 0.834, not significantly different from TF-IDF and frozen features on the same 1,000 labels, and significantly below the off-the-shelf SST-2 model (p = 0.0012). And full fine-tuning cost the model some of its pretrained skill: word-guessing loss 3.381 → 4.07.

Figure 8.4.2. Source CPU reference run: the custom-loop checkpoint was chosen on 200 validation reviews and then reported on 1,000 test reviews. The small experiment is not an optimised comparison of all training recipes.