Preparing Data for Fine-Tuning

Data preparation is a pipeline, and each stage answers a question:

1. Task: what goes in, what comes out, and what does each label mean?
2. Quality: is the data what we think it is? Empty texts, markup, oddities?
3. Cleaning and deduplication: remove noise, and make sure each example appears once.
4. Formatting: shape each example the way the model expects it, whether text plus label for a classifier or a conversation for a chat model.
5. Splits: separate training, validation and test data, reproducibly.
6. Leakage: make sure no evaluation example, or anything derived from it, reaches training.
7. Balance: check how the labels are distributed, and choose metrics that suit it.
8. Length: fit each example into the model's token limit without losing what matters.
9. Packaging: tokenise and store the result in a form training code can stream.

The order matters. Several of the worst mistakes, such as splitting before deduplicating, come from doing the right steps in the wrong order.

Figure 8.2.1. Group duplicate examples before splitting; keep validation and test out of training updates. Fit learned preprocessing on train, then apply the declared policy to held-out data.

Before touching the data, be precise about the task: one IMDb review in, one label out, positive or negative. The labels are the model's expected output, so we need to know exactly what they mean. The first place to look is the dataset card (Lesson 7.2):

The card tells us the task (binary sentiment classification), the size (25,000 training and 25,000 test reviews) and one crucial detail: the reviews are highly polar. The dataset's authors kept strongly positive and strongly negative reviews, judged from the reviewers' own star ratings, and left out the lukewarm middle. So a model trained on IMDb has never seen a genuinely mixed review labelled as such, which helps explain the lukewarm-review behaviour we saw in Lesson 7.5.

The card is also a warning about documentation. Its licence is other, and 16 places in it say "More Information Needed", including languages and how the labels were made. Many dataset cards are incomplete; when one is, find the original source (here the authors' website and paper) before relying on the data.

Now the data itself:

Two columns: text and label, with label names neg (0) and pos (1). That is our expected output: an integer class, 0 or 1.

Key terms

• Expected output (target): what the model should produce for each input; here, a label 0 or 1.
• Labelling rule: how the labels were decided; for IMDb, from the reviewers' star ratings, keeping only clearly positive or negative reviews.
• Highly polar: strongly one-sided; IMDb excludes mixed or neutral reviews.

Data rarely arrives clean. A few cheap checks catch most surprises: lengths, empty entries, markup and stray content.

Over all 50000 reviews:

• Lengths vary enormously, from 4 words to 2470, with a median of 173. Length is the subject of Section 9.
• No review is empty.
• 58.4% contain HTML tags. Almost all are <br /> line breaks (201935 of them), plus a handful of italics tags. These are left over from the web pages the reviews were collected from, and they mean nothing to a sentiment model.
• 119 reviews contain a web address.

The shortest reviews are worth a look, because extremes often reveal problems:

They are short but genuine. "Read the book, forget the movie!" is a perfectly clear negative review. The third shows something else: "More suspenseful, more subtle, much, much more disturbing...." is labelled negative, yet on its own it reads as praise. It's probably comparing the film unfavourably with something else. Some labels can only be understood with context the text doesn't contain, so a perfect score is not a realistic target.

Common mistake: skipping a manual look at the data. Summary statistics can't tell you that the shortest "reviews" are real, or that a label needs context. Reading a few examples, chosen by a rule such as the shortest or the first ones, takes minutes and often changes your plan.

Cleaning removes content that carries no signal for the task. For IMDb that means the <br /> tags, which we replace with spaces (the italics tags are too rare to matter). We'll keep the web addresses: they're rare, and a reviewer linking to something is part of what they wrote.

Deduplication matters more. A review that appears twice is weighted twice in training, and if its copies end up on both sides of a split, the model is tested on something it has already seen. We look for exact copies, and for near-copies that differ only in capitalisation, punctuation or spacing:

Each official half contains exact duplicates: 96 extra rows in train and 199 in test. Normalising text adds only 3 more near-copies in each half (99 and 202), so IMDb's duplicates are mostly exact. Most importantly, 123 reviews appear in both official halves, so evaluating on the official test set without deduplication would test the model on 123 reviews it trained on.

Do duplicates at least agree on their labels?

Yes: no duplicated text carries conflicting labels, so there's no label noise to resolve here. If copies of a text disagreed, we'd have to decide which label is right, or drop them. The most repeated training review appears 3 times. It reads like a description of a TV show, a reminder that "movie reviews" scraped from the web aren't always about movies.

Key terms

• Cleaning: removing content that carries no signal for the task, such as leftover HTML markup.
• Exact duplicate: an example identical to another, character for character.
• Near-duplicate: an example that differs from another only trivially (here: case, punctuation, spacing).
• Normalisation: transforming text into a standard form so that trivially different texts compare as equal.

Visual walkthrough: Keep duplicate groups in one split.

1. Find identical examples. A: “good film” · B: “bad film” · C: “good film”. This exact-match toy policy groups A and C; real preprocessing and near-duplicate policies must be declared.

2. Group before splitting. Group 1: {A, C} · Group 2: {B}. Splitting rows first could place the same text in train and test and inflate evaluation.

3. Assign whole groups. Train: {A, C} · Test: {B}. This two-group example explains the boundary; it is far too small for a useful model evaluation.

4. Fit only on train. Learn preprocessing from train → apply to test. Test data never supplies gradients or fitted vocabulary/statistics under this protocol. Validation is a separate group-held-out split.

The app adds a paused animation of this same supplied example.

The same data can be shaped for different kinds of model. For a classifier, as in Lessons 8.3–8.5, an example is just the text and an integer label. For an instruction-tuned chat model, as in Lesson 8.6, the same example becomes a short conversation: an instruction containing the review, and the answer written out as words, wrapped in the model's chat template (Lesson 8.1):

The classification example is {'text': ..., 'label': 0}. The instruction example is the same review and label, rewritten as a user turn ("Is this movie review positive or negative?") and an assistant turn ("negative"), inside the special markers SmolLM2 was trained with. Only the tokeniser was downloaded for this, a few megabytes.

Instruction formatting turns any labelled dataset into training data for a chat model. The choices you make (the wording of the instruction, the exact answer words, the system message) become part of what the model learns, so keep them consistent across every example. Lesson 8.6 shows how such examples are trained on, including the important detail of which tokens the loss is computed on.

Key terms

• Instruction formatting: rewriting a labelled example as an instruction and a response for a chat model.
• User / assistant turn: the parts of a conversation written by the user and by the model.

Module 4 introduced the three splits. Training data fits the weights, validation data chooses settings and checkpoints, and test data is used once at the end to estimate real-world performance. Here is the function that builds the split used throughout Modules 7 and 8, with the steps in the right order: deduplicate each half, drop test reviews that also appear in train, clean, then carve validation out of the training half with a fixed seed.

The result has 22,404 training reviews, 2,500 validation reviews and 24,678 test reviews, each close to half positive. val_eval and test_eval are the fixed 1,000-review samples used for most evaluations. The fingerprints are the split's identity card: any notebook that prints the same fingerprints is evaluating on exactly the same reviews, so results can be compared fairly.

Why fix the seed? Change it, and you get a different split:

With seed 1, the validation set's fingerprint changes, and only 226 of its 2,500 reviews were in seed 0's validation set, about what chance predicts. Results measured on the two splits are not directly comparable. Fixing the seed, and printing a fingerprint to prove it, is what lets Lessons 8.3–8.7 compare their numbers.

Key terms

• Training / validation / test split: data for fitting weights / for choosing settings / for one final estimate.
• Fingerprint: a short hash of a split's contents that proves which examples it contains.
• Reproducible split: a split that comes out the same every time, by fixing the random seed.

Leakage is any way information from evaluation data reaches training. It makes scores optimistic, sometimes wildly so. The most common cause is doing steps in the wrong order. Here's what would have happened if we had split IMDb's training half before removing duplicates:

Splitting first leaves 16 validation reviews with an identical copy in the training part. The model would be "validated" partly on reviews it had memorised. Deduplicating first, as our function does, leaves none, and no test review appears in training either.

Leakage takes other forms too:

• Leakage through preprocessing. Anything fitted on data, such as a vocabulary, a scaler or a choice of settings, must be fitted on training data only. Choosing a setting by its test score is leakage too.
• Leakage through groups. Several reviews by the same person, or about the same film, can share phrasing. If one goes to training and another to test, the test is easier than real use. When groups matter, split by group.
• Contamination of a pretrained model. In Lesson 7.7 we found a model fine-tuned on IMDb's training half, which contains our validation set, and that had chosen its checkpoint on IMDb's test half. Its validation score measured memory. Pretraining can contaminate too: a model pretrained on a large web crawl may have read IMDb reviews, or even the benchmark itself. DistilBERT was pretrained on books and Wikipedia (Lesson 7.3), which makes this unlikely for our model, but for models trained on web crawls you often can't rule it out. The safest defence is evaluation data the model can't have seen, such as data collected after its training.

Common mistake: splitting first and cleaning later. Deduplication, filtering and any fitted preprocessing must happen before the split (for deduplication) or on the training split only (for anything learned from data). Otherwise evaluation data leaks into training.

IMDb is exactly balanced (the official halves each have 12,500 reviews of each class), and our splits stay close to half positive. Real data often isn't balanced: complaints might be 5% of support tickets, fraud far less of transactions. Let's see what imbalance does to evaluation.

We need a real classifier's predictions, so we load the SST-2 model from Module 7 and run it on 500 validation reviews. The encode function fits each review into 128 tokens in one of three ways, which Section 9 explains and compares. For now we use the version labelled head+tail.

Now we build an imbalanced evaluation set by a fixed rule: the first 225 negative and the first 25 positive reviews from those 500, so 90% negative. We compare the SST-2 model with a "model" that always says negative:

The do-nothing classifier wins on accuracy, 0.900 against 0.848, because it is right about every negative review, and negatives are 90% of the set. It's also useless: its recall on positive reviews is 0.000 and its F1 score for the positive class is 0.000. The real model finds 0.800 of the positive reviews and 0.853 of the negative ones. On imbalanced data, accuracy rewards ignoring the rare class, so report per-class recall, the F1 score for the class you care about, and the confusion matrix (Lesson 7.5), and always compare with the majority-class baseline.

Imbalance in training data causes a related problem: a model can learn to favour the common class. Common remedies are to resample (show the rare class more often), to weight the loss towards the rare class, or to move the decision threshold, each chosen on validation. IMDb needs none of these, but your data might.

Key terms

• Class imbalance: some labels being much more common than others.
• Majority-class baseline: always predicting the most common label; any useful model must beat it on the metrics that matter.
• Recall: the share of a class's real examples that the model identifies.
• F1 score: a single score combining precision (how many predicted positives are right) and recall; useful when one class is rare.

Our model reads at most 512 tokens, and in Module 8 we'll train with only 128 to keep training affordable on a CPU. How much of a typical review fits?

The median validation review is 214 tokens, and one in ten is longer than 608. At 128 tokens, 85.6% of reviews don't fit; even at 512, 13.9% don't. So which 128 tokens we keep is a real decision. There are three simple strategies:

• head: keep the beginning, the usual default (Lesson 7.4);
• tail: keep the end;
• head+tail: keep the first quarter of the budget and the last three quarters, skipping the middle.

Here's what each keeps of one 215-token review:

Head keeps the opening and stops mid-review. Tail and head+tail both keep the final verdict ("it just has to be seen."). Head+tail also keeps the opening. Which matters more is a question for measurement. Here are all three strategies with the SST-2 model at 128 tokens, on the same 500 validation reviews, with sign tests on paired predictions:

Keeping the end of the review matters a great deal. Head scores 0.842; tail 0.888 and head+tail 0.880. Both beat head-only clearly (sign tests p = 0.014 and 0.042). Reviewers often summarise their verdict at the end, and the head-only cut throws it away. Between tail and head+tail, though, there's no meaningful difference: 19 reviews favour tail, 15 favour head+tail, p = 0.61.

We'll use head+tail for the rest of Module 8. It is statistically not significantly different in this paired comparison from tail on validation, and it keeps the opening as well, which helps reviews that state their verdict first. Either would be a defensible choice. Head-only would not.

Key terms

• Token budget: the maximum number of tokens an example may use; here 128.
• Head, tail and head+tail truncation: keeping the beginning, the end, or both ends of a text that exceeds the budget.

Common mistake: accepting the default truncation without checking it. Keeping only the beginning is the default almost everywhere, and for IMDb at 128 tokens it cost about four points of accuracy against keeping the end. Measure the options on validation.

Finally, we package everything for training. We take the 1,000 training reviews that Lessons 8.3–8.5 train on, the 1,000 val_eval reviews and the 1,000 test_eval reviews, apply head+tail truncation to 128 tokens, and store token IDs and labels in a DatasetDict (Lesson 7.2). We don't pad here: a data collator pads each batch to its own longest member when training (Lesson 7.4).

Three splits of 1,000 examples, each with input_ids and labels, and no example longer than 128 tokens. A data collator turns any 8 examples into a padded batch, adding the attention mask:

The batch has the shapes training code expects: input_ids and attention_mask of shape (8, 128), and 8 labels. Saved with save_to_disk, the whole prepared dataset takes 1.5 MB, in Arrow format. Lessons 8.3–8.5 rebuild exactly the same data with the same functions, so that each notebook stands alone; the fingerprints confirm they match.

Key terms

• DatasetDict: a dictionary of named Dataset splits (Lesson 7.2).
• Data collator: a function that combines examples into a padded batch.
• save_to_disk: saves a dataset in Arrow format for fast reloading.

Let's reassemble the pipeline, with what each step found on IMDb.

We defined the task (one review in, a 0/1 label out) and learned from the dataset card that IMDb uses polar rating labels (negative ratings at most 4/10, positive ratings at least 7/10); the prose can still contain mixed opinions, though the card itself is incomplete: 16 sections say "More Information Needed". Quality checks on 50000 reviews found no empty texts but 58.4% containing leftover HTML, mostly <br /> line breaks, which we cleaned. Deduplication found 96 duplicate rows in train, 199 in test and 123 reviews in both halves, with only 3 more near-copies per half after normalising, and no conflicting labels. The same review can be formatted for a classifier or, through a chat template, for an instruction model.

The split deduplicates first, then carves validation with a fixed seed, and fingerprints prove which reviews each notebook uses. A different seed gives a different split, sharing only 226 of 2,500 validation reviews. Doing the steps in the wrong order would have leaked 16 validation reviews into training. On an artificially imbalanced set, a do-nothing classifier scored 0.900 accuracy against the real model's 0.848, while finding none of the positive reviews: use recall and F1. Finally, length management mattered most of all: at 128 tokens, keeping the end of each review raised accuracy from 0.842 (head) to 0.880 (head+tail) or 0.888 (tail), and we chose head+tail. The prepared dataset is saved for Lessons 8.3–8.5.

Figure 8.2.2. Counts and token-budget comparisons from the supplied IMDb audit. Exact deduplication defines this course split; the separate normalised audit is not silently substituted for it.