Dataset and DataLoader: Feeding the Model

Module 1 trained the network using the whole training dataset at once. This is called full-batch gradient descent.

Suppose we have 36,168 training examples.

With full-batch training:

36,168 examples
      ↓
    model
      ↓
    loss
      ↓
gradient
      ↓
update parameters once

One complete pass through the training data produces one parameter update.

With mini-batch training, we divide the training data into smaller groups. For example, with batch_size=256:

36,168 examples
      ↓
  ┌───────┐
  │ batch │ 256 examples → update
  ├───────┤
  │ batch │ 256 examples → update
  ├───────┤
  │ batch │ 256 examples → update
  ├───────┤
  │  ...  │
  └───────┘

Now the model makes many parameter updates during one pass through the dataset.

A common explanation says:

"We use mini-batches because the whole dataset might not fit in GPU memory."

That is true for large datasets, especially images, but it is not the main reason in this experiment. This Bank Marketing dataset is small enough to fit comfortably in memory.

So this section asks a more useful question:

What does mini-batching give us when memory is not the problem?

The answer is mainly more parameter updates per epoch.

The next cells measure that difference.

We first train a small neural network using all training examples in one batch.

The important sequence inside the loop is:

1. clear old gradients
2. run the model on the data
3. calculate the loss
4. calculate gradients with backpropagation
5. update the parameters

Because the entire dataset is used at once, this sequence happens once per epoch.

Now we use batch_size=256.

The DataLoader divides the training set into batches. For every batch, we perform the same five-step training sequence:

batch → model → loss → gradients → parameter update

Because there are many batches in one epoch, the model updates its parameters many times per epoch.

First, an important definition:

An epoch = one complete pass through the training dataset.

With full-batch training, the entire dataset is processed once and then the model is updated once.

So:

1 epoch → 1 update
20 epochs → 20 updates

With batch_size=256, the training set is divided into about 142 batches:

1 epoch → about 142 updates
20 epochs → about 2,840 updates

So both approaches have seen the same training examples after 20 epochs, but the model has been given many more opportunities to change its parameters when using mini-batches.

That is the key idea:

Mini-batching is not only a memory-saving technique. It also controls how often the model updates its parameters.

Memory still becomes very important for large datasets. An entire dataset of images can be far too large to place on a GPU at once. But for this small tabular dataset, memory is not the limiting factor.

There is another difference between full-batch and mini-batch training.

A gradient tells the optimizer which direction the model's parameters should move to reduce the loss.

With full-batch training, the gradient is calculated using every training example. It is therefore the exact average gradient for that dataset.

With mini-batch training, the gradient is calculated using only a small sample of the data. It is therefore an estimate of the full gradient.

For example:

Full batch:
all examples → exact average gradient

Mini-batch:
256 examples → approximate gradient

Because different mini-batches contain different examples, their gradients are slightly different. This variation is called gradient noise.

"Noise" does not necessarily mean "bad."

It means that the optimizer does not take exactly the same direction it would have taken using the entire dataset. That variation can sometimes help optimization explore the loss surface rather than following one perfectly deterministic path.

The next experiment measures how closely several mini-batch gradients point in the same direction as the exact full-batch gradient.

The order of operations in a data pipeline matters because it determines whether your validation and test results are trustworthy.

The rule is:

Split the data first. Then learn preprocessing rules using the training split only.

Suppose we want to standardise a feature.

Standardisation means:

new value = (old value - mean) / standard deviation

If we calculate the mean and standard deviation using all the data, the calculation includes the validation and test examples.

That means information from the held-out data has influenced the numbers used to prepare the training data.

The model has not directly seen the validation labels, but the preprocessing step has still used information from the validation data.

That is a form of data leakage.

The safe sequence is:

1. Split the data
        ↓
2. Calculate preprocessing values from TRAIN only
        ↓
3. Transform TRAIN using those values
        ↓
4. Transform VALIDATION using the SAME values
        ↓
5. Transform TEST using the SAME values

For example:

X_tr, X_va = split(X)

scaler.fit(X_tr)

X_tr = scaler.transform(X_tr)
X_va = scaler.transform(X_va)

Notice that we do not call scaler.fit() on X_va.

The validation set is supposed to behave like unseen data.

stratify=y_all tells train_test_split to preserve approximately the same class proportions in each split.

Our target has two classes:

yes → client subscribed
no  → client did not subscribe

If the complete dataset contains about 11.7% yes and 88.3% no, stratification aims to keep roughly that same proportion in the training, validation, and test sets.

This is useful because a random split without stratification can accidentally produce slightly different class balances.

Important distinction:

Stratification controls class proportions. It does not solve every possible problem with how a dataset should be split.

For example, if the same person appears many times, or if rows from the same patient belong together, we may need a group-based split instead. We will return to that idea later.

The difference between the correctly calculated scaling statistics and the leaked statistics is small here.

That is actually an important lesson.

Data leakage does not have to produce an obviously ridiculous result. It can make a validation score look only slightly better while still making the evaluation method incorrect.

A code review should care about whether the pipeline is logically correct, not only whether the mistake changes today's number by a lot.

The safe pattern is always:

X_tr, X_va = split(X)          # split first
scaler.fit(X_tr)               # learn preprocessing from train only
X_tr = scaler.transform(X_tr)  # transform train
X_va = scaler.transform(X_va)  # transform validation with same scaler

Do not fit a second scaler on validation or test data.

A neural network ultimately needs numbers.

For this lesson, we want to give the model a matrix shaped like:

(N, F)

where:

• N = number of examples;
• F = number of input features.

For example:

(36,168, 50)

means:

• 36,168 training examples;
• 50 numerical input features for each example.

The Bank Marketing dataset contains both numeric and categorical columns, so we cannot simply pass the original table directly to a neural network.

We need to decide how each type of column should become numbers.

Suppose the job column contains:

admin.
blue-collar
management
retired
student

A tempting solution is:

admin.       → 0
blue-collar  → 1
management   → 2
retired      → 3
student      → 4

The problem is that these numbers create a fake meaning.

The model sees 4 and 3 as numbers, so it can learn that 4 is larger than 3, and that the distance from 0 to 4 is four times the distance from 0 to 1.

But there is no natural numerical distance between job categories.

student is not "twice as much job" as management.

For nominal categories—categories that are names rather than ordered quantities—we therefore usually use one-hot encoding.

One-hot encoding creates one indicator column for each category:

job=admin.       job=blue-collar  job=management  ...
     1                  0               0
     0                  1               0
     0                  0               1

Each row has a 1 for its category and 0 for the others.

This gives the model category membership without inventing an ordering.

One-hot encoding is a good default for the relatively small categorical columns in this dataset.

The trade-off is that one categorical column can become several numerical columns. Nine categorical columns therefore expand the input matrix from the original table to roughly 44 indicator columns.

For a much larger number of categories, another approach is a learned embedding.

An embedding gives each category a small trainable vector instead of creating a separate 0/1 column for every category. Language models use the same general idea when they turn token IDs into vectors.

For this lesson, one-hot encoding is simpler and completely adequate.

One subtle exception is month. Months have a natural order and also form a cycle: December is next to January. Treating them as ordinary categories loses that structure. We keep one-hot encoding here for simplicity, but it is a useful feature-engineering idea to revisit later.

The goal is to turn the mixed table into:

a clean float32 matrix
        ↓
that the neural network can accept directly

That final check is useful because the validation data should not have been used to learn its own scaling statistics.

If the validation features have exactly zero mean and unit standard deviation because you fitted a scaler directly on the validation set, that is a warning sign.

The validation set should be transformed using the scaler learned from training data.

Before evaluating a classifier, always ask:

How common is the positive class?

Here, "positive" means the client subscribed to the term deposit.

If only 11.7% of examples are positive, then 88.3% are negative.

A model that simply predicts:

"no" for everyone

would already achieve 88.3% accuracy.

That does not mean the model has learned anything useful.

This is why class balance should be understood before interpreting accuracy.

Accuracy is misleading for this dataset because the negative class is much more common than the positive class.

Two metrics used here are more informative:

• ROC-AUC: measures how well the model ranks positive examples above negative examples. A useful way to think about it is: if we randomly choose one subscriber and one non-subscriber, how often does the model give the subscriber the higher score?
• Average precision (PR-AUC): focuses on precision and recall for the positive class. This is useful when the positive class is relatively rare.

A useful fact about average precision is that its baseline depends on the positive-class rate. Here, that baseline is around 0.117 rather than 0.5.

We will use these metrics later rather than treating accuracy as the headline result.

A PyTorch Dataset is simply an object that knows how to answer two questions:

len(dataset)

How many examples are available?

and:

dataset[i]

Give me example number i.

In a custom PyTorch dataset, these are normally implemented as:

def __len__(self):
    return number_of_examples

def __getitem__(self, i):
    return one_example

This is called a protocol: a small set of methods that an object agrees to provide so that other PyTorch components know how to use it.

The important point is that PyTorch does not care where the example comes from.

It could come from:

• a tensor already in RAM;
• a CSV file;
• an image on disk;
• an audio file;
• a database;
• an API;
• or another source.

As long as the dataset can answer "how many examples?" and "give me example i", it can be used as a PyTorch Dataset.

Xtr_enc is a NumPy array containing the encoded training features.

torch.from_numpy(...) creates a PyTorch tensor that uses the same underlying data when possible.

We do this because PyTorch models and training operations work with PyTorch tensors.

TensorDataset is PyTorch's ready-made dataset for tensors.

Here it receives:

Xtr_t → training features
ytr_t → training labels

For index i, it returns:

(Xtr_t[i], ytr_t[i])

So it already implements the __len__ and __getitem__ behaviour we need.

This separation explains the job of each object.

Dataset answers:

"What is example number i?"

DataLoader answers:

"Which examples should I request, and how should I group them into batches?"

For example:

Dataset
  ↓
example 0
example 1
example 2
...
example 36,167

DataLoader
  ↓
batch 1: examples 0–255
batch 2: examples 256–511
...

Because these responsibilities are separate, you can use the same Dataset with different batch sizes:

DataLoader(train_ds, batch_size=32)
DataLoader(train_ds, batch_size=256)
DataLoader(train_ds, batch_size=1024)

The dataset does not need to change.

TensorDataset is convenient when your data is already stored in tensors.

You write your own Dataset when you need the dataset to do something while producing an example.

Common reasons include:

• reading an image from disk;
• applying an augmentation;
• normalising an example;
• reading an audio file;
• returning several pieces of information;
• performing a transformation that you do not want to calculate for the entire dataset ahead of time.

The key idea is:

__getitem__ is where you define how one example is obtained and prepared.

Here we write the same idea ourselves.

The class stores the features and labels, then defines:

• __len__ → number of examples;
• __getitem__(i) → feature and label for example i.

Writing it ourselves gives us a place to add extra behaviour, such as noise or transformations.

So far, the examples have already been stored in RAM.

That is convenient, but real datasets can be much larger than available memory.

Imagine a dataset containing:

1,000,000 images

It may be wasteful or impossible to load every image into RAM before training.

Instead, we can store the files on disk and have the Dataset read an image only when that image is requested.

This is called lazy loading.

Dataset created
     ↓
store file paths
     ↓
training asks for example 153
     ↓
read image 153 from disk
     ↓
return image

The image does not have to remain in memory as part of the entire dataset.

We will demonstrate this using scikit-learn's small load_digits dataset. The examples are written as PNG files first so that we can practise the same pattern used by much larger image datasets.

The folder structure uses one directory per class, which is also the convention expected by torchvision.datasets.ImageFolder.

Everything so far had the data in RAM already, which makes Dataset look like a formality. The protocol earns its keep when the data does not fit — 100,000 images, a directory of audio files, a set of documents too large to hold at once.

To make this concrete rather than hypothetical, we will write scikit-learn's load_digits out as individual PNG files in a folder tree, then read them back one at a time. The dataset is small enough to run anywhere and the pattern is identical to what you would use for a million images.

The folder layout below — one directory per class — is the near-universal convention, and it is what torchvision.datasets.ImageFolder expects.

This custom dataset does not store all image pixels in one large tensor.

Instead, it stores the locations of the image files. When __getitem__(i) is called, it opens the corresponding file, converts it into a tensor, and returns the image and its label.

That is the basic pattern behind many real-world image datasets.