Why Deep Learning?

You arrive at this course already able to do a great deal. You can load a messy dataset, clean it, explore it, encode categoricals, split it properly, fit a model, tune it, and evaluate it honestly. That is not a small skill set. Most of what follows depends on it.

So it is worth being precise about what is actually new here, because the honest answer is narrower than the marketing suggests.

In everything you have built so far, you decided what the model would look at. You chose the columns. You created the ratios, the log transforms, the interaction terms, the aggregations over time windows. The learning algorithm optimised coefficients or split points, but the description of each example — the thing the algorithm actually consumed — came from your judgement.

Deep learning moves that description inside the model. The network is handed something closer to the raw signal and learns, as part of the same optimisation that fits the final prediction, what the useful intermediate descriptions are.

That single shift is what this entire course is built on, and it is what makes foundation models possible. Here is the road ahead:

You are at the far left. By Module 10 you will be at the far right, and each arrow will be something you have actually run.

Rather than argue about pipelines in the abstract, let us construct a problem where the difference becomes visible in a single plot.

The problem is deliberately tiny: two features, two classes, one thousand points. There is no missing data, no leakage, no class imbalance, no categorical encoding to worry about. Every complication you normally spend your time on has been removed. Only the shape of the decision boundary remains.

That is the point. When a model fails here, there is nowhere to hide.

Look at it for a moment before moving on. A human sees the structure instantly. The two arms are obvious, the rule separating them is obvious, and you could label a new point by eye without hesitation.

Hold onto that, because the model is about to fail at something you can do effortlessly.

A logistic regression draws exactly one straight line through your feature space. Every point on one side receives one label, every point on the other side receives the other. That constraint is not a weakness in the implementation, not a bug, and not something a better optimiser would fix. It is the definition of the model.

Now look again at what we are asking it to separate. Walk outward along either spiral arm and you cross from one class to the other, repeatedly. No single straight line survives that journey. You have been handed a straight ruler and asked to trace a coil.

Before you run the next cell, commit to a number. What test accuracy do you expect?

Around 0.59. Barely better than a coin flip, and the picture shows exactly why: one flat blue region, one flat orange region, a single straight seam between them.

Here is the part that matters most, and it is easy to skim past.

Nothing is broken. The data loaded correctly. The labels are perfect. There is no missing data, no leakage, no imbalance. The optimiser converged. There is no bug in the code. The model did precisely what it was designed to do, and the task was simply outside its reach.

That distinction — between a model that failed and a model class that cannot express the answer — is one of the most useful diagnostic instincts you will build in this course. in Module 3 you will need it constantly.

You are not stuck. You have a whole professional toolkit for exactly this situation, and you have used it many times: if the model cannot express the boundary in the current feature space, change the feature space.

Go back to how you described the rule in Section 2. You almost certainly reached for distance from the centre and angle around the centre. That instinct is correct, and we can write it down.

Each point has a radius \(r = \sqrt{x_1^2 + x_2^2}\) and an angle \(\theta = \arctan2(x_2, x_1)\). Now here is the piece of domain knowledge that unlocks the problem. We built the spirals so that a point's angle grows in lockstep with its radius — that is what makes it a spiral rather than a circle. So if we subtract off the rotation that the radius accounts for, what remains should depend only on which arm the point belongs to.

Define an aligned angle:

For one arm \(\alpha\) lands near \(0\), for the other near \(\pi\). Feed the model \(\sin\alpha\) and \(\cos\alpha\) and the two classes pull apart cleanly. We use sine and cosine rather than \(\alpha\) itself because angles wrap around, and a raw wrapped angle would put nearby points at opposite ends of the number line.

This is unrolling the spiral into a straight strip. Same points, same labels, different description.

From 0.59 to 0.99. The model never changed — it is the same logistic regression, still drawing a single straight line. What changed is the space it draws that line in. In the aligned-angle space, one straight cut is all the problem ever needed.

This is worth stating plainly because it is the hinge of the whole argument:

The limitation was never in the classifier. It was in the representation.

You did.

And that is fine — for this problem. You could see the data in two dimensions, you knew the generating process, and the geometry was simple enough to reason about over a coffee. Under those conditions, hand-engineering is not merely acceptable, it is often the superior engineering decision: it is fast, cheap, interpretable, and it encodes real knowledge that no amount of data can supply.

So ask the uncomfortable question. What are the actual preconditions for what you just did?

• You could visualise the data. Two dimensions. Now imagine 4,096.
• You knew the generating process. You wrote it. Real data rarely comes with its equations attached.
• The right transformation was compact — one line of trigonometry. Not every structure compresses that well.
• The features were stable. Change the centre, the number of revolutions, or the noise level and polar_features degrades.

Strip away any one of those and the approach starts to strain. Strip away all four — which is the normal condition for images, audio, and text — and it collapses entirely. Section 10 shows you exactly what that collapse looks like.

This is the gap deep learning was built to fill. Not "features are bad." Rather: when you cannot invent the features, you need a method that discovers them from the data, as part of solving the task.

Now we hand a neural network the raw x1 and x2 columns. No radius. No angle. No trigonometry. Nothing of what you worked out in Section 4.

We use scikit-learn's MLPClassifier here deliberately, because the point of this section is the result, not the mechanism. Lesson 1.2 opens up the mechanism by hand, and Lesson 1.3 rebuilds it in PyTorch. For now, treat it as a black box that we are about to justify opening.

The boundary curves. It wraps around the arms, follows the spiral outward, and does so without anyone telling it that spirals have a radius and an angle.

Something specific happened inside that model, and naming it correctly matters more than being impressed by it. The network's first layer transformed (x1, x2) into 64 new numbers. The second layer transformed those into 64 more. Only the final layer performed the classification — and it is, in essence, a logistic regression, exactly like the one that failed in Section 3.

So the last layer is the same model that could not solve this problem. What changed is that it now sits on top of a representation the network built for itself. The earlier layers are the feature engineering. They were fitted, not written.

If this lesson stopped at Section 6, you would leave with a conclusion that is both tidy and wrong: neural networks learn their own features, therefore neural networks are better.

They are not better. They are different, and the difference pays off only under specific conditions. The fastest way to see this is to run the comparison on the kind of data most working data scientists actually face.

We use the Telco Customer Churn dataset — 7,043 customers, 20 predictors, a binary churn label. It is tabular, mixed categorical and numeric, moderately imbalanced. If you have worked in industry, you have met a dozen datasets shaped exactly like this.

Both models get the same split, the same preprocessing, and a fair shot.

Gradient boosting wins on quality and costs roughly a tenth of the training time. It also needed no feature scaling, exposes feature importances for free, and has far fewer knobs to get wrong.

This is not a fluke of one dataset or one seed. On medium-sized tabular data with meaningful engineered columns, gradient-boosted trees remain a genuinely strong default in 2026 — a fact that survives every wave of deep learning enthusiasm, and one that experienced practitioners take for granted.

Why? The spiral had geometric structure in raw coordinates that a network could exploit. The churn table does not. Its columns are already meaningful, human-curated quantities — tenure, monthly charge, contract type. Someone already did the representation learning, by hand, when they designed the database schema. There is very little left for a network to discover.

That gives you a usable heuristic, and it is worth committing to memory:

Deep learning pays off where the raw signal is high-dimensional, redundant, and structured in ways humans cannot easily write down — pixels, waveforms, text. On well-curated tabular data, it frequently does not.

Three experiments, three results. Let us extract the principle carefully, because loose versions of it are everywhere and most of them are wrong.

Here is the principle:

Representation learning means the transformation from raw input to useful description is learned from data, jointly with the task, rather than specified in advance by a person.

Three things this does not say, each corresponding to a claim you will encounter and should push back on:

• It does not say feature engineering is obsolete. Section 4 outperformed Section 6.
• It does not say networks always win. Section 7 says the opposite, on the data type most companies hold most of.
• It does not say the work vanishes. It moves — into architecture choice, data preparation, and training dynamics. Modules 2 through 4 are about that relocated work.

What it does say is that a door opened. Once representations are learnable, you can attack problems where nobody knows how to write the features down — and you can reuse a representation learned on one task for a different task. That second consequence is transfer learning, it is the basis of every pretrained model you will touch from Module 7 onward, and it is why this idea matters far beyond the accuracy numbers above.

A tempting summary of Section 6 is that the network needed no preparation. That is false, and believing it will cost you real time in Module 3.

Watch what happens when we remove one preprocessing step from the churn network — the standardisation that we quietly applied.

One column spans single digits, another spans thousands. Gradient descent takes steps that are the same size in every direction, so a feature measured in thousands dominates the update and a feature measured in tens is effectively ignored for a long time. Module 3 makes this precise. For now, note the empirical fact: the "no feature engineering needed" model lost real performance from a missing three-line transformation.

So where does the human work actually live in a deep learning project in 2026?

• Input scaling and normalisation — never optional, as you just measured.
• Data quality and labelling — the single largest lever on final performance, and Module 8 is devoted to it.
• Architecture choice — a convolution encodes "nearby pixels are related"; attention encodes "any token may relate to any other". These are engineered inductive biases, chosen by you.
• Tokenisation for text, sample rate for audio, image resizing and augmentation for vision — all human decisions made before the model sees anything.
• Loss design — what you optimise defines what you get.

The work did not disappear. It changed shape, and it moved up a level of abstraction.

The spirals had two features. You could plot them, reason about them, and invent a transformation over lunch. Now we scale the input up, and the argument stops being a matter of taste.

These are 8×8 handwritten digits — 64 pixels each, the smallest realistic image data that exists. Modern images are hundreds of times larger.

Now try to do what you did in Section 4. Write a feature that separates 3 from 8.

Not a model — a feature. An explicit arithmetic function of those 64 pixel values that is reliably larger for one digit than the other. Give yourself two honest minutes before running the next cell.

The distributions sit almost on top of each other. And this was the easy pair, on 8×8 images, distinguishing only two classes.

Now consider what you are actually up against. Distinguishing all ten digits needs features robust to stroke thickness, slant, position, size and writing style. A 224×224 colour photograph has 150,528 numbers. The pixel at position 9,412 means nothing on its own — meaning lives in relationships between pixels, across many scales at once, and those relationships shift when the object moves two pixels to the left.

No one hand-writes that. People genuinely tried, for decades, with real ingenuity — SIFT, HOG, wavelet banks, Gabor filters. Those were serious engineering achievements, and learned representations comprehensively outperformed them.

You now have the argument in full, so it is worth being explicit about what the next nine modules will and will not do.

This course is the engineering layer between the classical machine learning you already know and the generative AI work that follows. The question it answers is: how does an AI engineer take a neural network or a pretrained model from raw data to a working, evaluated, deployable system?

This course is not a mathematics course. You will trace backpropagation by hand in Lesson 1.2 because doing so makes training failures debuggable, not because derivations are the goal. There will be no proofs.

It is not a history course. Convolutional networks and recurrent networks receive proportionate conceptual treatment in Module 5 — enough to understand why they mattered and what replaced them.

It is not an LLM applications course. Retrieval, agents, and tool use belong to the Generative AI module that follows. Here we build the foundation those systems stand on.

It is not a model zoo tour. Fewer models, understood properly, beats many models demonstrated superficially.

And it is deliberately hands-on. Somewhere between half and two-thirds of your time will be spent writing, running, breaking and debugging code — which is where the understanding actually forms.

One habit to start today, since Module 10 will ask for it as a deliverable: keep a short log of every experiment. What did I try, why, what happened, what did I learn, what would I change? Four lines per experiment. This single practice separates engineers who improve steadily from those who keep rediscovering the same result.