From Machine Learning to Deep Learning
What changes when models learn representations from raw data?
You will learn
Hands-on: A first PyTorch network, trained until you can watch the loss fall.
Lessons
-
Lesson 1.1: Where Machine Learning Runs Out of Road
Deep learning is not "a stronger model". The one real change is that the description of each example is learned from data instead of written by you. This lesson shows the idea with three experiments: a straight-line model that cannot solve a spiral, a...
Not started Start -
Lesson 1.2: The Learning Mechanism, Traced By Hand
You become the optimiser. With a network of just nine numbers and four training examples, you compute the forward pass, the loss, every gradient and one parameter update, all by hand. Then PyTorch reproduces your arithmetic exactly. After this lesson,...
Not started Start -
Lesson 1.3: Your First PyTorch Network
Everything you did by hand in Lesson 1.2, now in idiomatic PyTorch: tensors (shape, dtype, device), a model as an nn.Module, a numerically stable loss on logits, an optimizer, and the six-line training loop. You train it on the spirals, read its loss...
Not started Start
Module challenge
Module 1 challenge: from features to a trained network
8 scenario questions that mix ideas from all 3 lessons. One question at a time, an explanation after each, and your score at the end. Pass mark 75%, so 6 of 8.
Best after you have practised all 3 lessons.
A colleague's logistic regression scores 0.60 on a problem whose classes form concentric rings. They propose collecting five times more data. What do you say?
Lesson 1.1: the limit is the model class's expressiveness, not the data. A radius feature makes the rings linearly separable; a network can learn such a feature itself.
You have 12,000 rows of well-curated tabular data (income, tenure, contract type…) and need a defensible model by Friday. What is the strong first choice, and why?
Lesson 1.1 §7: on curated tables boosting is typically as good or better, faster, and easier to explain. Deep learning pays off on raw, high-dimensional signals.
A network has layers [100, 50, 50, 1] and no activation functions. How many parameters does it have, and how expressive is it?
(100·50+50) + (50·50+50) + (50·1+1) = 5,050 + 2,550 + 51 = 7,651. Without activations the layers compose into one linear map (Lesson 1.2 §4).
For one example, a ReLU neuron's pre-activation is −0.4. During backpropagation, what does that neuron pass back for this example?
The ReLU mask cuts the path: no activity, no responsibility, no update from this example (Lesson 1.2 §8).
Your binary classifier's training loop starts at loss 3.9 before any training step. The labels are 30% positive. What is your first suspicion?
Lesson 1.3 §6's sanity check costs nothing. Common causes: a sigmoid before BCEWithLogitsLoss, float labels in the wrong shape, or a label-encoding error.
Training loss falls smoothly to a plateau; validation accuracy sits at 0.61 on the spirals, where the reference model reaches 0.95. Nothing crashes. Which silent bug fits best?
A smooth, healthy-looking curve that plateaus at logistic-regression accuracy is the signature of collapsed linear layers (Lesson 1.3 §13). A huge learning rate thrashes; missing zero_grad is erratic.
Which order of lines inside one training step is correct?
Clear old gradients, forward, loss, backward to fill .grad, then update (Lesson 1.3 §8).
Your loss curve oscillates wildly from the first epoch and ends higher than it started. Gradients were checked and are correct. What do you change first?
Every step overshoots the valley and lands further up the other side: the too-large learning-rate signature (Lesson 1.2 §11).