Deep Learning Engineering
How do I run deep learning experiments professionally?
You will learn
Hands-on: A gradient-boosting vs neural-network bake-off — choose on constraints, not hype.
Lessons
-
Lesson 4.1: Data You Can Trust
How do you know your validation score is telling the truth? This lesson checks the trust you have placed in every split since Module 2. You audit Bank Marketing for problems isna() cannot see (sentinels, placeholders, near-constant columns), write a split...
Not started Start -
Lesson 4.2: Experiments You Can Reproduce
If you report a number today, can anybody get it again tomorrow? You separate three meanings of reproducible (repeatable, portable, robust), seed every generator a run touches and give the DataLoader its own, drive every run from one settings dictionary...
Not started Start -
Lesson 4.3: Memory, Batch Size and Precision
Will this run fit in the memory you have? You count bytes: every number costs 4 bytes in fp32, gradients add one more copy of the model, and AdamW adds two, so training with AdamW needs about four times the model's own memory before a single activation....
Not started Start -
Lesson 4.4: The Bake-Off: Gradient Boosting against a Neural Network
Which of two models would you ship, and could you defend the choice to someone who disagrees? You write the rules before the first result, then run the Module 3 network, frozen, against gradient boosting on identical rows across ten splits. Each model on...
Not started Start
Module challenge
Module 4 challenge: experiments you can defend
8 scenario questions that mix ideas from all 4 lessons. One question at a time, an explanation after each, and your score at the end. Pass mark 75%, so 6 of 8.
Best after you have practised all 4 lessons.
A hospital readmission model is evaluated on a random row split; several rows belong to each patient. The validation AUC is excellent. What is the concern and the fix?
Rows from one group are more alike than random rows, so the model can recognise the patient (Lesson 4.1 §9).
Rows are stored in date order and the success rate rose tenfold over three years. The bank will use the model on next month's customers. Which split answers that question?
The random split's 0.78 became 0.61 on the chronological split (Lesson 4.1 §7). Test the deployment condition.
Between the training period and the test period, the yes-rate went from 6.7% to 47.1%, and the customers phoned changed too. Which statement is right?
Lesson 4.1 §8. Adversarial validation (AUC 0.94) confirmed the periods look very different.
You set seed_everything(42) once at the top of a notebook, then rerun a training cell. The result differs from the first run. Why?
Lesson 4.2 §2.
Seed-to-seed spread of test AUC is 0.001; split-to-split spread is 0.013. A change improves test AUC by 0.004 on one split. What can you conclude?
The unpaired noise (0.013) swamps 0.004, but a paired comparison cancels the shared split difficulty and can detect it (Lessons 4.2 §8, 4.4 §2).
Fine-tuning a 7B model in fp32 with AdamW, before activations. Roughly how much memory, and what would change the picture most?
Weights + gradients + two AdamW moments = 4 copies of 28 GB (Lesson 4.3 §1–2).
Training in fp16 without a loss scaler, many small weights stop changing and nothing warns you. What is happening?
Lesson 4.3 §3; try it in the precision lab.
Model B beats model A on 10 of 10 paired splits by +0.008 AUC, but A is 2× faster at prediction and one file smaller, and under the chronological split A wins by 0.03. What should the decision memo say?
Lesson 4.4 §10: a decision rule, not a winner. Accuracy is one column; robustness under shift can matter more.