Learning Representations from Images and Text
How do neural networks learn useful representations from different types of data?
You will learn
Hands-on: Train a Fashion-MNIST CNN, learn text embeddings, and compare raw inputs with learned features.
Lessons
-
Lesson 5.1: Understanding Representation Learning
What changes when a network learns its own features instead of being given them?
Not started Start -
Lesson 5.2: How Neural Networks Learn from Images
How can a network take advantage of the fact that, in an image, position and neighbourhood matter?
Not started Start -
Lesson 5.3: How Neural Networks Learn from Text
How can a network turn words, which are just symbols, into numbers that capture meaning?
Not started Start -
Lesson 5.4: From Sequences to Attention
How can a network use the order of a sequence, and why did that lead to attention?
Not started Start -
Lesson 5.5: Connecting Image and Text Representations
How can images and text be represented in a way that lets us compare them directly?
Not started Start -
Lesson 5.6: Synthesis: Putting Representation Learning into Practice
How can we measure, rather than just eyeball, what a learned representation has gained over raw data?
Not started Start
Module challenge
Module 5 challenge: Learning Representations from Images and Text
8 scenario questions that mix ideas from all 6 lessons. One question at a time, an explanation after each, and your score at the end. Pass mark 75%, so 6 of 8.
Best after you have practised all 6 lessons.
What does a representation describe?
An example in features useful for a task. Reread Lesson 5.1, section 1.
What is shared when a convolution filter moves?
Its weights are reused at different spatial positions. Reread Lesson 5.2, section 1.
Which step makes token vectors learn useful task information?
Training the embedding parameters through an objective. Reread Lesson 5.3, section 1.
Why do averaged word embeddings fail on reversed word order?
Averaging the same vectors gives the same result regardless of order. Reread Lesson 5.4, section 1.
Do independently trained image and text vectors align merely because their widths match?
No; equal width permits arithmetic but does not establish alignment. Reread Lesson 5.5, section 1.
Why use the same downstream probe family across representations?
It helps compare how accessible task information is in each representation. Reread Lesson 5.6, section 1.
Why split a network into an encoder and a head?
The encoder builds features and the head maps them to task outputs. Reread Lesson 5.1, section 2.
What is the PyTorch shape for 16 RGB images of size 224 by 224?
(16, 3, 224, 224). Reread Lesson 5.2, section 2.