In this assignment, students build several feedforward neural networks for face recognition using TensorFlow. Students train both shallow and deep networks to classify faces of famous actors. The assignment serves as an introduction to TensorFlow. Visualization of the weights to explain how the networks work is emphasized. Students modify and extend the handout code and write completely new code rather than ``fill in" missing pieces in the handout code. This allows students to experience the development of a machine learning system from scratch, using only code that is similar to what can be found on the internet.
Students train a one-hidden-layer neural network for face classification. Students then improve the performance of their system by build a convolutional network and using transfer learning. Students use the activations of AlexNet pretrained on ImageNet as the features for their face classifier.
Students visualize the weights of the neural networks they train. Visualizing the weights allows students to understand feedforward one-hidden-layer neural networks in terms of template matching, and allows students to explore overfitting. Students can demonstrate that, with enough hidden units, a one-hidden-layer neural network will ``memorize" faces. Students are asked to explore a visualization strategy of their choice for understanding the convolutional network they train (e.g., Guided Backpropagation).
TensorFlow is introduced by giving students working code that trains a one-hidden-layer neural network on the MNIST dataset. We also provide a beginner-friendly implementation of AlexNet in TensorFlow, along with weights pretrained on ImageNet.
Visualization of the parameters of a single-hidden-layer neural network that sometimes ``memorizes" faces because it contains a very large number of hidden units. Each image is a visualization of the incoming weights of one unit in the hidden layer (credit: Davi Frossard).
|Guided Backpropagation visualization of the pixels that were the most important for classifying the image above as Angie Harmon (credit: Davi Frossard).|
The assignment can be completed with a single CPU, but students should be encouraged to experiment with cloud services and GPUs.
Students build feedforward neural networks for face recognition using TensorFlow. Students then visualize the weights of the neural networks they train. The visualization allows students to understand feedforward one-hidden layer neural networks in terms of template matching, and allows students to explore overfitting. Using a framework such as TensorFlow allows for the students to be able to run a variety of experiments in order to obtain interesting visualizations. In the second part of the assignment, students use transfer learning and build a convolutional network to improve the performance of their face recognition system. For bonus marks, students visualize what the convolutional network is doing. Students work with a bare-bones and comprehensible implementation of AlexNet pretrained on ImageNet, and with a TensorFlow implementation of a neural network that classifies MNIST digits.
Feedforward neural networks, face recognition, weight visualization, overfitting, transfer learning, convolutional neural networks.
Third and fourth year students in Intro ML classes. Students need to have had practice with making medium-sized (several hundreds of lines of code) programs.
Students find this assignment quite difficult. Spending more than 10 hours on it is not unusual.
|Dependencies||Students should have a good understanding of feedforward neural networks, including a brief explanation of visualizing the weights of the neural network. Students should have had a brief introduction to ConvNets and transfer learning. Slides are available upong request.|
Handout: html (markdown source).