A Gentle Introduction to the Backpropagation Algorithm and Feedforward Networks

A Gentle Introduction to the Backpropagation Algorithm and Feedforward Networks - Java Edition

Michael Wollowski and
Oscar Youngquist
Rose-Hulman Institute of Technology

Summary This assignment is designed for students to learn about the algorithmic underpinnings of feedforward neural networks and the backpropagation algorithm. The given software closely follows the perceptron and backpropagation algorithms as explained in the 3rd edition of the Russell and Norvig AIMA textbook. As part of this assignment, students are asked to implement the training component of a given of a perceptron network and to modify the training component of a feedforward network. In order to develop an understanding of the effect that the different tuning parameters have on the performance of the backpropagation algorithm students are asked to experiment with various tuning parameters so as to implement the AND and XOR Boolean functions. Students are given an implementation of a feedforward network and are asked to implement a digit recognition network, using the MNIST data set.

Topics artificial neural networks, perceptron networks, feedforward networks, backpropagation, application of backpropagation to digit recognition

Audience The intended audience are undergraduate students in an introductory course to artificial intelligence, a course that includes an introduction to neural networks.

Difficulty The implementation of the training component of the perceptron network, the AND experiments and the accompanying write-up of the experiments take about an hour. The XOR experiments take about an hour, including the write-up of the experiments. Developing the code for the curation of the MNIST training code, running the experiments and documenting them takes about 5 hours.

Strengths We provide Java starter code here. To obtain solution files, please contact Michael Wollowski. Java should be familiar to many students. The code is richly commented.

Weaknesses Much of the current neural networks is written in Python. We are currently working on a python version of this assignment.

Dependencies A basic understanding of perceptrons and feedforward networks, such as covered in sections 18.7.2 and 18.7.4 of the Russell and Norvig. Familiarity with Java.

Variants This assignment will make a good prerequisite to the following MAIA assignments related to neural networks: Neural Networks for Face Recognition with TensorFlow as well as A Project on Gesture Recognition with Neural Networks for "Introduction to Artificial Intelligence" Classes

Link to assignment

Summary	This assignment is designed for students to learn about the algorithmic underpinnings of feedforward neural networks and the backpropagation algorithm. The given software closely follows the perceptron and backpropagation algorithms as explained in the 3rd edition of the Russell and Norvig AIMA textbook. As part of this assignment, students are asked to implement the training component of a given of a perceptron network and to modify the training component of a feedforward network. In order to develop an understanding of the effect that the different tuning parameters have on the performance of the backpropagation algorithm students are asked to experiment with various tuning parameters so as to implement the AND and XOR Boolean functions. Students are given an implementation of a feedforward network and are asked to implement a digit recognition network, using the MNIST data set.
Topics	artificial neural networks, perceptron networks, feedforward networks, backpropagation, application of backpropagation to digit recognition
Audience	The intended audience are undergraduate students in an introductory course to artificial intelligence, a course that includes an introduction to neural networks.
Difficulty	The implementation of the training component of the perceptron network, the AND experiments and the accompanying write-up of the experiments take about an hour. The XOR experiments take about an hour, including the write-up of the experiments. Developing the code for the curation of the MNIST training code, running the experiments and documenting them takes about 5 hours.
Strengths	We provide Java starter code here. To obtain solution files, please contact Michael Wollowski. Java should be familiar to many students. The code is richly commented.
Weaknesses	Much of the current neural networks is written in Python. We are currently working on a python version of this assignment.
Dependencies	A basic understanding of perceptrons and feedforward networks, such as covered in sections 18.7.2 and 18.7.4 of the Russell and Norvig. Familiarity with Java.
Variants	This assignment will make a good prerequisite to the following MAIA assignments related to neural networks: Neural Networks for Face Recognition with TensorFlow as well as A Project on Gesture Recognition with Neural Networks for "Introduction to Artificial Intelligence" Classes