This assignment asks students to build a program that plays the game of Set. Set is a game of visual perception and reasoning: in it players look through a group of face-up cards that contain a variety of shapes. Each of those shapes has four attributes: color, texture, form, and number. The game's goal is to find a trio of cards among which these attributes have all identical or all distinct values. As an example, the figure below shows nine of the 81 Set cards:
Within this image the central card, the "east" card, and the "northeast" card form a set. We can verify this through the four attributes:
Broader goals The goals of this assignment are twofold: first, it stands alone as an introduction to image-processing. As we use it, however, it is more important as a motivating introduction to the OpenCV library, the largest and most ubiquitous software foundation for real-time vision processing. Many courses and research projects build atop that library. Yet for all of its strengths, OpenCV continues to be a challenge to newcomers. This assignment offers students a relatively smooth and task-directed path through which undergraduates can get Set with OpenCV before applying it in more advanced investigations.
This assignment provides an introduction to computer vision. In particular, it reinforces color segmentation, morphology and binary image processing, and it helps students get comfortable with real-time pixel processing and OpenCV's interface to it.
On the other hand, this assignment does not reinforce game-search.
In fact, one of the realizations that students often articulate is that humans and computers have very different difficulties with the game. Humans identify the cards effortlessly and struggle to find a set of three that meet the constraints. For a program (or, at least, the programmer) the difficult part is determining the shapes. Identifying the triples of cards that share or differ in all attributes turns out to be much more straightforward to specify computationally.
|Audience||The intended audience for this assignment is undergraduates with experience in C or C++. In particular, no AI background is needed beyond the self-contained lecture for introducing and explaining the assignment.|
|Difficulty||This assignment is considered challenging, but doable, in general. Its difficulty can be varied by providing less - or more - scaffolding for the OpenCV library of image processing routines provided as starter code. Students in our AI Robotics course are given two weeks to complete both this introductory assignment and to integrate vision into a physical system they are working on.|
|Strengths||Set is a game many students like. They often enjoy building a game-player whose fundamental capability (vision) is very different than most or all of the AI agents/game players they know. In addition, this assignment is designed as a lead-in to further work, perhaps open-ended, using image processing. It provides motivation for some of the learning curve required to use the ubiquitous, but still idiosyncratic, OpenCV library.|
|Weaknesses||As noted, this assignment is best used as a stepping stone for student proejcts involving real-time vision. If an instructor wanted to use it as a self-contained introduction to image processing, the messiness of OpenCV - whose researchware origins still show - should be abstracted away. This could be done using OpenCV's Python interface, for example. However, because we do use this assignment as a gateway to undergraduate-accessible real-time vision, we have not yet created the scaffolding needed for an easily-deployed, self-contained Python version of this project.|
This assignment's primary purpose lies in the variants that students can build on top of it. In the past, our students have built a wide variety of robotic vision projects atop their Set experience:
|Links to resources and materials||