Augmented Reality

Students learn augmented reality coding concepts such as motion sensing, color calibration and gesture detection to build interactive experiences.

  • GRADES 5-7

Answer Key

Module 4: Window Wiper

Module 6: Doodle Tool

Module 8: Art Studio

Module 9: Quiz









Lesson 4: Tracking Motion

Course: | iPad Web

  • Introduction
  • Concepts
  • Window Wiper Intro
  • Window Wiper
  • Doodle Tool Intro
  • Doodle Tool
  • Art Studio Intro
  • Art Studio
  • Quiz

Lesson Plan

Lesson: Tracking Motion

Time: 40+ mins


In this lesson, students will learn how to track brightly colored real-world objects as they move! Before starting the modules, gather several brightly colored objects (e.g., pieces of paper, Snap Cubes, pencil cap erasers) for your students to use as calibration objects.

New Code Blocks

  • : Detects whatever color is the strongest behind the Actor.
  • : Search the video and find the location of the specified color, where the tolerance value sets how precise the Actor should follow the color being tracked.
  • : Move the Actor to the location of the color being tracked. Note that you need to use the “find color on video with tolerance” block at least once before the “found color position x/y” blocks will work correctly.
  • : Detect the strongest color behind the Actor and save that color in the “Color” variable.
  • : Move the Actor to the specified x- and y-coordinates.
  • : Set the pen’s color to the specified color.
  • : Set the pen’s thickness to the specified value.
  • : Put the pen down to draw an image on the Stage.
  • : Clear all markings made by the pen.
  • : React when the reset Actor is touched.
  • : Adjust the value of the specified variable.


  • Variable: A value or piece of information that is stored, and can change over time
  • Track: To follow or find an object
  • Color tracking: Detect and follow an object, based on color
  • Tynker calibration: A two-step process of first detecting, then saving the object’s color in a variable
  • Tolerance value: Sets how precise the Actor should follow the color being tracked, where a small tolerance is more precise


Students will...
  • Use color tracking to follow an object
  • Use color tracking and pen blocks to turn an Actor into a virtual pen
  • Create an art studio with buttons that can change the pen’s color


  • Computers or iPads (1 per student) with a working camera and student account access to
  • Brightly colored objects (e.g., pieces of bright colored paper, Snap Cubes, pencil cap erasers)

Warm-Up (5 minutes)

1. Tell students to take out a piece of paper and a pencil.
2. Tell students to answer the short-response question:
  • If you created an “augmented reality” art game or project, what would it be? What are some objects you might have the player draw?

Getting Started (5 minutes)

1. Use your projector to display “Module 2: Concepts.”
2. Play each concept (Get Color, Find Color, Found X and Y) and read the captions to the class.

Activities (30 minutes)

Facilitate as students complete all Tracking Motion modules on their own:
1. Introduction (Video)
  • This short video introduces the Tracking Motion lesson.
2. Concepts (Video)
  • This video introduces three coding concepts: “Get Color,” “Find Color,” and “Found X and Y.”
  • As students calibrate a color, make sure they hold their bright colored object to the grey circle for 5 seconds.
3. Window Wiper Intro (Video)
  • This short video introduces the Window Wiper puzzle module.
4. Window Wiper (Puzzle)
  • In this puzzle module, students will need to program a squeegee to follow an object to help clean the window. But there’s a catch! Students need to clean the dirt off the window in less than 45 seconds.
  • When the program starts, tell students to place their bright colored object on the squeegee for 5 seconds.
  • How to play: Once students finish calibrating and programming their squeegee, tell them to control the squeegee with their bright colored object and move it around the screen to clean the entire window.
5. Doodle Tool Intro (Video)
  • This short video introduces the Doodle Tool DIY (do it yourself) activity.
6. Doodle Tool (DIY)
  • In this DIY, students will use color tracking and pen blocks to turn an Actor into a virtual pen to draw pictures.
  • Tell students this DIY activity combines color tracking with Tynker’s pen tools.
  • Once the program starts, tell students to place their bright colored object on the grey circle for 5 seconds.
  • Encourage students to draw silly pictures on their face (e.g., new hairdo, glasses, cat ears).
7. Art Studio Intro (Video)
  • This short video introduces the Art Studio DIY activity.
8. Art Studio (DIY)
  • In this DIY, students expand on doddle concepts learned in the previous DIY activity to program an art studio!
  • In “Step 6” of the tutorial, check that students are dragging the blocks of code to the appropriate Actor (i.e., drag the “when touching blue occurs - set pen color” blocks to the blue Actor)
  • How to play: Tell students to draw with an object as they did in the “Doddle Tool DIY,” but this time, move their pen over the colored buttons to change the drawing color or clear the screen! Click the Stage to turn the pen on or off.
9. Quiz (Multiple-Choice)
  • Students will be tested on Tracking Motion concepts.

Optional Activities (20 minutes each)

Artists Needed!
1. Hand students a piece of paper and a pencil.
2. Tell them to draw a simple image (e.g., sun, smiley face, house, flower).
3. Collect the drawings and randomly hand them out to your students.
4. Tell students to open “Module 8: Art Studio” and digitally create the simple image they were handed.
5. Remind students to click the Stage to turn the pen on or off.

Review Questions
Discuss the following with students:
  • How can you track the location of a moving object in a video? (Track its color)
  • What is calibration? (Detecting and saving an object's color) What blocks do you use for the two "steps" of calibration? (Get Color and Set Variable)
  • How do you get the location of a color in a video? (use the “go to” and “found color position” code blocks)


CCSS-Math: MP.1
CCSS-ELA: SL.3.1, SL.3.3, W.3.1.C, W.3.2, SL.4.1, SL.4.1.C, W.4.2, SL.5.1, SL.5.1.C, SL.5.1.D, W.5.2, SL.6.1, SL.6.1.C, W.6.2, SL.7.1, SL.7.1.C, SL.7.1.D, W.7.2, SL.8.1, SL.8.1.C, SL.8.1.D, W.8.2
CSTA: 1A-AP-09, 1A-AP-10, 1A-AP-11, 1A-AP-14