Pixy2 has added the ability to detect and track lines. Line-following is a popular robotics demo/application because it is relatively simple to implement and gives a robot simple navigation abilities. Most line-following robots use discrete photosensors to distinguish between the line and the background. This method can be effective, but it tends to work best with only thick lines, and the sensing is localized making it difficult for the robot to predict the direction of the line or deal with intersections.

Pixy2 and pan-tilt on top of Zumo base, looking down and following lines, sporting wig, little baseball cap

Pixy2 attempts to solve the more general problem of line-following by using its image (array) sensor. When driving a car, your eyes take in lots of information about the road, the direction of the road (is there a sharp curve coming up?) and if there is an intersection ahead. This information is important! Similarly, each of Pixy2's camera frames takes in information about the line being followed, its direction, other lines, and any intersections that these lines may form. Pixy2's algorithms take care of the rest. Pixy2 can also read simple barcodes, which can inform your robot what it should do – turn left, turn right, slow down, etc. Pixy2 does all of this at 60 frames-per-second.

Pixy2's algorithms can detect dark lines on a light background or vice-versa. You can also tell Pixy2 that you are only interested in lines within a certain range of widths. Pixy2 goes through each frame finding lines that meet your criteria and determining where each line begins and ends within the frame.

Normally, a robot is only interested in the line that it is currently following. If new lines appear in the frame, you typically don't want your robot to switch and start following those lines. Pixy2 tracks each line, determining where each line moves in subsequent frames, so the line your robot is following remains so unless you tell it otherwise. The line your robot is currently following is called the Vector. Pixy tells you where the Vector starts and ends in each frame, so you can use that information to determine your robot's motion. So, for example, if the Vector heads toward the right in the frame, your robot should start turning right. If the Vector heads toward the left, your robot should start turning left, etc. We have example programs that run on an Arduino that show how to do this.

Intersections can be challenging for a line-following robot because they can take so many different shapes and forms. Will your robot deal well with T-shaped intersections, but fail when it sees Y-shaped intersections? What about 4 or even 5-way intersections? What if the intersections and lines are hand-drawn or “curvy”? Pixy2 can handle these cases. For example, Pixy2 tells your program that it has detected an intersection ahead, how many branches are in the intersection and the angles of each branch. Your program can then decide which branch it wants to take and then communicate it to Pixy2. Pixy2 will then make the branch its new Vector line, and your robot will in essence “take the branch” by following the new Vector's direction.

Note, to get PixyMon to label intersections like this, be sure to set Delayed turn in the the Expert configuration tab, otherwise Pixy2 will choose the turn/branch immediately and the intersection will no longer be relevant and essentially disappear in PixyMon.

In addition to detecting lines and intersections, Pixy2 can detect small barcodes. The barcodes can be used to tell your program what to do at the next intersection, or provide a hint, such as “slow down”. Pixy2 can detect up to 16 different codes and report to your program when it detects each one. Your program can choose how it reacts to each barcode.

Pixy2 tries to send only the most relevant information to your program:

• It finds the best Vector candidate and begins tracking it from frame to frame. The Vector is often the only feature Pixy2 provides to your program, and it's the only feature your program typically needs when following a line.
• It detects intersections and reports them after they have met the filtering constraint.
• It detects barcodes and reports them after they have met the filtering constraint.
• It reports each new barcode and intersection one time so your program doesn't need to keep track of which features it has/hasn't seen previously.
• It “executes” branches by assigning the Vector to the branch line that your program chooses.

But you may want to use Pixy2's line tracking algorithms in a different way that requires that Pixy2 share more of the information that it is gathering. For example, you may want your program (not Pixy) to choose the Vector line. If so, Pixy2 can provide your program with all lines, intersections and barcodes that it detects, regardless of filtering constraints or the state of the Vector. Pixy2 will still track each line, intersection and barcode from frame to frame and provide your program with this information, but your program can choose to use the information or ignore it. In other words, your program will be unconstrained regarding the information it receives from Pixy2.

Pixy2 has an integrated light source for when the lighting needs a boost. The light source is particularly useful when your robot is performing line-following. It gives Pixy2 plenty of light regardless of the level of ambient light and it reduces motion blur, which can be an issue when your robot is going fast and the ground is zipping by underneath it. The integrated light source emits around 20 lumens, about twice the amount an iPhone in flashlight mode emits.

The line tracking API can be found here.

Information about running and configuring the line tracking program in PixyMon can be found here.

This guide will help you get familiarized with the line tracking algorithm and get it up and running on your Arduino.