Pixy2 Quick Links
Basics
Connecting Pixy2 to…
Pixy2 Modes / Programs
Software and Support
Sidebar
wiki:v2:pixy2_full_api
Table of Contents
Pixy2 General API
Pixy2's general API can be called through the base Pixy2 object, for example:
Pixy2 pixy; pixy.init(); // call init() member function
See also the Color Connected Components API, Line tracking API, and Video API for other functionality.
For a byte-level reference to the protocol, check out the new Pixy2 Serial Protocol - Packet Reference
Member functions
int8_t init(uint32_t arg [optional])
init() should be called before your program communicates with Pixy. init() returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t getVersion()
getVersion() queries and receives the firmware and hardware version of Pixy2., which is put in the version member variable. It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds. It is called automatically as part of init().
int8_t changeProg(const char *prog)
changeProg() instructs Pixy2 to switch programs. It takes a string argument that indicates the name of the program. The argument can be a partial string of the program name as long as it is unique with respect to the other program names. It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t setServos(uint16_t s0, uint16_t s1)
setServos() sets the servo positions of servos plugged into Pixy2's two RC servo connectors. The servo values range between PIXY_RCS_MIN_POS (0) and PIXY_RCS_MAX_POS (1000). It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t setCameraBrightness(uint8_t brightness)
setCameraBrightness() sets the relative exposure level of Pixy2's image sensor. Higher values result in a brighter (more exposed) image. It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t setLED(uint8_t r, uint8_t g, uint8_t b)
setLED() sets Pixy2's RGB LED value. The three arguments sets the brightness of the red, green and blue sections of the LED. It will override Pixy2's own setting of the RGB LED. It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t setLamp(uint8_t upper, uint8_t lower)
setLamp() turns on/off Pixy2's integrated light source. The upper argument controls the two white LEDs along the top edge of Pixy2's PCB. The lower argument sets the RGB LED, causing it to turn on all three color channels at full brightness, resulting in white light. Both arguments are binary, zero or non-zero. It returns an error value (<0) if it fails and 0 (PIXY_RESULT_OK) if it succeeds.
int8_t getResolution()
getResolution() gets the width and height of the frames used by the current program. After calling this function, the width and height can be found in the frameWidth and frameHeight member variables. Note, calling changeProg() will call getResolution() automatically and update frameWidth and frameHeight.
Pixy2 will automatically change programs if, for example, you call ccc.getBlocks() from the color connected components program followed by line.getMainFeatures() from the line tracking program. These “automatic program changes” will not update frameWidth and frameHeight member variables.
int8_t getFPS()
getFPS() gets Pixy2's framerate. The framerate can range between 2 and 62 frames per second depending on the amount of light in the environment and the min frames per second setting in the Camera configuration tab. This function can also serve as a simple indicator of the amount of light in the environment. That is, low framerates necessarily imply lower lighting levels.
Member variables
Version *version
This contains the Pixy2's version information as a result of getVersion(). Inside the Version struct:
void print()
Calling this prints the version information to the console.
uint16_t hardware
This variable contains the hardware version number.
uint8_t firmwareMajor
This variable contains the firmware major version number.
uint8_t firmwareMinor
This variable contains the firmware minor version number.
uint16_t firmwareBuild
This variable contains the firmware build version number.
char *firmwareType
This variable contains the a string description of the firmware type.
uint16_t frameWidth
This variable contains the width of the frame in pixels. This field is updated upon calling init() and upon calling changeProg() as each program may have a unique frame size.
uint16_t frameHeight
This variable contains the height of the frame in pixels. This field is updated upon calling init() and upon calling changeProg() as each program may have a unique frame size.
Pixy2CCC ccc
This is an instance of the Color Connected Components object associated with the color_connected_components program.
Pixy2Line line
This is an instance of the Line Tracking object associated with the line_tracking program.
Error codes
PIXY_RESULT_ERROR
General error result
PIXY_RESULT_BUSY
This is returned when Pixy has no new data. It is used in polling mode, as opposed to blocking mode.
PIXY_RESULT_CHECKSUM_ERROR
This is returned when the data packet received contains a checksum error.
PIXY_RESULT_TIMEOUT
This is returned when Pixy2 has taken too long to return its result.
PIXY_RESULT_BUTTON_OVERRIDE
This is returned when the user is interacting with the button (e.g. teaching Pixy a signature) and can't perform the requested action.
Color Connected Components API
This API can be called through the ccc member of the Pixy2 object, for example:
Pixy2 pixy; pixy.ccc.getBlocks();
See also the Pixy2 General API, Line tracking API, and Video API for other functionality.
For a byte-level reference to the protocol, check out the new Pixy2 Serial Protocol - Packet Reference
Firmware versions 3.0.11 and greater will automatically switch to the color_connected_components program when making requests through the color connected components API. Older firmware versions will return errors when attempting to call these functions while the color connected components program is not running.
A good program to run to familiarize yourself the CCC algorithm is the ccc_hello_world example found in the Arduino library.
Member functions
int8_t getBlocks(bool wait [optional], uint8_t sigmap [optional], uint8_t maxBlocks [optional])
getBlocks() gets all detected blocks in the most recent frame. The new data is then available in the blocks member variable. The returned blocks are sorted by area, with the largest blocks appearing first in the blocks array. It returns an error value (<0) if it fails and the number of detected blocks (>=0) if it succeeds.
Setting wait to false causes getBlocks() to return immediately if no new data is available (polling mode). Setting wait to true (default) causes getBlocks() to block (wait) until the next frame of block data is available. Note, there may be no block data if no objects have been detected.
sigmap is a bitmap of all 7 signatures from which you wish to receive block data. For example, if you are only interested in block data from signature 1, you would pass in a value of 1. If you are interested in block data from both signatures 1 and 2, you would pass in a value of 3. If you are interested in block data from signatures 1, 2, and 3, you would pass a value of 7, and so on. The most-significant-bit (128 or 0x80) is used for color-codes. A value of 255 (default) indicates that you are interested in all block data.
maxblocks indicates the maximum number of blocks you wish to receive. For example, passing in a value of 1 would return at most 1 block. A value of 255 (default) indicates that you are interested in all blocks.
Member variables
uint8_t numBlocks
The number of blocks contained in the blocks member variable.
Block blocks[]
This array contains all of the block data as a result of getBlocks(). The blocks in this array are sorted by area, with the largest blocks appearing first in the blocks array.
Inside the Block struct:
void print()
Calling this prints the block information to the console.
uint16_t m_signature
This variable contains the signature number or color-code number.
uint16_t m_x
This variable contains the x location of the center of the block. The value ranges between 0 and frameWidth (315).
uint16_t m_y
This variable contains the y location of the center of the block. The value ranges between 0 and frameHeight (207).
uint16_t m_width
This variable contains the width of the block. The value ranges between 0 and frameWidth (316).
uint16_t m_height
This variable contains the height of the block. The value ranges between 0 and frameHeight (208).
int16_t m_angle
This variable contains the angle of color-code in degrees. The value ranges between -180 and 180. If the block is a regular signature (not a color-code), the angle value will be 0.
uint8_t m_index
This variable contains the tracking index of the block. When Pixy2 detects a new block, it will add it to a table of blocks that it is currently tracking and assign it a tracking index. It will then attempt to find the block (and every block in the table) in the next frame by finding its best match. Each block index will be kept for that block until it either leaves Pixy2's field-of-view, or Pixy2 can no longer find the object in subsequent frames (because of occlusion, lack of lighting, etc.)
uint8_t m_age
This variable contains the number of frames a given block has been tracked. When the age reaches 255, it remains at 255.
Line tracking API
This API can be called through the line member of the Pixy2 object, for example:
Pixy2 pixy; pixy.line.getMainFeatures();
See also the Pixy2 General API, Color Connected Components API, and Video API for other functionality.
For a byte-level reference to the protocol, check out the new Pixy2 Serial Protocol - Packet Reference
Firmware versions 3.0.11 and greater will automatically switch to the line_tracking program when making requests through the line tracking API.
A good program to run to familiarize yourself the Line Tracking algorithm is the line_hello_world example found in the Arduino Pixy2 library. This quickstart guide will get you off to a good start.
Whoa, this looks complicated
The line-tracking algorithm is really easy to use, especially if you stick to getMainFeatures(), which does it's best to keep your program simple, and it does a pretty good job. If you haven't stepped through this quickstart guide, do it!
The getAllFeatures() function is for more advanced programs and applications. Steer clear of it if you're just getting started!
Member functions
int8_t getMainFeatures(uint8_t features [optional], bool wait [optional])
This function gets the latest features including the Vector, any intersection that connects to the Vector, and barcodes. The results are returned in the variables vectors, intersections, and barcodes, respectively.
The optional features argument is a bitwise-ORing of LINE_VECTOR (1), LINE_INTERSECTION (2), and LINE_BARCODE (4), depending on what features you are interested in. All features, if present, are returned by default.
Setting wait to false causes getMainFeatures() to return immediately if no new data is available (polling mode). Setting wait to true (default) causes getMainFeatures() to block (wait) until the next frame of line feature data is available, unless the current frame of features hasn't been returned before, in which case, it will return immediately with the current frame's features. Note, there may be no feature data if no features have been detected.
If successful, the result is a bitwise-ORing of LINE_VECTOR (1), LINE_INTERSECTION (2), and LINE_BARCODE (4), depending on the data returned. If unsuccessful it returns an error value (<0).
getMainFeatures() tries to send only the most relevant information. Some notes:
- The line tracking algorithm finds the best Vector candidate and begins tracking it from frame to frame 1). The Vector is often the only feature getMainFeatures() returns.
- Intersections are reported after they meet the filtering constraint. Only intersections that connect to the Vector are reported.
- Barcodes are reported after they meet the filtering constraint.
- Each new barcode and intersection is reported only one time so your program doesn't need to keep track of which features it has/hasn't seen previously.
int8_t getAllFeatures(uint8_t features [optional], bool wait [optional])
This function returns all lines, intersections and barcodes that the line tracking algorithm detects. The results are returned in the variables vectors, intersections, and barcodes, respectively.
The optional features argument is a bitwise-ORing of LINE_VECTOR (1), LINE_INTERSECTION (2), and LINE_BARCODE (4), depending on what features you are interested in. All detected features are returned by default.
Setting wait to false causes getMainFeatures() to return immediately if no new data is available (polling mode). Setting wait to true (default) causes getMainFeatures() to block (wait) until the next frame of line feature data is available. Note, there may be no feature data if no features have been detected.
If successful, the result is a bitwise-ORing of LINE_VECTOR (1), LINE_INTERSECTION (2), and LINE_BARCODE (4), depending on the data returned. If unsuccessful it returns an error value (<0).
int8_t setMode(uint8_t mode)
This function sets various modes in the line tracking algorithm. The mode argument consists of a bitwise-ORing of the following bits:
LINE_MODE_TURN_DELAYED
Normally, the line tracking algorithm will choose the straightest path (branch) when encountering an intersection. 2) Setting LINE_MODE_TURN_DELAYED will prevent the line tracking algorithm from choosing the path automatically. This is useful if your program doesn't know beforehand which path it wants to take at the next intersection.
LINE_MODE_MANUAL_SELECT_VECTOR
Normally, the line tracking algorithm will choose what it thinks is the best Vector line automatically. Setting LINE_MODE_MANUAL_SELECT_VECTOR will prevent the line tracking algorithm from choosing the Vector automatically. Instead, your program will need to set the Vector by calling setVector().
LINE_MODE_WHITE_LINE
Normally, the line tracking algorithm will try to find dark lines on a light background (black lines). Setting LINE_MODE_WHITE_LINE will instruct the line tracking algorithm to look for light lines on a dark background (white lines).
int8_t setNextTurn(int16_t angle)
This function tells the line tracking algorithm which path it should take at the next intersection.
Turn angles are specified in degrees, with 0 being straight ahead, left being 90 and right being -90 (for example), although any valid angle value can be used. Valid angles are between -180 and 180.
setNextTurn() will remember the turn angle you give it, and execute it at the next intersection. The line tracking algorithm will then go back to the default turn angle for subsequent intersections.
Upon encountering an intersection, the line tracking algorithm will find the path in the intersection that matches the turn angle most closely.
int8_t setDefaultTurn(int16_t angle)
This function tells the line tracking algorithm which path to choose by default upon encountering an intersection. The line tracking algorithm will find the path in the intersection that matches the default turn angle most closely. A call to setNextTurn() will supercede the default turn angle for the next intersection.
Turn angles are specified in degrees, with 0 being straight ahead, left being 90 and right being -90 (for example), although any valid angle value can be used. Valid angles are between -180 and 180.
int8_t setVector(uint8_t index)
If the LINE_MODE_MANUAL_SELECT_VECTOR mode bit is set, the line tracking algorithm will no longer choose the Vector automatically. Instead, setVector() will set the Vector by providing the index of the line.
int8_t reverseVector()
The Vector has a direction. It normally points up, from the bottom of the camera frame to the top of the camera frame for a forward-moving robot. Calling reverseVector() will invert the vector. This will typically cause your robot to back-up and change directions.
Member variables
uint8_t numVectors
The number of lines in the vectors variable.
Vector vectors[]
This array contains either all of the detected lines if getAllFeatures() is called or the Vector if getMainFeatures() is called.
Inside the Vector struct:
void print()
Calling this prints the vector information to the console.
uint8_t m_x0
This variable contains the x location of the tail of the Vector or line. The value ranges between 0 and frameWidth (79) 3)
uint8_t m_y0
This variable contains the y location of the tail of the Vector or line. The value ranges between 0 and frameWidth (52).
uint8_t m_x1
This variable contains the x location of the head (arrow end) of the Vector or line. The value ranges between 0 and frameWidth (79).
uint8_t m_y1
This variable contains the y location of the head (arrow end) of the Vector or line. The value ranges between 0 and frameWidth (52).
uint8_t m_index
This variable contains the tracking index of the Vector or line. When Pixy2 detects a new line, it will add it to a table of lines that it is currently tracking. It will then attempt to find the line (and every line in the table) in the next frame by finding its best match. Each line index will be kept for that line until the line either leaves Pixy2's field-of-view, or Pixy2 can no longer find the line in subsequent frames (because of occlusion, lack of lighting, etc.)
uint8_t m_flags
This variable contains various flags that might be useful.
LINE_FLAG_INTERSECTION_PRESENT: This flag is only available if getMainFeatures() is called and the Vector is provided. This flag indicates that an intersection was detected but may not have met the filtering constraint.
LINE_FLAG_INVALID: This flag is only available if getAllFeatures() is called. This flag indicates that the line has been detected but has not met the filtering constraint.
uint8_t numIntersections
The number of intersections in the intersections variable.
Intersection intersections[]
This array contains either all of the detected intersections if getAllFeatures() is called or a single intersection that connects to the Vector if present and if getMainFeatures() is called.
Inside the intersection struct:
void print()
Calling this prints the intersection information to the console.
uint8_t m_x
This variable contains the x location of the intersection. The value ranges between 0 and frameWidth (79).
uint8_t m_y
This variable contains the y location of the intersection. The value ranges between 0 and frameHeight (52).
uint8_t m_n
This variable contains the number of lines (branches) in the intersection and in the m_intLines array.
IntersectionLine m_intLines[]
This array contains the lines in the intersection.
Inside the IntersectionLine struct:
uint8_t m_index
This variable contains the tracking index of the line.
int16_t m_angle
This variable contains the angle in degrees of the line.
uint8_t numBarcodes
The number of barcodes in the barcodes variable.
Barcode barcodes[]
This array contains the detected barcodes.
Inside the Barcode struct:
void print()
Calling this prints the barcode information to the console.
uint8_t m_x
This variable contains the x location of the barcode. The value ranges between 0 and frameWidth (79).
uint8_t m_y
This variable contains the y location of the barcode. The value ranges between 0 and frameHeight (52).
uint8_t m_flags
This variable contains various flags that might be useful.
LINE_FLAG_INVALID: This flag is only available if getAllFeatures() is called. This flag indicates that the barcode has been detected but has not met the filtering constraint.
uint8_t m_code
This variable contains the value of the code, which can range between 0 and 15.
Video API
This API can be called through the video member of the Pixy2 object, for example:
Pixy2 pixy; uint8_t r, g, b; pixy.video.getRGB(50, 75, &r, &g, &b); // get red, green, blue values at location x=50, y=75
See also the Pixy2 General API, Color Connected Components API, and Line tracking API for other functionality.
For a byte-level reference to the protocol, check out the new Pixy2 Serial Protocol - Packet Reference
Firmware versions 3.0.11 and greater will automatically switch to the video program when making requests through the video API.
Take a look at the video_get_rgb example in the Arduino Pixy2 library.
Note, full frame-grabs are only supported through the USB interface and libpixyusb, not through the SPI, I2C or UART serial interfaces (i.e. Arduino). Libpixyusb includes a frame-grab example called get_raw_frame. It uses a special function called getRawFrame() that isn't part of the video API. We could add this, but where would a 65K-byte raw frame fit in Arduino memory? :(
Member functions
int8_t getRGB(uint16_t x, uint16_t y, uint8_t *r, uint8_t *g, uint8_t *b, bool saturate [optional])
This is currently the only function supported by the video program. It takes an x and y location in the image, and pointers to red, green and blue variables that get written into by the getRGB() function with the red, green and blue values of the pixel at that location in the image. The individual values of red, green and blue vary from 0 to 255.
The optional saturate argument when set to true (default) will scale all RGB values such that the greatest of the three values (r, g and b) is maximized (saturated) at 255. When set to false, the unmodified RGB values are returned.
Instead of using just one pixel, getRGB() takes a 5×5 section of pixels centered at the x, y location and performs an average of all 25 pixels to obtain a representative result. Locations on the edge or close to the edge of the image are allowed, but will result in fewer pixels being averaged.
The image coordinates are shown below.
The width and height values are both available through pixy.frameWidth and pixy.frameHeight, if you don't want to remember their specific values.
Note, PixyMon displays the x, y mouse location in image coordinates in the lower right corner of the main window to help determine the coordinates to request from getRGB().
1)
unless the LINE_MODE_MANUAL_SELECT_VECTOR mode bit is set through setMode(), in which case the Vector is set through setVector()
2)
but this can be changed by setting the default turn angle through setDefaultTurn() – normally the default turn angle is 0 (straight ahead.)
3)
The raw resolution of the line tracking algorithm is 632×108. The higher horizontal resolution is important for barcode detection/decoding. The line/vector coordinates have lower resolution because they are formed on a 79×52 occupancy grid and memory constraints prevent higher resolutions.
wiki/v2/pixy2_full_api.txt · Last modified: 2018/10/10 17:00 by pixycam
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
