This API can be called through the video member of the Pixy2 object, for example:

Pixy2 pixy;
uint8_t r, g, b; 

pixy.video.getRGB(50, 75, &r, &g, &b);  // get red, green, blue values at location x=50, y=75

See also the Pixy2 General API, Color Connected Components API, and Line tracking API for other functionality.

For a byte-level reference to the protocol, check out the new Pixy2 Serial Protocol - Packet Reference

Firmware versions 3.0.11 and greater will automatically switch to the video program when making requests through the video API.

Take a look at the video_get_rgb example in the Arduino Pixy2 library.

Note, full frame-grabs are only supported through the USB interface and libpixyusb, not through the SPI, I2C or UART serial interfaces (i.e. Arduino). Libpixyusb includes a frame-grab example called get_raw_frame. It uses a special function called getRawFrame() that isn't part of the video API. We could add this, but where would a 65K-byte raw frame fit in Arduino memory? :(

This is currently the only function supported by the video program. It takes an x and y location in the image, and pointers to red, green and blue variables that get written into by the getRGB() function with the red, green and blue values of the pixel at that location in the image. The individual values of red, green and blue vary from 0 to 255.

The optional saturate argument when set to true (default) will scale all RGB values such that the greatest of the three values (r, g and b) is maximized (saturated) at 255. When set to false, the unmodified RGB values are returned.

Instead of using just one pixel, getRGB() takes a 5×5 section of pixels centered at the x, y location and performs an average of all 25 pixels to obtain a representative result. Locations on the edge or close to the edge of the image are allowed, but will result in fewer pixels being averaged.

The image coordinates are shown below.

Video image coordinates from Pixy2's perspective

The width and height values are both available through pixy.frameWidth and pixy.frameHeight, if you don't want to remember their specific values.

Note, PixyMon displays the x, y mouse location in image coordinates in the lower right corner of the main window to help determine the coordinates to request from getRGB().