Table of Contents

How to talk to Pixy

We currently have software support for the following microcontrollers:

You may want Pixy to talk to a controller that's not on our official list, and that's totally fine – Pixy is easy to strike up a conversation with! Pixy has three separate methods of communication:

Picking the right interface

You first need to determine how you want to talk to Pixy. Pixy sends block information to Arduino at 1 Mbits/second over SPI because the Arduino SPI clock rate is set to 1 MHz (it can be set higher, however). This means Pixy can send more than 6000 detected objects per second or 135 detected objects per frame (Pixy processes at 50 frames per second.) But it's still possible that the serial link is too slow to send back all of the objects before the next frame is processed and new objects are ready to send over serial. This can especially happen with UART serial, which is typically only 19.2 kbaud, but can be as high as 115 kbaud. When this happens, the unsent object blocks from the previous frame are tossed and the new object blocks from the new frame are sent instead. This way, the most recent information is prioritized and always sent. And since the objects are sorted by size, the larger the object, the more priority it gets also. Consider these guidelines:

Setting the interface

You can configure the output interface through PixyMon. The “Data out port” parameter in the “Interface” tab determines which interface you're using.

Note: if you are running the Lego firmware, PixyMon will not display an Interface tab! If you want to use Pixy Lego with non-Lego controllers, please see this page.

adc46178730fa1742baa6d9ab5da823d0a60969f.jpg

Note, the USB interface and protocol is always enabled, while each of the interfaces above can only be enabled one at a given time.

Notes on each interface

Below are the pinouts of Pixy's I/O connector, where all of the serial (SPI, I2C, UART) and analog/digital interfaces are available.

c38c5bf8e350433e686aca84ca8bab657844d78c.jpg

Looking at the back of Pixy, the pins of the I/O port are in the following order, with pin 1 in the top left:

1   2
3   4
5   6
7   8
9   10

SPI

The ICSP SPI interface operates as an SPI slave. It is designed around the Arduino's ICSP port, which doesn't have a slave select signal. The default data rate is 1 mbits/sec, but this can be increased by modifying the Pixy.h file in the Pixy Arduino library. The protocol has checksums to deal with bit errors, but bear in mind that the ribbon cable isn't shielded! The specific type of SPI that Pixy supports:

Pixy also supports SPI with slave select (SPI with SS).

Here's how to hook up your controller's SPI to Pixy (if you are not using an Arduino and the supplied cable):

I2C

Here's how to hook up your controller's I2C to Pixy:

Note, when talking to more than one Pixy over I2C, you will need to configure a different I2C address for each Pixy so they don't step on each other. You can make a “multi-crimp cable”, meaning you can take a 10-conductor ribbon cable and crimp N 10-pin IDC connectors to it and plug into to your N Pixys. That is, when selecting I2C as an interface, all signals on Pixy's I/O connector go into a high-impedance state and won't interfere with each other, waste power, etc.

UART

Here's how to hook up your controller's UART to Pixy:

Analog and digital output

Here's how to hook up your controller's ADC and digital I/O to Pixy:

Note, all digital output signals on Pixy are 3.3V CMOS logic. All digital input signals on Pixy are 5V tolerant.

The serial protocol

Whether you're using SPI, I2C or UART serial, the protocol is exactly the same.

Object block format

Bytes    16-bit word    Description
----------------------------------------------------------------
0, 1     y              sync: 0xaa55=normal object, 0xaa56=color code object
2, 3     y              checksum (sum of all 16-bit words 2-6, that is, bytes 4-13)
4, 5     y              signature number
6, 7     y              x center of object
8, 9     y              y center of object
10, 11   y              width of object
12, 13   y              height of object

To mark between frames an extra sync word (0xaa55) is inserted. This means that a new image frame is indicated by either:

So, a typical way to parse the serial stream is to wait for two sync words and then start parsing the object block, using the sync words to indicate the start of the next object block, and so on.

Control data sent to Pixy

You can send control data to Pixy to control the pan/tilt servos, adjust the camera brightness and set the LED color. Like above, each 16-bit word is sent little endian (least significant byte sent first).

Pan/tilt servo control

Bytes    16-bit word   Description
----------------------------------------------------------------
0, 1     y             servo sync (0xff00)
2, 3     y             servo 0 (pan) position, between 0 and 1000
4, 5     y             servo 1 (tilt) position, between 0 and 1000

Camera brightness (exposure) control

Bytes    16-bit word   Description
----------------------------------------------------------------
0, 1     y             camera brightness sync (0xfe00)
2        n             brightness value

LED control

Bytes    16-bit word   Description
----------------------------------------------------------------
0, 1     y             LED sync (0xfd00)
2        n             red value
3        n             green value
4        n             blue value

Writing the code

It's good to start by just parsing the sync words to separate the frames. You can then count how many frames you get in 1 second. It should be 50, which is a good way to see if you're on the right track. Here's some code that does just that. 

#define PIXY_START_WORD             0xaa55
#define PIXY_START_WORD_CC          0xaa56
#define PIXY_START_WORDX            0x55aa

typedef enum
{
  NORMAL_BLOCK,
  CC_BLOCK // color code block
} BlockType;

static BlockType g_blockType; // use this to remember the next object block type between function calls

int getStart(void)
{
  uint16_t w, lastw;

  lastw = 0xffff; // some inconsequential initial value

  while(1)
  {
    w = getWord();
    if (w==0 && lastw==0)
      return 0; // in I2C and SPI modes this means no data, so return immediately
    else if (w==PIXY_START_WORD && lastw==PIXY_START_WORD)
    {
      g_blockType = NORMAL_BLOCK; // remember block type
      return 1; // code found!
    }
    else if (w==PIXY_START_WORD_CC && lastw==PIXY_START_WORD)
    {
      g_blockType = CC_BLOCK; // found color code block
      return 1;
    }
    else if (w==PIXY_START_WORDX) // this is important, we might be juxtaposed
      getByte(); // we're out of sync! (backwards)
    lastw = w; // save
  }
}

The missing routine is getWord(), which is fairly straightforward: 

extern uint8_t getByte(void); // external, does the right things for your interface

uint16_t getWord(void)
{
  // this routine assumes little endian
  uint16_t w;
  uint8_t c;
  c = getByte();
  w = getByte();
  w <<= 8;
  w |= c;
  return w;
}

So, something like this to verify that you're getting 50 frames/sec: 

int main()
{
  int i=0, curr, prev=0;

  // look for two start codes back to back
  while(1)
  {
    curr = getStart());
    if (prev && curr) // two start codes means start of new frame
      printf("%d", i++);
    prev = curr;
  }
}

but there's an important exception with SPI, so read “SPI tries its best to confuse things” below, if you're using SPI.

Parsing the rest of the object block is fairly straightforward, and the real meat of the code: 

#define PIXY_ARRAYSIZE              100

typedef struct
{
  uint16_t signature;
  uint16_t x;
  uint16_t y;
  uint16_t width;
  uint16_t height;
  uint16_t angle; // angle is only available for color coded blocks
} Block;

static int g_skipStart = 0;
static Block *g_blocks;

uint16_t getBlocks(uint16_t maxBlocks)
{
  uint8_t i;
  uint16_t w, blockCount, checksum, sum;
  Block *block;

  if (!g_skipStart)
  {
    if (getStart()==0)
      return 0;
  }
  else
    g_skipStart = 0;

  for(blockCount=0; blockCount<maxBlocks && blockCount<PIXY_ARRAYSIZE;)
  {
    checksum = getWord();
    if (checksum==PIXY_START_WORD) // we've reached the beginning of the next frame
    {
      g_skipStart = 1;
      g_blockType = NORMAL_BLOCK;
      return blockCount;
    }
    else if (checksum==PIXY_START_WORD_CC)
    {
      g_skipStart = 1;
      g_blockType = CC_BLOCK;
      return blockCount;
    }
    else if (checksum==0)
      return blockCount;

    block = g_blocks + blockCount;

    for (i=0, sum=0; i<sizeof(Block)/sizeof(uint16_t); i++)
    {
      if (g_blockType==NORMAL_BLOCK && i>=5) // no angle for normal block
      {
        block->angle = 0;
        break;
      }
      w = getWord();
      sum += w;
      *((uint16_t *)block + i) = w;
    }

    // check checksum
    if (checksum==sum)
      blockCount++;
    else
      printf("checksum error!\n");

    w = getWord();
    if (w==PIXY_START_WORD)
      g_blockType = NORMAL_BLOCK;
    else if (w==PIXY_START_WORD_CC)
      g_blockType = CC_BLOCK;
    else
      return blockCount;
  }
}

This code assumes that the sync word has already been read when it's called. Probably the only part of the above code that requires some explaining is the gskipStart variable. It's there because we might read a sync instead of a checksum if the last block is read. gskipStart tells us that we've already read the sync.

The code above copies blocks that it reads into the g_blocks array, which needs to be initialized, of course, so something like this: 

void init()
{
  g_blocks = (Block *)malloc(sizeof(Block)*PIXY_ARRAYSIZE);
}

OK, so we've handled the object data coming from Pixy, what about control data sent to Pixy? You know, to control the pan/tilt servos, set the brightness of the camera, or set the LED color: 

#define PIXY_SERVO_SYNC             0xff
#define PIXY_CAM_BRIGHTNESS_SYNC    0xfe
#define PIXY_LED_SYNC               0xfd

extern int sendByte(uint8_t byte);

int send(uint8_t *data, int len)
{
  int i;
  for (i=0; i<len; i++)
    sendByte(data[i]);

  return len;
}

int setServos(uint16_t s0, uint16_t s1)
{
  uint8_t outBuf[6];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_SERVO_SYNC;
  *(uint16_t *)(outBuf + 2) = s0;
  *(uint16_t *)(outBuf + 4) = s1;

  return send(outBuf, 6);
}

int setBrightness(uint8_t brightness)
{
  uint8_t outBuf[3];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_CAM_BRIGHTNESS_SYNC;
  outBuf[2] = brightness;

  return send(outBuf, 3);
}

int setLED(uint8_t r, uint8_t g, uint8_t b)
{
  uint8_t outBuf[5];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_LED_SYNC;
  outBuf[2] = r;
  outBuf[3] = g;
  outBuf[4] = b;

  return send(outBuf, 5);
}

And like getByte(), the sendByte() routine is an external routine that does the right things for your interface.

SPI tries its best to confuse things

So this all makes sense, except that SPI on Pixy has some curveballs to throw at us:

So here's the code for SPI that takes these caveats into consideration: 

#define PIXY_SYNC_BYTE              0x5a  // to sync SPI data
#define PIXY_SYNC_BYTE_DATA         0x5b  // to sync/indicate SPI send data
#define PIXY_OUTBUF_SIZE            64

// SPI sends as it receives so we need a getByte routine that
// takes an output data argument
extern uint8_t getByte(uint8_t out);

// variables for a little circular queue for SPI output data
static uint8_t g_outBuf[PIXY_OUTBUF_SIZE];
static uint8_t g_outLen = 0;
static uint8_t g_outWriteIndex = 0;
static uint8_t g_outReadIndex = 0;

uint16_t getWord()
{
  // ordering is big endian because Pixy is sending 16 bits through SPI
  uint16_t w;
  uint8_t c, cout = 0;

  if (g_outLen)
  {
    w = getByte(PIXY_SYNC_BYTE_DATA);
    cout = g_outBuf[g_outReadIndex++];
    g_outLen--;
    if (g_outReadIndex==PIXY_OUTBUF_SIZE)
      g_outReadIndex = 0;
  }
  else
    w = getByte(PIXY_SYNC_BYTE); // send out sync byte
  w <<= 8;
  c = getByte(cout); // send out data byte
  w |= c;

  return w;
}

int send(uint8_t *data, int len)
{
  int i;

  // check to see if we have enough space in our circular queue
  if (g_outLen+len>PIXY_OUTBUF_SIZE)
    return -1;

  g_outLen += len;
  for (i=0; i<len; i++)
  {
    g_outBuf[g_outWriteIndex++] = data[i];
    if (g_outWriteIndex==PIXY_OUTBUF_SIZE)
      g_outWriteIndex = 0;
  }
  return len;
}

You see that We implement a little circular queue for the sent data because receiving and sending are bound together.

OK, so that's everything. Here's the complete code for your reference: 

#include <inttypes.h>
#include <stdio.h>
#include <stdlib.h>

// Are you using an SPI interface?  if so, uncomment this line
#define SPI

#define PIXY_ARRAYSIZE              100
#define PIXY_START_WORD             0xaa55
#define PIXY_START_WORD_CC          0xaa56
#define PIXY_START_WORDX            0x55aa
#define PIXY_SERVO_SYNC             0xff
#define PIXY_CAM_BRIGHTNESS_SYNC    0xfe
#define PIXY_LED_SYNC               0xfd
#define PIXY_OUTBUF_SIZE            64

#define PIXY_SYNC_BYTE              0x5a
#define PIXY_SYNC_BYTE_DATA         0x5b

// the routines
void init();
int getStart(void);
uint16_t getBlocks(uint16_t maxBlocks);
int setServos(uint16_t s0, uint16_t s1);
int setBrightness(uint8_t brightness);
int setLED(uint8_t r, uint8_t g, uint8_t b);

// data types
typedef enum
{
    NORMAL_BLOCK,
    CC_BLOCK // color code block
} BlockType;

typedef struct
{
  uint16_t signature;
  uint16_t x;
  uint16_t y;
  uint16_t width;
  uint16_t height;
  uint16_t angle; // angle is only available for color coded blocks
} Block;

// communication routines
static uint16_t getWord(void);
static int send(uint8_t *data, int len);

#ifndef SPI //////////// for I2C and UART

extern uint8_t getByte(void);
extern int sendByte(uint8_t byte);

uint16_t getWord(void)
{
  // this routine assumes little endian
  uint16_t w;
  uint8_t c;
  c = getByte();
  w = getByte();
  w <<= 8;
  w |= c;
  return w;
}

int send(uint8_t *data, int len)
{
  int i;
  for (i=0; i<len; i++)
    sendByte(data[i]);

  return len;
}

#else ///////////// SPI routines

// SPI sends as it receives so we need a getByte routine that
// takes an output data argument
extern uint8_t getByte(uint8_t out);

// variables for a little circular queue
static uint8_t g_outBuf[PIXY_OUTBUF_SIZE];
static uint8_t g_outLen = 0;
static uint8_t g_outWriteIndex = 0;
static uint8_t g_outReadIndex = 0;

uint16_t getWord()
{
  // ordering is big endian because Pixy is sending 16 bits through SPI
  uint16_t w;
  uint8_t c, cout = 0;

  if (g_outLen)
  {
    w = getByte(PIXY_SYNC_BYTE_DATA);
    cout = g_outBuf[g_outReadIndex++];
    g_outLen--;
    if (g_outReadIndex==PIXY_OUTBUF_SIZE)
      g_outReadIndex = 0;
  }
  else
    w = getByte(PIXY_SYNC_BYTE); // send out sync byte
  w <<= 8;
  c = getByte(cout); // send out data byte
  w |= c;

  return w;
}

int send(uint8_t *data, int len)
{
  int i;

  // check to see if we have enough space in our circular queue
  if (g_outLen+len>PIXY_OUTBUF_SIZE)
    return -1;

  g_outLen += len;
  for (i=0; i<len; i++)
  {
    g_outBuf[g_outWriteIndex++] = data[i];
    if (g_outWriteIndex==PIXY_OUTBUF_SIZE)
      g_outWriteIndex = 0;
  }
  return len;
}

#endif //////////////// end SPI routines

static int g_skipStart = 0;
static BlockType g_blockType;
static Block *g_blocks;

void init()
{
  g_blocks = (Block *)malloc(sizeof(Block)*PIXY_ARRAYSIZE);
}

int getStart(void)
{
  uint16_t w, lastw;

  lastw = 0xffff;

  while(1)
  {
    w = getWord();
    if (w==0 && lastw==0)
      return 0; // no start code
    else if (w==PIXY_START_WORD && lastw==PIXY_START_WORD)
    {
      g_blockType = NORMAL_BLOCK;
      return 1; // code found!
    }
    else if (w==PIXY_START_WORD_CC && lastw==PIXY_START_WORD)
    {
      g_blockType = CC_BLOCK; // found color code block
      return 1;
    }
    else if (w==PIXY_START_WORDX)
#ifdef SPI
      getByte(0); // we're out of sync! (backwards)
#else
      getByte(); // we're out of sync! (backwards)
#endif
    lastw = w;
  }
}

uint16_t getBlocks(uint16_t maxBlocks)
{
  uint8_t i;
  uint16_t w, blockCount, checksum, sum;
  Block *block;

  if (!g_skipStart)
  {
    if (getStart()==0)
      return 0;
  }
  else
    g_skipStart = 0;

  for(blockCount=0; blockCount<maxBlocks && blockCount<PIXY_ARRAYSIZE;)
  {
    checksum = getWord();
    if (checksum==PIXY_START_WORD) // we've reached the beginning of the next frame
    {
      g_skipStart = 1;
      g_blockType = NORMAL_BLOCK;
      return blockCount;
    }
    else if (checksum==PIXY_START_WORD_CC)
    {
      g_skipStart = 1;
      g_blockType = CC_BLOCK;
      return blockCount;
    }
    else if (checksum==0)
      return blockCount;

    block = g_blocks + blockCount;

    for (i=0, sum=0; i<sizeof(Block)/sizeof(uint16_t); i++)
    {
      if (g_blockType==NORMAL_BLOCK && i>=5) // no angle for normal block
      {
        block->angle = 0;
        break;
      }
      w = getWord();
      sum += w;
      *((uint16_t *)block + i) = w;
    }

    // check checksum
    if (checksum==sum)
      blockCount++;
    else
      printf("checksum error!\n");

    w = getWord();
    if (w==PIXY_START_WORD)
      g_blockType = NORMAL_BLOCK;
    else if (w==PIXY_START_WORD_CC)
      g_blockType = CC_BLOCK;
    else
      return blockCount;
  }
}

int setServos(uint16_t s0, uint16_t s1)
{
  uint8_t outBuf[6];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_SERVO_SYNC;
  *(uint16_t *)(outBuf + 2) = s0;
  *(uint16_t *)(outBuf + 4) = s1;

  return send(outBuf, 6);
}

int setBrightness(uint8_t brightness)
{
  uint8_t outBuf[3];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_CAM_BRIGHTNESS_SYNC;
  outBuf[2] = brightness;

  return send(outBuf, 3);
}

int setLED(uint8_t r, uint8_t g, uint8_t b)
{
  uint8_t outBuf[5];

  outBuf[0] = 0x00;
  outBuf[1] = PIXY_LED_SYNC;
  outBuf[2] = r;
  outBuf[3] = g;
  outBuf[4] = b;

  return send(outBuf, 5);
}