====== How to talk to Pixy ====== We currently have software support for the following microcontrollers: * [[wiki:v1:Hooking_up_Pixy_to_a_microcontroller_-28like_an_arduino-29|Arduino]] * [[wiki:v1:Hooking_up_Pixy_to_a_Raspberry_Pi|Raspberry Pi]] * [[wiki:v1:Hooking_up_Pixy_to_a_Beaglebone_Black|BeagleBone Black]] You may want Pixy to talk to a controller that's not on our official list, and that's totally fine -- Pixy is easy to strike up a conversation with! Pixy has three separate methods of communication: * **Serial:** this includes SPI, I2C and UART interfaces. The serial interfaces all use a simplified protocol with a small code and memory footprint. The code is simple to port to different microcontrollers. This is the method that is used for communication with Arduino. Using the serial protocol, Pixy sends complete information about what it detects, and it accepts simple commands for setting the pan/tilt servos, etc. * **USB:** the USB interface is intended for microcontrollers that have more memory resources (RAM and flash). The code for the USB protocol has a larger memory footprint. This is the method that is used to talk to the Raspberry Pi and BeagleBone Black, and it is used by PixyMon to stream live video and read/write configuration information. * **Analog/digital:** this is the simplest interface and doesn't even have a protocol. ===== Picking the right interface ===== You first need to determine how you want to talk to Pixy. Pixy sends block information to Arduino at 1 Mbits/second over SPI because the Arduino SPI clock rate is set to 1 MHz (it can be set higher, however). This means Pixy can send more than 6000 detected objects per second or 135 detected objects per frame (Pixy processes at 50 frames per second.) But it's still possible that the serial link is too slow to send back all of the objects before the next frame is processed and new objects are ready to send over serial. This can especially happen with UART serial, which is typically only 19.2 kbaud, but can be as high as 115 kbaud. When this happens, the unsent object blocks from the previous frame are tossed and the new object blocks from the new frame are sent instead. This way, the most recent information is prioritized and always sent. And since the objects are sorted by size, the larger the object, the more priority it gets also. Consider these guidelines: * If your controller is Linux-based and has a USB host port, use [[wiki:v1:Building_the_libpixyusb_example_on_Linux|libpixyusb]]. It's fairly easy to port over and USB is the fastest interface. * if your controller supports SPI, use that. It's typically faster than I2C and UART, or * if your controller supports I2C, use that. It's about the same speed as UART serial, but more flexible, or * if your controller supports UART serial, use that, or * if your controller doesn't support any of these interfaces, consider using simple analog and digital output. It's the simplest interface possible! ===== Setting the interface ===== You can configure the output interface through PixyMon. The "Data out port" parameter in the "Interface" tab determines which interface you're using. **Note: if you are running the Lego firmware, PixyMon will not display an Interface tab!** If you want to use Pixy Lego with non-Lego controllers, please see [[wiki:v1:Can_I_use_my_LEGO_Pixy_with_Arduino|this page.]] {{wiki:img:adc46178730fa1742baa6d9ab5da823d0a60969f.jpg}} * Arduino ICSP SPI - this is the default port that uses 3 wires (pins 1, 3, and 4 of the I/O connector) and is used to communicate with Arduino through the ICSP connector. This version of SPI does not use a slave select signal. * SPI with SS - this is the same as the Arduino ICSP SPI except that it includes support for Slave Select through pin 7 (SPI SS). That is you need to drive SPI SS low before sending/receiving each byte. * I2C - this is a multi-drop 2-wire port (pins 5 and 9 of the I/O connector) that allows a single master to communicate with up to 127 slaves (up to 127 Pixys). You can configure the I2C address through the "I2C address" parameter. * UART - this is the common "serial port" (pins 1 and 4 of the I/O connector). Pixy receives data via pin 1 (input) and transmits data via pin 4 (output). You can configure the UART baudrate through the "UART baudrate" parameter. * analog/digital x - this will output the x value of the largest detected object as an analog value between 0 and 3.3V (pin 3). It also outputs whether an object is detected or not as a digital signal (pin 1 of the I/O connector). * analog/digital y - this will output the y value of the largest detected object as an analog value between 0 and 3.3V (pin 3). It also outputs whether an object is detected or not as a digital signal (pin 1 of the I/O connector). * LEGO I2C - if you have a [[http://www.lego.com/en-us/mindstorms/?domainredir=mindstorms.lego.com|LEGO Mindstorms]] EV3 or NXP controller, use this mode. It's I2C, but uses the LEGO protocol. Note, the USB interface and protocol is always enabled, while each of the interfaces above can only be enabled one at a given time. ===== Notes on each interface ===== Below are the pinouts of Pixy's I/O connector, where all of the serial (SPI, I2C, UART) and analog/digital interfaces are available. {{wiki:img:c38c5bf8e350433e686aca84ca8bab657844d78c.jpg?400}} Looking at the back of Pixy, the pins of the I/O port are in the following order, with pin 1 in the top left:\\ 1 2 3 4 5 6 7 8 9 10 ==== SPI ==== The ICSP SPI interface operates as an SPI slave. It is designed around the Arduino's ICSP port, which doesn't have a slave select signal. The default data rate is 1 mbits/sec, but this can be increased by modifying the Pixy.h file in the Pixy Arduino library. The protocol has checksums to deal with bit errors, but bear in mind that the ribbon cable isn't shielded! The specific type of SPI that Pixy supports: - Data is sent most significant bit first - SPI SCK is low when idle - Data bits are latched on the rising edge of SPI SCK - Slave select is active low - 3.3V outputs, 5V tolerant Pixy also supports SPI with slave select (SPI with SS). Here's how to hook up your controller's SPI to Pixy (if you are not using an Arduino and the supplied cable): - Pin 10 ➜ your controller's ground signal - Pin 1 (SPI MISO) ➜ your controller's SPI MISO signal - Pin 4 (SPI MOSI) ➜ your controller's SPI MOSI signal - Pin 3 (SPI SCK) ➜ your controller's SPI SCK signal - Pin 7 (SPI SS) ➜ your controller's SPI SS signal (if you're using SPI with SS) ==== I2C ==== * The I2C interface operates as an I2C slave and requires polling. * There are weak 4.7K pullups to 5V on SDA and SCL signals, via R14 and R15. * I2C signals are 5V tolerant. * The I2C address can be configured in the "Interface" tab of the Configure Parameters dialog in PixyMon * There is an Arduino example that uses I2C. Run it by selecting File➜Examples➜i2c in the Arduino IDE. You'll need to make a special cable. Here's how to hook up your controller's I2C to Pixy: - Pin 10 ➜ your controller's ground signal - Pin 9 (I2C SDA) ➜ your controller's I2C SDA signal - Pin 5 (I2C SCL) ➜ your controller's I2C SCL signal Note, when talking to more than one Pixy over I2C, you will need to configure a different I2C address for each Pixy so they don't step on each other. You can make a "multi-crimp cable", meaning you can take a 10-conductor ribbon cable and crimp N 10-pin IDC connectors to it and plug into to your N Pixys. That is, when selecting I2C as an interface, all signals on Pixy's I/O connector go into a high-impedance state and won't interfere with each other, waste power, etc. ==== UART ==== * The UART interface is 8 data bits, 1 stop bit, no parity, no handshaking * The baudrate can be configured in the "Interface" tab of the Configure dialog in PixyMon * RX signal (pin 1) is 5V tolerant input * TX signal (pin 4) is 0 to 3.3V signal output * Baudrates up to 230 kbaud supported. * There is an Arduino example that uses UART serial. Run it by selecting File➜Examples➜uart in the Arduino IDE. You'll need to make a special cable. Here's how to hook up your controller's UART to Pixy: - Pin 10 ➜ your controller's ground signal - Pin 1 (UART RX) ➜ your controller's UART TX output - Pin 4 (UART TX) ➜ your controller's UART RX input ==== Analog and digital output ==== * Pixy has a single analog (DAC) output, so there are two modes for analog/digital output. * Mode 4 outputs the x value of the center of the biggest detected object to pin 3 of the I/O connector. * Mode 5 outputs the y value of the biggest detected object to pin 3 of the I/O connector. * Pin 1 goes high (3.3V) when an object is detected, and low (0V) when no object is detected. * Pixy's digital output is 0 to 3.3V logic and can source/sink 5 mA. * Pixy's analog (DAC) output ranges between 0 to 3.3V with roughly 200 ohm impedance. * Pixy's analog (DAC) output voltage is directly, linearly proportional to the object position in the image (depending on the mode) * In mode 4 (x mode) the analog output is 0V if object is on the far left of the image and 3.3V is object on the far right (PixyMon perspective). * In mode 5 (y mode) the analog output is 0V if object is on the bottom of the image and 3.3V is object on the top (PixyMon perspective). Here's how to hook up your controller's ADC and digital I/O to Pixy: - Pin 10 ➜ your controller's ground signal - Pin 3 (DAC OUT) ➜ one of your controller's ADC input signals - Pin 1 (GPIO0) ➜ one of your controller's digital input signals Note, all digital output signals on Pixy are 3.3V CMOS logic. All digital input signals on Pixy are 5V tolerant. ===== The serial protocol ===== Whether you're using SPI, I2C or UART serial, the protocol is exactly the same. * The protocol is data-efficient binary. * The objects in each frame are sorted by size, with the largest objects sent first. * You can configure the maximum number of objects sent per image frame ("Max blocks" parameter). * SPI and I2C operate in "slave mode" and rely on polling to receive updates. * When there are no detected objects (no data) Pixy sends zeros if the interface is SPI or I2C (since Pixy is a slave, it has to send something). * Each object is sent in an "object block" (see below). * All values in the object block are 16-bit words, sent least-signifcant byte first (little endian). So, for example, when sending the sync word 0xaa55, Pixy sends 0x55 (first byte) then 0xaa (second byte). ==== Object block format ==== Bytes 16-bit word Description ---------------------------------------------------------------- 0, 1 y sync: 0xaa55=normal object, 0xaa56=color code object 2, 3 y checksum (sum of all 16-bit words 2-6, that is, bytes 4-13) 4, 5 y signature number 6, 7 y x center of object 8, 9 y y center of object 10, 11 y width of object 12, 13 y height of object To mark between frames an extra sync word (0xaa55) is inserted. This means that a new image frame is indicated by either: - Two sync words sent back-to-back (0xaa55, 0xaa55), or - a normal sync followed by a color-code sync (0xaa55, 0xaa56). So, a typical way to parse the serial stream is to wait for two sync words and then start parsing the object block, using the sync words to indicate the start of the next object block, and so on. ==== Control data sent to Pixy ==== You can send control data to Pixy to control the pan/tilt servos, adjust the camera brightness and set the LED color. Like above, each 16-bit word is sent little endian (least significant byte sent first). ==== Pan/tilt servo control ==== Bytes 16-bit word Description ---------------------------------------------------------------- 0, 1 y servo sync (0xff00) 2, 3 y servo 0 (pan) position, between 0 and 1000 4, 5 y servo 1 (tilt) position, between 0 and 1000 ==== Camera brightness (exposure) control ==== Bytes 16-bit word Description ---------------------------------------------------------------- 0, 1 y camera brightness sync (0xfe00) 2 n brightness value ==== LED control ==== Bytes 16-bit word Description ---------------------------------------------------------------- 0, 1 y LED sync (0xfd00) 2 n red value 3 n green value 4 n blue value ===== Writing the code ===== It's good to start by just parsing the sync words to separate the frames. You can then count how many frames you get in 1 second. It should be 50, which is a good way to see if you're on the right track. Here's some code that does just that. #define PIXY_START_WORD 0xaa55 #define PIXY_START_WORD_CC 0xaa56 #define PIXY_START_WORDX 0x55aa typedef enum { NORMAL_BLOCK, CC_BLOCK // color code block } BlockType; static BlockType g_blockType; // use this to remember the next object block type between function calls int getStart(void) { uint16_t w, lastw; lastw = 0xffff; // some inconsequential initial value while(1) { w = getWord(); if (w==0 && lastw==0) return 0; // in I2C and SPI modes this means no data, so return immediately else if (w==PIXY_START_WORD && lastw==PIXY_START_WORD) { g_blockType = NORMAL_BLOCK; // remember block type return 1; // code found! } else if (w==PIXY_START_WORD_CC && lastw==PIXY_START_WORD) { g_blockType = CC_BLOCK; // found color code block return 1; } else if (w==PIXY_START_WORDX) // this is important, we might be juxtaposed getByte(); // we're out of sync! (backwards) lastw = w; // save } } The missing routine is getWord(), which is fairly straightforward: extern uint8_t getByte(void); // external, does the right things for your interface uint16_t getWord(void) { // this routine assumes little endian uint16_t w; uint8_t c; c = getByte(); w = getByte(); w <<= 8; w |= c; return w; } So, something like this to verify that you're getting 50 frames/sec: int main() { int i=0, curr, prev=0; // look for two start codes back to back while(1) { curr = getStart()); if (prev && curr) // two start codes means start of new frame printf("%d", i++); prev = curr; } } but there's an **important exception with SPI**, so read **"SPI tries its best to confuse things"** below, if you're using SPI. Parsing the rest of the object block is fairly straightforward, and the real meat of the code: #define PIXY_ARRAYSIZE 100 typedef struct { uint16_t signature; uint16_t x; uint16_t y; uint16_t width; uint16_t height; uint16_t angle; // angle is only available for color coded blocks } Block; static int g_skipStart = 0; static Block *g_blocks; uint16_t getBlocks(uint16_t maxBlocks) { uint8_t i; uint16_t w, blockCount, checksum, sum; Block *block; if (!g_skipStart) { if (getStart()==0) return 0; } else g_skipStart = 0; for(blockCount=0; blockCount=5) // no angle for normal block { block->angle = 0; break; } w = getWord(); sum += w; *((uint16_t *)block + i) = w; } // check checksum if (checksum==sum) blockCount++; else printf("checksum error!\n"); w = getWord(); if (w==PIXY_START_WORD) g_blockType = NORMAL_BLOCK; else if (w==PIXY_START_WORD_CC) g_blockType = CC_BLOCK; else return blockCount; } } This code assumes that the sync word has already been read when it's called. Probably the only part of the above code that requires some explaining is the g_skipStart variable. It's there because we might read a sync instead of a checksum if the last block is read. g_skipStart tells us that we've already read the sync. The code above copies blocks that it reads into the g_blocks array, which needs to be initialized, of course, so something like this: void init() { g_blocks = (Block *)malloc(sizeof(Block)*PIXY_ARRAYSIZE); } OK, so we've handled the object data coming from Pixy, what about control data sent to Pixy? You know, to control the pan/tilt servos, set the brightness of the camera, or set the LED color: #define PIXY_SERVO_SYNC 0xff #define PIXY_CAM_BRIGHTNESS_SYNC 0xfe #define PIXY_LED_SYNC 0xfd extern int sendByte(uint8_t byte); int send(uint8_t *data, int len) { int i; for (i=0; i And like getByte(), the sendByte() routine is an external routine that does the right things for your interface. ==== SPI tries its best to confuse things ==== So this all makes sense, except that SPI on Pixy has some curveballs to throw at us: - SPI is a simultaneous send/receive interface, so our getByte() routine instead of just returning a received data byte, needs to accept an output data byte too. - To save CPU, Pixy configures its SPI controller with 16-bit words instead of 8. This works great, but the 16-bit words are sent big-endian instead of little-endian. - Pixy relies on sync bytes sent to it to make sure it has good bit-sync, so you need to send a sync byte every other byte when talking to Pixy over SPI. This also solves the data imbalance problem with Pixy and SPI -- that is, there's a lot more data being sent by Pixy than being received by Pixy. The sync bytes allow Pixy to separate the filler data from the valid data. So here's the code for SPI that takes these caveats into consideration: #define PIXY_SYNC_BYTE 0x5a // to sync SPI data #define PIXY_SYNC_BYTE_DATA 0x5b // to sync/indicate SPI send data #define PIXY_OUTBUF_SIZE 64 // SPI sends as it receives so we need a getByte routine that // takes an output data argument extern uint8_t getByte(uint8_t out); // variables for a little circular queue for SPI output data static uint8_t g_outBuf[PIXY_OUTBUF_SIZE]; static uint8_t g_outLen = 0; static uint8_t g_outWriteIndex = 0; static uint8_t g_outReadIndex = 0; uint16_t getWord() { // ordering is big endian because Pixy is sending 16 bits through SPI uint16_t w; uint8_t c, cout = 0; if (g_outLen) { w = getByte(PIXY_SYNC_BYTE_DATA); cout = g_outBuf[g_outReadIndex++]; g_outLen--; if (g_outReadIndex==PIXY_OUTBUF_SIZE) g_outReadIndex = 0; } else w = getByte(PIXY_SYNC_BYTE); // send out sync byte w <<= 8; c = getByte(cout); // send out data byte w |= c; return w; } int send(uint8_t *data, int len) { int i; // check to see if we have enough space in our circular queue if (g_outLen+len>PIXY_OUTBUF_SIZE) return -1; g_outLen += len; for (i=0; i You see that We implement a little circular queue for the sent data because receiving and sending are bound together. OK, so that's everything. Here's the complete code for your reference: #include #include #include // Are you using an SPI interface? if so, uncomment this line #define SPI #define PIXY_ARRAYSIZE 100 #define PIXY_START_WORD 0xaa55 #define PIXY_START_WORD_CC 0xaa56 #define PIXY_START_WORDX 0x55aa #define PIXY_SERVO_SYNC 0xff #define PIXY_CAM_BRIGHTNESS_SYNC 0xfe #define PIXY_LED_SYNC 0xfd #define PIXY_OUTBUF_SIZE 64 #define PIXY_SYNC_BYTE 0x5a #define PIXY_SYNC_BYTE_DATA 0x5b // the routines void init(); int getStart(void); uint16_t getBlocks(uint16_t maxBlocks); int setServos(uint16_t s0, uint16_t s1); int setBrightness(uint8_t brightness); int setLED(uint8_t r, uint8_t g, uint8_t b); // data types typedef enum { NORMAL_BLOCK, CC_BLOCK // color code block } BlockType; typedef struct { uint16_t signature; uint16_t x; uint16_t y; uint16_t width; uint16_t height; uint16_t angle; // angle is only available for color coded blocks } Block; // communication routines static uint16_t getWord(void); static int send(uint8_t *data, int len); #ifndef SPI //////////// for I2C and UART extern uint8_t getByte(void); extern int sendByte(uint8_t byte); uint16_t getWord(void) { // this routine assumes little endian uint16_t w; uint8_t c; c = getByte(); w = getByte(); w <<= 8; w |= c; return w; } int send(uint8_t *data, int len) { int i; for (i=0; iPIXY_OUTBUF_SIZE) return -1; g_outLen += len; for (i=0; i=5) // no angle for normal block { block->angle = 0; break; } w = getWord(); sum += w; *((uint16_t *)block + i) = w; } // check checksum if (checksum==sum) blockCount++; else printf("checksum error!\n"); w = getWord(); if (w==PIXY_START_WORD) g_blockType = NORMAL_BLOCK; else if (w==PIXY_START_WORD_CC) g_blockType = CC_BLOCK; else return blockCount; } } int setServos(uint16_t s0, uint16_t s1) { uint8_t outBuf[6]; outBuf[0] = 0x00; outBuf[1] = PIXY_SERVO_SYNC; *(uint16_t *)(outBuf + 2) = s0; *(uint16_t *)(outBuf + 4) = s1; return send(outBuf, 6); } int setBrightness(uint8_t brightness) { uint8_t outBuf[3]; outBuf[0] = 0x00; outBuf[1] = PIXY_CAM_BRIGHTNESS_SYNC; outBuf[2] = brightness; return send(outBuf, 3); } int setLED(uint8_t r, uint8_t g, uint8_t b) { uint8_t outBuf[5]; outBuf[0] = 0x00; outBuf[1] = PIXY_LED_SYNC; outBuf[2] = r; outBuf[3] = g; outBuf[4] = b; return send(outBuf, 5); }