Arduino + accelerometer as joystick for Second Life

Update – example code available here.

Here’s something I hacked together earlier this year & although I didn’t end up developing it further it may prove interesting to others.


First off a quick video showing the Arduino + accelerometer ‘joystick’ controlling the avatar & then the flycam in the official Second Life client. If this piques your interest, read on to find out how (& why) it was done.

I wanted to investigate how I might use real world orientation data, such as that recorded by an accelerometer attached to an Arduino, to control a Second Life avatar without having to modify the source code of the Second Life client. This approach would have two major advantages;

  • conceivably much less work required
  • compatibility with any Second Life client sans modification – no reliance on a bespoke modified client

Unfortunately the official Second Life client (upon which all third party clients are based) doesn’t really present any interfaces for input devices apart from the usual mouse, keyboard & joystick. But some work that my colleague John was doing at the time getting XBox controllers to move avatars using the joystick interface got me thinking & a Google search for ‘Arduino joystick’ led to my discovery of the Arduino UNO Joystick HID firmware based on LUFA (the Lightweight USB Framework for AVRs) which allows an Arduino to appear to a computer as a standard USB HID joystick, instead of as a serial device, by reprogramming the USB-to-serial converter. Of course this ‘joystick’ can in fact be used by any program/game, not just Second Life.

Note: this is only possible with Arduino Uno & Mega, which use an ATMega chip for USB-to-serial conversion. It does not work with older Arduino, such as the Duemilanove, which use a FTDI chip. My experiments used an Uno R3 which has the ATMega16U2, check compatibility before you attempt the following with a Uno/Mega R1/R2 which use the ATMega8U2.

Reprogramming the ATMega16U2 involves putting the Arduino into DFU mode & using dfu-programmer. Unfortunately the latest version of dfu-programmer (0.5.4) doesn’t know about the ATMega16U2 so has to be patched. Grab the latest dfu-programmer source & apply this patch as discussed in this thread on the Arduino.cc forums.

With your Arduino connected to your computer as normal you should see something similar if you run lsusb.

[root@flatline /]# lsusb
Bus 003 Device 007: ID 2341:0043 Arduino SA Uno R3 (CDC ACM)

To enter DFU mode find the 6-pin AVR header near the USB socket & briefly connect the 2 pins closest to the USB socket (see picture beneath, click for full size) using the tip of a screwdriver, a paperclip, piece of wire, etc.

The Arduino should no longer register in lsusb but a myserious Atmel Corp. device should have appeared in its place. The serial port (eg /dev/ttyACM0) should also have disappeared.

[root@flatline /]# lsusb
Bus 003 Device 011: ID 03eb:2fef Atmel Corp.

You can then go ahead & erase the original firmware & flash the joystick HID firmware.

[root@flatline /]# dfu-programmer atmega16u2 erase
root@flatline /]# dfu-programmer atmega16u2 flash Arduino-joystick-0.1.hex 
Validating...
4076 bytes used (33.17%)
[root@flatline /]# dfu-programmer atmega16u2 reset

And you’re done! At this point you will need to physically disconnect & reconnect the Arduino for the computer to recognise it as a joystick. After you have done so, lsusb should report something like this.

[root@flatline /]# lsusb
Bus 003 Device 012: ID 03eb:2043 Atmel Corp. LUFA Joystick Demo Application

With the joystick firmware in place you will no longer be able to upload sketches as normal. If you don’t have a USBtiny ISP or similar you will have to revert back to the original USB-to-serial firmware each time you want to upload a new sketch. This process is exactly the same as above, but substituting the joystick .hex file with the USB-to-serial one.

[root@flatline /]# dfu-programmer atmega16u2 erase
[root@flatline /]# dfu-programmer atmega16u2 flash Arduino-usbserial-atmega16u2-Uno-Rev3.hex 
Validating...
4034 bytes used (32.83%)
[root@flatline /]# dfu-programmer atmega16u2 reset

Once again, you will have to disconnect & reconnect the Arduino for your computer to register the change.


As for the sketch itself, mapping accelerometer readings to the joystick axes is simply a case of inserting them into the correct variables in the joyReport struct & sending it over Serial – take a look at the example sketch that comes with the joystick firmware & you should soon see how to do it. Beneath is a rudimentary example using readings from the Honeywell HMC6343 from Sparkfun (see here for how to use this with Arduino), mapping the accelerometer’s roll to the joystick’s X axis & its pitch to the Y axis.

 
#include <Wire.h>
 
#define HMC6343_ADDRESS 0x19
#define HMC6343_HEADING_REG 0x50

// data structure as defined by the joystick firmeware
struct {
    int8_t x;
    int8_t y;
    uint8_t buttons;
    uint8_t rfu;
} joyReport;

void setup() {
  Wire.begin();          // initialize the I2C bus
  Serial.begin(115200);  // initialize the serial bus
}
 
void loop() {
  byte highByte, lowByte;
 
  Wire.beginTransmission(HMC6343_ADDRESS);    // start communication with HMC6343
  Wire.write(0x74);                           // set HMC6343 orientation
  Wire.write(HMC6343_HEADING_REG);            // send the address of the register to read
  Wire.endTransmission();
 
  Wire.requestFrom(HMC6343_ADDRESS, 6);       // request six bytes of data from the HMC6343
  while(Wire.available() < 1);                // busy wait while there is no byte to receive
 
  highByte = Wire.read();
  lowByte = Wire.read();
  float heading = ((highByte << 8) + lowByte) / 10.0; // heading in degrees
 
  highByte = Wire.read();
  lowByte = Wire.read();
  float pitch = ((highByte << 8) + lowByte) / 10.0;   // pitch in degrees
 
  highByte = Wire.read();
  lowByte = Wire.read();
  float roll = ((highByte << 8) + lowByte) / 10.0;    // roll in degrees

  joyReport.buttons = 0;
  joyReport.rfu = 0;
  
  joyReport.x = constrain(((int)(map(roll, -90, 90, -100, 100))), -100, 100);
  joyReport.y = constrain(((int)(map(pitch, -90, 90, -100, 100))), -100, 100);
  
  Serial.write((uint8_t *)&joyReport, 4);

  delay(100); // do this at approx 10Hz
}

When you fire up your Second Life client (either the official client or a third-party client) go into Preferences -> Move & View -> Other Devices to open the Joystick Configuration window & you should see something like the screenshot beneath (click for full size). Note ‘Arduino Arduino Joystick’ has been recognised – however also note that it is a limitation of the client that it only recognises the first joystick device connected to the computer. Depending on how you have mapped your axes & what you want to control you will have to change the numbers in this window accordingly – with the above sketch 0 is the X axis & 1 is the Y axis (-1 disables a control).


In the end this approach proved to be unsuitable for my purposes, due to the difficulty of mapping readings to discrete virtual world orientations rather than to relative movements from the previous orientation. But it was still interesting to do :)