iPhone 4 Gyro Control in Unity

iPhone 4 Gyro Control in Unity

 The Write Up.

This idea spawned from seeing an iPhone app called Cameraman for Maya by Wes Mcdermott (www.the3dninja.com). The basic principle is you can use the gyro inside an iPhone 4 to record motion capture data. Really useful if you want to get an authentic shaky cam effect. I’ve been planning some larger projects in my head and figured something like this would be really useful for getting some animation clips for the camera in cutscenes.

Initially I thought this would be really easy in Unity. It has a remote iPhone app and I can just use that. I had a friendly peer-imposed deadline of 24 hours from Prime_31 so I figured why not :).

Unfortunately gyroscope support has only been added to Unity in the very latest version (3.4) and the remote hasn’t been updated to send Gyroscope data to Unity when using it as a remote. So that was out, (although that would also have limited me to having a usable system only on Mac). So I tried something else.

(If your not interested in the background and just want to get it going on your own projects, goto the bottom of the post to get the download links and how-to)

 

OSC?

For the uninitiated OSC stands for open sound control and is a messaging format optimized to be super fast over current networks. It’s primary use is in audio, controlling synthesizers and synchronising hardware with software. However due to its light weightness its really good for real time stuff, I did a quick search for iPhone apps that support OSC and found a BEAUTY. This is part 1 of the puzzle to get this thing running.

 
Control

The app is called control. It’s a single dev’s work by the name of Charlie Roberts (www.charlie-roberts.com/Control/). If it wasn’t for this I wouldn’t have made my 24 hour deadline. It’s basically a customizable interface allowing access to all sensor and touch events and send the raw data with OSC. The great thing about it is the interfaces are in JSON format so writing your own is a piece of cake. Oh yeah, and it’s free! Here’s an example interface for Control:

 

loadedInterfaceName = "template";
interfaceOrientation = "portrait";
 
pages = [[
{
    "name": "refresh",
    "type": "Button",
    "bounds": [.6, .9, .2, .1],
    "startingValue": 0,
    "isLocal": true,
    "mode": "contact",
    "ontouchstart": "interfaceManager.refreshInterface()",
    "stroke": "#aaa",
    "label": "refrsh",
},
]
];

So I made an interface that sends the gyroscope information 100 times a second and also has 4 triggerable buttons on the screens (used for movement in Unity). Once you’ve got all this sending data, it has to be received by something. So another app was required here. (Sending the gyro at 100hz was absolutely fine as long as my wireless signal was strong. Any sort of interference or low signal caused a few drops here and there)

 
Osculator

Another wonderful application that I utilised is Osculator (Mac only unfortunately. I didn’t search for a Windows alternative but OSC is an open format so should be plenty of software out there!). It is available as a demo which functions fully apart from a pause every 10 minutes or so for 20 seconds. This application receives all the OSC data you can throw at it then does whatever you want with it. You can see in the screenshots it provides these great live views for any parameters you send it so I get a nice graph showing me exactly whats happening with the gyroscope. On my osculator setup shown in the screenshot you can see I mapped each gyro output (pitch, roll and yaw) to a different joystick axis and then assigned the 4 on screen buttons to joystick buttons. Now just have to hook them up in the Unity editor and half the puzzle is solved.

 

Osculator Setup

 
Unity

Once we are in Unity we just set up the 3 axes in the Input Manager and write some script to handle it. The scripting is pretty straight forward. Just adding a little code to convert the axes into the transform data and everything starts moving around as it should. Initially when I did this I had the animation creation as a 2 step process where you would make your data whilst playing, then create the animation file and edit it after the game has ended and your back in editor. Now I’ve streamlined it thanks to Unitys AssetDatabase API I can create the animation asset whilst your playing and store it in your Assets folder. (This will obviously only work in the editor still, it wont work if you export. That was never my intention). Anyway that was the final hurdle so now, we are done :) It took me 20 hours in total to reasearch and complete the prototype.

 

 
Trying it Yourself!

If you’d like to give this a go yourself… Well, cool. But there’s a few prerequisites.

  • Control Application for iOS (iTunes Link)
  • Osculator for Mac. (Demo)
  • iOS with Gyro (iPhone 4 or iPad 2)
  • Your computer and iOS device on the same network.
 
Okay I have all that, I still wanna do this!
  • Start Osculator and grab my patch file from here. Load it and leave it running.
  • Start Control on your device: In the app, goto ‘Interfaces‘ click the plus (+). Point it at the URL http://www.quickfingers.net/unitygyro/layout.js (This will download the interface to your phone so it’ll always be there from now on) You should see it in the list as “Unity Controller“. Now goto ‘Destinations‘ and your computer’s IP address should be there, tap it. Then back to ‘Interfaces‘ and tap Unity Controller.
  • Inside Osculator you should see things starting to flash.
  • Now open a new or existing Unity Project
  • Download the GyroRecorder.cs file and attach it to the camera. (Or anything you want to record animation for)
  • Download the InputManager.asset file and overwrite the one in your projects /Library folder. (Be aware this will destroy your Input Managers settings so don’t do this on some pre existing project with complex button mapping setup).
  • Restart Unity.
  • With Control and Osculator running, hit play in Unity and everything should just work!
  • Hit ‘Zero‘ on the device to reset the current gyroscope position as 0,0,0.
  • If your getting weird behaviour and flipping make sure to lock your devices orientation. This uses Core Motion so the accelerometer does come into play. I had weird effects when the orientation was detected. It’s better to just lock it

The rest should be pretty straight forward! Hit the record button to start tracking input. If you stop the engine or press stop it’ll output to a .anim file in the Assets root directory. Okay enjoy!