Mirrorshades project update – new camera mounts, IndoorAtlas into Unity

Scroll down for a fun video if you don’t want to read ;)

A long overdue update on Mirrorshades, my project that aims to let you walk around wearing an Oculus Rift using cameras to see your real surroundings, whilst using the IndoorAtlas indoor positioning system to track your position & move you around a Unity environment that you can switch to viewing through the Rift whenever you want.

New camera mounts

I realised from William Steptoe’s Rift-based AR platform (incidentally a much more professionally approached endeavour than mine!) that I had made a glaring error with my camera mount by having the cameras horizontal rather than vertical. The Rift’s 1280×800 display is split vertically into two 640×800 segments, one for each eye, so the area that each camera renders to is actually ‘portrait’ rather than ‘landscape’. So I went back to the 3D printer & made some new mounts.

rift

They’re much simpler, still allow the interoccular/interpupillary distance to be altered & I switched from using metal hex spacers & washers to using rubber washers which both makes toe-in adjustments easier & keeps the sensors closer to the eyes so the ‘eyes on stalks’ feeling of the cameras being physically several inches in front of your eyes should be marginally reduced over the old mount.

back

side

IndoorAtlas

I’ve now got things set up such that position data from an Android device using IndoorAtlas is dumped into a MySQL database & a Unity app with a nice model of the building I work in (obtained from another student – my 3D modelling skills are much more rudimentary!) queries the database for the current location of the device & can then use it to move a camera controller.

As a quick first test & to show things starting to work, I simply scripted a sphere with a camera pinned above it to move instantaneously to each new position value & built the app for Android so I could quickly try it out without having to carry around a laptop. This is obviously a very rudimentary approach – for a proper implementation you would almost certainly want to move the marker/camera smoothly, maybe with some pathfinding &/or extrapolation.

This could of course have been achieved with a single device by integrating the IndoorAtlas code into the Unity app, but by using two devices I could start playing around straight away & as the Rift will most likely be running from a Windows laptop carried in a backpack it was a good test to use a separate device for collecting the position data.

Next steps?

Next comes the fun part – building the app for Windows & walking around with the Rift on, switching between looking at the real world through the cameras & the Unity world when pressing a button!

IndoorAtlas Beta – track phone from another device

As you may have seen on YouTube I’ve recently been experimenting with the beta of IndoorAtlas, a service that allows a smartphone to use its magnetometer to determine its position within a building by measuring fluctuations to the Earth’s magnetic field caused by metal building materials;

I’m not so much interested in using IndoorAtlas to make a location aware smartphone app, but instead in using it to provide indoor positioning to a Unity application that displays to an Oculus Rift. To do this I had to make the positioning data available to devices other than the phone itself; the IndoorAtlas API allows a device to obtain its own position, but not to make this position available to other devices (obviously there are privacy concerns involved here!).

So as a quick hack (I believe the research term is ‘proof of concept’) I modified the example Eclipse project that comes with the IndoorAtlas Android API download to additionally submit the position data that the phone receives to a MySQL database on one of my servers, which can then be accessed by whatever I see fit. I borrowed a lot of code from here.

On the phone, it’s simply a case of editing the onServiceUpdate method to build a HTTP POST (with the relevant data as POST parameters) & to send it to a piece of server side PHP which adds or updates the details for the particular phone to the database. The PHP looks something like this;

<?php

// array for JSON response
$response = array();

// check for required fields in POST data
if (isset($_POST['deviceId']) && isset($_POST['buildingId']) && isset($_POST['levelId']) && isset($_POST['floorplanId']) && isset($_POST['latitude']) && isset($_POST['longitude']) && isset($_POST['x']) && isset($_POST['y'])	&& isset($_POST['i']) && isset($_POST['j']) && isset($_POST['heading']) && isset($_POST['probability']) && isset($_POST['roundtrip']) && isset($_POST['time'])) {

	// extract data from POST into variables
	$deviceId = $_POST['deviceId'];
	$buildingId = $_POST['buildingId'];
	$levelId = $_POST['levelId'];
	$floorplanId = $_POST['floorplanId'];
	$latitude = $_POST['latitude'];
	$longitude = $_POST['longitude'];
    $x = $_POST['x'];
    $y = $_POST['y'];
    $i = $_POST['i'];
    $j = $_POST['j'];
    $heading = $_POST['heading'];
    $probability = $_POST['probability'];
	$roundtrip = $_POST['roundtrip'];
    $time = $_POST['time'];

	// include database connect class
	require_once __DIR__ . '/db_connect.php';

 	// connect to the database
 	$db = new DB_CONNECT();

	// check whether primary key already exists
	$result = mysql_query("SELECT * FROM devicelocations WHERE deviceId = '$deviceId';");
	$arr = mysql_fetch_array($result);

	// If the select statement returns the empty set, sizeof will return 1. Otherwise sizeof will return 28 (14 columns in a row).

	// if the deviceID does not already exist in the database, do an INSERT
	if (sizeof($arr) == 1) {
		$result = mysql_query("INSERT INTO devicelocations (deviceId, buildingId, levelId, floorplanId, latitude, longitude, x, y, i, j, heading, probability, roundtrip, time) VALUES ('$deviceId', '$buildingId', '$levelId', '$floorplanId', '$latitude', '$longitude', '$x', '$y', '$i', '$j', '$heading', '$probability', '$roundtrip', '$time');");
	}
	// if the deviceID does already exist in the database, do an UPDATE
	else {
		$result = mysql_query("UPDATE devicelocations SET buildingId = '$buildingId', levelId = '$levelId', floorplanId = '$floorplanId', latitude = '$latitude', longitude = '$longitude', x = '$x', y = '$y', i = '$i', j = '$j', heading = '$heading', probability = '$probability', roundtrip = '$roundtrip', time = '$time' WHERE deviceId = '$deviceId';");
	}

    // check if row inserted or not
    if ($result) {
        // successfully inserted into database
        $response["success"] = 1;
        $response["message"] = "Entry successfully inserted or updated.";

        // echoing JSON response
        echo json_encode($response);
    } else {
        // failed to insert row
        $response["success"] = 0;
        $response["message"] = "Entry was not successfully inserted or updated.";

        // echoing JSON response
        echo json_encode($response);
    }

} else {
    // required POST field is missing
    $response["success"] = 0;
    $response["message"] = "Required POST field(s) missing";

    // echoing JSON response
    echo json_encode($response);
}

?>

Once the data are in the database, I can access them from pretty much anything I want. As a quick test I put together a simple Web page that displays the floor plan image with a red dot to represent the phone’s last known location as well as dumping the data in text form;

map

The PHP for this page looks something like this (it’s a quick hardcoded hack, but you get the idea);

<?php

echo "<head>";
echo "<meta http-equiv='refresh' content='2.5'>";
echo "</head>";

require_once __DIR__ . '/db_connect.php';

$db = new DB_CONNECT();

$result = mysql_query("SELECT * FROM devicelocations");

$arr = mysql_fetch_array($result);

$deviceId = ($arr[0]);
$buildingId = ($arr[1]);
$levelId = ($arr[2]);
$floorplanId = ($arr[3]);
$latitude = ($arr[4]);
$longitude = ($arr[5]);
$x = ($arr[6]);
$y = ($arr[7]);
$i = ($arr[8]);
$j = ($arr[9]);
$heading = ($arr[10]);
$probability = ($arr[11]);
$roundtrip = ($arr[12]);
$time = ($arr[13]);

echo "<div style='position: relative; left: 0; top: 0;'>";
echo "<img src='map.gif' style='position: relative; top: 0; left: 0;'/>";
echo "<img src='splodge.png' width = 1% style='position: absolute; top: ";
echo ($j)/10;
echo "; left: ";
echo ($i)/10;
echo ";'/>";
echo "</div>";

echo "deviceId: ";
echo $deviceId;
echo "<br/>";
echo "buildingId: ";
echo $buildingId;
echo "<br/>";
echo "levelId: ";
echo $levelId;
echo "<br/>";
echo "floorplanId: ";
echo $floorplanId;
echo "<br/>";
echo "latitude: ";
echo $latitude;
echo "<br/>";
echo "longitude: ";
echo $longitude;
echo "<br/>";
echo "x: ";
echo $x;
echo "<br/>";
echo "y: ";
echo $y;
echo "<br/>";
echo "i: ";
echo $i;
echo "<br/>";
echo "j: ";
echo $j;
echo "<br/>";
echo "heading: ";
echo $heading;
echo "<br/>";
echo "probability: ";
echo $probability;
echo "<br/>";
echo "roundtrip: ";
echo $roundtrip;
echo "<br/>";
echo "time: ";
echo $time;

?>

Next will be to modify a player/camera controller in Unity so that it moves according to the position of the phone!

unity