Augmented reality? Local Positioning System?

It occurred to me the car video would be a good target for augmented reality. I already have overlays, but you could insert location-based data over the video, for example the other cars, pit stops, navigation, maybe even explosions!

The trouble (as with any augmented reality system) is tracking distance, rotation, angle etc so as to place the augmented items correctly on screen. For a while I have been considering the idea of a LPS (local positioning system), like GPS, for accurate location of items in a small area. Fast compass measurements would also be necessary.


I recently attended a talk on lightweight java game library, which is mainly a simple, fast wrapper for OpenGL, so rendering objects is actually pretty easy :) Minecraft uses LWJGL.

Just thoughts, so far...

WiFi IP camera

While I haven't posted for a while, there was some fervent activity for a bit. My hackerspace comrade tjhowse found that there are cheap WiFi cameras you can get with an ARM processor and embedded ucLinux, which are in fact quite hackable. They also have pan-tilt stepper motors, but they have been geared way down so they are incredibly slow. Finally, they have decent IR leds so they can see in the dark; they are designed as controllable IP security cameras after all.

The innocent Wifi camera before being hacked.

The default web UI is quite decent.

I won't go into detail right now but the photo album is here and here are some threads on the topic:

http://groups.google.com/group/hackerspace_brisbane/browse_thread/thread/f5eae6b6d9922e00

http://groups.google.com/group/hackerspace_brisbane/browse_thread/thread/7336980c0f887d25

Added wires for serial connection and power.

Ultimately though, I found that:

• The latency is actually quite low on a non-congested wireless network - as little as 35ms, which is quite adequate. 
• The latency spikes a bit sometimes, not sure if this is congestion or CPU problems on the system
• It is hard to connect up the Arduino over serial reliably, though I have basically solved that. 
• It is great to have clear video without interference and static
• However, the video is too slow. Framerate is uneven and low, and there is a significant delay - no good for what should be a fast-paced reflexive game. Not sure if this is due to CPU power, the capture device, or network problems.

So it was fun hacking it, and in fact I might keep it for data comms. Maybe remove the camera and just run the base board. That will require hacking the firmware, but that's a solved problem.

I was able to run the camera off the car's battery, through the Arduino's regulator! The reg got hot quickly though.

Debug pin with software serial

I have been intending to re-purpose the hardware serial interface on the JeeNode to run IR comms, because being hardware and buffered it won't miss data if it's doing something else other than listening, as it would with any software serial library.

But at the same time, while doing development, I like to have debug output. Until I actually start doing the IR thing, I can still use the serial port for this. But I would like to keep that ready to do at any time. Or I can send them over the RF12, but I would like to reduce my wireless usage as much as possible, and not clutter up my UI's text window. Again, until I have multiple cars going, wireless contention isn't as much of an issue, but I prefer to do things properly the first time as long as it isn't going to be harder than fixing it later.

Then I thought, I could use software serial. It's slower and ties up the CPU, but I don't have a lot of or very frequent debug output, so it should be fine. Furthermore, I can just turn off debug output when it's not useful (mainly, when the car is moving around). I added #define and #if-#endif statements to turn the serial output on or off entirely, in a method like this (from memory):

#define DEBUG_ENABLED 1

void debugln(char[] msg) {
  #if DEBUG_ENABLED
  debugSerial.println(msg);
  #endif
}

So if I turn off DEBUG_ENABLED, then calls in the code such as debugln("Car ready") won't have any impact. And including a library should only take up some memory.

First I tried NewSoftSerial, which I have tested out before and decided was too slow (with Arduino and XBee). This caused my steering servo to freak out at the same interval as the messages being sent, no matter what pin I tried to use for debug. I also didn't see any serial output. I think the freak out was because NSS uses interrupts, which is probably shared with either that servo output (it's probably a PWM pin, I didn't check) or the RF12 code. I don't need or want an interrupt-enabled library because this is going to be used sparingly and only for output.

Side note: it's annoying that every such library demands you give it both an Rx and Tx pin. If I only want to send, why do I have to give it a receive pin?

So I tried out SoftwareSerial instead, which is part of the core Arduino library. It didn't cause any apparent problems with servo or RF12, but it also didn't seem to work at all. No matter what pin I chose, from a digital to an analog, or a few other pins on the JeeNode ISP or other headers which are not used at the moment, nothing came out. I set it to 4800 baud (9600 being the max anyway) but the LEDs didn't even light up. I added Serial.println after the debug and that was still outputting.

In short: is the RF12, Ports, or Servo library interfering with SoftwareSerial?

Update: it turns out you need to set the Tx pin to OUTPUT mode. I would have thought the library would do that, I really would.

So that started working, but then I had a problem with debugging just after sending a status update: the output was scrambled. I guessed that the rf12 library was sending and that was throwing off the serial timings. I asked on the JeeLabs forums if there was a way to fix it, and jcw himself (creator of Jeenode) replied. If you add a rf12_sendWait(1); after the sendStart command, it waits for the send to complete before returning to the program flow. That's a double-edged sword; You can guarantee your timings won't be thrown off by the background sending, but at the same time you delay returning to the program, I'm not sure for how long. If I want to keep the loop very tight, it might be a problem. I could also reduce the size of my sends as much as possible. But so far it seems ok.

Forum post is at http://forum.jeelabs.net/node/259

Now with pan/tilt mount

I thought of a quick and dirty way to test how the pan/tilt mount would work with the new car, without having to build the whole support and everything...

That tape is actually pretty sturdy... but I haven't driven it around much. At all actually. There isn't much room and it's late and the car is noisy. But it should be pretty awesome :)

This new car with its 7.2V battery (which after charging is like 8.5V) also means I can run the camera off the same battery, yay! So I don't need an extra battery holder for that. I will also add a MOSFET so that when I turn on the car, it allows current to the camera - so a single switch to turn everything on/off.

I'll record some video from the car camera when I am able to drive it around.

I also need to update that UI - I have the prototype pretty much done, I just have to do the work to put it into the app.

New car test through laptop

We took the new RC car for a spin with the UI and the camera. Buzz wanted to try out his own camera, which is much more expensive and better than mine, so we mounted it.

Camera mounted to car with additional rails... and tape.

 

That's a true hack job there :) TJ suggested putting rails on it between the back and front shocks. We'll get around to actually cutting them to size, tapping and screwing them, but for now they're zip tied on. Then a plastic light cover is taped between the rods, and the camera, antenna and wires are taped down everywhere :)

You can also see the JeeNode at the front. It turns out to be really easy to control both steering, and throttle with an ESC, as they both just use standard servo control. It actually saves me 2 pins compared to the other car!

View from car camera

 Buzz took it for a spin. He was determined to try to drive it under the cars parked in the space. We taped a small flashlight to the front so we could see.

Unfortunately I didn't record any video footage from this. We were also still getting used to the fast car and the controls were twitchy; with just a slightly too-long keypress, I drove the car into a pole so hard the zip-tied rails slid forward out of the rear zip ties. Buzz got it stuck under a car. I tried to drive it outside which I knew was dangerous if I lost jeenode signal and therefore control. I lost both video and control signals at once momentarily, and Buzz went to get the car. He found it trying to reverse while hard up, and making a bad electronics smell. We're not entirely sure what made the smell, and everything still works, but I think it was probably the ESC - it's the only thing that was a bit warm.

I have been aware of the danger of driving out of signal for some time, because there's nothing stopping the car from continuing if it never gets the stop command. There's no way to detect a JeeNode signal in range except to receive data. It wasn't that much of a concern with the smaller car, and I was just careful. So now I have a really compelling reason, and I've done it - if the car hasn't received a command to go forward, reverse or stop for 250ms, it stops. This also requires a change to the UI to send an update every 150ms, just to keep it going if for example you hold down forward for over 250ms, where before there would be no need to send an update.

Fancy new UI

I've been working on dramatic improvements to the UI - position indicators, more gauges, and building in the support for a few features to come. Tell me if it looks familiar at all ;)

This would of course finally be full screen, with the video behind, and work in widescreen or 4:3.

New car! New control!

Bigger, faster, actually has suspension, proportional steering and throttle, and simplified electronics through an ESC - awesome :)
Also the battery is higher voltage (7.2V) so I should be able to run the camera off the same battery!
However, the canopy is very flimsy so I'll have to create a new mount for the camera + gun.

Oh and what's that behind it?....

It's an old set of steering wheel and pedals that I picked up for $10 at an op shop! It's for a PS2 or Xbox original, but as Xbox is just USB, at Hackerspace tonight I lopped off the end and after a lot of cursing about anti-standard wire colours, got it recognised as a USB device under Windows. Have yet to try Mac. How awesome will it be to drive that car with a proper wheel and pedals!