User Interface

I decided I should post some of the UI and hopefully some video; this is after all, or should be, a very video and UI-heavy project. Work on the hardware side has left the software side behind.

So here is a screenshot of my current basic UI! Click for the full-size version.

The camera view is shown with EasyCapViewer (mac) or AmCap (Windows). Processing is used to create a basic control and indicator interface between the user and the serial port. 

Let's break down that UI a bit:

The camera is controlled by moving the mouse cursor within this grey box. The mouse position is mapped to degrees for the servos; left is as far as the pan servo will turn left, middle is straight forward, at the bottom is as far as the tilt servo can go down. This is not ideal as the mouse can go out of the box while you're watching the video, and you have no idea of absolute position; I often look down to see where the bonnet of the car is so I can tell where it's pointing!

The 4 squares at the top indicate the movement that should occur; basically they map the WASD keys to forward, back, left and right; they turn yellow when the relevant key is pressed. This was used for debugging.

Next the two V labels indicate voltages sent back by the arduino every second. Vcc is the inbuilt voltage meter of the arduino and Vcar is the voltage of the car battery, max 6V, which is input with a 1/3 voltage divider so that the arduino could even read 9V if I was running it at 3.3V. It cannot read a voltage above Vcc. There can be variation in these voltages near max capacity, but as it is currently well below that they are the same.

There is a text box which allows me to type any command to the arduino; currently it uses a simple string-based command system which is easy to debug but not the most efficient. It might be improved later but works for now.

Finally the ping time; yes, latency! It measures the round-trip time of a timestamped message to be sent, received and processed by the full stack, sent every second. One of the first things I implemented with the string-based command system was an echo response, so this sends a message of "ping <current-time>" where current-time is the number of ms since the program was started. When an echo is received, it compares the time in the message with the current time and there is your latency. The latency bounces between 33 and 66ms at the moment; I have another post to write about that. It does not measure lost packets, but if it has been over a second since it received a ping echo, it says "Ping timeout!!"

As for the video, well, it's not great, but I'm actually pretty impressed given the amount I paid for it all! There is some bands of interference near the top and bottom of the image; this is due to the wireless transmission. Bypassing the wireless makes the image clearer. But there should be a better wireless camera kit on the way.

So this is far from ideal. I want the video fullscreen, with the mouse free to move anywhere, I want the voltages and latency overlaid on the video like a HUD, and I want some graphics to indicate camera angle overlaid as well. There will be more overlays later too.

I met a new guy at Hackerspace who has experience with doing games and video programming and is interested in the project, which is great! He's a Windows guy, which is less great, but we'll figure it out :) So hopefully I will get some of this improved soon.

Need to fix up the camera feed

The biggest current problems I have with Phase 1 are just around video, mainly due to buying cheap shit electronics from China - the camera image itself is surprisingly clear, but the refresh rate is low and you can't see anything when there's vibration (ie driving fast). I have ordered a much better camera and mounting system from hobbyking, but seems they're waiting for stock to come in.

And then there's the very cheap USB video capture devices I bought - they were so cheap I got 4 for $36 including shipping! They're apparently DC60 devices; DC60+ are apparently *completely* different and do work on mac and better in general. That's probably why they were so cheap. They're ok for just checking what the camera is seeing, but the driver support is very low, there are no drivers for mac, so I can't write a program to integrate the video from it. There is only one decent program I've found that lets you see the image, EasyCapViewer, which is a mac port of a Linux driver. Unfortunately, it's only a standalone program, and doesn't create devices I could use in another program. And it's written in some language I don't know which uses .m source files - Objective-C maybe? I would rather write in Python or even Java. I've hacked up a basic UI in Processing but it doesn't integrate with the video at all.

So I'm also looking for better capture devices but everything on ebay is variations of the same chip. Also everything is geared towards either viewing your console game in real time or capturing VHS tapes to DVD, so there aren't general drivers that you could plug into any software. Nothing seems available in USB which is a shame because I wanted to use my laptop so I can move around with it. I will try a PC video card capture just to see if that works better.

Servo control and Arduino PWM outputs

tl;dr Arduino PWM (ie analogWrite) doesn't work for servos. Use the built-in Servo library or one of the many variations and deal with the loss of some analog output pins or having to call a function at least every 50ms.

I had problems with the servos being a bit jerky, or twitchy when not being moved. I'm using the built-in Servo library, but was annoyed that it disabled PWM outputs on some pins - I used these pins for the motor control, and this particular issue wasted several hours debugging. After I discovered this, I was annoyed because I had found a convenient place to plug in motor control and was too lazy to move it.

I tried the SoftwareServo library and found it no better, in fact possibly more twitchy, sometimes not responding at all - but this may have been due to power problems. But I put a LED on the output pin and it seemed to flicker inconsistently, due to it not being a hardware output and dependent on processing loop time, whereas it should have been a smooth or at least consistent flicker.

I thought all this was silly, why were there different libraries for servo control, and both had annoying issues? I could just control the servos directly with the hardware PWM, right?

Well I tried just that, and the servos did very weird and completely wrong things - a single change it degree made it move a little, or maybe 90 degrees.

Turns out that servo motors are not controlled by PWM, or Pulse Width Modulation, at least that which is output by the Arduino (the timing is probably wrong). They are controlled by Pulse Position Modulation (PPM). The wikipedia page on servos is a little misleading on this matter, although it is possible there are servos that are controlled by different kinds of PWM.

All this behavior finally became clear when I found this post on the arduino forums: 

Help with PWM for controlling servos and speed con

So after moving my motor PWM pins, making sure the arduino power was isolated from the servo power, replacing the oversized pan servo with a smaller, faster, lower-power one, and replacing the tilt servo with one that hadn't been put through the first batch of testing where it held up the camera itself, I have a stable camera :)

JeeNode for wireless comms

I just came across this kit, looks pretty cool. I currently use the RBBB Kit, so the JeeNode is just that + an onboard wireless serial device. Would work well in my case, because I already use the RBBB on a breadboard with an offboard XBee as wireless serial.

You'd need the PC end as well to be able to communicate with it. I'd like to know how hard they might be to configure to talk to different devices.

Don't know much about the serial device itself; whether it does spread spectrum, buffering, meshing etc; you do get a lot with the XBees. But this might turn out to still be quite acceptable.

Looks like they are available in Australia at toysdownunder, but the board is out of stock and they don't seem to have the host, which makes it all quite moot.

Xbee communication

I've had some odd problems with the XBee modules I have.

tl;dr I changed them from ZIGBEE (coordinator/end) to ZNET 2.5 (coordinator/end) and it seems to have fixed it.
If you want to know more, read on.

As it happens, when I went to buy them from littlebird electronics, they only had series 2.5 modules. I did read that they are not compatible with series 1, but thought that it was likely that 2.5 would be overall better. As it turns out, series 1 may have been better, at least where one is communicating in a transparent serial way, between just two nodes. Later, the features of 2.5 may come in more useful.

Another bugbear of the 2.5 series is that they don't do I/O line passing, ie the serial control signals like RTS, CTS etc, which means you can't treat them like a full transparent serial interface. Which means you can't set them up to be able to upload arduino code wirelessly. This is not essential as my car will not be inaccessible, but would have been nice. There may be ways to do it by sending a reset command at the right time, then hitting upload at the right time, but meh.

Back to my problem. When I originally configured them, I followed a tutorial, and was delighted to see they worked very well, fast and reliable! Later they started doing odd stuttering things, where the car would just stop responding, then maybe several seconds later suddenly start moving, like it had some of the commands buffered and had just gotten to them. This may be partially due to power issues I had. Then I stopped receiving any data at all from the car, which was annoying as I needed to debug the controls (the car would send back what action it had just performed).

I had, as per the tutorial, configured the modules as "ZIGBEE COORDINATOR AT" and "ZIGBEE ROUTER/END DEVICE AT". What this means is that they use the Zigbee protocol, one is the coordinator and the other is the end device, and they interface using the AT protocol. In a mesh, there can be multiple end devices which can either talk both ways with the coordinator, or use the broadcast address to talk to all at once. For a single peer to peer connection, imitating a wired serial connection, the devices are configured with eachother's high and low serial numbers, so they only talk to eachother. The AT protocol is the way one configured the old dialup modems. The other option is API which I won't go into here.

Another tutorial contradicted this and said I should configure it as "ZNET 2.5 COORDINATOR AT" and "ZNET 2.5 ROUTER/END DEVICE AT". I was dubious that this would fix it, because it had worked earlier, and the whole coordinator/end thing sounded basically the same. Also I had real trouble reconfiguring one of them; it seemed to not respond to attempts to rewrite the firmware. I eventually found a post here which worked for me; without having to wire up a reset switch or something, this was a lot simpler. It boils down to:

1. pull the module out of the usb or breakout board
2. write the firmware with X-CTU
3. it will complain that it can't find the device. when this happens, smoothly reinsert the module (don't miss or bend any pins).
4. it will write the firmware, and possibly complain about not being able to set the AT commands. but the important part is done.
5. re-read the module, and you will need to configure it again as it has had defaults written.

I configured the modules to talk to eachother peer-to-peer again, and it's worked great since!

Here is a page with a lot of useful notes about XBee modules in general:
Xbee Znet and gps (RoboSci Blog)

Onboard Arduino and XBee

The "Arduino" that I'm using in this project is actually an arduino-compatible Really Bare Bones Board (RBBB). This was both cheaper and more appropriate for this project, because I would rather plug it into a regular breadboard to have more pins available. Luckily a breadboard I had already happened to have fit quite neatly into the car!

Since the control would be in the car, I had no use for the hacked handheld controller any more. I had to hack the car's receiver end instead and figure out how I could drive it. It had a number of transistors which appeared to form two variations of an H-bridge, one for the forward/reverse motor, one for steering. It also had a small PCB offshoot which probably did the signal processing. It connected to the main PCB with a number of pins. I made use of an oscilloscope which had been recently loaned to the hackerspace, and mapped out the pins as follows:

If we make the top left pin 1 and the top right 18 (going down and back up the other side), then pins 1, 3, 4 and 10 are of interest. All I have to do is send a 5V signal from the arduino to drive the car that way. You also need to connect up the ground to make them common (or the 5V signal won't be registered). Also, being a H-bridge, you must be careful not to send both forward and reverse signals at once, which shortcircuits around the motor and causes "shoot-through", driving all the current straight to ground and possibly damaging or burning out components.

Unfortunately I didn't get a photo of the car's hacked circuit board before I closed it up to test it, but here are some photos of the mounted wireless circuit. Hmm... these seem to be mirrored. Stupid mac camera software. 

Controller mounted to car body and secured with electrical tape. That's the Xbee module sticking out the side.

Pan-tilt camera

This is fairly straightforward. One servo controls horizontal movement (pan) of the rest of the structure, and the tilt servo controls vertical movement. I put it on the roof of the vehicle.

Hole for the pan servo to hold the structure on top.

I didn't want to cut a huge hole for the big servo, nor have the mechanism mounted high. But the servo's mounting holes were too low. So I built up the rest of the space with polymorph. This stuff is awesome! You grab some of the pellets, throw them in hot water, when they turn transparent they are ready. Fish them out with a spoon and mould it as you like. It cools back to solid white and is a relatively hard plastic. If you mess up or want to change it, and chuck it back in hot water to soften it and you can use it again! If it's stuck to something that might be damaged by hot water, either cut it off or maybe you could use steam.

Polymorph to build up some space, with screw thread

I screwed screws into it while it was still somewhat soft, to give it a thread. I am able to screw and unscrew it quite securely :) It wasn't quite flat on top, but you can file or cut it down and even retain the leftovers! 

Wireless camera and pan/tilt mount, roughly attached with polymorph and blutack.

I stuck the first one together with blutack and polymorph.

The pan servo is unneccessarily huge and slow. Soon to be replaced.

At this time I used separate power for the car, onboard controller, and camera. 9V batteries don't last very long.

Rear and batteries.

I'll get some photos of the new mount. Or maybe skip that entirely and get a better one. 

PC-based wireless control

The first "wireless" control I had was the handheld battery powered one that came with the car. But since I wanted it controlled through the PC, I had two options:

• hack the handheld controller so I could use the existing radio control from the arduino.
  
• hack the car's controller, instruct it from the arduino, and get commands to the car somehow.

The second one required me to have the arduino and xbees, but they hadn't arrived, and it turned out to be dead easy to drive the handheld controller; I just had to pull a line from 5V to ground to imitate a button press. Easier than having to wire a transistor for each button to send the 9V from the battery.

I hacked the original RC controller in order to send the commands from the PC, from the arduino.

The controller plugged into the breadboard.

I also didn't have a wireless camera, but since I had a PC-controlled car, I had to try it out! I blutacked a cheap toy webcam to the car and could drive it about 5m with some USB extensions.

Early days. 

Success! Remote control car with camera!

The Project

I'm starting this build blog to document my progress, as well as the problems I have encountered and their solutions during my project, in order to help others and collaborate.

The project has multiple phases planned, depending on how successful each phase is. The first phase is this:

• PC-based remote control;
• WASD keyboard controls
  
• video on-screen
• mouse camera control

Ideally also:

• full screen video and mouse
• with overlays
  
• battery level and lag overlay
• speed control
• protection logic eg auto shutoff when no signal received.
• smooth video even when travelling fast or rough
  
• upload code changes to arduino remotely
• a single power supply

At this point, all of the minimum features are working, and many of the ideal features! I have speed control (controlled with numbers 1 through 0), the car sends back the battery level every second, there is a latency-measuring method, and these are shown on the control program, although not an overlay. The circuit power (5V) runs off the car batteries which are 6V, but the camera has its own power because it wants 7-9V.

I am aware that it is not an original idea to put a camera on a car, nor to control it from a computer. The idea is a smooth game-style interface, and I have a much greater goal in mind which I don't want to reveal yet :) There is much more planned, I have 4 phases sketched out, and will describe them when I start them ;) I might start the next phase soon, while still solving some of the existing problems.

In building this, I have tried to use mostly off-the-shelf components, such as an existing RC car, rather than building from the ground up. A system designed for this specific purpose might perform better, but I just want to get something up and running. In some cases this is cheaper, or easier, or just makes it easier to reproduce. I don't have enough knowledge to build all the different parts, so I'd rather just try to interface them together.

I have implemented this essentially with an arduino (compatible) mounted on a breadboard, hacking into the existing circuit board on the car to control the speed and direction, using XBee series 2.5 2mW radios for communication. The arduino also controls two servos for the pan-tilt mechanism, which was built first by myself, then improved with help from Buzz from HSBNE. The camera is independent of the circuit, having its own antenna and power source, currently.

Here's my photo album for the car, starting with the original circuit and camera mount.

This project has been built with significant assistance and advice of the members of Hackerspace Brisbane, and built and tested at the space.

I'll post about some of the problems I've had so far, and new stuff as it happens!