The Dark Pi Rises

Did you think that the Raspberry Pi will only be limited to benign uses? Did you think that the Raspberry Pi will always bask in the light of day? Well this Raspberry Pi...has embraced the darkness.

Okay I might have overdone it a bit =) no thanks to iMovie's trailer templates! This will be a series of tutorials and code on the construction of Xaver Mk.2. Watch this space! For the next few weeks this article will be a work in progress.

The Architecture

To use the Raspberry Pi in a robot, we need to get the Raspberry Pi mobile first. I used a Tecknet IEP387 USB power pack which provides 7000mAh at 5 volts, which is enough to power the Raspberry Pi for several hours. The power pack has 2 USB power outputs, and I used the first one for the Raspberry Pi, the second one for the powered USB hub using a USB-A to 3.5mm DC jack cable.

I then connected an Arduino to the Raspberry Pi using the USB cable. This enables the Arduino to receive commands from the Raspberry Pi using the serial interface (/dev/ttyACM0), and the Arduino then deals with the hardware such as LEDs and motors. This set-up enables me to keep adding things to be controlled - currently 2 servos, a 5mW red laser diode, a white LED, a 940nm IR LED, as well as the main car motor. This set-up is only limited by the number of available pins on the Arduino (currently 3 motor outputs and 3 digital pins are unused).

Finally, I connected a PS3 eye to the USB hub to serve as the robot's onboard camera. I also removed the IR filter on this camera. This makes colours look a bit strange in daylight, but is essential for the night vision capability of Xaver Mk.2.

Setting up the wifi

First, you need to install the Raspbian image into an SD card. Once the image is loaded on the SD card, it will be recognised on a linux computer as containing 2 partitions. On the 2nd partition, navigate to /etc/network and as root, edit the file 'interfaces'. Put:

auto wlan0
iface wlan0 inet dhcp
    wpa-ssid "ssid"
    wpa-psk "password"

Replacing, of course, the ssid and password with your own network ssid and password. The Raspberry Pi forums and website contains more details on getting wifi set up on the Raspberry Pi. One tip - use a USB wifi dongle which uses a Ralink or Atheros chipset. Unfortunately, the very small wifi dongles (like the Edimax EW-7811un) use a Realtek chipset which is not as capable on linux yet. If you're going for those wifi dongles anyway, I recommend using the Adafruit Occidentalis version of Raspbian, as they get recognized automatically.

Once you start the Raspberry Pi, it should then connect automatically to the access point. You can connect the Raspberry Pi to a monitor to see its IP address, or you can use Nmap and scan for open SSH ports.

So assuming everything went well, log in through SSH:

arthur@macbook $ ssh pi@192.168.1.135
pi@192.168.1.135's password:

Now you have control over your Raspberry Pi without connecting it to a keyboard or monitor!

Setting up the webcam

After some trouble with other webcams, I settled on using the PS3 eye webcam. You can buy used ones from game stores for about £5. One good thing about the PS3 eye is its extremely fast frame rates at low resolution (60-120 fps), and for £5, that's very good value for money!

To use it, let's install some required programs:

sudo apt-get install gstreamer-tools gstreamer0.10-plugins-bad gstreamer0.10-plugins-good v4l-utils

There are several ways to stream the webcam output to the network, but I settled on GStreamer because 1) I couldn't get VLC to work, 2) ffserver, motion and mjpeg consumed more CPU when I tried them. The following command gives me 10fps at 320x240, at only 25-30% CPU usage:

gst-launch -v v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,width=320,height=240,framerate=\(fraction\)30/1 ! queue ! videorate ! video/x-raw-yuv,framerate=10/1 ! jpegenc ! multipartmux ! tcpserversink host=192.168.1.135 port=5000 sync=false

UPDATE: The Raspberry Pi Foundation has updated the firmware to enable H.264 encode on the GPU - I'll be working on getting this working as this could enable phenomenal improvement in the webcam streaming while using *less* CPU!

To open the Raspberry Pi's stream on your desktop, just fire up VLC and click 'File->Open Network Stream' and type in 'tcp://192.168.1.135:5000' in the location bar (replace the IP address with your Raspberry Pi's IP address!).

Now you should have 'live' streaming from your Raspberry Pi!

Using a PS3 controller

In one of my earlier posts, I enabled control of a robotic arm through a PS3 sixaxis controller. To do this, we need to install the qtsixa package on the LAPTOP/DESKTOP we're using to control the robot:

sudo add-apt-repository ppa:falk-t-j/qtsixa
sudo apt-get update
sudo apt-get install qtsixa

This should hopefully pull in the sixad daemon which we need to recognise the PS3 sixaxis as a joystick. Alternatively, you can just use a normal PC joystick.

Follow the instructions on the qtsixa website to connect your PS3 controller to your computer. Finally, you need to install pygame (again, still on the laptop/desktop you're using to control the Raspberry Pi):

sudo apt-get install python-pygame

Setting up ad-hoc wifi

To actually be able to use wi-fi outdoors, I followed debian's documentation on setting up an ad-hoc network:

RASPBERRY PI /etc/network/interfaces:

auto wlan0
iface wlan0 inet static
    address 192.168.1.1
    netmask 255.255.255.0
    gateway 192.168.1.2
    wireless-channel 1
    wireless-essid MYNETWORK
    wireless-mode ad-hoc

On the linux laptop: use gnome network manager to connect to MYNETWORK, but set a manual IP address of 192.168.1.2, with a gateway of 192.168.1.1. Take note that ad-hoc is NOT supported in all of linux wifi drivers. (Ralink 2500 does not support it, neither do most Realtek chips)

All in all, the control architecture of Xaver Mk.2 can be represented by the diagram below:

PS3 + TCP client in python:

#!/usr/bin/env python
 
import pygame
import usb.core
import time
#import serial
 
#DEVICE = '/dev/ttyACM0'
#BAUD = 9600
#ser = serial.Serial(DEVICE, BAUD)
 
pygame.init()
j = pygame.joystick.Joystick(0)
j.init()
 
print 'Initialized Joystick : %s' % j.get_name()
 
threshold = 0.30
throttle_prev = 0
steering_prev = 0
laser_prev = 0
 
import socket               # Import socket module
 
s = socket.socket()         # Create a socket object
host = '192.168.1.135' #socket.gethostname() # Get local machine name
port = 12345                # Reserve a port for your service.
 
print 'Connecting to ', host, port
s.connect((host, port))
 
def sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera):
    byte1 = chr(steering_set+throttle_set)
    byte2 = chr(camera*8+laser_set*4+int(infrared_set)*2+int(light_set))
    s.send(byte1+byte2)
 
try:
    command = (0,0,0)
    lighton = False
    lc = 0
    oldtime = 0
    oldtime2 = 0
    oldtime3 = 0
    oldtime4 = 0
    light_set = False
    infrared_set = False
    while True:
        pygame.event.pump()
        steering = j.get_axis(0)
        throttle = j.get_axis(3)
        light = j.get_button(12)
        laser = j.get_button(11)
        infrared = j.get_button(15)
        camera_left = j.get_button(5)
        camera_right = j.get_button(7)
        #############################################
        throttle_set = int((-throttle*7.5)+7.5)
        steering_set = int((steering*7.5)+7.5)*16
        laser_set = int(laser)
        camera = 0
        if light:
            if (time.time()- oldtime) > 0.3:
                light_set = not light_set
                oldtime = time.time()
                sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
                print 'light: ', light
        if infrared:
            if (time.time()- oldtime2) > 0.3:
                infrared_set = not infrared_set
                oldtime2 = time.time()
                sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
                print 'light: ', light
        if throttle_set != throttle_prev:
            sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
            print 'throttle: ', throttle_set
        if steering_set != steering_prev:
            sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
            print 'steering: ', steering_set
        if laser_set != laser_prev:
            sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
            print 'pew, pew' 
        if camera_left:
            if (time.time()- oldtime3) > 0.05:
                camera = 2
                oldtime3 = time.time()
                sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)
        if camera_right:
            if (time.time()- oldtime4) > 0.05:
                camera = 1
                oldtime4 = time.time()
                sendcommand(steering_set,throttle_set,light_set,infrared_set,laser_set,camera)             	
        throttle_prev = int(throttle_set)
        steering_prev = int(steering_set)
        laser_prev = laser_set
 
except KeyboardInterrupt:
    j.quit()

The TCP server running on Xaver:

#!/usr/bin/python           
# Xaver's TCP server. Runs on the Raspberry Pi.
 
import socket               # Import socket module
s = socket.socket()         # Create a socket object
host = '192.168.1.135' # Get local machine name
port = 12345                # Reserve a port for your service.
 
import serial
 
DEVICE = '/dev/ttyACM0' # the arduino serial interface (use dmesg when connecting)
BAUD = 9600
ser = serial.Serial(DEVICE, BAUD)
 
print 'Server started!'
print 'Waiting for clients...'
 
s.bind((host, port))        # Bind to the port
s.listen(5)                 # Now wait for client connection.
c, addr = s.accept()     # Establish connection with client.
print 'Got connection from', addr
while True:
   msg = c.recv(2) # get 2 bytes from the TCP connection
   ser.write(msg)  # write the 2 bytes to the serial interface

The Arduino sketch:

#include <AFMotor.h>
#include <Servo.h> 
 
Servo steering;
Servo cservo;
AF_DCMotor motor(3, MOTOR12_64KHZ); // create motor #3, 64KHz pwm
int throttle = 0;
int ctlbyte = 0;
int ctlbyte2 = 0;
float tval = 7.0;
float sval = 7.0;
int steering_pos = 96;
int camera_pos = 90;
 
void setup() {
  Serial.begin(9600);
  motor.setSpeed(0);
  motor.run(RELEASE);
  steering.attach(10); 
  steering.write(96);
  cservo.attach(9);
  cservo.write(90);
  pinMode(A5,OUTPUT);
  pinMode(A4,OUTPUT);
  pinMode(A3,OUTPUT);
  digitalWrite(A5,LOW);
  digitalWrite(A4,LOW);
  digitalWrite(A3,LOW);
}
 
void loop() {
  if (tval == 7.0) {
    motor.setSpeed(0);
    motor.run(RELEASE);
  }
  if (sval == 7.0) {
    steering.write(96);
  } 
  if (Serial.available()==2)
  {
    ctlbyte = Serial.read();
    tval = (float) (ctlbyte & 0x0F);
    sval = (float) (ctlbyte & 0xF0);
    sval = sval/16.0;
    //set zero positions
    if (tval == 7.0) {
      motor.setSpeed(0);
      motor.run(RELEASE);
    }
    if (sval == 7.0) {
      steering.write(96);
    }    
    //motor forward
    if (tval > 7.0) {
      throttle = (int) (100.0 + ((tval - 7.0)/8.0)*150.0);
      motor.setSpeed(throttle);
      //Serial.println(throttle);
      motor.run(FORWARD);
    }
    //motor backward
    if (tval < 7.0) {
      throttle = (int) (100.0 + (-(tval-7.0)/7.0)*150.0);
      motor.setSpeed(throttle);
      //Serial.println(throttle);
      motor.run(BACKWARD);      
    }
    //going right
    if (sval > 7.0) {
      steering_pos = (int) 96.0 + ((sval - 7.0)/8.0)*40.0;
      steering.write(steering_pos);
    }
    //going left
    if (sval < 7.0) {
      steering_pos = (int) 96.0 - ((-(sval-7.0)/7.0)*26.0);
      steering.write(steering_pos);    
    }
    ctlbyte2 = Serial.read(); //read 2nd control byte
    if ((ctlbyte2 & 0x01)==1) {
      digitalWrite(A5,HIGH); //light on
    }
    if ((ctlbyte2 & 0x01)==0) {
      digitalWrite(A5,LOW); //light off
    }
    if ((ctlbyte2 & 0x02)==2) {
      digitalWrite(A4,HIGH); //IR on
    }
    if ((ctlbyte2 & 0x02)==0) {
      digitalWrite(A4,LOW); // IR off
    }
    if ((ctlbyte2 & 0x04)==4) {
      digitalWrite(A3,HIGH); //laser on
    }
    if ((ctlbyte2 & 0x04)==0) {
      digitalWrite(A3,LOW); // laser off
    }    
    if ((ctlbyte2 & 0x18)==8) { //turn camera left
      camera_pos = camera_pos + 5;
      cservo.write(camera_pos);
    }
    if ((ctlbyte2 & 0x18)==16) { //turn camera right
      camera_pos = camera_pos - 5;
      cservo.write(camera_pos);      
    }    
  }
}
Tags: 

Comments

My today's visit to raspberrypi.org blog is much more exciting than any of my previous visits. Your Pi + Robotics projects are very interesting. I am curious to know about how does the camera connection and how does Pi + Arduino communication happen...

Good luck,

-Kri

This looks great, and thanks for posting. Any chance you could fill us in on what wifi hardware you're using?

I'll be following your blog! P.S. Looks like the embedded object (below the first still of the Dark Pi) is not working FWIW. I assume the Pi connects to the Arduino via TTL serial? Also, have you looked at the Beagle Bone? Can you compare the Beagle to the Pi?
Thanks!
Ralph - ralphsrobots.com

The video started working, it is a riot! Nice Job.

can you send me some details of the video editing tools you used to create your work, It looks great and i'd like to know how you did it.

Very good work! I have the same idea since one year ago, but i was using a mini-ITX as the raspberry, now got a rasp and i'm remaking the project with it. I really appreciate your work, It will help my project to get ended.
sorry for my bad english!

Nice project! I am actually planning on combining an Arduino and a Raspberry Pi for an autonomous vehicle that I am working on but I am not quite as far along on it as you are with yours. Keep up the good work.

Thanks for the awesome post! I'm using your method for streaming from my PS Eye on the Raspi with the official 2012-08-16 Raspbian distro but it's using 60-70% CPU for me... any ideas so to why? The only thing I've changed is overclocking to 800Mhz.

It's possibly because the v4l settings need to be set. You need v4l-utils and type:

v4l2-ctl --list-formats

Then select 320x240 resolution by typing:

v4l2-ctl --set-fmt-video=width=320,height=240,pixelformat=0

Replace the pixelformat with the correct number (I don't have my pi powered on right now to check the exact value). Hope this helps.

Vivo en Uruguay. Hay algun proveedor al que pueda comprar un Raspberry Pi por internet?

Great little project you've done with the Pi :-)

Was reading over on the Foundation forum that you were now doing this with an adhoc wi-fi network. WOuld you mind giving some details on how you managed to do this? I've been struggling with it for about a month.
-
Mike

Hey, im trying the same streaming with a Microsoft Camera on linux, it just works with the same configuration you have in here, but i have a one second delay between the movement and the streamed. Is there a way to perform this?
Thanks

is there code missing? what code goes on the raspi and what is used on the laptop.
Thanks.

Hi! Thank you for sharing your setup.

Can you please say what USB wifi dongle you are using? And why is there an antenna connected to a white USB dongle?
Also you are using the "EXTREME 1:10" for your rover?
Is this the Adafruit motor shield you refer http://www.adafruit.com/products/81 ?

The wifi dongle doesn't have a brand on it, and I've thrown away the box. All I know is that it uses a Ralink chipset - plenty of other dongles have the same chipset, use those. The TP-Link TL-WN722N also works well.

2nd question: yes.

3rd question: Yes it's a 1:10 radio controlled car.

4th question: yes.

Thanks! :)
I am now testing a PS3 Eye toy webcam.

Hi i would like to set up a drone of my own, I am having trouble with the tcp server for the raspberry pi.
Your help would be greatly appreciated.

Hi, great idea and execution. I have all the parts of my machine figured out but am really struggling writing the tcp stack for the raspberry pi. Any help or a pointer in the right direction would be greatly appreciated.

Cheers, Gavinu

Hi amazing stuff here

i followed your instructions and i open the stream in vlc but i am only getting a green screen. do you have any pointers because im stuck

Which USB hub are you using?

Hey, what kind of latency is there between the ps3 eye and vlc?

Hello,
I am having trouble setting up my wifi. How would I set this up at my school where I need to sign in with a username and a password. How would I set this up using wlan0? It is a WPA-2 network and find an answer online. If you know of a good tutorial, that would be great as well.

Thanks,
Michael