Raspberry Pi OpenCV Pan & Tilt Face Tracker

Create your own face tracking, pan and tilt camera on the Raspberry Pi!

This tutorial will demonstrate use of the OpenCV (computer vision) library to identify and track faces on the raspberry pi using two servos and a USB webcam. For the interested, I previously covered a more thorough overview of the installation of OpenCV from source here, however, I have found that the apt package is sufficient for all but the most bleeding edge of projects.

This project is based on the OpenCV face tracking example that comes along with the source-based distribution. In short, it performs face detection using haar-like features to analyze the video frame, and locates any faces present within it. In the live video window, identified faces are surrounded with a red rectangle. For additional information about how face detection works as well as the specifics of face detection with OpenCV, I recommend this article by Robin Hewitt.

Using the coordinates of the rectangle vertices, my script calculates the (X,Y) position of the center of the face. If the face is sufficiently on the left side of the screen, the pan servo will progressively rotate leftward, on the right side, rightward. This is likewise performed for the tilt servo as well, if the face is in the upper portion of the screen, it will pan upward, in the lower portion, pan downward. If the face is detected reasonably within the center of the image, no action is performed by the servos. This prevents unnecessary jitter once the camera has locked itself on the face.

Hardware

Parts needed:

  • 512 MB raspberry pi
  • 2x Hobby servos (Turnigy 9g fom Hobby King)
  • Pan & tilt bracket (from Foxtech FPV)
  • USB webcam (Microsoft LifeCam Show from Amazon)
  • Power supply
  • Hook-up wire
  • Raspberry Pi enclosure (from Built to Spec)

Assembly

Connect the red, power lines of the servos to +5v, the black ground lines to GND, and the yellow signal lines to the desired output pins, GPIO pins 22 and 23 in the example. Here is a diagram of the completed circuit  (created with Fritzing):

Raspberry Pi OpenCV Pan & Tilt Face Tracker

 

Software

Get the source.

The first step of this procedure is to install the required libraries and packages using the Raspberry Pi package manager. Open up terminal shell and run:

sudo apt-get update && sudo apt-get install git python-opencv python-all-dev libopencv-dev

This command will pull down all of the required packages (about 215 MB worth) including the git version control system, as well as the OpenCV development headers and Python bindings. The next step is to configure the Raspberry Pi for use with multiple pulse width modulation (PWM) outputs. Normally, the Raspberry Pi has only one channel of PWM output. However, thanks to the efforts of Richard Hirst, eight channels of PWM can be used through the use of this servoblaster kernel driver.The driver creates a device file, /dev/servoblaster to which commands can be sent to control the servos. The commands take the form “=” with servo number representing the desired servo (0-7 in this case) and servo position representing the pulse width in units of 10 µs. For example, to send servo 3 a pulse width of 120 µs: echo 3=120 > /dev/servoblaster To configure the servoblaster on the Raspberry Pi, first pull down the sources from Richard’s Github repo:

git clone https://github.com/richardghirst/PiBits.git

Then change into the servo blaster directory, and install the module:

cd PiBits/ServoBlaster

make install_autostart

This command also sets up the necessary udev rules for accessing the /dev/servoblaster device. Note: using the ’install_autostart ’ command will set up a Raspberry Pi to load the servoblaster kernel module on every boot. If you don’t want this behavior, execute ‘make install’ instead. In either case, the module will not yet be loaded so go ahead and load it into the kernel using modprobe:

sudo modprobe servoblaster

Now that all the prerequisites have been installed and the servo blaster device configured, it’s time to get the actual face tracking and servo moving code. Clone my repo from Github here:

git clone https://github.com/mitchtech/py_servo_facetracker

Now change directory into the created folder, and run the script like this:

cd py_servo_facetracker

python ./facetracker_servo_gpio.py 0

If you have a different servo bracket configuration, the pan and tilt axis may need to be inverted. To do so, invert the sign on the values of panStepSize and tiltStepSize.  Similarly, increasing or decreasing these values will change the sensitivity of the movement, larger numbers corresponding to more degrees moved per face detection frame.

Raspberry Pi Google Talk Robot

Google Talk/Chat/Messenger is normally used by humans to interact with other humans. However, its underlying technology can also be used as a mechanism to implement software robots.  Internet bots, also known as web robots, WWW robots or simply ‘bots’ can also utilize the technology to perform automated functions over the web.  There are many such bots in existence, offering a diverse spectrum of services from jokes ([email protected]) to URL Shortening using bit.ly ([email protected]), even mathematical calculation ([email protected]).  Using such bots is quick and easy to configure, all that must be done is to add the bot as a contact to your messaging account.  Then, whenever you desire data from the service, simply text the command to the bot and it will respond with the respective message.

The Raspi Bot is essentially the same as any other automated Internet robot.  To configure it, it must first have its own e-mail address associated with a Google talk account.  This e-mail address must also be added as a contact with the account that wishes to communicate to the bot.  Then, whenever the script is running on the remote machine, it will log into Google chat and appear as a friend in your contact list.

The software itself is essentially just a Python daemon script that is a wrapper around the XMPP protocol.  When executed, the script will sign in to Google talk using its own username and password.  The Python script is derived from the open source project pygtalkrobot: An open source python gtalk(google talk) bot framework using XMPPPY and PyDNS libraries, that also references the source code of python-jabberbot.

Software

The Raspi Bot requires several additional Python modules for use.  The easiest way to install these is with the python pip package manager. If you don’t have it installed, you can install them using apt-get:

sudo apt-get install python-pip git-core python2.7-dev

Then update the easy_install index:

sudo easy_install -U distribute

And install the GPIO, xmpppy, and pydns modules:

sudo pip install RPi.GPIO xmpppy pydns

Then clone my repo for the Raspi Gtalk robot:

git clone https://github.com/mitchtech/raspi_gtalk_robot.git

Now change into the newly created directory:

cd raspi_gtalk_robot

Finally, you will need  to configure the Raspi Bot’s Gtalk username and password.  This is done by editing the fields BOT_GTALK_USER, BOT_GTALK_PASS, and BOT_ADMIN, on lines 31-33 in the raspiBot.py file.  It is recommended, though not required, to give the Raspi Bot its own Gmail account. Since access to the Raspberry Pi GPIO pins is restricted, the script needs to be run with sudo:

sudo python ./raspiBot.py

This basic sample script supports the following commands:

  • [pinon|pon|on|high] [pin] : turns on the specified GPIO pin
  • [pinoff|poff|off|low] [pin] : turns off the specified GPIO pin
  • [write|w] [pin] [state] : writes specified state to the specified GPIO pin
  • [read|r] [pin]: reads the value of the specified GPIO pin
  • [available|online|busy|dnd|away|idle|out|xa] [arg1] : set gtalk state and status message to specified argument
  • [shell|bash] [arg1] : executes the specified shell command argument after ‘shell’ or ‘bash’

For example, sending the message “pinon 10″ will turn on GPIO pin #10, “read 8″ will read the current state of GPIO pin 8, or “bash ps” to execute the shell command ‘ps’.

Hardware

The video demonstration uses a slide switch connected to GPIO pin 8 and an LED connected to GPIO pin 10.

Use case #1: Home automation

One of the most obvious usages of this technology is for home automation purposes. The Raspi Bot can be accessed anywhere with Google talk, which to my understanding, is nearly every system in existence.  Send the Raspi Bot messages to turn on and off lights and other electrical appliances.

This is also useful to provide immediate notification in the event of intrusion detection. The Raspi Bot can be supplemented with additional security sensors, including infrared motion, and ultrasonic distance sensors. If any pre-programmed sensor violates any predefined condition, you can be immediately notified via message from the Raspi Bot.

Use case #2: Remote shell

The Raspi Bot can be used essentially as a remote shell.  In this configuration, every message sent to the Raspi Bot will be interpreted as a shell command with the output piped back to the user in the form of a response message.  Obviously, this could raise some security concerns. To protect against misuse, the Raspi Bot will only respond to Google chat messages from the Google user designated as the administrator of the bot.  By default, messages from any other user will simply be ignored.

The following is a small subset of the relatively benign commands possible to be run remotely via the Raspi Bot:

  • vmstat – system activity, hardware and information
  • uptime – how long the system has been running
  • w – logged in users and their process activity
  • ps – reports a snapshot of the current processes
  • free – physical and swap memory usage
  • iostat – average CPU load, disk activity

Arguably, disclosure of any amount of information about system can be considered a security issue such as that reported by some of the above tools.  For users more concerned about convenience over security exposure, much more elaborate commands can be run, such as executing additional scripts or accessing private data.

Use case #3: Remote Reboot

Another problem that can be solved by the Raspi Bot is frozen remote machines. We’ve all been there before, attempting to access a remote machine only to find it to be completely non-responsive to any form of remote login.  These cases, we (or maybe a system administrator somewhere) would usually have to make a trip to the physical location of the server and push ‘the big red button’ to reboot the affected machine.  The situation can be eliminated completely by deputizing a Raspi Bot as a remote reboot agent. This can be done by adding bot controlled relay(s) to the power supplies of the machines.  In the event any of the machines controlled by the Raspi Bot becomes non-responsive, simply send the appropriate Google talk message to flip the respective relay, and reboot the affected machine.

Raspberry Pi + PWM RGB LED Strip

This tutorial demonstrates how to easily use a Raspberry Pi to drive 12V RGB LED strips using Pulse Width Modulation (PWM).  Out of the box, the Raspberry Pi has only one GPIO pin that is capable of pulse width modulation (PWM).  However, thanks to the efforts of Richard Hirst and his servoblaster kernel module, standard GPIO pins can be used to perform PWM.

Note: The flashing of the LED strip due to PWM is only noticeable in the uploaded video; in reality, the colors progress smoothly without any visible flashing.

Hardware

Parts needed:

  • Raspberry Pi
  • 3 x TIP120 power transistors
  • RGB LED strip
  • Perfboard/Breadboard
  • Hook-up wire

Assembly

Connect a 12V power supply to the 12V pad on the RGB LED strip, connect the base of each TIP120 power transistor to its respective GPIO pin (pins 18, 23, and 24 in this example), connect the collector of each to its matching pad on the LED strip, and finally, connect the emitters to common ground for both the Raspberry Pi and the 12V power supply. Here is a diagram of the circuit  (created with Fritzing):

Software

Configure servoblaster.

The next step is to configure the servoblaster kernel driver. The driver creates a device file, /dev/servoblaster to which commands can be sent to control the servos. The commands take the form “=” with servo number representing the desired servo (0-7 in this case) and servo position representing the pulse width in units of 10 µs. For example, to send servo 3 a pulse width of 120 µs:

echo 3=120 > /dev/servoblaster

To setup the servo blaster on the Raspberry Pi, we will need to have git installed to pull down the sources. If you don’t have it installed already, open a terminal and run:

sudo apt-get install git-core

Then pull down the sources from Richard’s Github repo:

git clone https://github.com/richardghirst/PiBits.git

Now change into the servo blaster directory:

cd PiBits/ServoBlaster

And compile and install the module:

make install_autostart

This command also sets up the necessary udev rules for accessing the /dev/servoblaster device. Note: using the ’install_autostart ’ command will set up a raspberry pi to load the servo blaster kernel module on every boot. If you don’t want this behavior, execute ‘make install’ instead. In either case, the module will not yet be loaded so go ahead and install it using modprobe:

sudo modprobe servoblaster

Sample code usage.

Now that all the prerequisites have been installed and the servo blaster device configured, on to the actual sample code. The python sample script uses PWM to fade from blue to violet, to red, to yellow, to green, to teal, back to blue.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
#!/usr/bin/env python
import time
import os
STEP = 100
DELAY = 0.5
def pwm(pin, angle):
print servo[ + str(pin) + ][ + str(angle) + ]
cmd = echo + str(pin) + = + str(angle) + > /dev/servoblaster
os.system(cmd)
time.sleep(DELAY)
while True:
for i in range(0, 8):
for j in range(1, 249, STEP):
pwm(i,j)
for i in range(0, 8):
for j in range(249, 1, (STEP*-1)):
pwm(i,j)

Easily connect Raspberry Pi to Gmail, Facebook, Twitter & more!

Easily connect your Raspberry Pi to web services and social networks! This tutorial demonstrates how to painlessly send and receive Gmail on the Raspberry Pi from Python, which in turn, allows you to easily connect it to web services and social networks like Facebook, Twitter, and more! This would normally be well beyond the abilities of most users due to the inherent complexities of programming through social media APis, client/server authentication, etc. However, with the easy-to-use web service Swiss Army knife ifttt (if this then that) anyone with even the most basic programming skills can dramatically expand the capabilities of their Raspberry Pi.

How does this work?

If you are not yet aware, ifttt is a great tool to simplify interaction with many social networks and other web services. It operates with the premise that when some specific action occurs it should perform some other predefined action. The interface is intuitive and very easy to quicky understand. The bridge then is connecting the Raspberry Pi to any of these supported services, which in turn, enables the use of all other services that can respond to that respective trigger. In an earlier  tutorial, I  previously demonstrated how to connect your Raspberry Pi to your Dropbox account using SSHFS. This is another perfectly viable option to establish two way communication with ifttt. However, instead of using email as the primary communication medium, it relies on Dropbox and the filesystem itself. Depending on your application, this may or may not be a better option.

Getting Started

The tutorial uses the most recent Raspbian wheezy image, (2012-08-16-wheezy-raspbian)but should largely generalize for not only other Raspberry Pi distributions, but most other linux distributions as well (especially those derived from Debian, such as Ubuntu and Mint). The majority of the complexity and functionality comes from the feedparser Python module, which should be available essentially anywhere with Python support, even including Windows and OSX distributions too. In any case, begin by setting up the necessary accounts.

Account Setup

As you would undoubtedly expect, the ability to send and receive Gmail is predicated upon having a Gmail account. You are welcome to use your personal Gmail account if you wish, however, I recommend creating a separate account for the Raspberry Pi as it provides the most flexibility. If you would like to expand beyond the capabilities of Gmail on the Raspberry Pi, you will also need an account on ifttt. As with Gmail, you can either share your accounts with the Raspberry Pi, or create a separate account specifically for this purpose. Your decision here might be dependent on which services will be necessary for your own specific application.

Install Packages

Independent of whether you are using one or multiple Gmail or ifttt accounts, you will first need to install some required packages. Open up a terminal on the Raspberry Pi and install the Python development headers and the pip package manager:

sudo apt-get install python-pip python2.7-dev

Next, for pip to work correctly you will need to update to a newer version of distribute using the easy_install utility:

sudo easy_install -U distribute

Once that completes, install the feedparser module with the pip package manager:

sudo pip install feedparser

Now that all of the prerequisites are installed, we can move on to the actual code!

Checking Gmail by the Raspberry Pi

This is an example script that will check the Gmail of the specified user, and display the subject line of all unread e-mails.

#!/usr/bin/env python
import feedparser
USERNAME = [email protected]
PASSWORD = password
response = feedparser.parse(https:// + USERNAME + : + PASSWORD + @mail.google.com/gmail/feed/atom)
unread_count = int(response[feed][fullcount])
for i in range(0,unread_count):
print ( + str((i+1)) + / + str(unread_count) + ) + response[items][i].title
view rawcheck_gmail.py hosted with ❤ by GitHub

Sending Gmail from the Raspberry Pi

This is an example script that can be used to send plain text e-mails with Gmail to the specified mail to e-mail address.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
#!/usr/bin/env python
import smtplib
from email.mime.text import MIMEText
USERNAME = [email protected]
PASSWORD = password
MAILTO = [email protected]
msg = MIMEText(This is the body of the email)
msg[Subject] = The email subject
msg[From] = USERNAME
msg[To] = MAILTO
server = smtplib.SMTP(smtp.gmail.com:587)
server.ehlo_or_helo_if_needed()
server.starttls()
server.ehlo_or_helo_if_needed()
server.login(USERNAME,PASSWORD)
server.sendmail(USERNAME, MAILTO, msg.as_string())
server.quit()
view rawsend_gmail.py hosted with ❤ by GitHub

Raspberry Pi OpenCV Pan & Tilt Face Tracker Diagram

Sending Attachments through Gmail from the Raspberry Pi

This is a more complex example script that can be used to attach files to emails sent from Gmail. It sends an array of attached files to an array of recipients.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
#!/usr/bin/env python
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
import os
USERNAME = [email protected]
PASSWORD = password
def sendMail(to, subject, text, files=[]):
assert type(to)==list
assert type(files)==list
msg = MIMEMultipart()
msg[From] = USERNAME
msg[To] = COMMASPACE.join(to)
msg[Date] = formatdate(localtime=True)
msg[Subject] = subject
msg.attach( MIMEText(text) )
for file in files:
part = MIMEBase(application, octet-stream)
part.set_payload( open(file,rb).read() )
Encoders.encode_base64(part)
part.add_header(Content-Disposition, attachment; filename=”%s
% os.path.basename(file))
msg.attach(part)
server = smtplib.SMTP(smtp.gmail.com:587)
server.ehlo_or_helo_if_needed()
server.starttls()
server.ehlo_or_helo_if_needed()
server.login(USERNAME,PASSWORD)
server.sendmail(USERNAME, to, msg.as_string())
server.quit()
sendMail( [[email protected]],
this is the subject,
this is the body text of the email,
[photo.jpg,text_file.txt] )

Scheduling tasks with cron

Often, we want to run tasks at periodic intervals, like to poll Gmail for example. Cron, the UNIX time-based job scheduler, is an easy way to run regular tasks on the Raspberry Pi (or other *nix). Cron is a daemon (like a Web server) that is used to execute commands or scripts automatically at a specified time and date interval. To use it, open the global crontab (cron table) for editing:

sudo crontab -e

In the text editor, you’ll see a commented out section of text describing how to configure tasks with cron. To add your own task, simply add a line to the end of the file. For example, to run the check_gmail.

 

For more detail: Raspberry Pi OpenCV Pan & Tilt Face Tracker


About The Author

Ibrar Ayyub

I am an experienced technical writer holding a Master's degree in computer science from BZU Multan, Pakistan University. With a background spanning various industries, particularly in home automation and engineering, I have honed my skills in crafting clear and concise content. Proficient in leveraging infographics and diagrams, I strive to simplify complex concepts for readers. My strength lies in thorough research and presenting information in a structured and logical format.

Follow Us:
LinkedinTwitter

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top