Create your own face tracking, pan and tilt camera on the Raspberry Pi!
This tutorial will demonstrate use of the OpenCV (computer vision) library to identify and track faces on the raspberry pi using two servos and a USB webcam. For the interested, I previously covered a more thorough overview of the installation of OpenCV from source here, however, I have found that the apt package is sufficient for all but the most bleeding edge of projects.
This project is based on the OpenCV face tracking example that comes along with the source-based distribution. In short, it performs face detection using haar-like features to analyze the video frame, and locates any faces present within it. In the live video window, identified faces are surrounded with a red rectangle. For additional information about how face detection works as well as the specifics of face detection with OpenCV, I recommend this article by Robin Hewitt.
Using the coordinates of the rectangle vertices, my script calculates the (X,Y) position of the center of the face. If the face is sufficiently on the left side of the screen, the pan servo will progressively rotate leftward, on the right side, rightward. This is likewise performed for the tilt servo as well, if the face is in the upper portion of the screen, it will pan upward, in the lower portion, pan downward. If the face is detected reasonably within the center of the image, no action is performed by the servos. This prevents unnecessary jitter once the camera has locked itself on the face.
Hardware
Parts needed:
- 512 MB raspberry pi
- 2x Hobby servos (Turnigy 9g fom Hobby King)
- Pan & tilt bracket (from Foxtech FPV)
- USB webcam (Microsoft LifeCam Show from Amazon)
- Power supply
- Hook-up wire
- Raspberry Pi enclosure (from Built to Spec)
Assembly
Connect the red, power lines of the servos to +5v, the black ground lines to GND, and the yellow signal lines to the desired output pins, GPIO pins 22 and 23 in the example. Here is a diagram of the completed circuit (created with Fritzing):
Software
Get the source.
The first step of this procedure is to install the required libraries and packages using the Raspberry Pi package manager. Open up terminal shell and run:
sudo apt-get update && sudo apt-get install git python-opencv python-all-dev libopencv-dev
This command will pull down all of the required packages (about 215 MB worth) including the git version control system, as well as the OpenCV development headers and Python bindings. The next step is to configure the Raspberry Pi for use with multiple pulse width modulation (PWM) outputs. Normally, the Raspberry Pi has only one channel of PWM output. However, thanks to the efforts of Richard Hirst, eight channels of PWM can be used through the use of this servoblaster kernel driver.The driver creates a device file, /dev/servoblaster to which commands can be sent to control the servos. The commands take the form โ=โ with servo number representing the desired servo (0-7 in this case) and servo position representing the pulse width in units of 10 ยตs. For example, to send servo 3 a pulse width of 120 ยตs: echo 3=120 > /dev/servoblaster To configure the servoblaster on the Raspberry Pi, first pull down the sources from Richardโs Github repo:
git clone https://github.com/richardghirst/PiBits.git
Then change into the servo blaster directory, and install the module:
cd PiBits/ServoBlaster
make install_autostart
This command also sets up the necessary udev rules for accessing the /dev/servoblaster device. Note: using the โinstall_autostart โ command will set up a Raspberry Pi to load the servoblaster kernel module on every boot. If you donโt want this behavior, execute โmake installโ instead. In either case, the module will not yet be loaded so go ahead and load it into the kernel using modprobe:
sudo modprobe servoblaster
Now that all the prerequisites have been installed and the servo blaster device configured, itโs time to get the actual face tracking and servo moving code. Clone my repo from Github here:
git clone https://github.com/mitchtech/py_servo_facetracker
Now change directory into the created folder, and run the script like this:
cd py_servo_facetracker
python ./facetracker_servo_gpio.py 0
If you have a different servo bracket configuration, the pan and tilt axis may need to be inverted. To do so, invert the sign on the values of panStepSize and tiltStepSize. Similarly, increasing or decreasing these values will change the sensitivity of the movement, larger numbers corresponding to more degrees moved per face detection frame.
Raspberry Pi Google Talk Robot
Google Talk/Chat/Messenger is normally used by humans to interact with other humans. However, its underlying technology can also be used as a mechanism to implement software robots. Internet bots, also known as web robots, WWW robots or simply โbotsโ can also utilize the technology to perform automated functions over the web. There are many such bots in existence, offering a diverse spectrum of services from jokes (jokes@askme.im) to URL Shortening using bit.ly (url@askme.im), even mathematical calculation (math@bot.im). Using such bots is quick and easy to configure, all that must be done is to add the bot as a contact to your messaging account. Then, whenever you desire data from the service, simply text the command to the bot and it will respond with the respective message.
The Raspi Bot is essentially the same as any other automated Internet robot. To configure it, it must first have its own e-mail address associated with a Google talk account. This e-mail address must also be added as a contact with the account that wishes to communicate to the bot. Then, whenever the script is running on the remote machine, it will log into Google chat and appear as a friend in your contact list.
The software itself is essentially just a Python daemon script that is a wrapper around the XMPP protocol. When executed, the script will sign in to Google talk using its own username and password. The Python script is derived from the open source project pygtalkrobot: An open source python gtalk(google talk) bot framework using XMPPPY and PyDNS libraries, that also references the source code of python-jabberbot.
Software
The Raspi Bot requires several additional Python modules for use. The easiest way to install these is with the python pip package manager. If you donโt have it installed, you can install them using apt-get:
sudo apt-get install python-pip git-core python2.7-dev
Then update the easy_install index:
sudo easy_install -U distribute
And install the GPIO, xmpppy, and pydns modules:
sudo pip install RPi.GPIO xmpppy pydns
Then clone my repo for the Raspi Gtalk robot:
git clone https://github.com/mitchtech/raspi_gtalk_robot.git
Now change into the newly created directory:
cd raspi_gtalk_robot
Finally, you will need to configure the Raspi Botโs Gtalk username and password. This is done by editing the fields BOT_GTALK_USER, BOT_GTALK_PASS, and BOT_ADMIN, on lines 31-33 in the raspiBot.py file. It is recommended, though not required, to give the Raspi Bot its own Gmail account. Since access to the Raspberry Pi GPIO pins is restricted, the script needs to be run with sudo:
sudo python ./raspiBot.py
This basic sample script supports the following commands:
- [pinon|pon|on|high] [pin] : turns on the specified GPIO pin
- [pinoff|poff|off|low] [pin] : turns off the specified GPIO pin
- [write|w] [pin] [state] : writes specified state to the specified GPIO pin
- [read|r] [pin]: reads the value of the specified GPIO pin
- [available|online|busy|dnd|away|idle|out|xa] [arg1] : set gtalk state and status message to specified argument
- [shell|bash] [arg1] : executes the specified shell command argument after โshellโ or โbashโ
For example, sending the message โpinon 10โณ will turn on GPIO pin #10, โread 8โณ will read the current state of GPIO pin 8, or โbash psโ to execute the shell command โpsโ.
Hardware
The video demonstration uses a slide switch connected to GPIO pin 8 and an LED connected to GPIO pin 10.
Use case #1: Home automation
One of the most obvious usages of this technology is for home automation purposes. The Raspi Bot can be accessed anywhere with Google talk, which to my understanding, is nearly every system in existence. Send the Raspi Bot messages to turn on and off lights and other electrical appliances.
This is also useful to provide immediate notification in the event of intrusion detection. The Raspi Bot can be supplemented with additional security sensors, including infrared motion, and ultrasonic distance sensors. If any pre-programmed sensor violates any predefined condition, you can be immediately notified via message from the Raspi Bot.
Use case #2: Remote shell
The Raspi Bot can be used essentially as a remote shell. In this configuration, every message sent to the Raspi Bot will be interpreted as a shell command with the output piped back to the user in the form of a response message. Obviously, this could raise some security concerns. To protect against misuse, the Raspi Bot will only respond to Google chat messages from the Google user designated as the administrator of the bot. By default, messages from any other user will simply be ignored.
The following is a small subset of the relatively benign commands possible to be run remotely via the Raspi Bot:
- vmstat โ system activity, hardware and information
- uptime โ how long the system has been running
- w โ logged in users and their process activity
- ps โ reports a snapshot of the current processes
- free โ physical and swap memory usage
- iostat โ average CPU load, disk activity
Arguably, disclosure of any amount of information about system can be considered a security issue such as that reported by some of the above tools. For users more concerned about convenience over security exposure, much more elaborate commands can be run, such as executing additional scripts or accessing private data.
Use case #3: Remote Reboot
Another problem that can be solved by the Raspi Bot is frozen remote machines. Weโve all been there before, attempting to access a remote machine only to find it to be completely non-responsive to any form of remote login. These cases, we (or maybe a system administrator somewhere) would usually have to make a trip to the physical location of the server and push โthe big red buttonโ to reboot the affected machine. The situation can be eliminated completely by deputizing a Raspi Bot as a remote reboot agent. This can be done by adding bot controlled relay(s) to the power supplies of the machines. In the event any of the machines controlled by the Raspi Bot becomes non-responsive, simply send the appropriate Google talk message to flip the respective relay, and reboot the affected machine.
Raspberry Pi + PWM RGB LED Strip
This tutorial demonstrates how to easily use a Raspberry Pi to drive 12V RGB LED strips using Pulse Width Modulation (PWM). Out of the box, the Raspberry Pi has only one GPIO pin that is capable of pulse width modulation (PWM). However, thanks to the efforts of Richard Hirst and his servoblaster kernel module, standard GPIO pins can be used to perform PWM.
Note: The flashing of the LED strip due to PWM is only noticeable in the uploaded video; in reality, the colors progress smoothly without any visible flashing.
Hardware
Parts needed:
- Raspberry Pi
- 3 x TIP120 power transistors
- RGB LED strip
- Perfboard/Breadboard
- Hook-up wire
Assembly
Connect a 12V power supply to the 12V pad on the RGB LED strip, connect the base of each TIP120 power transistor to its respective GPIO pin (pins 18, 23, and 24 in this example), connect the collector of each to its matching pad on the LED strip, and finally, connect the emitters to common ground for both the Raspberry Pi and the 12V power supply. Here is a diagram of the circuit (created with Fritzing):
Software
Configure servoblaster.
The next step is to configure the servoblaster kernel driver. The driver creates a device file, /dev/servoblaster to which commands can be sent to control the servos. The commands take the form โ=โ with servo number representing the desired servo (0-7 in this case) and servo position representing the pulse width in units of 10 ยตs. For example, to send servo 3 a pulse width of 120 ยตs:
echo 3=120 > /dev/servoblaster
To setup the servo blaster on the Raspberry Pi, we will need to have git installed to pull down the sources. If you donโt have it installed already, open a terminal and run:
sudo apt-get install git-core
Then pull down the sources from Richardโs Github repo:
git clone https://github.com/richardghirst/PiBits.git
Now change into the servo blaster directory:
cd PiBits/ServoBlaster
And compile and install the module:
make install_autostart
This command also sets up the necessary udev rules for accessing the /dev/servoblaster device. Note: using the โinstall_autostart โ command will set up a raspberry pi to load the servo blaster kernel module on every boot. If you donโt want this behavior, execute โmake installโ instead. In either case, the module will not yet be loaded so go ahead and install it using modprobe:
sudo modprobe servoblaster
Sample code usage.
Now that all the prerequisites have been installed and the servo blaster device configured, on to the actual sample code. The python sample script uses PWM to fade from blue to violet, to red, to yellow, to green, to teal, back to blue.
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
#!/usr/bin/env python
import time
import os
STEP = 100
DELAY = 0.5
def pwm(pin, angle):
print โservo[โ + str(pin) + โ][โ + str(angle) + โ]โ
cmd = โecho โ + str(pin) + โ=โ + str(angle) + โ > /dev/servoblasterโ
os.system(cmd)
time.sleep(DELAY)
while True:
for i in range(0, 8):
for j in range(1, 249, STEP):
pwm(i,j)
for i in range(0, 8):
for j in range(249, 1, (STEP*-1)):
pwm(i,j)
|
Easily connect Raspberry Pi to Gmail, Facebook, Twitter & more!
Easily connect your Raspberry Pi to web services and social networks! This tutorial demonstrates how to painlessly send and receive Gmail on the Raspberry Pi from Python, which in turn, allows you to easily connect it to web services and social networks like Facebook, Twitter, and more! This would normally be well beyond the abilities of most users due to the inherent complexities of programming through social media APis, client/server authentication, etc. However, with the easy-to-use web service Swiss Army knife ifttt (if this then that) anyone with even the most basic programming skills can dramatically expand the capabilities of their Raspberry Pi.
How does this work?
If you are not yet aware, ifttt is a great tool to simplify interaction with many social networks and other web services. It operates with the premise that when some specific action occurs it should perform some other predefined action. The interface is intuitive and very easy to quicky understand. The bridge then is connecting the Raspberry Pi to any of these supported services, which in turn, enables the use of all other services that can respond to that respective trigger. In an earlier tutorial, I previously demonstrated how to connect your Raspberry Pi to your Dropbox account using SSHFS. This is another perfectly viable option to establish two way communication with ifttt. However, instead of using email as the primary communication medium, it relies on Dropbox and the filesystem itself. Depending on your application, this may or may not be a better option.
Getting Started
The tutorial uses the most recent Raspbian wheezy image, (2012-08-16-wheezy-raspbian)but should largely generalize for not only other Raspberry Pi distributions, but most other linux distributions as well (especially those derived from Debian, such as Ubuntu and Mint). The majority of the complexity and functionality comes from the feedparser Python module, which should be available essentially anywhere with Python support, even including Windows and OSX distributions too. In any case, begin by setting up the necessary accounts.
Account Setup
As you would undoubtedly expect, the ability to send and receive Gmail is predicated upon having a Gmail account. You are welcome to use your personal Gmail account if you wish, however, I recommend creating a separate account for the Raspberry Pi as it provides the most flexibility. If you would like to expand beyond the capabilities of Gmail on the Raspberry Pi, you will also need an account on ifttt. As with Gmail, you can either share your accounts with the Raspberry Pi, or create a separate account specifically for this purpose. Your decision here might be dependent on which services will be necessary for your own specific application.
Install Packages
Independent of whether you are using one or multiple Gmail or ifttt accounts, you will first need to install some required packages. Open up a terminal on the Raspberry Pi and install the Python development headers and the pip package manager:
sudo apt-get install python-pip python2.7-dev
Next, for pip to work correctly you will need to update to a newer version of distribute using the easy_install utility:
sudo easy_install -U distribute
Once that completes, install the feedparser module with the pip package manager:
sudo pip install feedparser
Now that all of the prerequisites are installed, we can move on to the actual code!
Checking Gmail by the Raspberry Pi
This is an example script that will check the Gmail of the specified user, and display the subject line of all unread e-mails.
|
#!/usr/bin/env python
import feedparser
USERNAME = โusername@gmail.comโ
PASSWORD = โpasswordโ
response = feedparser.parse(โhttps://โ + USERNAME + โ:โ + PASSWORD + โ@mail.google.com/gmail/feed/atomโ)
unread_count = int(response[โfeedโ][โfullcountโ])
for i in range(0,unread_count):
print โ(โ + str((i+1)) + โ/โ + str(unread_count) + โ) โ + response[โitemsโ][i].title
|
Sending Gmail from the Raspberry Pi
This is an example script that can be used to send plain text e-mails with Gmail to the specified mail to e-mail address.
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
#!/usr/bin/env python
import smtplib
from email.mime.text import MIMEText
USERNAME = โusername@gmail.comโ
PASSWORD = โpasswordโ
MAILTO = โmailto@gmail.comโ
msg = MIMEText(โThis is the body of the emailโ)
msg[โSubjectโ] = โThe email subjectโ
msg[โFromโ] = USERNAME
msg[โToโ] = MAILTO
server = smtplib.SMTP(โsmtp.gmail.com:587โ)
server.ehlo_or_helo_if_needed()
server.starttls()
server.ehlo_or_helo_if_needed()
server.login(USERNAME,PASSWORD)
server.sendmail(USERNAME, MAILTO, msg.as_string())
server.quit()
|
Sending Attachments through Gmail from the Raspberry Pi
This is a more complex example script that can be used to attach files to emails sent from Gmail. It sends an array of attached files to an array of recipients.
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
#!/usr/bin/env python
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.Utils import COMMASPACE, formatdate
from email import Encoders
import os
USERNAME = โusername@gmail.comโ
PASSWORD = โpasswordโ
def sendMail(to, subject, text, files=[]):
assert type(to)==list
assert type(files)==list
msg = MIMEMultipart()
msg[โFromโ] = USERNAME
msg[โToโ] = COMMASPACE.join(to)
msg[โDateโ] = formatdate(localtime=True)
msg[โSubjectโ] = subject
msg.attach( MIMEText(text) )
for file in files:
part = MIMEBase(โapplicationโ, โoctet-streamโ)
part.set_payload( open(file,โrbโ).read() )
Encoders.encode_base64(part)
part.add_header(โContent-Dispositionโ, โattachment; filename=โ%sโโ
% os.path.basename(file))
msg.attach(part)
server = smtplib.SMTP(โsmtp.gmail.com:587โ)
server.ehlo_or_helo_if_needed()
server.starttls()
server.ehlo_or_helo_if_needed()
server.login(USERNAME,PASSWORD)
server.sendmail(USERNAME, to, msg.as_string())
server.quit()
sendMail( [โmailto@gmail.comโ],
โthis is the subjectโ,
โthis is the body text of the emailโ,
[โphoto.jpgโ,โtext_file.txtโ] )
|
Scheduling tasks with cron
Often, we want to run tasks at periodic intervals, like to poll Gmail for example. Cron, the UNIX time-based job scheduler, is an easy way to run regular tasks on the Raspberry Pi (or other *nix). Cron is a daemon (like a Web server) that is used to execute commands or scripts automatically at a specified time and date interval. To use it, open the global crontab (cron table) for editing:
sudo crontab -e
In the text editor, youโll see a commented out section of text describing how to configure tasks with cron. To add your own task, simply add a line to the end of the file. For example, to run the check_gmail.
For more detail: Raspberry Pi OpenCV Pan & Tilt Face Tracker
