Noodle: a crowdsourced robot
Noodle is a small device with the I/O of a machine but the thoughts and feelings of a human. You can program it to monitor your physical space and react to changes in the environment with words, images, sounds and decisions.
For example, you could program it to watch your front door, and anytime someone arrives, determine if they look friendly or scary, and either let them in or call for help.
This Instructable contains three main sections: setting up the Raspberry Pi, connecting all the electronics, and creating the enclosure. Depending on what tools and materials you have access to, and how closely you want to recreate Noodle, feel free to only follow the first one or two sections. Noodle’s “brain” will still function without a body, and without all the components, or even in a different body if you’re interested in creating a new enclosure!
List of materials
We had the opportunity of creating Noodle at the Autodesk Pier 9 workshop space, which has pretty much everything you could possibly need to develop something like Noodle. But because the development process lasted over the course of a month in multiple parts of the workshop, it’s difficult to document every single item that we used. Basically, you should be able to recreate Noodle with access to a decent makerspace workshop that has some spare hardware for working with computers, and tools for basic electronics hacking.
Here are some of the parts we used (most of them available from Adafruit):
- Raspberry Pi Model B
- Raspberry Pi Camera
- 5V 1A (1000mA) USB port power supply – UL Listed
- 8GB Card with NOOBS 1.3
- USB cable – A/MicroB
- USB Powered Speakers
- Adafruit Assembled Pi Cobbler Breakout + Cable for Raspberry Pi
- Miniature WiFi (802.11b/g/n) Module: For Raspberry Pi and more
- NTSC/PAL (Television) TFT Display – 2.0″ Diagonal
- USB Battery Pack for Raspberry Pi – 10000mAh – 2 x 5V @ 2A
- RCA (Composite Video, Audio) Cable 6 feet
- Blue Snowflake Microphone
- 4-port USB Hub
sparkfun: 6 – $9.00 – PRT-00437 – USB Male Type A Connector ($1.50 ea.)
- 5V to 12V boost converter
We also used a monitor, mouse, keyboard, and the wireless connection at Pier 9. We had our laptops to work with most of the time. After setting everything up, you can unplug the keyboard, mouse, and screen from the Pi and manipulate it from your laptop. For some of the devices above, they also needed USB cables for power or connectivity. If you are planning on making the enclosure relatively small, be prepared to strip, break, and reconnect the USB cables. It might be good to have extra USB connectors if this is your first time.
For fabrication we used:
- Soldering iron
- Wire stripper
- Wire cutter
- Hot glue gun
- Standard multimeter
- Bench power supply
- Epilog Legend 36EXT Laser Cutter
- Objet Connex500 3d Printer
Step 1: Setting up the raspberry pi
The very first step is to plug some peripherals into the Raspberry Pi and then set up all the software on the Pi. After that, we can tear apart the peripherals and figure out how to get them into the enclosure.
Start by plugging the keyboard, mouse, and external monitor into the Pi. The Pi might need to update itself, so instead of plugging in the wireless adapter, try plugging it into your network with an ethernet cable at this point.
If your SD card doesn’t already have a copy of NOOBS on it, make sure to follow the instructions here before turning on the Pi.
Once you have an SD card with a copy of NOOBS, plug the SD card into the Pi and plug your 5V USB power adapter into the Pi to turn it on. For this project, we’ve selected to use the Raspbian OS. Select it from the NOOBS menu, and the commands will guide you through the rest of the installation process.
When you get to a blue screen that asks you to configure the options with which Raspbian is configured, there are two options you need to set: “ssh” should be enabled, and “camera” should be enabled.
After the installation process is complete, you’ll be greeted with a command line asking you to log in. Use the default username/password of pi/raspberry to log in. You might consider setting noodle to auto-login http://elinux.org/RPi_Debian_Auto_Login If you would like to use a GUI to navigate, type “startx”. But all of the following commands will work just fine from the text-only command line interface.
Another technique for editing and checking on the Pi files from another computer is to use the Pi as an external hard drive. If you follow these instructions.
You can set up the Pi as a Samba server, and then edit and view files from your laptop.
Some of the following steps require you to edit files. To do this you can either use nano on the command line, one of the text editing apps from the GUI, or edit the files on your laptop via the Samba disk.
Step 2: Working with the camera
By enabling camera support during setup, we have access to two tools, “raspivid” and “raspistill”. To test the camera, turn off the Pi (type “sudo poweroff” and unplug it), plug in the camera by lifting the zero-insertion-force socket, making sure the pins from the cable are in contact with the pins on the socket and not backwards, then latch the socket in place and plug the Pi back in.
After booting up and logging in, type “raspistill -o image.jpg”. After the Pi takes a photo, you should be able to type “ls -lh” and see “image.jpg” next to a filesize like 2MB or 3MB.
Step 3: Working with the miniature LCD screen
By default, the current version of Raspbian outputs only to HDMI. In order to allow the Raspberry Pi to output to the composite video (yellow plug), you need to open the file /boot/config.txt and comment out the line reading “hdmi_force_hotplug=1” by adding a “#” to the beginning of the line.
Once that line is commented out, unplug the HDMI output and the next time you restart the Pi it will use the composite output. The composite output might not look perfectly centered, might not fill the screen, or it might be too high resolution, in which case you can check out this tutorial http://learn.adafruit.com/using-a-mini-pal-ntsc-display-with-a-raspberry-pi/configure-and-test showing how to set the resolution and the overscan (padding).
For now, power the LCD with bench power. Later we will get it running off USB power.
Step 4: Working with the speakers
The Pi has a very basic built in pulse width modulation sound output. This means it can’t output high-quality sound the way your laptop computer can, but it can approximate sounds with very quick pulses of varying length. The speakers we used from Adafruit are USB powered, but this power can come from anywhere. If your Pi is out of USB ports at this point, you can either plug in the USB hub or use your laptop’s USB to power the speakers. To play sounds on your Pi you can use command line tools like aplay for .wav files, or omxplayer for far more file f
ormats. The Pi should already have a few sounds on it, type this to play one of them: “omxplayer /usr/share/scratch/Media/Sounds/Effects/WaterDrop.mp3”. If you set up Samba, you can also try copying audio files from your computer to the Pi, and then using omxplayer to play them.
Step 5: Working with the microphone
While the Pi has built in sound output, it doesn’t have any input. So instead of just plugging in a microphone, we have to use a device that’s a microphone with a built-in sound card (the Blue Snowflake). Fortunately, you don’t need to install any extra drivers to get the audio input working. After plugging in the microphone, you can test it with the arecord application. Try entering on the command line “arecord -D plughw:1,0 -d 5 -f S16_LE -c1 -r22050 -t wav out.wav”. This will record 5 seconds of audio from the microphone at 16 bit resolution and 22,050 Hz sample rate and save it to “out.wav”. Then try “omxplayer out.wav” to listen to the recorded audio. If you are using a different microphone or have a different hardware setup you might need to change the “plughw” options. The code autodetects your microphone from the USB device name.
Step 6: Working with wifi
Setting up the wifi with the Pi is one of the things that’s easiest using a GUI rather than by editing the config file directly. Adafruit has instructions for both here:
The only difficulty we ran into was setting the encryption. The default encryption wasn’t the correct one for our network, so we opened our Network Preferences in OS X on our laptop, and looked for more information about our network under the 802.1X Tab after clicking on the “Advanced” Wi-Fi button. The correct setting for the Pi was an authentication type that was named similarly to the authentication type listed on OS X.
If you are using an unsecured network, or a network without 802.1X authentication, the tutorial above should be enough.
Step 7: Installing node.js
The code for Noodle is built with two parts: a Node.js server that handles task management and the user interface, and Python functions that interface to the Pi hardware. The Pi already includes Python, but we need to install Node.
We followed some of the instructions from here:
First we entered the following series of commands at the command line to download Node, decompress the archive, create a directory for it, and move it to that directory:
tar xvzf node-v0.10.22-linux-arm-pi.tar.gz
sudo mkdir /opt/node
sudo cp -r node-v0.10.22-linux-arm-pi/* /opt/node
Finally, we edit the file “/etc/profile” by adding the following two lines before the “export PATH” line:
This makes it possible for the command line to know where to find node when we type “node”.
The tutorial above has some good advice about using a static IP address from your Pi also. Here’s a tutorial that’s only about using a static IP:
This is helpful because it means the Pi will have the same address when you turn it off and turn it back on. Otherwise, it might receive a different address from your router. Another approach instead of using a static IP is to set up the “raspberrypi.local” address, which is much easier to remember than an IP address:
Not all networks support the .local domain. For example, when we were working at Pier 9 we couldn’t access the Pi using “raspberrypi.local”.
Step 8: Installing Noodle and dependencies
Fortunately git ships with Raspbian, so we can grab all the code from GitHub very easily.
1. Get the code from github:
git clone [email protected]:lmccart/noodle.git
2. Install node dependencies:
3. Install other dependencies with pip and apt-get:
sudo apt-get update
python packages sudo apt-get install python-pip
sudo pip install boto (amazon web services)
sudo pip install -U socketIO-client (https://pypi.python.org/pypi/socketIO-client)
sudo apt-get install python-pyaudio
4. Install espeak (used for generating speech from text):
wget https://pypi.python.org/packages/source/p/pyttsx/pyttsx-1.1.tar.gz gunzip pyttsx-1.1.tar.gz
tar -xf pyttsx-1.1.tar
sudo python setup.py install
sudo apt-get install espeak
Step 9: Connecting with Amazon Mechanical Turk credentials
1. Sign up with Amazon Mechanical Turk as a requester:
Note that it takes ~24 hours for your account to get approved.
2. In addition to a requester account, you will need an Amazon Web Services account. Click the “Developer” tab, then click “Create AWS Account”.
3. Log into the panel http://aws.amazon.com/console/ and set up your access keys. In the upper right corner, click on your name, then “Security Credentials”
4. Choose “Access Keys”, then “Create New Access Key”. Download the file (this contains your Secret Access Key) and note the Access Key ID in the console. You will need both of these to login to noodle.
5. Choose “Services” in the upper left, and “S3” and make sure it’s activated.
For more detail: Noodle: a crowdsourced robot