Introduction
With the power of Raspberry Pi, Python, avrdude, a custom HAT with cable adapter, we developed our own spin on AVR programming that turned out pretty darn robust. In addition to creating a stand alone solution, it was a great experiment to learn more about running a Raspberry Pi “headless”, auto-launching python modules on bootup, file management, file parsing, general GPIO control, using the built-in SPI hardware on the Raspberry Pi, and even some logic level conversion.
The commit history tells a lot of this story, but this tutorial fills in some of the gaps and includes more general information that is helpful outside of our specific application.
Background
Over the past 10+ years, we have used a programmer called the AVR ISP MKII in SparkFun Production. It has served us well, but due to a couple pain points and finally an EOL notice, it was time to upgrade.
At first glance, one might think, “okay, what’s the latest programmer from ATMEL? Can’t we just swap them out for the old ones?” With our current programming methods, it turned out to be a bit more complicated than that. Also, we do a lot of AVR programming at SparkFun spread across 50+ products, so the problem was actually a pretty big one that needed a reliable and long term solution.
We use batch files a lot in production. If there is an executable GUI that we have to run to program an IC on a product, then we immediately try to find a command line version of that program and automate as much as possible.
With the MKII programmers, our two available options were: Use the GUI (AVR Studio), or do it all in command line. Once you know your commands, then you can put those into a batch file, and a technician can simply double click on a batch file to begin programming – wahoo!! This is how we’ve done it forever.
(Oldschool Memory Lane Side Note) Well, actually, right as I came on board in 2007, we were manually cutting and pasting the commands from text files into our command line window and reading all of the output to find “Flash verified.” Dang, what a pain. I remember thinking, “There’s got to be a better way!”. Then I went batch file crazy, and three years later did some pretty wonky DIY ganged programming. Look at this beast:
So all this to say: The problem was bigger than we thought. At the time, each of our programming procedures for AVR chips used a custom batch file that called STK500.exe, that was buried deep within the AVR studio file structure.
After exploring a few available off the shelf products, we quickly fell victim to the classic engineer’s curse and thought, “Hey, I could design this better myself!” As it usually always does, this sort of adventure became a long journey.
A stand-alone solution for programming AVR chips has always been a dream of mine. If there was an off the shelf solution (similar to the PICkit3 for PICs), then we probably would have migrated sooner. But there wasn’t, so we continued to use the MKIIs. With the day to day stuff and lots of new products coming out, it’s tough to make time for these sorts of projects. And so it was this solution (known as the Pi_Grammer around here) that has been a project of mine for almost two years now.
Although, it’s not really correct to call it “our own” because it truly builds upon many open source tools that were already available. This is our spin on how to create a stand-alone AVR programmer. So in true open source fashion, here is a tutorial to share what we learned along the way!
Pain Points w/ the AVR ISP MKII
Overall, the MKII programmers have done quite well, considering how long and how much we’ve used them. We are careful with our equipment, but after programming tens of thousands of ICs, you have to expect some wonkiness both in the hardware and the software.
My team and I are known as Quality Control here at SparkFun. One of our responsibilities is to design, build, document, and maintain the testing equipment for production. This not only includes the test jigs that we design, but also includes making sure that their windows desktop machines are working and have up to date (or hacked and working) drivers for all our testing needs.
If production sees any issues at all, they call us up, and we try and solve the issue at that moment. It’s usually pretty time sensitive, because we run such a lean ship down in production. Whatever build is held up, is probably going to go out of stock in the next few hours, and so it’s very important that we get up and programming immediately!
Back in the day (2008-ish), this was an hourly occurrence. When it was just me as the only QC person in production, I remember production issue response would eat up most of my day. A day with only one issue was considered a good one! Now we probably only see a couple issues a month.
So knowing this, it is clear to see that when any problem happens in production, it causes QC much pain.
The main pain points were as follows:
- Drivers – Windows updates conflicting with device drivers. All too often, we’d see the following message pop up on our programming batch files:
- The cause for this was usually the fact that Windows would do an auto update over the weekend, and then a tech comes in on Monday morning to see this failure in their batch file. UGH!As a side note, the last workaround that allowed us to use MKIIs on Windows 7 was a great little tool called Zadig. I recommend checking it out if you need to force your computer to use the darn driver you exactly want it to:
- AVR Studio Versions – New versions of AVR Studio conflicting with older versions. We started using the Atmel ICE for some programming of SAMD21 chips, and that needs the latest AVR Studio 6 or 7 (which comes with a new program called “ATPROGRAM”) All our old batch files call “STK500.exe” so that was an issue. It became a dance on each technician’s computer as to which version they had installed, in which order they were installed, and which “actual” un-install happened correctly. No fun.
- Ribbon Cables Wear and Tear – The ribbon cables failing to make a connection after too many cycles. Ribbon cables work great for 20, 30, 50? cycles, but after so long, those solder-less razor-blade connections get loose and no longer connect.
- Internal Servers Down – This last one was pretty rare, but occasionally our internal servers would go down, and the batch files wouldn’t run because there was no access to the network hosted hex file.
Solution: Stand-Alone AVR Programmers
How do we eliminate all of these problems for good? Make the new solution in-house and completely stand alone! No more auto updates!
First Attempt – Chip to Chip Programming
My first approach was to use another AVR chip to do the programming. It had been done before by a few others, and documented quite well.
I found that for some of our combined hex files, this wasn’t going to fit in the flash. Also, the process to get the hex data into an array and into the compiled code that lives on the programmer was a bit tedious. Either way, I jumped into this rabbit hole, and thought this was going to be the solution for RedBoard Programming. This was the twelve-at-a-time beast that I came up with:
Wow, crazy to think that was October 2015. Yikes, time has flown!
After some detailed production yield tracking, we ultimately decided that we still needed to test the boards individually, and so we found that this programmer didn’t give us any reductions in labor costs. A bit of a let down, but it also proved that this method of programming was too unreliable. After running a few thousand boards through this thing, we found that boards would often fail at programming on first attempt. Not acceptable for production use. A bit of a bummer, but hey you can’t be afraid to fail, and we learned a ton! It was also a great excuse to really try out our Actobotics line of products.
Enter the Raspberry Pi – Programming Over GPIO
After exploring the chip to chip programming option, and finding that it would not work for our application, we searched for another solution. This time we wondered if a Raspberry Pi might do the trick. A quick search of the internet lead us to a great tutorial by Adafruit that uses GPIO to “bit-bang” programming to AVR chips.
We got this up and running quite quickly. Stoked! Made a custom hat, so we could run this headless, and started migrating some of our kit programming procedures. A little slower than the MKII, but it was completely stand-alone and very reliable – I’m in!
Then I found that a second or two longer is actually a lot of labor and production was really not digging this.
Looking on a scope I saw the clock signal flying at 84K. This was leading to some programming times of around 4 or 5 seconds, and then an additional 4-5 seconds to verify. Unfortunately, this was not gonna fly in production and we needed to find a faster solution.
The Need for Speed – Hardware SPI
In search of faster solutions, we found another tutorial about using the SPI hardware on the Raspberry Pi.
KEVIN CUZNER: RASPBERRY PI AS AN AVR PROGRAMMER
Cool. This should do the trick. Well, also found that you need to also open up SPI hardware on the Pi. Check out this tutorial below on how to do that. Thanks Byron!
SPARKFUN'S TUTORIAL ON RASPBERRY PI SPI AND I2C
With this new method of programming over the hardware SPI, we were able to get much faster speeds for programming. We really only needed up to 2MHz for most of our chips (less than ¼ of clock speed is recommended).
After seeing some strange behavior in clock speeds, we found this article explains it well and shows all the available speeds:
RASPBERRY PI DOCUMENTATION: SPI
RC.LOCAL + Python + HAT = Headless Operation!
RC.LOCAL
A quick modification to rc.local to call Python at bootup means we can run this thing headless. Wahoo!
#!/bin/sh -e
# rc.local
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
# In order to enable or disable this script just change the execution
# bits.
# By default this script does nothing.
# Print the IP address
_IP=$(hostname –I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
python /home/pi/test.py &
exit 0
Python Script
Note the most important line here:
python /home/pi/test.py &
And don’t forget that “&” and the end of the command – that will actually let your pi continue on to bootup and run the OS!
Test.py basically waits to see a “button” pressed to engage programming (pi_program.sh). In this case, it is actually a capsense IC that is sending HIGH/LOW to a GPIO on the pi. To read the entire test.py, you can check it out on the repo here:
PI-GRAMMER GITHUB REPO: TEST.PY
And the pi_program.sh file is basically the single call to avrdude to actually do the programming. It looks something like this on most of our production machines:
#program flash and lock bits
sudo avrdude –p $DEVICE –C /home/pi/avrdude_gpio.conf –c linuxspi –P /dev/spidev0.0 –b 2000000 –D –v –u –U flash:w:$firmware:i –u –U lock:w:$LOCK:m 2>/home/pi/flash_results.txt
PI-GRAMMER GITHUB REPO: PI_PROGRAM.SH
Pi_Grammer HAT
The last ingredient is a custom hat that allows for triggering and LED indicators to the user. Also, added a shutdown button, for goodness sake!
Parsing Output from avrdude w/ Python
In order to blink those nice new LEDs on the HAT, we’d need the Python module to be able to parse the output from avrdude, and blink the success or fail LED as needed. Oh man, What a dream! Python makes this so easy.
A Little Background on Parsing
Often times, the readout from a program can be pretty lengthy (with useful information), but when you are programming tons and tons of boards, you just want to see Success or Failure in the readout and not have to look through a ton of text.
This is something that we added into our batch files years ago. Using our old batch files, we’ve been parsing “tokens” returned from stk500.exe and the code is cryptic. Here’s what a typical batch file might look like:
FOR /F “tokens=*” %%i IN (‘”%step1%”‘) DO (
if “%%i” == “Fuse bits verified successfully” echo Success at %$step1%
) & (
if “%%i” == “Target voltage is within 5%% of wanted value” (echo Target Voltage Good)
) & (
if “%%i” == “Target voltage is NOT within 5%% of wanted value” (echo %%i) & goto menu
) & (
if “%%i” == “WARNING! One or more operations failed! Please examine the output log above!” (echo Failed at %$step1%) & goto menu
) & (
if “%%i” == “Could not connect to STK500 V2 on USB” (echo %%i) & goto menu
) & (
if “%%i” == “Could not connect to AVRISP mkII on USB” (echo %%i) & goto MENU
)
Read More Info…
