Thermal cameras are very similar to standard cameras in that they record their images with light. The most significant distinction is that thermal cameras detect and filter light such that only the infrared region of the electromagnetic spectrum is recorded, not the visible region [read more about infrared cameras here and https:Available: [//thermalcameras. guide]. Infrared detectors were soon patented as a means of inferring temperature based on non-contact measurements, shortly after the discovery of the black-body radiation [for example, this was patented in 1939 ]. In the last few decades due to the advancement in the miniaturization of integrated circuits, the use of infrared detectors for applications in non-destructive tests, medical electronic devices, and motion detection of heated objects have been incorporated in the mainstream technology. The sensor used here is the MLX90640 [datasheet]; it is a 768-pixel (24 x 32) thermal camera. It employs a field of infrared detectors (and perhaps filters) for detecting the amount of radiation emitted by objects. The fairly high-resolution temperature maps will be extracted and recorded, other than using a Raspberry Pi computer — another device to be used is the MLX90640. With the parameters selected in Python, one will be able to stress the RPI, and get a 3 fps thermal camera with 240×320 pixel resolution by interpolating the MLX90640.
Parts List and Wiring
The parts list required for this tutorial is straightforward: one Raspberry Pi and one MLX90640. A Raspberry PI 4 will be used due to its higher processing capability to achieve, at least, a 3 fps visualization of all 768 pixels of the MLX90640. Some links for the RPI and MLX IR camera are given below for reference: Some links for the RPI and MLX IR camera are given below for reference:
-
MLX90640 Thermal Camera – $178.99 [Amazon], $95.00 [Our Store]
-
Raspberry Pi 4 Computer – $60.95 [Amazon], $55.00 [2GB from Our Store]
We could manage to get some from one of our partners because, at the time of writing, the MLX90640 IR camera is relatively expensive attributed to the following; It is going to be opened for sale on our store today. Comparatively, the commonly used AMG8833 of 8×8 pixel thermal camera is about $40-$50.
The wiring of the MLX90640 with the Raspberry Pi board is also described below (The wiring for the module in our store is the same). Other specialized places such as Ray PCB should be able to offer circuits where the MLX90640 and Raspberry Pi can be connected directly on the same PCB. Below is the direct wiring between the module and RPi 4:Below is the direct wiring between the module and RPi 4:
The MLX90640 and Raspberry Pi communicate via the I2C protocol, which uses the hardware pins 3/5 on the Pi (SDA/SCL).
Preparing a Raspberry Pi for the MLX90640
The Adafruit library will be used to read the MLX90640 thermal breakout board (though we’re using the Waveshare board). The following commands should be inputted into the terminal on the Raspberry Pi to ensure that the MLX90640 sensor can be visualized in Python [based on Adafruit’s tutorial]:
pi@raspberrypi:~ $ sudo pip3 install matplotlib scipy numpy
Additionally, the RPi needs I2C tools installed:
pi@raspberrypi:~ $ sudo apt-get install -y python-smbus
pi@raspberrypi:~ $ sudo apt-get install -y i2c-tools
Also ensure that the I2C is enabled (via the terminal here):
pi@raspberrypi:~ $ sudo nano /boot/config.txt
This should open up the boot file on the RPi. Scroll down to the dtparam=i2c_arm=on and make sure that it is uncommented:
With the I2C now enabled, reboot the RPi:
pi@raspberrypi:~ $ sudo reboot
Once the RPi restarts, assuming the MLX90640 is wired correctly, we can check the I2C port and ensure that the RPi registers the MLX90640. This can be done with the following command:
pi@raspberrypi:~ $ sudo i2cdetect -y 1
The following should be printed out on the terminal:
The number 33 printed out, which is the I2C address of the MLX90640 (0x33). This can be confirmed by looking at the MLX90640 datasheet.
At this point, the MLX90640 is ready to be read by the Raspberry Pi. However, since the Adafruit library is being used, a few other libraries need to be installed:
pi@raspberrypi:~ $ sudo pip3 install RPI.GPIO adafruit-blinka
pi@raspberrypi:~ $ sudo pip3 install adafruit-circuitpython-mlx90640
Next, the Python Integrated Development and Learning Environment (IDLE) is installed, but not necessarily required. An anaconda environment could also be used, but since the RPi is used here, we chose IDLE (for Python 3). IDLE, if not installed already, can be installed as follows:
pi@raspberrypi:~ $ sudo apt-get install idle3
Finally, open up IDLE or Anaconda and attempt to import the MLX90640 library from Adafruit using the following test code:
##################################
# MLX90640 Test with Raspberry Pi
##################################
#
import time,board,busio
import numpy as np
import adafruit_mlx90640
i2c = busio.I2C(board.SCL, board.SDA, frequency=400000) # setup I2C
mlx = adafruit_mlx90640.MLX90640(i2c) # begin MLX90640 with I2C comm
mlx.refresh_rate = adafruit_mlx90640.RefreshRate.REFRESH_2_HZ # set refresh rate
frame = np.zeros((24*32,)) # setup array for storing all 768 temperatures
while True:
try:
mlx.getFrame(frame) # read MLX temperatures into frame var
break
except ValueError:
continue # if error, just read again
# print out the average temperature from the MLX90640
print('Average MLX90640 Temperature: {0:2.1f}C ({1:2.1f}F)'.\
format(np.mean(frame),(((9.0/5.0)*np.mean(frame))+32.0)))
The code above should print out the average temperature read by the MLX90640. Pointing the MLX90640 sensor at the Raspberry Pi and resulted in an average temperature of 42.8°C (109.0°F).
When reading the MLX90640, an error may appear that cites a refresh rate issue. This can be avoided by amping up the rate of the I2C device on the RPi. To do this, we need to change the following back in the ‘config.txt’ file:
pi@raspberrypi:~ $ sudo nano /boot/config.txt
Scrolling down to the uncommented ‘dtparam=i2c_arm=on’ – we also want to add the following line that increases the I2C speed to 1Mbit/s:
Be cautious when increasing the I2C baud rate above the recommended speed (400kbit/s). This high speed can cause overheating of the Pi, so ensure that the board is properly ventilated or actively cooled. In an upcoming section, some routines for plotting the 24×32 temperature grid will be introduced, where this 1Mbit/s will be important for creating a near real-time thermal camera with the MLX90640 sensor.
Visualizing the MLX90640 – Real-Time Thermal Camera
The MLX90640 datasheet shows that the sensor’s pixel map begins at the top-right corner, starting with the topmost and rightmost point of its viewing window. This is shown in the illustration underneath:
This is crucial when plotting the output in Python and will assist in accurately mapping the thermal camera visualization. There are various techniques in Python for displaying the MLX90640’s output. The initial option is ‘imshow’, which enables users to see any picture. The ‘imshow’ function has a distinct origin setup, where its origin point is located at the top-left corner of the window. This indicates that the MLX90640 points must be adjusted horizontally when plotted to align with the correct mapping of the IR sensor illustrated. Below is a basic demonstration of the visualization of MLX90640 using the ‘imshow’ function in Python, with the code implementing left-right flipping.
##########################################
# MLX90640 Thermal Camera w Raspberry Pi
# -- 2Hz Sampling with Simple Routine
##########################################
#
import time,board,busio
import numpy as np
import adafruit_mlx90640
import matplotlib.pyplot as plt
i2c = busio.I2C(board.SCL, board.SDA, frequency=400000) # setup I2C
mlx = adafruit_mlx90640.MLX90640(i2c) # begin MLX90640 with I2C comm
mlx.refresh_rate = adafruit_mlx90640.RefreshRate.REFRESH_8_HZ # set refresh rate
mlx_shape = (24,32)
# setup the figure for plotting
plt.ion() # enables interactive plotting
fig,ax = plt.subplots(figsize=(12,7))
therm1 = ax.imshow(np.zeros(mlx_shape),vmin=0,vmax=60) #start plot with zeros
cbar = fig.colorbar(therm1) # setup colorbar for temps
cbar.set_label('Temperature [$^{\circ}$C]',fontsize=14) # colorbar label
frame = np.zeros((24*32,)) # setup array for storing all 768 temperatures
t_array = []
while True:
t1 = time.monotonic()
try:
mlx.getFrame(frame) # read MLX temperatures into frame var
data_array = (np.reshape(frame,mlx_shape)) # reshape to 24x32
therm1.set_data(np.fliplr(data_array)) # flip left to right
therm1.set_clim(vmin=np.min(data_array),vmax=np.max(data_array)) # set bounds
cbar.on_mappable_changed(therm1) # update colorbar range
plt.pause(0.001) # required
fig.savefig('mlx90640_test_fliplr.png',dpi=300,facecolor='#FCFCFC',
bbox_inches='tight') # comment out to speed up
t_array.append(time.monotonic()-t1)
print('Sample Rate: {0:2.1f}fps'.format(len(t_array)/np.sum(t_array)))
except ValueError:
continue # if error, just read again
The code above should output an image similar to the following:
If the user comments out the saving routine, the Raspberry Pi 4 should be able to plot roughly 2 images per second (2Hz sample rate). In the next section, the development of a real-time interpolation routine is introduced.
Real-Time Interpolation of MLX90640
The code shown above offers a straightforward and speedy method for visualizing the unprocessed data generated by the MLX90640 thermal array. Furthermore, our objective is to enhance the resolution of the IR camera, as well as to increase the speed of the plotting routine. Despite extensive testing and development, we were unable to exceed approximately 3 frames per second at the higher resolution. The newly selected resolution is 240×320 (76,800 pixels in total!), representing a significant increase from the previous native resolution. However, since we can accelerate the evaluation – the new rate of temporal sampling is quite satisfactory. Certainly, the Raspberry Pi does have its restrictions, especially when it comes to handling the images in this scenario. Finding a way to utilize a poorer quality screen may reduce processing time and lead to an even higher effective sampling rate.
The ‘ndimage’ toolkit in the Python library ‘scipy’ is capable of interpolating images through various methods for image processing tasks. By employing the ‘zoom’ feature within the ‘ndimage’ toolkit, the original 24×32 output of MLX90640 can be enhanced to 240×320 through the addition of the code mentioned earlier.
data_array = ndimage.interpolation.zoom(data_array,zoom=10)
This adjustment results in an effective frame rate of 1.6fps. This is a fairly slow frame rate, even for a thermal camera. As a result, the following codes will attempt to push the Raspberry Pi and Python to update the plot at a slightly higher frame rate.
— The first change involves invoking the ‘blitting’ method in matplotlib, which keeps the background of a matplotlib figure every loop and updates only the image section of the figure. This gives us an updated frame rate (depending on the RPI board) for the interpolated image. The code to enable ‘blitting’ is given below:
##########################################
# MLX90640 Thermal Camera w Raspberry Pi
# -- 2fps with Interpolation and Blitting
##########################################
#
import time,board,busio
import numpy as np
import adafruit_mlx90640
import matplotlib.pyplot as plt
from scipy import ndimage
i2c = busio.I2C(board.SCL, board.SDA, frequency=400000) # setup I2C
mlx = adafruit_mlx90640.MLX90640(i2c) # begin MLX90640 with I2C comm
mlx.refresh_rate = adafruit_mlx90640.RefreshRate.REFRESH_16_HZ # set refresh rate
mlx_shape = (24,32) # mlx90640 shape
mlx_interp_val = 10 # interpolate # on each dimension
mlx_interp_shape = (mlx_shape[0]*mlx_interp_val,
mlx_shape[1]*mlx_interp_val) # new shape
fig = plt.figure(figsize=(12,9)) # start figure
ax = fig.add_subplot(111) # add subplot
fig.subplots_adjust(0.05,0.05,0.95,0.95) # get rid of unnecessary padding
therm1 = ax.imshow(np.zeros(mlx_interp_shape),interpolation='none',
cmap=plt.cm.bwr,vmin=25,vmax=45) # preemptive image
cbar = fig.colorbar(therm1) # setup colorbar
cbar.set_label('Temperature [$^{\circ}$C]',fontsize=14) # colorbar label
fig.canvas.draw() # draw figure to copy background
ax_background = fig.canvas.copy_from_bbox(ax.bbox) # copy background
fig.show() # show the figure before blitting
frame = np.zeros(mlx_shape[0]*mlx_shape[1]) # 768 pts
def plot_update():
fig.canvas.restore_region(ax_background) # restore background
mlx.getFrame(frame) # read mlx90640
data_array = np.fliplr(np.reshape(frame,mlx_shape)) # reshape, flip data
data_array = ndimage.zoom(data_array,mlx_interp_val) # interpolate
therm1.set_array(data_array) # set data
therm1.set_clim(vmin=np.min(data_array),vmax=np.max(data_array)) # set bounds
cbar.on_mappable_changed(therm1) # update colorbar range
ax.draw_artist(therm1) # draw new thermal image
fig.canvas.blit(ax.bbox) # draw background
fig.canvas.flush_events() # show the new image
return
t_array = []
while True:
t1 = time.monotonic() # for determining frame rate
try:
plot_update() # update plot
except:
continue
# approximating frame rate
t_array.append(time.monotonic()-t1)
if len(t_array)>10:
t_array = t_array[1:] # recent times for frame rate approx
print('Frame Rate: {0:2.1f}fps'.format(len(t_array)/np.sum(t_array)))
The code above uses the 10x interpolation of the ‘zoom’ function to output a 240×320 thermal image. We were able to achieve roughly a 2.2 frames per second output rate with the blitting, however, if the image is shrunk by figsize=(9,5), it runs at roughly 3.4fps, and shrinking down to figsize=(5,3) results in about 5.4fps. One thing to note is that the refresh rate of the MLX90640 must also be changed to fit the different frame rates in order to keep up with the plotting. I don’t really recommend going over REFRESH_16_HZ due to the unstable nature of the images. Another thing to note is that the colorbar likely will not update unless ‘plt.pause(0.01)’ is called, and unfortunately this slows down the frame rate. Lastly, if the ‘set_clim’ function is removed and the user is able to set the clim before the loop, this can speed up the process as well. The downside is if the temperature changes, the colormap will not reflect this, obscuring the real-time visualization. With both the colorbar and clim functions commented out, we were able to get a frame rate of about 6.2fps when figsize() is set to (5,3). This is similar to if the RPI was running on a small display instead of a monitor (we were using an HD display monitor).
The resulting interpolated live plot should appear similar to the animation below (taken directly from our MLX90640 sensor):
The animation above is similar to what is expected from the code implementation at the interpolated resolution of 240×320. The MLX90640 was being refreshed at 8Hz, meaning the maximum resolution is 8fps. The GIF above also uses roughly a 5Hz frame rate, which can roughly be seen in the animation speed.
A Few Notes On The MLX90640 Real-Time Thermal Camera:
-
- The MLX90640 reads in a chess pattern, resulting in the pixelation observed during movement
- A refresh rate above 8Hz produces a large amount of noise – thus, 8fps is the recommended upper limit
- Shrinking the size of the figure results in faster processing and plotting
- Updating the temperature and colorbar limits slow down processing
- The Raspberry Pi is limited by its CPU and GPU. Running the real-time interpolation results in roughly 85% processor usage
Conclusion
The MLX90640 infrared thermal camera was introduced as a tool for visualizing the spatial distribution of temperatures across 768 (24×32) pixels. Using a Raspberry Pi, the MLX90640, and Python, a real-time temperature map was developed that operates at roughly 3-8 frames per second. The frame rate is limited by the CPU and GPU on the Raspberry Pi 4, where the upper limit of 8fps corresponds to a smaller figure display on the RPI. The thermal camera was further improved by interpolating pixels to 240×320, resulting in a smoother depiction of the temperature map. The thermal camera methods introduced in this tutorial have potential applications in non-destructive testing or experiments where distributed temperature maps are desired. Specific applications may be: electronics cooling, monitoring of moving parts with high frictions, and perhaps monitoring bodies for security or tracking. An infrared camera is particularly suitable for environments in low light because of the consistentcy of infrared radiation given off by bodies. Lastly, the MLX90640 IR sensor is a low-cost and efficient solution to monitoring spatial distributions of temperature, particularly for applications involving open-source tools that include Python, Arduino, and the Raspberry Pi platforms.
