Audio-visual face mask detection system on the Jetson Nano

This work implements face mask recognition system on the Jetson Nano using Jupyter Lab headlessly.

Currently, Face-mask recognition is considered the most significant part in computer vision and image analysis, and thus it receives much more research in its different components such as the enhancement of its algorithms or new approaches created to better detect and organize the face mask. As you know, The COVID-19 pandemic forced governments across the world to impose lockdowns to prevent virus transmissions. Research indicates that maintaining social distancing and wearing face masks while at work clearly reduces the risk of transmission.

In this tutorial, we’ll show you how to train your Nvidia Jetson Nano to recognize face masks using JupyterLab headlessly. After performing the initial analysis, the system classifies person as “wearing a mask” or flags as “not wearing a mask” and sends an instant alert, so you can take further action — dispatch a public audio announcement, send a custom message to a digital screen, or a personalized message to the person’s phone. Here, we will set-up our Nvidia Jetson Nano to send voice notifications via JupyterLab when a face mask is either recognized or not.

Prerequisites

Before you get started with the this tutorial, you will need the following:

• NVIDIA Jetson Nano Developer Kit
• Laptop or standalone PC
• High-performance microSD card: 32GB minimum
• MicroSD to SD adapter
• 5V 4A power supply with 2.1mm DC barrel connector
• USB WebCam for use with real-time applications. For live camera demonstrations, cameras like the Raspberry Pi Camera module are required. Here we’ll be using Arducam Complete High Quality Camera Bundle.

Additional Requirements

• A computer with the ability to flash your microSD card. Here we’ll be using laptop.
• Some experience with Python is helpful but not required.
• DL applications use a lot of compute resources and the heat sink may not be enough for the heat generated. Consider adding a cooling fan.

Let’s get started!

Step 1 – Burn NVIDIA Image to the SD card and power it up

For this course, we are running the Jetson Nano Developer Kit in a “headless” configuration. That means you do not hook up a monitor directly to the Jetson Nano Developer Kit. This method conserves memory resources on the Jetson Nano and has the added benefit of eliminating the requirement for extra hardware, i.e. a monitor, keyboard, and mouse.

NVIDIA provides a disk image that you can flash to a micro-SD card. You don’t have to deal with messy installs. Saving us the trouble of having to install applications and libraries ourselves and getting us up and running much faster. Make sure you are downloading NVIDIA DLI AI Jetson Nano SD Card Image v1.1.1.

• Download the DLI AI Jetson Nano SD Card Image.
• Download Etcher and install it. Burn OS image to MicroSD card.
• Plug in your webcam into one of the USB ports of your Raspberry Pi. If you are using a Raspberry Pi Camera there are a few extra steps involved. The Raspberry Pi Camera connects to the MIPI CSI port.
• Power up the Jetson Nano using DC barrel for 5V (make sure there is a green light besides the Micro-usb Port, remember to set the jumper for 5V DC).

Step 2 – Logging Into The JupyterLab Server

JupyterLab is a modern interactive development environment (IDE) that allows you to work with code, data, and the Jupyter notebook format. Jupyter Notebook is an open-source, interactive web application that allows you to write and run computer code in more than 40 programming languages, including Python, R, Julia, and Scala.

• Wait for 1 minutes for the bootup sequence to complete with JupyterLab Server running
• Connect the USB cable (Micro-B to Type-A).
• Your Jetson is already setup to automatically start a JupyterLab Server when it boots.
• Open browser on your laptop and access http://192.168.55.1:8888. Password is dlinano.

You will see below screen. Congratulations!

Step 3 – Running Jupyter Lab Remotely (optional)

Since Jetson runs headless, ie. without a graphical user interface, you’ll need to access Jupyter remotely via the WiFi network.

• Create Wireless Ad-Hoc Network from the Jetson Nano board. This method will allow you to wirelessly connect a laptop to your Jetson without any other networking infrastructure like router. This is the most robust way to connect, but you will have no internet connectivity on both your remote laptop and Jetson board.
• You can connect your Jetson to a wireless Wi-Fi network which has internet connectivity. In order to communicate with the Jetson board from remote laptop, a WiFi router is used. For this tutorial assume that the remote computer and the Nvidia Jetson board are connected to the same WIFI network.

Here we will be using second option.

Network Manager should be installed by default on Ubuntu Desktop installs, as well as most flavours of Ubuntu. mtui is a basic text-based user interface for managing Network Manager.

Run the following command launch the nmtui interface. Select Active a connection and hit OK.

sudo nmtui

Example output:

Select the interface which you want to bring online then hit Activiate button.

A ping test may be done in order to determine the status of your Internet connection.

ping google.com

If it is successful, to connect to your Jetson Nano via your local area network, you will need to know the IP address or the hostname of your device.

You can find out it by running the following command:

Hostname -I

You can unplug micro usb cable from the Jetson Nano and connect JupyterLab Server using network ip address.

You an browse http://IP_ADDRESS:8888/ to access the JupyterLab Server from remote computer.

Step 4 – Disable GUI on the Jetson nano (optional)

The Jetson comes with a desktop version of Ubuntu installed. If the GUI is not required for operation, you can disable it with the following command:

sudo systemctl set-default multi-user.target 

You should see a similar response in terminal on the Nvidia Jetson:

Removed /etc/systemd/system/default.target.Created symlink /etc/systemd/system/default.target → /lib/systemd/system/multi-user.target. 

This will free up extra memory that the window manager and desktop uses.

To get a glimpse of the memory usage, just run the top command:

top - 06:54:47 up 2 min, 1 user, load average: 0.36, 0.49, 0.22Tasks: 207 total,  1 running, 206 sleeping,  0 stopped,  0 zombie%Cpu(s): 0.0 us, 0.4 sy, 0.0 ni, 99.4 id, 0.1 wa, 0.0 hi, 0.1 si, 0.0 stKiB Mem : 4051256 total, 3426792 free,  283448 used,  341016 buff/cacheKiB Swap: 6219916 total, 6219916 free, 0 used. 3597892 avail Mem 

This command was run with GUI enabled as well to see difference:

top - 06:50:44 up 16 min, 2 users, load average: 0.04, 0.17, 0.24Tasks: 314 total,  1 running, 313 sleeping,  0 stopped,  0 zombie%Cpu(s): 1.2 us, 0.7 sy, 0.0 ni, 97.9 id, 0.0 wa, 0.1 hi, 0.1 si, 0.0 stKiB Mem : 4051256 total, 1558468 free, 1679124 used,  813664 buff/cacheKiB Swap: 6219916 total, 6219916 free, 0 used. 2205376 avail Mem 

To enable GUI again issue the command:

sudo systemctl set-default graphical.target

Step 5 – Upgrade all installed packages on your Jetson

At this point it’s a good idea to run some updates. You can do that by entering the commands below on the Nano.

sudo apt-get updatesudo apt-get upgrade

If yo you see:

Reading package lists... DoneE: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)E: Unable to lock directory /var/lib/apt/lists/

You can delete the lock file with the following command:

sudo rm /var/lib/apt/lists/lock

You may also need to delete the lock file in the cache directory

sudo rm /var/cache/apt/archives/locksudo rm /var/lib/dpkg/lock

Once you are done, you can reboot the Jetson Nano with this command:

sudo reboot now

Step 6 – Running the Camera code cell

• Navigate to the nvdli-nano – > classification folder with a double-click
• Open classification_interactive.ipynb file.

Source: Audio-visual face mask detection system on the Jetson Nano