The purpose of this report is to discuss and demonstrate the concept of designing and implementing a mobile robot capable of visually detecting and avoiding obstacles using a USB webcam. The system was ported to the Raspberry Pi hardware, a Linux based US$35 computer.This blog will highlight the important concepts that was needed to realise the successful implementation of a mobile robot capable of navigating within an indoor environment. Theproposed solution is intended to be used within a warehouse setting to transport goods from one location to the next without the need of human operators.
Robots already exist in the market but these robots are expensive to buy and are purposely designed to work in a single environmental setting such as warehouse, battlefield or at homes. These robots are capable of detecting obstacles using passive sensors such as ultrasonic sensors and infrared sensors and are sometime referred to as Autonomous Vehicles because of its mobility on ground surface. Traditionally, these autonomous systems are usually equipped with passive sensors such as ultrasonic distance sensors, infrared sensors or GPS to sense its position and environment. The presence of these sensors enables the robot to avoid collision with obstacles when placed with obstacles (Wang J. et al, 2011). However, with the advancement of low cost embedded computer system technology with high processing power, it is possible to use a more reliable and cost effective method of designing autonomous robots capable of detecting obstacles. One possible method would be to make use of computer vision algorithm that would be visually capable of detecting and avoiding obstacles. Sensor based method of detecting obstacle is limited to the proximity of the obstacles with respect to the distance of the robot, however, with vision based method, it is possible to detect and identify the obstacle before avoiding the obstacle.
Proposed Solution & Artefact:
The aim of this report is to design an autonomous robot that is capable of detecting and avoiding obstacles using USB webcam connected to a Raspberry Pi. This report will introduce basic concept of mobile robots and the technologies that are currently being employed within an indoor setting. The following steps were carried out in order to implement the propose system:
- Design an algorithm using computer vision framework such as SimpleCV that can detect obstacles.
- Implement the algorithm to a Raspberry Pi
- Implement motor control on Raspberry Pi using Python programming language
- Test and evaluate the performance of the proposed system using Raspberry Pi
Moreover, the following objectives were met in order to fully realise the proposed system:
- The range of obstacle detection can be made by modifying the algorithm as per demand.
- A geometric shape (i.e. Circle) of constant size will be placed on the obstacle for detection
- SimpleCV library communicates with Raspberry Pi to control the two DC motors
- The system is tested and evaluated by modifying the code and detecting the obstacle at varying distances
Hardware and Software list:
The proposed system was implemented on a Raspberry Pi using an USB webcam to capture images. The following are the list of hardware and software that were required to fully realise the proposed autonomous robot.
- Raspberry Pi Kit,
- USB webcam,
- two DC motors,
- H-Bridge (to control motor direction),
- SimpleCV python library,
- Ubuntu Operating System,
- Raspberry OS (Raspbian),
The proposed system is capable of detecting obstacle at various ranges. This was inspired by visual looming algorithm and was achieved by defining the minimum number of pixels within the code while measuring the distance between the camera and the obstacle.
Figure – The experimental setup used to find relationship between the project circle and the distance of the obstacle from the robot
The arrangement shown above can be made in the laboratory to measure the distance between the obstacle (circle) and the robot in order to determine the relationship between the minimum numbers of pixels which will correspond to the actual distance of the obstacle from the robot. As seen from the diagram above and discussion made in Chapter 2, the size of the circle is larger when the circle is closer together while it is smaller when the circle is further away. Hence, a set of test could be performed that would allow distance measurements of the obstacle against the robot when the number of pixels are changed within the algorithm. The obstacle will be move as shown in the diagram above so to find the right distance at which the algorithm detects the obstacle.
Figure – The concept of visual looming is inspired from medical science
Overall System Flowchart:
The following flowchart explains how the proposed vision based robot would be designed using Raspberry Pi and the SimpleCV framework. As shown on figure 6, an USB webcam will be connected to Raspberry Pi as an input to the computer. The USB webcam will scan incoming frames for a specific geometric shape i.e. Circle which will be placed on obstacles. The reason of marking the obstacles with geometrical shape i.e. circlewas decided merely to reduce time when preparing the algorithm. Usually, vision based obstacles detection is done using two cameras which create stereo vision to judge the distance between the robot and the obstacles (Neves& Matos, 2013). Hence, detecting a geometric shape i.e. circles using a single camera was achievedby calculating the radius of the circle would help the robot to detect the distance of the robot and the obstacles. This method of detecting an obstacle was inspired by vision looming method proposed by Sachin & Paolo (1998).
The proposed system will incorporate USB camera which will be connected to a Raspberry Pi. Raspberry Pi will be running a python script that will grab images in the video while looking for a geometrical shape i.e. circles. If the circle is detected then based on the location of the circle on the screen, the script will pull up or down the GPIO port pins that will be used to control the direction of the robot.
Computer vision algorithm structure:
The algorithm was prepared on an Ubuntu powered laptop before porting to a Raspberry Pi. This is due to the fact that Ubuntu and Raspbian are flavours of Linux, which makes porting and implementing the algorithm easier between cross-platforms. This gave me the flexibility to work with the algorithm without needing to constantly connect to my Raspberry Pi. Hence, the algorithm was implemented independently without worrying about the hardware interface.
The first step of designing a vision based system is to select the Region of Interest (ROI) which would detect the obstacle and help us with the algorithm preparation. The ROI in this case is a black circle that would be detected by the algorithm.After detection, the algorithm binarises the image so that only the region of interest is visible on the background. The algorithm would measure the coordinates of the circle on the screen and measure the coordinates with respect to what region of the screen the circle was detected. If the coordinate falls in the centre of the screen then the algorithm would move the robot accordingly as discussed earlier.
Before writing the algorithm, it was necessary to decide on the screen resolution which would allow the Raspberry Pi to detect and keep track of the obstacle. Since Raspberry Pi is a low powered computer with a system clock speed of 700MHz and a limited RAM of 512MB, therefore, high definition video input would consume a lot of processing power which will eventually cause the system unreliable. Hence, a middle ground need to be selected which would be sufficient for the Raspberry Pi computer to process the data from the camera.
After several trial and errors, a resolution of 320×240 was selected as discussed in the later section. This resolution was sufficient to detect the different color space and geometrical shapes. It was also found in the experiment that this resolution was not causing the input video to lag as was the case with higher resolution, particularly, 640×480 and above. This is discussed in detail in Section 4 of this report.
The first step of detecting obstacle from the image was to divide the screen in to three distinct segments as show in Figure 8. This was done to detect and track the position of the obstacle so that the respective motor action could be perform as discussed in the previous section. For this design project, the obstacles are represented by a specific symbol i.e. a circle; hence, the presence of a circle would register the presence of an obstacle in the algorithm.
For more detail: A Mobile Robot with Vision Based Obstacle Avoidance