A mobile robot with Vision based obstacle capability

I worked on designing a mobile robot that would be capable of detecting and avoiding obstacles visually by means of computer vision algorithm in a low cost computer such as a Raspberry Pi. I am sharing the outcome of my design project here so that people could replicate and improve upon the project. Do share your thoughts and question with me, if you have any.


The purpose of this report is to discuss and demonstrate the concept of designing and implementing a mobile robot capable of visually detecting and avoiding obstacles using a USB webcam. The system was ported to the Raspberry Pi hardware, a Linux based US$35 computer.This blog will highlight the important concepts that was needed to realise the successful implementation of a mobile robot capable of navigating within an indoor environment. Theproposed solution is intended to be used within a warehouse setting to transport goods from one location to the next without the need of human operators.


Robots already exist in the market but these robots are expensive to buy and are purposely designed to work in a single environmental setting such as warehouse, battlefield or at homes. These robots are capable of detecting obstacles using passive sensors such as ultrasonic sensors and infrared sensors and are sometime referred to as Autonomous Vehicles because of its mobility on ground surface. Traditionally, these autonomous systems are usually equipped with passive sensors such as ultrasonic distance sensors, infrared sensors or GPS to sense its position and environment. The presence of these sensors enables the robot to avoid collision with obstacles when placed with obstacles (Wang J. et al, 2011). However, with the advancement of low cost embedded computer system technology with high processing power, it is possible to use a more reliable and cost effective method of designing autonomous robots capable of detecting obstacles. One possible method would be to make use of computer vision algorithm that would be visually capable of detecting and avoiding obstacles. Sensor based method of detecting obstacle is limited to the proximity of the obstacles with respect to the distance of the robot, however, with vision based method, it is possible to detect and identify the obstacle before avoiding the obstacle.

Proposed Solution & Artefact:

The aim of this report is to design an autonomous robot that is capable of detecting and avoiding obstacles using USB webcam connected to a Raspberry Pi. This report will introduce basic concept of mobile robots and the technologies that are currently being employed within an indoor setting. The following steps were carried out in order to implement the propose system:

  • Design an algorithm using computer vision framework such as SimpleCV that can detect obstacles.
  • Implement the algorithm to a Raspberry Pi
  • Implement motor control on Raspberry Pi using Python programming language
  • Test and evaluate the performance of the proposed system using Raspberry Pi

Moreover, the following objectives were met in order to fully realise the proposed system:

  • The range of obstacle detection can be made by modifying the algorithm as per demand.
  • A geometric shape (i.e. Circle) of constant size will be placed on the obstacle for detection
  • SimpleCV library communicates with Raspberry Pi to control the two DC motors
  • The system is tested and evaluated by modifying the code and detecting the obstacle at varying distances

Hardware and Software list:

The proposed system was implemented on a Raspberry Pi using an USB webcam to capture images. The following are the list of hardware and software that were required to fully realise the proposed autonomous robot.


  • Raspberry Pi Kit,
  • USB webcam,
  • two DC motors,
  • H-Bridge (to control motor direction),
  • Chassis,
  • Cables
  • breadboard

Software package:

  • SimpleCV python library,
  • Ubuntu Operating System,
  • Raspberry OS (Raspbian),
  • Python2.7

The proposed system is capable of detecting obstacle at various ranges. This was inspired by visual looming algorithm and was achieved by defining the minimum number of pixels within the code while measuring the distance between the camera and the obstacle. The setup to this can be made within a laboratory as shown below:

Figure – The experimental setup used to find relationship between the project circle and the distance of the obstacle from the robot

The arrangement shown above can be made in the laboratory to measure the distance between the obstacle (circle) and the robot in order to determine the relationship between the minimum numbers of pixels which will correspond to the actual distance of the obstacle from the robot. As seen from the diagram above and discussion made in Chapter 2, the size of the circle is larger when the circle is closer together while it is smaller when the circle is further away. Hence, a set of test could be performed that would allow distance measurements of the obstacle against the robot when the number of pixels are changed within the algorithm. The obstacle will be move as shown in the diagram above so to find the right distance at which the algorithm detects the obstacle.


Figure – The concept of visual looming is inspired from medical science

Overall System Flowchart:

The following flowchart explains how the proposed vision based robot would be designed using Raspberry Pi and the SimpleCV framework. As shown on figure 6, an USB webcam will be connected to Raspberry Pi as an input to the computer. The USB webcam will scan incoming frames for a specific geometric shape i.e. Circle which will be placed on obstacles. The reason of marking the obstacles with geometrical shape i.e. circlewas decided merely to reduce time when preparing the algorithm. Usually, vision based obstacles detection is done using two cameras which create stereo vision to judge the distance between the robot and the obstacles (Neves& Matos, 2013). Hence, detecting a geometric shape i.e. circles using a single camera was achievedby calculating the radius of the circle would help the robot to detect the distance of the robot and the obstacles. This method of detecting an obstacle was inspired by vision looming method proposed by Sachin & Paolo (1998).


The proposed system will incorporate USB camera which will be connected to a Raspberry Pi. Raspberry Pi will be running a python script that will grab images in the video while looking for a geometrical shape i.e. circles. If the circle is detected then based on the location of the circle on the screen, the script will pull up or down the GPIO port pins that will be used to control the direction of the robot.


Computer vision algorithm structure:

The algorithm was prepared on an Ubuntu powered laptop before porting to a Raspberry Pi. This is due to the fact that Ubuntu and Raspbian are flavours of Linux, which makes porting and implementing the algorithm easier between cross-platforms. This gave me the flexibility to work with the algorithm without needing to constantly connect to my Raspberry Pi. Hence, the algorithm was implemented independently without worrying about the hardware interface.

The first step of designing a vision based system is to select the Region of Interest (ROI) which would detect the obstacle and help us with the algorithm preparation. The ROI in this case is a black circle that would be detected by the algorithm.After detection, the algorithm binarises the image so that only the region of interest is visible on the background. The algorithm would measure the coordinates of the circle on the screen and measure the coordinates with respect to what region of the screen the circle was detected. If the coordinate falls in the centre of the screen then the algorithm would move the robot accordingly as discussed earlier.


Before writing the algorithm, it was necessary to decide on the screen resolution which would allow the Raspberry Pi to detect and keep track of the obstacle. Since Raspberry Pi is a low powered computer with a system clock speed of 700MHz and a limited RAM of 512MB, therefore, high definition video input would consume a lot of processing power which will eventually cause the system unreliable. Hence, a middle ground need to be selected which would be sufficient for the Raspberry Pi computer to process the data from the camera.

After several trial and errors, a resolution of 320×240 was selected as discussed in the later section. This resolution was sufficient to detect the different color space and geometrical shapes. It was also found in the experiment that this resolution was not causing the input video to lag as was the case with higher resolution, particularly, 640×480 and above. This is discussed in detail in Section 4 of this report.


The first step of detecting obstacle from the image was to divide the screen in to three distinct segments as show in Figure 8. This was done to detect and track the position of the obstacle so that the respective motor action could be perform as discussed in the previous section. For this design project, the obstacles are represented by a specific symbol i.e. a circle; hence, the presence of a circle would register the presence of an obstacle in the algorithm.

Setting up the system:

As seen from the schematic diagram, the Raspberry Pi was connected to L293D H-bridge chip which in turn were connected to an external power source and two DC motors. The final system was attached to a WST chassis and connections from the Raspberry Pi were made on a breadboard and an USB webcam. The schematic Diagram above was made using EagleTool software.


Final System Design:


Conclusion and Future work:

The final system design did not require any traditional sensor based technology such as Infrared, Ultrasonic sensor etc. for detecting obstacle but by means of an USB webcam the system simply required the shape and size to be entered in to the programming environment of Raspberry Pi which would eventually allow the autonomous robot to detect and avoid colliding with any kind of obstacles as long as the geometric markers are placed and visible to the robot. It was also found that a robot with vision based capability is an economically option than sensor based robots. This is due to a fact that webcam on a robot is a kind of multipurpose sensor that can be used in various situation to perform multitude of task.

By testing the frame rate of the algorithm on Ubuntu powered laptop and comparing the result of the same algorithm on a Raspberry Pi, it was found that embedded computers such as a Raspberry Pi has a relatively satisfactory performance when running the algorithm given that Raspberry Pi has a processor speed of 700MHz. Since the algorithm was intentionally made simple to detect an obstacle marked with a circle of a constant size, therefore, the algorithm did a better job of detecting obstacle marked with a circle. However, this also shows the limitation of Raspberry Pi as evident from the frame rate obtained from the Raspberry Pi which drastically decreases due to a slower processor.

In the end, all objective were successfully met and an autonomous robot with vision based obstacle avoidance capability was designed and implemented on a Raspberry Pi. The robot was capable of controlling the motors of the chassis simply by detecting a special marker placed on the obstacle. The algorithm was made modular enough that the range of obstacle detection can be controlled simply by changing certain variables on the algorithm without needing to replace the webcam or any electronics.

Future Work:

Embedded Computer vision is a niche area of computer vision and embedded system engineering. Due to time constraint, a simple algorithm was designed and tested on a Raspberry Pi which was capable of detecting a circle. This method is not always reliable and has various limitations. For instance, the algorithm requires a better lighting condition so that the SimpleCV library could differentiate black blobs from other colours. Lens distortion from USB camera is minimised but it cannot be eliminate. The algorithm makes use of low resolution screen size to identify obstacle but low resolution creates false positive which affects the performance of the algorithm.

All these can be avoided if a more rigid approach to object detection is employed. This includes, but is not limited, to machine learning techniques which would allow the algorithm to learn to differentiate among different obstacles. This would allow the algorithm to detect obstacles with no marker attached to it. Moreover, with machine learning algorithm, the robot could also detect moving obstacle and could also identify people.

Leave a Reply