Building a Vision-Controlled Car Using Raspberry Pi—From Scratch

In this tutorial, we’ll create a car that controlled using a USB camera and a Raspberry Pi. We’ll use a wood plank as the car body, which will hold all the car’s components. The components are:

  • 1 USB camera
  • 1 Raspberry Pi (RPi)
  • 1 mini electronic breadboard (BB)
  • 4 DC toy motors
  • 4 car tires (and a couple more small things).

After fixing the component using cable ties, we’ll connect the Raspberry Pito the motors using its general purpose input output (GPIO) pins. Similar to a traffic sign, the motors will run when the dominant color in the captured images is green and stop when it’s red.

The tutorial has 2 experiments. The first one creates a car that moves without using the camera. This is just for testing to make sure everything works well on the simplest level. The second experiment uses the camera, which will be placed on the car for controlling its movement.

Before starting the tutorial, it’s preferred to have a beginners-level knowledge about electronics and to be able to create a circuit for lighting an LED using Raspberry Pi. Basic knowledge about how transistors and diodes work is also essential.

The steps of this tutorial are as follows:

  1. Preparing the Car Components
  2. Fixing the Components on the Wood Plank
  3. Understanding the Circuit Connected to the Motor
  4. Building the Circuit on the BB using Transistors, Resistors, and Diodes
  5. Connecting the BB to the RPi GPIO Pins
  6. Accessing RPi
  7. Controlling Car Motors
  8. Connecting a USB Camera to RPi
  9. Controlling the Motors using the Camera

Let’s start discussing each of these steps.

1. Preparing the Car Components

There are 12 different components required. Multiple pieces may be required from the same component. The summary is as follows:

1. (1) Raspberry Pi: Processes the captured images and generates signals over its GPIO output pins for running or stopping the motors.

2. (1) Mini Electronic BB: It holds the circuits connecting the motors to the RPi pins.

3. (4) DC Toy Motors: Such motors push the car forward.

4. (1) Wood Plank: Components in point 2, 3, and 4 will be placed on this plank. The plank is labeled according to the position of each component. M1, M2, M3, and M4 refer to the 4 motors.

5. (6) Plastic Cable Ties: Used for fixing the previous components (1 RPi, 1 BB, and 4 DC motors) on the wood plank. 6 cables are needed because there are 6 pieces to be fixed. Note that each component has two holes on their sides for passing the cable tie through the component. The holes for the BB are bounded in red in the wood plank image.

6. (4) Car Tires: Each tire is connected to one motor. Make sure that the tire fits the axis in the motor.

7. (13) Male/Male Jumper Wires: Such wires connect the components placed on the BB.

8. (3) Male/Female Jumper Wires: The female end of such wires is connected to the GPIO pins of the RPi. The male end is connected to the BB.

9. (2) NPN MPS2222A Transistors: The output current from the RPi GPIO pins does not allow the motor to work. The transistor amplifies such current. As an alternative to the MPS2222A, you could use a 2N2222A.

10. (2) 1N4148 Diodes: Used for passing the current in one direction. If the amplified signal produced by the transistor returns back to the GPIO pins, the pins will be damaged.

11. (2) 1K Resistors: Connects the transistor base terminal to the GPIO output pin.

12. (1) USB Camera: Used for capturing images and controlling the movement of the car. Based on the color in such images, the car motors may run or stop. It will be used in the second experiment.

Note that there are just 2 transistors, 2 resistors, and 2 diodes. Every single transistor-resistor-diode combination builds a single circuit connected to a single motor. In this tutorial, just 2 motors will be used for pushing the car, and thus just 2 circuits are required. This is why there are just 2 pieces for the transistor, resistor, and diode.

After placing all components required for building the body of the car on the wood plank according to the following figure, our next step is to use the cable tie for fixing them.

2. Fixing the Components on the Wood Plank

The following figure shows the front side of the wood plank after fixing the components using the cable ties. Make sure that no component is shaking, especially the motors. It’s also very important to use car tires that lift up the wood blank from the ground so that the back side of the cable ties are not touching the ground. This affects the movement of the car. Make sure the cables aren’t touching anything in the RPi to avoid damaging its elements.

After making sure that the car elements are placed on the wood plank correctly, our next step is to build the circuits that will be connected to both the motors and the RPi GPIO pins.

3. Understanding the Circuit Connected to the Motor

The BB holds two circuits, with each circuit connected to a single motor. The circuit diagram is given below. The circuit uses a transistor, a resistor, and a diode. Let’s understand the benefit of this circuit and how such components are connected to each other.

The maximum current produced by a GPIO pin in the RPi is around 16mA. This current is not sufficient for running the DC toy motor. Thus, we have to increase that current. This is done using the transistor, which amplifies the current.

The transistors have three legs: emitter, base, and collector. The base is the middle leg, and the other legs are on the sides of the transistor. It’s important to know the label of each leg in order to build the circuit successfully. You can search Google for more info.

The transistor emitter is connected to the ground (GND) RPi GPIO pin. The transistor base is connected to a 1K resistor, which is then connected to the RPi GPIO output pin.

The transistor collector is connected to the anode of the diode. The diode cathode is connected to the 5 voltage pin in the RPi GPIO. Some diodes have a bold black circle near the cathode terminal. This helps us identify the diode’s 2 terminals.

In order to check the diode polarity, we can use the multimeter. By connecting the diode terminals to that device, we can detect its polarity. When the device reads a value for the resistance, this means that the positive terminal of the device touches the anode of the diode and the negative terminal of the device touches the cathode.

The motor is connected in parallel to the diode. When the state of the GPIO output pin connected to the transistor is high, the NPN transistor will be active and current will pass from the emitter to the collector. The diode will be activated and thus the motor will run. The diode prevents the current from flowing in the opposite direction. If the amplified signal from the transistor flows back to the GPIO pin of the RPi, the pin will be damaged.

Note that reversing the connection of the motor will still make it work—but in the reverse direction. This helps to build a car that moves forward and backward.

After understanding the circuit, let’s build them in the BB.

4. Building the Circuit on the BB using Transistors, Resistors, and Diodes

The connections between the three components (transistor, resistor, and diode) on the BB are illustrated in the following figure. Remember to connect the correct terminals of the transistors and the diodes to each other. Let’s explore the connections for the circuit at the top of the figure. This circuit is connected to the motor labeled M2 on the wood plank.

The resistor is connected to the pins in columns E and F in row 4. The base of the transistor is connected to the pin in column G and row 4. Because columns F and G are connected to each other in the breadboard, then both the resistor and the base are connected to each other.

The anode of the diode is connected to the pin in column I and row 5. Similarly, the cathode of the diode is connected to the pin in column I but row 8. Note that the collector of the transistor is connected to the pin in row 5 and column G. Because columns I and G are connected to each other in the breadboard, then the collector of the transistor is connected to the anode of the diode.

According to the wood plank labels, the motor labeled M2 is connected in parallel with the diode, using its pins in column I and rows 5 and 8.

The transistor’s emitter is connected to the pin in row 4 and column G. According to the previous diagram, the emitter is connected to the GND pin of the RPi GPIO. This will be shown later.

There are 2 other connections to the RPi GPIO pins. The first one is between the cathode of the diode and a 5V GPIO pin. The second one is the connection between the terminal of the resistor in column E and row 4 and one of the GPIO output pins.

The previous connections will be repeated for the second circuit of the motor labeled M1 on the wood plank. As in the first circuit, there will be 3 missing connections to the RPi GPIO pins. Let’s show the connections between the BB and the RPi GPIO pins. Up to this time, all jumper wires used are male/male.

5. Connecting the BB to the Raspberry Pi GPIO Pins

There are three connections required between each circuit and the RPi GPIO pins. The first one connects one of the GND pins of the RPi to the emitter of the NPN transistor. The second connection connects the cathode of the diode to one of the 5V pins in the RPi. The third connection connects the resistor to one of the GPIO output pins.

The following diagram shows how the circuits created previously in the BB are connected to the GPIO pins of the RPi. According to the board numbering of the GPIO pins, pin number 4 is used for 5V, pin number 6 as a GND, and pin number 8 as the output pin. The 5V pin is connected to the BB using the pin in column J and row 28. From the same row in the BB, two connections will connect the cathode of the diode to the 5V pin of the RPi.

The GND pin of the RPI is connected to the BB using the pin in column A and row 23. As previously, two connections from that row are used for connecting the emitter to the ground. Finally, the output pin of the RPi is connected to column A in row 27. This row will have two connections to the resistor of each circuit. Note that the connections from the RPi pins to the BB use male/female jumper wires.

The wood plank will look as given in the following figure after building the circuits, connecting them to the RPi, and attaching the tires to the motors.

Note that the default state of the output pin of the GPIO is low. This means the transistor is not active. In order to activate the transistor, the state of this pin must be changed to high. To do that, we have to access the RPi in order to execute a Python script that changes the state of the GPIO output pin to high. This activates the transistors and thus runs the motors.

6. Accessing the Raspberry Pi

There are different ways of accessing the RPi. One way is to connect it to a display using the HDMI port in addition to a keyboard and a mouse using 2 USB ports. Another way is to connect to the Ethernet interface of your PC and establish a secure session (SSH) based on the IPv4 address assigned to the RPi Ethernet interface.

For instructions on establishing an SSH session between your PC and the RPi using an Ethernet cable, read my tutorial titled “Building an Image Classifier Running on Raspberry Pi”.

For instructions on creating a SSH session wirelessly between your PC and the RPi, read the first step (Connecting RPi to a PC using the Wireless Interface) of my article titled “Building Surveillance System using USB Camera and Wireless-Connected Raspberry Pi”.

After establishing the SSH session using the MobaXterm software, you’ll see the following screen assuming that the IPv4 address assigned to the RPi is 192.158.1.9.:

7. Controlling the Car’s Motors

After accessing the RPi, we can create a Python script to control the DC motors. We do this by changing the state of the GPIO output pin to high and low. We can drag and drop files from and to RPi SD card by simply using the MobaXterm secure file transfer protocol (SFTP) session that opens automatically after opening the SSH session. We can also create a file directly using the “Create new file” option in that software. Whatever method we use, we need to have a Python script with the following code:

import time
 import RPi.GPIO
 
 # Initializing the GPIO pins. The numbering using is board.
 RPi.GPIO.setmode(RPi.GPIO.BOARD)
 
 # Configuring the GPIO pin number 8 to be an output pin.
 RPi.GPIO.setup(8, RPi.GPIO.OUT)
 
 print("Running Motor.")
 # Running the motor by changing the GPIO output pin state to high.
 RPi.GPIO.output(8, RPi.GPIO.HIGH)
 
 # Leave the motor running for 3 seconds.
 time.sleep(3)
 
 # Stop the motor by changing the GPIO output pin state back to low.
 RPi.GPIO.output(8, RPi.GPIO.LOW)
 
 # cleanup all GPIO pins.
 print("Clean Up GPIO.")
 RPi.GPIO.cleanup()

This code imports the RPi.GPIO module to access the GPIO pins. The RPi.GPIO.setmode() function sets the pins numbering to “RPi.GPIO.BOARD”. This means the pin numbers are assigned to the board numbering. The following figure gives the numbers and locations of the GPIO pins according to the board’s numbering. Now it’s clear where exactly the pins that we’re using in this experiment are.

The above code configures the pin number 8 as an output pin using the RPi.GPIO.setup() function. Then its state is changed to high using the RPi.GPIO.output() function. At this time, the motor will run. It will still run for three seconds according to the time.sleep() function. After that time, its state will be returned back to low. Finally, the pins of the GPIO are returned to their default states by cleaning their current signals using the RPi.GPIO.cleanup() function.

A summary of the above work can be found in this video on my YouTube channel:

8. Connecting a USB Camera to the Raspberry Pi

After running the motors by changing the GPIO output pin state from low to high, we can extend that by using a USB camera. We can connect the camera to one of the USB ports of the RPi. Based on its captured images, we can determine whether the motors should run or not.

For instructions about how to access a USB camera from the RPi, you can read the previous 2 tutorials just mentioned above. After at least reading the first tutorial titled “Building an Image Classifier Running on Raspberry Pi”, you should be able to capture and save images using the PyGame module and process them.

Here is the Python code used to capture and save a single image using PyGame and display it:

import os

import pygame, sys



from pygame.locals import *

import pygame.camera



width = 320

height = 240



# initialise PyGame

pygame.init()

pygame.camera.init()

cam = pygame.camera.Camera("/dev/video0",(width,height))

cam.start()

# Setup a window for displaying the captured images

windowSurfaceObj = pygame.display.set_mode((width,height),1,16)

pygame.display.set_caption('Camera')

# Take a picture

image = cam.get_image()

# Display the picture

catSurfaceObj = image

windowSurfaceObj.blit(catSurfaceObj,(0,0))

pygame.display.update()

pygame.image.save(windowSurfaceObj, './test_picture' + str(k) + '.jpg')
cam.stop()

After doing that, we’re ready to continuously capture images, analyze their colors, and decide whether to run or stop the motors.

9. Controlling the Motors using the Camera

Each captured image using the USB camera connected to the RPi will be processed to determine whether its dominant color is green or red. If green, then the GPIO output pin state will be made high to move the car by running its motors. If the dominant color is red, then the GPIO output pin state will be set to low to stop the car. This is similar to the traffic sign.

Here’s the Python code used to determine the dominant color in the captured images and change the state of the GPIO output pin according to it:

import time
 import RPi.GPIO
 import numpy
 import os
 import pygame.camera
 import pygame
 
 #####GPIO#####
 # Initializing the GPIO bins. The numbering using is board.
 RPi.GPIO.setmode(RPi.GPIO.BOARD)
 
 # Configuring the GPIO bin number 22 to be an output bin.
 RPi.GPIO.setup(8, RPi.GPIO.OUT)
 
 #####PyGame#####
 # Initializing PyGame and the camera.
 pygame.init()
 pygame.camera.init()
 
 # Captured image dimensions. It should be less than or equal to the maximum dimensions acceptable by the camera.
 width = 320
 height = 240
 
 # Preparing a resizable window of the specified size for displaying the captured images.
 window = pygame.display.set_mode((width, height), pygame.RESIZABLE)
 
 # Specifying the camera source and the image dimensions.
 cam = pygame.camera.Camera("/dev/video0", (width, height))
 cam.start()
 
 for im_num in range(0, 2000):
     print("Image : ", im_num)
 
     im = cam.get_image()
 
     # Displaying the image on the window starting from the top-left corner.
     window.blit(im, (0, 0))
 
     # Refreshing the window.
     pygame.display.update()
 
     im = pygame.surfarray.array3d(window)
     r = numpy.mean(im[:, :, 0])
     g = numpy.mean(im[:, :, 1])
     b = numpy.mean(im[:, :, 2])
 
     if ((r - g) > 30 and (r - b) > 30):
         print("Red - Stop")
         try:
             RPi.GPIO.output(8, RPi.GPIO.LOW)
         except KeyboardInterrupt:  # CTRL+C
             print("Keyboard Interrupt.")
         except:
             print("Error occurred.")
     elif ((g - r) > 30 and (g - b) > 30):
         print("Green - Move")
         try:
             RPi.GPIO.output(8, RPi.GPIO.HIGH)
         except KeyboardInterrupt:  # CTRL+C
             print("Keyboard Interrupt.")
         except:
             print("Error occurred.")
 
 # Stopping the camera.
 cam.stop()
 
 # cleanup all GPIO bins.
 print("Clean Up GPIO.")
 RPi.GPIO.cleanup()

Simply, the average of each of the three RGB channels is calculated. If the average value of the red channel is larger than the average color of both the blue and green channels, then the dominant color is red and the car will stop. If the average color of the green channel is larger than the average of the other 2 channels, then the car moves. Note that the if statements use a margin of 30. This makes the decision more accurate.

After running this code, a window will appear that shows each captured image. When the dominant color in the image is green, the motors run and a print message appears on the terminal of the session. This is also the same for red.

I prepared two images, one red and the other is green as given in the following figure. That image is displayed on the screen of the PC. When the camera is placed in front of the green one, the motors run. Motors stop when in front of the red one.

The following video shows that the experiment works well and that the color is classified correctly:

For More Details

For Contacting the Author

Avatar photo

Fritz

Our team has been at the forefront of Artificial Intelligence and Machine Learning research for more than 15 years and we're using our collective intelligence to help others learn, understand and grow using these new technologies in ethical and sustainable ways.

Comments 0 Responses

Leave a Reply

Your email address will not be published. Required fields are marked *