Raspberry Pi Self-driving Car
Video Streaming Car Robot using Raspberry Pi & Camera
Building a Raspberry Pi Car Robot with Video Streaming
https://how2electronics.com/video-streaming-car-robot-using-raspberry-pi-camera/
Overview
In this guide, you will learn how to build your own Raspberry Pi Video Streaming Robot Car and create a First Person View (FPV) driving experience. For this, we will use a Robotic Car Kit called Picar-X Robot from Sunfounder.
The first step is to configure video streaming on the Raspberry Pi using the HAT & Raspberry Pi camera. The Robot is capable of Video Streaming with a Raspberry Pi Camera using OpenCV. The Robot can also control camera servos as Pan and Tilt functions as its head is movable. Apart from this you can connect everything together & drive the car in any direction.
The PiCar-X is an AI self-driving robot car that uses Raspberry Pi as its control center. The Robotic Kit is designed by SunFounder. The Kit comes with a Robot HAT for Raspberry Pi. The HAT integrates the motor driving, servo driving, and preset ADC, PWM, and digital pins to extend the functionality. The PiCar-X’s has a 2-axis camera module, ultrasonic module, and line tracking modules that can provide the functions of color/face/traffic signs detection, automatic obstacle avoidance, automatic line tracking, etc.
This project can be advanced to AI-based autonomous-driving cars and fully autonomous cars. The remote control enables valuable operational data to be obtained, thus laying the groundwork for gradually increasing autonomous driving performance in the future. Moreover, human assistance through remote driving can offer more flexibility and intelligence than a single artificial intelligence.
Bill of Materials
We need the following components along with the SunFounder Robot Car Kit to make a Video Streaming Car Robot using Raspberry Pi Camera. You can purchase all these components from Amazon links:
| S.N. | COMPONENTS | QUANTITY | PURCHASE LINKS |
|---|---|---|---|
| 1 | Raspberry Pi Ai Car Kit PiCar-X | 1 | Amazon | SunFounder |
| 2 | Raspberry Pi 4 | 1 | Amazon | SunFounder |
| 3 | Samsung 18650 Battery | 2 | AliExpress |
| 4 | SD Card 16/32 GB | 1 | Amazon | SunFounder |
| 5 | SD Card Adapter | 1 | Amazon | AliExpress |
| 6 | 18650 Battery Charger | 1 | Amazon | AliExpress |
Robot HAT cannot charge the battery, so you need to buy a battery charger at the same time.
Robot Assembly
The Sunfounder Picar-X Robot Kit comes with all the necessary components required for the Robot which include the following components. You need to buy a Raspberry Pi 4 Board and a pair of Samsung 18650 Batteries separately.
The most important part of this Kit is Robot HAT. With the Robot HAT board, the PiCar-X integrates left/right driving motors, servo motors for steering, and the camera’s pan/tilt functions, and pre-sets the Robot HAT’s ADC, PWM, and Digital I2C pins to allow for extensions to the standard functionality of the Raspberry Pi.
Both a speaker and a Bluetooth chip have been engineered into the Robot HAT for remote control of Text-to-Speech, sound effects, or even background music functionality.
To assemble the components together and make a perfect Robotic Car you can follow the Assembly Guide. The PDF Document has all the images and a pictorial view for easy help with assembly.
The Robotic Kit Part details, Assembly guide, HAT parts & functionality, calibration process, Robot Testing and Setting Up Rasbian OS has been explained in PiCar-X Getting Started guide.
Read the tutorial thoroughly to understand the Hardware & Software Setup Part.
Installing Python Modules & Libraries
The OS installation part and setup part can be followed on Raspberry Pi OS Setup. But for this robotic project part, it is recommended to install Raspberry Pi OS(Legacy).
You can either use the HDMI Screen or enable SSH and connect the Raspberry Pi to VNC Viewer using the local IP Address, username, and password. It is best to connect the Robot Wirelessly via a Remote Desktop as we need a remote control for it.
The Picar-X Robot should be turned ON and Raspberry Pi should be connected to the network. Now let’s install all the Python Modules and libraries.
Let’s first update the system using the following commands.
1 2 | sudo apt update sudo apt upgrade |
Then install the Python3-related packages.
1 | sudo apt install git python3-pip python3-setuptools python3-smbus |
Download & install the libraries for Robot-Hat.
1 2 3 4 | cd /home/pi/ git clone https://github.com/sunfounder/robot-hat.git cd robot-hat sudo python3 setup.py install |
Running ‘setup.py‘ will download some necessary components. Your download may have failed due to network issues. You may need to download it again at this point. See the following interface, type ‘Y‘, and press Enter.
Now, download and install the ;vilib‘ module.
1 2 3 4 | cd /home/pi/ git clone https://github.com/sunfounder/vilib.git cd vilib sudo python3 install.py |
Similarly download and install the ‘picar-x‘ module.
1 2 3 4 | cd /home/pi/ git clone -b v2.0 https://github.com/sunfounder/picar-x.git cd picar-x sudo python3 setup.py install |
This will download a large number of files, therefore it will take time.
Once the download is complete, run the ‘i2samp.sh‘ script to install the components required by the i2s amplifier, otherwise the picar-x will have no sound.
1 2 | cd /home/pi/picar-x sudo bash i2samp.sh |
Type ‘y‘ and press enter to continue running the script.
Type ‘y‘ and press enter to run ‘/dev/zero‘ in the background.
Type y and press enter to restart the Picar-X.
After restarting, the Raspberry Pi will connect to the Network and will play a sound.
If there is no sound after restarting, you may need to run the ‘i2samp.sh‘ script several times.
All the installation and setup part is complete here.
Python Code for Video Streaming Car Robot
Here is a complete Python Code for Video Streaming Car Robot using Raspberry Pi & Camera. But before that we need to fulfill some requirements for the fully functionality of this Robot:
- This project requires access to the Raspberry Pi desktop to view the footage taken by the camera module.
- You can connect a screen to the PiCar-X or refer to the tutorial Remote Desktop to access it with VNC or XRDP.
Once inside the Raspberry Pi desktop, open Terminal and type the following command to run it, or just open and run it with a Python editor.
1 2 | cd /home/pi/picar-x/example sudo python3 video_car.py |
The code is also located in the source code path ‘picar-x/example/video_car.py‘. You can directly open the code from this location. You can Modify/Reset/Copy/Run/Stop the code below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | # #!/usr/bin/env python3 print('Please run under desktop environment (eg: vnc) to display the image window') from utils import reset_mcu reset_mcu() from picarx import Picarx from vilib import Vilib from time import sleep, time, strftime, localtime import readchar manual = ''' Press key to call the function(non-case sensitive)： O: speed up P: speed down W: forward S: backward A: turn left D：turn right F: stop T: take photo ESC / Ctrl+C: quit ''' px = Picarx() def take_photo(): _time = strftime('%Y-%m-%d-%H-%M-%S',localtime(time())) name = 'photo_%s'%_time path = "/home/pi/Pictures/picar-x/" Vilib.take_photo(name, path) print('\nphoto save as %s%s.jpg'%(path,name)) def move(operate:str, speed): if operate == 'stop': px.stop() else: if operate == 'forward': px.set_dir_servo_angle(0) px.forward(speed) elif operate == 'backward': px.set_dir_servo_angle(0) px.backward(speed) elif operate == 'turn left': px.set_dir_servo_angle(-30) px.forward(speed) elif operate == 'turn right': px.set_dir_servo_angle(30) px.forward(speed) def main(): speed = 0 status = 'stop' Vilib.camera_start(vflip=False,hflip=False) Vilib.display(local=True,web=True) sleep(2) # wait for startup print(manual) while True: print("\rstatus: %s , speed: %s "%(status, speed), end='', flush=True) # readkey key = readchar.readkey().lower() # operation if key in ('wsadfop'): # throttle if key == 'o': if speed <=90: speed += 10 elif key == 'p': if speed >=10: speed -= 10 if speed == 0: status = 'stop' # direction elif key in ('wsad'): if speed == 0: speed = 10 if key == 'w': # Speed limit when reversing,avoid instantaneous current too large if status != 'forward' and speed > 60: speed = 60 status = 'forward' elif key == 'a': status = 'turn left' elif key == 's': if status != 'backward' and speed > 60: # Speed limit when reversing speed = 60 status = 'backward' elif key == 'd': status = 'turn right' # stop elif key == 'f': status = 'stop' # move move(status, speed) # take photo elif key == 't': take_photo() # quit elif key == readchar.key.CTRL_C or key in readchar.key.ESCAPE_SEQUENCES: print('\nquit ...') px.stop() Vilib.camera_close() break sleep(0.1) if __name__ == "__main__": try: main() except Exception as e: print("error:%s"%e) finally: px.stop() Vilib.camera_close() |
Testing Video Streaming Car Robot using Raspberry Pi & Camera
This program will provide a First Person View from the PiCar-X! Use the keyboards ‘WSAD‘ keys to control the direction of movement, and the ‘O‘ and ‘P‘ to adjust the speed.
Once the code is running, you can see what PiCar-X is shooting and control it by pressing the following keys.
- O: speed up
- P: speed down
- W: forward
- S: backward
- A: turn left
- D：turn right
- F: stop
- T: take photo
- ESC / Ctrl+C: quit
This is how you can make a Video Streaming Car Robot using Raspberry Pi & Camera Module with the application of OpenCV.
Video Tutorial & Guide
 |  | | ||
| | | ||
| | |||
|
| |
