Programming a real self-driving car

Self-Driving Car Engineer Nanodegree Capstone project

Team "Udanomous"

Team members :

Name	email
Mahmoud Kamel	mahmoudkamel2014@gmail.com
Longxing Tan	longxing.tan@daimler.com
Khaled Zoheir	khaled.zoheir.kz@gmail.com
Chun Hsien Ho	chunhsien@umich.edu
Emile Papillon	epapillon@live.ca

Version

Version	details
v1.0	first submission

Copyrights disclaimer : Some code or data was used from the web . The following websites were the source of this code and data:

• Autoware provides pure-pursuit waypoint follower https://github.com/CPFL/Autoware (author : Autoware)
• The traffic light detection is using a model pretrained on the COCO dataset (http://cocodataset.org/) and available at http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_11_06_2017.tar.gz

---

Programming a self-driving car : implementation

ROS System Architecture

The following figure shows the architecture implemented, as suggested by Udacity. It contains a perception subsystem consisting of the following nodes:

• Traffic light detection : Responsible of detecting traffic lights and classifying them. If the light is green, the car is free to circulate. Otherwise it must stop. The TL-Detector publishes the waypoint where the traffic lights are detected and the state of the traffic lights.

The Planning subsystem contains the following nodes :

• Waypoint loader : Initially, waypoints are published and loaded into the self-driving car system by the waypoint loader.
• Waypoint updater node : This node provides a set of 50 waypoints to be driven in front of the vehicle. The waypoint updater node is also responsible to take the traffic light waypoints and ramp-down the target velocity to stop at a target waypoint in case a red light is detected.

Finally, the Control subsystem consists of :

• Waypoint follower node : The waypoint follower provides target states for each waypoints. It is implemented as a pure-pursuit algorithm and provided by Autoware (see copyrights notice at the beginning of this document)

• Drive-by-wire (DBW) node : The drive by wire is the interface with the actuators and it publishes the brake, throttle and steering commands. It is implemented using a PID controller with target velocities (linear and angular) as twist commands.

The simulator receives actuator commands and provides sensor data.

Architecture Diagram :

Nodes implementation

Traffic light detection

The traffic light detection uses a tensorflow model pretrained on the COCO dataset that returns a set of corners delimiting the portion of the image where traffic light s are found. This information is then used by a model made in Keras and trained on the data provided in bag files by Udacity to lear to classify lights as 0 : red, 1: yellow or 2: green.

• 32 x 32 pixel squares are generated by the detector model
• A keras model then uses the 32 x 32 pixels portions of the original frame to classify the traffic light according to its state.
• The final output are one-hot encoded labels which are then used to command the vehicle to stop if a light is classified red.

Keras Model :

Path planning

The path planning uses a K dimension tree algorithm from sklearn library to find the next waypoint and appends a list of 50 waypoints ahead of the vehicle. This node also receives the traffic light waypoints and modifies the target velocity of waypoints ahead in order to stop at red traffic lights.

Control

The control subsystem is implemented using a PID. It consists of the waypoint follower node and the DBW node. The waypoint follower is responsible of defining target speed and angular velocity that are optimal to follow the target trajectory. It implements a pure pursuit waypoint following method. The waypoints generated are used by the DBW node that is implemented using a PID to issue the right throttle, brake and steering commands to follow the planned route.

Udacity Original README.md file contents

This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.

Please use one of the two installation options, either native or docker installation.

Native Installation

• Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.

• If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:

  • 2 CPU
  • 2 GB system memory
  • 25 GB of free hard drive space

  The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.

• Follow these instructions to install ROS

  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.

• Dataspeed DBW

  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)

• Download the Udacity Simulator.

Docker Installation

Install Docker

Build the docker container

docker build . -t capstone

Run the docker file

docker run -p 4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone

Port Forwarding

To set up port forwarding, please refer to the instructions from term 2

Usage

1. Clone the project repository

git clone https://github.com/udacity/CarND-Capstone.git

2. Install python dependencies

cd CarND-Capstone
pip install -r requirements.txt

3. Make and run styx

cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch

4. Run the simulator

Real world testing

1. Download training bag that was recorded on the Udacity self-driving car.
2. Unzip the file

unzip traffic_light_bag_file.zip

3. Play the bag file

rosbag play -l traffic_light_bag_file/traffic_light_training.bag

4. Launch your project in site mode

cd CarND-Capstone/ros
roslaunch launch/site.launch

5. Confirm that traffic light detection works on real life images