IMPORTANT

Because of the GitHub File size limit I had to store the model on Google drive here: https://drive.google.com/file/d/1mG7B_5HyQCaKOWWhuus7k5QUoOLku8_1/view?usp=sharing

Download the model from there and place it in the models folder of the light_classification package within the tl_detector package.

This is the project repo for the final project of the Udacity Self-Driving Car Nanodegree: Programming a Real Self-Driving Car. For more information about the project, see the project introduction here.

Please use one of the two installation options, either native or docker installation.

Native Installation

• Be sure that your workstation is running Ubuntu 16.04 Xenial Xerus or Ubuntu 14.04 Trusty Tahir. Ubuntu downloads can be found here.

• If using a Virtual Machine to install Ubuntu, use the following configuration as minimum:

  • 2 CPU
  • 2 GB system memory
  • 25 GB of free hard drive space

  The Udacity provided virtual machine has ROS and Dataspeed DBW already installed, so you can skip the next two steps if you are using this.

• Follow these instructions to install ROS

  • ROS Kinetic if you have Ubuntu 16.04.
  • ROS Indigo if you have Ubuntu 14.04.

• Dataspeed DBW

  • Use this option to install the SDK on a workstation that already has ROS installed: One Line SDK Install (binary)

• Download the Udacity Simulator.

Docker Installation

Install Docker

Build the docker container

docker build . -t capstone

Run the docker file

docker run -p 4567:4567 -v $PWD:/capstone -v /tmp/log:/root/.ros/ --rm -it capstone

Port Forwarding

To set up port forwarding, please refer to the instructions from term 2

Usage

1. Clone the project repository

git clone https://github.com/udacity/CarND-Capstone.git

2. Install python dependencies

cd CarND-Capstone
pip install -r requirements.txt

3. Make and run styx

cd ros
catkin_make
source devel/setup.sh
roslaunch launch/styx.launch

4. Run the simulator

Real world testing

1. Download training bag that was recorded on the Udacity self-driving car.
2. Unzip the file

unzip traffic_light_bag_file.zip

3. Play the bag file

rosbag play -l traffic_light_bag_file/traffic_light_training.bag

4. Launch your project in site mode

cd CarND-Capstone/ros
roslaunch launch/site.launch

5. Confirm that traffic light detection works on real life images

Results

The results of the project can be seen in the following video: https://youtu.be/BkCbJkp_KcI

The video is fairly long so watching it at 2x speed is probably preferable. The car was testing at various speeds up to 45 mph and still worked well for the most part. Once the car went past 40 mph there was strange behaviour when stopping and turning. This is likely due to the car having a fairly short lookahead horizon. Due to limitations with the workspace simulator I can only have the car look 50 waypoints ahead before there were noticeable performance issues. The higher speeds cause the car to run through these 50 waypoints significantly faster so the system can't respond fast enough. If I was able to run this on a local powerful machine I believe this limitations would be gone and I would be able to extend the lookahead range by quite a bit and handle higher speeds.

The car was only tested in simulation because it won't be run in the real world due to the pandemic. Because of this I only trained the traffic light classifier on simulated images so performance in the real world would likely be poor. The network could easily be retrained with a set of real world images or real world images mixed with simulated images and most likely obtain good results.

Acknowledgements

The walkthrough videos for the project were heavily referenced when I got stuck on certain topics or needed some guidance on the proper direction to go. I also referenced the student help board a few times when trying to workout why the simulator was running so poorly.