AI techniques of Tesla's self-driving cars

2022, May, 31

Explainer

Self driving cars need to know about all the objects around them.

What type of object is it ?

How far is it ?

There are few techniques for this:

  1. Light Detection And Ranging (LiDAR)
    • It uses low wavelength lasers that bounces off objects near by to provided distance from them.
  2. RADAR
    • Similar to LiDAR, but instead of lasers it uses radio waves to know the distance.
  3. Computer Vision
    • It uses inputs from cameras to gather data and do object detection via neural networks.

Most self-driving car manufacturers, like Alphabet's Waymo and Uber use a combo for all of these.

But Tesla is only relying on Computer Vision.

So Why Tesla has choosen 'Vision only' approach ?

  • Sensors for LiDAR & RADAR are expensive.
  • Specialized sensor hardware adds extra weight to the electric car which reduces it's range.
  • Camera hardware setup needed for 'Computer Vision only' approach is cheaper and simpler.
  • Humans have been driving by relying only on visual input, so Tesla believes that a system could be created which can do the same.

Let's try to understand some of the techniques Tesla is using.

  • Tesla cars have 8 cameras that cover 360 degrees of view.
  • Each camera has different caliberation (angle, height, field of view, etc.). These caliberation parameters of each camera are used to adjust the video feed and then it's send to neural network. This ensures that input data of the neural network is uniform.

8cams

  • Neural Network stitches the video streams together to create a 3D vector space map of the surroundings in real time.
  • For example, in this image there is a large trailer which appears in video feed of 5 cameras but neural network needs to detect this as a single object.

cam_joining

  • Tesla uses a processing architecture called HydraNet. It has a common backbone and multiple heads. Each head is an independent model. For example, there is a model for traffice light detection, pedestrian detection, etc.

hydranet

  • Each Tesla car has two independent processing chips on board. Cars can continue to drive even if one of the processing unit stops working.
  • When any Tesla car faces any unusual situation and it's (Fully Self Driving)FSD system does not function well, the data from it's cameras is used to create additional training scenarios for the models.

Reference