How Self Driving Cars Work

Just admit it, we are all waiting in anticipation for fully self-driving cars to become a reality. For those of us who grew up watching The Jetsons, our present commuting situation was supposed to look a whole lot different than it does now. So why aren’t we there yet? We have the code, the automation, the GPS, and robots, and cryptocurrencies, and unprecedented AI at our disposal. What is the hold up with self-driving cars? And how do self-driving cars even work?

With many self-driving success cases to back them, Google thought their self-driving Waymo would be ready by 2018, but it’s not. And Elon Musk wanted 2020 to be the year that Telsa released a self-driving car. But Autopilot, while functional, remains in the nascent stages of automated driving and is only fit for certain driving conditions. 

The partial answer to the question “Where is my self-driving car??!!”, is to understand how self-driving cars work or how they are intended to work. And what we need from the technology, traffic legislation, and infrastructure in order for us to see the proliferation of self-driving cars. 

What do cars need in order to be autonomous?

How self-driving cars works are that they need to be able to respond to the environment better than any human can. What does that mean? Well, when a human is driving, they are (or should be) constantly engaging with their environment in order to safely and appropriately responding to all of the objects and obstacles in its way.

So what does it take for an automated vehicle to do the same? Well, currently, with the existing limitations of self-driving vehicles, they have about 100 million lines of code and that number literally increased daily. 

For cars to be fully autonomous they rely on sensors, actuators, complex algorithms, machine learning systems, and powerful processors. 

That means that self-driving cars rely on an incredible amount of technology. To work safely, they require that the AI is able to: sense objects in the environment, gauge the distance and speed of the objects, acknowledge when sensors and cameras are disrupted by debris, discriminate between motorcycles and bicycles and animals, and humans.

The vehicle also needs to be able to take you to work when you ask it to, and not to the grocery store. That is, it needs to be able to follow instructions perfectly, and have strong GPS, which includes knowledge of main roads, back roads, residential roads, and construction zones. 

How do self driving cars work? It’s a moving question.

In short, self-driving cars need to be better than human drivers. And in order to do this, the technology needs to be incredibly robust. But more than that, we need roads and traffic laws that support the best use of this technology. And we need to navigate a world where human drivers interact with self-driving cars.

The rise of the robot cars: levels of automation

Another point that needs more clarity in understanding what self-driving vehicles are and what they need in order to work, is understanding the levels of automation. 

Tesla’s Autopilot, which I will explain in more detail shortly, still requires a fully attentive human. The Uber that struck and killed the Elaine Herzberg, the pedestrian who was walking her bicycle through a crosswalk, was a self-driving car with a human behind the wheel. Tragically, the driver was not able to override the vehicle, which fatally misidentified Herzberg in the crosswalk. 

The point is, there are different levels of human intervention that are either necessary or desired when it comes to automation. 

For clarity, the Society of Automotive Engineers (SAE) has defined 6 levels of driving automation ranging. They rely on levels that range from 0-5. Level 0, is a fully manual vehicle, while Level 5 is a fully autonomous vehicle. The description had been adopted by the U.S. Department of Transportation. 

Below is a depiction of the range of levels for driving automation:

how self driving cars work - levels of driving automation
Image from Synopsys

How do you drive an automated car?

Given the responsibility of a human driver, imagine what a machine driver must be able to do. Automated cars need to be able to take in information just like a human driver. In order to do so, they must create and maintain a map of their surroundings. Using multiple sensors situated in different parts of the vehicle, the vehicle reads the surroundings and must correctly categorize the objects in their environment.  

This also requires radar and lidar sensors to monitor the position of the vehicles in the surrounding environment. It also needs video cameras in order to detect and understand traffic lights, read road signs. It must also track other vehicles and their changing speed, and perhaps most importantly, correctly identify pedestrians. 

Reading the environment

To read the environment, automated cars use lidar, which is light detection and ranging sensors. Lidar uses bounce pulses of light off the automated car’s surroundings, which then measure distances, detect the hedges and ditches of roads, as well as understand and identify lane markings. There are also ultrasonic sensors in the wheels which are there to detect curbs and other objects in order for the vehicle to park itself.

The software of the vehicle must then correctly process all of the input it receives from the sensors. This information is used to chart a course to the desired location safely. Which means it sends information to the vehicle’s actuators. The actuators control acceleration, braking, as well as steering. 

Along with the information received from lidar, there are also hard-coded rules. Hard-coded rules are set with algorithms to avoid obstacles and use predictive modeling and object recognition based on previous data collection. This data helps the software follow traffic rules and navigate expected and unexpected obstacles.

What Are We Waiting On?

Right now Tesla’s Autopilot is available for use. However, it is NOT fully automated. It is somewhere between Level 2 and 3. A person is required and must have their hands on the wheel, and be able to override the system. Essentially, it is also only approved to be used on the highway, and not in pedestrian areas. 

Fully autonomous, Level 5, vehicles have been undergoing testing and trial uses around the world for about a decade now. The most tested data has currently been collected in California and Arizona. So there are many instances and test cases where fully automated vehicles were in use. However, they are still deemed not ready and therefore are not yet general public. 

In fact, many in the field suggest that we’re still several years away from the personal use of a fully self-driving car. There is still a wide range of challenges before self-driving vehicles will be available for regular consumer use. 

The challenges include the technology self-driving vehicles rely on, the environment and infrastructure for self-driving vehicles, as well as ethical concerns primarily related to safety and responsibility. 

how self driving cars work - technology is evolving

Breaking Down the Challenges

Let’s look at these issues more closely. 

Lidar and Radar

Lidar stands for Light Detection and Ranging. It is a remote sensing method that uses light in the form of a pulsed laser to measure the distance from objects to the Earth. Lidar is expensive and essential, so there is a fine balance between range and resolution. There is also the worry about multiple autonomous cars driving on the same road. Will their lidar signals interfere with one another? Also, if multiple radio frequencies are available, there is a concern that there will not be enough of a frequency range that will effectively support the mass use of autonomous cars.  

Weather Conditions

Changes in weather conditions may not only affect the ability of the vehicle to drive. But more importantly, they may affect the ability for cameras and sensors to track lane markings and traffic signs. (Does the vehicle recognize the stop-sign covered in snow? The one you know to stop for because your drive through the intersection every day). Self-driving cars still need to drive safely even when road markings are obscured by water, oil, ice, or debris.

Traffic Conditions and Laws

Present self-driving cars are more successful on the highway, that is because there are fewer changing variables. The main variable is that most cars are driven by humans. Until the day when the situation is the reverse, self-driving cars must consider the variability of human drivers and roads built for human-driven vehicles. 

State vs. Federal Regulation

Currently, legislation is decided state-by-state in the United States. To prevent “zombie cars,” cars driving around without passengers, lawmakers propose a tax. Other lawmakers have also proposed that all autonomous cars must be zero-emission, as well as the need to have panic buttons. However, if laws are different from one state to the next, it is hard to know how these vehicles will be able to travel effectively. 

Accident Liability

In fully autonomous cars there will be no steering wheel, so it is not clear that human drivers will be able to take over in the case of an emergency. In that case, where the vehicle is driving itself, who becomes responsible for accidents? Will the manufacturer or the passenger be responsible?

Artificial vs. Human Intelligence

One of the serious issues is the question of whether or not humans or machines make better drivers. One argument that humans are better drivers is that they are able to read ambiguous information from their surroundings, such as eye contact and facial expression from pedestrians, which allows one human to predict another human’s behavior. It is unclear if machines will be as effective at reading human behavior and then acting out of the best interest for everyone’s safety.

Models in the race for self driving cars

There are several car manufacturers investing millions in research and development for its own version of an autonomous car. As I have said, no one has a fully functioning autonomous vehicle yet (level 5), but many are getting close. That is, no one has a fleet of level 5 vehicles ready for regular consumer use. 

Right now the best autonomous vehicle for purchase is Tesla’s Autopilot because it requires fully human intervention. Many users are very pleased with the success, but a driver is still absolutely necessary. And, it is primarily made for highway driving, as there are still too many variables in other driving environments. 

So here is how Autopilot works.

how self driving cars work - Tesla's Autopilot


With multiple sensors around the vehicle, Tesla’s Autopilot system uses these sensors to help the car understand its environment. The car can then safely steer itself in most highway situations, responding to other moving vehicles and changing lanes effectively. 

Included in the hardware of Autopilot is a forward radar, as well as a forward-looking camera, a high-precision digitally-controlled electric assist braking system, and 12 long-range ultrasonic sensors placed around the car. The sensors are placed around the vehicle. The sensors are able to sense 16 feet around it in all directions, at all speeds.

The forward-facing camera is located on the top windshield, and it relies on a computer inside the camera that helps the car understand the obstacles which are ahead of the car. On top of that, radar enables the detection of cars and other moving objects.

How does Autopilot Activation work?

Activation of Autopilot works by simply pulling the cruise control towards the driver twice. Then the automation takes over steering and braking. And then to disengage Autopilot, the driver simply pushes the button on the end of the cruise control stalk, or pushes the stalk forward, or presses the brake. Autosteer is also disabled by manually turning the wheel. 

In Autopilot the vehicle drives itself, so it can steer in the lane, change lanes, change speeds, as well as brakes appropriately. There is also a control panel that shows the driver what the car is reading as traffic, obstacles, and lane markings.

Tesla is constantly learning from its fleet, and so the information is aggregated to use to improve driving. These updates are continually added to the vehicle to improve the system, just like your phone does.

“The driver is still responsible for, and ultimately in control of, the car. What’s more, you always have intuitive access to the information your car is using to inform its actions.” 

Says Tesla representative in a statement when Autopilot was first released.

how self driving cars work - Waymo

Who is WAYMO?

As Google’s arm of self-driving research and development, Waymo has launched actual autonomous rides in Arizona. But Waymo is also a specific vehicle:

“Waymo stands for a new way forward in mobility. We’re a self-driving technology company with a mission to make it safe and easy for people and things to move around. We’re determined to improve transportation for people around the world, building on technology developed in Google’s labs since 2009.”

In the test drives in Arizona, all vehicles have human drivers in the front seat as a safety precaution. However, Waymo has successfully tested fully autonomous cars, but when this product will be available for consumer use it is unclear.

Like all other self-driving vehicles, they rely on lidar, radar, and multiple cameras. Waymo is primarily software, which means it is developing the software for car manufacturers. Waymo began to develop its own hardware and sensors in-house in 2011. This was after Waymo determined that no other vehicle on the market provided “level 4” autonomous functionality.

Many believe that Waymo is the closest to achieving a truly self-driving vehicle for popular use. 


GM is another strong competitor for self-driving vehicle development as the company has billions of dollars in investments from SoftBank, Honda, GM, and T.Rowe Price Associates. Cruise also has 180 vehicles in testing, which is the second largest number of vehicles in testing. 

Like its competitors, the Cruise vehicles use machine learning techniques, cloud-based tools, and IoT sensors to gather data. The data is aggregated to program the cars to learn about its surroundings and then to make intelligent decisions.

Despite the fact that GM is in an excellent position, given it has a strong place in the automotive world, like others, the real launch of their self-driving vehicles is unclear. Everyone has been a bit too ambitious with their real launch of self-driving cars. And who can blame them? With so many pieces of the puzzle coming together, it seems like we could be there any day now.

Given their status in the world of automotive manufacturing, GM is in a great place to manufacture self-driving cars. General Motors positions Cruise to manufacture self-driving cars on an assembly line in Orion, Michigan, which means the company is capable of producing hundreds of thousands of vehicles per year.  


Argo AI is another strong competitor in the ring. They are an independent American company that started in 2017 and claims that they have been able to make progress faster than others. This is because they are focused on the self-driving system. So they have left car manufacturing and customer services to their partners.

Since they began, they have been able to secure a $1 billion investment from Ford Motor Company and Volkswagen had recently joined the partnership. With VW, Argo is looking at acquiring another $2.6 billion investment in the company. 

Nick Twork, senior communications counsel at Argo AI:

“What’s unique about Argo is that we’re developing the self-driving system. We are not developing a car, we’re not going to be the operators of the services. That’s something that our partners will be doing on their own or in conjunction with other partners potentially…In a very short period of time, we’ve been able to basically put the system at a level of maturity far beyond what other companies of our age have been able to do”

Twork asserts that Argo is much more focused than its competitors, leaving the rest to the car and customer service experts. 

Why we need autonomous cars

The concerns and limitations of self-driving technology are real. Coordinating the technology with the existing driving conditions is no small challenge. Not to mention the amount of legislation that will have to be in place before fully self-driving cars are able to take us from A to B.

But once the self-driving car is a reality, and there are more than just a few thousand users world wide, the benefits are literally limitless!

Mobility for everyone will increase. Particularly for those who suffer the most from limited mobility, such as the elderly, the disabled, and those remotely located. Not to mention the benefit of the convenience of cars that can run personal errands for you, taking door-to-door delivery to a new level of efficiency. 

When driving autonomously becomes safer

Moreover, while the main concern for the safety of self-driving cars is legitimate, once the technology has improved, driving will actually become much safer. No longer will we need to worry about driving impaired, or new drivers, or aging drivers. The machines will learn from each other and respond safely and appropriately. 

To put things in perspective, humans are not great drivers. Based on a current statistic, about 3500 people are killed every day in vehicle accidents. And these accidents are caused by drivers, not by their vehicles. 

Helping the environment

One of the most promising outcomes of self-driving cars is that they are poised to dramatically lower CO2 emissions. They will also ease the problem of commuter sprawl. One study found that the best case use of self-driving cars could to the following:

  • Reduce traffic congestion with 30% fewer vehicles on the road, as the need for personal vehicles will decrease.
  • Reduce transportation costs by 40%, as the cost of vehicles, fuel, and driving infrastructure will decrease.
  • There will be less need for more roads, and parking lots. This will free of valuable land for other important needs, such as housing or agriculture. 
  • Overall, it could reduce urban CO2 emissions by 80% worldwide. 

While eliminating vehicles from the road, and combating urban sprawl are important ecological issues, imagine the reduction in human hours spent on driving and communicating?

The future of self-driving cars

Just imagine a world where you could relax, work, or sleep while you drove for hours and hours. What would we all do with our new found time?

In the meantime, while we wait on the technology, the legislation, and the infrastructure, it is more likely that we will start to see public and mass forms of self-driving transportation occur faster. This is because the technology is still very expensive. So, personally investing in a self-driving car, even a Tesla is around $200,000 USD. 

That means that it will be more reasonable to expect cities and governments to invest in the technology first. Which could mean that we will see self-driving public transportation proliferated before we see more of Tesla’s vehicles on our roads. 

“The technology is not done. It’s not ready for commercial applications. The technology simply does not yet exist to support completely autonomous vehicles, which can navigate highways, congested metroplexes, or harsh driving conditions.”

Says Twork.

But software and AI and car manufactures are getting close. And so while it is still unclear at what point exactly we will see can expect to see self-driving vehicles, it is very likely that within the next 10 years we will see an increase in autonomous-type vehicles. 

And then we will all be able to live out our Jetsons or Star Trek fantasies. Or, just take a nap on the way to work – that would be nice, too. 

crypto social trading