Read on Twitter
It’s that time again!Let’s dive into the latest release of @Tesla FSD (full self-driving) with a rather eventful trip through midtown Sacramento.
Thread...
1. Before we get started, here’s a refresher on two important differentiators for FSD.
These differentiators will (eventually) help FSD scale, but they also make the self-driving problem significantly more complex.
In this thread, we’ll explore examples of this. https://twitter.com/olivercameron/status/1319835517094014977
These differentiators will (eventually) help FSD scale, but they also make the self-driving problem significantly more complex.
In this thread, we’ll explore examples of this. https://twitter.com/olivercameron/status/1319835517094014977
2. Let’s talk roundabouts.
Roundabouts are not yet consistently handled well by FSD.
See here how FSD appeared to attempt to drive through the center median, requiring intervention.
This is why drivers need to always be alert with their hands on the wheel.
Roundabouts are not yet consistently handled well by FSD.
See here how FSD appeared to attempt to drive through the center median, requiring intervention.
This is why drivers need to always be alert with their hands on the wheel.
3. My theory is that FSD did not reliably detect this as a roundabout in real-time.
Note how the center circle is properly outlined, but its colored red (like lane lines).
In other roundabout clips, that center circle is colored purple.
Note how the center circle is properly outlined, but its colored red (like lane lines).
In other roundabout clips, that center circle is colored purple.
4. It appears FSD has different behaviors if a roundabout is missing in Google Maps, like in this clip.
With no roundabout noted in Google Maps, FSD appears to handle the roundabout a tad fast and furious.
I am curious to see how FSD handles other situations with stale maps.
With no roundabout noted in Google Maps, FSD appears to handle the roundabout a tad fast and furious.
I am curious to see how FSD handles other situations with stale maps.
5. In most cases, roundabouts appear to be handled very cautiously.
In some cases, like this clip, too cautiously (requiring intervention).
In some cases, like this clip, too cautiously (requiring intervention).
6. Here’s another example of very cautious navigation around a roundabout.
7. Let’s talk about decoding road-rules in real-time, instead of utilizing a high-resolution map with rules pre-encoded.
Here, FSD needs to merge, but for some reason does not attempt to until it’s too late (requiring intervention).
This is why drivers always need to be alert.
Here, FSD needs to merge, but for some reason does not attempt to until it’s too late (requiring intervention).
This is why drivers always need to be alert.
8. Note the road ahead is closed in this clip.
The driver is manually driving here, but FSD appears (if you look at the viz) to want to navigate through the road blocked by the fence.
If left to its own devices, would FSD have driven to the fence, or would it re-route?
The driver is manually driving here, but FSD appears (if you look at the viz) to want to navigate through the road blocked by the fence.
If left to its own devices, would FSD have driven to the fence, or would it re-route?
9. Inferring lane lines is one of the strengths of FSD, but here’s an example where it struggles.
FSD cuts very close to the curb while navigating roadway with light rail tracks.
Without a pre-recorded map, this will continue to be a challenge.
FSD cuts very close to the curb while navigating roadway with light rail tracks.
Without a pre-recorded map, this will continue to be a challenge.
10. FSD appears to stop quite far back at certain unprotected turns.
Why?
For a robot to infer, in real-time, the most useful position for the best visibility is very challenging. At @voyage, we encode this starter position into the map (and nudge if need be).
Why?
For a robot to infer, in real-time, the most useful position for the best visibility is very challenging. At @voyage, we encode this starter position into the map (and nudge if need be).
11. Such a task (choosing where to stop and wait) requires a high-level of reasoning, to understand what is occluding your view and if you can safely move forward.
Here is another example of this, eventually requiring intervention.
Here is another example of this, eventually requiring intervention.
12. Each of the above challenges were sourced from this video, which is from around 24 minutes of driving.
Watch the full video for other, less interesting interventions.
Watch the full video for other, less interesting interventions.
13. That said, there is a lot to admire about the technology behind FSD.
I remain impressed with the ability of FSD to understand the drivable surface ahead (i.e. free space). It is likely the best algorithm of its kind in the world.
I remain impressed with the ability of FSD to understand the drivable surface ahead (i.e. free space). It is likely the best algorithm of its kind in the world.
14. I also remain impressed with their traffic light recognition. I haven’t seen any red lights blazed (yet), and they appear to detect traffic lights (and the signal) from quite a distance.
15. I outlined a few other impressive areas in prior tweets.
There were fewer wow-worthy clips this time around, but that doesn’t make the system any less impressive. Boring is good. https://twitter.com/olivercameron/status/1319835688460640256
There were fewer wow-worthy clips this time around, but that doesn’t make the system any less impressive. Boring is good. https://twitter.com/olivercameron/status/1319835688460640256
16. After reviewing this and many other trips, I still struggle to see FSD evolving into a system requiring no driver intervention in months, not years.
I think it is likely we have to settle in for the long-haul.
I think it is likely we have to settle in for the long-haul.
17. I often hear that @Tesla’s data advantage will enable it to solve the self-driving problem far faster than others.
If that were the case, given the many, many years of data @Tesla has collected, you have to ask yourself why the system does not yet reflect that advantage.
If that were the case, given the many, many years of data @Tesla has collected, you have to ask yourself why the system does not yet reflect that advantage.
18. That said, if we can be patient, and if FSD is tested responsibly (a big if), I see no reason why we cannot be excited about what’s to come.
This technology will eventually save lives, and that is something we can all agree is a good thing.
This technology will eventually save lives, and that is something we can all agree is a good thing.
