Inspired by AGI Friday self driving tesla reports.
I previously had the Comma 2 “developer kit” which shipped a cheap smartphone stripped of a battery and strapped to a 3D printed case. You would mount this dead center in your windshield, and through the usage of a device called a panda, the Comma 2 would replace CAM bus signals sent by your car’s native lane keep assist cruise control system with commands sent by the Comma 2 through a man in the middle attack. I luckily had (and still have) a toyota corolla hatchback 2019 which was sufficiently advanced enough to have a native cruise control and lane keep functionality that both supplied high torque for autonomous wheel steering, and capability of having cruise control work at speeds as slow as zero miles per hour.
Later toyota models improved the security on their CAM busses, so later model years are incompatible with Comma and the Openpilot software that runs on them.
In actuality, there’s three self driving or driver assist packages I’ll be assessing today.
A) Stock - Toyota Safety Sense 2.0
TSS 2.0 ships with four standout features that I have used over my years of owning the car.
i) Emergency braking - doesn’t work when you’d expect it to, but saved my butt at least twice over my years of ownership.
One thumb up. Not good, but better than not being there.
ii) Passive steering assist - it is always detecting lane lines and helps correct the wheel steering if you accidentally drive over a line
One thumb up. Lane line detection sucked and sometimes it nudges you in the wrong direction, especially exits if you forget to signal.
iii and iv) radar cruise control and lane keep assist - radar cruise control keeps a set distance to the vehicle ahead, and uses the camera to help keep the car in the lane. Requires you to have hands on the wheel at all times, but IN THEORY you could let go of the wheel for about five seconds and be fine LIKE MOST CARS but the lane keep assist is so crappy that it barely does anything most of the time - it just reduces fatigue compared to steering by yourself.
iii) two thumbs up. the radar cruise control tuning by either mobieye or those japanese engineers is just m’wah chef’s kiss. they know how to make a radar cruise control that’s highly comfortable and keeps a great distance to the car ahead. a lifesaver for long journeys.
iv) sucks. you can’t rely on it. it reduces fatigue, but even in common situations like GOING STRAIGHT this has noticeable “ping pong” between the lane lines, like a - not this extreme, but like a bowling ball bouncing between bumpers. Better than nothing. Reluctant one thumb up.
v) distracted driver prevention
It just forces you to hold the wheel at all times. If it doesn’t detect input for ten ish seconds, system disengages.
overall system
B) Comma Two
Comma two keeps stock i) in place, in theory, but disables ii) for unassisted driving, and replaces iii) and iv) with openpilot, though I’ll break it out into different categories.
Context: My commute was three hours in total round trip, so 1.5 hours each way. I would get on a road, especially a highway, turn on openpilot, and the car would drive itself ON THE HIGHWAY as like a computer assisted version of cruise control that was good enough you could take your hands off the wheel (which most openpilot users do) entirely and let the vehicle drive itself.
iii) Openpilot Cruise Control (aka, MPC default longitudinal)
The Comma two’s cruise control evolved significantly over the course of the time I owned it. First, openpilot used radar only to determine distance to the vehicle ahead. However, since every openpilot vehicle sends training data to comma.ai by default, openpilot later shipped vision based lead car distance detection, which became the default, and radar was DISABLED. I will talk about the radar disabled version.
Openpilot cruise control has two sides. First, lead vehicle detection.
Vehicle detection was good. It could detect stopped vehicles (with vision) and stop for them. Toyota safety sense did not detect stopped vehicles, as radar had no easy way of distinguishing stationary cars from stationary objects, and thus only tracked moving objects. Comma’s switch to vision only lead car detection made perfect sense in this context, whereas previously before with the radar based approach, the camera I believe was a fallback for when radar failed to detect a stopped car. The switch from radar to vision also enabled openpilot cruise control for non radar equipped cars.
However, it detects lead cars at a shorter distance than radar did. Which lead to problems with the second part of openpilot cruise control, MPC.
MPC (model predictive control) represented openpilot’s largely handcoded decision tree for the car’s cruise control behavior. It was… good enough, but because of the poor lead car detection, drove like a jerk, accelerating before suddenly stopping. I have to note, it did this in the most odd, computer way possible, as the MPC system HIGHLY prioritized reducing jerk, or change in acceleration, so it felt like you were on some futuristic smooth butt sled from the jetsons as your self driving toyota corolla brake-checked someone on the most dangerous interstate in the united states. (I-4)
The brake checking occured because the MPC system was eager to reach a top speed defined by the user using the standard cruise control input buttons (you enabled openpilot by using the same buttons in your toyota that you would use to enable stock cruise control) but higher speeds meant less time to detect an oncoming car using vision, so when the vehicle travelling at 70mph saw a vehicle in front going at 50, it would suddenly stop SOOOOOO SMOOOTHLY brake checking whoever happened to be following you as your accelerated forward. The combination of smooth braking and speed was genuinely inhuman and made me ignore how much of a jerk the handcoded driving policy ended up unintentionally being.
The MPC also did not stop for traffic lights or stop signs, but neither did TSS which was explicitly designed as a cruise control system, so.
One thumb up. It was smooth and good for the car occupant, but not great.
Because comma two ran open source software, some people used forks that preserved TSS cruise control but used openpilot only for steering. I never resorted to that but did install a fork after Comma 2 stopped receiving updates, which brings me to my next point…
iv) Openpilot Steering (aka, lateral)
This had two variations, both useable.
- Laneful
Uses computer vision to detect lanelines, and anchors your car to the dead center of the lanelines. It was SOOOOO precise that it felt like you were riding a rollercoaster or a tracked vehicle like a train. Laneful openpilot was a revelation, and even worked in the abscence of lanelines, like in construction zones.
- Laneless
A fully end to end ai system which simply took a video feed from the front camera and outputted a steering angle, without any intermediary hand labeling. Not as good as laneful, even though it could in theory handle more situations. Prone to drifting out of lanes, but significantly better than toyota safety sense. Still good enough for most users to take their hands off the wheels.
v) Distracted driver detection
Openpilot is good enough that most users take their hands off the steering wheel entirely. For legal reasons, openpilot and comma ai don’t advise this, but honestly ya. The system is that dang good, even in the comma two iteration. However, Comma Ai didn’t want blood on their hands, so they developed a driver monitoring system that checked that your eyes were open and you were staring straight ahead. It was very, very strict. If you wore polarized sunglasses that blocked its ability to see your eyeballs, it wouldn’t let you activate the openpilot system AT ALL.
It was supposed to stop openpilot if the user closed their eyes, were not looking mostly straight ahead, even if openpilot was already engaged.
I’ll give it two thumbs up as it caught me falling asleep at the wheel at 3 AM once and I decided to pull over and consider what I was doing with my life that I was falling asleep at 3 AM. The auditory alerts it gives are VERY LOUD and the system was extremely strict.
Overall system rating - thumb and a half up.
Caveat - Florida is hot. So hot. So I would remove my comma two from the dashboard daily to keep it out of the heat, because if the SMARTPHONE IN A 3D PRINTED CASE overheated your self driving system was not usable for a half hour while it cooled down. The comma two was mounted to the dashboard via 3M VHB. So I removed and reinstalled it daily. One day while the car was driving THE MOUNT FELL OFF and I rapidly unplugged the comma and took over control because the driver monitoring system had trained me well to always be staring straight ahead. But yeah, THAT HAPPENED. It didn’t happen again, but after a few months, my comma failed to boot, and I relegated it to my discarded tech heap. I think the device saved my life at least twice and made my three hours of driving daily usuable, so despite that I still rate this device highly.
I believe the failure was from the fact that VHB is highly built for shear forces, forces parallel to the mount (being mounted on a windshield, most of the force is being done along the correct axis for the tape) but not peel forces, which the daily removal and attachment probably weakened the mount and caused it to fail.
On a more sentimental note, I drove my date home from my first kiss using this device. So I will always have a fondness for my comma two. Despite the fact that I traded it in for the
C) COMMA FOUR
I skipped Comma 3 and 3X because I was poor, but now I have money again and oh boy. Tiny device, dedicated cooling system, injection molding, custom boards, this thing looks like a real product, and apparently their MTBF has lowered to the point where you would feel comfortable having this drive your car, wow… (Comma 2 MTBF was like one hundred hours, which I had had the comma 2 device crash on me mid drive.. at least twice. Maybe three times. Comma 3 brought it up to one thousand hours iirc, and I assume the numbers have just gotten better with each generation, though automotively speaking you want your MTBF to be much, much higher. No idea what it is for the four.)
The comma two was sketchy as fluff, friends. But this thing? oh boy.
iii) Openpilot Cruise Control longitudinal
- MPC cruise control (lead car detection + handcoded driving policy)
- This doesn’t feel like it’s improved, though in theory stopped car detection should have been improved. I thought it felt worse at first, but then I realized toyota safety sense which I had been using for a long time had a longer detection radius for MOVING vehicles.
- End to End longitudinal (fully AI velocity control)
- OH BOY. This got good. This got really good! You just feed it a video feed and the model outputs a velocity directly. It drives so well. It drives fantastic! It drives like a safe grandma. Absolutely fantastic. Blows TSS out of the water, as this can detect stopped cars from way farther away than TSS and MPC based lead car detection. It just works. The only nibble is that it takes a while to accelerate, but tbh I mean - this just slaps. Absolutely dashtastic.
Two thumbs up. What I always wanted. The latter is still underneath the experimental branch, but I’m such a happy camper.
iv) Openpilot lateral
Laneful has been removed, and laneless is now default. It is now much better at highway situations and stays within the lines pretty much always, unlike laneless from comma two days which drifted out of lanes. However, it still sucks at windy roads which comma two laneful was good at.
Overall, I’d give it one and a half thumbs up. It does great in everyday driving, I just wish it could handle curvy roads better. I WILL SAY, it can now handle the curvy road outside my house that starts my commute everyday that the comma two did not, so that’s a definite improvement! but it’s not quite there yet.
v) driver monitoring
Less strict. I described comma two driver monitoring to a friend as a fussy, concerned mom. I have actually yet to have had the car trigger an alert for me yet.
Overall, two thumbs up first impressions. Best in class outside of a tesla or waymo, does everything you need except windy roads or any kind of navigation. But fully AI end to end cruise control kicks absolute bass. I love it!