What, a billionaire that isn’t actually a technological genius? I know, shocking, right? It’s almost like billionaires make their vast mountains of cash from labour and market exploitation, not merit… But yes, Elon Musk is determined to make the same catastrophic yet straightforward mistake with AI time and again. Unfortunately, this is a huge problem, as he is pivoting Tesla from an EV giant to an AI platform that “brings AI into the physical world.” Yet, the reason Tesla’s FSD drives like a geriatric, drunken dementia patient, and Optimus makes C3PO look agile can all be traced back to this moron not learning the fundamental basics of the technology behind these systems.
Rodney Brooks has been calling out Musk on this topic for nearly a decade now, and if anyone is qualified for that role, it is him, as he is one of the world’s leading roboticists. In a recent essay, he criticised the humanoid robotics industry and its investors, pointing out that they are wasting their money and that this bubble is doomed to collapse. But his critique was targeted towards Tesla’s Optimus robot program in particular.
The problem? The data they are using to train the robots’ AI.
Companies like Tesla are training these AIs by feeding them videos of humans performing tasks, like folding clothes, with very little additional data, and Brooks simply pointed out that this approach is “pure fantasy thinking”.
Why? Because we humans use far more than just visual data to complete these tasks.
Brooks points out that we use touch significantly more than visual data to finish tasks. Our hands have 17,000 specialised, highly sensitive touch receptors capable of feeling changes as little as 40 μm (about half the width of a human hair), all working at around one billion bits per second. In other words, a single human hand feeds our brain over two gigabytes of highly detailed data every second! We need all of this fidelity, combined with remarkably dexterous and fast hands, just to do simple tasks. Without it, or with reduced sensation of touch, when we rely mostly on vision, things become much harder.
Yet robots like Optimus have hands with exponentially less touch fidelity. Not to mention that, while their hands are impressive, they still have significantly lower dexterity and speed compared to our digits.
This creates a huge problem.
The AI is being trained on an incomplete data set. As Brooks points out, “We don’t have such a tradition for touch data,” so we aren’t collecting that data for these AIs to be trained on, leaving the AI with a gaping blind spot.
Take the folding clothes example: we use touch to assess a material’s pliability, weight and texture, and that is what enables us to quickly and accurately fold it. An AI being trained to fold clothes solely based on visual data of a human folding clothes will not have access to the same data.
All an AI can do is recognise and replicate patterns in data. That’s it. It isn’t cognitive and doesn’t understand what it is doing. So, in this example of folding clothes, the data set is incomplete, making it difficult for the AI to interpret the actual pattern that is occurring. How can it accurately replicate it? It can’t.
Essentially, the training data is not sufficiently analogous. The human completing the task and making the training data has access to exponentially more data and higher fidelity data than what is being fed to the AI meant to replicate said task. They are making decisions on data that the AI doesn’t have access to.
This is why Brooks points out that this approach will make Optimus impossible to optimise, and the entire project doomed to fail.
Having appropriate training data is a basic tenet of AI technology. It is one of the first things you are expected to learn. So the fact that Musk has made this mistake, not just on this multi-billion-dollar AI project but on several others too, is utterly shocking.
Take Tesla’s FSD (Full Self Driving) autonomous driving system. Every other major self-driving system uses a plethora of different sensors to gain a clear picture of the world around it, but not FSD. Musk forced engineers to go with a computer vision-only approach, which means that the only way this system can understand the world around it is through nine video cameras dotted around the car. Why did Musk do this? Well, according to him, we mainly use vision to navigate the road, and so an AI should be able to do the same.
Frustratingly, human vision is not the same as a video camera. You see so much more!
A human eye has roughly 576 megapixels and over 20 stops of dynamic range (a measurement of how much detail can be seen in the shadows and highlights), with a variable frame rate of 30 to 60 frames per second. It also has incredible 3D information, not just from the binocular effect of two forward-facing eyes, but also from parallax, as we often move our head side to side to gain even more three-dimensional accuracy.
Compare that to the cameras Tesla uses, which have five megapixels, less than ten stops of dynamic range, and a limit of 36 frames per second. And while, yes, the nine cameras might be dotted around the car for the 360 vision, there is no binocular or parallax effect to gain accurate 3D information. 3D has to be inferred from a single camera feed.
Not only does Tesla Vision function in a totally different way from how we “see” the world when driving, but again, its fidelity is a joke compared to human vision. We can see critical details in highlights and shadows, like a cyclist coming out of a dark tunnel or the words on a bright sign in the sun, when all the Tesla Vision will see is a black and white image. Our higher resolution helps us pick out and identify complex objects, like establishing whether or not that blob on the side of the road is a broken-down motorbike that we need to avoid, when the only thing Tesla Vision will see is a jumble of pixels. Our highly accurate 3D sensors enable us to judge distances with incredible accuracy in an instant, while the Tesla Vision is still stuck guessing how close that oncoming vehicle actually is.
Still, Tesla trains its FSD AI on data collected from these cameras while its customers are driving on public roads.
Again, the drivers in this training data are making decisions based on micro-details that can’t be seen in the data used to train the AI. The AI can’t see the cause-and-effect pattern of the human driver’s thought process, so how on Earth can it replicate it accurately? It can’t. It won’t understand the patterns it is seeing, interpret phantom patterns that don’t really exist, or associate the wrong cause and effect, because it can’t see what caused the driver to make a particular manoeuvre.
Musk is trying to force AI to be more human, when it can never be. A self-driving car “sees” the world differently and has different driving capabilities than a human driver. So, its “thought process”, or the patterns it is replicating, needs to match these differences. In other words, the training data needs to be based on the data feeds the AI has available to it, not what a human has access to.
Now, FSD has a whole host of other nail-biting issues (read more here), but this training data issue is one of the big reasons why the current version of FSD can only manage 493 miles between critical disengagements, compared to Waymo’s 17,000. And that is the reason Tesla is so far behind in the robotaxi race.
Again, this is a basic tenet of AI. The only reason LLMs’ chatbot AIs are even passable these days is not due to access to more data or more data centres, but access to the correct kind of data prepared in the right way. You can’t just shove any old human-derived data into these things and expect results. But that is time-consuming, expensive, and limiting, as it means Musk will have to arduously create reams of training data in-house, rather than just stealing it from someone else, or half-arsing it. But the fact that Musk keeps making this same mistake, despite the experts’ warning him against it, not only shows that he doesn’t understand the basics of AI but that he is dangerously ignorant and swimming in his own Kool-Aid.
Thanks for reading! Don’t forget to check out my YouTube channel for more from me, or Subscribe. Oh, and don’t forget to hit the share button below to get the word out!
Sources: Tech Crunch, RB, Will Lockett, Will Lockett, Will Lockett, IE, Crunch Base, Jalopnik, The Guardian, Lasik, ARP, AAO, Healthline, PC Mag, BBC, BI, Tech News, ME
For some reason, this post isn't appearing in my Substack feeds, although I got the newsletter in my email. Don't know if this is just me, but all my other subscriptions are working fine
But it's not just Tesla. The entire US financial system, and therefore much of the world system, is now based on fantasy - crypto, AI revolution, MAGA economics. The end will be ugly, but who knows when it will come.