OpenAI's "Revolutionary" o1 Model Has A Fatal Flaw
OpenAI's "Revolutionary" o1 Model Has A Fatal Flaw
I was right; it's all just smoke and mirrors.
One of the biggest faults of LLM (Large Language Model) AIs is their hallucinations. These AIs simply can’t stop spouting demonstrably false claims. For example, Elon Musk’s own Grok AI has accused him of going to court for paedophilia, and ChatGPT has even said Musk has been dead for the last few years. So it was huge news when OpenAI announced that its latest model, o1, manages to solve this problem by fact-checking itself and spending more time considering all parts of a question. If true, this would be a monumental leap forward for the AI industry. So, how does it do this? Well, spoiler alert: it can’t.
I’ve already gone over o1, also known as “Strawberry,” and how it works (click here to read more). But the basic gist is that the model automates a fairly successful prompting technique known as chain-of-thought. This is where you ask the AI to explain each step of its logic. Now, this is anthropomorphising the machine a bit here, as it isn’t actually explaining to you what it is doing. Instead, it breaks up the question into steps and solves each step in a chain, hence the name. This gives the AI the ability to correct its hallucinations, as if a previous step contains a false fact or piece of logic, the AI gets a second chance to correct it in the next step.
Sounds like an elegant solution, right? A simple modification to the front end of the AI enables it to solve the most infuriating problem with AI chatbots.
Well, experts have now managed to get their grubby little mitts on o1 and examine this approach, enabling them to look past the marketing bumph. They found a serious problem with the model. You see, rather than helping the AI solve its hallucinations, it might actually make the problem worse!
With a normal chatbot, there is only one place hallucinations can arise: the generated content. But with a chain-of-thought model, there are actually three: the generated content, the steps in the chain-of-thought (which you get to see), and the chain-of-thought that is hidden from view.
That’s right, apparently, the chain-of-thought you get to see isn’t the one o1 actually used to process your prompt. Instead, it is more of a concocted summary. Why? Well, OpenAI has said that if they displayed the actual chain-of-thought, it would display the secret IP that helps o1 run. Which is not so open for a company called OpenAI.
This can make finding, diagnosing, and correcting hallucinations horrifically hard! Firstly, small false facts or logical flaws can be hard to spot, even if spelled out for you, and can have massively detrimental effects. So, even hallucinations in the displayed chain-of-thought can be hard to find and correct, both for the AI itself and the user.
But what about hallucinations in the hidden chain-of-thought? You can’t see them at all, and the summarised process to create the displayed chain-of-thought could gloss over or hide hallucinations in the hidden chain-of-thought, making it nearly impossible to identify and correct. This issue is compounded by the fact that recent studies have found that as LLMs get larger, they become worse at solving basic tasks like arithmetic. These are the kind of basic steps that would be in the hidden chain-of-thought and glossed over or summarised in the displayed one.
As such, o1 might actually do a better job at hiding hallucinations and make it harder to correct them than to fact-check itself.
Consequently, several AI experts have recommended being aware of this flaw and double-checking that there are no hints of hidden hallucinations in the displayed chain-of-thought. Some have even described using o1 as a “large leap of faith.” So, maybe, o1 isn’t the revolution OpenAI make it out to be.
Thanks for reading! Content like this doesn’t happen without your support. So, if you want to see more like this, don’t forget to Subscribe and help get the word out by hitting the share button below.
Sources: Tech Crunch, Forbes, SWW, The Information, Futurism