Even AI Companies Know Their Models Can't Be Trusted
It's all hype, no trousers.
You could fit every techbro CEO’s colossal, distended ego inside the gaping chasm between what AI promised to do and reality. Every now and then, we get a perfect snapshot of this discrepancy, and internet sleuths have just provided us with another one. Microslop has been shoving Copilot down users’ throats for a while now, heavily pushing it as the future of professional productivity. However, outlets like TechCrunch, TechRadar, PCMag, and others recently broke the news that Microsoft’s Terms of Service (ToS) state that Copilot is for entertainment purposes only, sparking a tsunami of online derision. Unfortunately, this is so much worse than people realise, because this isn’t just limited to Copilot, and the implications of this run significantly deeper than you might think.
The Terms of Service
To use Copilot, like most modern software, you have to agree to its Terms of Service (ToS). So, let’s start with what Copilot’s ToS actually says. In the “Code of Conduct” section, it states that “Copilot is for entertainment purposes only. It can make mistakes, and it may not work as intended. Don’t rely on Copilot for important advice. Use Copilot at your own risk.” In other words, Copilot isn’t accurate or reliable enough to be trusted with even remotely important tasks or as a source of information, meaning it should be treated more like a toy than a tool.
But Microsoft also doubled down on that “at your own risk” statement. In the “Important Disclosures And Warnings” section, it states that “You agree to indemnify us and hold us harmless (including our affiliates, employees and any other agents) from and against any claims, losses, and expenses (including attorneys’ fees) arising from or relating to your use of Copilot, including without limitation your use, sharing, or publication of any Prompt, Responses, or Creations, or your breach of these Terms or violation of applicable law.” In layman’s terms, if you use Copilot as a tool and not just for entertainment (you know, like how it is advertised), then you can’t hold Microsoft liable for any damage it may cause to you personally or the business you work for.
So, let me get this straight — this tool, which has been slated as the next big thing in professional productivity, is so catastrophically unreliable that Microsoft had to not only explicitly state it is just for entertainment purposes but also force users to completely surrender any right to hold them accountable for damages caused by using this unreliable, inaccurate AI as a professional productivity tool. That doesn’t seem right, does it?
Some have pointed out that these ToS only cover Copilot as a personal chatbot and don’t apply to Copilot business tools. But that isn’t really a defence. For one, many businesses still use these “individual” versions of Copilot as a chatbot assistant. Secondly, people also use Copilot as a professional assistant chatbot in both their personal and professional lives, as that is essentially how it has been advertised. But finally, these Copilot business tools are based on the same AI models, so they will have similar issues.
And here is the thing: it isn’t just Microslop pulling this legalese bulls**t.
Anthropic, the less-evil AI company, is also doing this. Their consumer ToS (when viewed from a European IP) states that their services are “non-commercial use only” and that users “agree not to use our Services for any commercial or business purposes and we (and our Providers) have no liability to you for any loss of profit, loss of business, business interruption, or loss of business opportunity.” I wonder how many developers vibecoding with Claude Code know that.
To be fair to Anthropic, they are still better than Microsoft. They don’t completely retract any liability they have for damage their models may cause — but they do heavily restrict it. The ToS states that Anthropic’s “total liability to you for any loss or damage arising out of or in connection with these Terms, whether in contract (including under any indemnity), tort (including negligence) or otherwise will be limited to the greater of: (a) the amount you paid to us for access to or use of the Services in the six months prior to the event giving rise to the liability, and (b) £100.”
This is the exact same schtick Microsoft has pulled. The AI is so inaccurate that it can’t be trusted with meaningful tasks, so they claim it can’t be used for commercial purposes (even though that is primarily how it has been marketed) and then retract as much liability as possible for their AI’s actions.
OpenAI is not much better.
Their ToS does not contain an agreement not to use their services commercially. But it probably should, as Copilot is practically just a reskinned ChatGPT. Therefore, this lack of a disclaimer can more likely be attributed to Sam Altman’s far looser and riskier “break things and move fast” mentality than an improved product.
What it does include is a giant section in which it painstakingly tells users not to trust the output of their AI. For example, one section reads, “Output may not always be accurate. You should not rely on Output from our Services as a sole source of truth or factual information, or as a substitute for professional advice.” It goes on to say, “You must evaluate Output for accuracy and appropriateness for your use case, including using human review as appropriate, before using or sharing Output from the Services.” In other words, it shouldn’t be used to automate anything professional, and its output should always be reviewed by a human. That is so close to saying “not for commercial use” without actually saying it.
Also, they fully remove any liability they might incur for their AI’s actions. The ToS states, “NEITHER WE NOR ANY OF OUR AFFILIATES OR LICENSORS WILL BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, OR EXEMPLARY DAMAGES, INCLUDING DAMAGES FOR LOSS OF PROFITS, GOODWILL, USE, OR DATA OR OTHER LOSSES, EVEN IF WE HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.” And you can tell they really meant this because they literally wrote it in block capitals.
These AI companies are trying to have their cake and eat it too. Supposedly, these AIs are so good that they can replace workers and take businesses to the next level, which is one of the main pumps inflating the AI bubble. Yet they still have to warn users not to trust their AIs, as they are dangerously inaccurate, as well as explicitly rejecting any liability for the tools they release into the world. It’s either one or the other — you can’t have it both ways.
The Liability Paradox
All three ToS documents say virtually the same thing. Firstly, the AI is unreliable and can’t be trusted. Secondly, it can and will cause damage if unchecked, and these companies won’t take liability for such damage. And thirdly, these AIs need to be micromanaged by humans checking every single output to work.
This creates a bit of a paradox. AI is meant to be a productivity tool, capable of augmenting or automating tasks and jobs. But, if it is such an inaccurate liability that a human has to micromanage it, how on Earth can it boost productivity?
I discussed this subject in a previous article, but a number of studies, such as this one from the University of Melbourne, demonstrate that AI is too unreliable to be a productivity tool because it requires more time to micromanage the AI in order to catch and correct its constant errors than it saves.
This Can’t Be Solved
So, what are the implications of all of this?
Well, let’s start with the fact that we knew this was the case, and it isn’t going to improve.
Take this study from Carnegie Mellon University, which found that even the best “agentic” AIs fail basic tasks 70% of the time. Or, what about this study that found that the best current AIs failed 97.5% of realistic real-world freelancing jobs given to them due to AI hallucinations and total failures? We know that these AIs are just statistical models; they aren’t thinking machines, meaning they will statistically get things wrong and hallucinate very frequently. We shouldn’t be surprised that AI companies’ small print warns against using their models commercially and washes their hands of any meaningful liability. Everyone knows, including AI companies, that these models are dangerously inaccurate.
But the promise that these AIs may eventually be good enough is what is driving the currently insane levels of investment in AI. In order to produce AI models capable of replacing human workers, hyperscalers are literally pouring hundreds of billions of dollars into building larger data centres and providing AI models with more compute power.
Only, even OpenAI knows that isn’t going to happen. They published a study last year that found that increasing the computing power behind AI, or providing it with more data, can’t reduce AI “hallucinations” (or error rate) below its current level. In fact, they found no viable way to reduce AI hallucinations, which strongly implies these models are doomed to remain as unreliable as they currently are. A more recent paper by Vishal Sikka and his son Varin Sikka has supported this idea, revealing that AIs are mathematically incapable of being reliable or carrying out computational and agentic tasks beyond a certain complexity.
So, what does that say about the hundreds of billions of dollars being poured into AI to “unlock” it?
The Big Implication
Quite simply, all of these factors point to generative AI being a dead-end economic bubble. The hype and investment are not at all reflected in reality. These ToS documents prove that these AI companies are, at least to some degree, aware of this discrepancy. They know their models aren’t good enough. They know they can and will cause damage. They know they aren’t going to meaningfully improve. Yet, rather than stopping the madness, they are actively attempting to protect themselves while still forcing the hype and inflating the bubble, all to line their already cavernously deep pockets. This is a tale of greed and deception, and we all know how it will end.
Thanks for reading! Everything expressed in this article is my opinion, and should not be taken as financial advice or accusations. Don’t forget to check out my YouTubechannel for more from me, or Subscribe. Oh, and don’t forget to hit the share button below to get the word out!