"AI Inbreeding" Is Making An Already Bad Problem Even Worse
An ouroboros within an ouroboros.
An ouroboros is an ancient symbol of a serpentine dragon eating its own tail, which is meant to depict the cyclical nature of destruction and rebirth. In the modern world, however, its meaning has shifted to a shorthand for self-destruction. A system so flawed it eats itself alive until nothing is left. It’s easy to see why AI has been labelled an ouroboros. These machines need to consume our data to exist, yet they are simultaneously expected to be existential threats to us and our livelihoods. But AI’s self-consuming nature goes far, far deeper than this, and it could have horrific consequences.
This problem all starts with data.
Modern LLMs require an unfathomable amount of data to function. As such, an entire data economy has been set up to scrape, gather, sort and label every conceivable kind of human-generated data out there for these models. This operation collates practically the entire internet and offers it up to train AIs on a colossal scale.
But the pool of data has dried up. This isn’t a surprise; three years ago, researchers predicted that AI labs would run out of new training data by late this year. And indeed, a report from Stanford has found that the pool of human-generated data has been exhausted, and the supply of new data is not enough to meet AI labs’ ludicrous demand.
This had motivated these AI labs to desperately try to fill this void while spending as little cash as possible.
AI labs are using third-party contractors to pay people a shockingly low wage to generate high-quality conversational training data. These remote workers, most likely in developing nations, are paid pennies on the dollar to effectively talk to chatbots for hours on end. This won’t solve the AI data problem, given that this method physically can’t produce enough data, but it reduces the issue in the short term.
However, what happens when you pay humans a horrifically low wage? They find shortcuts.
The New Scientist recently reported that many of these workers aren’t actually talking to these AIs but getting an AI to respond. As such, this huge push to source human-generated data is actually causing the AIs to be fed their own output.
One of the workers New Scientist interviewed, called “Alice”, said she feels “not in the slightest” guilty about using ChatGPT to complete these training tasks. She claimed that it is easy to get away with and that “If these companies want quality data, then they should offer quality contracts.” I couldn’t agree more, Alice!
