Growing Trend in AI Training: Specialists Educate Advanced Models
STOCKHOLM/SAN FRANCISCO - Once reliant on vast teams of low-cost workers for fundamental tasks like distinguishing simple objects, advanced artificial intelligence (AI) models such as ChatGPT and Cohere now necessitate a more sophisticated and specialised touch. These AI systems require training from individuals with specialised expertise, ranging from historians and scientists to financial analysts and medical professionals.
Cohere, a formidable competitor to OpenAI and valued at over $5 billion, is a prime example of this evolved approach. Co-founder Evan Chang remarked, "A year ago, we could have successfully hired undergrads just to generally teach AI how to improve. Now we have doctors who are licensed to teach models how to behave in medical settings, or financial analysts or accountants."
To fulfil these sophisticated training needs, Cohere collaborates with startup Invisible Tech. Founded in 2015 as a workflow automation company for clients like DoorDash, Invisible Tech pivoted significantly after partnering with OpenAI in 2022. "OpenAI came to us with a problem that when you asked a question in an early version of ChatGPT, it would make you delirious. You couldn’t trust the answer," explained Francis Pedraza, founder of Invisible Tech.
Invisible Tech now employs thousands of trainers working remotely across more than 100 countries. These trainers, some holding advanced degrees or specialised certifications, are pivotal in teaching AI models to reduce errors, known in industry jargon as "hallucinations." Compensation varies significantly, with rates up to $40 per hour depending on the work's complexity and location. Some companies like Outlier offer up to $50 per hour, while Labelbox provides up to $200 per hour for highly specialised topics like quantum physics.
These partnerships aim to address significant challenges AI models face, notably the generation of inaccurate information. A case in point occurred in 2023 when Google’s chatbot gave incorrect information about the satellite that first captured images of a planet outside Earth’s solar system. Such hallucinations can severely impact AI's business appeal, prompting companies to rigorously refine their training methodologies.
Invisible Tech has become a key partner in this space, working with various AI giants, including Cohere, AI21, and Microsoft, to provide targeted training. While Microsoft has not confirmed its engagement with Invisible, Cohere and AI21 have verified their collaborations.
OpenAI employs a dedicated "Human Data Team" that collaborates with AI trainers from Invisible Tech and other vendors to collect specific data to refine its models like ChatGPT. "OpenAI has some of the most amazing computer scientists in the world, but they are not necessarily experts on Swedish history or chemistry questions or biology questions," Pedraza noted.
The AI training industry is becoming increasingly competitive. Scale AI, a private startup valued at $14 billion, also provides training data and AI coaching services and counts OpenAI among its clients.
Pedraza highlighted that Invisible Tech, profitable since 2021, operates with a business model where 70% of ownership lies with the team and only 30% with investors. The company has raised $8 million in seed capital, with unconfirmed reports suggesting a valuation near half a billion dollars.
The shift from low-cost data classification in mainly African and Asian countries to highly remunerated specialised training roles has widened access for experts from various fields. These experts can become AI trainers without necessarily knowing how to code. Cohere’s Chang anticipates continued growth in the sector, describing his inbox as "full of new companies popping up here and there," reflecting a burgeoning market for generating data to train AI models.
The sophistication of AI models and their expanding functionality continue to fuel demand for specialised training, creating new economic opportunities for individuals globally. As AI companies persist in launching more advanced models, reliance on specialised human trainers appears set to increase.
Source: Noah Wire Services