Picture a freelancer on Upwork taking on a real paying job: a 3D model for an engagement ring, a short animated ad, a dimensioned floor plan for a bathroom remodel. A year ago, if you handed that same brief to an AI agent, it would get it right about 2 or 3 times out of 100. As of this month, according to a benchmark built specifically to test this, it gets it right more than 4 times out of 10.
That jump comes from the Remote Labor Index (RLI), a benchmark run by the Center for AI Safety (CAIS) and Scale Labs that tests AI agents on real freelance projects, not toy coding puzzles. Anthropic’s new Fable 5.2 model just topped the leaderboard with a 41.25% automation rate. If you freelance, hire freelancers, or are simply wondering how fast AI is catching up to skilled human work, this benchmark is one of the clearest signals we have.
What the Remote Labor Index Actually Measures
Most AI benchmarks test narrow skills in a lab setting. RLI does something different. It uses 240 real projects sourced from 358 verified professionals on Upwork, covering 23 categories such as 3D and CAD design, architecture, graphic design, video and animation, audio editing, data analysis, and web development. Each project comes with the client’s original brief, the input files, and the actual human-made deliverable the client paid for and accepted.
The average project took a human professional about 29 hours and was worth roughly $632.60. When an AI agent attempts one of these projects, trained human evaluators compare its work directly against the professional’s deliverable and ask a simple question: would a reasonable client accept this? Only projects judged as good as, or better than, the human’s work count toward the “automation rate,” which is the headline number everyone quotes.
How Fast This Number Has Moved
When RLI launched, the best-performing AI agent automated just 2.5% of projects. By mid-2026, Anthropic’s Fable 5 had pushed that to 15.8%, described at the time as the automation rate roughly quadrupling in under eight months. Now, with Fable 5.2, the top score sits at 41.25%, more than double the next closest model, Anthropic’s own Opus 5.5 at 21.25%, with OpenAI’s GPT 6 Astra close behind at 20.83%.
Scale Labs reported that Fable 5.2 made its biggest gains in audio editing, architectural design, and 3D modeling, and was the first model to solve roughly 50 tasks that no earlier agent had managed. Even so, the system still struggles with animation work and industrial product design, which tells you this progress is real but uneven, not a clean sweep across every kind of freelance work.
Where AI Still Falls Short
It is worth being honest about what “41%” does not mean. It does not mean AI can now do nearly half of all freelance work competently. CAIS, which co-built the benchmark, has been direct about this: the most common failure mode is simply poor quality work that looks “child-like” or amateurish, followed by incomplete deliverables such as truncated videos or missing files, and outright technical errors like corrupted or unusable file formats.
- Poor quality output (the single biggest failure category)
- Incomplete deliverables, like partial videos or missing files
- Technical and file-format errors
- Inconsistencies across a project, such as a 3D design that changes between views
The researchers also found something that surprised them: AI judges used to grade this kind of work still badly overestimate quality compared to trained human evaluators, sometimes by two to three times. In other words, even the AI tools built to check AI work are not yet reliable enough to replace a human client’s judgment. That gap matters, because it means the real-world automation rate is almost certainly lower than what an automated AI reviewer alone would report.
What This Means If You Freelance or Hire Freelancers
From my own experience working on websites, online tools, and digital projects, the lesson here is not panic, it is paying attention to where the gap is actually closing. AI is pulling ahead fastest in tasks with a clear, checkable output: a report, a logo, a data pull, a short audio edit. It is still weak at longer, more judgment-heavy creative work, and at anything that needs a human to genuinely understand context the way a client relationship does.
Tip: if a chunk of your freelance income comes from short, templated deliverables, that is exactly the kind of work AI agents are getting good at fastest. Shifting toward projects that need ongoing client communication, nuanced creative judgment, or review of AI-produced drafts is a more defensible position than competing purely on speed.
This also connects to a pattern we have covered before on BrightMindAI: employers increasingly reward people who know how to direct and check AI output, not just people who can produce work from scratch. If you want to see how that is already showing up in hiring data, our piece on what separates AI power users from everyone else at work is a good next read, and our guide on what AI skills are actually worth on your CV breaks down which specific skills employers are paying for.
If you are earlier in your career, it is also worth reading our look at why the AI job gap for new graduates has been widening, and if you are curious whether letting an AI agent apply to jobs on your behalf is actually a good idea, we covered that too in AI agents applying for jobs.
Common Questions
Is 41% the share of all freelance work AI can now do?
No. It is the share of 240 specific real-world test projects where an AI agent’s output matched or beat the professional’s deliverable, across a defined set of 23 categories. It is a strong signal of the trend, not a claim about the whole freelance economy.
Which AI model performed best?
Anthropic’s Fable 5.2, at a 41.25% automation rate, more than double the second-place model, Anthropic’s own Opus 5.5 at 21.25%, with OpenAI’s GPT 6 Astra just behind at 20.83%.
What kinds of work is AI still bad at?
Animation, industrial product design, and anything needing sustained visual or logical consistency across many files. The most common failures overall were simply low-quality or incomplete work.
Should freelancers be worried right now?
The direction of travel is clear and worth planning around, but the data itself shows AI still fails most projects outright, and even AI graders overestimate AI quality. There is time to adapt, and the smartest move is building skills in directing, reviewing, and combining AI output rather than competing purely on raw output speed.
Final Takeaway
The honest takeaway from this benchmark is not that AI is about to replace freelancers wholesale. It is that the pace of change is fast enough that waiting and seeing is not a strategy. A number that moved from 2.5% to 15.8% to 41.25% in about a year and a half will not sit still. Whatever work you do, now is a good time to get specific about which parts of it an AI agent could realistically attempt today, and which parts still clearly need you.








0 Comments