Google Updates Android AI Coding Benchmarks, Highlights Claude Fable 5

Google has introduced changes to its evaluation process for AI models focused on Android coding, updating its rankings with Claude Fable 5 leading the pack.

Earlier this year, Google launched the “Android Bench,” a ranking system designed to assess AI models based on their proficiency in Android development rather than general performance. This tool offers developers a specialized benchmark to compare AI capabilities specifically within Android coding tasks.

The company announced modifications to the benchmarking methodology, transitioning to the standardized Harbor framework. Previously, models such as GPT-5.5, Claude Opus 4.7, and Gemini 3.1 Pro Preview were assessed using the mini-swe-agent v1 benchmark tool, which was developed for broader applications.

By adopting the Harbor framework, Google enables Android developers to utilize consistent tools for evaluating AI models tailored to specific use cases. Additionally, Google is inviting users to submit Android development tasks to be incorporated into the evaluation process, allowing the system to test how models perform in various scenarios. Developers are also encouraged to contribute their own benchmark results.

Google emphasized its commitment to openness and transparency by making the original methodology and test harness publicly accessible on GitHub. The company stated, “You’ve asked for a way to provide feedback on our dataset, so now we’re taking collaboration a step further by giving you, the Android developer community, a chance to shape Android Bench.”

Alongside the methodological update, Google refreshed the Android Bench rankings using the new framework. All AI models, including both closed-weight and open-weight variants, were reevaluated under the revised testing criteria.

Claude Fable 5 secured the highest position with a score of 84.5, maintaining a lead of 4.3 points over GPT-5.5, which scored 80.2. Claude Sonnet 5 ranked nearly 10 points below Fable 5. These results suggest that Anthropic’s performance claims are credible, despite Fable 5 still operating under significant restrictions.

Most AI models from the previous Android Bench list received updated scores due to the changes in evaluation standards. The new rankings reflect the adjusted framework and scoring system.