Claude Opus 4.6 and MathArena

[Oktai15]

According to Epoch.AI, Claude Opus 4.6 is the first model by Anthropic that works pretty good on hard math tasks (see https://x.com/epochairesearch/status/2019852613672665193).

Is it possible to add Claude Opus 4.6 to some recent benchmarks (Project Euler, ArxivMath, Apex, Kanagroo 2025)? For some benchmarks you added previous models of Anthropic, but, as far as I see, you don’t add their new models anymore.

Anthropic is one of the industry leaders, it would be cool to see the results for their best model also, as you support it for OpenAI/Google and open-source models if you have enough credits for it.

[JasperDekoninck]

So far we did not add many Anthropic models since they are very expensive and not really good at math. We also saw the improvement from Opus 4.6 and are currently looking into getting the funding to make running it possible :)

[Oktai15]

@JasperDekoninck thanks for answer and great to hear that you are trying to make it possible!

Another similar question about adding SOTA models: do you have a plan to add OpenAI’s Pro line (e.g. GPT 5 Pro) and Google’s Deep Think line (e.g. Gemini 3 Deep Think)? According to company reports, they perform extremely well especially on math tasks.

[JasperDekoninck]

GPT-5.2-Pro etc. are unfortunately extremely expensive, so we keep it only for our our smaller (proof-based) evaluations. Gemini 3 Deep Think does not even have an API, so its impossible to run on any larger scale.

We have an update on both of them though once we can publish our eval on the Putnam (have to wait until the human results get published)