GPT-6 Astra Solves 98 Percent of Novel Theoretical Physics Problems in New TheoPhysEval Benchmark

A new benchmark called TheoPhysEval has sent shockwaves through academia: OpenAI's GPT-6 Astra scored roughly 98 percent on fifty unpublished, frontier-level theoretical physics problems - after catching errors made by the very experts who wrote the questions. The findings, described in a paper on arXiv (2610.16733), come from a collaboration spanning the International Centre for Theoretical Sciences, the Tata Institute of Fundamental Research, UT Austin, the University of Tokyo, Harvard, UPenn, Duke, and Google DeepMind.

Building a Benchmark AI Cannot Memorize Evaluating today's large models is hard because problem banks on the internet have already been absorbed into training data. To probe genuine reasoning ceilings, dozens of prominent physicists - including Suvrat Raju, Sudip Chakraborty, and A. Ramesh Chandra - contributed fifty hardcore problems drawn exclusively from their own unpublished research notes, spanning quantum field theory, holography, and adjacent fields.

The difficulty bar was set brutally high: even a top world expert in the relevant subfield needs a full day or longer to solve one problem without AI assistance, while experts from neighboring fields may need enormous time or fail entirely.

Near-Perfect Scores - and the AI Grading the Examiners When the fifty problems were fed to frontier models, GPT-6 Astra initially scored about 88 percent and Claude Opus 5.5 scored 92 percent. The twist came next: when experts audited the grading, Astra proved remarkably stubborn about its answers - and the audit revealed that the model had caught computational mistakes made by the question writers themselves. After the human experts acknowledged the errors, Astra's score rose from 88 to 98 percent.

The result goes beyond benchmark bragging rights. A model identifying flaws in problem sets authored by leading theoretical physicists - in quantum field theory and holography, no less - suggests frontier systems are operating at the edge of expert-level original reasoning rather than sophisticated recall.

Science Problem-Solving Up 17x in Two Years The benchmark lands alongside a separate MIT measurement finding that the ability of frontier AI to solve theoretical science problems has surged seventeen-fold over the past two years. The researchers warn that science is on the verge of a major shake-up: after mathematics, quantum field theory, cosmology, and economics are increasingly being absorbed into AI-assisted workflows.

For research institutions, the implications are double-edged. TheoPhysEval offers a genuinely contamination-resistant yardstick for scientific reasoning, but its results indicate that unpublished-notes methodology - the current gold standard for avoiding data leakage - may itself have a short shelf life as models close in on expert performance across the physical sciences.

Key Data Points TheoPhysEval: 50 unpublished frontier problems from quantum field theory and holography GPT-6 Astra: 88 percent initial score, rising to 98 percent after expert errors were corrected Claude Opus 5.5: 92 percent; other tested models included Gemini 3.8 Flash and open-source DeepSeek-V4.1-Flash Contributing institutions: ICTS, TIFR, UT Austin, University of Tokyo, Harvard, UPenn, Duke, Google DeepMind MIT measurement: frontier AI scientific problem-solving capability up 17x in two years Paper: arXiv 2610.16733 Sources: 36Kr, Shinny Yuan (New Intelligence), arXiv, threerouter