Best AI model solves 12% of expert tasks in SemiAnalysis test

The signal

The best AI model completed 12% of expert tasks in a new test

What happened. On 2 November SemiAnalysis, a research firm that covers chips and AI, published GDPx, a benchmark of agent tasks in 11 fields, including chip design, cybersecurity, biology, finance, law and software engineering (Kan et al., 2026). The best model scored 12%. The authors say their finance and legal tasks are so hard that almost every model scores exactly zero. Claude Opus 5.5 was the best model overall, and every leading model was strong in some fields and weak in others. Fourteen specialist firms wrote the tasks, and the full article is for paying subscribers.

Why it matters. The authors argue that older benchmarks such as GDPval, released in September 2025, use short, fully specified tasks that no longer test today's agents. The headline score a vendor quotes tells you little about your own specialist work. Results vary by field, so a model that ranks first overall can be the wrong choice for yours.

What to do. Before you buy or switch, give two or three models ten real tasks from your team and have an expert grade the output.

Abu Dhabi now checks permits with OpenAI models, and an official signs off

What happened. On 2 November the Abu Dhabi government and OpenAI launched the Machine-Readable Government programme (Abu Dhabi Media Office, 2026). It turns a set of government rules into a form AI systems can read, so they can help officials check routine permit and certificate applications. Each check points back to the rule it applied, and officials keep the final decision. The first pilot is already live at the agriculture and food safety authority, for food imports, with computing in data centres inside the UAE. Two other agreements signed the same day bring OpenAI models to ADNOC, in the energy sector, and to investment analysis at Mubadala.

Why it matters. This is a working design for AI in regulated decisions: written rules as input, a trace to the rule for every answer, and a person who decides. The same design fits internal compliance work, such as supplier checks or export rules, where an auditor will ask why a decision was made.

Try this today

List your team's heavy AI tasks in ten minutes, before ChatGPT and Codex share one limit

What it is: From December, ChatGPT and Codex, OpenAI's coding agent, will draw on one shared usage allowance on the Plus, Pro and Business plans, measured in tokens, the units of text a model reads and writes.

Who it is for: Team leads and IT managers who pay for ChatGPT seats and whose teams also use Codex.

Access: Paid: Plus, Pro and Business plans, listed on OpenAI's pricing page. Combined remaining usage will show in ChatGPT under Settings, then Usage.

First step: Ask your three heaviest users to list the Codex and advanced-model tasks they ran last week, because those will draw most from the shared pool.

Caution: OpenAI has not published the size of the allowances. Instant chat does not count, while advanced models and complex tasks use more; at the limit you wait for the reset, upgrade or buy credits.

(OpenAI, n.d.)

Keep your US patent searches running: create a USPTO account before 7 November

What it is: From 7 November 2026, Patent Public Search, the public search tool for US patents and published applications run by the USPTO, will require users to sign in with a USPTO.gov account.

Who it is for: Patent, R&D and technology scouting teams, and anyone who checks US patents; my note on what to fix first in patent analysis covers the wider workflow.

Access: A USPTO.gov account, created on the MyUSPTO sign-up page; no fee is listed there.

First step: Open the sign-up page, create an account, and ask each person in your team who searches US patents to do the same today.

Caution: The notice does not say what changes for automated or bulk use, so check any tool or script that queries the search without a login.

(United States Patent and Trademark Office, n.d.)

Worth watching

Kimi promises open weights for K3.1 Pro by 4 November

Kimi says it will release the full weights of Kimi K3.1 Pro by 4 November (Kimi, n.d.). Open weights would let companies run the model on their own servers. The model is already available in Kimi's apps and API, with a context of 1 million tokens, and Kimi reports 65.7% on a coding-agent test against 66.4% for Claude Opus 5.5, by its own measurement. The weights are announced, not yet released.

South Korea budgets 4.7 trillion won for its own top AI model

On 2 November President Lee Jae Myung presented a record 2027 budget of 820.9 trillion won to the National Assembly, up 12.8% (Lim & Lee, 2026). It sets aside 4.7 trillion won for a sovereign AI model with the "world's highest performance", and 21.3 trillion won, nearly double the 2026 level, for chips, physical AI and AI data centres. The Assembly now has to approve it. Yesterday's edition covered xAI's plan to watermark Grok API text by default.

Repos worth a look

google-research/harness2

harness2 from Google Research improves an AI agent by rewriting the setup around the model, its prompts, tools and memory, instead of retraining the model (Google Research, 2026). It needs no answer key: it tests changes on the task itself and keeps a log of lessons for later tasks. One round raised scores by up to 11.8 percentage points on a legal-work test and 11.5 points on an office-work test, across two different models. About 22,000 stars on 2 November 2026, Apache 2.0 licence. Forward it to whoever builds your internal agents; the README says it is not an officially supported Google product and that runs incur model costs.

yaojingang/GEOInflow

GEOInflow shows which AI crawlers visit your website, which pages they read, and how many visitors and conversions arrive from AI search (yaojingang, 2026). It runs on your own server, needs Docker and at least 6 GiB of memory, and works without an AI model key. 784 stars on 2 November 2026, MIT licence. Give it to your web or marketing analyst, and change the default admin password at first login. We covered research on predicting which sources AI answer engines cite on 1 November; GEOInflow measures the other side, what the crawlers take from your site.

The CERNY.AI read

Today's items point the same way: AI is good enough to use and not good enough to trust unchecked. GDPx puts the best model at 12% on expert tasks, and Abu Dhabi's answer is a design where every AI check cites its rule and a person signs. Access is tightening at the same time: the USPTO adds a login, and OpenAI meters chat and agents from one pool. Organisations in competitive and patent intelligence should stop choosing models from public leaderboards and build a small test set of their own tasks, graded by their own experts, before each renewal. Building that test set is part of my competitive intelligence consulting.

One question for your next meeting

If we gave our three most common expert tasks to an AI model today, who would grade the output, and against what standard? The CI maturity quick check shows whether your team has that standard written down.

Sources

Abu Dhabi Media Office. (2026, November 2). Khaled bin Mohamed bin Zayed meets Sam Altman and witnesses signing of agreements with OpenAI. https://www.mediaoffice.abudhabi/en/crown-prince-news/khaled-bin-mohamed-bin-zayed-meets-sam-altman-and-witnesses-signing-of-agreements-with-openai/
Google Research. (2026). harness2 [Computer software]. GitHub. https://github.com/google-research/harness2
Kan, M., Megalaa, A., & Patel, D. (2026, November 2). GDPx: Measuring frontier model capabilities across the most valuable domains. SemiAnalysis. https://newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilities
Kimi. (n.d.). Kimi K3.1 Pro tech blog: Advancing open intelligence. https://www.kimi.ai/blog/kimi-k3-1-pro
Lim, H. J., & Lee, J. (2026, November 2). South Korea targets 'world's highest performance' sovereign AI model in record new budget. CNBC. https://www.cnbc.com/2026/11/02/south-korea-ai-spending-budget.html
OpenAI. (n.d.). Understanding ChatGPT and Codex usage limits. OpenAI Help Center. https://help.openai.com/en/articles/20001625-understanding-chatgpt-and-codex-usage-limits
United States Patent and Trademark Office. (n.d.). Patent Public Search. https://www.uspto.gov/patents/search/patent-public-search
yaojingang. (2026). GEOInflow [Computer software]. GitHub. https://github.com/yaojingang/GEOInflow