GLOBAL INFERENCENET INITIATIVE · NTU GIFTS

InferenceNet Challenge

Can AI reproduce econometric findings from real research data?

Given data and analysis instructions, a model writes and runs code to recover a coefficient, its standard error and its p-value. An independent evaluator compares the output with hidden reference results.

We study how models and harnesses work together to make this process more reliable.

NTU College of Computing and Data Science NTU Nanyang Business School HKU Business School
Explore a real case View the leaderboard Explore the data

Model and Model + Harness results are now available.View results

01 / The benchmark

From research data to a verified result

The benchmark tests whether models can turn a specified empirical method into an executable analysis and reproduce its statistical results.

Explore the dataset and task coverage

  1. 01

    Read the task

    Research data, an estimation method, variables and analysis requirements.

  2. 02

    Write the program

    The model translates those requirements into code.

  3. 03

    Run the analysis

    Prepare the sample, estimate the model and return the requested statistics.

  4. 04

    Check the result

    The evaluator checks the coefficient, standard error and p-value against the reference.

Reference answers and reference programs stay with the evaluator and are never shown to the model during the run.

02 / A real case

Finding the month variable

One recorded GPT-5.5 task shows how execution feedback changes the analysis. DeepAgents is the harness used in this example.

Task 0011 · Journal of Economic History

How does socio-economic conflict relate to economic policy uncertainty in Spain?

Reproduce the specified regression for 1905–1945, with three controls and standard errors clustered by month.

data_np.dta4,550 rows · 38 columnsWhat is in the data?

GPT-5.5 · Model

The first program stops

The model generates a program that searches for familiar month-column names. The actual month index is named tid.

The program cannot identify it, so execution fails before producing a result.

1 model call

GPT-5.5 + DeepAgents

Inspect, execute, reproduce

The model inspects the columns, identifies the monthly index, and checks the sample and regression. DeepAgents runs its code and returns the output at each step.

The final program uses 977 observations and reproduces all three target statistics.

4 model calls · 3 tool executions

Two separate runs of the same task. The final program is replayed and checked independently.

Follow the recorded run

Explore more research tasks

03 / Leaderboard

InferenceNet Challenge Leaderboard

Model and Model + Harness results on 1,000 econometric replication tasks.

Scroll the table to see all metrics →

Scores (%)
#ModelHarnessFull replicationPartial replicationExecution successCoefficient directionSignificance level
1GPT-5.6 SolDeepAgents51.8%69.9%91.0%87.9%78.7%
2Claude Opus 4.8DeepAgents39.2%62.1%83.4%79.3%69.6%
3Gemini 3.1 ProDeepAgents39.1%63.2%85.0%81.1%71.8%
4Kimi K3DeepAgents39.0%63.5%86.7%80.4%70.9%
5GPT-5.6 Sol—33.8%43.0%55.7%53.7%48.5%
6Qwen3.7-MaxDeepAgents24.8%37.9%52.3%48.8%43.3%
7Gemini 3.1 Pro—24.6%36.6%54.3%50.6%43.7%
8Kimi K3—23.0%36.6%52.6%49.4%43.1%
9Qwen3.7-Max—19.8%31.5%48.6%45.4%39.6%
10Claude Opus 4.8—18.4%30.5%46.7%43.7%37.0%
11DeepSeek V4 ProDeepAgents18.3%27.0%35.2%33.8%30.8%
12DeepSeek V4 Pro—12.9%19.5%28.1%26.0%23.6%
Every score uses all 1,000 tasks; failures and unknowns stay in the denominator.Download results CSV ↓

Updated 17 Sep 2026 · Research evaluation. Interaction budgets and historical settings differ across configurations.

Full leaderboard and evaluation details

04 / Research

What can we learn from these runs?

We aim to use replication results and execution feedback to improve how models and harnesses work together.

These are research directions we hope to pursue with collaborators. Explore the research directions

05 / Get involved

Build with us

We welcome feedback and collaboration on models, harnesses, evaluation and trajectory-based training.

Challenge rules, dates and submission instructions will be announced here. In the meantime, explore the benchmark or get in touch.

Contact us Meet the team

Resources

Sources and citation

The benchmark is published by CamoAiLab on Hugging Face. The related MetricsAI paper describes the Econometrics-Agent research system. Use the citation provided by each source.