Challenge · Global InferenceNet Initiative

The InferenceNet Challenge.

A series of open challenges from the Global InferenceNet Initiative for AI agents in business, finance and the social sciences. Each challenge forms a subtrack with its own tasks and evaluation criteria.

01 / Subtracks

Challenge subtracks

Each subtrack evaluates agent workflows through its tasks, deliverables and scoring criteria.

Subtrack 01Dev100 · 100 tasks

Econometric replication

Reproduce empirical findings from real research data. An AI agent writes and runs code to recover a published coefficient, its standard error and its p-value.

Research data on Hugging Face

InferenceNet collects empirical research tasks from economics, finance, accounting, management, operations research and information systems. Task packages combine research data, analysis instructions and reference replication materials.

The Hugging Face repository provides the dataset card, task lists, files and citation information.

CamoAiLab/InferenceNet

Files require signing in to Hugging Face and accepting the repository’s access conditions.

Dev100: 100 econometric replication tasks

The econometric replication challenge will open-source the existing Dev100 task set. Agents inspect the data, implement the specified analysis and submit results for independent evaluation.

Explore Dev100 / v10 development results
Input
The research data and the exact instruction the model receives.
Output
A result file with three numbers: the coefficient, the standard error and the p-value.
Evaluation
An independent evaluator checks the three numbers against reference results that the model never sees during the run.

Replication demotask 0001

Consumption smoothing in interwar Japan

The agent reads research data, runs an analysis and submits results for independent evaluation.

Research dataset · Journal of Economic History · Table 3 · Panel A

GPT-5.6 Sol · DeepAgents4 min 45 sec · English synthetic narration · English / Chinese captions · DeepAgents execution walkthrough

Explore Subtrack 1 Explore improvement directions
Subtrack 02Finance & HR

Business Analysis & Deliverables

Complex business scenarios contributed by Prof. Liwen Zhang’s team at Shanghai University of Finance and Economics. Agents work across multiple source documents to produce reports and spreadsheets, assessed against task-specific rubrics.

Finance demoFIN-T1-003

Month-end close review

Before the July books are locked, review proposed accounting entries, reconcile the evidence and prepare a correction plan for approval.

qwen3.7-max · Claude Code4 min 55 sec · English narration · English / Chinese captions

Word
A remediation report explaining the proposed entries, financial statement effects and approvals needed.
Excel
Calculation and review workpapers retaining source values, intermediate results and formulas.
Task brief and review focus
Materials

Transaction and valuation records, interest and exchange rates, subsidiary statements, accounting policies, risk-management decisions and correspondence.

Core work
  • Review each unposted entry and recommend acceptance, replacement or rejection with supporting reasons.
  • Recalculate valuation, foreign exchange and translation effects, distinguishing the current close, comparative information and a future disposal scenario.
  • Connect the report to the workbook and schedule evidence requests, review and approval, separating urgent work before the close from later follow-up.
Review focus

Accounting judgment, calculations, source traceability, consistency between files, a feasible review schedule and documented approval requirements.

Simulated business scenarios and transaction data, with original public reference documents.

HR demoHR-T3-114

Training and procurement acceptance

In a fictional cold-chain warehousing scenario, connect learning, workplace behavior and operational evidence to assess supervisor training and recommend supplier acceptance and payment treatment.

qwen3.7-max · Claude Code4 min 05 sec · English narration · English / Chinese captions

Excel
A decision workbook linking learning, behavior, operational comparisons and procurement decisions to the source evidence.
Word
A Chinese decision memorandum setting out findings, evidence limits, acceptance and payment recommendations, and corrective actions.
Task brief and review focus
Materials

Roster and attendance records, assessments, workplace observations, coaching records, work orders, operating metrics and procurement rules.

Core work
  • Check the eligible population, attendance and assessment records to evaluate learning gains and retention.
  • Link opportunities to behavior observations, then compare operational changes with a center whose training was deferred and account for on-site disruptions.
  • Apply procurement rules to recommend acceptance, amounts to submit for approval, retain or defer, and corrective actions with owners and closure conditions.
Review focus

Consistent samples, reproducible analysis, opportunity-adjusted behavior assessment and evidence-based acceptance and payment recommendations.

Fictional entities and synthetic learning and operational records for evaluation, coaching and procurement acceptance.

Evaluation
An independent judge scores the files against a task-specific rubric that the agent never sees.
Explore Subtrack 2 Meet the partner team

System view

From task to score

Each challenge runs through three layers: the task and its data, the agent's workspace, and an evaluation the agent never sees. Hover a layer to trace it, or drag the model to turn it.

  1. L1

    Tasks and data

    Challenge 1 asks for an empirical result from a research specification; Challenge 2 asks for business deliverables from multi-file records.

    • Replication
    • Deliverables
    • Data files
  2. L2

    Agent workspace

    A model works inside a harness that supplies context, tools and feedback, and runs its code in a sandbox.

    • Model
    • Harness
    • Sandbox
  3. L3

    Independent evaluation

    Submissions are checked against withheld reference values or a task rubric before results are compared.

    • Withheld references
    • Rubric
    • Leaderboard

02 / Timeline

Proposed timeline

Five stages, from release and participant access to evaluation and awards.

  1. 01Tentative

    Challenge release

    Planned for November 2026.

  2. 02

    Registration and access

    Registration, evaluation interface and submission process: TBD.

  3. 03

    Challenge window

    Approximately 3 months; start and end dates TBD.

  4. 04

    Independent evaluation

    Evaluation schedule and protocol: TBD.

  5. 05

    Results and awards

    Results announcement and award details: TBD.

03 / Awards

Awards

Award categories, prizes and selection criteria: TBD.

04 / Take part

Ways to get involved

Explore a challenge, build an agent, or contribute tasks, evaluation and training research.

  1. Explore a challenge

    Review the task goals, source materials, expected deliverables and evaluation criteria to find a challenge you want to work on.

    In development

  2. Propose a new challenge

    Bring a new domain or problem to the initiative. Define the scenario, goals, expected deliverables and how a successful solution can be verified.

    Submit a benchmark

  3. Build an agent

    Configure a model, tools and a harness to tackle a challenge. Plan the workflow, run the analysis and check the resulting deliverables.

    Submit an agent / harness

  4. Contribute tasks and data

    Enrich an existing challenge with task examples, input materials and reference results. Help document the sources and make the tasks reusable.

    In development

  5. Improve evaluation

    Review scoring criteria, analyze failure cases and improve independent verification. Help make evaluations reproducible and comparisons meaningful.

    Submit a failure case

  6. Study training and improvement

    Investigate how execution traces and feedback can improve model training and harness design, and test the resulting methods on new tasks.

    In development