AI Agents for Business, Finance and Social Science
Global InferenceNet Initiative sets out a vision of shared infrastructure for AI agents in business, finance and social science. InferenceNet Challenge advances this vision by building an open-source community, bringing together challenging problems, and developing a standard framework for model training and evaluation.
Using the household panel, estimate the effect of treat × post on log consumption with household and quarter fixed effects. Restrict to 2008–2012 and cluster standard errors by village. Report coefficient, SE and p-value.
Month-end close review. Reconcile the general ledger against sub-ledgers, assess hedge and disposal treatments under policy, and link each correction proposal to the review workpapers.
Both challenges test complete agent workflows: interpret domain requirements, plan and execute analysis, revise through tool feedback, and submit results for independent evaluation.
Can an AI agent translate a research specification into a working analysis and recover the requested empirical result from the data?
Can an AI agent reconcile business evidence, make defensible decisions and turn them into reports and workbooks that withstand review?
Each challenge has its own evaluation criteria. Task packages, submission rules and dates will be announced separately.
Subtracks, timeline and awardsInspect the data, construct the required sample and variables, implement the estimator, then debug and revise using execution feedback.
Read source documents, reconcile records and rules, perform calculations, then reopen and revise the report and workbook.
Translate controls, interactions, fixed effects and clustered errors into code. A program can run successfully yet estimate the wrong quantity.
Resolve inconsistent evidence and apply professional rules. Calculations, explanations and recommendations must agree across documents.
A result file containing the target coefficient, standard error and p-value, computed by the agent’s analysis program.
A Word report and Excel workbook with traceable evidence, calculations and explicit decisions, following the task’s format requirements.
Compare the submitted statistics with withheld reference results, examining numerical agreement, effect direction and significance.
Check the files against source evidence and task-specific rubrics, scoring correctness, traceability and violations of required rules.
See how agents work within InferenceNet’s task and evaluation framework to tackle research and business problems: interpreting requirements, calling tools, refining their work through feedback, and undergoing independent evaluation.
The agent inspects household panel data, implements the research specification in regression code, and revises it using execution feedback. The framework then runs the submitted program and checks the coefficient, standard error and p-value against reference results.
Explore Challenge 1The agent reviews financial records and accounting policies, plans month-end corrections, and revises its work using tool feedback. It delivers a Word report and Excel workpapers, which the framework evaluates against the task’s evidence and scoring criteria.
Explore Challenge 2Challenges supply the tasks, harnesses turn models into agents, and independent evaluation decides what counts. Hover a layer to trace it, or drag the model to turn it.
Econometric replication and business analysis tasks, each with its own data and files.
Models act inside harnesses that supply context, tools and feedback. Dev100 compares 14 models across 4 harnesses.
Withheld references and rubrics score each run; results feed the leaderboard and self-evolving harness research.
Four ways InferenceNet Challenge advances the Initiative: community collaboration, challenging problems, standard evaluation, and model training and improvement.
Connect researchers, developers and domain experts to share code, tools, methods and results. Help teams build on each other’s work, solve problems together and accelerate research and applications.
Collaborate with usCollect demanding problems from research and professional practice, with data, task materials, deliverables and evaluation criteria. Build reusable tasks that test agents’ ability to complete complex work.
Explore the challengesStandardize task interfaces, execution records and independent evaluation protocols. Verify results against task-specific criteria so models and harnesses can be assessed reproducibly and compared under consistent conditions.
Explore the evaluation setupUse task data, verified trajectories and failure feedback to study model training and harness optimization. Test improvements on new tasks, turning evaluation findings into evidence for better systems.
Explore training and improvement researchWe welcome benchmark tasks, domain partners and collaborators working on models, harnesses and evaluation.
Bring your domain expertise, a new challenge, an agent implementation or evaluation ideas. Work with the community to turn challenging problems into reusable tasks, reproducible evaluation and better agents.
Two finance and HR cases follow agent workflows from task planning and evidence analysis to document delivery and independent review.
Explore the new subtrackThe existing 100-task Dev100 set will be open-sourced for the econometric replication challenge.
The v10 report covers 14 models and four harnesses on 100 development tasks. These are research results, not official challenge rankings.
View Dev100 results