This repository contains the reference agent implementation for APEX-Agents 1.1. The tasks, world seeds, and shared runtime images are in the Harbor Hub and Hugging Face datasets.
The agent is a minimal LiteLLM tool-calling loop over the world's MCP tools. Tool output is truncated to 200 lines or 32K characters and the agent has a default of 100 total steps (turns) and a 10,800 second timeout, matching the leaderboard runs.
Install Docker and uv, then install Harbor:
uv tool install harbor==0.20.0Clone this repository, then set the API key for your model provider and any
grader credentials in .env:
git clone https://github.com/Mercor-Intelligence/apex_loop_truncated_tools_agent.git
cd apex_loop_truncated_tools_agent
cp .env.example .envTo reproduce the leaderboard's model settings, set APEX_MODEL_EXTRA_ARGS in .env. It is a JSON object of extra LiteLLM arguments sent with every model call. When it is unset, the model runs with provider defaults (for Claude, without extended thinking). The leaderboard's Claude Opus 5.5 run used:
APEX_MODEL_EXTRA_ARGS='{"output_config": {"effort": "max"}, "thinking": {"type": "adaptive", "display": "summarized"}, "max_tokens": 128000}'Run a task:
PYTHONPATH="$PWD/apex_loop_truncated_tools_agent" \
harbor run \
--env-file .env \
-d mercor/apex-agents-1-1@1.1.1 \
-i mercor/128-jr-1-f7f95d92 \
-a apex_loop_truncated_tools_agent:ApexLoopTruncatedToolsAgent \
-m anthropic/claude-opus-5Run the benchmark:
PYTHONPATH="$PWD/apex_loop_truncated_tools_agent" \
harbor run \
--env-file .env \
-d mercor/apex-agents-1-1@1.1.1 \
-a apex_loop_truncated_tools_agent:ApexLoopTruncatedToolsAgent \
-m anthropic/claude-opus-5Install the hf CLI, then download the delivery and load the three shared
images:
hf download mercor/apex-agents-v1.1 \
--repo-type dataset \
--revision 1.1.1 \
--local-dir apex-agents-v1.1
bash apex-agents-v1.1/environment/load_images.shRun a task:
PYTHONPATH="$PWD/apex_loop_truncated_tools_agent" \
harbor run \
--env-file .env \
-p apex-agents-v1.1/tasks/128-jr-1-f7f95d92 \
-a apex_loop_truncated_tools_agent:ApexLoopTruncatedToolsAgent \
-m anthropic/claude-opus-5 \
-yRun the benchmark:
PYTHONPATH="$PWD/apex_loop_truncated_tools_agent" \
harbor run \
--env-file .env \
-p apex-agents-v1.1/tasks \
-a apex_loop_truncated_tools_agent:ApexLoopTruncatedToolsAgent \
-m anthropic/claude-opus-5Replace the model with any LiteLLM-compatible provider/model.
@misc{bennett2026apexagents11,
title = {Introducing APEX--Agents 1.1},
author = {Bennett, Austin and Datta, Akul and Vidgen, Bertie},
year = {2026},
month = {September},
howpublished = {Mercor},
url = {https://www.mercor.com/blog/introducing-apex-agents-1-1/}
}
@misc{vidgen2026apexagents,
title = {APEX--Agents},
author = {Vidgen, Bertie and Mann, Austin and Fennelly, Abby and Wright Stanly, John and Rothman, Lucas and Burstein, Marco and Benchek, Julien and Ostrofsky, David and Ravichandran, Anirudh and Sur, Debnil and Venugopal, Neel and Hsia, Alannah and Robinson, Isaac and Huang, Calix and Varones, Olivia and Khan, Daniyal and Haines, Michael and Richards, Zach and Mahapatra, Chirag and Foody, Brendan and Nitski, Osvald},
year = {2026},
howpublished = {arXiv},
url = {https://arxiv.org/pdf/2601.14242}
}