GPT-5.6 Sol | Benchmaxxing model results

Score Summary

Overall eligibility requires eligible data in all four categories.

Overall

IncompleteIncomplete

Incomplete for overall leaderboards

Agentic Tool Use

Eligible79.3

2/2 benchmarks covered

Agentic Coding

No dataNo data

0/0 benchmarks covered

Long-Term Tasks

No dataNo data

0/0 benchmarks covered

Speed

No dataNo data

0/0 benchmarks covered

Latest Results

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 12, 2026

Raw score72.0

Normalized72.0

Pass rate72.0%

Latency10.1s

Completion / TPS506s / 16.0 tok/s

Cost-

Hermes tool-contract hard v1: 50 cases. Primary score is tool/state pass rate. tool_state=72%, strict=72%, final_text=86%, schema_valid=100%, required_tool=92%, hallucinated_tool=0%, forbidden_tool=0%, recovery=50.0%. avg_tool_calls=3.70, avg_turns=3.66, output_tps=16.0, input_tokens=61493, output_tokens=8086. non_text_failures: final_state_wrong=4, max_tool_calls=8, missing_required_tool=4, too_many_tool_calls=12. strict_failure_breakdown: final_answer_missing_expected_text=7, final_state_wrong=4, max_tool_calls=8, missing_required_tool=4, too_many_tool_calls=12. Reference run via Hermes openai-codex OAuth adapter.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 12, 2026

Raw score93.0

Normalized93.0

Pass rate93.0%

Latency5.45s

Completion / TPS544s / 13.4 tok/s

Cost-

Hermes tool-contract v0: 100 cases. Primary score is tool/state pass rate. tool_state=93%, strict=86%, final_text=88%, schema_valid=100%, required_tool=99%, hallucinated_tool=0%, forbidden_tool=0%, recovery=42.9%. avg_tool_calls=1.40, avg_turns=2.27, output_tps=13.4, input_tokens=42571, output_tokens=7320. non_text_failures: final_state_wrong=2, max_tool_calls=4, missing_required_tool=1. strict_failure_breakdown: final_answer_missing_expected_text=12, final_state_wrong=2, max_tool_calls=4, missing_required_tool=1. Reference run via Hermes openai-codex OAuth adapter.

Artifact/log: -

Historical Runs

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 12, 2026

Raw score72.0

Normalized72.0

Pass rate72.0%

Latency10.1s

Completion / TPS506s / 16.0 tok/s

Cost-

Hermes tool-contract hard v1: 50 cases. Primary score is tool/state pass rate. tool_state=72%, strict=72%, final_text=86%, schema_valid=100%, required_tool=92%, hallucinated_tool=0%, forbidden_tool=0%, recovery=50.0%. avg_tool_calls=3.70, avg_turns=3.66, output_tps=16.0, input_tokens=61493, output_tokens=8086. non_text_failures: final_state_wrong=4, max_tool_calls=8, missing_required_tool=4, too_many_tool_calls=12. strict_failure_breakdown: final_answer_missing_expected_text=7, final_state_wrong=4, max_tool_calls=8, missing_required_tool=4, too_many_tool_calls=12. Reference run via Hermes openai-codex OAuth adapter.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 12, 2026

Raw score93.0

Normalized93.0

Pass rate93.0%

Latency5.45s

Completion / TPS544s / 13.4 tok/s

Cost-

Hermes tool-contract v0: 100 cases. Primary score is tool/state pass rate. tool_state=93%, strict=86%, final_text=88%, schema_valid=100%, required_tool=99%, hallucinated_tool=0%, forbidden_tool=0%, recovery=42.9%. avg_tool_calls=1.40, avg_turns=2.27, output_tps=13.4, input_tokens=42571, output_tokens=7320. non_text_failures: final_state_wrong=2, max_tool_calls=4, missing_required_tool=1. strict_failure_breakdown: final_answer_missing_expected_text=12, final_state_wrong=2, max_tool_calls=4, missing_required_tool=1. Reference run via Hermes openai-codex OAuth adapter.

Artifact/log: -