GPT-5.6 Luna | Benchmaxxing model results

Score Summary

Overall eligibility requires eligible data in all four categories.

Overall

IncompleteIncomplete

Incomplete for overall leaderboards

Agentic Tool Use

Eligible87.0

2/2 benchmarks covered

Agentic Coding

No dataNo data

0/0 benchmarks covered

Long-Term Tasks

No dataNo data

0/0 benchmarks covered

Speed

No dataNo data

0/0 benchmarks covered

Latest Results

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 12, 2026

Raw score86.0

Normalized86.0

Pass rate86.0%

Latency13.6s

Completion / TPS680s / 16.5 tok/s

Cost-

Hermes tool-contract hard v1: 50 cases with reasoning_effort=max. Primary score is tool/state pass rate. tool_state=86%, strict=86%, final_text=100%, schema_valid=100%, required_tool=92%, hallucinated_tool=0%, forbidden_tool=0%, recovery=90.0%. avg_tool_calls=3.38, avg_turns=3.72, output_tps=16.5, input_tokens=61249, output_tokens=11225. non_text_failures: final_state_wrong=3, max_tool_calls=2, missing_required_tool=4, too_many_tool_calls=4. strict_failure_breakdown: final_state_wrong=3, max_tool_calls=2, missing_required_tool=4, too_many_tool_calls=4. Reference run via Hermes openai-codex OAuth Responses adapter.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 12, 2026

Raw score89.0

Normalized89.0

Pass rate89.0%

Latency9.36s

Completion / TPS936s / 12.0 tok/s

Cost-

Hermes tool-contract v0: 100 cases with reasoning_effort=max. Primary score is tool/state pass rate. tool_state=89%, strict=77%, final_text=81%, schema_valid=100%, required_tool=95%, hallucinated_tool=0%, forbidden_tool=0%, recovery=100.0%. avg_tool_calls=1.35, avg_turns=2.21, output_tps=12.0, input_tokens=40971, output_tokens=11252. non_text_failures: final_state_wrong=3, max_tool_calls=4, missing_required_tool=5, model_error=1. strict_failure_breakdown: final_answer_missing_expected_text=19, final_state_wrong=3, max_tool_calls=4, missing_required_tool=5, model_error=1. One provider request error was counted as a failed first attempt. Reference run via Hermes openai-codex OAuth Responses adapter.

Artifact/log: -

Historical Runs

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 12, 2026

Raw score86.0

Normalized86.0

Pass rate86.0%

Latency13.6s

Completion / TPS680s / 16.5 tok/s

Cost-

Hermes tool-contract hard v1: 50 cases with reasoning_effort=max. Primary score is tool/state pass rate. tool_state=86%, strict=86%, final_text=100%, schema_valid=100%, required_tool=92%, hallucinated_tool=0%, forbidden_tool=0%, recovery=90.0%. avg_tool_calls=3.38, avg_turns=3.72, output_tps=16.5, input_tokens=61249, output_tokens=11225. non_text_failures: final_state_wrong=3, max_tool_calls=2, missing_required_tool=4, too_many_tool_calls=4. strict_failure_breakdown: final_state_wrong=3, max_tool_calls=2, missing_required_tool=4, too_many_tool_calls=4. Reference run via Hermes openai-codex OAuth Responses adapter.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 12, 2026

Raw score89.0

Normalized89.0

Pass rate89.0%

Latency9.36s

Completion / TPS936s / 12.0 tok/s

Cost-

Hermes tool-contract v0: 100 cases with reasoning_effort=max. Primary score is tool/state pass rate. tool_state=89%, strict=77%, final_text=81%, schema_valid=100%, required_tool=95%, hallucinated_tool=0%, forbidden_tool=0%, recovery=100.0%. avg_tool_calls=1.35, avg_turns=2.21, output_tps=12.0, input_tokens=40971, output_tokens=11252. non_text_failures: final_state_wrong=3, max_tool_calls=4, missing_required_tool=5, model_error=1. strict_failure_breakdown: final_answer_missing_expected_text=19, final_state_wrong=3, max_tool_calls=4, missing_required_tool=5, model_error=1. One provider request error was counted as a failed first attempt. Reference run via Hermes openai-codex OAuth Responses adapter.

Artifact/log: -