Qwen AgentWorld 35B A3B oQ4 MLX | Benchmaxxing model results

Score Summary

Overall eligibility requires eligible data in all four categories.

Overall

IncompleteIncomplete

Incomplete for overall leaderboards

Agentic Tool Use

Eligible75.3

2/2 benchmarks covered

Agentic Coding

No dataNo data

0/0 benchmarks covered

Long-Term Tasks

No dataNo data

0/0 benchmarks covered

Speed

No dataNo data

0/0 benchmarks covered

Latest Results

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 14, 2026

Raw score70.0

Normalized70.0

Pass rate70.0%

Latency18.9s

Completion / TPS945s / 35.0 tok/s

Cost$0.00

Hermes tool-contract hard v1: 50 cases. Primary score is tool/state pass rate. tool_state=70%, strict=70%, final_text=96%, schema_valid=100%, required_tool=78%, hallucinated_tool=0%, forbidden_tool=0%, recovery=100.0%. avg_tool_calls=2.98, avg_turns=3.94, output_tps=35.0, input_tokens=137305, output_tokens=33051. non_text_failures: final_state_wrong=11, max_tool_calls=2, missing_required_tool=11, too_many_tool_calls=2. strict_failure_breakdown: final_answer_missing_expected_text=2, final_state_wrong=11, max_tool_calls=2, missing_required_tool=11, too_many_tool_calls=2. Local OMLX AgentWorld oQ4 run.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 14, 2026

Raw score85.0

Normalized85.0

Pass rate85.0%

Latency10.2s

Completion / TPS1018s / 39.3 tok/s

Cost$0.00

Hermes tool-contract v0: 100 cases. Primary score is tool/state pass rate. tool_state=85%, strict=81%, final_text=91%, schema_valid=100%, required_tool=90%, hallucinated_tool=0%, forbidden_tool=0%, recovery=66.7%. avg_tool_calls=1.31, avg_turns=2.25, output_tps=39.3, input_tokens=108131, output_tokens=39972. non_text_failures: final_state_wrong=4, max_tool_calls=5, missing_required_tool=10. strict_failure_breakdown: final_answer_missing_expected_text=9, final_state_wrong=4, max_tool_calls=5, missing_required_tool=10. Local OMLX AgentWorld oQ4 run.

Artifact/log: -

Historical Runs

2 eval runs

Hermes Tool Contract Hard v1Agentic Tool Use / Hermes Agent Evals hermes_tool_contract_hard_v1

EligibleAug 14, 2026

Raw score70.0

Normalized70.0

Pass rate70.0%

Latency18.9s

Completion / TPS945s / 35.0 tok/s

Cost$0.00

Hermes tool-contract hard v1: 50 cases. Primary score is tool/state pass rate. tool_state=70%, strict=70%, final_text=96%, schema_valid=100%, required_tool=78%, hallucinated_tool=0%, forbidden_tool=0%, recovery=100.0%. avg_tool_calls=2.98, avg_turns=3.94, output_tps=35.0, input_tokens=137305, output_tokens=33051. non_text_failures: final_state_wrong=11, max_tool_calls=2, missing_required_tool=11, too_many_tool_calls=2. strict_failure_breakdown: final_answer_missing_expected_text=2, final_state_wrong=11, max_tool_calls=2, missing_required_tool=11, too_many_tool_calls=2. Local OMLX AgentWorld oQ4 run.

Artifact/log: -

Hermes Tool ContractAgentic Tool Use / Hermes Agent Evals hermes_tool_contract_v0

EligibleAug 14, 2026

Raw score85.0

Normalized85.0

Pass rate85.0%

Latency10.2s

Completion / TPS1018s / 39.3 tok/s

Cost$0.00

Hermes tool-contract v0: 100 cases. Primary score is tool/state pass rate. tool_state=85%, strict=81%, final_text=91%, schema_valid=100%, required_tool=90%, hallucinated_tool=0%, forbidden_tool=0%, recovery=66.7%. avg_tool_calls=1.31, avg_turns=2.25, output_tps=39.3, input_tokens=108131, output_tokens=39972. non_text_failures: final_state_wrong=4, max_tool_calls=5, missing_required_tool=10. strict_failure_breakdown: final_answer_missing_expected_text=9, final_state_wrong=4, max_tool_calls=5, missing_required_tool=10. Local OMLX AgentWorld oQ4 run.

Artifact/log: -