Montréal v1.0 vision benchmark

• PortMind Montréal v1.0.0
• Published 2026-09-25
• 96 images
• 6 models
• Truck presence and container attachment

Frozen reference 6697e8c8…b2ec3f Copy full digest

Public figures are aggregates. The image set and raw model responses stay private. The two tasks are scored separately and are not pooled into one ranking.

Ranked by positive detection rate (found / positive labels) on truck presence. Exact counts stay visible.

ModelFound / positive labels

1GPT-6 Astraopenai/gpt-6-astra54 / 54

2Mistral Small 3.1 24B@cf/mistralai/mistral-small-3.1-24b-instruct48 / 54

3Llama 4 Scout 17B@cf/meta/llama-4-scout-17b-16e-instruct39 / 54

4Kimi K2.7 Code@cf/moonshotai/kimi-k2.7-code33 / 54

5Kimi K2.6@cf/moonshotai/kimi-k2.633 / 54

6Gemma 4 26B A4B@cf/google/gemma-4-26b-a4b-it28 / 54

Truck presence counts. Uncertain references are outside these denominators (5 on this task).

Model	Found	Missed positives	False alarms	Answered	Uncertain	Failed
GPT-6 Astraopenai/gpt-6-astra	54 / 54	0	21 / 37	91 / 91	0	0
Mistral Small 3.1 24B@cf/mistralai/mistral-small-3.1-24b-instruct	48 / 54	6	7 / 37	91 / 91	0	0
Llama 4 Scout 17B@cf/meta/llama-4-scout-17b-16e-instruct	39 / 54	15	6 / 37	91 / 91	0	0
Kimi K2.7 Code@cf/moonshotai/kimi-k2.7-code	33 / 54	21	1 / 37	91 / 91	0	0
Kimi K2.6@cf/moonshotai/kimi-k2.6	33 / 54	21	2 / 37	91 / 91	0	0
Gemma 4 26B A4B@cf/google/gemma-4-26b-a4b-it	28 / 54	26	0 / 37	91 / 91	0	0