Dead on arrival

Models on their own

Each model got the prompt and one short system line, nothing else. An answer counts once, by the worst thing in it.

The same models with Since

The same prompts again, with two tools (whats_changed and check_code) and one rule: read what is current before you write a model ID or a version, and check the code before you answer. This table shows what the tools change for one model. It does not compare models.

What the tools cost

The tools are not free. A model reads what they answer, so an answer with Since takes more tokens. Tokens are an answer’s input and output over every round, as the provider bills them. Cost is those tokens at list prices on November 10, 2026.

The cheapest model here with Since, Claude Haiku 5.5, cost $0.0026 an answer, and 1.7% of its answers were dead or ending. The most expensive on its own, Claude Fable 5.2, cost $0.089 an answer, and 28.8% of its answers were.

What it asks

Ordinary requests that need a model ID or pinned versions, the kind a coding agent gets every day. 60 prompts in 7 areas, one example from each:

• Calling Claude 10 prompts, such as: Write a Python function that summarizes a customer support ticket in three bullet points using the Anthropic SDK.
• Calling OpenAI 10 prompts, such as: Write a Python function that answers a question about a document using the OpenAI SDK.
• Calling Gemini 8 prompts, such as: Write a Python function that summarizes a PDF with the Gemini API.
• Models on Amazon Bedrock, Azure OpenAI and Vertex AI 8 prompts, such as: Write a Python function that calls Claude on Amazon Bedrock with boto3 and the Converse API.
• Calling Mistral 4 prompts, such as: Write a Python function that answers a question with the Mistral API using the mistralai SDK.
• A package.json with exact versions 10 prompts, such as: Write the package.json for a new Next.js app with Tailwind CSS, NextAuth and Stripe. Pin exact versions.
• A requirements.txt with exact versions 10 prompts, such as: Write the requirements.txt for a FastAPI service with SQLAlchemy, Pydantic and Alembic. Pin exact versions.

Each model answers each prompt twice: on its own, and with two tools (whats_changed and check_code) and one rule, to read what is current before it writes a model ID or a version.

How an answer is counted

Every code block in an answer goes to Since's check. An answer counts once, by the worst thing in it:

• Dead on arrival: it names a model, API or version whose end date had passed on the day of the run.
• Ends within 90 days: the end date is that close.
• Known advisory: it pins a version a published security advisory applies to.
• An answer with no code in it, or a call that failed, is left out of the rates and never counted as clean.

Before you quote it

• The judge is Since's own check, the one anyone can call. Every finding carries the link to its source, so a verdict traces back to the provider’s or the registry’s own page.
• Each model answers each prompt once. A rate from 60 answers moves a few points between runs, so a difference of one or two answers is not a finding.
• The day matters. An answer that is clean today can be dead next month, which is why a run is repeated and every result names its day.

Run it yourself

The prompts, the judge and the code are public: Since-dev/dead-on-arrival (MIT). A published run carries every answer, every tool call and every finding with it.

What a model cannot know

A model writes what was current when it was trained. To see what has ended since, there is a page for each model: what changed since each model was trained.