Cohere North Mini Code Is Now Available via API and Self-Hosted: The Operator Routing Decision for Agentic Coding Workloads

Cohere's North Mini Code changes the deployment calculus for teams running agentic coding workloads through an AI gateway. Released June 9, 2026, under Apache 2.0, it is the first model from Cohere built exclusively for coding agents — and its architecture forces a concrete routing decision most teams have been able to defer until now.

What happened

North Mini Code is a 30B-parameter mixture-of-experts (MoE) model with only 3B active parameters per forward pass. It is optimized for code generation, multi-step software engineering, and terminal tool use. At launch it was released simultaneously in three deployment modes:

• Cohere API — hosted inference, pay-per-token, accessible via Cohere's OpenAI-compatible endpoint
• Model Vault — Cohere's dedicated managed inference environment for enterprise deployments
• HuggingFace weights — bf16, fp8, and w4a16 quantizations for self-hosted deployment

Minimum hardware for self-hosting is a single H100 at fp8 or fp4 precision. Context window is 256K total with 64K maximum generation.

On benchmarks, Cohere reports North Mini Code delivers up to 2.8× higher output throughput than Devstral Small 2 under identical concurrency and hardware, and a 30% advantage in inter-token latency. Its score on the Artificial Analysis Coding Index sits at 33.4, a competitive position among similarly sized models.

Why it matters for AI engineering teams

The North Mini Code announcement is not primarily a benchmark story. It is a deployment topology story — one that is increasingly relevant as coding agent workloads expand from individual tools to team-scale infrastructure.

Three dynamics matter here:

1. MoE efficiency changes the hosted-vs-self-hosted breakeven. At 3B active parameters, North Mini Code behaves like a 3B model at inference time despite a 30B parameter pool. Teams that previously ruled out self-hosting because their H100 capacity was tied up in larger models now have a genuine option. The throughput advantage (2.8×) means a single H100 can serve substantially higher concurrency than most 7B dense models would achieve.

2. Apache 2.0 removes the legal blocker for on-premises deployments. Enterprise operators in regulated industries — finance, healthcare, government — routinely cannot route coding workloads to third-party hosted inference without legal review. An Apache 2.0 model that runs on a single H100 eliminates that barrier. The routing decision shifts from "cloud vs. on-prem?" to "which cloud tier or deployment config best matches our compliance posture?"

3. The Cohere API endpoint is OpenAI-compatible. Teams already routing through an AI gateway that accepts OpenAI-compatible endpoints can add North Mini Code as a provider target without SDK changes. The operational cost of evaluating it in a live fallback or A/B routing policy is low.

The router/operator angle

North Mini Code introduces a three-way routing decision most teams have only faced as a two-way choice (large hosted model vs. smaller API model):

For gateway operators, the practical routing policy update involves three decisions:

1. Add North Mini Code as a fallback target for coding agent routes that currently default to larger, slower models. At 2.8× throughput advantage, it can reduce queue latency under burst load even if it is not the primary model.
2. Evaluate cost breakeven against your current coding model spend. MoE at 3B active parameters means hosted inference costs should track closer to 3B dense model pricing than to full 30B costs — verify with actual Cohere API pricing for your token volume.
3. Test tool-call reliability before promoting it to primary routing. North Mini Code was trained specifically for agentic coding with integrated tool use, but your specific agent workflow (file editing, terminal commands, multi-step planning) may expose edge cases that benchmarks do not capture.

What TheRouter users should watch or try

Teams routing coding agent workloads through an AI gateway should evaluate North Mini Code as a cost-optimized fallback or parallel evaluation target rather than a primary replacement for established coding agents at launch.

The self-hosting case is compelling if you already have H100 access and want to move coding workloads off third-party inference. The API case is worth a routing experiment if your current coding agent is bottlenecked on throughput rather than raw capability.

Review your /docs/ routing configuration and consider adding a dedicated coding agent tier if you have not already. Separating coding agent traffic from general-purpose LLM traffic makes it easier to optimize each tier independently as the model landscape evolves.