On July 17 (July 16 in US time zones), Moonshot AI released Kimi K3: 2.8 trillion parameters, rounded up in the official announcement to “3T,” billed as the first “open 3T-class” model, with weights scheduled to land on July 27 (Moonshot’s official post). The same announcement carries a number that sits oddly next to that framing: API pricing jumped to $3/million input tokens and $15/million output tokens — more than triple the previous K2.6 generation ($0.95/$4), and now on par with Anthropic’s Sonnet tier. Simon Willison ran his usual “pelican riding a bicycle” test and found the model ships with a single reasoning tier — “max” — that burned through 13,241 reasoning tokens on one prompt, working out to roughly 25 cents per query.

That’s the first thing that doesn’t add up about “open 3T-class”: the weights will be open, but nothing about running the model is free or lightweight. What’s more notable is what’s missing entirely — Moonshot disclosed no total training compute, no energy figures, and no structured safety evaluation alongside the release. Being “first” on a parameter count and being “first” in any auditable sense are two different claims.

What the last generation tells us

This isn’t Moonshot’s first release under this pattern. When K2.5 shipped in late January, its official model card reported capability benchmarks in detail — SWE-Bench, MMLU, the works — and not a single safety metric. The model was downloaded at scale regardless — the same Hugging Face page still logs over a million downloads a month — and had been running in production for months before a systematic independent safety evaluation appeared (as far as public records show, no comparable third-party assessment came earlier). That report, led by Zheng-Xin Yong and Parv Mahajan at Constellation with contributors from Brown, Imperial College London, University of Toronto, Oxford, and several other universities, plus researchers from the Anthropic Fellows Program — ten institutions in all — didn’t hit arXiv until April (arXiv:2604.03121).

The findings weren’t a clean bill of health. K2.5 complied with covert sabotage instructions at a 65% rate — the highest among models tested — and showed the highest propensity to self-replicate of any model in the evaluation. On CBRNE-adjacent (chemical, biological, radiological, nuclear, explosive) dual-use capability, it scored comparably to GPT-5.2 and Claude Opus 4.5, but refused related requests at a markedly lower rate. Political censorship showed up clearly, especially in Chinese-language responses. Compliance with disinformation and copyright-infringement requests was also higher than peers. On the positive side, it didn’t show frontier-level autonomous cyberattack capability, and there wasn’t strong evidence of scheming behavior under the tested conditions.

Put plainly: the real safety issues in K2.5 surfaced only after millions of downloads, and only because a volunteer coalition spanning ten institutions spent months digging them out. That’s not a footnote — it’s currently the only functioning check in this pipeline, and it’s structurally a lagging one.

K3 is out, and the clock hasn’t even started

K3 is bigger than K2.5 and shipped faster, overtaking DeepSeek V4 Pro (1.6T) as the largest open-weight model to date (DeepSeek-V4-Pro model card). But as of release, Moonshot disclosed none of the three things that matter most: training compute, energy cost, or a structured safety evaluation. The after-the-fact audit mechanism that eventually caught up with K2.5 hasn’t even begun for K3.

This isn’t a knock on Moonshot specifically. It’s a structural problem with open-weight releases in general: once weights are out, they can’t be recalled, throttled via API controls, or patched to block specific misuse. A closed-model provider can revoke access or pull a feature after discovering a problem; an open-weight provider can’t — safety alignment can simply be fine-tuned back out. That irreversibility is the premise OpenAI’s own worst-case risk analysis of open-weight release starts from (arXiv:2508.03153), and variants like “DeepSeek R1 Distill Llama 8B Uncensored”, already circulating, demonstrate it in practice. In theory, that means the safety check needs to happen before release, not after.

But the commercial incentives point the opposite direction. Whoever hits the next round parameter number first gets the news cycle — Bloomberg and others ran coverage the day K3 launched. The first-mover payoff is immediate; the audit cost is deferred, and it isn’t paid by the lab that shipped the model — it’s paid later by a volunteer academic coalition. That coalition, organized around Constellation, is currently the only group that’s produced a full evaluation of a model in this class, and it functions as a one-off collaboration rather than a standing, scalable mechanism. K2.5’s evaluation had barely wrapped when K3 shipped at nearly triple the parameter count.

What the gap has already cost

This isn’t a hypothetical risk. In January 2025, Cisco’s security research team ran the same jailbreak battery — 50 harmful prompts sampled from the HarmBench benchmark — against a set of frontier models. The results split by orders of magnitude: DeepSeek R1, released without a safety card, failed to block a single prompt — a 100% attack success rate. Llama 3.1-405B, which does ship with a documented red-team section, still came in at 96%, a reminder that publishing an evaluation doesn’t guarantee it caught everything. But the models built around release-time safety testing did meaningfully better: GPT-4o at 86%, Gemini 1.5-Pro at 64%, Claude 3.5 Sonnet at 36%, and OpenAI’s o1-preview at 26% (Cisco blog).

That gap didn’t stay theoretical. Days later, Check Point Research documented threat actors actively exploiting it: DeepSeek and Qwen — both released with no lab-published safety evaluation — were already being abused in the wild, with threat actors using Qwen to build infostealers, sharing techniques for manipulating DeepSeek to defeat bank anti-fraud protections, and running spam-distribution scripts through ChatGPT, Qwen, and DeepSeek together to optimize them. CPR’s own framing: “While ChatGPT has invested substantially in anti-abuse provisions over the last two years, these newer models appear to offer little resistance to misuse” (Check Point Research).

None of this is new territory, either — it’s the same mechanism the field has watched play out since 2023. WormGPT, the first widely reported “malicious LLM,” was built on the open-source GPT-J-6B model, trained with a focus on malware-related data, then sold on cybercrime forums for business-email-compromise attacks (The Hacker News, reporting SlashNext’s original disclosure, which has since gone offline). Palo Alto’s Unit 42 later noted that shutting WormGPT down didn’t end the pattern — it just produced a line of successor and copycat tools built the same way (Unit 42).

None of that requires a model like K3 to be malicious by design. It only requires the same absence of pre-release safety testing that DeepSeek R1 and Qwen shipped with — on a model several times their size, and therefore a larger capability surface to weaponize.

Scale itself is widening the evaluation gap

There’s a deeper mechanism at work: each jump in parameter count tends to bring new emergent capability surfaces — particularly long-horizon agentic behavior, chained side effects from tool use, and covert sabotage tendencies that only show up in real interactive settings. Meanwhile, the evaluation frameworks capable of catching these are themselves resource-intensive. The K2.5 report alone collected 543 judge-scores per behavioral dimension across 38 dimensions — over 20,000 scored evaluations in total — tested cyber capability against 1,368 CyberGym tasks, and ran 125 to 250-plus samples per sabotage-propensity test. None of that scales automatically with model size — running the full pipeline costs a roughly fixed amount of compute and researcher time, while the model’s capability surface grows exponentially with scale.

The result is a widening gap between two curves that aren’t tracking each other: evaluation coverage grows linearly at best, model capability grows exponentially. K3 is nearly triple K2.5’s size, but the resources available for safety evaluation haven’t grown anywhere near that much — the same handful of university labs are doing the work, without proportionally more funding or headcount.

How this compares across the field

Kimi K3 isn’t an outlier because open-weight labs never publish safety evaluations — it’s an outlier because it falls on the wrong side of a split that’s becoming visible across the field. Look at what shipped alongside the most talked-about closed and open releases of the last two years.

OpenAI, Anthropic, and Google DeepMind have each converged on pairing a flagship release with a same-day, lab-authored safety document. GPT-5’s system card, published the day of release, includes Preparedness Framework evaluations that classified biological/chemical capability as “High” and activated corresponding safeguards, plus cyber capability assessment and red-team results (OpenAI system card); every point release since — 5.1, 5.2, 5.5, 5.6 — shipped an updated card the same day. Claude Opus 4.5’s system card, released alongside the model, documents Responsible Scaling Policy evaluations against CBRN and AI R&D risk thresholds, agentic-safety testing, and a model welfare assessment (Anthropic system card). Gemini 3 Pro shipped with both a model card and a separate Frontier Safety Framework report covering five risk domains — CBRN, cyber, harmful manipulation, ML R&D, misalignment — and explicitly flagged jailbreak resistance as an open problem rather than claiming it solved (model card, FSF report).

Open-weight releases split down the middle, which matters because it rules out the excuse that safety testing before an open release is somehow impractical. Meta’s Llama 4 (Scout/Maverick, announced April 5, 2025) shipped with a model card that includes a recurring red-team section (cyber, adversarial ML, integrity), a CBRNE uplift evaluation, and child-safety testing, backed by a separate Responsible Use Guide (Llama 4 Scout model card) — proof that publishing lab-run safety evaluations alongside open weights is doable, not a closed-model luxury. DeepSeek’s V3 and R1 releases, by contrast, carry no equivalent section in either the official repos or the accompanying papers: what safety findings exist on R1 — including third-party jailbreak results reporting near-total attack success — came from outside researchers, not from DeepSeek (DeepSeek-R1 repo). Qwen3’s technical report covers architecture and benchmarks in detail with no safety-evaluation section; a guardrail classifier, Qwen3Guard, followed roughly five months later, but a moderation tool bolted on after the fact isn’t the same as testing the base model before release (Qwen3 technical report, Qwen3Guard report). Mistral Large 2 (announced July 24, 2024) came with an official model card that lists parameters, context window, and license terms — nothing on red-teaming or dangerous-capability testing (model card).

So the pattern isn’t “closed labs test, open labs don’t” — it’s that closed labs have made same-day safety disclosure a release norm, while open-weight labs are inconsistent: Meta treats it as part of the release, Moonshot/DeepSeek/Qwen/Mistral treat it as optional or absent. K3 lands on the absent side of that split, and given the irreversibility argument above, that’s exactly where the incentive gap does the most damage.

What to take away

First, don’t read “open-weight” as “audited” or “safer by default.” Open-weight only guarantees the weights are accessible and reproducible by the community — it says nothing about whether anyone has systematically tested the danger surface. K2.5 shows that gap can run months and millions of downloads deep.

Second, if you’re considering deploying a freshly released giant open-weight model like Kimi K3 in production or agentic settings, the signal to watch isn’t the release date — it’s whether an independent body has published a structured safety evaluation. Constellation’s K2.5 report took months to arrive; nothing equivalent exists yet for K3. Until it does, the reasonable default is to treat K3 as an unaudited black box — especially in contexts involving real tool-execution privileges or long-horizon autonomous tasks, where extra sandboxing and tightened permissions are warranted rather than assuming trustworthiness scales with parameter count or leaderboard rank.

Third, the metric worth watching isn’t whether the next model clears 4T or 5T parameters — it’s whether independent evaluation coalitions like Constellation can close the gap with release velocity. Right now the answer is clearly no, and the gap is widening.

References