Announcement and the headline number
On August 14, Zhipu (also known as Z.ai) unveiled GLM-5.3, a model focused on programming. In the technical release, it immediately compared the new model with leading international developments. The centerpiece was its result on the CyberGym benchmark: 84.5% versus 83.8% for Anthropic Mythos 5 and 83.6% for OpenAI GPT-5.6 Sol. That advantage looks solid and earned the announcement a spot in news headlines. But if you dig deeper, it turns out this win is almost the only place where GLM-5.3 genuinely outpaces its competitors. The developer itself essentially admits this, albeit in a veiled way.
An important clarification: this is not about comprehensive security, but about finding vulnerabilities in source code. This is precisely the point that often gets lost when the news is retold, creating a false impression of leadership.
CyberGym: why a narrow test doesn't make a model a leader
CyberGym is fairly simple: models are given source code, and the model must find a vulnerability in it and confirm it. This is certainly a useful skill, but it reflects only the first stage of a real cyberattack. Zhipu itself calls CyberGym the narrowest of the three published security metrics. The other two tests, ExploitBench and ExploitGym, show that with the full "find → exploit" cycle, things are not so rosy for GLM-5.3. Moreover, the gap on CyberGym is just 0.7 percentage points — given a single run (pass@1 on 1,507 tasks) and no variance reported, this difference is unlikely to be statistically significant. Simply put, this result doesn't prove superiority; it only indicates that the models are roughly on par in one specific narrow task.
ExploitBench and ExploitGym: the real state of affairs
Let's start with ExploitBench. Here GLM-5.3 scores 54.4%, which is double what its predecessor achieved (24.4%). The growth is impressive, but it falls short of competitors: Anthropic Mythos 5 scores 78.0%, and OpenAI GPT-5.6 Sol scores 76.5%. The dynamics on ExploitGym are even more telling. In two hours, GLM-5.3 completes 105 tasks; in six hours, 130. Mythos 5 solves 181 and 247 tasks in the same timeframes, respectively. The gap is multiples. Zhipu itself notes: the further along the "vulnerability discovery → exploitation" chain, the greater the model's lag. In other words, if CyberGym is the only point where GLM-5.3 holds its own, then as soon as the task becomes more complex, the model loses more and more noticeably.
Coding: mixed results and odd comparisons
Since the model is positioned as a tool for developers, it's reasonable to expect strong results on coding benchmarks. In reality, things are mixed. In the main table, Zhipu compares GLM-5.3 with Anthropic Opus 4.8 — and it wins on some tests and loses on others. In the performance charts, a different model is listed as the rival — Claude Fable 5 — and on Zhipu's internal test, the newcomer trails behind. This approach — comparing against different models in different sections — makes it impossible to see the full picture. Moreover, GLM-5.3's evaluation was run inside an Anthropic agent, namely Claude Code 2.1.207. This means we're seeing results not of a standalone model, but of a "model + environment" combination, which also affects the final metrics.
Methodological caveats and acknowledging its own weaknesses
Zhipu makes a curious remark in its report: its cybersecurity capability "is growing fastest precisely where it lags the most." This is an honest admission that the baseline for comparison is currently weak and progress is starting from a low point. At the same time, the comparison methodology raises questions. Different sections of the report use different Anthropic models — Anthropic Opus 4.8, Claude Fable 5, and Anthropic Mythos 5. Choosing a rival "to fit a specific test" can create a distorted impression. Plus, the time budgets for ExploitGym are normalized based on throughput from Artificial Analysis; the conversion coefficients are given for Kimi K3 and Qwen3.8 Max, but not for Mythos 5. Without these coefficients, competitor results may not be fully comparable. Still, Zhipu is taking the right step: GLM-5.3's weights will be published, whereas Anthropic's comparable work remains under restricted access. This gives independent researchers a chance to verify the claimed numbers.
2,436 vulnerabilities: numbers that need context
The report also details joint work with Chinese security teams. The model analyzed real open-source projects and ultimately found 2,436 vulnerabilities across 269 repositories. These are already filtered and deduplicated results that have undergone expert review. The severity distribution looks like this:
- critical: 107;
- high: 990;
- medium: 1,286;
- low: 53.
Impressive, but there's a nuance. The summary panel in the same release lists 1,097 "critical and high" findings — which is exactly the sum of 107 and 990. However, the report text describes these 1,097 as "medium-high." Some news outlets apparently copied the second version without noticing the discrepancy. This inconsistency suggests the report was put together in a hurry. Another curious fact: the oldest vulnerability found dates back to 1981, and the average age of findings is 26.6 years. In other words, the model excels at finding long-known issues, but its success in detecting fresh vulnerabilities is far less obvious.
Conclusions
The GLM-5.3 announcement is a story of how marketing and reality diverge. On one narrow benchmark, the model did come out on top, but more comprehensive tests bring us back down to earth: there's a serious gap behind the leaders in both cybersecurity and coding. At the same time, Zhipu deserves respect for its openness: it publishes both the weights and data about the model's weak points. There's clear significant progress over the previous version, which makes it possible to look to the future with cautious optimism. But calling GLM-5.3 a security leader based on a single number from a single run would be an overstatement.
