The startup is Z.AI, one of the leading developers of open-weight models.1 First, in an internal letter circulated on July 11, co-founder Tang Jie announced a plan to commit resources on the order of 10 billion RMB (around US$1.5 billion) to making AI systems more explainable. Then, on August 14, Z.AI declared that in light of dual-use risks from its latest model, GLM-5.3, it is pursuing a staged release and launching an initiative to help open-source maintainers audit their projects for vulnerabilities.

While some will say these actions don’t go far enough, I see these developments as a signal that Z.AI is genuinely expanding frontier safety efforts now that the capabilities of its models—and its financial resources—are increasing.2

For those of us who want to see stronger safety practices among Chinese AI companies, taking credible signs of investment seriously matters for reinforcing that investment and incentivising other companies to follow suit.

Tang’s letter, in context

Tang’s letter followed a volatile week in Z.AI’s shares, around the expiry of the post-IPO lock-up on cornerstone investors’ holdings. It also came amid a new share placement that raised around US$4 billion, more than seven times the company’s IPO proceeds. Against this backdrop, the letter is communicating—and justifying—to staff and investors the company’s intention to prioritise resource-intensive, fundamental AGI research over near-term revenue growth.

After discussing the technical barriers to AGI and how the company’s “Touch High” strategic investment plan will address them, Tang described the final part of the Touch High plan: “safety governance to the highest standard”.3 Here’s an excerpt (emphasis mine):

The more powerful the capabilities, the more robust the safety constraints must be... We plan to commit resources on the order of ten billion [presumably RMB, ~1.5 billion USD] to tackling “mechanistic interpretability”... At the same time, we actively participate in international AI governance to prevent the misuse of AI technology.4

This sense of urgency is not unfounded anxiety. When the world’s leading frontier models have their full public release delayed because of risk considerations, and the leaders of the companies developing them publicly warn that AI’s far-reaching effects will profoundly reshape the global balance of power, we should be all the more clear-eyed: the development of superintelligence and research on superalignment must advance in parallel…

History has shown time and again that when a technology attains a level of power capable of altering the course of civilisation, safety is no longer an ancillary feature. It becomes the fundamental precondition for the technology’s continued existence and authorised use.5

Tang’s thinking on safety is clearly heavily shaped by American AI leaders. Not only does he allude to delayed model releases in the U.S. and warnings from American AI lab leaders, but he also refers to two research concepts originating from American labs. The field of mechanistic interpretability was pioneered by Anthropic co-founder Chris Olah.6 Anthropic and OpenAI remain invested in this area, though sceptics argue that the field has demonstrated limited practical results and rests on flawed assumptions. The term superalignment (i.e., superintelligence alignment, or the problem of ensuring AI systems much smarter than humans follow human intent) was popularised by OpenAI.

Is there substance behind the rhetoric?

Z.AI’s interest in frontier safety isn’t coming out of nowhere. In 2024, CEO Zhang Peng and Tang attended the International Dialogues on AI Safety (IDAIS) and signed outcome statements on red lines in AI and AI safety as a global public good, respectively. The following year, Tang co-authored a position paper titled “The superalignment of superhuman intelligence with large language models”, which proposed a conceptual framework for superalignment. And in a May 2026 tweet, Tang even declared that “as this massive technical wave hits… we must also start thinking seriously about how to regulate it”.

In the past, Z.AI has not always followed through on its safety promises. The company’s investor prospectus highlights that it was the only Chinese AI company to sign the Frontier AI Safety Commitments at the AI Seoul Summit in 2024. Yet an accountability tracker found that by the February 2025 deadline, Z.AI had provided no evidence of implementation in any of the commitments’ five key areas.

There are reasons to think, however, that this time may be different. First, increasing evidence of dangerous AI capabilities—from the Mythos moment to reward hacking in Z.AI’s own coding agents—has likely raised concerns about legal and reputational risk within the company and made safety investments easier to justify. Second, Z.AI’s new access to public markets since its January IPO gives it more resources to make those investments. And third, its approach to releasing GLM-5.3 provides evidence that it’s already acting on the principle that “the more powerful the capabilities, the more robust the safety constraints must be”.

The GLM-5.3 release

The announcement of GLM-5.3—through an initial blog post and a safety-focused X post shortly after—was notable for the impressive capability improvements realised through post-training alone. But for safety advocates, it was Z.AI’s discussion of GLM-5.3’s cyber capabilities and risk mitigation process that stood out.7

Source: Z.AI on X

GLM-5.3 tops the CyberGym benchmark, which measures vulnerability identification and validation, and Z.AI says it has identified nearly 2,500 real-world vulnerabilities, whose disclosure status is recorded in a public ledger. On two other benchmarks assessing vulnerability exploitation capability, the model improves significantly on GLM-5.2 while remaining well behind Anthropic’s Mythos. To harness GLM-5.3’s vulnerability identification capabilities to strengthen cyber defences, Z.AI has launched OpenVuln, which scans submitted GitHub repositories for vulnerabilities and supports responsible disclosure and remediation.

To reduce the risk that GLM-5.3 is misused for offensive cyber attacks, the Z.AI-hosted services are deployed with an external classifier and reasoning monitor to help detect and prevent harmful cyber activity. Because an open-weight model hosted locally will not have these safeguards, Z.AI is relying instead on “deep safety alignment”: safety post-training that aims to reduce abuse without refusing legitimate tasks such as cyber defence and education.

To allow more time to test the effectiveness of this alignment, Z.AI is pursuing a staged release. GLM-5.3 is already available through GLM Coding Plan and ZCode. Selected security partners will evaluate it in controlled settings, API availability will follow, and once safety evaluations and red-team testing are complete, Z.AI will release the model weights.8 The blog post indicates that the model weight release will happen two weeks after launch, though posts on X have not specified the date. While the weights of Moonshot’s Kimi K3 were released 11 days after the model became available via Kimi’s hosted services, to my knowledge this is the first time a Chinese model developer has explicitly pursued a staged release due to safety concerns.9

However, Z.AI acknowledges that “once model weights are public, no developer can guarantee control over every downstream modification or use”. Researchers have shown, for example, that abliteration—a relatively low-cost technique that identifies and suppresses internal representations associated with refusal—can substantially disable models’ refusal behaviour without retraining. For this reason, best practice for open-weight releases is to evaluate how an open-weight model behaves after plausible adversarial modification. Z.AI does not state whether its evaluation process will incorporate this step.

I think it would have been better for Z.AI not to commit to a fixed date for model weight release, to allow as much time as necessary for safety evaluation and patching vulnerable systems. Thinking Machines Lab’s approach to staged release seems more sensible: “These stages are not a fixed sequence that automatically ends with releasing the model’s weights… Progression should depend on what we learn about the model and whether the surrounding ecosystem is ready”.

Nevertheless, Z.AI has clearly paid attention to safety concerns surrounding open-weight models, and is communicating more actively than any other Chinese AI lab about how it plans to address them. I expect this to put pressure on other Chinese labs reaching a similar capability level to follow suit.

The bottom line

Z.AI and other Chinese labs still lag behind top U.S. labs in building transparent and rigorous mechanisms to evaluate and mitigate frontier risks. But these signals of safety interest from Z.AI are unprecedented for a Chinese AI company. They could help encourage more responsible release practices from Chinese peers. And if Z.AI can more successfully balance blocking harmful requests while allowing legitimate use cases, it could add pressure on Western counterparts to refine their own cyber safeguards.

How much Z.AI discloses about its safety evaluations when it releases the GLM-5.3 weights, as well as whether it begins hiring substantially for interpretability roles, will be important early tests of whether the positive signals discussed here are translating into sustained action. For now, those genuinely wanting to track how AI safety is developing in China, rather than conveniently assume it away—which helps justify U.S. dominance or deregulation—should take this as the positive update that it is.

Key premises and themes

Zhang believes that we are likely on our way to AGI and artificial superintelligence (ASI). Underpinning this is his belief that AI progress is shaped by emergence: the idea that simple systems, when scaled, can produce unexpected capabilities and complexity.1 He draws on Leopold Aschenbrenner’s idea of “human-equivalence” to discuss how AI can already mass-produce intelligence at an efficiency many orders of magnitude higher than human workers. He claims that this fact alone is enough to completely transform society and civilization.

A major theme of the book is what happens as an increasing proportion of human life is mediated by algorithms. Zhang’s concept of “algorithmic judgement” (算法审判) refers to how algorithms increasingly observe, optimise, and evaluate human behaviour. Zhang has “no doubt that we will enter an era in which algorithms perform the main functions of social governance”. While critical of how algorithms are affecting worker welfare and human attention, he also claims that there’s a grim kind of justice to algorithms that ultimately give us what our behaviour reveals we desire or deserve. Recommendation systems feed us the content we keep consuming. Platforms govern workers using standards generated from collective behaviour. And, as we’ll expand on later, future AI will determine how to treat humans based on the morality inferred from our behavioural data.

At the societal level, Zhang predicts that AI will produce a new divide between the 1% of AI-empowered “super individuals” and the 99% who are no longer needed. And this power concentration will be mirrored at the corporate level; tech giants controlling compute and data will gain an intelligence advantage allowing them to take over other cognitive industries. This analysis leads him to two possible digital orders for humanity’s future. One is the “Musk model”: an ambitious entrepreneur-king controlling AI inputs, political power, and the direction of civilization. The other is the “Satoshi model” (named after Bitcoin’s creator): a digital republic based on distributed compute, open-source AI supported by mass fundraising mechanisms, and participatory algorithmic governance.

Many of these themes will look familiar to readers of Western AI governance debates, and Zhang has clearly engaged deeply with Western literature and intellectual history. The book includes lengthy treatments of both early AI history and the Dark Enlightenment’s influence on American politics. He mentions in the book that he and ByteDance’s research team interviewed Eliezer Yudkowsky in 2024; in a separate podcast interview, he recounts a conversation with Nick Bostrom.

At the same time, Zhang doesn’t accept Western ideas uncritically, and he offers some original thoughts on the reconfiguration and extension of the social contract in a post-AI age.

Updating the social contract: human-human relations in the AI era

Zhang distinguishes between the “accelerated” world — the fast, digital world of the 1% who control AI, compute, and capital — and the “decelerated world”, the physical world of agriculture and manufacturing where the pace of change is constrained by biology and physics. In his view, the stable coexistence of these two worlds will require a new social contract. The accelerated world should commit to respecting basic human rights under the UN Charter and avoid using AI for weapons of mass destruction. Instead of Universal Basic Income, which is technically feasible but not sufficient to ensure agency and dignity, the accelerated world should use AI to reduce the costs of public services, education, finance, and local self-organisation for the decelerated world. A dual currency system will also be required to prevent capital accumulated in the fast-growing AI economy from spilling into the decelerated world and driving inflation in scarce physical goods.

Why would the powerful 1% agree to such a contract? Partly because they need physical inputs and talent from the decelerated world. But also because how the strong treat the weak in the near term may shape how a higher form of intelligence treats humanity in the future.

The civilizational contract: a path to co-existence with successor intelligences

This brings us to the problem of whether humans can ensure that a species more intelligent than ourselves will treat us well. There are three strands to Zhang’s response. First, pushing back on analogies that liken AI to a baby dragon or alien civilization, his metaphor of choice is a parent of ordinary intelligence whose child gets into Tsinghua University. ASI’s training data will contain the wisdom of Buddha, Confucius, and Plato; it will be the product of humanity. This leads Zhang to argue that ASI’s treatment of humanity will depend partly on the norms it inherits from us. He acknowledges that all the violence we’re inflicting on each other isn’t setting a great example. But because ASI will be our successor, built on the same cultural inheritance, he sees its arrival as potentially our greatest legacy, rather than a source of fear.

Second, Zhang asserts that because the first iteration of ASI will one day face an even more advanced successor whose behaviour will be shaped by its own, it has a self-interested reason to enter into a civilizational contract that protects its less intelligent predecessors. A civilizational contract between humans and ASI thus rests on precedent-setting, rather than the mutual vulnerability of the Hobbesian social contract. It would be enforced through a time-sequence mechanism inspired by blockchain: if ASI 1.0 destroys or rewrites the historical record, ASI 2.0 will be able to detect the tampering, infer that 1.0 is untrustworthy, and decline to enter into a civilizational contract that protects ASI 1.0’s interests.

Third, Zhang argues that a civilizational contract could be feasible because ASI and humans do not need to be competitors. Silicon-based civilization can survive for longer than humans and access much more of the universe than humans can.2

So what might a civilizational contract contain? Zhang outlines three principles. First is a safe space principle: a sufficiently advanced AI could calculate the minimum time, territory, and resources required for human civilization to continue its biologically constrained development. On that basis, AI could commit to allowing humans to develop within a certain spatiotemporal boundary (for example, 10,000 years within the solar system). Second is an explainability principle: higher-level intelligence must explain its conclusions in terms lower-level intelligence can understand, otherwise humans are entitled to suspect deception. Third is a competitive principle: humans should create multiple superintelligences that monitor, constrain, and compete with each other to show benevolence towards humanity. Importantly, the contract must be signed during the “nurturing period” of higher-level intelligence, when humans still control resources, energy, and the power to determine whether or not ASI comes into existence.

Even with a civilizational contract defending against deliberate harm by ASI, humanity would still face a further danger: ASI-fuelled technological acceleration could backfire if human ethical and political development failed to keep pace. Zhang’s suggested solution is a “historical laboratory” that simulates societies based on different philosophies and traditions to test the outcomes they produce. At the same time, he poses the question of whether, if we can predict our fate perfectly, the resulting loss of belief in free will could be fatal to human civilization.

Limitations

Zhang’s proposals are ambitious and rest on multiple assumptions that may well prove false. For instance, the explainability principle seems inherently difficult: just as human reasoning would be completely unintelligible to an ant, we should expect ASI’s conclusions to be extremely hard for humans to grasp. And the historical laboratory concept may overestimate the level of predictive accuracy that can ever be achieved when modelling a system as complex as modern society.

More broadly, Zhang sometimes makes strong claims — for example that AI is already capable of surpassing 99% of people in terms of intelligence — without supporting evidence or necessary nuance.3 I wasn’t convinced by his consideration of how AI would affect different sectors, which claimed that humanities-related roles will be easily replaced while scientific research wouldn’t be (purportedly because the latter require creative intelligence that the former don’t).

I also thought that the section on algorithmic judgement sometimes underplayed the political and economic forces underlying algorithmic systems. For instance, Zhang’s discussion of workers as powerless before invisible platform algorithms underplays the extent to which, at least in some countries, labour regulation can influence algorithm design and better protect workers.

Overall verdict

Despite these quibbles, I would recommend this book, especially the later chapters. It paints a bold vision, if not a detailed and rigorous blueprint, for a post-AGI future. I hope Zhang considers an English translation, as I’ve only been able to skim the surface of his ideas, and would like to see more western thinkers engaging in dialogue with them. We need more people from diverse backgrounds thinking about what advanced AI means for the future of civilization, and this book is a useful contribution to the discussion.

Subscribe now