From April to June 2024, a sequence of events transformed public discourse on AI insiders’ right to warn the public about risks. At the center of this transformation was Daniel Kokotajlo, whose decision to speak publicly about his concerns at OpenAI helped set in motion the collective movement that resulted in the Right to Warn letter—and ultimately changed the conversation about insiders’ right to speak up in the public interest.

The Cost of Speaking Up Freely

On 11 May, in a comment on LessWrong, Daniel publicly confirmed that he had chosen not to sign OpenAI’s non-disparagement agreement — and that the equity he forfeited represented approximately 85% of his family’s net worth. News outlets subsequently estimated the amount as approximately $1.7 to $2 million in vested equity.

Even while going public, Daniel noted he remained bound by confidentiality obligations from when he joined the company:

“To clarify: I did sign something when I joined the company, so I’m still not completely free to speak (still under confidentiality obligations). But I didn’t take on any additional obligations when I left.”

At the same time, he was clear that his motivations were about retaining the right to speak up:

“…Basically I wanted to retain my ability to criticize the company in the future. I’m not sure what I’d want to say yet though & I’m a bit scared of media attention.”

Daniel’s LessWrong’s public statement on this matter, however, quickly gained traction, going viral on X and drawing media attention.

On 17 May, Vox’s Kelsey Piper published an investigation into OpenAI’s exit agreements, explicitly referencing Daniel’s disclosure. The reporting revealed that OpenAI had been pressuring departing employees into signing exit agreements containing extremely broad, legally binding non-disparagement (although potentially not valid) and non-disclosure provisions. By signing, employees were permanently restricted from mentioning the existence or terms of the NDA, publicly criticizing OpenAI, or doing anything which might cause the company financial or reputational damage [4]. If they did not agree to sign, or violated the contract’s terms, the employees risked losing vested equity worth potentially millions of dollars [5]. This appears to have been a common consequence under OpenAI’s departure agreements since 2017 for employees who did not sign [6]. The exit agreements also prohibited employees from acknowledging their existence [7], the scope of these restrictions was not publicly known before Daniel’s disclosure. As Zvi Mowshowitz noted:

“No one knew about this until recently, because until Daniel Kokotajlo everyone signed, and then they could not talk about it. Then Daniel refused to sign, Kelsey Piper started reporting, and a lot came out.”

In a later interview, Piper confirmed that within three hours of the article’s publication, OpenAI contacted Daniel directly. The following day, Sam Altman posted on X stating that OpenAI would not cancel any former employee’s vested equity. OpenAI subsequently sent an internal memo to former employees confirming that it “has not canceled, and will not cancel, any Vested Units” regardless of whether they had signed the agreement [8].

On 24 May, one LessWrong user reflected on the sequence of events:

“So first the 85% net worth thing went quite viral several times and made Daniel Kokotajlo a bit of a heroic figure on Twitter.

Then Kelsey Piper’s reporting pushed OpenAI to give back Daniel’s vested units. I think it’s likely that Kelsey used elements from this discussion as initial hints for her reporting and plausible that the discussion sparked her reporting, I’d love to have her confirmation or denial on that.”

However, this account of the causal chain has not been independently confirmed by Kelsey herself.

The Start of Coalition Building: From Individual to Movement

In the weeks that followed, a broader effort began to take shape, though little is publicly known about how the coalition formed.

An Associated Press report published on the day of the letter’s release described both Daniel Kokotajlo and fellow former OpenAI engineer Daniel Ziegler as co-organizers. Harvard Law Professor Lawrence Lessig represented the group pro bono.

Daniel Ziegler had worked at OpenAI from 2018 to 2021 and helped develop techniques that would later underpin ChatGPT. As he told the AP, the development of more powerful AI systems was “moving fast and there are a lot of strong incentives to barrel ahead without adequate caution.”

The coalition they helped assemble brought together former OpenAI researchers including Jacob Hilton, William Saunders, and Carroll Wainwright; former Google DeepMind employee Ramana Kumar; current DeepMind researcher Neel Nanda; and six anonymous signatories; four of whom were current OpenAI employees and two of whom were former employees. The result was a letter representing voices from across the frontier AI landscape.

Restrictive NDAs

In May 2024, prior to the Right to Warn letter, OpenAI’s CEO Sam Altman posted an apology on X about the company’s NDAs, promising not to enforce the most restrictive provisions, as well as highlighting that he had not known about equity clawback provisions being present in the agreements [12]. However, evidence to the contrary was shared by Vox, who reported that OpenAI’s senior leadership, including Altman, had been aware of, and signed-off on, the clawback provisions [13].

OpenAI shared an internal memo (first reported by Bloomberg) with past and current employees stating that “Regardless of whether you executed the [nondisparagement] agreement, we write to notify you that OpenAI has not cancelled, and will not cancel, any vested units” [14]. The company also promised to release former employees from existing non disparagement obligations (unless the nondisparagement provision was mutual) [15].

To date, five OpenAI employees have disclosed that the company has released them from the restrictive agreements [16]. However, according to a LessWrong post co-authored by Adam Scholl (a researcher at Missing Measures), over 500 people may have signed these agreements [17].

In the same memo, OpenAI promised to remove nondisparagement clauses from the company’s standard exit agreements [18]. While this represented meaningful progress, as LawfareMedia observed, their promises “did not change the underlying legal reality that allowed OpenAI to propose the NDAs in the first place, and that would allow any other frontier AI company to propose similarly broad contractual restrictions in the future.” [19]

The whistleblowers were represented pro bono by Harvard Law Professor Lawrence Lessig, who initially approached Daniel Kokotajlo after the Vox news story about OpenAI’s restrictive NDAs [23]. Lawrence argued for establishing “Right to Warn” protections, analogizing them as to having fire alarms in schools [24]. He stated that, to protect the former employees, all legal options would be examined including suing OpenAI if necessary [25].

In response to OpenAI’s official statement (see above), Daniel Ziegler – a former OpenAI employee and signee of the Right to Warn letter – told CNN that it’s important to remain skeptical of the company’s commitment to transparency:

“It’s really hard to tell from the outside how seriously they’re taking their commitments for safety evaluations and figuring out societal harms, especially as there is such strong commercial pressures to move very quickly…It’s really important to have the right culture and processes so that employees can speak out in targeted ways when they have concerns“ [26].

His statement was referenced by several other media outlets thereafter.

In July 2024, just under a month after the issuing of the Right to Warn letter, whistleblowers from OpenAI filed a legal complaint about the same concerns with the US Securities and Exchange Commission (SEC). The complaint alleged that OpenAI’s NDAs were a violation of federal whistleblower laws, and thereby illegally prohibited employees from warning regulators about serious AI risks [27].