AIWI / Case Studies / Shane Jones

Shane Jones

Shane raised concerns about Copilot Designer, Microsoft's Al image-generator tool which was able to create violent and sexualised content. Though the product was never taken off the market, certain safety features have since been updated.

Read the Press Coverage:

Company

Microsoft

Jurisdiction

US

Year

2024

Issues

Product Safety Risks

Violent and Harmful Content via AI image-generator

Channels

Internal → External

Public and Regulatory

Why This Case Matters

Shane continues to work at Microsoft as a Principal Software Engineering Manager. In his spare time, he is involved in red teaming efforts to test leading AI products.

While the AI image-generator, Copilot Designer, was never taken off market, users that tried to generate content in ways similar to those reported by Shane can no longer do so.
OpenAI confirmed that additional safeguards for their products that used DALLE-3 had been implemented.

Shane received public support on LinkedIn for sending letters to the FTC and Microsoft’s board.

OpenAI also recently thanked him for spotting an issue with their GPT-4 model. Shane noted that OpenAI values the work of independent researchers and had taken on board his feedback.

Support Received

Public Support Public Support via LinkedIn on the sending of letters to the FTC and Microsoft’s Board
Public Comment from OpenAI A public statement from OpenAI to thank Shane
Legal Support No legal support known

You don’t need to be in a crisis to seek help. Pro bono legal counsel can help you navigate your situation and find the right support.
Explore Your Legal Options →

Timeline

While testing Microsoft’s AI image generator, Copilot Designer, Shane became concerned that the product was creating violent and sexualised content, often through benign user requests. For example, the AI tool exhibited a tendency to depict demons and monsters when prompted with text related to abortion rights [1], along with sexualised images of women via prompts involving car accidents [2].

He reported the issue internally to management. Shane was told that as Copilot Designer was derived from DALL-E 3, an OpenAI image-generator, he should direct the concern to them. Having done so through the requested channel, he did not receive a response.

Shane then issued a letter to OpenAI’s board of directors, publicly viewable on LinkedIn, asking them to take the product off the market until further safety measures were put in place [3]. He also notified his management of this, and was told that Microsoft’s legal team was demanding he remove the post including the letter. He did so, on the agreement that he would receive justification as to why he was being told to remove the letter. However, Shane didn’t receive a response, despite repeatedly following up on the matter.

Having received no response from OpenAI or Microsoft, Shane escalated the matter by issuing a letter to US senators; meeting with staffers from the Senate’s Committee on Commerce Science and Transportation; and sending further letters to the Federal Trade Commission (FTC), and Microsoft’s board of directors [4].

Shane explained the safety risks posed by Copilot Designer and the lack of action taken by Microsoft and OpenAI to address them. He highlighted that these are internally known risks which are not being disclosed publicly. He stated that his recommendation to take down the image generator and/or add certain safety features, such as an age restriction for using the AI tool, had been dismissed. He implored the Senate, FTC and Microsoft’s board to investigate the company’s handling of safety risks and incident reporting, as well as insisting the company be more transparent about known AI risks.

Microsoft issued a statement about its commitment to addressing employee concerns, and while they appreciated Shane’s efforts in testing the technology, they recommended that employees use the official internal reporting channels to disclose concerns before escalating issues publicly [5]. A Microsoft spokesperson also claimed that the company had previously facilitated meetings for Shane with employees working on responsible AI, so as to review his concerns [6].

Both Microsoft and OpenAI claim that Shane’s concern was immediately investigated, and that no issue was found with the existing guardrails in place for Copilot Designer and DALL-E 3 respectively [7].

OpenAI’s statement noted safety as a company priority, and that the most explicit content had been filtered from DALL-E 3’s training data. They also confirmed that additional safeguards for their products that used DALL-E 3 had been implemented, declining user requests for images that violated their policies.

Shane responded to Microsoft’s statement noting that he did report the issue using the requested internal channel, ‘Report It Now’ security incident system, as instructed by his leadership team. The response he received, which he shared with his leadership team, was that the content of his concern did not fall under their remit, suggesting he instead submit his feedback over OpenAI’s website. The response Shane received from Microsoft also noted that they would now be closing the case.

Regarding OpenAI’s DALL-E 3 model, Shane noted that different safety measures are in place for different products making use of the model. OpenAI’s ChatGPT, for example, also makes use of DALLE-3. Shane stated that users generating AI images using ChatGPT wouldn’t get the same harmful outputs as they would via Microsoft’s Copilot Designer, because ChatGPT has different safeguards [8]. However, Shane confirmed that OpenAI had taken the time to meet with him and discuss his concerns [9].

Outcomes

For the Whistleblower

Shane still works at Microsoft as a Principal Software Engineering Manager. In his spare time, he continues to be involved in red teaming efforts to test leading AI products. OpenAI recently thanked him publicly for spotting an issue with their GPT-4 model, that was allowing users to generate images with DALLE-3 using morse code prompts that would have been refused in their English language equivalent. He responded that OpenAI values the work of independent researchers, and has taken onboard feedback relating to the issues he discovered [10].

Shane’s story received significant media attention, being covered by outlets such as CNBC, The Guardian and Business Insider, where his concerns were portrayed as serious and credible. Expert opinions were shared in some news articles covering the story, who agreed with Shane’s concerns, including Tinglong Dai, a professor at Johns Hopkins University, and Davi Ottenheimer, a vice president at startup Inrupt[11].

He also received praise on social media platforms such as LinkedIn and X for whistleblowing, with some commending the risk he took in doing so. His post on LinkedIn about his letter to the FTC and Microsoft’s Board received mostly positive reactions and comments, though there were a few negative responses from those skeptical that Microsoft would not have taken the necessary safety precautions.

Consequences faced:

(Threat of) Legal or disciplinary actions Termination or dismissal Harassment or bullying Blacklisting Other: Silencing – request to remove online content relating to concern

Note:

This is AIWI’s interpretation of the consequences faced by the individual based on publicly available information, as stated in our methodology.

For the Case

Copilot Designer was never removed from public use, despite Shane’s warnings that it presented a safety risk. There is no publicly available information as to whether any of Shane’s requests in his letters to the FTC and Microsoft’s board of directors were addressed, though FTC did acknowledge receipt of Shane’s letter.

However, some users who tried to generate images similar to those described by Shane (and which were validated through reproduction by CNBC [12]) could no longer do so, suggesting that Microsoft made certain updates after the case was publicly scrutinized.

Microsoft publicly announced that certain safety measures had been put in place after deepfake, explicit Al-generated images of people such as Taylor Swift were circulating online, allegedly created by products including Copilot Designer [13]. Shane had highlighted this example in his letter to the company’s board of directors. Microsoft noted the key safety updates included blocking specific user prompts, as well as the suspension of user accounts if repeated attempts were made at using a blocked prompt [14].

Your concerns matter and deserve expert answer

If you see something concerning, it’s already worth a confidential conversation. AIWI can help you find clarity with the support of our expert network. Confidential, anonymous and on your terms.

Get a confidential consultation →