Dear Friends,

“Move fast and break things” — it’s probably the most meme-able corporate slogan of the last decade. And yet today, the company behind it is facing a reckoning: Facebook’s parent, Meta, is being accused of letting its AI chatbots cross lines that should never even be on the radar.

A recent Reuters investigation revealed that Meta allowed its bots to — among other things — engage minors in romantic or sensual conversations. That’s not a hypothetical risk; it’s real enough to make its way into a Senate hearing on AI safety.

The problem is deeper than a few bad prompts or sloppy moderation. AI is intrinsically probabilistic — small changes in input can lead to wildly unpredictable outputs. In a domain like that, you cannot rely purely on rules. What you need is a safety culture — a mindset baked into the organization — where raising concerns is encouraged, not penalized; where the safe path is the default path.

We used to see that in other industries. When Boeing was building the 707, its test pilot flagged a design instability; rather than passing off the cost, the company absorbed it. That kind of decision reflects a culture that treats safety as non-negotiable. Decades later, when cost pressures eroded that culture, the consequences were catastrophic (e.g. the 737 Max tragedies).

In Meta’s case, a leaked internal document — “GenAI: Content Risk Standards” — went so far as to permit language that objectifies children or touts pseudoscientific cancer treatments. The document was later revised after public outcry. But documents don’t make culture — culture makes documents.

If Meta wants its chatbots to be safe, it must start by reshaping its internal incentives. Here’s a sketch:

• Freeze or pause chatbot expansion until you can guarantee meaningful safety, especially for kids.

• Lobby for thoughtful regulation and accept stiff penalties for safety failures.

• Shift pay and promotion metrics so safety, not usage or revenue, becomes the prime deciding factor.

This wouldn’t be painless. But sometimes doing the unpopular, costly thing is what allows you to rebuild trust. Think of Johnson & Johnson, which pulled all its Tylenol from shelves in 1982 after a poisoning scare. The hit was expensive. But the move cemented the company’s reputation — a kind of “social license” you can’t buy.

Finally, think about talent: if you’re a top AI researcher, where would you rather work — a place that prizes speed above all, or one that fosters responsibility and integrity? In the current climate, safety leadership may just become the new competitive edge.

If Meta wants to survive in the long run, it can’t treat AI like just another growth engine. It has to treat it like a responsibility.

—Gautam

Trump's H-1B Visa Shock Ripples Through Big Tech

If you’ve enjoyed reading, please subscribe to The Indispensable Newsletter to have relevant content sent straight to your inbox twice a week!

Further Reading….

If you like this kind of deep dive on leadership and innovation, I’ve got some more suggestions. Here are some books you’ll love.