The Enterprise Reality: AI Must Be Built for Real Human Behaviour
Part 2 of 2:
Public AI shows the risks. Enterprise AI must solve them.
Public AI Shows Us the Risks. Enterprise AI Must Solve Them.
The 47,000 ChatGPT logs are a helpful reminder of something we already knew deep down: public AI is built for everyone, which means it protects no one in particular. It’s designed for convenience, creativity, curiosity, and the occasional late-night existential spiral. And that’s fine – as long as you’re not handling sensitive information, regulated data, or anything with a legal department attached to it.
But organisations don’t have the luxury of “fine.” They live in a world of consequences, contracts, and compliance. For them, AI can’t simply be helpful. It has to be safe. Predictable. Transparent. And, most importantly, built around the way people actually behave, not the way we wish they would.
That’s where enterprise AI begins. Not with model size or clever prompts, but with a simple question:
How do we protect people even on their most distracted day?
Everything else flows from that.
The Enterprise Risk Landscape: What Happens When Real Life Meets Public AI
Let’s talk about the uncomfortable part first. The legal system has already shown that it sees no special privacy halo around AI conversations. In the United States, a judge recently ordered OpenAI to hand over twenty million private chats. 20. Million. Anonymised, of course – though anyone who works in data protection knows that “anonymised” often means “you’d better hope no one tries too hard.”
Combine that with the behaviour shown in the ChatGPT dataset – oversharing, emotional disclosure, accidental confessions, medical details –, and you get a perfect storm. Not because AI is malicious. Not because people are reckless. But because these systems were never designed for sensitive information in the first place.
Public AI has to keep logs.
Courts can demand logs.
Users paste confidential content into those logs.
And organisations are left with their fingers crossed, hoping none of it comes back to them later.
Hope is lovely in personal life. It is less useful in compliance.
The Hard Truth: You Can’t Train Your Way Out of Human Behaviour
Many companies respond to AI risks with a familiar package: guidelines, workshops, best practices, and the occasional all-staff memo beginning with “please do not paste confidential information into ChatGPT.”
It’s well-intentioned; responsible, in fact. It’s also doomed(!).
People don’t overshare because they’re rebellious. They overshare because they’re human, and because they’re trying to get work done quickly. Because the deadline is in half an hour. Because the person who normally helps is on holiday. And because the chatbot is simply… there.
If the 47,000 logs show anything, it’s that oversharing isn’t an edge case. It’s the default. And if it’s the default, no amount of training will eliminate it.
The only sustainable solution is to build systems that assume this behaviour and protect users automatically. The same way modern cars assume you sometimes forget your seatbelt or try to reverse into a bollard.
Real safety comes from design, not discipline.
Privacy by Design, Not by Policy: Why Anonymisation Must Happen Before AI Touches Anything
This is the point in the conversation where “privacy by design” finally earns its name. Not as a slogan or a compliance checkbox. But as the backbone of how enterprise AI should work. Especially for industries handling sensitive personal data – like healthcare, but increasingly every sector that works with real-world human information.
If sensitive data reaches a model before anonymisation, the harm has already happened. It doesn’t matter how politely the privacy policy explains it afterwards.
So in a well-built enterprise AI environment, anonymisation isn’t a patch. It’s the front door.
Here’s what that looks like in practice:
- Automatic detection of personal data, medical data, financial details, institutional names, and other identifiers the moment a file is uploaded.
- Tokenisation and Named Entity Recognition (NER)-based anonymisation, replacing sensitive elements before they touch any model.
- A human redaction loop, not for decoration, but for accuracy – especially when Optical Character Recognition (OCR) does what OCR sometimes does.
- Secure deanonymisation, performed only at the very end on a protected server.
- Tamper-evident auditing, because if you can’t see who did what and when, you don’t have compliance.
- Role-based access, ensuring only the right people see the right things at the right stage.
This isn’t about being strict; it’s about being realistic. It’s about creating an environment where the system is trustworthy because it’s built to catch the things humans naturally miss.
A good privacy-by-design system behaves like a quietly competent colleague: always watching the details, always covering your blind spots, never making you feel foolish for needing the help.
The Future of Enterprise AI: Built for Real People, Not Ideal Users
If public AI shows us anything, it’s that we cannot design enterprise systems around tidy assumptions. We have to design them around real life – which is famously untidy.
The future belongs to AI that:
- assumes oversharing will happen
- prevents risks structurally, not socially
- offers anonymization as a first-class option when workflows require it
- records interactions transparently
- lets organisations choose the right model for each task
- keeps data sovereign and properly contained
- knows when to say “no”
- supports, rather than surprises, compliance teams
In other words, AI that behaves like part of the organisation, not an unpredictable guest.
And perhaps most importantly, AI that treats privacy not as an act of willpower, but as an architectural guarantee for the workflows that need it most.
Because the strongest AI systems of the next decade won’t just be clever. They’ll be kind. Respectful. Boring in the best possible way. They’ll blend into daily workflows without creating risk in their wake.
They’ll protect people quietly, automatically, reliably – the way all good systems should.
The Next Decade Belongs to Systems That Protect Us as We Are
If Part I showed anything, it’s that humans are wonderfully, predictably human. We seek connection. We want help. We lean into anything that listens. And we occasionally trust machines a little too much.
None of that is going to change.
So the responsibility falls not on people, but on the systems we build for them. Systems that expect human behaviour and that respect it. Systems that catch mistakes before they matter. And systems that protect us not on our best day, but on our ordinary ones.
Enterprise AI doesn’t need to be dramatic. It just needs to be designed with care.
Because in the end, the real breakthrough isn’t intelligence. It’s responsibility.
And if the 47,000 chats tell us anything, it’s that the AI world is finally ready to have that conversation.
