Data Privacy and AI: What Changes When a Model Reads Data

BusinessSafety and governancePublished Updated By Simon Budziak

Data privacy is the right of people to control how information about them is collected, used and shared. For a company using AI it means a model, an agent or a vendor may process personal data only for a stated purpose, on a legal basis, with the minimum data needed, and without leaking it to anyone else.

What changes for data privacy when you use AI?

Three new leak paths open. Staff paste personal data into chat tools the company never approved, which is shadow AI. A vendor may keep prompts or use them to improve its models unless the contract or settings say otherwise. And an agent with broad access can pull personal data into an answer for someone who should never see it. Most AI privacy failures are configuration and permission failures, not model failures.

What does the GDPR require when AI processes personal data?

The core principles in Article 5 still apply: purpose limitation, data minimization, accuracy and storage limitation. Article 25 requires data protection by design and by default, and Article 35 requires a data protection impact assessment when processing, especially with new technologies, is likely to result in a high risk to people (Regulation (EU) 2016/679, read 12 November 2026). A new AI use of old data is a new purpose, and it needs its own check. Where the data lives matters too, which is the subject of data residency.

Data privacy vs data security vs data governance: what’s the difference?

Question it answers Typical control
Data privacy May we use this person’s data for this purpose? Legal basis, consent, minimization, retention limits
Data security Can an attacker reach the data? Encryption, access control, monitoring
Data governance Who owns the data and sets its rules? Owners, policies, catalog

Privacy sits on top of security: you need both, and one does not imply the other.

How do we set a privacy baseline for a client’s AI tools?

When we onboard a client onto a SaaS tool that handles their data, such as a meeting recorder, we first write down a configuration baseline for retention, access, sharing and residency, so later drift is visible. Our defaults are strict. Share nothing with attendees automatically, share links only with people added by name, opt out of letting the vendor improve its AI with the client’s data, even when anonymized, and connect only the integrations in use. We keep recording consent notices switched on and say it out loud as well, because under the GDPR that obligation is ours, not the vendor’s. A privacy setting nobody wrote down is a setting nobody will notice changing.

Written with AI assistance and reviewed by Simon Budziak. The production notes come from systems Soba Labs builds and runs.

Frequently asked questions

Is data privacy the same as data security?

No. Security keeps unauthorized people out of the data. Privacy decides what even authorized people and systems may do with personal data, so a perfectly secure system can still breach privacy by using data for a purpose nobody agreed to.

Can I put customer data into an AI tool?

Only if the use fits the purpose you collected it for, you have a legal basis, the vendor contract limits what the provider may do with it, and training on your data is switched off. Check the plan you actually pay for, since defaults differ between consumer and business tiers.

Does the GDPR apply to AI?

Yes. The GDPR applies whenever personal data is processed, whether by a person, a database or a model, and the EU AI Act adds its own duties on top rather than replacing it.

Summarize this page with

See this working in a system we built