What DPDP Expects From Your Models When AI Makes Decisions

Part one of this series looked at training data. This part looks at the other end of the pipeline: what a model produces and what DPDPA expects when that output touches a real person's rights or interests.

Most engineering teams treat a model's output as a result, a number, a recommendation, or a decision. Under DPDPA, an output that relates to an identifiable person is personal data too, generated rather than collected, and it carries its own set of obligations.

A Model's Output Can Be Personal Data Even If Its Input Was Anonymised

This is the part that surprises most engineering teams. A model can be trained entirely on anonymised data and still produce an output that is, in legal terms, personal data about a specific individual. A credit risk score, a churn prediction, a fraud flag, and a recommended interest rate – each of these is generated information that relates to and identifies a person, even if the underlying model never saw that person's name during training.

DPDPA does not distinguish between collected data and inferred data. Both are processed. Both entail the same obligations regarding purpose, accuracy, and the rights of the person whose data it is.

Automated Decision-Making Carries Its Own Set Of Expectations

When a model's output drives a decision that affects a person, whether that is approving a loan, flagging a transaction, or prioritising a support ticket, the stakes change; the person affected by that decision has a legitimate interest in understanding, at some level, why the system reached that conclusion.

This is where explainability stops being a research topic and becomes a compliance requirement. A model does not need to expose its full architecture or weights to satisfy this. It needs to be able to produce a reasonable explanation for a specific decision when one is requested, in plain language a Data Principal can actually use. If a Data Principal exercises their right to question a decision and the answer is "the model said so, that answer will not hold up.

The Purpose Limitation Problem In Model Outputs

A model built to predict customer churn for retention purposes produces a score. That score, once generated, is a tempting input for other teams: marketing wants to use it for targeting, collections wants to use it for prioritisation, sales wants to use it for lead scoring. Each of these is a new purpose, and DPDPA's purpose limitation principle applies to the output exactly as it applies to the original input data.

An output generated for one purpose does not automatically become available for every purpose someone in the organisation finds useful for it. This sounds obvious stated plainly, but it is one of the most common ways model outputs drift outside their original consent boundary, because outputs feel like internal artefacts rather than personal data, even though legally they often are exactly that.

Logging Outputs Without Creating A New Liability

There is a tension worth naming directly. Logging model inputs and outputs is good practice for debugging, monitoring, and audit purposes. It is also, itself, a form of processing personal data, and those logs need the same security safeguards, retention discipline, and access controls as any other store of personal data.

A common failure mode is a logging system that captures every prediction indefinitely, with broad internal access, because nobody has assigned ownership of the log's lifecycle. That log becomes a growing liability rather than a useful audit trail. The fix is not to stop logging. It is to apply retention limits, access controls, and a clear purpose to the logs themselves, the same discipline that incident management practice already expects of any system holding personal data.

A Practical Checklist For Model Outputs

1. Treat any output that relates to an identifiable person as personal data, regardless of whether the training input was anonymised.
2. For decisions with real consequences, confirm the system can produce a plain language explanation for a specific outcome on request.
3. Check whether an output generated for one purpose is being reused for a different purpose elsewhere in the organisation.
4. Apply retention limits and access controls to prediction logs, not just to the original input data.
5. Document who is accountable for explaining an automated decision if a Data Principal challenges it.

How Privy By IDfy Supports Model Output Governance

Privy by IDfy extends visibility beyond input data to the outputs and decisions an AI system generates. InspectAI scans digital journeys for consent misalignments, which includes flagging when a model's output is being used for a purpose the original consent never covered.

Data Principal rights workflows connect to this same layer, so when a person questions an automated decision, the response is built on a real record of what data and purpose drove that decision, not a reconstruction assembled after the fact.

Conclusion

A model's output is not exempt from DPDPA just because it was generated rather than collected. If it relates to an identifiable person, the same obligations around purpose, accuracy, and explainability apply to it that apply to any other personal data the enterprise holds. Teams that build explainability and purpose tracking into their output handling now will be far better placed to answer a data principal's question, or a regulator's, when it eventually comes.

If you want to discuss how to bring this kind of output governance into your AI systems, or you would like a demo of how Privy by IDfy connects consent to AI-driven decisions, write to shivani@idfy.com.

FAQ's

Is a model's output considered personal data under DPDPA? Yes, if the output relates to and can identify a specific person, such as a risk score or a recommendation, it is personal data regardless of whether the training input was anonymised.

Does DPDPA require AI decisions to be explainable? DPDPA does not use the word 'explainability' directly, but a Data Principal's rights to question and challenge processing means a system needs to be able to produce a reasonable explanation for a decision affecting them.

Can a model output generated for one purpose be reused for another? Not automatically. Purpose limitation applies to outputs the same way it applies to the data that produced them, so reusing an output for a new purpose needs its own lawful basis.

Do prediction logs count as personal data that needs protection? Yes. Logs of model inputs and outputs that relate to identifiable people are personal data and need the same retention limits, access controls, and security safeguards as any other store of personal data.

What should a team do first to govern model outputs better? Start by mapping which outputs relate to identifiable people, then confirm that the purpose each output is actually being used for matches the purpose it was generated under.