When Systems Cannot Do What the Policy Promises

Abstract

An organization preparing for the Digital Personal Data Protection Act 2023 can exist in two different states of compliance at the same time. In the first state, the documents are in order. The privacy notice has been drafted, the consent language reviewed, the retention policy approved, and the incident response plan filed. In the second state, the systems are examined, and a different picture appears. Personal data sits in places the documents never mention, consent is recorded but nothing reads the record, retention periods exist on paper while every table grows without limit, and the logs a breach report would depend on were never switched on. The distance between these two states is the subject of this article.

The first article in this series established that the obligations of the Act resolve into system requirements. This second article examines what happens when those requirements are assumed rather than verified, and it does so at the level where the assumption breaks, which is the level of databases, applications, infrastructure, and vendor relationships. Each of these layers has its own characteristic gaps, and each gap has a specific way of turning a documented obligation into an unperformed one. The article walks through the four layers in turn, describing the gaps that appear most often and the reasons they form in otherwise well run organizations.

The examination matters because enforcement will concentrate on exactly this distance. A rights request, a consent withdrawal, or a breach does not test what an organization wrote. It tests what the organization can do, within defined time periods, with evidence retained at every step. The penalty schedule of the Act attaches its largest figures to failures of security safeguards and breach notification, and both of those failures occur in systems rather than in documents. An organization that knows the size and location of its own gap can close it in an orderly way before the core obligations commence around May 2027. An organization that has never measured the gap will learn its size from the first real test.

The article closes with a working method for that measurement. A readiness assessment of the kind described here does not review documents against a checklist. It traces each obligation end to end through the organization’s actual systems, exercises the paths a data principal or a regulator would exercise, and records where each path completes, where it stalls, and where it does not exist. The output is a mapped, prioritized, and honest account of the distance between the compliance an organization believes it has and the compliance it can demonstrate, which is the starting point of every remediation program worth the name.

1. Two States of Compliance

1.1 The document state and the system state

The document state of compliance is easy to reach and easy to see. It consists of the artifacts a compliance program is expected to produce, including the privacy notice, the consent language, the retention policy, the data processing agreements, the incident response plan, and the training records. These artifacts can be produced quickly, reviewed by counsel, and presented to a board, and their completion feels like progress because it is visible and countable. Nothing in this article argues against producing them. The argument concerns what their existence does and does not establish, because a complete document set establishes intent, and the Act regulates conduct.

The system state of compliance is reached differently and seen differently. It is established by observation rather than by drafting, and the observations are specific. Can the organization produce, for one named individual, every item of personal data it holds about them, within its published response period? When a person withdraws consent for marketing, does the marketing platform stop within a defined time, and is the stop recorded? When the retention period for a data category expires, does anything delete the data, and does anything prove the deletion ran? Each question has a verifiable answer, and the honest answers define the organization’s real position under the law. The document state and the system state coincide only when someone has done the work of connecting them, and that work is the substance of a compliance program rather than its afterthought.

1.2 How the gap forms in well run organizations

It would be a mistake to read the gap between documents and systems as evidence of negligence, because it forms through the ordinary operation of a growing company. Products are built under schedule pressure, and data collection expands feature by feature without a register recording the expansion. Teams adopt tools independently, and each tool receives a copy of customer data as part of doing its job. Engineers create data extracts to answer legitimate business questions, and the extracts remain after the questions are answered. Marketing platforms accumulate lists, support systems accumulate conversations, and analytics pipelines accumulate events, all of it useful and none of it mapped. Several years of this produces an estate that no individual in the organization can describe completely, which is the normal condition of companies that have never had a legal reason to maintain the description.

The DPDP Act supplies that legal reason, and it changes the standard against which the accumulated estate is judged. Before the Act, an unmapped copy of customer data in a forgotten analytics bucket was untidy. After the core obligations commence, the same copy is personal data the organization processes without being able to account for it, invisible to rights requests, excluded from retention schedules, and unexamined in any breach investigation. Nothing about the data changed. The obligations around it changed, and the gap this article examines is precisely the set of places where accumulated practice has not yet been brought under the new standard. Understanding that the gap is normal is the first step. Measuring it is the second, and the remaining sections describe where to look.

2. The Gap in the Data Layer

2.1 Personal data the organization cannot see

The first and largest gap sits in the data layer, and it takes the form of personal data that exists outside the organization’s awareness. The known locations are the production databases behind the main product, and even a modest company holds personal data in many places beyond them. Application logs capture email addresses and identifiers in request records. Analytics events carry user properties into third party platforms. Support tickets contain whatever customers chose to write, which frequently includes phone numbers, addresses, and account details. Marketing tools hold exported lists that were uploaded for a single campaign and never removed. Spreadsheets on shared drives hold extracts made for reporting, reconciliation, or a one time analysis. Message channels hold customer details pasted between colleagues to resolve an issue.

Every one of these locations is inside the scope of the Act, because the law attaches to the data rather than to the systems the organization considers official. A rights request must cover the spreadsheet as fully as the production database. An erasure must reach the analytics platform as fully as the customer table. A breach of the shared drive is a breach of personal data with the same notification duties as a breach of the primary systems. This is why discovery, meaning the systematic identification of where personal data actually resides, is the first act of any genuine readiness program. Discovery is also the step most often skipped, because it is slower than drafting and its output is uncomfortable, and a program that skips it is documenting an estate it has not seen.

2.2 One person, many records

The second gap in the data layer concerns identity. The obligations of the Act are owed to individuals, and the systems of most organizations are not organized around individuals. The same person exists as a customer identifier in the product database, an email address in the marketing platform, a ticket requester in the support system, a device identifier in the analytics pipeline, and a counterparty name in the invoicing tool, and nothing connects these representations. When a rights request arrives from that person, the organization must locate all of them, and the connection work that was never done in advance must be done during the response period, by hand, with no assurance of completeness.

Closing this gap means building what the first article called a purpose aware data map, extended with identity resolution, so that the organization can traverse from one individual to every record held about them across every system. In practice this becomes a maintained inventory that records, for each system, what personal data it holds, under which purposes, keyed by which identifiers, and joined to the other systems through which correspondences. The inventory is unglamorous and it is the load bearing structure of everything else, because access, correction, erasure, withdrawal propagation, and breach scoping all reduce to traversals of it. An organization that can answer the question of where one named person’s data sits, quickly and completely, has closed the data layer gap. An organization that cannot answer it holds obligations it cannot yet perform.

3. The Gap in the Application Layer

3.1 Consent that is recorded but not enforced

The application layer produces a subtler gap, and it appears in organizations that have already done visible consent work. The consent screens exist, the choices are captured, and a record is written, which satisfies every review that stops at the interface. The gap sits behind the interface, in the question of what reads the record. In many implementations the answer is nothing. The marketing platform sends to its full list regardless of recorded preferences. The analytics pipeline processes every event regardless of recorded purposes. The recommendation system trains on all available data regardless of who consented to what. The consent record has become an archive of intentions that the processing systems never consult.

Enforcement of consent is an integration project, and its shape follows from the purposes involved. Each processing system either checks consent state before acting, or receives only data sets that were filtered against consent upstream, and both patterns require the consent record to be authoritative, current, and reachable from the systems that need it. Withdrawal sharpens the requirement, because a withdrawal is a change of state that must alter the behavior of every connected system within a defined time, and the alteration must be observable afterward. An organization can test its own position in an afternoon by withdrawing consent on a test account and watching what actually changes. Where the honest answer is that nothing changes, the application layer gap has been located.

3.2 Rights features that were never built

The rights of data principals require application features, and in most products those features do not yet exist, because nothing before the Act required them. Access requires the ability to assemble and deliver an individual’s data in readable form. Correction requires an interface or a process through which corrections enter the systems and propagate to copies. Erasure requires a deletion capability that operates at the level of one individual across all holdings, which is a different engineering problem from dropping a table or expiring a partition. Grievance redressal requires a case handling workflow with ownership, timelines, and records. Nomination requires the identity model to represent a person authorized to act for another under defined conditions.

Each of these features carries real build effort, and the effort is regularly underestimated because the features sound administrative. Individual level erasure in particular reaches into schema design, foreign key relationships, event streams, and derived data, and retrofitting it into a system that never anticipated deletion is among the larger single work items in a readiness program. The realistic approach is to treat the rights features as a product roadmap in their own right, with the same scoping, estimation, and sequencing any roadmap receives. Programs that treat them this way finish. Programs that treat rights as a paragraph in a policy discover the roadmap later, with less time to execute it.

4. The Gap in the Infrastructure Layer

4.1 Copies that outlive the erasure

Below the applications, the infrastructure layer holds personal data in forms the application teams rarely think about, and these forms open a gap between what erasure means in a policy and what it accomplishes in practice. Database backups contain every record as of the moment of the backup, including records erased afterward. Read replicas and caches hold copies with their own lifecycles. Data warehouse snapshots preserve historical states by design. Object storage buckets hold exports, dumps, and archives accumulated over years. Log retention keeps request histories that embed identifiers. An erasure that removes an individual from the live systems while all of these remain untouched has changed the surface and left the depths as they were.

The first article noted that the defensible pattern combines live system erasure with expiry driven deletion in backup cycles and restoration procedures that reapply erasures. The readiness question for this article is whether that pattern actually exists in the organization’s infrastructure, and the examination is concrete. What is the backup retention period, and is it documented and enforced. Do restoration procedures include a step that replays erasures performed since the backup date, and has the step ever been exercised. Which caches and replicas hold personal data, and what bounds their lifetimes. Which buckets hold historical exports, and when were their contents last reviewed against retention rules. Organizations that can answer these questions have engineered their erasure. Organizations that cannot are promising a capability their infrastructure does not deliver.

4.2 Logging below the standard that breach reporting assumes

The breach reporting obligation examined in the first article assumes an infrastructure that most organizations have not yet built. Reporting within seventy two hours of awareness presumes that awareness arrives quickly, and quick awareness rests on access logs across the systems holding personal data, alerting that surfaces unusual patterns, and an escalation path that produces a declared incident with an owner. The Rules add a specific floor by requiring logs to be retained for at least one year. Against this standard, the common condition of mid sized organizations shows a recognizable set of shortfalls. Logging is enabled on some systems and absent on others. Retention is whatever the default was. Logs exist but nothing watches them. Alerts exist but route to a channel nobody owns.

The infrastructure gap here has a property worth stating plainly, which is that it cannot be closed after it is needed. Every other gap in this article can in principle be remediated late, at higher cost and risk. Logs are different, because a breach investigation can only draw on the records that were being written before the breach, and no remediation performed afterward can recover observation that never happened. This gives logging and monitoring a natural priority in any remediation sequence, ahead of work items that appear more visible. An organization that switches on complete access logging with one year retention early in its program has purchased the evidentiary foundation for every incident that follows, whatever else remains unfinished at the time.

5. The Gap in Third Party Processing

5.1 Vendors holding data without a deletion path

The fourth layer of the gap sits outside the organization entirely, in the vendors and platforms that process personal data on its behalf. The Act holds the data fiduciary answerable for this processing, and requires that erasure extend to processors, which converts every vendor relationship involving personal data into a compliance dependency. The readiness questions follow directly. Which vendors hold personal data, and is the list complete rather than remembered. Does each relationship rest on a contract with data protection obligations, deletion duties, and breach notification duties running back to the fiduciary. When the organization erases an individual, does an instruction reach each vendor holding that individual’s data, and does a confirmation come back. When a vendor suffers a breach, does the organization learn of it within a time that leaves room for its own seventy two hour obligation.

In most organizations the honest answers are uneven, because the vendor estate accumulated the same way the data estate did, tool by tool, under schedule pressure, without a register. The remediation is a vendor inventory joined to the data map, so that every system in the map that belongs to a third party carries its contractual status, its deletion path, and its notification terms alongside it. Renegotiating agreements takes calendar time that engineering cannot compress, since it depends on counterparties, which argues for starting the vendor workstream early in a program rather than treating it as a closing formality. A fiduciary whose own systems are compliant while its processors are ungoverned has closed three layers of the gap and left the fourth open, and the Act does not grade the layers separately.

5.2 Integrations the policy has never heard of

A narrower but recurring version of the vendor gap involves integrations that move personal data without appearing in any document. A developer connects the product to an error tracking service, and stack traces begin carrying user identifiers to an external platform. A growth team installs an attribution tool, and device level data begins flowing to it. A plugin added to the website begins setting identifiers and sending events. None of these integrations passed through procurement, none appears in the processing description, and each of them is processing personal data on the organization’s behalf under the meaning of the law. The privacy notice describes a data flow diagram that stopped being accurate the day after it was drawn.

The remedy is procedural as much as technical. Technically, an organization can enumerate outbound data flows from its applications and its web properties, and the enumeration regularly surprises the teams who commissioned it. Procedurally, the lesson of the first article’s discussion of notices applies here in full, which is that changes to data flows must pass through a checkpoint that updates the map, the vendor register, and the disclosures together. The checkpoint costs little when it is part of the ordinary release process, and its absence is how an organization that completed a careful readiness program in one year drifts quietly out of compliance over the following two.

6. Measuring the Gap: A Working Method

6.1 Tracing obligations end to end

A readiness assessment that measures the real gap does not begin with a questionnaire about policies. It begins by selecting the concrete paths the law will exercise and walking them through the actual systems. The assessment team submits an access request for a real test individual and follows it from intake to delivered response, recording every system touched, every manual step, and every point where the trail goes cold. It withdraws consent on a test account and observes which downstream systems change behavior, over what period, with what record. It runs a tabletop breach exercise against a plausible scenario and measures the distance from first signal to a draft Board report, noting which facts could not be established because the logs to establish them do not exist. It selects an expired retention category and checks whether anything deleted the data.

Each trace produces the same kind of output, which is a specific, located, verifiable finding rather than an impression. The access trace shows that the support platform and two marketing tools were never queried because nobody responsible for the response knew they held personal data. The withdrawal trace shows the record updating while the email platform continues sending. The breach exercise shows that awareness would have taken days because the relevant alerting does not exist, and that the seventy two hour report could not have named the affected individuals because the access logs cover only one system. Findings of this kind cannot be argued with, and they convert the readiness question from an opinion into an inventory. For organizations meeting the thresholds of a Significant Data Fiduciary, the same tracing discipline extends to the additional obligations of that status, including the annual data protection impact assessment, the independent audit, and the appointment of a data protection officer based in India, each of which is either demonstrably in place or is a finding.

6.2 Reading the results into a plan

The raw findings become a program when they are read against three questions. The first is severity, meaning which findings expose the organization to the largest obligations it cannot perform, with anything touching breach detection, notification, and security safeguards weighted by the penalty schedule that reaches 250 crore rupees. The second is effort, meaning which findings are configuration changes, which are integration projects, and which are schema level engineering, since these differ by an order of magnitude in cost and duration. The third is dependency, meaning which findings block others, with the data map blocking nearly everything and logging blocking every future investigation.

Reading the findings this way produces a remediation sequence rather than a list, and the sequence is the real deliverable of an assessment. It states what will be fixed, in what order, by whom, at what cost, and by what date, and it gives leadership an honest position to manage instead of a general sense of exposure. The assessment also produces a second deliverable that is easy to overlook, which is the baseline. Every claim the organization later makes about its progress, to its board or to a regulator, gains credibility when it can be stated against a measured starting point, and the measured starting point can only be captured once, before the remediation begins.

7. Closing the Gap Before May 2027

7.1 What remediation actually consists of

Remediation work sorts into recognizable streams once the assessment has located the gap. The data stream builds and maintains the inventory and the identity resolution across systems, and it runs first because every other stream consumes its output. The application stream implements consent enforcement, withdrawal propagation, and the rights features, treated as a product roadmap with real engineering estimates. The infrastructure stream brings logging, monitoring, retention execution, and backup handling up to the standard the obligations assume, with logging switched on at the earliest possible date for the reasons given in section four. The vendor stream completes the register, closes the contractual gaps, and establishes the deletion and notification paths, started early because its schedule belongs partly to counterparties. The documentation stream runs alongside all of them, capturing each control as it reaches its working state so that the final document set describes a system that exists.

The composition of effort across these streams is worth stating because it corrects a common budgeting error. In a typical mid sized organization, the engineering streams carry most of the cost and most of the calendar time, and the documentation stream carries the least, which is close to the inverse of how compliance budgets are usually imagined. An organization that allocates its program budget primarily to document production has funded the fifth stream and starved the four that determine the outcome. The corrected allocation follows the assessment findings, and it is one more reason the assessment comes first, since a program budgeted before the gap was measured is a program budgeted against a guess.

7.2 Time, sequence, and the honest calendar

The calendar arithmetic for this work is straightforward and worth doing explicitly. A readiness assessment of the kind described in section six occupies four to eight weeks in an organization of moderate complexity. The remediation streams that follow occupy six to nine months when they run with committed ownership, and longer when they compete with feature work for the same engineers. Vendor renegotiations run on their own clock within that period. Ahead of the commencement of core obligations around May 2027, an organization beginning in the second half of 2026 can complete the sequence with modest margin, and each quarter of delay converts directly into compression of the engineering streams, which are the streams that compress worst.

The second half of the calendar argument concerns what the finished state buys. An organization that closes its gap in an orderly way arrives at commencement able to answer rights requests within its published periods, able to demonstrate consent enforcement, able to execute and prove erasure, and able to produce a breach report from logs that were running all along. The same organization holds a document set whose every statement can be verified against a working control, which is the position from which regulatory questions are answered briefly and confidently. That position is available to any organization that measures its gap while there is still time to close it, and the measurement, as this article has tried to show, is a defined piece of work with a defined method rather than a mystery.

Conclusion

The gap between documented compliance and implemented compliance is not a rhetorical device. It is a measurable distance with a known anatomy, and this article has traced that anatomy through four layers. In the data layer it appears as personal data the organization cannot see and identities it cannot connect. In the application layer it appears as consent that nothing enforces and rights features that were never built. In the infrastructure layer it appears as copies that outlive erasure and logging below the standard that breach reporting assumes. In the vendor layer it appears as processors without deletion paths and integrations no document describes. Each form of the gap turns an obligation the organization has accepted on paper into an obligation it cannot perform, and the Act, from around May 2027, will test performance.

The first article in this series established what the obligations of the DPDP Act ask of systems. This second article has examined where systems fall short of the documents written above them, and how the shortfall is measured honestly. The third and final article describes the engagement that closes the distance, covering how a properly structured compliance program moves from assessment through design and implementation to sustained adherence, what each phase produces, and how an organization evaluating outside help can tell substantive work from the appearance of it. The series ends where every real program begins, with the decision to measure before the deadline does the measuring instead.