Part 5 · When It Goes Wrong

5 · Respond

When something goes wrong: detecting the breach, running the 72-hour clock, and surviving a regulator investigation. This is the shortest part of the playbook because most of the work (instrumentation, logging, IAM) was already done in Parts 2 to 4. Respond is where that investment pays off, or doesn't. And a "breach" is broader than "hacker stole the database." A misconfigured S3 bucket, an accidental DELETE FROM users without a backup, or a laptop with customer emails left in a taxi all count (Art. 4(12)).

5.1Breach detection

References

GDPR Art. 32 (security of processing: includes "ability to ensure the ongoing confidentiality, integrity, availability and resilience" of systems; the ability to detect is implicit in the obligation to notify within 72 hours of becoming aware per Art. 33).

The ruleYou can't notify on a breach you never saw. Baseline egress and access, scan for public exposure and leaked credentials, and make findings page on-call, not sit on a dashboard.

Why we care

Most enforcement actions cite delayed detection as an aggravating factor. Marriott didn't detect the Starwood intrusion for roughly four years. The ICO's penalty notice called out the absence of effective monitoring on the reservation database as a primary failing (ICO, 2020). If your logs don't surface anomalies, you are not "secure by design" in the regulator's view.

Key patterns

  • Anomalous data egress. Baseline normal byte volumes per service/user/role; alert on multi-sigma deviations. VPC Flow Logs, CloudTrail data events. Flag any single principal exporting >Nx the 30-day p99 from a PII-bearing data store.
  • Access pattern anomalies. UEBA-style detection: "this engineer just ran a full-table scan on users at 02:47 from a new ASN." Database audit logs (RDS Enhanced Monitoring / Performance Insights, native engine audit logs) wired to your SIEM with per-user/per-table baselines.
  • Leaked-credential monitoring. GitHub secret scanning + push protection on all org repos, plus a secondary scanner (TruffleHog, Gitleaks) in CI. Subscribe to native leaked-credential webhooks. AWS auto-quarantines leaked IAM keys, but you still need to rotate and audit usage.
  • Public exposure scanners. Daily scan for public S3 buckets, public RDS snapshots, overly permissive security groups (0.0.0.0/0 on non-web ports). AWS Config rules, plus a third-party (Wiz, Orca, Prowler, ScoutSuite) for broader coverage.
  • SIEM/SOAR basics. Centralise logs (CloudTrail, GuardDuty, audit logs, app logs) into one place: Splunk, Elastic, or a cloud-native data lake (Security Lake). Define detection-as-code (Sigma rules, Panther, Splunk ES) in version control. Wire a SOAR (Tines, Torq, native playbooks) for auto-containment of high-confidence findings: disable IAM key, quarantine instance, snapshot disk.

Anti-patterns / gotchas

  • Treating GuardDuty findings as a dashboard nobody looks at. They must page on-call, or they don't exist.
  • Detection coverage gaps in "boring" data stores: ElasticSearch, Redis, MongoDB, S3 are where most public-internet exposures actually happen, not the production RDBMS.
  • 90-day log retention. Many breaches (Marriott ~4 years; supply-chain compromises: months) span longer than your log window. Retain security-relevant logs ≥13 months in cheap tiered storage (S3 Glacier).
Cloud notes: AWS · K8s
  • AWS: GuardDuty (foundational; EKS + S3 + Malware + RDS protection plans), Security Hub, Macie for S3 PII discovery, CloudTrail data events, Detective, IAM Access Analyzer for external exposure.
  • Kubernetes: Falco for runtime threat detection (syscall-level), Tetragon for eBPF observability, audit policy on the API server, admission controllers (OPA/Gatekeeper, Kyverno) to prevent the misconfigurations that become tomorrow's breach.

5.2Breach notification

  Tool: who do I notify?   Deep-dive: the 72-hour clock
References

GDPR Art. 33 (notification to supervisory authority), Art. 34 (communication to data subjects). Authoritative guidance: EDPB Guidelines 9/2022 on personal data breach notification (updates and partially supersedes WP29 Guidelines wp250rev.01).

The ruleThe 72-hour clock starts at awareness with reasonable certainty, not at the breach. Notify the lead supervisory authority (phased if needed); tell affected individuals too when risk is high.

Why we care

The clock starts when the controller becomes "aware" of a breach with reasonable certainty (not when notification is convenient, not when legal is ready, not when the post-mortem is written). Engineers are usually the people who first see the anomaly. The moment you write "looks like the database was exfiltrated" in Slack is, in practice, when the clock can be argued to have started.

Key patterns

  • "Becoming aware" is stricter than engineers expect. Per EDPB guidance, the controller is aware when it has a "reasonable degree of certainty that a security incident has occurred that has led to personal data being compromised." A short period of investigation to establish that certainty is permitted, but stalling, or treating "we're still investigating" as a way to dodge the clock, is not. The Irish DPC's reasoning in several Meta decisions (EDPB binding decisions) hinges on when the controller should reasonably have been aware.
  • What must be in the notification (Art. 33(3)): (a) nature of the breach, including categories and approximate numbers of data subjects and records affected; (b) DPO contact point; (c) likely consequences; (d) measures taken or proposed to mitigate. If you don't have all the info at hour 72, notify with what you have and supplement in phases. Art. 33(4) explicitly permits phased notification.
  • Communicating to data subjects (Art. 34). Required when the breach is "likely to result in a high risk." Use the ENISA severity methodology (the 2013 ENISA/DPA methodology, still the canonical reference) as a defensible scoring framework. Exemptions: data was effectively encrypted with keys not compromised; the risk has been mitigated subsequently; or notification would involve disproportionate effort (in which case a public communication is acceptable).
  • Processor's duty (Art. 33(2)). A processor must notify the controller "without undue delay" after becoming aware. There is no 72-hour grace period for processors; the clock belongs to the controller and starts from their awareness. In practice, your DPA should specify a tight processor notification SLA (commonly 24-48 hours, sometimes shorter). If you operate a SaaS, build a breach notification workflow that emits structured notifications to controller customers automatically (signed email + portal entry).
  • Drill the breach response runbook at least annually. Tabletop with engineering, security, legal, DPO, comms. Simulate "GuardDuty fired at 23:00 on a Friday on the customer-PII account" and time the path from alert to draft notification. If you can't draft an Art. 33 notification within 24 hours of detection, you cannot reliably meet 72 hours when reality is messier.

Anti-patterns / gotchas

  • The "72 hours from the breach" myth. It is 72 hours from awareness. Conversely: do not assume awareness only starts when you've fully scoped the incident. Wilful blindness ("we didn't look at the alert") doesn't reset the clock.
  • Notifying every incident reflexively, or never. Art. 33 has a risk threshold: "unless the breach is unlikely to result in a risk to the rights and freedoms of natural persons." Document that risk assessment for every incident. Regulators will ask why you decided not to notify.
  • Forgetting one-stop-shop. For cross-border processing, notify your lead supervisory authority (the SA of your main establishment), not all 27. Get this wrong and you may face parallel investigations.
Cloud notes: AWS · K8s
  • AWS publishes its customer breach notification commitments in its DPA (AWS GDPR DPA). Read it. It defines what to expect when your processor (the hyperscaler) has a breach.
  • For your own SaaS, automate the path: an internal "declare incident" command in your ChatOps that creates an immutable incident record, starts evidence preservation (snapshots, log exports to a write-once bucket), and pages the DPO. The timestamp on that record is your defensible "awareness" timestamp.
  • Reference cases on what gets fined when detection/notification fails:
    • British Airways (ICO, 2020): £20M (reduced from £183M proposed). Magecart-style skimming via a compromised Modernizr script on the payment page; the breach went undetected for over two months. Engineering lessons: file integrity monitoring on production JS, strict CSP with script-src allow-listing, SRI on third-party scripts. (ICO action page).
    • Marriott (ICO, 2020): £18.4M. Inherited Starwood breach undetected from 2014 to 2018. Lessons: M&A due diligence must include security posture and historical compromise hunting; deploy detection across acquired estates before integration.
    • Meta / Facebook (DPC Ireland, 2022): €265M for the data-scraping incident affecting ~533M users. The fine was for inadequate technical and organisational measures (Art. 25, Art. 32), not non-notification, but the engineering lesson is the same: rate limits and bot defences on contact-discovery and similar enumerable APIs. (DPC announcement).

5.3Working with the DPO and regulators

References

GDPR Art. 37 (designation of the DPO), Art. 38 (position), Art. 39 (tasks). Supervisory authority powers: Art. 58. Aggravating/mitigating factors for fines: Art. 83(2).

The ruleSnapshot evidence before you remediate, page the DPO before any external word, and cooperate. The artefacts regulators want are the logs, diagrams, and DPIAs you kept years ago.

Why we care

Once a regulator opens an investigation, what they ask for is overwhelmingly engineering artefacts: log extracts, access control snapshots, system diagrams, change-management records, DPIA documents, dates of awareness. If those artefacts don't exist or aren't preserved with integrity, the investigation goes badly regardless of how good your intentions were.

Key patterns

  • Escalation triggers - when an engineer must page the DPO immediately: confirmed or strongly suspected unauthorised access to a PII store; confirmed data exfiltration; public exposure of a resource containing PII (regardless of duration); ransomware or wiper malware on systems processing personal data; loss/theft of devices with unencrypted PII; mis-sent bulk communication exposing recipients to each other (the "BCC vs CC" classic). The runbook should make paging the DPO a single command, not a debate.
  • Evidence preservation from minute one. Snapshot affected EBS volumes before any remediation. Copy CloudTrail log slices for the relevant window to a write-once bucket (S3 Object Lock in Compliance mode). Capture in-memory state if feasible (EC2 SSM Run Command + LiME, or GRR / commercial DFIR). Do this before cleanup.
  • Chain of custody basics. For each artefact: who collected it, when (UTC), from which system, with which command, and the SHA-256 hash. A simple evidence/<incident-id>/manifest.csv in an immutable bucket is sufficient for most cases. If a regulator (or court) later asks "how do we know this log wasn't edited?", you need an answer.
  • Runbook patterns for the first 24 hours. Pre-written: declare-incident command, containment playbooks per asset class (compromised IAM key, compromised EC2 instance, exposed S3 bucket, compromised K8s pod), evidence-collection playbook, DPO/legal notification template, Art. 33 draft template. Store them in the same repo as your IaC: they are infrastructure.
  • How regulator investigations typically unfold. (1) Initial notification under Art. 33. (2) Follow-up questionnaire, usually weeks later, dozens of detailed questions covering technical and organisational measures dating back years. (3) Possible on-site or document inspection. (4) Statement of preliminary findings, right to respond. (5) Decision. Timelines are months to years. Engineers will be pulled in repeatedly to produce evidence. Log retention and DPIA hygiene done years ago is what saves you.

Anti-patterns / gotchas

  • Rebuilding the affected system before snapshotting it. Once you terraform apply over compromised infrastructure, the forensic trail is gone. Containment ≠ destruction.
  • The DPO finds out from Twitter. If a customer or journalist tells the DPO first, you have lost trust internally and the response will be reactive. Hard rule: the DPO is paged before any external communication, including status pages.
  • Treating the regulator as adversarial from minute one. Regulators distinguish between organisations that cooperate, self-report promptly, and demonstrate learning versus those that minimise, delay, and resist. Cooperation is an explicit mitigating factor under Art. 83(2).
Cloud notes: AWS · K8s
  • AWS: EBS snapshots (encrypted, copied to a forensic account), S3 Object Lock for log preservation, SSM Run Command for evidence collection without SSH, Detective and Security Lake for cross-account investigation. The AWS Security Incident Response Guide is the canonical reference.
  • Kubernetes: Volume snapshots via CSI, pod-level evidence capture (kubectl cp, ephemeral debug containers), audit log export, Falco event archive. Treat compromised pods as evidence, not garbage: cordon and snapshot before deleting.

Closing thoughts

Three takeaways:

  • The 72-hour clock starts at awareness with reasonable certainty, not at the breach and not at the conclusion of investigation. The moment an engineer reasonably concludes "personal data was likely compromised," the clock is running. Design runbooks and your awareness-timestamp practice accordingly.
  • Detection is half the regulator's question. Marriott, BA, and most headline GDPR breach fines turn on detection failures or response delays, not on the existence of the breach itself.
  • Evidence preserved is evidence available. Snapshot first, remediate second. Immutable storage for security logs is cheap; reconstructing destroyed evidence under regulator scrutiny is not.

At a glance

72h  notification window

Art. 33/34  notify the SA & data subjects

~4 yrs  Marriott breach undetected

The series

Part 5 of 5

Detect the breach, run the 72-hour clock, and survive the investigation: the payoff for everything built in Parts 1-4.

Prev

← Part 4 · Operate

Series

All concepts →