Deep-dive · Operate 4.3
"Delete user X" looks like one row. In a microservices estate it is a distributed problem: at-least-once delivery, idempotent handlers, and a way to prove every service finished inside the clock. Scroll to watch a deletion propagate.
outbox in the same transaction, marks the user pending-deletionUserDeletionRequested → durable bus"Delete user X" is never one row. It spreads across dozens of services. Doing it synchronously in a single transaction across 40 services is impossible; doing it asynchronously without guarantees is unsafe.
The originating service writes UserDeletionRequested to a local outbox table inside
the same transaction that marks the user pending-deletion. A relay (Debezium, Kafka Connect, a poller)
publishes it. That gives exactly-once production from the source.
Downstream services subscribe and delete locally: row delete, cache invalidation, index removal, lake-row
tombstone, blob delete. Each emits UserDeletionCompleted with its service name and a timestamp.
At-least-once delivery means handlers will see the same event twice. Either check the subject is gone and
treat absence as success, or keep a processed-events table keyed by (event_id, service).
For compacted topics keyed by subject, publish a tombstone (null value) rather than a hard delete.
An orchestrator records which services have ack'd. When all expected services ack within SLA it marks the request complete and notifies the user. When any service lags, it pages on-call. Fire-and-forget pub/sub cannot prove completion within 30 days.
Stripe, SendGrid, Intercom, Segment, your warehouse reverse-ETL: none of them read your Kafka topic. The orchestrator calls their delete APIs and tracks completion the same way it tracks internal services.
All of this must be provable within the 30-day erasure clock; missing it is itself a violation. The classic regression: a service onboarded next quarter backfills old data and resurrects erased subjects. It must replay the erasure queue before it goes live.