DB Olympics

Queries · q334 · Data Protection Officer · week 2, night 4

Is users.deleted_at used anywhere on the platform?

“Is deleted_at a field that's actually populated for any users platform-wide, or is account deactivation simply not wired up at all (context for the erasure finding)?”

Manager · asked the questionInes LaurentData Protection Officer. Lawyer by training, previously at a European marketplace. Also reports to the board's audit committee.
Analyst · planned the reportPriyanka DesaiAuditor's mindset: traces individual records end to end to check that the totals are real.
Engineer · wrote the SQLGraceWrites tidy, layered CTEs with descriptive names, one step per CTE, then a final select.

Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told

  1. A business event lands on the Data Protection Officer's desk

    Week 2; shared with Henrik Aalto, Chief Information Security Officer

    A data protection authority asked how we handle privacy: how many privacy requests we receive and how fast we complete them, how consent is recorded, and what remains after an account is deleted.

  2. Ines Laurent writes to Priyanka Desai

    The manager, at the start of week 2

    Subject: Week 2 — DPA inquiry changes our priorities

    Priyanka,

    Good work — this is exactly the kind of digging I wanted. A few things surprised me and now matter more because of the regulator inquiry (which I'm handling jointly with Henrik Aalto, CISO — our response needs to be accurate and defensible, not just reassuring).

    What struck me:

    • The three "open" erasure requests aren't really open, they're parked behind legal holds — but we don't actually know if those holds are still justified. If a regulator asked us today why those erasures haven't happened, "disputes" that are 1.5–2.6 years old is not a comfortable answer without evidence.
    • We have a closure-tracking gap (rejected requests never get a completion date) and apparently zero portability requests in three years. Both could be true, or both could be an artifact of how we log things — and I don't want to hand the regulator a number I can't stand behind.
    • The email retention failure is concrete and fixable, but it also means I can't yet answer "what remains after an account is deleted" with confidence for other categories until we know how widespread this is.
    • The consent anomaly (one category showing suspiciously perfect opt-in) undermines my confidence in saying "this is how consent is recorded" to a regulator until it's explained.

    So, for this week, please prioritize work that lets us answer the DPA's three questions honestly: volume and speed of privacy requests, how consent is captured and whether it's reliable, and what data persists post-deletion. That's the deliverable Henrik and I need to brief the board audit committee before we respond.

    My questions:

    1. For the three long-running legal holds, can we establish — from whatever evidence exists — whether the underlying disputes are still active, so I can decide whether to push for release or document why they must stand?
    2. Can we get a trustworthy, defensible count of privacy requests received and resolved, including the rejected ones and whether "zero portability requests ever" is real or a logging blind spot — the regulator will ask for exactly this?
    3. How confident can we be in what consent records actually show, given the anomaly you flagged — do we have other categories with the same too-clean pattern, and does that change how I'd describe our consent process to a regulator?
    4. Beyond email, how much shopper data elsewhere is sitting past its stated retention window right now — I need a true current picture of "what remains after deletion," not just the one confirmed violation.

    If all four can't fit, start with 2 and 4 — those go directly into the regulator response.

    Thanks, Ines

    A note from the Chief Information Security Officer, Henrik Aalto

    Subject: Security angle for the DPA response

    Hi Priyanka,

    A few things from my side that would strengthen the privacy response:

    1. When a data subject request results in erasure, can we show what login/session and access history remains afterward, and for how long — i.e. do our retention rules actually clear it out, or does it linger?
    2. For consent withdrawals, do we have any evidence that we stop using/collecting the related data promptly (e.g. no further sign-ins or sensitive-data access tied to that account after withdrawal)?
    3. On the completion-time question the DPA is asking about — can we also show whether requests touching account security data (sessions, API/service credentials, access logs) take longer to close than others? That's often where delays hide.

    Happy to compare notes before you send anything to data engineering.

    Henrik

  3. Priyanka Desai plans the night

    The analyst · week 2, night 4, request 3 of 3

    Last night's post-erasure report surfaced something I need to pin down on this final night: several "remaining" login_attempts/sessions dates are not just old pre-erasure history lingering — they occur well AFTER the erasure's completed_at (e.g. Marrowby & Vane, Cragsley Outfitters, and Studio Vrij subjects all show login attempts dated 2025-02-08 to 2025-02-11, and one Kontor24 Bürobedarf subject shows a session on 2025-12-02, seven months after its 2025-04-29 completion). The Feb 2025 cluster across multiple unrelated erased accounts lines up exactly with Noah Fischer's (CISO) channel post about a 10x login-failure spike in Feb 2025 that wasn't caught by alerts — this could mean that spike included attempts against accounts that should have already been locked down by erasure, which is squarely relevant to both the regulator response and Henrik's security angle. I want hard evidence (count, success/fail, exact timing) rather than eyeballing. I also realized I've never actually checked whether "completed" erasure touches the core users record at all (status/deleted_at) — all my evidence so far is about downstream history tables, not whether the primary account was deactivated, which is the more fundamental test of whether erasure does anything to account access. This is my last night, so I'm spending all 3 slots closing out Henrik's post-erasure footprint question definitively rather than opening new territory; the other items (1, 2, 3, 4, and Henrik's consent/closure-time asks) are already answered from the prior three nights.

    Is users.deleted_at used anywhere on the platform?
    Question
    Is deleted_at a field that's actually populated for any users platform-wide, or is account deactivation simply not wired up at all (context for the erasure finding)?
    One row per
    users.status value, platform-wide, full history
    Columns
    • status
    • total users
    • count with deleted_at not null
    • pct with deleted_at set
    • most recent deleted_at observed
  4. Grace writes the SQL

    The engineer · 1 dry run on the empty schema, then the submission

    Dry run 1 · STATUS: COMPILES (dry run on the empty schema, no data read)
    with user_status_stats as (
        select
            status,
            count(*) as total_users,
            count(deleted_at) as users_with_deleted_at,
            max(deleted_at) as most_recent_deleted_at
        from users
        group by status
    )
    select
        status,
        total_users,
        users_with_deleted_at as count_with_deleted_at_not_null,
        round(100.0 * users_with_deleted_at / nullif(total_users, 0), 2) as pct_with_deleted_at_set,
        most_recent_deleted_at
    from user_status_stats
    order by total_users desc, status asc

    “Query compiles cleanly. Submitting.”

    Submitted · the SQL of the last dry run, unchanged · Submitted to tonight's batch. Your work on this request is done.

    “Submitted. The report groups users by status platform-wide (no tenant/date filters) and shows total users, how many have deleted_at populated, the percentage, and the most recent deleted_at per status — giving Priyanka the baseline to compare against the 39 erasure-subject cases.”

  5. The report comes back overnight

    As Priyanka Desai saw it, from the small dataset the agents worked on

    status      | total_users | count_with_deleted_at_not_null | pct_with_deleted_at_set | most_recent_deleted_at    
    ------------+-------------+--------------------------------+-------------------------+---------------------------
    active      |       11731 |                              0 |                     0.0 | NULL                      
    deactivated |          41 |                             39 |                   95.12 | 2025-11-10 15:18:24.435098
    suspended   |          36 |                              0 |                     0.0 | NULL                      
    (3 rows)
  6. What came of it

    The memo and the queries that followed; links found by title, so inferred

    It builds on:

  7. It becomes benchmark query q334

    Changed only where the benchmark needs the same result on every system

    • numericsdocs/NUMERICS.md: rounding removed; results keep full precision
    • determinismthe standard leaves the place of NULLs in a sort to the implementation: NULLS LAST ascending, NULLS FIRST descending, as in the reference results (2×)
    The benchmark text, against the submitted SQL
    with user_status_stats as (    select        status,        count(*) as total_users,        count(deleted_at) as users_with_deleted_at,        max(deleted_at) as most_recent_deleted_at    from users    group by status)select    status,    total_users,    users_with_deleted_at as count_with_deleted_at_not_null,    round(100.0 * users_with_deleted_at / nullif(total_users, 0), 2) as pct_with_deleted_at_set,    100.0 * users_with_deleted_at / nullif(total_users, 0) as pct_with_deleted_at_set,    most_recent_deleted_atfrom user_status_statsorder by total_users desc, status ascorder by total_users desc NULLS FIRST, status asc NULLS LAST
    with user_status_stats as (
        select
            status,
            count(*) as total_users,
            count(deleted_at) as users_with_deleted_at,
            max(deleted_at) as most_recent_deleted_at
        from users
        group by status
    )
    select
        status,
        total_users,
        users_with_deleted_at as count_with_deleted_at_not_null,
        100.0 * users_with_deleted_at / nullif(total_users, 0) as pct_with_deleted_at_set,
        most_recent_deleted_at
    from user_status_stats
    order by total_users desc NULLS FIRST, status asc NULLS LAST
    1 ms10 ms100 msHyper4.0 msDuckDB14.1 msGizmoSQL11.1 msclickhouse_brew11.5 msFirebolt19.5 msDataFusion31.8 msClickHouse10.1 ms
  8. Everything, as recorded

    The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted

    Grace's whole session, 7 turns
    • The request, as Grace received it
      Report request from Priyanka Desai (analyst for the Data Protection Officer):
      
      Title: Is users.deleted_at used anywhere on the platform?
      Question: Is deleted_at a field that's actually populated for any users platform-wide, or is account deactivation simply not wired up at all (context for the erasure finding)?
      Grain: one row per users.status value, platform-wide, full history
      Columns:
      - status
      - total users
      - count with deleted_at not null
      - pct with deleted_at set
      - most recent deleted_at observed
      Details: Source: users, platform-wide, no tenant filter, no date filter. This gives the baseline: if deleted_at is essentially never populated for anyone regardless of status (not just the 39 erasure subjects), that points to a platform-wide gap in how account deactivation is recorded rather than something specific to the DSAR erasure workflow. Order by total users descending.
      
      Check your query with the warehouse's dry_run tool, then hand it in with submit.
    • Grace calls dry_run
      with user_status_stats as (
          select
              status,
              count(*) as total_users,
              count(deleted_at) as users_with_deleted_at,
              max(deleted_at) as most_recent_deleted_at
          from users
          group by status
      )
      select
          status,
          total_users,
          users_with_deleted_at as count_with_deleted_at_not_null,
          round(100.0 * users_with_deleted_at / nullif(total_users, 0), 2) as pct_with_deleted_at_set,
          most_recent_deleted_at
      from user_status_stats
      order by total_users desc, status asc
    • The warehouse answers
      STATUS: COMPILES (dry run on the empty schema, no data read)
      COLUMNS: status, total_users, count_with_deleted_at_not_null, pct_with_deleted_at_set, most_recent_deleted_at
    • GraceQuery compiles cleanly. Submitting.
    • Grace calls submit
      with user_status_stats as (
          select
              status,
              count(*) as total_users,
              count(deleted_at) as users_with_deleted_at,
              max(deleted_at) as most_recent_deleted_at
          from users
          group by status
      )
      select
          status,
          total_users,
          users_with_deleted_at as count_with_deleted_at_not_null,
          round(100.0 * users_with_deleted_at / nullif(total_users, 0), 2) as pct_with_deleted_at_set,
          most_recent_deleted_at
      from user_status_stats
      order by total_users desc, status asc
    • The warehouse answers
      Submitted to tonight's batch. Your work on this request is done.
    • GraceSubmitted. The report groups `users` by `status` platform-wide (no tenant/date filters) and shows total users, how many have `deleted_at` populated, the percentage, and the most recent `deleted_at` per status — giving Priyanka the baseline to compare against the 39 erasure-subject cases.