Queries · q325 · Data Protection Officer · week 2, night 1
“Beyond the confirmed email_log failure, how much data in the other four policy-covered tables is currently sitting past its stated retention window?”
Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told
Week 2; shared with Henrik Aalto, Chief Information Security Officer
A data protection authority asked how we handle privacy: how many privacy requests we receive and how fast we complete them, how consent is recorded, and what remains after an account is deleted.
The manager, at the start of week 2
Subject: Week 2 — DPA inquiry changes our priorities
Priyanka,
Good work — this is exactly the kind of digging I wanted. A few things surprised me and now matter more because of the regulator inquiry (which I'm handling jointly with Henrik Aalto, CISO — our response needs to be accurate and defensible, not just reassuring).
What struck me:
So, for this week, please prioritize work that lets us answer the DPA's three questions honestly: volume and speed of privacy requests, how consent is captured and whether it's reliable, and what data persists post-deletion. That's the deliverable Henrik and I need to brief the board audit committee before we respond.
My questions:
If all four can't fit, start with 2 and 4 — those go directly into the regulator response.
Thanks, Ines
Subject: Security angle for the DPA response
Hi Priyanka,
A few things from my side that would strengthen the privacy response:
Happy to compare notes before you send anything to data engineering.
Henrik
The analyst · week 2, night 1, request 3 of 3
Ines said to lead with priorities 2 and 4 if slots are tight, so tonight is entirely those two. For priority 2, I'm not just re-stating last week's 60-request total — Claire Dubois (Risk & Payments) posted ~48 requests company-wide in the same channel, which doesn't match my count, so before anything goes to the regulator I need an auditable type-x-status breakdown that either reconciles the two numbers or pinpoints exactly where they diverge. I'm also treating "zero portability requests ever" as a hypothesis to test rather than a fact: since data_subject_requests.data_export_id links to data_exports, I can check whether export jobs are actually being run for shoppers outside the formal DSAR table — if so, that's a logging blind spot, not a true zero. For priority 4, last week I only checked 2 of the 6 policy-covered tables (email_log confirmed broken, audit_log mostly fine with a narrow gap); tonight I'm closing out the remaining four (events, job_runs, page_views, system_health_checks) at a compact one-row-per-table grain so Ines has the full current picture of "what persists past its stated window," not just the one confirmed violation. I'm deferring the legal-hold evidence question (priority 1), the consent anomaly (priority 3), and Henrik's three asks to night 2, since those need either external evidence (disputes) or more design thought (what "touches security data" means for a request) that I don't want to rush tonight.
The engineer · 1 dry run on the empty schema, then the submission
WITH policy AS ( SELECT entity_table, MIN(retention_days) AS retention_days, MIN(action_on_expiry) AS action_on_expiry FROM retention_policies WHERE entity_table IN ('events','job_runs','page_views','system_health_checks') GROUP BY entity_table ), events_stats AS ( SELECT 'events' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.occurred_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.occurred_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'events'))) AS exceeding_rows FROM events t0 ), job_runs_stats AS ( SELECT 'job_runs' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.queued_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.queued_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'job_runs'))) AS exceeding_rows FROM job_runs t0 ), page_views_stats AS ( SELECT 'page_views' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.viewed_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.viewed_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'page_views'))) AS exceeding_rows FROM page_views t0 ), system_health_checks_stats AS ( SELECT 'system_health_checks' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.checked_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.checked_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'system_health_checks'))) AS exceeding_rows FROM system_health_checks t0 ), all_stats AS ( SELECT * FROM events_stats UNION ALL SELECT * FROM job_runs_stats UNION ALL SELECT * FROM page_views_stats UNION ALL SELECT * FROM system_health_checks_stats ) SELECT a.entity_table AS entity_table, COALESCE(p.retention_days, 0) AS stated_retention_days, COALESCE(p.action_on_expiry, 'unknown') AS action_on_expiry, a.total_rows AS total_rows, a.oldest_age_days AS oldest_row_age_days, a.exceeding_rows AS rows_exceeding_retention, ROUND(100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0), 2) AS pct_exceeding_retention FROM all_stats a LEFT JOIN policy p ON p.entity_table = a.entity_table ORDER BY pct_exceeding_retention DESC, a.entity_table ASC
“Query compiles cleanly and returns the expected one-row-per-table grain with all requested columns. Submitting.”
“Submitted. The report gives one row per policy-covered table (events, job_runs, page_views, system_health_checks) with stated retention days, action on expiry, total rows, oldest row age, count and percentage of rows past the retention window — ordered worst-first by percentage exceeding, exactly as requested.”
As Priyanka Desai saw it, from the small dataset the agents worked on
entity_table | stated_retention_days | action_on_expiry | total_rows | oldest_row_age_days | rows_exceeding_retention | pct_exceeding_retention ---------------------+-----------------------+------------------+------------+---------------------+--------------------------+------------------------ system_health_checks | 35 | delete | 129780 | 39 | 14832 | 11.43 job_runs | 90 | delete | 49247 | 94 | 2281 | 4.63 events | 1095 | delete | 360478 | 1100 | 1324 | 0.37 page_views | 1095 | delete | 396998 | 1100 | 1449 | 0.36 (4 rows)
The memo and the queries that followed; links found by title, so inferred
It builds on:
Changed only where the benchmark needs the same result on every system
WITH policy AS ( SELECT entity_table, MIN(retention_days) AS retention_days, MIN(action_on_expiry) AS action_on_expiry FROM retention_policies WHERE entity_table IN ('events','job_runs','page_views','system_health_checks') GROUP BY entity_table),events_stats AS ( SELECT 'events' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.occurred_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.occurred_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'events'))) AS exceeding_rows FROM events t0),job_runs_stats AS ( SELECT 'job_runs' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.queued_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.queued_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'job_runs'))) AS exceeding_rows FROM job_runs t0),page_views_stats AS ( SELECT 'page_views' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.viewed_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.viewed_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'page_views'))) AS exceeding_rows FROM page_views t0),system_health_checks_stats AS ( SELECT 'system_health_checks' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.checked_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.checked_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'system_health_checks'))) AS exceeding_rows FROM system_health_checks t0),all_stats AS ( SELECT * FROM events_stats UNION ALL SELECT * FROM job_runs_stats UNION ALL SELECT * FROM page_views_stats UNION ALL SELECT * FROM system_health_checks_stats)SELECT a.entity_table AS entity_table, COALESCE(p.retention_days, 0) AS stated_retention_days, COALESCE(p.action_on_expiry, 'unknown') AS action_on_expiry, a.total_rows AS total_rows, a.oldest_age_days AS oldest_row_age_days, a.exceeding_rows AS rows_exceeding_retention, ROUND(100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0), 2) AS pct_exceeding_retention 100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0) AS pct_exceeding_retentionFROM all_stats aLEFT JOIN policy p ON p.entity_table = a.entity_tableORDER BY pct_exceeding_retention DESC, a.entity_table ASCORDER BY pct_exceeding_retention DESC NULLS FIRST, a.entity_table ASC NULLS LAST
WITH policy AS ( SELECT entity_table, MIN(retention_days) AS retention_days, MIN(action_on_expiry) AS action_on_expiry FROM retention_policies WHERE entity_table IN ('events','job_runs','page_views','system_health_checks') GROUP BY entity_table ), events_stats AS ( SELECT 'events' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.occurred_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.occurred_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'events'))) AS exceeding_rows FROM events t0 ), job_runs_stats AS ( SELECT 'job_runs' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.queued_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.queued_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'job_runs'))) AS exceeding_rows FROM job_runs t0 ), page_views_stats AS ( SELECT 'page_views' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.viewed_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.viewed_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'page_views'))) AS exceeding_rows FROM page_views t0 ), system_health_checks_stats AS ( SELECT 'system_health_checks' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.checked_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.checked_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'system_health_checks'))) AS exceeding_rows FROM system_health_checks t0 ), all_stats AS ( SELECT * FROM events_stats UNION ALL SELECT * FROM job_runs_stats UNION ALL SELECT * FROM page_views_stats UNION ALL SELECT * FROM system_health_checks_stats ) SELECT a.entity_table AS entity_table, COALESCE(p.retention_days, 0) AS stated_retention_days, COALESCE(p.action_on_expiry, 'unknown') AS action_on_expiry, a.total_rows AS total_rows, a.oldest_age_days AS oldest_row_age_days, a.exceeding_rows AS rows_exceeding_retention, 100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0) AS pct_exceeding_retention FROM all_stats a LEFT JOIN policy p ON p.entity_table = a.entity_table ORDER BY pct_exceeding_retention DESC NULLS FIRST, a.entity_table ASC NULLS LAST
The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted
Report request from Priyanka Desai (analyst for the Data Protection Officer): Title: Retention reality check beyond email: events, job_runs, page_views, system_health_checks Question: Beyond the confirmed email_log failure, how much data in the other four policy-covered tables is currently sitting past its stated retention window? Grain: one row per entity_table: events, job_runs, page_views, system_health_checks Columns: - entity_table - stated retention_days (from retention_policies, confirmed uniform across tenants last week) - action_on_expiry - total rows currently in the live table (platform-wide) - oldest row age in days (as of 2026-01-05) - rows exceeding stated retention (count) - pct of rows exceeding stated retention Details: Source: retention_policies for the stated retention_days/action_on_expiry per table (one value each, already confirmed uniform across all 120 tenants last week), joined against each live table using its natural timestamp: events.occurred_at, job_runs.queued_at, page_views.viewed_at, system_health_checks.checked_at. Use the literal date 2026-01-05 for all age math (never CURRENT_DATE/now()). This mirrors last week's email_log/audit_log checks but at a compact one-row-per-table summary so all four fit on one page, giving a complete current picture of retention-window violations across every policy-covered table, not just the one confirmed so far. Order by pct of rows exceeding retention, descending (worst first). Check your query with the warehouse's dry_run tool, then hand it in with submit.
dry_runWITH policy AS ( SELECT entity_table, MIN(retention_days) AS retention_days, MIN(action_on_expiry) AS action_on_expiry FROM retention_policies WHERE entity_table IN ('events','job_runs','page_views','system_health_checks') GROUP BY entity_table ), events_stats AS ( SELECT 'events' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.occurred_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.occurred_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'events'))) AS exceeding_rows FROM events t0 ), job_runs_stats AS ( SELECT 'job_runs' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.queued_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.queued_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'job_runs'))) AS exceeding_rows FROM job_runs t0 ), page_views_stats AS ( SELECT 'page_views' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.viewed_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.viewed_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'page_views'))) AS exceeding_rows FROM page_views t0 ), system_health_checks_stats AS ( SELECT 'system_health_checks' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.checked_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.checked_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'system_health_checks'))) AS exceeding_rows FROM system_health_checks t0 ), all_stats AS ( SELECT * FROM events_stats UNION ALL SELECT * FROM job_runs_stats UNION ALL SELECT * FROM page_views_stats UNION ALL SELECT * FROM system_health_checks_stats ) SELECT a.entity_table AS entity_table, COALESCE(p.retention_days, 0) AS stated_retention_days, COALESCE(p.action_on_expiry, 'unknown') AS action_on_expiry, a.total_rows AS total_rows, a.oldest_age_days AS oldest_row_age_days, a.exceeding_rows AS rows_exceeding_retention, ROUND(100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0), 2) AS pct_exceeding_retention FROM all_stats a LEFT JOIN policy p ON p.entity_table = a.entity_table ORDER BY pct_exceeding_retention DESC, a.entity_table ASC
STATUS: COMPILES (dry run on the empty schema, no data read) COLUMNS: entity_table, stated_retention_days, action_on_expiry, total_rows, oldest_row_age_days, rows_exceeding_retention, pct_exceeding_retention
submitWITH policy AS ( SELECT entity_table, MIN(retention_days) AS retention_days, MIN(action_on_expiry) AS action_on_expiry FROM retention_policies WHERE entity_table IN ('events','job_runs','page_views','system_health_checks') GROUP BY entity_table ), events_stats AS ( SELECT 'events' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.occurred_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.occurred_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'events'))) AS exceeding_rows FROM events t0 ), job_runs_stats AS ( SELECT 'job_runs' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.queued_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.queued_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'job_runs'))) AS exceeding_rows FROM job_runs t0 ), page_views_stats AS ( SELECT 'page_views' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.viewed_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.viewed_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'page_views'))) AS exceeding_rows FROM page_views t0 ), system_health_checks_stats AS ( SELECT 'system_health_checks' AS entity_table, COUNT(*) AS total_rows, (DATE '2026-01-05' - CAST(MIN(t0.checked_at) AS DATE)) AS oldest_age_days, COUNT(*) FILTER (WHERE t0.checked_at < (DATE '2026-01-05' - (SELECT retention_days FROM policy WHERE policy.entity_table = 'system_health_checks'))) AS exceeding_rows FROM system_health_checks t0 ), all_stats AS ( SELECT * FROM events_stats UNION ALL SELECT * FROM job_runs_stats UNION ALL SELECT * FROM page_views_stats UNION ALL SELECT * FROM system_health_checks_stats ) SELECT a.entity_table AS entity_table, COALESCE(p.retention_days, 0) AS stated_retention_days, COALESCE(p.action_on_expiry, 'unknown') AS action_on_expiry, a.total_rows AS total_rows, a.oldest_age_days AS oldest_row_age_days, a.exceeding_rows AS rows_exceeding_retention, ROUND(100.0 * a.exceeding_rows / NULLIF(a.total_rows, 0), 2) AS pct_exceeding_retention FROM all_stats a LEFT JOIN policy p ON p.entity_table = a.entity_table ORDER BY pct_exceeding_retention DESC, a.entity_table ASC
Submitted to tonight's batch. Your work on this request is done.