DB Olympics

Queries · q303 · Head of Data · week 2, night 2

Existing customer/audience definitions already in use

“Do any marketing segments or analytics cohorts already encode a definition like 'customers who purchased recently' or 'all customers', so we can certify a customer count the business already uses rather than inventing a new one?”

Manager · asked the questionRosa Van den BergHead of Data. Took over the analytics team last week. Before the team's numbers go to the board she wants to know how far they can be trusted.
Analyst · planned the reportYuki TanakaDetail-oriented: picks specific merchants, products or days and looks at them closely.
Engineer · wrote the SQLViktorOld-school warehouse engineer: explicit JOIN chains straight to the base tables, derived tables, GROUP BY with CASE pivots, few CTEs.

Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told

  1. A business event lands on the Head of Data's desk

    Week 1; shared with Tobias Lindqvist, Chief Financial Officer

    Before the board pack goes out, Data must certify the key metrics. Do revenue, orders, customers, refunds and payouts agree wherever they are recorded, and where they do not, why not and by how much?

  2. Rosa Van den Berg writes to Yuki Tanaka

    The manager, at the start of week 2

    Subject: Good work — a few things I need pinned down before we certify

    Yuki,

    This is exactly the kind of digging I wanted — especially catching that invoices weren't a real fifth number, and that your first "20-40% gap" read was wrong and you went back to check it. That's the instinct I want on this team.

    Three things surprised me and need explaining before I can tell Tobias we're ready:

    1. Even with invoices out of the picture, order revenue, payments captured and payout gross still don't match each other (332,982 vs 323,026 vs 322,182 in your December example). You've explained orders-to-payments very well at the order level — but what explains that same gap when we look at it as one portfolio total for a month? Is it simply orders placed near month-end that haven't been paid yet, or something else?
    2. The refund/payout traceability gap worries me more than you've flagged it. Before we loop in engineering, I want a sense of scale and shape: is this refund-timing effect spread evenly across all merchants, or concentrated in a few — because if it's concentrated, "we can't point to which payout absorbed it" becomes a real merchant-facing payout-support problem, not just a reporting footnote.
    3. On customers — I don't want to just pick two numbers ourselves. I want to understand what each of those two counts is actually used for elsewhere in the business today, so we're not inventing a new definition the rest of the company doesn't already use.

    Questions for this week:

    • For the revenue gap between orders and money actually collected, at the portfolio level — can you tell me how much of it is pure timing (uncollected yet) versus something that never gets collected at all?
    • For the refund/payout gap — is it concentrated in particular merchants or spread thinly across all of them, and does the size of it grow or shrink over the past year?
    • Between the two customer counts, which one is already being used anywhere else in the business (support, billing, marketing), so we're not asking the board to adopt a brand-new definition?
    • On the B2B receivables — rather than spending a slot rebuilding the merchant list this week, can you give me a rough sense of whether wholesalers are large enough a slice of revenue that leaving that line out of Friday's pack is actually a problem?

    Keep the FX-adjusted view for USD/NOK on your list — that one's solid and the board will ask about it.

    Thanks for the pace you're keeping on this.

    Rosa

  3. Yuki Tanaka plans the night

    The analyst · week 2, night 2, request 2 of 3

    Three things from last night change my plan. (1) The age-bucket report is reassuring but not a clean monotonic ramp: match rate is 100% at 0-7 days, dips to 83.4% at 8-14 days, climbs through the 90s, then settles around 98.9-100.1% from 91 days onward - so the true permanent "never collected" floor looks like just ~1-2%, but that mid-range dip needs an explanation before I tell Rosa this is purely timing; my best guess is a payment method with inherent capture lag (bank transfer/wire), which I want to test directly. (2) The refund-concentration report confirms Rosa's worry: the top 4 merchants (Cragsley Outfitters, Ampfeld Elektronik, Marrowby & Vane, Studio Vrij) account for 70.8% of all late-processed refunds in 2025, with Cragsley alone at 36.2% and still growing (+36.2% H1-to-H2) - this is genuinely concentrated, not spread thin, and it sits on our single biggest merchant. (3) The industry-revenue report was the big surprise: B2B wholesale (Office Supplies + Industrial Supplies) is 38.9% of 2025 collected revenue across just 3 tenants - far too material to leave out of the board pack, which changes the stakes on getting that receivables number right. But it also only found 3 B2B tenants by industry label, while Daniel Kim's team (last week's channel) referenced 5 B2B wholesale merchants - that gap is worth resolving properly now, by identifying B2B tenants operationally (via actual CRM deal activity) rather than by industry-text label, and recomputing their receivables on the corrected set. I'm also starting on Rosa's customer-definition-usage question tonight by surveying what audience/segment definitions already exist and are actively used elsewhere in the business.

    Existing customer/audience definitions already in use
    Question
    Do any marketing segments or analytics cohorts already encode a definition like 'customers who purchased recently' or 'all customers', so we can certify a customer count the business already uses rather than inventing a new one?
    One row per
    named audience definition (one row per segment or cohort)
    Columns
    • source
    • tenant_name
    • name
    • is_dynamic
    • member_count
    • rules_definition
    • last_updated
  4. Viktor writes the SQL

    The engineer · 1 dry run on the empty schema, then the submission

    Dry run 1 · STATUS: COMPILES (dry run on the empty schema, no data read)
    SELECT
        'segment' AS source,
        t.name AS tenant_name,
        s.name AS name,
        s.is_dynamic AS is_dynamic,
        s.member_count AS member_count,
        substring(cast(s.rules AS varchar), 1, 500) AS rules_definition,
        s.last_refreshed_at AS last_updated
    FROM segments s
    JOIN tenants t ON t.id = s.tenant_id
    UNION ALL
    SELECT
        'cohort' AS source,
        t.name AS tenant_name,
        c.name AS name,
        c.is_dynamic AS is_dynamic,
        c.member_count AS member_count,
        substring(cast(c.definition AS varchar), 1, 500) AS rules_definition,
        c.computed_at AS last_updated
    FROM cohorts c
    JOIN tenants t ON t.id = c.tenant_id
    ORDER BY member_count DESC, tenant_name ASC, name ASC
    LIMIT 50

    “Query compiles cleanly. Submitting.”

    Submitted · the SQL of the last dry run, unchanged · Submitted to tonight's batch. Your work on this request is done.

    “Submitted. The report pulls all segments and cohorts rows, tagged by source, joined to tenants for the name, with the rules/definition JSON cast to text and truncated to 500 characters, ordered by member_count descending and capped at the top 50 — matching the requested grain and columns exactly.”

  5. The report comes back overnight

    As Yuki Tanaka saw it, from the small dataset the agents worked on

    source  | tenant_name            | name                                | is_dynamic | member_count | rules_definition                         | last_updated       
    --------+------------------------+-------------------------------------+------------+--------------+------------------------------------------+--------------------
    segment | Cragsley Outfitters    | All subscribers                     | true       |          849 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Cragsley Outfitters    | Lapsed 90 days                      | true       |          400 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Customers before the 2023 migration | false      |          348 | {"signed_up_before":"2023-01-01"}        | 2023-01-01 04:00:00
    segment | Cragsley Outfitters    | VIP customers                       | true       |          343 | {"all":[{"field":"orders_count","op":"gt | 2025-12-31 02:17:00
    segment | Marrowby & Vane        | All subscribers                     | true       |          316 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Brennquell Kaffee      | All subscribers                     | true       |          179 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    cohort  | Brennquell Kaffee      | Customers before the 2023 migration | false      |          175 | {"signed_up_before":"2023-01-01"}        | 2023-01-01 04:00:00
    segment | Marrowby & Vane        | Lapsed 90 days                      | true       |          155 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    cohort  | Marrowby & Vane        | Customers before the 2023 migration | false      |          154 | {"signed_up_before":"2023-01-01"}        | 2023-01-01 04:00:00
    segment | Ampfeld Elektronik     | All subscribers                     | true       |          136 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Cragsley Outfitters    | New customers (30 days)             | true       |          132 | {"all":[{"field":"first_order_days_ago", | 2025-12-31 02:17:00
    segment | Studio Vrij            | All subscribers                     | true       |          129 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Marrowby & Vane        | VIP customers                       | true       |          126 | {"all":[{"field":"orders_count","op":"gt | 2025-12-31 02:17:00
    segment | Brennquell Kaffee      | VIP customers                       | true       |          111 | {"all":[{"field":"orders_count","op":"gt | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2024-12                   | false      |          109 | {"signed_up_from":"2024-12-01","signed_u | 2025-01-01 04:00:00
    segment | Petalcourt Beauty      | All subscribers                     | true       |          101 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Cinder Peak Sports Co. | All subscribers                     | true       |           98 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2023-11                   | false      |           97 | {"signed_up_from":"2023-11-01","signed_u | 2023-12-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-11                   | false      |           95 | {"signed_up_from":"2025-11-01","signed_u | 2025-12-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2024-11                   | false      |           93 | {"signed_up_from":"2024-11-01","signed_u | 2024-12-01 04:00:00
    segment | Fennel & Flint         | All subscribers                     | true       |           93 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2025-10                   | false      |           83 | {"signed_up_from":"2025-10-01","signed_u | 2025-11-01 04:00:00
    segment | Brennquell Kaffee      | Lapsed 90 days                      | true       |           80 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    cohort  | Studio Vrij            | Customers before the 2023 migration | false      |           77 | {"signed_up_before":"2023-01-01"}        | 2023-01-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2023-12                   | false      |           74 | {"signed_up_from":"2023-12-01","signed_u | 2024-01-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2024-05                   | false      |           72 | {"signed_up_from":"2024-05-01","signed_u | 2024-06-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-05                   | false      |           71 | {"signed_up_from":"2025-05-01","signed_u | 2025-06-01 04:00:00
    cohort  | Ampfeld Elektronik     | Customers before the 2023 migration | false      |           70 | {"signed_up_before":"2023-01-01"}        | 2023-01-01 04:00:00
    segment | Ampfeld Elektronik     | Lapsed 90 days                      | true       |           70 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    segment | Studio Vrij            | Lapsed 90 days                      | true       |           69 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2024-10                   | false      |           68 | {"signed_up_from":"2024-10-01","signed_u | 2024-11-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-07                   | false      |           68 | {"signed_up_from":"2025-07-01","signed_u | 2025-08-01 04:00:00
    segment | Linden & Loom          | All subscribers                     | true       |           67 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2024-04                   | false      |           63 | {"signed_up_from":"2024-04-01","signed_u | 2024-05-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-04                   | false      |           63 | {"signed_up_from":"2025-04-01","signed_u | 2025-05-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-09                   | false      |           61 | {"signed_up_from":"2025-09-01","signed_u | 2025-10-01 04:00:00
    segment | Gratlinie Outdoor      | All subscribers                     | true       |           60 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2024-07                   | false      |           59 | {"signed_up_from":"2024-07-01","signed_u | 2024-08-01 04:00:00
    segment | Wiesgrund Wohnen       | All subscribers                     | true       |           59 | {"all":[{"field":"marketing_opt_in","op" | 2025-12-31 02:17:00
    segment | Marrowby & Vane        | New customers (30 days)             | true       |           58 | {"all":[{"field":"first_order_days_ago", | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2024-03                   | false      |           57 | {"signed_up_from":"2024-03-01","signed_u | 2024-04-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-03                   | false      |           57 | {"signed_up_from":"2025-03-01","signed_u | 2025-04-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-06                   | false      |           56 | {"signed_up_from":"2025-06-01","signed_u | 2025-07-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2025-08                   | false      |           56 | {"signed_up_from":"2025-08-01","signed_u | 2025-09-01 04:00:00
    cohort  | Studio Vrij            | Signed up 2025-05                   | false      |           56 | {"signed_up_from":"2025-05-01","signed_u | 2025-06-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2023-04                   | false      |           55 | {"signed_up_from":"2023-04-01","signed_u | 2023-05-01 04:00:00
    segment | Cinder Peak Sports Co. | Lapsed 90 days                      | true       |           54 | {"all":[{"field":"last_order_days_ago"," | 2025-12-31 02:17:00
    cohort  | Cragsley Outfitters    | Signed up 2023-03                   | false      |           54 | {"signed_up_from":"2023-03-01","signed_u | 2023-04-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2024-06                   | false      |           54 | {"signed_up_from":"2024-06-01","signed_u | 2024-07-01 04:00:00
    cohort  | Cragsley Outfitters    | Signed up 2023-10                   | false      |           53 | {"signed_up_from":"2023-10-01","signed_u | 2023-11-01 04:00:00
    (50 rows)
  6. What came of it

    The memo and the queries that followed; links found by title, so inferred

    It builds on:

  7. It becomes benchmark query q303

    Changed only where the benchmark needs the same result on every system

    • spellingfetch first
    • determinismtie-breaker: no unique key in scope; rows that tie on all output columns are identical
    • determinismthe standard leaves the place of NULLs in a sort to the implementation: NULLS LAST ascending, NULLS FIRST descending, as in the reference results (7×)
    • placeholderfetch first
    The benchmark text, against the submitted SQL
    SELECT    'segment' AS source,    t.name AS tenant_name,    s.name AS name,    s.is_dynamic AS is_dynamic,    s.member_count AS member_count,    substring(cast(s.rules AS varchar), 1, 500) AS rules_definition,    s.last_refreshed_at AS last_updatedFROM segments sJOIN tenants t ON t.id = s.tenant_idUNION ALLSELECT    'cohort' AS source,    t.name AS tenant_name,    c.name AS name,    c.is_dynamic AS is_dynamic,    c.member_count AS member_count,    substring(cast(c.definition AS varchar), 1, 500) AS rules_definition,    c.computed_at AS last_updatedFROM cohorts cJOIN tenants t ON t.id = c.tenant_idORDER BY member_count DESC, tenant_name ASC, name ASCLIMIT 50ORDER BY member_count DESC NULLS FIRST, tenant_name ASC NULLS LAST, name ASC NULLS LAST, source NULLS LAST, is_dynamic NULLS LAST, rules_definition NULLS LAST, last_updated NULLS LAST{{fetch_first(50)}}
    SELECT
        'segment' AS source,
        t.name AS tenant_name,
        s.name AS name,
        s.is_dynamic AS is_dynamic,
        s.member_count AS member_count,
        substring(cast(s.rules AS varchar), 1, 500) AS rules_definition,
        s.last_refreshed_at AS last_updated
    FROM segments s
    JOIN tenants t ON t.id = s.tenant_id
    UNION ALL
    SELECT
        'cohort' AS source,
        t.name AS tenant_name,
        c.name AS name,
        c.is_dynamic AS is_dynamic,
        c.member_count AS member_count,
        substring(cast(c.definition AS varchar), 1, 500) AS rules_definition,
        c.computed_at AS last_updated
    FROM cohorts c
    JOIN tenants t ON t.id = c.tenant_id
    ORDER BY member_count DESC NULLS FIRST, tenant_name ASC NULLS LAST, name ASC NULLS LAST, source NULLS LAST, is_dynamic NULLS LAST, rules_definition NULLS LAST, last_updated NULLS LAST
    FETCH FIRST 50 ROWS ONLY
    1 ms10 ms100 msHyper3.9 msDuckDB3.1 msGizmoSQL6.4 msclickhouse_brew69.4 msFirebolt25.2 msDataFusion16.3 msClickHousewrong result
    • ClickHouserow count 50 vs 531
  8. Everything, as recorded

    The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted

    Viktor's whole session, 7 turns
    • The request, as Viktor received it
      Report request from Yuki Tanaka (analyst for the Head of Data):
      
      Title: Existing customer/audience definitions already in use
      Question: Do any marketing segments or analytics cohorts already encode a definition like 'customers who purchased recently' or 'all customers', so we can certify a customer count the business already uses rather than inventing a new one?
      Grain: named audience definition (one row per segment or cohort)
      Columns:
      - source
      - tenant_name
      - name
      - is_dynamic
      - member_count
      - rules_definition
      - last_updated
      Details: Population: all rows from the 'segments' table (marketing module) UNION ALL all rows from the 'cohorts' table (analytics module), across all tenants. source = literal 'segment' or 'cohort' depending on origin table. tenant_name = joined from tenants. name = segments.name or cohorts.name. is_dynamic = the respective is_dynamic boolean column. member_count = the respective member_count column. rules_definition = the rules (segments) or definition (cohorts) json column, cast to text (truncate to a reasonable length, e.g. 500 characters, if needed for display). last_updated = segments.last_refreshed_at or cohorts.computed_at. Order by member_count descending, limited to the top 50 rows by member_count.
      
      Check your query with the warehouse's dry_run tool, then hand it in with submit.
    • Viktor calls dry_run
      SELECT
          'segment' AS source,
          t.name AS tenant_name,
          s.name AS name,
          s.is_dynamic AS is_dynamic,
          s.member_count AS member_count,
          substring(cast(s.rules AS varchar), 1, 500) AS rules_definition,
          s.last_refreshed_at AS last_updated
      FROM segments s
      JOIN tenants t ON t.id = s.tenant_id
      UNION ALL
      SELECT
          'cohort' AS source,
          t.name AS tenant_name,
          c.name AS name,
          c.is_dynamic AS is_dynamic,
          c.member_count AS member_count,
          substring(cast(c.definition AS varchar), 1, 500) AS rules_definition,
          c.computed_at AS last_updated
      FROM cohorts c
      JOIN tenants t ON t.id = c.tenant_id
      ORDER BY member_count DESC, tenant_name ASC, name ASC
      LIMIT 50
    • The warehouse answers
      STATUS: COMPILES (dry run on the empty schema, no data read)
      COLUMNS: source, tenant_name, name, is_dynamic, member_count, rules_definition, last_updated
    • ViktorQuery compiles cleanly. Submitting.
    • Viktor calls submit
      SELECT
          'segment' AS source,
          t.name AS tenant_name,
          s.name AS name,
          s.is_dynamic AS is_dynamic,
          s.member_count AS member_count,
          substring(cast(s.rules AS varchar), 1, 500) AS rules_definition,
          s.last_refreshed_at AS last_updated
      FROM segments s
      JOIN tenants t ON t.id = s.tenant_id
      UNION ALL
      SELECT
          'cohort' AS source,
          t.name AS tenant_name,
          c.name AS name,
          c.is_dynamic AS is_dynamic,
          c.member_count AS member_count,
          substring(cast(c.definition AS varchar), 1, 500) AS rules_definition,
          c.computed_at AS last_updated
      FROM cohorts c
      JOIN tenants t ON t.id = c.tenant_id
      ORDER BY member_count DESC, tenant_name ASC, name ASC
      LIMIT 50
    • The warehouse answers
      Submitted to tonight's batch. Your work on this request is done.
    • ViktorSubmitted. The report pulls all `segments` and `cohorts` rows, tagged by `source`, joined to `tenants` for the name, with the rules/definition JSON cast to text and truncated to 500 characters, ordered by `member_count` descending and capped at the top 50 — matching the requested grain and columns exactly.