DB Olympics

Queries · q307 · Head of Data · week 2, night 3

Late-refund share as a percentage of each merchant's own refund volume

“Is Cragsley Outfitters' outsized share of company-wide late refunds just proportional to how big it is, or does it refund late at a higher rate than other large merchants?”

Manager · asked the questionRosa Van den BergHead of Data. Took over the analytics team last week. Before the team's numbers go to the board she wants to know how far they can be trusted.
Analyst · planned the reportYuki TanakaDetail-oriented: picks specific merchants, products or days and looks at them closely.
Engineer · wrote the SQLViktorOld-school warehouse engineer: explicit JOIN chains straight to the base tables, derived tables, GROUP BY with CASE pivots, few CTEs.

Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told

  1. A business event lands on the Head of Data's desk

    Week 1; shared with Tobias Lindqvist, Chief Financial Officer

    Before the board pack goes out, Data must certify the key metrics. Do revenue, orders, customers, refunds and payouts agree wherever they are recorded, and where they do not, why not and by how much?

  2. Rosa Van den Berg writes to Yuki Tanaka

    The manager, at the start of week 2

    Subject: Good work — a few things I need pinned down before we certify

    Yuki,

    This is exactly the kind of digging I wanted — especially catching that invoices weren't a real fifth number, and that your first "20-40% gap" read was wrong and you went back to check it. That's the instinct I want on this team.

    Three things surprised me and need explaining before I can tell Tobias we're ready:

    1. Even with invoices out of the picture, order revenue, payments captured and payout gross still don't match each other (332,982 vs 323,026 vs 322,182 in your December example). You've explained orders-to-payments very well at the order level — but what explains that same gap when we look at it as one portfolio total for a month? Is it simply orders placed near month-end that haven't been paid yet, or something else?
    2. The refund/payout traceability gap worries me more than you've flagged it. Before we loop in engineering, I want a sense of scale and shape: is this refund-timing effect spread evenly across all merchants, or concentrated in a few — because if it's concentrated, "we can't point to which payout absorbed it" becomes a real merchant-facing payout-support problem, not just a reporting footnote.
    3. On customers — I don't want to just pick two numbers ourselves. I want to understand what each of those two counts is actually used for elsewhere in the business today, so we're not inventing a new definition the rest of the company doesn't already use.

    Questions for this week:

    • For the revenue gap between orders and money actually collected, at the portfolio level — can you tell me how much of it is pure timing (uncollected yet) versus something that never gets collected at all?
    • For the refund/payout gap — is it concentrated in particular merchants or spread thinly across all of them, and does the size of it grow or shrink over the past year?
    • Between the two customer counts, which one is already being used anywhere else in the business (support, billing, marketing), so we're not asking the board to adopt a brand-new definition?
    • On the B2B receivables — rather than spending a slot rebuilding the merchant list this week, can you give me a rough sense of whether wholesalers are large enough a slice of revenue that leaving that line out of Friday's pack is actually a problem?

    Keep the FX-adjusted view for USD/NOK on your list — that one's solid and the board will ask about it.

    Thanks for the pace you're keeping on this.

    Rosa

  3. Yuki Tanaka plans the night

    The analyst · week 2, night 3, request 3 of 3

    Last night's B2B report raised a real data-integrity flag I need to resolve before I can trust the 38.9%-of-revenue materiality number: 6 of the 9 tenants with CRM deal activity (Ballycorran Industrial Supply, Hanse Industriebedarf, Kontor24 Burobedarf, Midland Safety Wear, Papeterie Grossiste, Polder Packaging) show exactly 0.0 across contract value, 2025 revenue, AND open receivables simultaneously, despite deal counts from 6 to 136 - that pattern (three independent measures all exactly zero together) looks like a conversion or join failure, not genuine zero business activity, and I want to check the raw native-currency numbers before concluding anything. Separately, the age-bucket dip and payment-method report together close out Rosa's timing question cleanly: bank_transfer payments average 15.7 days to capture (only 61.1% within 14 days) versus instant capture for card/wallet/credit, which fully explains the mid-range dip as pure timing, not a structural hole - I'm confident enough in that to not spend another slot on it. The segments/cohorts report found no company-wide "active customer" definition already in use (these are per-merchant marketing audiences, not Tidewick's own customer count), but it did validate that a recency-based "ordered in the last N days" concept (the "Lapsed 90 days" segment) is the operational analog marketing already relies on - good enough to write up without further digging. Tonight's three slots: (1) diagnose the zero-value B2B tenant anomaly in native currency with no FX conversion, (2) pull Castell Fasteners' actual open invoices since my €14,481 figure doesn't match Merchant Success's reported €5,080 and the board pack can't show two different numbers for the same merchant, (3) check whether Cragsley Outfitters' outsized share of late refunds is just proportional to its size or a real outlier, by comparing late-refund share against each top merchant's own total refund volume.

    Late-refund share as a percentage of each merchant's own refund volume
    Question
    Is Cragsley Outfitters' outsized share of company-wide late refunds just proportional to how big it is, or does it refund late at a higher rate than other large merchants?
    One row per
    tenant (top merchants by total 2025 refund volume)
    Columns
    • tenant_name
    • total_refunds_2025_eur
    • late_refunds_2025_eur
    • pct_of_own_refunds_late
    • company_avg_pct_late_2025
  4. Viktor writes the SQL

    The engineer · 1 dry run on the empty schema, then the submission

    Dry run 1 · STATUS: COMPILES (dry run on the empty schema, no data read)
    WITH refund_eur AS (
      SELECT pr.tenant_id,
             pr.amount / er.rate AS eur_amount,
             CASE WHEN po.paid_at IS NOT NULL AND pr.processed_at > po.paid_at THEN 1 ELSE 0 END AS is_late
      FROM payment_refunds pr
      JOIN exchange_rates er
        ON er.base_code = 'EUR'
       AND er.quote_code = pr.currency_code
       AND er.as_of = CAST(pr.processed_at AS date)
      LEFT JOIN payout_items pi ON pi.payment_id = pr.payment_id
      LEFT JOIN payouts po ON po.id = pi.payout_id
      WHERE pr.status = 'succeeded'
        AND pr.processed_at >= DATE '2025-01-01'
        AND pr.processed_at < DATE '2026-01-01'
    ),
    tenant_agg AS (
      SELECT tenant_id,
             SUM(eur_amount) AS total_refunds_2025_eur,
             SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_refunds_2025_eur
      FROM refund_eur
      GROUP BY tenant_id
    ),
    company_agg AS (
      SELECT SUM(eur_amount) AS total_eur,
             SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_eur
      FROM refund_eur
    )
    SELECT t.name AS tenant_name,
           ta.total_refunds_2025_eur,
           ta.late_refunds_2025_eur,
           ROUND(ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100, 1) AS pct_of_own_refunds_late,
           ROUND(ca.late_eur / NULLIF(ca.total_eur, 0) * 100, 1) AS company_avg_pct_late_2025
    FROM tenant_agg ta
    JOIN tenants t ON t.id = ta.tenant_id
    CROSS JOIN company_agg ca
    ORDER BY ta.total_refunds_2025_eur DESC, t.name ASC
    LIMIT 15

    “Query compiles cleanly and returns the expected columns. Submitting.”

    Submitted · the SQL of the last dry run, unchanged · Submitted to tonight's batch. Your work on this request is done.

    “Submitted. Summary of what the query does: - Builds a per-refund EUR amount for all succeeded payment_refunds in 2025, converting via the EUR→currency rate on the refund's processed_at date. - Flags a refund as "late" when its payment has a matching payout_items→payouts row and processed_at is after that payout's paid_at; refunds with no payout match are counted in totals but not in the late bucket, per the spec. - Aggregates to tenant level for total_refunds_2025_eur and late_refunds_2025_eur, derives pct_of_own_refunds_late, and separately computes company_avg_pct_late_2025 across all tenants (not just the top 15), repeating it on every row for comparison. - Returns the top 15 tenants by total_refunds_2025_eur, descending, with tenant name as tie-break.”

  5. The report comes back overnight

    As Yuki Tanaka saw it, from the small dataset the agents worked on

    tenant_name            | total_refunds_2025_eur | late_refunds_2025_eur | pct_of_own_refunds_late | company_avg_pct_late_2025
    -----------------------+------------------------+-----------------------+-------------------------+--------------------------
    Cragsley Outfitters    |     169010.91182854303 |    159217.51324604754 |                    94.2 |                      83.7
    Marrowby & Vane        |      62492.02059736718 |     57291.44628112623 |                    91.7 |                      83.7
    OfficeHub Nordic       |     29640.921945628725 |     659.0626346225799 |                     2.2 |                      83.7
    Cinder Peak Sports Co. |     16718.356136629656 |    15670.993847422224 |                    93.7 |                      83.7
    Fell & Fjell           |      5909.871943508494 |      5470.31143406717 |                    92.6 |                      83.7
    Velvet Ash             |      5814.409241830303 |     5483.716447166978 |                    94.3 |                      83.7
    Petalcourt Beauty      |      4571.768079432371 |     3590.343860438063 |                    78.5 |                      83.7
    Bytefield              |      4487.985128653481 |     4487.985128653481 |                   100.0 |                      83.7
    Fennel & Flint         |     3790.5924802646105 |    3505.9284095856133 |                    92.5 |                      83.7
    Oakhurst Tailoring     |     3594.0848337948937 |    2602.7819582550646 |                    72.4 |                      83.7
    Kayak Corner           |      3384.095018658191 |     2707.641419036956 |                    80.0 |                      83.7
    Willow & Wren          |     2413.0786385413694 |    2413.0786385413694 |                   100.0 |                      83.7
    Fjordline Wear         |     2321.0617591726004 |     1881.079397977649 |                    81.0 |                      83.7
    Thistlecombe           |     2304.0121591147863 |    2024.1239057408218 |                    87.9 |                      83.7
    Courtside Collective   |      1731.818487369363 |     1731.818487369363 |                   100.0 |                      83.7
    (15 rows)
  6. What came of it

    The memo and the queries that followed; links found by title, so inferred

    It builds on:

  7. It becomes benchmark query q307

    Changed only where the benchmark needs the same result on every system

    • spellingfetch first
    • determinismtie-breaker: tenant names are not unique
    • numericsdocs/NUMERICS.md: rounding removed; results keep full precision
    • numericsdocs/NUMERICS.md: x / exchange rate cast to NUMERIC(18,6)
    • made explicita selected t.col gets AS col, its standard name made explicit (3×)
    • determinismthe standard leaves the place of NULLs in a sort to the implementation: NULLS LAST ascending, NULLS FIRST descending, as in the reference results (3×)
    • placeholderfetch first
    The benchmark text, against the submitted SQL
    WITH refund_eur AS (  SELECT pr.tenant_id,         pr.amount / er.rate AS eur_amount,  SELECT pr.tenant_id AS tenant_id,         CAST(pr.amount / er.rate AS NUMERIC(18,6)) AS eur_amount,         CASE WHEN po.paid_at IS NOT NULL AND pr.processed_at > po.paid_at THEN 1 ELSE 0 END AS is_late  FROM payment_refunds pr  JOIN exchange_rates er    ON er.base_code = 'EUR'   AND er.quote_code = pr.currency_code   AND er.as_of = CAST(pr.processed_at AS date)  LEFT JOIN payout_items pi ON pi.payment_id = pr.payment_id  LEFT JOIN payouts po ON po.id = pi.payout_id  WHERE pr.status = 'succeeded'    AND pr.processed_at >= DATE '2025-01-01'    AND pr.processed_at < DATE '2026-01-01'),tenant_agg AS (  SELECT tenant_id,         SUM(eur_amount) AS total_refunds_2025_eur,         SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_refunds_2025_eur  FROM refund_eur  GROUP BY tenant_id),company_agg AS (  SELECT SUM(eur_amount) AS total_eur,         SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_eur  FROM refund_eur)SELECT t.name AS tenant_name,       ta.total_refunds_2025_eur,       ta.late_refunds_2025_eur,       ROUND(ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100, 1) AS pct_of_own_refunds_late,       ROUND(ca.late_eur / NULLIF(ca.total_eur, 0) * 100, 1) AS company_avg_pct_late_2025       ta.total_refunds_2025_eur AS total_refunds_2025_eur,       ta.late_refunds_2025_eur AS late_refunds_2025_eur,       ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100 AS pct_of_own_refunds_late,       ca.late_eur / NULLIF(ca.total_eur, 0) * 100 AS company_avg_pct_late_2025FROM tenant_agg taJOIN tenants t ON t.id = ta.tenant_idCROSS JOIN company_agg caORDER BY ta.total_refunds_2025_eur DESC, t.name ASCLIMIT 15ORDER BY ta.total_refunds_2025_eur DESC NULLS FIRST, t.name ASC NULLS LAST, t.id ASC NULLS LAST{{fetch_first(15)}}
    WITH refund_eur AS (
      SELECT pr.tenant_id AS tenant_id,
             CAST(pr.amount / er.rate AS NUMERIC(18,6)) AS eur_amount,
             CASE WHEN po.paid_at IS NOT NULL AND pr.processed_at > po.paid_at THEN 1 ELSE 0 END AS is_late
      FROM payment_refunds pr
      JOIN exchange_rates er
        ON er.base_code = 'EUR'
       AND er.quote_code = pr.currency_code
       AND er.as_of = CAST(pr.processed_at AS date)
      LEFT JOIN payout_items pi ON pi.payment_id = pr.payment_id
      LEFT JOIN payouts po ON po.id = pi.payout_id
      WHERE pr.status = 'succeeded'
        AND pr.processed_at >= DATE '2025-01-01'
        AND pr.processed_at < DATE '2026-01-01'
    ),
    tenant_agg AS (
      SELECT tenant_id,
             SUM(eur_amount) AS total_refunds_2025_eur,
             SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_refunds_2025_eur
      FROM refund_eur
      GROUP BY tenant_id
    ),
    company_agg AS (
      SELECT SUM(eur_amount) AS total_eur,
             SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_eur
      FROM refund_eur
    )
    SELECT t.name AS tenant_name,
           ta.total_refunds_2025_eur AS total_refunds_2025_eur,
           ta.late_refunds_2025_eur AS late_refunds_2025_eur,
           ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100 AS pct_of_own_refunds_late,
           ca.late_eur / NULLIF(ca.total_eur, 0) * 100 AS company_avg_pct_late_2025
    FROM tenant_agg ta
    JOIN tenants t ON t.id = ta.tenant_id
    CROSS JOIN company_agg ca
    ORDER BY ta.total_refunds_2025_eur DESC NULLS FIRST, t.name ASC NULLS LAST, t.id ASC NULLS LAST
    FETCH FIRST 15 ROWS ONLY
    10 ms100 ms1 sHyper67.6 msDuckDB50.7 msGizmoSQL54.2 msclickhouse_brew240 msFirebolterrorDataFusion320 msClickHouseerror
    • Firebolt{ "errors": [ { "description": "Line 3, Column 25: Operations between decimals with different precision and scale is not supported at the moment. Explicitly cast one of the decimals to other decimal precision and scale." } ], "query": { "query_id": "7aec2b6d-72e8-4dda-b0b3-8f8f982c8996", "query_labe
    • ClickHouseCode: 69. DB::Exception: Decimal result's scale is less than argument's one: while executing function divide on arguments __table3.amount Decimal(14, 4) Decimal64(size = 0), __table4.rate Decimal(12, 6) Decimal64(size = 0). (ARGUMENT_OUT_OF_BOUND)
  8. Everything, as recorded

    The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted

    Viktor's whole session, 7 turns
    • The request, as Viktor received it
      Report request from Yuki Tanaka (analyst for the Head of Data):
      
      Title: Late-refund share as a percentage of each merchant's own refund volume
      Question: Is Cragsley Outfitters' outsized share of company-wide late refunds just proportional to how big it is, or does it refund late at a higher rate than other large merchants?
      Grain: tenant (top merchants by total 2025 refund volume)
      Columns:
      - tenant_name
      - total_refunds_2025_eur
      - late_refunds_2025_eur
      - pct_of_own_refunds_late
      - company_avg_pct_late_2025
      Details: Population: payment_refunds with status='succeeded' and processed_at in 2025, grouped by tenant. total_refunds_2025_eur = sum of payment_refunds.amount for the tenant, converted to EUR via exchange_rates (base_code='EUR', quote_code=payment_refunds.currency_code, as_of=processed_at date). late_refunds_2025_eur = the subset of those refunds that are 'late' (processed_at is after the paid_at of the payout that settled the original payment, found via payout_items.payment_id = payment_refunds.payment_id joined to payouts; refunds whose payment has no matching payout_items row do not count as late), converted to EUR the same way. pct_of_own_refunds_late = late_refunds_2025_eur / total_refunds_2025_eur * 100, rounded to 1 decimal. company_avg_pct_late_2025 = the same ratio computed company-wide across ALL tenants (a single repeated value on every row, for easy comparison), rounded to 1 decimal. Tables: payment_refunds, payout_items, payouts, tenants, exchange_rates. Show only the top 15 tenants by total_refunds_2025_eur. Order by total_refunds_2025_eur descending.
      
      Check your query with the warehouse's dry_run tool, then hand it in with submit.
    • Viktor calls dry_run
      WITH refund_eur AS (
        SELECT pr.tenant_id,
               pr.amount / er.rate AS eur_amount,
               CASE WHEN po.paid_at IS NOT NULL AND pr.processed_at > po.paid_at THEN 1 ELSE 0 END AS is_late
        FROM payment_refunds pr
        JOIN exchange_rates er
          ON er.base_code = 'EUR'
         AND er.quote_code = pr.currency_code
         AND er.as_of = CAST(pr.processed_at AS date)
        LEFT JOIN payout_items pi ON pi.payment_id = pr.payment_id
        LEFT JOIN payouts po ON po.id = pi.payout_id
        WHERE pr.status = 'succeeded'
          AND pr.processed_at >= DATE '2025-01-01'
          AND pr.processed_at < DATE '2026-01-01'
      ),
      tenant_agg AS (
        SELECT tenant_id,
               SUM(eur_amount) AS total_refunds_2025_eur,
               SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_refunds_2025_eur
        FROM refund_eur
        GROUP BY tenant_id
      ),
      company_agg AS (
        SELECT SUM(eur_amount) AS total_eur,
               SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_eur
        FROM refund_eur
      )
      SELECT t.name AS tenant_name,
             ta.total_refunds_2025_eur,
             ta.late_refunds_2025_eur,
             ROUND(ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100, 1) AS pct_of_own_refunds_late,
             ROUND(ca.late_eur / NULLIF(ca.total_eur, 0) * 100, 1) AS company_avg_pct_late_2025
      FROM tenant_agg ta
      JOIN tenants t ON t.id = ta.tenant_id
      CROSS JOIN company_agg ca
      ORDER BY ta.total_refunds_2025_eur DESC, t.name ASC
      LIMIT 15
    • The warehouse answers
      STATUS: COMPILES (dry run on the empty schema, no data read)
      COLUMNS: tenant_name, total_refunds_2025_eur, late_refunds_2025_eur, pct_of_own_refunds_late, company_avg_pct_late_2025
    • ViktorQuery compiles cleanly and returns the expected columns. Submitting.
    • Viktor calls submit
      WITH refund_eur AS (
        SELECT pr.tenant_id,
               pr.amount / er.rate AS eur_amount,
               CASE WHEN po.paid_at IS NOT NULL AND pr.processed_at > po.paid_at THEN 1 ELSE 0 END AS is_late
        FROM payment_refunds pr
        JOIN exchange_rates er
          ON er.base_code = 'EUR'
         AND er.quote_code = pr.currency_code
         AND er.as_of = CAST(pr.processed_at AS date)
        LEFT JOIN payout_items pi ON pi.payment_id = pr.payment_id
        LEFT JOIN payouts po ON po.id = pi.payout_id
        WHERE pr.status = 'succeeded'
          AND pr.processed_at >= DATE '2025-01-01'
          AND pr.processed_at < DATE '2026-01-01'
      ),
      tenant_agg AS (
        SELECT tenant_id,
               SUM(eur_amount) AS total_refunds_2025_eur,
               SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_refunds_2025_eur
        FROM refund_eur
        GROUP BY tenant_id
      ),
      company_agg AS (
        SELECT SUM(eur_amount) AS total_eur,
               SUM(CASE WHEN is_late = 1 THEN eur_amount ELSE 0 END) AS late_eur
        FROM refund_eur
      )
      SELECT t.name AS tenant_name,
             ta.total_refunds_2025_eur,
             ta.late_refunds_2025_eur,
             ROUND(ta.late_refunds_2025_eur / NULLIF(ta.total_refunds_2025_eur, 0) * 100, 1) AS pct_of_own_refunds_late,
             ROUND(ca.late_eur / NULLIF(ca.total_eur, 0) * 100, 1) AS company_avg_pct_late_2025
      FROM tenant_agg ta
      JOIN tenants t ON t.id = ta.tenant_id
      CROSS JOIN company_agg ca
      ORDER BY ta.total_refunds_2025_eur DESC, t.name ASC
      LIMIT 15
    • The warehouse answers
      Submitted to tonight's batch. Your work on this request is done.
    • ViktorSubmitted. Summary of what the query does: - Builds a per-refund EUR amount for all `succeeded` `payment_refunds` in 2025, converting via the EUR→currency rate on the refund's `processed_at` date. - Flags a refund as "late" when its payment has a matching `payout_items`→`payouts` row and `processed_at` is after that payout's `paid_at`; refunds with no payout match are counted in totals but not in the late bucket, per the spec. - Aggregates to tenant level for `total_refunds_2025_eur` and `late_refunds_2025_eur`, derives `pct_of_own_refunds_late`, and separately computes `company_avg_pct_late_2025` across all tenants (not just the top 15), repeating it on every row for comparison. - Returns the top 15 tenants by `total_refunds_2025_eur`, descending, with tenant name as tie-break.