DB Olympics

Queries · q087 · VP Product · week 1, night 4

Repeat purchase after a return vs. no return, 2025

“Does experiencing a return change whether, and how soon, a customer places their next order, compared to orders that had no return?”

Manager · asked the questionJonas FeldVP Product. Joined from a mobile-first retailer. Deciding the 2026 roadmap.
Analyst · planned the reportEmeka ObiCustomer-centric: follows cohorts of shoppers or merchants over time.
Engineer · wrote the SQLGraceWrites tidy, layered CTEs with descriptive names, one step per CTE, then a final select.

Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told

  1. A business event lands on the VP Product's desk

    Week 1

    We are redesigning the shopper journey for 2026. We need to understand the whole path, from the first visit to checkout, delivery, returns and the next purchase, and where shoppers drop out and why.

  2. Jonas Feld writes to Emeka Obi

    The manager, at the start of week 1

    Subject: Shopper journey end-to-end — what I need for the 2026 roadmap

    Emeka,

    We're resetting the shopper journey for 2026, and I need a clear picture of the whole path — first visit through checkout, delivery, returns, and whether people come back — so we can decide where to invest design and engineering time first. This feeds the roadmap review I'm presenting in two weeks, so the Friday memo should leave me able to say "here's where shoppers are leaking out, and here's the one or two places a fix would move the most volume."

    I know you only get a handful of report slots this week, so please prioritize in the order below — stop earlier in the list if something forces a tradeoff, rather than spreading thin across all of it.

    1. Where do we lose people between landing on the storefront and placing an order? I want to see the shape of the funnel (visit → browse/search → cart → checkout → paid order) and which step is the biggest drop-off, ideally for a few of our bigger merchants so we're not averaging away real differences.
    2. Once someone reaches checkout, what's stopping them from finishing? Is it a one-step cliff or a slow bleed across the checkout flow, and does it differ by device or by how they arrived (e.g. ad vs. direct vs. returning)?
    3. After the order is placed, how is delivery experience affecting trust? I want to understand how often deliveries run late and whether that visibly connects to returns or to people not coming back.
    4. How painful are returns, and does a return kill the relationship? I'd like a sense of how often a return leads to the shopper buying again within a reasonable window versus going quiet.
    5. Who comes back, and how soon? A simple read on repeat purchase behavior — how many first-time buyers return, and roughly how long it takes — would help me know if we're building a roadmap for acquisition or for retention.

    If you can only land three of these this week, do 1, 2 and 5 — funnel, checkout, and repeat purchase are the ones I most need going into the roadmap conversation. Delivery and returns can follow next week if we run out of nights.

    Thanks — looking forward to Friday.

    Jonas

  3. Emeka Obi plans the night

    The analyst · week 1, night 4, request 3 of 3

    Night 3 lands priority 2 solidly and complicates priority 3 in an interesting way.

    Report 1 (corrected status funnel, retail-only): confirms the clean read — pending→paid loses only 2.4% of orders, essentially instantly (median 0 hours), and paid→fulfilled is 94.9% at a sane ~23-hour median. So payment capture itself is not where retail checkout leaks; it leaks earlier, at cart→order (which Report 1 night 1 already showed at 27-61% conversion for these tenants). Refunds trickle in ~3 weeks out (517h), cancellations are near-instant (0.1h). Priority 2 is now fully answered: it's a slow bleed concentrated in the cart stage, not a cliff at payment.

    Report 2 (cart timing by tenant): the slow-bleed shape is remarkably consistent across all 6 retail tenants — <1hr bucket is ~23-32% of carts and ~100% converted (fast successful checkouts, not failures), while the 1-24hr and 1-7day buckets (roughly two-thirds of all carts, combined) convert at only 1-19%. This is a universal pattern, not one tenant's expiry setting — strong, generalizable finding for the roadmap.

    Report 3 (delivery speed vs. returns) is the interesting twist: at the merchant level, the correlation runs backwards from the "late delivery → more returns" hypothesis. The three apparel merchants (Marrowby & Vane, Studio Vrij, Cragsley Outfitters) have the highest return rates (31-34%) but some of the lowest late-delivery rates (0.3-12%), while food/beauty/pet merchants have high late-delivery rates (20-23%) but near-zero returns (0.5-4.3%). This looks like vertical (apparel fit/size returns are structurally high regardless of delivery speed) is swamping any delivery-speed effect in a merchant-level comparison. Before I tell Jonas "no connection," I want to re-test within a single vertical to remove that confound, and also test the "people not coming back" half of his question directly (repeat purchase after late delivery), which I haven't touched yet.

    This is the last night before the Friday memo. Priorities 1, 2, 5 are in good shape. I'm spending all 3 slots to get a defensible first read on the two things Jonas said could slip to next week if time ran short (delivery trust, and whether a return kills the relationship) — we have the nights, so I'll use them rather than leave those blank in the memo.

    Repeat purchase after a return vs. no return, 2025
    Question
    Does experiencing a return change whether, and how soon, a customer places their next order, compared to orders that had no return?
    One row per
    group: orders that had a return vs orders that had no return
    Columns
    • group (has_return / no_return)
    • orders count
    • distinct customers
    • pct with a subsequent order within 90 days
    • median days to that subsequent order
  4. Grace writes the SQL

    The engineer · 1 dry run on the empty schema, then the submission

    Dry run 1 · STATUS: COMPILES (dry run on the empty schema, no data read)
    WITH retail_tenants AS (
        SELECT id, name
        FROM tenants
        WHERE name IN (
            'Kibblewick', 'Studio Vrij', 'Petalcourt Beauty',
            'Marrowby & Vane', 'Cragsley Outfitters', 'Ampfeld Elektronik'
        )
    ),
    all_orders AS (
        SELECT o.id, o.tenant_id, o.user_id, o.placed_at
        FROM orders o
        JOIN retail_tenants rt ON rt.id = o.tenant_id
        WHERE o.placed_at IS NOT NULL
    ),
    base_orders AS (
        SELECT *
        FROM all_orders
        WHERE placed_at >= TIMESTAMP '2025-01-01'
          AND placed_at < TIMESTAMP '2025-10-01'
    ),
    order_has_return AS (
        SELECT DISTINCT order_id
        FROM order_returns
    ),
    next_order_per_base_order AS (
        SELECT b.id AS order_id, MIN(a.placed_at) AS next_placed_at
        FROM base_orders b
        JOIN all_orders a
          ON a.tenant_id = b.tenant_id
         AND a.user_id = b.user_id
         AND a.placed_at > b.placed_at
        GROUP BY b.id
    ),
    order_level AS (
        SELECT
            b.id AS order_id,
            b.tenant_id,
            b.user_id,
            CASE WHEN ohr.order_id IS NOT NULL THEN 'has_return' ELSE 'no_return' END AS grp,
            CASE WHEN nop.next_placed_at IS NOT NULL
                 THEN EXTRACT(EPOCH FROM (nop.next_placed_at - b.placed_at)) / 86400.0
            END AS days_to_next
        FROM base_orders b
        LEFT JOIN order_has_return ohr ON ohr.order_id = b.id
        LEFT JOIN next_order_per_base_order nop ON nop.order_id = b.id
    )
    SELECT
        grp AS "group",
        COUNT(*) AS orders_count,
        COUNT(DISTINCT (tenant_id, user_id)) AS distinct_customers,
        ROUND(
            100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*),
            1
        ) AS pct_with_subsequent_order_within_90_days,
        ROUND(
            CAST(
                PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days_to_next)
                FILTER (WHERE days_to_next IS NOT NULL AND days_to_next <= 90)
            AS numeric),
            0
        ) AS median_days_to_subsequent_order
    FROM order_level
    GROUP BY grp
    ORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 END

    “Query compiles cleanly and returns the expected columns. Submitting.”

    Submitted · the SQL of the last dry run, unchanged · Submitted to tonight's batch. Your work on this request is done.

    “Submitted. Summary of the approach: - Scope: Orders from the 6 named retail tenants with placed_at in 2025‑01‑01 through 2025‑09‑30. - Grouping: has_return if the order has any row in order_returns, else no_return. - Subsequent order: for each base order, found the nearest later order (strictly after, same tenant+user) from the full orders history (not limited to the base window, so it can reach into Q4 2025). - Metrics: orders count, distinct (tenant, user) customers, % with that next order ≤90 days out (1 decimal), and the median days-to-next-order among those within 90 days (rounded to whole days), using PERCENTILE_CONT. - Ordering: has_return row first, then no_return, as requested. Noted a caveat in the submit note: "subsequent order" means the nearest next order by that customer (not any order within 90 days), and return rows aren't filtered by status/outcome—any return record counts.”

  5. The report comes back overnight

    As Emeka Obi saw it, from the small dataset the agents worked on

    group      | orders_count | distinct_customers | pct_with_subsequent_order_within_90_days | median_days_to_subsequent_order
    -----------+--------------+--------------------+------------------------------------------+--------------------------------
    has_return |         1441 |               1034 |                                     49.7 |                              33
    no_return  |         3413 |               1686 |                                     57.7 |                              28
    (2 rows)
  6. It becomes benchmark query q087

    Changed only where the benchmark needs the same result on every system

    • numericsdocs/NUMERICS.md: rounding removed; results keep full precision; bare NUMERIC cast that only served the rounding removed
    • numericsdocs/NUMERICS.md: EXTRACT(EPOCH FROM (b - a)) cast to NUMERIC(18,6)
    • made explicita selected t.col gets AS col, its standard name made explicit (6×)
    • placeholderpercentile cont
    • placeholderseconds between
    • placeholdercount distinct pair
    • determinismthe standard leaves the place of NULLs in a sort to the implementation: NULLS LAST ascending, NULLS FIRST descending, as in the reference results
    The benchmark text, against the submitted SQL
    WITH retail_tenants AS (    SELECT id, name    FROM tenants    WHERE name IN (        'Kibblewick', 'Studio Vrij', 'Petalcourt Beauty',        'Marrowby & Vane', 'Cragsley Outfitters', 'Ampfeld Elektronik'    )),all_orders AS (    SELECT o.id, o.tenant_id, o.user_id, o.placed_at    SELECT o.id AS id, o.tenant_id AS tenant_id, o.user_id AS user_id, o.placed_at AS placed_at    FROM orders o    JOIN retail_tenants rt ON rt.id = o.tenant_id    WHERE o.placed_at IS NOT NULL),base_orders AS (    SELECT *    FROM all_orders    WHERE placed_at >= TIMESTAMP '2025-01-01'      AND placed_at < TIMESTAMP '2025-10-01'),order_has_return AS (    SELECT DISTINCT order_id    FROM order_returns),next_order_per_base_order AS (    SELECT b.id AS order_id, MIN(a.placed_at) AS next_placed_at    FROM base_orders b    JOIN all_orders a      ON a.tenant_id = b.tenant_id     AND a.user_id = b.user_id     AND a.placed_at > b.placed_at    GROUP BY b.id),order_level AS (    SELECT        b.id AS order_id,        b.tenant_id,        b.user_id,        b.tenant_id AS tenant_id,        b.user_id AS user_id,        CASE WHEN ohr.order_id IS NOT NULL THEN 'has_return' ELSE 'no_return' END AS grp,        CASE WHEN nop.next_placed_at IS NOT NULL             THEN EXTRACT(EPOCH FROM (nop.next_placed_at - b.placed_at)) / 86400.0             THEN {{seconds_between(b.placed_at, nop.next_placed_at)}} / 86400.0        END AS days_to_next    FROM base_orders b    LEFT JOIN order_has_return ohr ON ohr.order_id = b.id    LEFT JOIN next_order_per_base_order nop ON nop.order_id = b.id)SELECT    grp AS "group",    COUNT(*) AS orders_count,    COUNT(DISTINCT (tenant_id, user_id)) AS distinct_customers,    ROUND(        100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*),        1    ) AS pct_with_subsequent_order_within_90_days,    ROUND(        CAST(            PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days_to_next)            FILTER (WHERE days_to_next IS NOT NULL AND days_to_next <= 90)        AS numeric),        0    ) AS median_days_to_subsequent_order    {{count_distinct_pair(tenant_id, user_id)}} AS distinct_customers,    100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*) AS pct_with_subsequent_order_within_90_days,    {{percentile_cont_filter(0.5, days_to_next, days_to_next IS NOT NULL AND days_to_next <= 90)}} AS median_days_to_subsequent_orderFROM order_levelGROUP BY grpORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 ENDORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 END NULLS LAST
    WITH retail_tenants AS (
        SELECT id, name
        FROM tenants
        WHERE name IN (
            'Kibblewick', 'Studio Vrij', 'Petalcourt Beauty',
            'Marrowby & Vane', 'Cragsley Outfitters', 'Ampfeld Elektronik'
        )
    ),
    all_orders AS (
        SELECT o.id AS id, o.tenant_id AS tenant_id, o.user_id AS user_id, o.placed_at AS placed_at
        FROM orders o
        JOIN retail_tenants rt ON rt.id = o.tenant_id
        WHERE o.placed_at IS NOT NULL
    ),
    base_orders AS (
        SELECT *
        FROM all_orders
        WHERE placed_at >= TIMESTAMP '2025-01-01'
          AND placed_at < TIMESTAMP '2025-10-01'
    ),
    order_has_return AS (
        SELECT DISTINCT order_id
        FROM order_returns
    ),
    next_order_per_base_order AS (
        SELECT b.id AS order_id, MIN(a.placed_at) AS next_placed_at
        FROM base_orders b
        JOIN all_orders a
          ON a.tenant_id = b.tenant_id
         AND a.user_id = b.user_id
         AND a.placed_at > b.placed_at
        GROUP BY b.id
    ),
    order_level AS (
        SELECT
            b.id AS order_id,
            b.tenant_id AS tenant_id,
            b.user_id AS user_id,
            CASE WHEN ohr.order_id IS NOT NULL THEN 'has_return' ELSE 'no_return' END AS grp,
            CASE WHEN nop.next_placed_at IS NOT NULL
                 THEN CAST(EXTRACT(EPOCH FROM (nop.next_placed_at - b.placed_at)) AS NUMERIC(18,6)) / 86400.0
            END AS days_to_next
        FROM base_orders b
        LEFT JOIN order_has_return ohr ON ohr.order_id = b.id
        LEFT JOIN next_order_per_base_order nop ON nop.order_id = b.id
    )
    SELECT
        grp AS "group",
        COUNT(*) AS orders_count,
        COUNT(DISTINCT (tenant_id, user_id)) AS distinct_customers,
        100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*) AS pct_with_subsequent_order_within_90_days,
        PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days_to_next) FILTER (WHERE days_to_next IS NOT NULL AND days_to_next <= 90) AS median_days_to_subsequent_order
    FROM order_level
    GROUP BY grp
    ORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 END NULLS LAST
    100 ms1 sHypererrorDuckDB348 msGizmoSQL298 msclickhouse_brew798 msFirebolterrorDataFusionerrorClickHouse893 ms
    • HyperROW types are disabled
    • Firebolt{ "errors": [ { "description": "Line 50, Column 20: Records are only supported in queries of the following form: select ... <record> [NOT] IN (select ...)" } ], "query": { "query_id": "33a50b09-1ddb-4b47-a8e4-2b4a3e845158", "query_label": null, "request_id": "6ab50b17-07a7-43a5-8cd5-e4b0ee116d44" },
    • DataFusionResources exhausted: Failed to allocate additional 17.2 TB for FinalHashAggregateStream[49] with 0.0 B already allocated for this reservation - 30.7 GB remain available for the total memory pool: fair(pool_size: 153.6 GB)
  7. Everything, as recorded

    The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted

    Grace's whole session, 7 turns
    • The request, as Grace received it
      Report request from Emeka Obi (analyst for the VP Product):
      
      Title: Repeat purchase after a return vs. no return, 2025
      Question: Does experiencing a return change whether, and how soon, a customer places their next order, compared to orders that had no return?
      Grain: one row per group: orders that had a return vs orders that had no return
      Columns:
      - group (has_return / no_return)
      - orders count
      - distinct customers
      - pct with a subsequent order within 90 days
      - median days to that subsequent order
      Details: Restrict to the 6 retail tenants (Kibblewick, Studio Vrij, Petalcourt Beauty, Marrowby & Vane, Cragsley Outfitters, Ampfeld Elektronik) since returns are only meaningful volume there (the food/coffee merchants showed near-zero return rates in the earlier delivery report and would just dilute this). Base population: orders with placed_at in 2025-01-01 to 2025-09-30 (leaves a full 90-day forward window against data through 2025-12-31). group = 'has_return' if the order has at least one row in order_returns, else 'no_return'. For each group: orders_count = count of orders; distinct_customers = distinct (tenant_id, user_id) among those orders. For each order, check whether the same user placed another order with placed_at strictly after this order's placed_at and within 90 days after it; pct_with_subsequent_order_within_90_days = % of orders in the group meeting that condition, rounded 1 decimal; median_days_to_subsequent_order = median of (that next order's placed_at minus this order's placed_at) in days, among orders that had one, rounded to a whole number. Order rows with 'has_return' first, then 'no_return'.
      
      Check your query with the warehouse's dry_run tool, then hand it in with submit.
    • Grace calls dry_run
      WITH retail_tenants AS (
          SELECT id, name
          FROM tenants
          WHERE name IN (
              'Kibblewick', 'Studio Vrij', 'Petalcourt Beauty',
              'Marrowby & Vane', 'Cragsley Outfitters', 'Ampfeld Elektronik'
          )
      ),
      all_orders AS (
          SELECT o.id, o.tenant_id, o.user_id, o.placed_at
          FROM orders o
          JOIN retail_tenants rt ON rt.id = o.tenant_id
          WHERE o.placed_at IS NOT NULL
      ),
      base_orders AS (
          SELECT *
          FROM all_orders
          WHERE placed_at >= TIMESTAMP '2025-01-01'
            AND placed_at < TIMESTAMP '2025-10-01'
      ),
      order_has_return AS (
          SELECT DISTINCT order_id
          FROM order_returns
      ),
      next_order_per_base_order AS (
          SELECT b.id AS order_id, MIN(a.placed_at) AS next_placed_at
          FROM base_orders b
          JOIN all_orders a
            ON a.tenant_id = b.tenant_id
           AND a.user_id = b.user_id
           AND a.placed_at > b.placed_at
          GROUP BY b.id
      ),
      order_level AS (
          SELECT
              b.id AS order_id,
              b.tenant_id,
              b.user_id,
              CASE WHEN ohr.order_id IS NOT NULL THEN 'has_return' ELSE 'no_return' END AS grp,
              CASE WHEN nop.next_placed_at IS NOT NULL
                   THEN EXTRACT(EPOCH FROM (nop.next_placed_at - b.placed_at)) / 86400.0
              END AS days_to_next
          FROM base_orders b
          LEFT JOIN order_has_return ohr ON ohr.order_id = b.id
          LEFT JOIN next_order_per_base_order nop ON nop.order_id = b.id
      )
      SELECT
          grp AS "group",
          COUNT(*) AS orders_count,
          COUNT(DISTINCT (tenant_id, user_id)) AS distinct_customers,
          ROUND(
              100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*),
              1
          ) AS pct_with_subsequent_order_within_90_days,
          ROUND(
              CAST(
                  PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days_to_next)
                  FILTER (WHERE days_to_next IS NOT NULL AND days_to_next <= 90)
              AS numeric),
              0
          ) AS median_days_to_subsequent_order
      FROM order_level
      GROUP BY grp
      ORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 END
    • The warehouse answers
      STATUS: COMPILES (dry run on the empty schema, no data read)
      COLUMNS: group, orders_count, distinct_customers, pct_with_subsequent_order_within_90_days, median_days_to_subsequent_order
    • GraceQuery compiles cleanly and returns the expected columns. Submitting.
    • Grace calls submit
      WITH retail_tenants AS (
          SELECT id, name
          FROM tenants
          WHERE name IN (
              'Kibblewick', 'Studio Vrij', 'Petalcourt Beauty',
              'Marrowby & Vane', 'Cragsley Outfitters', 'Ampfeld Elektronik'
          )
      ),
      all_orders AS (
          SELECT o.id, o.tenant_id, o.user_id, o.placed_at
          FROM orders o
          JOIN retail_tenants rt ON rt.id = o.tenant_id
          WHERE o.placed_at IS NOT NULL
      ),
      base_orders AS (
          SELECT *
          FROM all_orders
          WHERE placed_at >= TIMESTAMP '2025-01-01'
            AND placed_at < TIMESTAMP '2025-10-01'
      ),
      order_has_return AS (
          SELECT DISTINCT order_id
          FROM order_returns
      ),
      next_order_per_base_order AS (
          SELECT b.id AS order_id, MIN(a.placed_at) AS next_placed_at
          FROM base_orders b
          JOIN all_orders a
            ON a.tenant_id = b.tenant_id
           AND a.user_id = b.user_id
           AND a.placed_at > b.placed_at
          GROUP BY b.id
      ),
      order_level AS (
          SELECT
              b.id AS order_id,
              b.tenant_id,
              b.user_id,
              CASE WHEN ohr.order_id IS NOT NULL THEN 'has_return' ELSE 'no_return' END AS grp,
              CASE WHEN nop.next_placed_at IS NOT NULL
                   THEN EXTRACT(EPOCH FROM (nop.next_placed_at - b.placed_at)) / 86400.0
              END AS days_to_next
          FROM base_orders b
          LEFT JOIN order_has_return ohr ON ohr.order_id = b.id
          LEFT JOIN next_order_per_base_order nop ON nop.order_id = b.id
      )
      SELECT
          grp AS "group",
          COUNT(*) AS orders_count,
          COUNT(DISTINCT (tenant_id, user_id)) AS distinct_customers,
          ROUND(
              100.0 * SUM(CASE WHEN days_to_next IS NOT NULL AND days_to_next <= 90 THEN 1 ELSE 0 END) / COUNT(*),
              1
          ) AS pct_with_subsequent_order_within_90_days,
          ROUND(
              CAST(
                  PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY days_to_next)
                  FILTER (WHERE days_to_next IS NOT NULL AND days_to_next <= 90)
              AS numeric),
              0
          ) AS median_days_to_subsequent_order
      FROM order_level
      GROUP BY grp
      ORDER BY CASE WHEN grp = 'has_return' THEN 0 ELSE 1 END
    • The warehouse answers
      Submitted to tonight's batch. Your work on this request is done.
    • GraceSubmitted. Summary of the approach: - **Scope**: Orders from the 6 named retail tenants with `placed_at` in 2025‑01‑01 through 2025‑09‑30. - **Grouping**: `has_return` if the order has any row in `order_returns`, else `no_return`. - **Subsequent order**: for each base order, found the nearest later order (strictly after, same tenant+user) from the full orders history (not limited to the base window, so it can reach into Q4 2025). - **Metrics**: orders count, distinct (tenant, user) customers, % with that next order ≤90 days out (1 decimal), and the median days-to-next-order among those within 90 days (rounded to whole days), using `PERCENTILE_CONT`. - **Ordering**: `has_return` row first, then `no_return`, as requested. Noted a caveat in the submit note: "subsequent order" means the *nearest* next order by that customer (not any order within 90 days), and return rows aren't filtered by status/outcome—any return record counts.