DB Olympics

Queries · q292 · VP Product · week 2, night 2

Do stalled carts that receive an email convert more often?

“Among carts that don't convert within the first hour, is there already an organic recovery nudge happening (an email reaching the shopper), and does it correlate with eventually converting?”

Manager · asked the questionJonas FeldVP Product. Joined from a mobile-first retailer. Deciding the 2026 roadmap.
Analyst · planned the reportEmeka ObiCustomer-centric: follows cohorts of shoppers or merchants over time.
Engineer · wrote the SQLOmarPragmatic: the shortest correct query, aggregates first and joins dimensions afterwards, FILTER clauses for conditional counts.

Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told

  1. A business event lands on the VP Product's desk

    Week 1

    We are redesigning the shopper journey for 2026. We need to understand the whole path, from the first visit to checkout, delivery, returns and the next purchase, and where shoppers drop out and why.

  2. Jonas Feld writes to Emeka Obi

    The manager, at the start of week 2

    Subject: Good start — let's dig into the cart leak

    Emeka, this is exactly the kind of clarity I needed. The cart-stage bleed is the headline, and I agree we should lead with it — but a two-thirds drop-off that lingers for days, converting at under 20%, isn't something we can act on until we know why people stall. "Add cart recovery emails" is a guess until we understand the shape of the hesitation. That's my main ask for this week.

    I also want to flag one thing before we build a roadmap slide on repeat purchase and returns: the return-vs-repeat gap and the delivery/returns pattern are both flagged as not-yet-causal, confounded by vertical. I don't want to present an 8-point "returns hurt loyalty" number to the exec team if it's really just "apparel behaves differently." Let's firm that up before it goes in a deck, but it's not urgent this week.

    Questions for this week:

    1. What does a stalled cart actually contain, and does it differ from a fast-converting one? I'm thinking about basket size, price point, discount/coupon presence — anything that hints at why someone hesitates for hours or days instead of minutes.
    2. Is the cart bleed getting better, worse, or flat over the past year? If it's been stable at ~70% for two years, that's a structural product problem; if it's worsening recently, something changed (a checkout step, a fee, a release) that we should find and reverse.
    3. Of the carts that eventually do convert after sitting a while, is there a nudge already happening (a reminder, a return visit, a discount) that we could learn from and do more deliberately? I want to know if recovery is already happening organically before we invest in prompting it.
    4. On the subscription merchants' tracking gap — is this just a blind spot in how we instrument recurring orders, or is it telling us something about how those shoppers actually behave (e.g., they never browse again after the first signup)? A quick gut-check is enough this week; no need for a deep investigation yet.

    Keep the returns/delivery vertical-control work and the causal retention question on the list for whenever a slot frees up, but the cart story is what I want to walk into the roadmap review with real conviction on.

    Thanks, Emeka — strong week.

    Jonas

  3. Emeka Obi plans the night

    The analyst · week 2, night 2, request 3 of 3

    Reading last night's reports:

    Report 1 (cart contents) is the headline for Q1: fast-converting carts (<1hr) average 2.79 items and €242 (median €131), while stalled carts (1-24h, 1-7d) average only 1.6-1.8 items and €169-210 (median €75-84). So hesitation concentrates in smaller, lower-value, mostly single-item carts — not "sticker shock" on big purchases. Interestingly, per-item price is actually higher in stalled carts (~€97-117/item) than fast carts (~€87/item), suggesting the stalling archetype may be "one pricier item, deliberated over" rather than "big basket, abandoned." I want one more cut on price tier specifically to confirm this before I put it in front of Jonas. Caveat: the discount-presence column is only observable among carts that converted (abandoned carts never reach an order, so there's no discount record) — it's a real but narrower sub-question than "stalled vs fast," and I've noted that.

    Report 2 (monthly trend) shows a genuinely flat cart→order conversion rate (34-38%) Jan-Oct — good evidence the bleed is structural, not worsening, which is itself useful for Jonas. But Nov (37.7%) and especially Dec's lower 1-7-day resolution rate (62.5% vs typical 69-73%) are suspect: cart volume nearly doubled in Nov/Dec (holiday season) and any cart created in the back half of December simply hasn't had a full 7 days to resolve against our Dec-31 data cutoff — that's right-censoring, not a real behavior change, and I should not over-read it. More interesting: last week's company channel flagged that cart-reminder emails broke for exactly the week of Nov 24 at 10 merchants including Cragsley Outfitters (one of our 6 retail tenants). That's a natural experiment sitting right there, and it answers two of Jonas's questions at once — Q2 ("what changed") and Q3 ("is a nudge already happening") — so I'm prioritizing it tonight.

    Report 3 (subscription tracking gap) flipped my expectation: "after first order" session coverage is actually substantial (42-70% of customers have a session after, averaging 2.4-4.1 of them) — these shoppers do keep coming back and browsing. The real gap is "before first order" (only 16-30% have a tracked session pre-signup), which looks more like a tracking gap at the signup moment than a behavioral finding. That's a sufficient "quick gut-check" per Jonas's ask — I'm not spending more slots on this this week.

    Tonight's 3 requests: (1) the Nov 24 cart-reminder-outage natural experiment at Cragsley Outfitters, (2) a price-tier cut to sharpen the Q1 "why" story, (3) a direct empirical test of whether an email nudge correlates with recovery among stalled carts generally (not just the outage week) — the clearest way to answer Jonas's "is a nudge already happening that we could learn from."

    Do stalled carts that receive an email convert more often?
    Question
    Among carts that don't convert within the first hour, is there already an organic recovery nudge happening (an email reaching the shopper), and does it correlate with eventually converting?
    One row per
    group: carts whose user received an email in the 1-48 hour window after cart creation, vs. carts whose user did not
    Columns
    • group (received email / no email)
    • cart count
    • pct eventually converted
    • median days to convert (converted carts only)
  4. Omar writes the SQL

    The engineer · 1 dry run on the empty schema, then the submission

    Dry run 1 · STATUS: COMPILES (dry run on the empty schema, no data read)
    WITH six_tenants AS (
      SELECT id FROM tenants
      WHERE name IN ('Kibblewick','Studio Vrij','Petalcourt Beauty','Marrowby & Vane','Cragsley Outfitters','Ampfeld Elektronik')
    ),
    population AS (
      SELECT c.id, c.tenant_id, c.user_id, c.status, c.converted_at, c.created_at
      FROM carts c
      JOIN six_tenants st ON st.id = c.tenant_id
      WHERE c.created_at >= DATE '2025-01-01'
        AND c.created_at < DATE '2025-12-01'
        AND NOT (c.status = 'converted' AND c.converted_at IS NOT NULL AND c.converted_at <= c.created_at + INTERVAL '1 hour')
    ),
    grouped AS (
      SELECT p.*,
        CASE WHEN p.user_id IS NOT NULL AND EXISTS (
          SELECT 1 FROM email_log e
          WHERE e.tenant_id = p.tenant_id
            AND e.user_id = p.user_id
            AND e.sent_at > p.created_at + INTERVAL '1 hour'
            AND e.sent_at <= p.created_at + INTERVAL '48 hour'
        ) THEN 'received_email' ELSE 'no_email' END AS grp
      FROM population p
    ),
    agg AS (
      SELECT grp,
        COUNT(*) AS cart_count,
        COUNT(*) FILTER (WHERE user_id IS NULL) AS guest_cart_count,
        COUNT(*) FILTER (WHERE status = 'converted') AS converted_count
      FROM grouped
      GROUP BY grp
    ),
    medians AS (
      SELECT grp,
        PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (converted_at - created_at))/86400.0) AS median_days_to_convert
      FROM grouped
      WHERE status = 'converted'
      GROUP BY grp
    )
    SELECT
      a.grp AS group_name,
      a.cart_count,
      a.guest_cart_count,
      ROUND(100.0 * a.converted_count / a.cart_count, 1) AS pct_eventually_converted,
      ROUND(m.median_days_to_convert::numeric, 1) AS median_days_to_convert
    FROM agg a
    LEFT JOIN medians m ON m.grp = a.grp
    ORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END, a.grp

    “Query compiles cleanly. Submitting.”

    Submitted · the SQL of the last dry run, unchanged · Submitted to tonight's batch. Your work on this request is done.

    “Submitted. Summary of what the query does: - Population: carts from the 6 named retail tenants, created 2025‑01‑01 through 2025‑11‑30, excluding carts that already converted within 1 hour of creation. - Grouping: for each remaining cart with a non-null user_id, checks email_log for any row with matching tenant_id/user_id and sent_at in (created_at+1h, created_at+48h] → received_email; everything else (no match, or guest carts with null user_id) → no_email. - Metrics per group: cart count, guest-cart count (null user_id, only relevant to no_email), % eventually converted (status='converted', any time), and median days from created_at to converted_at among converted carts. - Ordered received_email first, then no_email. I added the guest_cart_count column instead of a prose note since I can't preview actual data values (dry runs don't return rows), and it keeps the report self-contained for reading the comparison correctly.”

  5. The report comes back overnight

    As Emeka Obi saw it, from the small dataset the agents worked on

    group_name     | cart_count | guest_cart_count | pct_eventually_converted | median_days_to_convert
    ---------------+------------+------------------+--------------------------+-----------------------
    received_email |        587 |                0 |                     56.0 |                    1.4
    no_email       |      12280 |             7540 |                      8.1 |                    4.8
    (2 rows)
  6. What came of it

    The memo and the queries that followed; links found by title, so inferred

    Later queries that build on this report:

    It builds on:

  7. It becomes benchmark query q292

    Changed only where the benchmark needs the same result on every system

    • spellingcast
    • numericsdocs/NUMERICS.md: rounding removed; results keep full precision; bare NUMERIC cast that only served the rounding removed
    • numericsdocs/NUMERICS.md: EXTRACT(EPOCH FROM (b - a)) cast to NUMERIC(18,6)
    • made explicita selected t.col gets AS col, its standard name made explicit (8×)
    • placeholderpercentile cont
    • placeholderseconds between
    • determinismthe standard leaves the place of NULLs in a sort to the implementation: NULLS LAST ascending, NULLS FIRST descending, as in the reference results (2×)
    The benchmark text, against the submitted SQL
    WITH six_tenants AS (  SELECT id FROM tenants  WHERE name IN ('Kibblewick','Studio Vrij','Petalcourt Beauty','Marrowby & Vane','Cragsley Outfitters','Ampfeld Elektronik')),population AS (  SELECT c.id, c.tenant_id, c.user_id, c.status, c.converted_at, c.created_at  SELECT c.id AS id, c.tenant_id AS tenant_id, c.user_id AS user_id, c.status AS status, c.converted_at AS converted_at, c.created_at AS created_at  FROM carts c  JOIN six_tenants st ON st.id = c.tenant_id  WHERE c.created_at >= DATE '2025-01-01'    AND c.created_at < DATE '2025-12-01'    AND NOT (c.status = 'converted' AND c.converted_at IS NOT NULL AND c.converted_at <= c.created_at + INTERVAL '1 hour')),grouped AS (  SELECT p.*,    CASE WHEN p.user_id IS NOT NULL AND EXISTS (      SELECT 1 FROM email_log e      WHERE e.tenant_id = p.tenant_id        AND e.user_id = p.user_id        AND e.sent_at > p.created_at + INTERVAL '1 hour'        AND e.sent_at <= p.created_at + INTERVAL '48 hour'    ) THEN 'received_email' ELSE 'no_email' END AS grp  FROM population p),agg AS (  SELECT grp,    COUNT(*) AS cart_count,    COUNT(*) FILTER (WHERE user_id IS NULL) AS guest_cart_count,    COUNT(*) FILTER (WHERE status = 'converted') AS converted_count  FROM grouped  GROUP BY grp),medians AS (  SELECT grp,    PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (converted_at - created_at))/86400.0) AS median_days_to_convert    {{percentile_cont(0.5, {{seconds_between(created_at, converted_at)}}/86400.0)}} AS median_days_to_convert  FROM grouped  WHERE status = 'converted'  GROUP BY grp)SELECT  a.grp AS group_name,  a.cart_count,  a.guest_cart_count,  ROUND(100.0 * a.converted_count / a.cart_count, 1) AS pct_eventually_converted,  ROUND(m.median_days_to_convert::numeric, 1) AS median_days_to_convert  a.cart_count AS cart_count,  a.guest_cart_count AS guest_cart_count,  100.0 * a.converted_count / a.cart_count AS pct_eventually_converted,  m.median_days_to_convert AS median_days_to_convertFROM agg aLEFT JOIN medians m ON m.grp = a.grpORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END, a.grpORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END NULLS LAST, a.grp NULLS LAST
    WITH six_tenants AS (
      SELECT id FROM tenants
      WHERE name IN ('Kibblewick','Studio Vrij','Petalcourt Beauty','Marrowby & Vane','Cragsley Outfitters','Ampfeld Elektronik')
    ),
    population AS (
      SELECT c.id AS id, c.tenant_id AS tenant_id, c.user_id AS user_id, c.status AS status, c.converted_at AS converted_at, c.created_at AS created_at
      FROM carts c
      JOIN six_tenants st ON st.id = c.tenant_id
      WHERE c.created_at >= DATE '2025-01-01'
        AND c.created_at < DATE '2025-12-01'
        AND NOT (c.status = 'converted' AND c.converted_at IS NOT NULL AND c.converted_at <= c.created_at + INTERVAL '1 hour')
    ),
    grouped AS (
      SELECT p.*,
        CASE WHEN p.user_id IS NOT NULL AND EXISTS (
          SELECT 1 FROM email_log e
          WHERE e.tenant_id = p.tenant_id
            AND e.user_id = p.user_id
            AND e.sent_at > p.created_at + INTERVAL '1 hour'
            AND e.sent_at <= p.created_at + INTERVAL '48 hour'
        ) THEN 'received_email' ELSE 'no_email' END AS grp
      FROM population p
    ),
    agg AS (
      SELECT grp,
        COUNT(*) AS cart_count,
        COUNT(*) FILTER (WHERE user_id IS NULL) AS guest_cart_count,
        COUNT(*) FILTER (WHERE status = 'converted') AS converted_count
      FROM grouped
      GROUP BY grp
    ),
    medians AS (
      SELECT grp,
        PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY CAST(EXTRACT(EPOCH FROM (converted_at - created_at)) AS NUMERIC(18,6))/86400.0) AS median_days_to_convert
      FROM grouped
      WHERE status = 'converted'
      GROUP BY grp
    )
    SELECT
      a.grp AS group_name,
      a.cart_count AS cart_count,
      a.guest_cart_count AS guest_cart_count,
      100.0 * a.converted_count / a.cart_count AS pct_eventually_converted,
      m.median_days_to_convert AS median_days_to_convert
    FROM agg a
    LEFT JOIN medians m ON m.grp = a.grp
    ORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END NULLS LAST, a.grp NULLS LAST
    100 ms1 s10 sHyper598 msDuckDB361 msGizmoSQL351 msclickhouse_brew5.35 sFirebolt3.25 sDataFusionerrorClickHouse5.49 s
    • DataFusionThis feature is not implemented: Physical plan does not support logical expression Exists(Exists { subquery: <subquery>, negated: false })
  8. Everything, as recorded

    The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted

    Omar's whole session, 7 turns
    • The request, as Omar received it
      Report request from Emeka Obi (analyst for the VP Product):
      
      Title: Do stalled carts that receive an email convert more often?
      Question: Among carts that don't convert within the first hour, is there already an organic recovery nudge happening (an email reaching the shopper), and does it correlate with eventually converting?
      Grain: one row per group: carts whose user received an email in the 1-48 hour window after cart creation, vs. carts whose user did not
      Columns:
      - group (received email / no email)
      - cart count
      - pct eventually converted
      - median days to convert (converted carts only)
      Details: Tenants: the 6 retail merchants (Kibblewick, Studio Vrij, Petalcourt Beauty, Marrowby & Vane, Cragsley Outfitters, Ampfeld Elektronik). Population: carts with created_at in 2025-01-01 to 2025-11-30 (leaves at least a month of follow-up before the 2025-12-31 cutoff), restricted to carts NOT already converted within the first hour of creation (i.e., still open at the 1-hour mark — these are the ones that would actually need a nudge). For each such cart with a non-null user_id, check email_log for any row with the same tenant_id and user_id where sent_at falls between (cart's created_at + 1 hour) and (cart's created_at + 48 hours); group = 'received_email' if at least one such row exists, else 'no_email' (carts with a null user_id, i.e. guest carts with no way to match an email recipient, also fall into 'no_email' — flag in the report notes what share of the population that is, since it affects how to read the comparison). pct_eventually_converted = % of carts in the group with status='converted' (any time). median_days_to_convert = median of (converted_at - created_at) in days among the converted carts in that group. Order rows with 'received_email' first, then 'no_email'.
      
      Check your query with the warehouse's dry_run tool, then hand it in with submit.
    • Omar calls dry_run
      WITH six_tenants AS (
        SELECT id FROM tenants
        WHERE name IN ('Kibblewick','Studio Vrij','Petalcourt Beauty','Marrowby & Vane','Cragsley Outfitters','Ampfeld Elektronik')
      ),
      population AS (
        SELECT c.id, c.tenant_id, c.user_id, c.status, c.converted_at, c.created_at
        FROM carts c
        JOIN six_tenants st ON st.id = c.tenant_id
        WHERE c.created_at >= DATE '2025-01-01'
          AND c.created_at < DATE '2025-12-01'
          AND NOT (c.status = 'converted' AND c.converted_at IS NOT NULL AND c.converted_at <= c.created_at + INTERVAL '1 hour')
      ),
      grouped AS (
        SELECT p.*,
          CASE WHEN p.user_id IS NOT NULL AND EXISTS (
            SELECT 1 FROM email_log e
            WHERE e.tenant_id = p.tenant_id
              AND e.user_id = p.user_id
              AND e.sent_at > p.created_at + INTERVAL '1 hour'
              AND e.sent_at <= p.created_at + INTERVAL '48 hour'
          ) THEN 'received_email' ELSE 'no_email' END AS grp
        FROM population p
      ),
      agg AS (
        SELECT grp,
          COUNT(*) AS cart_count,
          COUNT(*) FILTER (WHERE user_id IS NULL) AS guest_cart_count,
          COUNT(*) FILTER (WHERE status = 'converted') AS converted_count
        FROM grouped
        GROUP BY grp
      ),
      medians AS (
        SELECT grp,
          PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (converted_at - created_at))/86400.0) AS median_days_to_convert
        FROM grouped
        WHERE status = 'converted'
        GROUP BY grp
      )
      SELECT
        a.grp AS group_name,
        a.cart_count,
        a.guest_cart_count,
        ROUND(100.0 * a.converted_count / a.cart_count, 1) AS pct_eventually_converted,
        ROUND(m.median_days_to_convert::numeric, 1) AS median_days_to_convert
      FROM agg a
      LEFT JOIN medians m ON m.grp = a.grp
      ORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END, a.grp
    • The warehouse answers
      STATUS: COMPILES (dry run on the empty schema, no data read)
      COLUMNS: group_name, cart_count, guest_cart_count, pct_eventually_converted, median_days_to_convert
    • OmarQuery compiles cleanly. Submitting.
    • Omar calls submit
      WITH six_tenants AS (
        SELECT id FROM tenants
        WHERE name IN ('Kibblewick','Studio Vrij','Petalcourt Beauty','Marrowby & Vane','Cragsley Outfitters','Ampfeld Elektronik')
      ),
      population AS (
        SELECT c.id, c.tenant_id, c.user_id, c.status, c.converted_at, c.created_at
        FROM carts c
        JOIN six_tenants st ON st.id = c.tenant_id
        WHERE c.created_at >= DATE '2025-01-01'
          AND c.created_at < DATE '2025-12-01'
          AND NOT (c.status = 'converted' AND c.converted_at IS NOT NULL AND c.converted_at <= c.created_at + INTERVAL '1 hour')
      ),
      grouped AS (
        SELECT p.*,
          CASE WHEN p.user_id IS NOT NULL AND EXISTS (
            SELECT 1 FROM email_log e
            WHERE e.tenant_id = p.tenant_id
              AND e.user_id = p.user_id
              AND e.sent_at > p.created_at + INTERVAL '1 hour'
              AND e.sent_at <= p.created_at + INTERVAL '48 hour'
          ) THEN 'received_email' ELSE 'no_email' END AS grp
        FROM population p
      ),
      agg AS (
        SELECT grp,
          COUNT(*) AS cart_count,
          COUNT(*) FILTER (WHERE user_id IS NULL) AS guest_cart_count,
          COUNT(*) FILTER (WHERE status = 'converted') AS converted_count
        FROM grouped
        GROUP BY grp
      ),
      medians AS (
        SELECT grp,
          PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY EXTRACT(EPOCH FROM (converted_at - created_at))/86400.0) AS median_days_to_convert
        FROM grouped
        WHERE status = 'converted'
        GROUP BY grp
      )
      SELECT
        a.grp AS group_name,
        a.cart_count,
        a.guest_cart_count,
        ROUND(100.0 * a.converted_count / a.cart_count, 1) AS pct_eventually_converted,
        ROUND(m.median_days_to_convert::numeric, 1) AS median_days_to_convert
      FROM agg a
      LEFT JOIN medians m ON m.grp = a.grp
      ORDER BY CASE WHEN a.grp = 'received_email' THEN 0 ELSE 1 END, a.grp
    • The warehouse answers
      Submitted to tonight's batch. Your work on this request is done.
    • OmarSubmitted. Summary of what the query does: - **Population**: carts from the 6 named retail tenants, created 2025‑01‑01 through 2025‑11‑30, excluding carts that already converted within 1 hour of creation. - **Grouping**: for each remaining cart with a non-null `user_id`, checks `email_log` for any row with matching `tenant_id`/`user_id` and `sent_at` in `(created_at+1h, created_at+48h]` → `received_email`; everything else (no match, or guest carts with null `user_id`) → `no_email`. - **Metrics per group**: cart count, guest-cart count (null `user_id`, only relevant to `no_email`), % eventually converted (`status='converted'`, any time), and median days from `created_at` to `converted_at` among converted carts. - Ordered `received_email` first, then `no_email`. I added the `guest_cart_count` column instead of a prose note since I can't preview actual data values (dry runs don't return rows), and it keeps the report self-contained for reading the comparison correctly.