Queries · q260 · Head of Growth Marketing · week 2, night 2
“Controlling for category, do loyalty members actually come back and buy again more than non-members, or does that pattern not hold either once you look within a vertical?”
Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told
The manager, at the start of week 2
Subject: Good progress — four things I want nailed down this week
Hi Sofie,
Thanks for this — genuinely useful, and the vertical breakdown on discounting is exactly the kind of "don't trust the pooled number" catch I was worried about. A few things surprised me and I want to go deeper before I take anything to leadership.
What surprised me / needs explaining:
My questions for this week, roughly in priority order:
What I need this for: I'm trying to turn this into a short list of 2-3 concrete recommendations for merchants and for our own roadmap — where to tell them to pull back (discounting), where to double down (referrals), and where we might be quietly burning money (loyalty payouts). I'd rather have fewer, firmer answers than a wider, shakier set.
Thanks again — this is coming together well.
Chloé
The analyst · week 2, night 2, request 2 of 3
Night 1 settled Q1: discounting's real cost is heavily vertical-dependent. Apparel, Beauty, Home & Living, and Sports & Outdoor show discounted orders costing 17-42 percentage points more of GMV (combined discount+refund) than full-price orders in the same vertical — genuine margin leakage worth telling merchants to rein in. Food & Coffee and Pet Supplies show even bigger leakage % (83-89%), but that's near-free subscription trial offers, a different (acquisition-cost) story, not wasteful promo. The coupon-code report found only 5 distinct code templates drive almost all 2025 redemptions platform-wide, with WELCOME10 alone responsible for €51k net cost (discount+refund) — confirms and quantifies Oliver's finding, and the monthly trend confirms leakage worsened into Q3/Q4, tracking the refund-rate creep CEO/CFO flagged. Q1 is in good shape for the memo. Tonight I'm moving to Q2 (loyalty): Ingrid's team already gave the company-wide cost number (19.4% of member revenue redeemed, €669k outstanding liability) and the category-mix correction on returns, so I'm pushing that down to the vertical level (cost) and adding the one 'worth' metric Chloé asked for that we're missing — retention/repeat-purchase rate, controlled by category. Third slot starts Q3 (referrals) at the same vertical grain for comparability, to see where reach is least tapped.
The engineer · 1 dry run on the empty schema, then the submission
WITH loyalty_industries AS ( SELECT DISTINCT t.industry FROM loyalty_programs lp JOIN tenants t ON t.id = lp.tenant_id WHERE lp.is_active = true AND t.industry IS NOT NULL ), orders_2025 AS ( SELECT o.id AS order_id, o.user_id, o.tenant_id, t.industry, o.placed_at, o.grand_total, o.currency_code, CASE WHEN EXISTS ( SELECT 1 FROM loyalty_accounts la WHERE la.user_id = o.user_id AND la.tenant_id = o.tenant_id ) THEN 'loyalty member' ELSE 'non-member' END AS segment FROM orders o JOIN tenants t ON t.id = o.tenant_id WHERE o.placed_at >= DATE '2025-01-01' AND o.placed_at < DATE '2026-01-01' AND t.industry IN (SELECT industry FROM loyalty_industries) ), orders_eur AS ( SELECT oe.*, CASE WHEN oe.currency_code = 'EUR' THEN oe.grand_total ELSE oe.grand_total / er.rate END AS grand_total_eur FROM orders_2025 oe LEFT JOIN exchange_rates er ON er.base_code = 'EUR' AND er.quote_code = oe.currency_code AND er.as_of = CAST(oe.placed_at AS DATE) ), user_stats AS ( SELECT industry, segment, user_id, COUNT(*) AS order_count, COUNT(DISTINCT CAST(placed_at AS DATE)) AS distinct_order_days FROM orders_eur GROUP BY industry, segment, user_id ), agg_users AS ( SELECT industry, segment, COUNT(*) AS users_count, COUNT(*) FILTER (WHERE distinct_order_days >= 2) AS repeat_users, AVG(order_count) AS avg_orders_per_user FROM user_stats GROUP BY industry, segment ), agg_orders AS ( SELECT industry, segment, AVG(grand_total_eur) AS avg_order_value_eur FROM orders_eur GROUP BY industry, segment ) SELECT au.industry, au.segment, au.users_count, ROUND(100.0 * au.repeat_users / NULLIF(au.users_count,0), 1) AS repeat_rate_pct, ROUND(au.avg_orders_per_user::numeric, 2) AS avg_orders_per_user, ROUND(ao.avg_order_value_eur::numeric, 0) AS avg_order_value_eur FROM agg_users au JOIN agg_orders ao ON ao.industry = au.industry AND ao.segment = au.segment ORDER BY au.industry ASC, au.segment ASC
“The query compiles and returns the expected columns. Submitting.”
“Submitted. Summary of key modeling choices (also noted for the analyst):
- Segment: a user is a "loyalty member" if any loyalty_accounts row exists for their (user_id, tenant_id), independent of program status.
- Repeat rate: based on ≥2 distinct placed_at calendar days in 2025 (not just ≥2 order rows), matching "distinct orders (by placed_at date)" in the request.
- avg_orders_per_user: counts all 2025 orders per user (not deduped by day).
- Industry scope: an industry qualifies if any tenant in it has an active loyalty program, but once qualified, all orders/users from all tenants in that industry are included in the comparison — this is the natural reading of "industries with an active loyalty program only" at the grain requested (one row per industry), but flagged in case the intent was to restrict to only the tenant(s) with the active program.
- FX: grand_total converted to EUR via exchange_rates (EUR base) as of the order's placed_at date, with EUR orders passed through at 1.0.”
As Sofie Nilsen saw it, from the small dataset the agents worked on
industry | segment | users_count | repeat_rate_pct | avg_orders_per_user | avg_order_value_eur ----------------------+----------------+-------------+-----------------+---------------------+-------------------- Apparel & Footwear | loyalty member | 619 | 58.5 | 2.53 | 165 Apparel & Footwear | non-member | 2140 | 59.6 | 2.62 | 189 Beauty | loyalty member | 45 | 62.2 | 2.84 | 102 Beauty | non-member | 447 | 65.1 | 3.22 | 96 Books & Stationery | loyalty member | 8 | 62.5 | 3.13 | 69 Books & Stationery | non-member | 47 | 63.8 | 2.85 | 48 Consumer Electronics | loyalty member | 25 | 72.0 | 2.28 | 446 Consumer Electronics | non-member | 250 | 45.2 | 2.08 | 1114 Food & Coffee | loyalty member | 7 | 100.0 | 7.71 | 61 Food & Coffee | non-member | 831 | 85.1 | 8.64 | 30 Home & Living | loyalty member | 26 | 65.4 | 2.23 | 144 Home & Living | non-member | 710 | 52.0 | 2.26 | 149 Jewelry & Accessories | non-member | 29 | 41.4 | 1.72 | 217 Pet Supplies | loyalty member | 4 | 75.0 | 4.00 | 49 Pet Supplies | non-member | 229 | 80.3 | 6.28 | 37 Sports & Outdoor | loyalty member | 128 | 53.9 | 2.56 | 355 Sports & Outdoor | non-member | 330 | 49.7 | 2.09 | 372 (17 rows)
The memo and the queries that followed; links found by title, so inferred
It builds on:
Changed only where the benchmark needs the same result on every system
WITH loyalty_industries AS ( SELECT DISTINCT t.industry SELECT DISTINCT t.industry AS industry FROM loyalty_programs lp JOIN tenants t ON t.id = lp.tenant_id WHERE lp.is_active = true AND t.industry IS NOT NULL),orders_2025 AS ( SELECT o.id AS order_id, o.user_id, o.tenant_id, t.industry, o.placed_at, o.grand_total, o.currency_code, o.user_id AS user_id, o.tenant_id AS tenant_id, t.industry AS industry, o.placed_at AS placed_at, o.grand_total AS grand_total, o.currency_code AS currency_code, CASE WHEN EXISTS ( SELECT 1 FROM loyalty_accounts la WHERE la.user_id = o.user_id AND la.tenant_id = o.tenant_id ) THEN 'loyalty member' ELSE 'non-member' END AS segment FROM orders o JOIN tenants t ON t.id = o.tenant_id WHERE o.placed_at >= DATE '2025-01-01' AND o.placed_at < DATE '2026-01-01' AND t.industry IN (SELECT industry FROM loyalty_industries)),orders_eur AS ( SELECT oe.*, CASE WHEN oe.currency_code = 'EUR' THEN oe.grand_total ELSE oe.grand_total / er.rate ELSE CAST(oe.grand_total / er.rate AS NUMERIC(18,6)) END AS grand_total_eur FROM orders_2025 oe LEFT JOIN exchange_rates er ON er.base_code = 'EUR' AND er.quote_code = oe.currency_code AND er.as_of = CAST(oe.placed_at AS DATE)),user_stats AS ( SELECT industry, segment, user_id, COUNT(*) AS order_count, COUNT(DISTINCT CAST(placed_at AS DATE)) AS distinct_order_days FROM orders_eur GROUP BY industry, segment, user_id),agg_users AS ( SELECT industry, segment, COUNT(*) AS users_count, COUNT(*) FILTER (WHERE distinct_order_days >= 2) AS repeat_users, AVG(order_count) AS avg_orders_per_user AVG(CAST(order_count AS NUMERIC(18,6))) AS avg_orders_per_user FROM user_stats GROUP BY industry, segment),agg_orders AS ( SELECT industry, segment, AVG(grand_total_eur) AS avg_order_value_eur FROM orders_eur GROUP BY industry, segment)SELECT au.industry, au.segment, au.users_count, ROUND(100.0 * au.repeat_users / NULLIF(au.users_count,0), 1) AS repeat_rate_pct, ROUND(au.avg_orders_per_user::numeric, 2) AS avg_orders_per_user, ROUND(ao.avg_order_value_eur::numeric, 0) AS avg_order_value_eur au.industry AS industry, au.segment AS segment, au.users_count AS users_count, 100.0 * au.repeat_users / NULLIF(au.users_count,0) AS repeat_rate_pct, au.avg_orders_per_user AS avg_orders_per_user, ao.avg_order_value_eur AS avg_order_value_eurFROM agg_users auJOIN agg_orders ao ON ao.industry = au.industry AND ao.segment = au.segmentORDER BY au.industry ASC, au.segment ASCORDER BY au.industry ASC NULLS LAST, au.segment ASC NULLS LAST
WITH loyalty_industries AS ( SELECT DISTINCT t.industry AS industry FROM loyalty_programs lp JOIN tenants t ON t.id = lp.tenant_id WHERE lp.is_active = true AND t.industry IS NOT NULL ), orders_2025 AS ( SELECT o.id AS order_id, o.user_id AS user_id, o.tenant_id AS tenant_id, t.industry AS industry, o.placed_at AS placed_at, o.grand_total AS grand_total, o.currency_code AS currency_code, CASE WHEN EXISTS ( SELECT 1 FROM loyalty_accounts la WHERE la.user_id = o.user_id AND la.tenant_id = o.tenant_id ) THEN 'loyalty member' ELSE 'non-member' END AS segment FROM orders o JOIN tenants t ON t.id = o.tenant_id WHERE o.placed_at >= DATE '2025-01-01' AND o.placed_at < DATE '2026-01-01' AND t.industry IN (SELECT industry FROM loyalty_industries) ), orders_eur AS ( SELECT oe.*, CASE WHEN oe.currency_code = 'EUR' THEN oe.grand_total ELSE CAST(oe.grand_total / er.rate AS NUMERIC(18,6)) END AS grand_total_eur FROM orders_2025 oe LEFT JOIN exchange_rates er ON er.base_code = 'EUR' AND er.quote_code = oe.currency_code AND er.as_of = CAST(oe.placed_at AS DATE) ), user_stats AS ( SELECT industry, segment, user_id, COUNT(*) AS order_count, COUNT(DISTINCT CAST(placed_at AS DATE)) AS distinct_order_days FROM orders_eur GROUP BY industry, segment, user_id ), agg_users AS ( SELECT industry, segment, COUNT(*) AS users_count, COUNT(*) FILTER (WHERE distinct_order_days >= 2) AS repeat_users, AVG(CAST(order_count AS NUMERIC(18,6))) AS avg_orders_per_user FROM user_stats GROUP BY industry, segment ), agg_orders AS ( SELECT industry, segment, AVG(grand_total_eur) AS avg_order_value_eur FROM orders_eur GROUP BY industry, segment ) SELECT au.industry AS industry, au.segment AS segment, au.users_count AS users_count, 100.0 * au.repeat_users / NULLIF(au.users_count,0) AS repeat_rate_pct, au.avg_orders_per_user AS avg_orders_per_user, ao.avg_order_value_eur AS avg_order_value_eur FROM agg_users au JOIN agg_orders ao ON ao.industry = au.industry AND ao.segment = au.segment ORDER BY au.industry ASC NULLS LAST, au.segment ASC NULLS LAST
The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted
Report request from Sofie Nilsen (analyst for the Head of Growth Marketing):
Title: Loyalty worth by vertical: repeat-purchase rate
Question: Controlling for category, do loyalty members actually come back and buy again more than non-members, or does that pattern not hold either once you look within a vertical?
Grain: one row per industry and segment (loyalty member vs non-member), industries with an active loyalty program only
Columns:
- industry - tenants.industry
- segment - 'loyalty member' or 'non-member'
- users_count - distinct users in that segment who placed >=1 order with placed_at in 2025 in this industry
- repeat_rate_pct - share of those users with >=2 distinct orders (by placed_at date) in 2025, *100, rounded to 1 decimal
- avg_orders_per_user - average count of 2025 orders per user in that segment, rounded to 2 decimals
- avg_order_value_eur - average grand_total (EUR) of that segment's 2025 orders in this industry, rounded to 0 decimals
Details: Source: orders joined to tenants (industry) and loyalty_accounts (member match on user_id+tenant_id), same member/non-member classification as last week. Restrict to industries with an active loyalty program (loyalty_programs.is_active=true for some tenant in that industry). Filter placed_at in 2025. Convert grand_total to EUR using exchange_rates as of placed_at date (EUR=rate 1). Order by industry ascending, then segment ascending ('loyalty member' before 'non-member'). Expect ~9 industries x 2 rows.
Check your query with the warehouse's dry_run tool, then hand it in with submit.dry_runWITH loyalty_industries AS ( SELECT DISTINCT t.industry FROM loyalty_programs lp JOIN tenants t ON t.id = lp.tenant_id WHERE lp.is_active = true AND t.industry IS NOT NULL ), orders_2025 AS ( SELECT o.id AS order_id, o.user_id, o.tenant_id, t.industry, o.placed_at, o.grand_total, o.currency_code, CASE WHEN EXISTS ( SELECT 1 FROM loyalty_accounts la WHERE la.user_id = o.user_id AND la.tenant_id = o.tenant_id ) THEN 'loyalty member' ELSE 'non-member' END AS segment FROM orders o JOIN tenants t ON t.id = o.tenant_id WHERE o.placed_at >= DATE '2025-01-01' AND o.placed_at < DATE '2026-01-01' AND t.industry IN (SELECT industry FROM loyalty_industries) ), orders_eur AS ( SELECT oe.*, CASE WHEN oe.currency_code = 'EUR' THEN oe.grand_total ELSE oe.grand_total / er.rate END AS grand_total_eur FROM orders_2025 oe LEFT JOIN exchange_rates er ON er.base_code = 'EUR' AND er.quote_code = oe.currency_code AND er.as_of = CAST(oe.placed_at AS DATE) ), user_stats AS ( SELECT industry, segment, user_id, COUNT(*) AS order_count, COUNT(DISTINCT CAST(placed_at AS DATE)) AS distinct_order_days FROM orders_eur GROUP BY industry, segment, user_id ), agg_users AS ( SELECT industry, segment, COUNT(*) AS users_count, COUNT(*) FILTER (WHERE distinct_order_days >= 2) AS repeat_users, AVG(order_count) AS avg_orders_per_user FROM user_stats GROUP BY industry, segment ), agg_orders AS ( SELECT industry, segment, AVG(grand_total_eur) AS avg_order_value_eur FROM orders_eur GROUP BY industry, segment ) SELECT au.industry, au.segment, au.users_count, ROUND(100.0 * au.repeat_users / NULLIF(au.users_count,0), 1) AS repeat_rate_pct, ROUND(au.avg_orders_per_user::numeric, 2) AS avg_orders_per_user, ROUND(ao.avg_order_value_eur::numeric, 0) AS avg_order_value_eur FROM agg_users au JOIN agg_orders ao ON ao.industry = au.industry AND ao.segment = au.segment ORDER BY au.industry ASC, au.segment ASC
STATUS: COMPILES (dry run on the empty schema, no data read) COLUMNS: industry, segment, users_count, repeat_rate_pct, avg_orders_per_user, avg_order_value_eur
submitWITH loyalty_industries AS ( SELECT DISTINCT t.industry FROM loyalty_programs lp JOIN tenants t ON t.id = lp.tenant_id WHERE lp.is_active = true AND t.industry IS NOT NULL ), orders_2025 AS ( SELECT o.id AS order_id, o.user_id, o.tenant_id, t.industry, o.placed_at, o.grand_total, o.currency_code, CASE WHEN EXISTS ( SELECT 1 FROM loyalty_accounts la WHERE la.user_id = o.user_id AND la.tenant_id = o.tenant_id ) THEN 'loyalty member' ELSE 'non-member' END AS segment FROM orders o JOIN tenants t ON t.id = o.tenant_id WHERE o.placed_at >= DATE '2025-01-01' AND o.placed_at < DATE '2026-01-01' AND t.industry IN (SELECT industry FROM loyalty_industries) ), orders_eur AS ( SELECT oe.*, CASE WHEN oe.currency_code = 'EUR' THEN oe.grand_total ELSE oe.grand_total / er.rate END AS grand_total_eur FROM orders_2025 oe LEFT JOIN exchange_rates er ON er.base_code = 'EUR' AND er.quote_code = oe.currency_code AND er.as_of = CAST(oe.placed_at AS DATE) ), user_stats AS ( SELECT industry, segment, user_id, COUNT(*) AS order_count, COUNT(DISTINCT CAST(placed_at AS DATE)) AS distinct_order_days FROM orders_eur GROUP BY industry, segment, user_id ), agg_users AS ( SELECT industry, segment, COUNT(*) AS users_count, COUNT(*) FILTER (WHERE distinct_order_days >= 2) AS repeat_users, AVG(order_count) AS avg_orders_per_user FROM user_stats GROUP BY industry, segment ), agg_orders AS ( SELECT industry, segment, AVG(grand_total_eur) AS avg_order_value_eur FROM orders_eur GROUP BY industry, segment ) SELECT au.industry, au.segment, au.users_count, ROUND(100.0 * au.repeat_users / NULLIF(au.users_count,0), 1) AS repeat_rate_pct, ROUND(au.avg_orders_per_user::numeric, 2) AS avg_orders_per_user, ROUND(ao.avg_order_value_eur::numeric, 0) AS avg_order_value_eur FROM agg_users au JOIN agg_orders ao ON ao.industry = au.industry AND ao.segment = au.segment ORDER BY au.industry ASC, au.segment ASC
Submitted to tonight's batch. Your work on this request is done.