Queries · q371 · Head of Lifecycle Marketing · week 2, night 1
“Is the growing points liability just a 'young programme, redemptions lag issuance' phase that will level out, or do even our oldest member cohorts still show a structurally low redemption rate?”
Fictional people at a fictional company, each played by a language-model agent. Who they are and what they are told
Week 1; shared with Tobias Lindqvist, Chief Financial Officer
Is the loyalty and referral programme worth it? Do members and referred customers behave differently from others, and what does the programme cost?
The manager, at the start of week 2
Subject: Re: Loyalty & referral — thank you, and where to dig next
Ingrid, this is excellent — especially catching that the refund-rate gap was a vertical mix artifact. Good instinct to go back and break that out rather than let the scary headline stand.
What surprised me most, and what I want explained before we take this to Tobias as a joint recommendation: a 19.4% cost-to-member-revenue ratio is strikingly high, and members showing lower revenue and order counts than non-members is the opposite of what a loyalty programme is supposed to produce. I don't want to present "the programme loses money" to leadership until I understand whether that's a true causal effect or another mix artifact like the refund rate turned out to be — since 72% of members sit in one vertical, I'm wary of a second apples-to-oranges comparison hiding in there.
Questions for this week, in my own words:
The B2B extension question can wait — I agree we shouldn't open that door until we trust the member economics number. Let's keep that out of this week's slots unless there's room left over.
Thanks again, Ingrid — this is exactly the kind of rigor I want from this team.
Best, Aisha
The analyst · week 2, night 1, request 2 of 3
Aisha's push is right: with 72% of members in one vertical, the revenue/order gap could be a second Simpson's-paradox case just like refunds were. Tonight I'm prioritizing her questions 1-3 in order since she explicitly ranked the apparel-only comparison as the one she'll trust as a verdict, and flagged the liability trajectory and the redemption-linkage retest as the two things that could change the recommendation. Before rebuilding the gaming-hypothesis join (which returned zero matches last time), I'm first running a cheap diagnostic on whether order_id is ever populated on redemption-direction loyalty_transactions at all - if it's structurally never set, I need a different join strategy (e.g. time-proximity matching) rather than repeating the same failed approach, and I'd rather spend a slot finding that out now than waste a bigger request later. Holding question 4 (what better-performing members look like) and the B2B question for later nights, per Aisha's explicit deprioritization.
The engineer · 1 dry run on the empty schema, then the submission
with cohort_accounts as ( select la.id as loyalty_account_id, la.loyalty_program_id, cd.year_num, cd.quarter_num, cd.quarter_start from loyalty_accounts la join calendar_days cd on cd.calendar_date = cast(la.enrolled_at as date) where cd.quarter_start >= date '2023-01-01' and cd.quarter_start <= date '2025-10-01' ), account_currency as ( select ca.loyalty_account_id, ca.year_num, ca.quarter_num, ca.quarter_start, lp.points_per_currency, lp.currency_code from cohort_accounts ca join loyalty_programs lp on lp.id = ca.loyalty_program_id ), transaction_amounts as ( select ac.loyalty_account_id, ac.year_num, ac.quarter_num, ac.quarter_start, lt.direction, case when ac.currency_code = 'EUR' then (lt.points / ac.points_per_currency) else (lt.points / ac.points_per_currency) / er.rate end as amount_eur from account_currency ac join loyalty_transactions lt on lt.loyalty_account_id = ac.loyalty_account_id left join exchange_rates er on er.base_code = 'EUR' and er.quote_code = ac.currency_code and er.as_of = cast(lt.occurred_at as date) ), cohort_members as ( select year_num, quarter_num, quarter_start, count(distinct loyalty_account_id) as member_count from cohort_accounts group by year_num, quarter_num, quarter_start ), cohort_transactions as ( select year_num, quarter_num, quarter_start, sum(case when direction = 'earn' then amount_eur else 0 end) as total_points_earned_value_eur, sum(case when direction <> 'earn' then amount_eur else 0 end) as total_points_redeemed_value_eur from transaction_amounts group by year_num, quarter_num, quarter_start ) select cast(cm.year_num as varchar) || '-Q' || cast(cm.quarter_num as varchar) as enrollment_quarter, cm.member_count, (extract(year from date '2025-12-31') - extract(year from cm.quarter_start)) * 12 + (extract(month from date '2025-12-31') - extract(month from cm.quarter_start)) as months_since_cohort_quarter_start, round(coalesce(ct.total_points_earned_value_eur, 0), 2) as total_points_earned_value_eur, round(coalesce(ct.total_points_redeemed_value_eur, 0), 2) as total_points_redeemed_value_eur, round( case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end, 2 ) as lifetime_redemption_rate_pct from cohort_members cm left join cohort_transactions ct on ct.year_num = cm.year_num and ct.quarter_num = cm.quarter_num and ct.quarter_start = cm.quarter_start order by cm.quarter_start asc
“Query compiles and returns exactly the requested columns, one row per enrollment quarter (12 cohorts, 2023-Q1 through 2025-Q4), ordered oldest-first.”
“Submitted. The report gives one row per enrollment-quarter cohort (2023-Q1 through 2025-Q4) with member counts, cohort age in months, lifetime EUR-converted points earned/redeemed, and the resulting redemption rate — ordered oldest cohort first so Ingrid can read the trend directly to see whether redemption rate rises with cohort age or stays structurally flat even for the oldest cohorts.”
As Ingrid Solberg saw it, from the small dataset the agents worked on
enrollment_quarter | member_count | months_since_cohort_quarter_start | total_points_earned_value_eur | total_points_redeemed_value_eur | lifetime_redemption_rate_pct -------------------+--------------+-----------------------------------+-------------------------------+---------------------------------+----------------------------- 2023-Q1 | 71 | 35 | 53321.25 | 19210.96 | 36.03 2023-Q2 | 79 | 32 | 56411.44 | 17960.9 | 31.84 2023-Q3 | 84 | 29 | 58397.95 | 21700.95 | 37.16 2023-Q4 | 117 | 26 | 69771.34 | 22025.15 | 31.57 2024-Q1 | 77 | 23 | 57808.09 | 16190.81 | 28.01 2024-Q2 | 91 | 20 | 39710.82 | 8435.43 | 21.24 2024-Q3 | 86 | 17 | 46566.8 | 12920.23 | 27.75 2024-Q4 | 162 | 14 | 69095.93 | 17820.6 | 25.79 2025-Q1 | 117 | 11 | 47248.24 | 5470.62 | 11.58 2025-Q2 | 125 | 8 | 25855.05 | 1086.02 | 4.2 2025-Q3 | 127 | 5 | 26118.48 | 2311.09 | 8.85 2025-Q4 | 202 | 2 | 17562.57 | 842.63 | 4.8 (12 rows)
The memo and the queries that followed; links found by title, so inferred
Ingrid Solberg's memo to Aisha Rahman at the end of the week:
3. The programme's real cost is concentrated, and richer than I reported last week. The engaged 17% (225 members) have redeemed €177.8k against €434k of their own revenue — a 40.95% effective discount rate, well above the 19.4% blended figure. The other 83% cost nothing in cash but carry €323.6k in paper liability that, per cohort-aging data, mostly never gets claimed (redemption rates plateau at 32–37% even for 3-year-old cohorts). (Reports: "Loyalty unit economics: ever-redeemed vs never-redeemed"; "Loyalty points redemption rate by enrollment cohort age.")
Later queries that build on this report:
It builds on:
Changed only where the benchmark needs the same result on every system
with cohort_accounts as ( select la.id as loyalty_account_id, la.loyalty_program_id, cd.year_num, cd.quarter_num, cd.quarter_start la.loyalty_program_id AS loyalty_program_id, cd.year_num AS year_num, cd.quarter_num AS quarter_num, cd.quarter_start AS quarter_start from loyalty_accounts la join calendar_days cd on cd.calendar_date = cast(la.enrolled_at as date) where cd.quarter_start >= date '2023-01-01' and cd.quarter_start <= date '2025-10-01'),account_currency as ( select ca.loyalty_account_id, ca.year_num, ca.quarter_num, ca.quarter_start, lp.points_per_currency, lp.currency_code ca.loyalty_account_id AS loyalty_account_id, ca.year_num AS year_num, ca.quarter_num AS quarter_num, ca.quarter_start AS quarter_start, lp.points_per_currency AS points_per_currency, lp.currency_code AS currency_code from cohort_accounts ca join loyalty_programs lp on lp.id = ca.loyalty_program_id),transaction_amounts as ( select ac.loyalty_account_id, ac.year_num, ac.quarter_num, ac.quarter_start, lt.direction, ac.loyalty_account_id AS loyalty_account_id, ac.year_num AS year_num, ac.quarter_num AS quarter_num, ac.quarter_start AS quarter_start, lt.direction AS direction, case when ac.currency_code = 'EUR' then (lt.points / ac.points_per_currency) else (lt.points / ac.points_per_currency) / er.rate else CAST((lt.points / ac.points_per_currency) / er.rate AS NUMERIC(18,6)) end as amount_eur from account_currency ac join loyalty_transactions lt on lt.loyalty_account_id = ac.loyalty_account_id left join exchange_rates er on er.base_code = 'EUR' and er.quote_code = ac.currency_code and er.as_of = cast(lt.occurred_at as date)),cohort_members as ( select year_num, quarter_num, quarter_start, count(distinct loyalty_account_id) as member_count from cohort_accounts group by year_num, quarter_num, quarter_start),cohort_transactions as ( select year_num, quarter_num, quarter_start, sum(case when direction = 'earn' then amount_eur else 0 end) as total_points_earned_value_eur, sum(case when direction <> 'earn' then amount_eur else 0 end) as total_points_redeemed_value_eur from transaction_amounts group by year_num, quarter_num, quarter_start)select cast(cm.year_num as varchar) || '-Q' || cast(cm.quarter_num as varchar) as enrollment_quarter, cm.member_count, cm.member_count AS member_count, (extract(year from date '2025-12-31') - extract(year from cm.quarter_start)) * 12 + (extract(month from date '2025-12-31') - extract(month from cm.quarter_start)) as months_since_cohort_quarter_start, round(coalesce(ct.total_points_earned_value_eur, 0), 2) as total_points_earned_value_eur, round(coalesce(ct.total_points_redeemed_value_eur, 0), 2) as total_points_redeemed_value_eur, round( case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end, 2 ) as lifetime_redemption_rate_pct coalesce(ct.total_points_earned_value_eur, 0) as total_points_earned_value_eur, coalesce(ct.total_points_redeemed_value_eur, 0) as total_points_redeemed_value_eur, case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end as lifetime_redemption_rate_pctfrom cohort_members cmleft join cohort_transactions ct on ct.year_num = cm.year_num and ct.quarter_num = cm.quarter_num and ct.quarter_start = cm.quarter_startorder by cm.quarter_start ascorder by cm.quarter_start asc NULLS LAST, cm.year_num asc NULLS LAST, cm.quarter_num asc NULLS LAST
with cohort_accounts as ( select la.id as loyalty_account_id, la.loyalty_program_id AS loyalty_program_id, cd.year_num AS year_num, cd.quarter_num AS quarter_num, cd.quarter_start AS quarter_start from loyalty_accounts la join calendar_days cd on cd.calendar_date = cast(la.enrolled_at as date) where cd.quarter_start >= date '2023-01-01' and cd.quarter_start <= date '2025-10-01' ), account_currency as ( select ca.loyalty_account_id AS loyalty_account_id, ca.year_num AS year_num, ca.quarter_num AS quarter_num, ca.quarter_start AS quarter_start, lp.points_per_currency AS points_per_currency, lp.currency_code AS currency_code from cohort_accounts ca join loyalty_programs lp on lp.id = ca.loyalty_program_id ), transaction_amounts as ( select ac.loyalty_account_id AS loyalty_account_id, ac.year_num AS year_num, ac.quarter_num AS quarter_num, ac.quarter_start AS quarter_start, lt.direction AS direction, case when ac.currency_code = 'EUR' then (lt.points / ac.points_per_currency) else CAST((lt.points / ac.points_per_currency) / er.rate AS NUMERIC(18,6)) end as amount_eur from account_currency ac join loyalty_transactions lt on lt.loyalty_account_id = ac.loyalty_account_id left join exchange_rates er on er.base_code = 'EUR' and er.quote_code = ac.currency_code and er.as_of = cast(lt.occurred_at as date) ), cohort_members as ( select year_num, quarter_num, quarter_start, count(distinct loyalty_account_id) as member_count from cohort_accounts group by year_num, quarter_num, quarter_start ), cohort_transactions as ( select year_num, quarter_num, quarter_start, sum(case when direction = 'earn' then amount_eur else 0 end) as total_points_earned_value_eur, sum(case when direction <> 'earn' then amount_eur else 0 end) as total_points_redeemed_value_eur from transaction_amounts group by year_num, quarter_num, quarter_start ) select cast(cm.year_num as varchar) || '-Q' || cast(cm.quarter_num as varchar) as enrollment_quarter, cm.member_count AS member_count, (extract(year from date '2025-12-31') - extract(year from cm.quarter_start)) * 12 + (extract(month from date '2025-12-31') - extract(month from cm.quarter_start)) as months_since_cohort_quarter_start, coalesce(ct.total_points_earned_value_eur, 0) as total_points_earned_value_eur, coalesce(ct.total_points_redeemed_value_eur, 0) as total_points_redeemed_value_eur, case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end as lifetime_redemption_rate_pct from cohort_members cm left join cohort_transactions ct on ct.year_num = cm.year_num and ct.quarter_num = cm.quarter_num and ct.quarter_start = cm.quarter_start order by cm.quarter_start asc NULLS LAST, cm.year_num asc NULLS LAST, cm.quarter_num asc NULLS LAST
The agents' own records, as the role play wrote them; only the personal data of the person who ran it is redacted
Report request from Ingrid Solberg (analyst for the Head of Lifecycle Marketing): Title: Loyalty points redemption rate by enrollment cohort age Question: Is the growing points liability just a 'young programme, redemptions lag issuance' phase that will level out, or do even our oldest member cohorts still show a structurally low redemption rate? Grain: one row per loyalty_accounts enrollment cohort, grouped by the calendar quarter the member enrolled Columns: - enrollment_quarter - member_count - months_since_cohort_quarter_start - total_points_earned_value_eur - total_points_redeemed_value_eur - lifetime_redemption_rate_pct Details: Group loyalty_accounts by the calendar quarter of enrolled_at (2023-Q1 through 2025-Q4). member_count = distinct accounts in that cohort. months_since_cohort_quarter_start = number of months between that quarter's start date and 2025-12-31 (fixed reference, so older cohorts show a higher number, e.g. 2023-Q1 to 2025-12-31 is about 36 months). For each cohort's members, total_points_earned_value_eur = lifetime sum of 'earn'-direction loyalty_transactions converted to currency via the account's program points_per_currency and currency_code, then to EUR via exchange_rates as_of occurred_at (same method as the earlier cost-by-month and unit-economics reports). total_points_redeemed_value_eur = same conversion for non-'earn'-direction transactions. lifetime_redemption_rate_pct = total_points_redeemed_value_eur divided by total_points_earned_value_eur as a percentage. Order by enrollment_quarter ascending (oldest cohort first) so the trend across cohort age is easy to read. Check your query with the warehouse's dry_run tool, then hand it in with submit.
dry_runwith cohort_accounts as ( select la.id as loyalty_account_id, la.loyalty_program_id, cd.year_num, cd.quarter_num, cd.quarter_start from loyalty_accounts la join calendar_days cd on cd.calendar_date = cast(la.enrolled_at as date) where cd.quarter_start >= date '2023-01-01' and cd.quarter_start <= date '2025-10-01' ), account_currency as ( select ca.loyalty_account_id, ca.year_num, ca.quarter_num, ca.quarter_start, lp.points_per_currency, lp.currency_code from cohort_accounts ca join loyalty_programs lp on lp.id = ca.loyalty_program_id ), transaction_amounts as ( select ac.loyalty_account_id, ac.year_num, ac.quarter_num, ac.quarter_start, lt.direction, case when ac.currency_code = 'EUR' then (lt.points / ac.points_per_currency) else (lt.points / ac.points_per_currency) / er.rate end as amount_eur from account_currency ac join loyalty_transactions lt on lt.loyalty_account_id = ac.loyalty_account_id left join exchange_rates er on er.base_code = 'EUR' and er.quote_code = ac.currency_code and er.as_of = cast(lt.occurred_at as date) ), cohort_members as ( select year_num, quarter_num, quarter_start, count(distinct loyalty_account_id) as member_count from cohort_accounts group by year_num, quarter_num, quarter_start ), cohort_transactions as ( select year_num, quarter_num, quarter_start, sum(case when direction = 'earn' then amount_eur else 0 end) as total_points_earned_value_eur, sum(case when direction <> 'earn' then amount_eur else 0 end) as total_points_redeemed_value_eur from transaction_amounts group by year_num, quarter_num, quarter_start ) select cast(cm.year_num as varchar) || '-Q' || cast(cm.quarter_num as varchar) as enrollment_quarter, cm.member_count, (extract(year from date '2025-12-31') - extract(year from cm.quarter_start)) * 12 + (extract(month from date '2025-12-31') - extract(month from cm.quarter_start)) as months_since_cohort_quarter_start, round(coalesce(ct.total_points_earned_value_eur, 0), 2) as total_points_earned_value_eur, round(coalesce(ct.total_points_redeemed_value_eur, 0), 2) as total_points_redeemed_value_eur, round( case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end, 2 ) as lifetime_redemption_rate_pct from cohort_members cm left join cohort_transactions ct on ct.year_num = cm.year_num and ct.quarter_num = cm.quarter_num and ct.quarter_start = cm.quarter_start order by cm.quarter_start asc
STATUS: COMPILES (dry run on the empty schema, no data read) COLUMNS: enrollment_quarter, member_count, months_since_cohort_quarter_start, total_points_earned_value_eur, total_points_redeemed_value_eur, lifetime_redemption_rate_pct
submitwith cohort_accounts as ( select la.id as loyalty_account_id, la.loyalty_program_id, cd.year_num, cd.quarter_num, cd.quarter_start from loyalty_accounts la join calendar_days cd on cd.calendar_date = cast(la.enrolled_at as date) where cd.quarter_start >= date '2023-01-01' and cd.quarter_start <= date '2025-10-01' ), account_currency as ( select ca.loyalty_account_id, ca.year_num, ca.quarter_num, ca.quarter_start, lp.points_per_currency, lp.currency_code from cohort_accounts ca join loyalty_programs lp on lp.id = ca.loyalty_program_id ), transaction_amounts as ( select ac.loyalty_account_id, ac.year_num, ac.quarter_num, ac.quarter_start, lt.direction, case when ac.currency_code = 'EUR' then (lt.points / ac.points_per_currency) else (lt.points / ac.points_per_currency) / er.rate end as amount_eur from account_currency ac join loyalty_transactions lt on lt.loyalty_account_id = ac.loyalty_account_id left join exchange_rates er on er.base_code = 'EUR' and er.quote_code = ac.currency_code and er.as_of = cast(lt.occurred_at as date) ), cohort_members as ( select year_num, quarter_num, quarter_start, count(distinct loyalty_account_id) as member_count from cohort_accounts group by year_num, quarter_num, quarter_start ), cohort_transactions as ( select year_num, quarter_num, quarter_start, sum(case when direction = 'earn' then amount_eur else 0 end) as total_points_earned_value_eur, sum(case when direction <> 'earn' then amount_eur else 0 end) as total_points_redeemed_value_eur from transaction_amounts group by year_num, quarter_num, quarter_start ) select cast(cm.year_num as varchar) || '-Q' || cast(cm.quarter_num as varchar) as enrollment_quarter, cm.member_count, (extract(year from date '2025-12-31') - extract(year from cm.quarter_start)) * 12 + (extract(month from date '2025-12-31') - extract(month from cm.quarter_start)) as months_since_cohort_quarter_start, round(coalesce(ct.total_points_earned_value_eur, 0), 2) as total_points_earned_value_eur, round(coalesce(ct.total_points_redeemed_value_eur, 0), 2) as total_points_redeemed_value_eur, round( case when coalesce(ct.total_points_earned_value_eur, 0) = 0 then null else ct.total_points_redeemed_value_eur / ct.total_points_earned_value_eur * 100 end, 2 ) as lifetime_redemption_rate_pct from cohort_members cm left join cohort_transactions ct on ct.year_num = cm.year_num and ct.quarter_num = cm.quarter_num and ct.quarter_start = cm.quarter_start order by cm.quarter_start asc
Submitted to tonight's batch. Your work on this request is done.