Discounts vs repeat rate: run the counterfactual before you add to Q4

The question on the table was "should we add $120,000 to the Q4 discount budget?" The answer the data gave was "hold, fix the leak, and test." Here is the method, with the numbers from the demo tenant so you can check the arithmetic.

1. Count the full cost first

Weekday 20% off vapes, July 1 – September 20, three stores: 9,120 redemptions, $86,400 in discount (average $9.47). Adult-use only; medical tickets are excluded from the start — they're a different population with different incentives.

2. Find the leakage

The rule as configured: Monday–Thursday, category = Vapes. Checked against ticket attributes:

That is money the discount was never meant to spend. The fix isn't analytical — it's a POS discount-rule correction, drafted as a task for the store lead. One store (on a CSV export with no rule ID on discount lines) had to be excluded from the leakage figure and says so on the report; partial data is labelled, not guessed.

3. Build the counterfactual

The real question: do discounted first-time buyers come back more?

• Population: first-time buyers, July 1 – August 20, with a 60-day follow-up window.
• Matching: store, weekday, basket band, category.
• Discounted first visit: 1,480 customers → 462 repeat (31.2%).
• Not discounted (matched): 2,960 → 1,024 repeat (34.6%).
• Difference: −3.4 points, 95% CI −6.3 to −0.5.

Unadjusted standard error: √(0.312·0.688/1480 + 0.346·0.654/2960) ≈ 0.0149 → ±2.9 points. The interval excludes zero; the discounted cohort came back less often. Caption on the report, every time: observational; unmeasured confounders possible.

4. Look at the curve, not just the rate

Second visit within 60 days: 33.5% overall. The cliff: 58% of two-visit customers never make a third within 90 days, and the drop concentrates on days 21–35. For the discounted cohort the cliff is 64%. The discount isn't building a habit; it's pulling forward a visit that would have happened and then losing the customer at the same point as everyone else.

5. Propose, don't decide

The agent's recommendation, with confidence moderate (matched cohorts, interval excludes zero):

1. Hold the Q4 budget — do not add $120,000.
2. Fix the leakage — POS rule correction, $18,740 a quarter at current run-rate.
3. Test — a day-25 win-back email to consented members, four weeks, 10% holdout, measured on third-visit rate. If the holdout shows lift with an interval that excludes zero, scale it; if not, the money stays held.

The agent proposes; it never changes a budget, a discount or a POS rule. A controller approved the hold and the test; the store lead applied the rule fix.

Why this matters under compression

Mean discount on THC products is already around 13% (pre-rolls ~20%) in early 2026. Every point of discount under compression is margin you can't get back by volume. The cheapest wins are leakage (free) and killing promos whose counterfactual is negative (also free). Adding budget is the expensive guess.

Checklist

• Full promo cost, adult-use only, by store.
• Leakage: redemptions vs rule attributes; POS rule fix drafted.
• Matched cohorts, first-visit based, 60-day window, k ≥ 8 per cell.
• Difference with a 95% CI and the observational caption.
• Retention curve; find the cliff.
• Recommendation = hold / fix / test with a holdout; a person approves.

BudAlly's promo ROI agent runs this on any POS and drafts the recommendation into the approval queue — the retail intelligence module.