Glossary

Holdout

A randomly selected share of an audience (BudAlly default 10%) that does not receive a campaign, so lift can be measured against it. Campaigns ship with one; lift is reported with a confidence interval or 'not significant'.

A holdout turns a campaign into a test. Comparing the customers who were held back with those who got the message tells a store whether the message changed what customers did, rather than taking credit for visits that would have happened anyway.

In BudAlly, lift is measured on the outcome the store chooses, with third-visit rate as the default. Every campaign draft carries its holdout, and the promo analysis recommends a holdout test rather than a budget change. A win-back offer is treated as a hypothesis, not a fix: scale it if the interval excludes zero, stop if it doesn't.

The demo tenant's example is a day-25 win-back email to consented members, run for four weeks and measured on third-visit rate. Only contacts with marketing consent proof get an email invite with a holdout. The loyalty module runs the rule pack and consent check first, then puts the draft in the approval queue with the holdout attached.