The estimand, accounting, diagnostics, and validation behind the membership analysis.
The synthetic panel contains 40,000 customers and 573,352 transactions from January 1, 2024 through December 31, 2025. Program launch is October 1, 2024: 274 pre-period days and 457 post-period days. There are 12,408 adopters and 27,592 non-adopters.
The matched analysis retains 12,153 adopter-control pairs, about 97.94% of adopters. Its target is the full post-launch-window contrast for retained adopters under their observed adoption timing. It is not a randomized offer ITT or a uniform duration-since-adoption effect.
The primary outcome is net product payments per customer per day, after discounts and redeemed reward coins. Membership fees are tracked separately. The secondary outcome is product value at modeled pre-discount prices.
Neither is contribution profit. Issued but unredeemed rewards, costs, and fee revenue recognition need separate accounting before a financial rollout decision. Physical units and willingness to pay are not measured by pre-discount product value.
A logistic propensity model uses pre-program covariates. Greedy one-to-one matching without replacement uses a default caliper of 0.03. Difference-in-differences is then computed within the matched pairs: (member post − member pre) − (control post − control pre).
The pre-spend standardized mean difference falls from 0.3961 to 0.00134. The largest absolute matched model-covariate SMD is approximately 0.0152. These are observed-balance diagnostics, not proof of exchangeability.
The net-payment estimate is −$0.0102538/day, paired SE $0.0066222, p = 0.1216, 95% CI [−$0.0232345, +$0.0027268]. The pre-discount estimate is +$0.1367276/day, 95% CI [+$0.1225555, +$0.1508997].
| Outcome | Estimate | 95% interval |
|---|---|---|
| Product value | +$0.1367 | +$0.1226 to +$0.1509 |
| Net payments | −$0.0103 | −$0.0232 to +$0.0027 |
Dots are matched DiD estimates; lines are 95% intervals. The 457-day view only rescales the same estimate, not a new forecast. Product value uses pre-discount prices; payments exclude membership fees. Neither measures profit.
Source and assumptions ↗Monthly specifications use pair-clustered covariance. Only 49.5% of adopters join in the launch month. Launch-aligned paths therefore mix cohorts and exposure durations; they should not be read as cohort-specific treatment dynamics.
The overall joint pretrend diagnostic has raw p = 0.0165 and Holm-adjusted p = 0.1154 across seven diagnostics. Report both: adjusted non-rejection does not establish parallel trends or practical equivalence.
Identification still requires assumptions about parallel counterfactual trends, selection, anticipation, and interference. Matching cannot remove unobserved confounding by itself.
At caliper 0.10 the net-payment estimate becomes nominally significant (p = 0.0467), compared with p = 0.1216 at the default. Statistical significance is therefore not stable across all matching specifications.
The in-store net-payment contrast is −$0.01210/day (nominal p = 0.0131); online is +$0.00184/day (p = 0.7123). These channel results do not establish a profitable overall program. The interactive dashboard includes four monthly outcomes and an additive trend-violation sensitivity control.
12,153 matched pairs. Unadjusted nominal p = 0.1216. The adjusted interval includes zero.
| Outcome | Estimate | 95% interval |
|---|---|---|
| Unadjusted | −$0.0103 | −$0.0232 to +$0.0027 |
| Trend-adjusted | −$0.0103 | −$0.0232 to +$0.0027 |
Trend adjustment subtracts an assumed untreated difference in pre-to-post daily spending changes from the estimate and both interval endpoints, holding the standard error fixed. This is a sensitivity scenario, not an estimated correction. Calipers use matching seed 42; each may retain different customers.
Source and assumptions ↗The shared generator constructs coupled factual and no-benefit worlds and verifies the factual reconstruction before producing counterfactual output. The known full-window net-payment ATT is −$0.0112774/day. The estimate differs by about +$0.0010236/day and contains that truth in its interval.
All six outcome/channel comparisons contain their coupled truth, but they are correlated outcomes in one world, not a coverage study.
A separate Monte Carlo exercise uses 20 configured-benefit and 20 zero-benefit worlds, each with 4,000 customers and matching refitted. Both scenarios cover truth in 20/20 intervals; each Wilson 95% interval is approximately 83.9% to 100%. Zero of 20 null worlds rejects at 5%. This is too small to establish precise coverage or Type I error, and it is not full-sample validation.
The pipeline regenerates data, runs 14 regression tests, executes the analysis and assumption audit, builds the dashboard and preview, and validates artifact consistency. The manifest records completion, package versions, source/data hashes, and phase logs. Separate JavaScript checks exercise dashboard bindings and sensitivity updates.
The implementation checks support this simulation. They do not establish real-world identification. The next organizational study should randomize an appropriate offer or rollout and measure contribution profit before a launch recommendation.