GeoLift: Models

Overview

GeoLift uses three complementary causal inference models. Running all three (default "geolift" / "all" mode) gives stronger confidence than relying on any single method.

Model	Approach	Best for
ANCOVA	Regression-based difference-in-differences	Clear pre/post comparison with parallel trends
Synthetic DiD	Weighted synthetic counterfactual	Building a close pre-period match from multiple controls
BSTS	Bayesian structural time series	Trend + seasonality forecasting with uncertainty bands

All three produce the same core outputs: total lift, relative lift, confidence interval, and p-value.

---

1) ANCOVA (Difference-in-Differences)

How it works

ANCOVA compares how the treatment market changed before vs after the intervention, relative to how control markets changed over the same periods.

It uses a regression with:

• treatment indicator,
• post-intervention indicator,
• and their interaction (the lift estimate).

Strengths

• Interpretable and widely used in experimentation
• Explicit parallel-trends check in the pre-period
• Control markets can be weighted by similarity

Safeguards

• Flags when pre-period trends diverge between test and controls
• Uses cluster-robust standard errors for geo-level data

Plain-language summary

"Did the treatment market grow more than controls after the campaign started, beyond what we'd expect from normal movement?"

---

2) Synthetic DiD (Synthetic Difference-in-Differences)

How it works

Synthetic DiD builds a synthetic version of the treatment market by blending multiple control markets. Weights are optimized so the synthetic market tracks the treatment geo closely in the pre-period.

After treatment starts:

• observed treatment performance is compared to the synthetic counterfactual
• the gap is the estimated lift

Strengths

• Can create a very tight pre-period fit by combining several controls
• Handles cases where no single control is a perfect match
• Provides a counterfactual series for visualization

Safeguards

• Pre-period fit reliability check
• Bootstrap resampling for confidence intervals
• Plausibility guard on extreme relative lift values

Plain-language summary

"If we blend the best control markets into one synthetic twin of our test market, how much did the real market outperform that twin after launch?"

---

3) BSTS (Bayesian Structural Time Series)

How it works

BSTS models the treatment market's time series using:

• local trend,
• weekly seasonality,
• and control market signals.

It forecasts what the treatment geo would have looked like without the intervention, then measures the post-period gap.

Strengths

• Naturally handles trend and seasonal structure
• Produces probabilistic uncertainty through simulation
• Strong when pre-period patterns are smooth and seasonal

Safeguards

• Bootstrap-based confidence intervals
• Pre-period fit metrics (correlation, MAE, RMSE)

Plain-language summary

"Based on pre-campaign patterns and control signals, what would sales have been without the intervention — and how far off was reality?"

---

Why run all three?

Reason	Explanation
Robustness	Different assumptions reduce single-model bias
Agreement check	If all three point the same way, confidence is higher
Diagnostic signal	Strong disagreement suggests weak controls or data issues
Default in AIGenie	"geolift" maps to "all" — all three run automatically

---

How to read model results together

1. Check if methods agree on direction (positive vs negative lift)
2. Compare magnitude — large gaps between methods warrant caution
3. Read confidence intervals — wide bands mean high uncertainty
4. Combine with quality status — significance alone is not enough

Agreement patterns

Pattern	What it suggests
All three significant, same direction	Strong signal
Two of three significant	Moderate signal — review the outlier
None significant, adequate power	Likely small or no effect
Methods disagree on sign	Review controls and data quality

---

Model selection (when not using default)

You can run individual methods instead of all three:

Method key	When to use
ancova	Standard DiD-style analysis
synthetic_did	When you want a synthetic counterfactual with visual trace
bsts	When trend/seasonality structure is important
all / geolift	Recommended default — runs all three

---

Key outputs per model

Each method returns:

Output	What it means
total_lift	Absolute incremental impact in outcome units
relative_lift	Proportional impact (0.08 = 8%)
confidence_interval	Range of plausible true effects
p_value	Statistical evidence against zero effect
mde	Smallest detectable effect at target power
model_fit_metrics	Pre-period fit quality (varies by method)

Synthetic DiD additionally provides a counterfactual time series for charting.

---

For the full pipeline sequence, see geolift-methodology. For interpreting final outputs, see geolift-results.