GeoLift

Overview

GeoLift: Overview

What is GeoLift?

GeoLift is a business measurement framework that answers one core question:

"How much incremental business impact came from our intervention?"

An intervention can be:

• a campaign launch,
• a pricing change,
• a promotion,
• or a regional roll-out.

GeoLift compares:

• treatment markets (where the intervention happened), and
• control markets (similar markets where it did not happen),

then estimates what would likely have happened without the intervention.

---

Why GeoLift matters for business

GeoLift helps teams make better decisions on:

• budget allocation,
• rollout vs pause decisions,
• market prioritization,
• and confidence in observed performance changes.

Instead of relying only on raw before/after trends, GeoLift separates:

• true intervention impact, and
• normal market movement/noise.

---

Core business capabilities

1) Incrementality measurement

• Estimates total lift and relative lift (%)
• Quantifies impact with confidence bounds
• Supports better ROI and scale decisions

2) Multi-model reliability

GeoLift in AIGenie evaluates impact using multiple methods:

• ANCOVA
• Synthetic DiD
• BSTS

Running multiple methods reduces dependency on a single modeling assumption.

3) Automated quality guardrails

The system includes:

• data validation checks,
• control-quality checks,
• power and detectability checks,
• and final quality status labels.

This helps prevent overconfident decisions from weak study setups.

4) Business-readable outputs

GeoLift outputs are designed for non-technical review:

• PDF summary for leadership consumption,
• compact quality status (Significant, Inconclusive, etc.),
• and standardized run metrics for audit and comparison.

---

Where GeoLift fits in the product experience

GeoLift is a structured measurement workflow with:

• job submission,
• background execution,
• artifact generation,
• and decision-focused interpretation.

---

Related documentation

• Methodology — how the pipeline works technically (geolift-methodology)
• Models — ANCOVA, Synthetic DiD, BSTS explained (geolift-models)
• Workflow — step-by-step process (geolift-workflow)
• Results — how to read and interpret outputs (geolift-results)

---

Business language for outcomes

When presenting GeoLift internally:

• Significant -> "We have reliable evidence the intervention moved outcomes."
• Inconclusive -> "No clear effect yet; signal may be small or noisy."
• Underpowered -> "Study setup is too weak for a definitive read."
• Data Issues -> "Input quality must be fixed before trusting results."
• No Results -> "Technical completion issue; rerun needed."

---

Typical stakeholders

• Growth and marketing leaders
• Category and merchandising teams
• Finance and planning teams
• Strategy and analytics teams
• Product/experimentation owners

---

Typical decisions GeoLift supports

• "Should we scale this intervention nationally?"
• "Which geos should be prioritized next?"
• "Do we continue, redesign, or stop this campaign?"
• "Is observed growth likely causal or coincidental?"

---

Quick value summary

GeoLift provides:

• better confidence in intervention decisions,
• clearer risk framing through uncertainty and quality signals,
• and stronger governance than simple trend reading.

Models

GeoLift: Models

Overview

GeoLift uses three complementary causal inference models. Running all three (default "geolift" / "all" mode) gives stronger confidence than relying on any single method.

Model	Approach	Best for
ANCOVA	Regression-based difference-in-differences	Clear pre/post comparison with parallel trends
Synthetic DiD	Weighted synthetic counterfactual	Building a close pre-period match from multiple controls
BSTS	Bayesian structural time series	Trend + seasonality forecasting with uncertainty bands

All three produce the same core outputs: total lift, relative lift, confidence interval, and p-value.

---

1) ANCOVA (Difference-in-Differences)

How it works

ANCOVA compares how the treatment market changed before vs after the intervention, relative to how control markets changed over the same periods.

It uses a regression with:

• treatment indicator,
• post-intervention indicator,
• and their interaction (the lift estimate).

Strengths

• Interpretable and widely used in experimentation
• Explicit parallel-trends check in the pre-period
• Control markets can be weighted by similarity

Safeguards

• Flags when pre-period trends diverge between test and controls
• Uses cluster-robust standard errors for geo-level data

Plain-language summary

"Did the treatment market grow more than controls after the campaign started, beyond what we'd expect from normal movement?"

---

2) Synthetic DiD (Synthetic Difference-in-Differences)

How it works

Synthetic DiD builds a synthetic version of the treatment market by blending multiple control markets. Weights are optimized so the synthetic market tracks the treatment geo closely in the pre-period.

After treatment starts:

• observed treatment performance is compared to the synthetic counterfactual
• the gap is the estimated lift

Strengths

• Can create a very tight pre-period fit by combining several controls
• Handles cases where no single control is a perfect match
• Provides a counterfactual series for visualization

Safeguards

• Pre-period fit reliability check
• Bootstrap resampling for confidence intervals
• Plausibility guard on extreme relative lift values

Plain-language summary

"If we blend the best control markets into one synthetic twin of our test market, how much did the real market outperform that twin after launch?"

---

3) BSTS (Bayesian Structural Time Series)

How it works

BSTS models the treatment market's time series using:

• local trend,
• weekly seasonality,
• and control market signals.

It forecasts what the treatment geo would have looked like without the intervention, then measures the post-period gap.

Strengths

• Naturally handles trend and seasonal structure
• Produces probabilistic uncertainty through simulation
• Strong when pre-period patterns are smooth and seasonal

Safeguards

• Bootstrap-based confidence intervals
• Pre-period fit metrics (correlation, MAE, RMSE)

Plain-language summary

"Based on pre-campaign patterns and control signals, what would sales have been without the intervention — and how far off was reality?"

---

Why run all three?

Reason	Explanation
Robustness	Different assumptions reduce single-model bias
Agreement check	If all three point the same way, confidence is higher
Diagnostic signal	Strong disagreement suggests weak controls or data issues
Default in AIGenie	"geolift" maps to "all" — all three run automatically

---

How to read model results together

1. Check if methods agree on direction (positive vs negative lift)
2. Compare magnitude — large gaps between methods warrant caution
3. Read confidence intervals — wide bands mean high uncertainty
4. Combine with quality status — significance alone is not enough

Agreement patterns

Pattern	What it suggests
All three significant, same direction	Strong signal
Two of three significant	Moderate signal — review the outlier
None significant, adequate power	Likely small or no effect
Methods disagree on sign	Review controls and data quality

---

Model selection (when not using default)

You can run individual methods instead of all three:

Method key	When to use
ancova	Standard DiD-style analysis
synthetic_did	When you want a synthetic counterfactual with visual trace
bsts	When trend/seasonality structure is important
all / geolift	Recommended default — runs all three

---

Key outputs per model

Each method returns:

Output	What it means
total_lift	Absolute incremental impact in outcome units
relative_lift	Proportional impact (0.08 = 8%)
confidence_interval	Range of plausible true effects
p_value	Statistical evidence against zero effect
mde	Smallest detectable effect at target power
model_fit_metrics	Pre-period fit quality (varies by method)

Synthetic DiD additionally provides a counterfactual time series for charting.

---

For the full pipeline sequence, see geolift-methodology. For interpreting final outputs, see geolift-results.

Workflow

GeoLift: Workflow

Workflow at a glance

1. Define intervention and markets
2. Prepare and validate data
3. Submit GeoLift run
4. System performs causal analysis
5. Review quality and impact outputs
6. Make a decision

---

Step 1: Define the question

Before running GeoLift, align on:

• intervention type (campaign, promo, pricing, etc.),
• treatment markets,
• treatment start date,
• primary KPI (units, revenue, conversions, etc.).

Good prompt for internal alignment: "What decision are we making with this result?"

---

Step 2: Prepare data inputs

Minimum practical expectations:

• clear date column,
• clear geo/location column,
• clear outcome metric,
• enough pre- and post-intervention history.

Best practice:

• keep metric definition stable through the study,
• avoid mixing structural breaks (major assortment/logging changes),
• ensure controls are not exposed to treatment spillover.

---

Step 3: Submit GeoLift job

GeoLift jobs are submitted and processed asynchronously:

• request is created,
• background worker executes analysis,
• run artifacts are generated.

In implementation terms, the platform routes this through causal inference job infrastructure (jobs.causal_inference).

Input use cases

GeoLift commonly runs in two input modes:

1. Test and control both provided

• You provide test_locations and control_locations.
• System runs causal estimation on the specified pair(s).

2. Test provided, control not provided

• You provide test_locations only.
• System derives/ranks controls from remaining candidate markets before running inference.

---

Step 4: Automated analysis sequence

Behind the scenes, GeoLift runs the following sequence:

1. Data mapping and transformation
2. Dataset validation and balancing
3. Treatment-date resolution
4. Pre-treatment market characterization
5. Test/control comparison setup
6. Multi-model impact estimation
7. Power and quality evaluation
8. Report + metrics publication

This sequence is built to protect teams from acting on weak or misleading runs.

---

Step 5: Outputs become available

After completion, teams can access:

• executive-style PDF report,
• run status and quality labels,
• per-method impact metrics,
• standardized output data for governance/review.

---

Step 6: Decision checkpoint

Use the output to choose one of these paths:

• Scale: strong evidence and acceptable risk
• Refine and re-test: mixed signals or weak controls
• Hold: no reliable impact evidence
• Fix and rerun: data quality or technical completion issue

---

Suggested operating cadence

For recurring interventions:

• run GeoLift on a planned cadence,
• compare run quality over time,
• track repeated learnings by intervention type/geo cluster.

This builds a repeatable evidence engine, not one-off analysis.

---

Pre-run checklist

• Intervention and KPI are clearly defined
• Treatment markets are finalized
• Controls are likely untreated
• Treatment date is accurate
• History window is sufficient
• Decision owner is identified

---

Post-run checklist

• Quality status reviewed first
• Effect size and uncertainty reviewed together
• Model agreement checked
• Practical significance assessed
• Explicit action (scale/refine/hold) documented

---

Common workflow pitfalls

• Running without a clear decision question
• Treating "not significant" as "no impact" without checking power
• Ignoring data issues and using results anyway
• Scaling based on point estimate alone
• Comparing runs with different KPI definitions

Methodology

GeoLift: Methodology

How GeoLift works technically

GeoLift measures incremental impact by building a counterfactual: what the treatment market would likely have done without the intervention. The gap between actual performance and that counterfactual is the estimated lift.

The full process has five stages:

1. Prepare data — map columns, clean the panel, validate quality
2. Characterize markets — learn pre-treatment behavior per geo
3. Set up test vs control — select or rank comparable control markets
4. Run causal models — estimate lift with one or more inference methods
5. Score reliability — power analysis, quality status, and report generation

Default GeoLift runs use all three models (ANCOVA, Synthetic DiD, BSTS) so results are not dependent on a single approach.

---

Stage 1: Data preparation

Column mapping

The pipeline auto-detects:

• date (week/day/period),
• location (market, store, DMA, etc.),
• outcome (sales, revenue, units, or another KPI).

You can also specify the outcome column explicitly when submitting a job.

Panel balancing

• Duplicate rows are aggregated
• Time index is normalized (weekly alignment where applicable)
• Missing values are handled so trends stay stable
• Geos with too much missing history may be dropped

Why it matters: sparse or inconsistent panels produce unreliable lift estimates.

Validation

Checks include:

• minimum number of geos and time periods,
• missingness and sparsity,
• structural readiness for causal modeling.

Validation issues feed into the final quality score.

---

Stage 2: Pre-treatment market characterization

Before the intervention date, each geo gets a fingerprint summarizing historical behavior:

• average outcome level,
• volatility,
• trend direction,
• seasonality-like patterns,
• stability signals.

Fingerprints are computed on pre-treatment data only so post-campaign changes do not bias control selection.

---

Stage 3: Test and control setup

GeoLift supports two input modes:

Mode	What you provide	What the system does
Explicit	Test + control markets	Runs inference on your specified pair(s)
Test only	Test market(s) only	Ranks and selects best controls from remaining geos

Controls are sorted by similarity to the treatment geo. Top matches are used for inference (typically capped for performance).

---

Stage 4: Causal inference

Each model estimates lift differently but answers the same question: how much extra outcome came from the intervention?

See geolift-models for a detailed breakdown of ANCOVA, Synthetic DiD, and BSTS.

Common outputs per method:

• total lift (absolute),
• relative lift (%),
• confidence interval,
• p-value,
• model fit diagnostics.

---

Stage 5: Power and quality scoring

Power analysis

Answers: "Could this study reliably detect an effect of realistic size?"

Key outputs:

• MDE — smallest lift the design can detect at target power
• power curve — detectability across effect sizes

If MDE cannot be computed, the run is treated as underpowered.

Quality status

Status	Meaning
Significant	At least one method found a reliable effect
Inconclusive	Models ran but no method reached significance
Underpowered	Design too weak for confident detection
Data Issues	Input quality problems dominate
No Results	Models failed to produce output

Quality combines significance evidence, effect size, and data-issue penalties.

---

Data requirements

Requirement	Practical guidance
History	Aim for 10+ pre-treatment weeks where possible
Post window	Enough weeks after intervention to observe effect
Geos	At least 1 treatment + 2 controls after cleaning
Frequency	Weekly time series is the typical grain
Outcome	Single stable KPI (sales, revenue, units, etc.)

---

Automated safeguards

• Parallel-trend checks (ANCOVA)
• Pre-period fit reliability (Synthetic DiD)
• Bootstrap/simulation-based confidence intervals
• Plausibility guards on extreme lift values
• Multi-method agreement as a sanity check

---

What can go wrong

• Wrong column mapped as outcome or location
• Treatment geo dropped due to missing data
• Weak or contaminated controls
• Methods disagree strongly on direction or magnitude
• Underpowered design mistaken for "no effect"

For interpretation guidance, see geolift-results.

Results

GeoLift: Results and Interpretation

What results you get

A typical GeoLift run provides:

• quality status and rationale,
• lift estimates (absolute and relative),
• uncertainty bounds,
• method-level comparisons,
• report artifacts for review.

---

Read results in this order

1. Quality status
2. Method agreement
3. Relative lift and confidence interval
4. Power/MDE signals
5. Plausibility

This order prevents overreacting to a single large point estimate.

---

Interpretation of status labels

Significant

Meaning:

• reliable evidence of intervention impact.

Action:

• consider controlled scale-up, budget reallocation, or geo expansion.

Inconclusive

Meaning:

• no method produced reliable significance, but run completed.

Action:

• refine intervention design, extend observation window, or adjust targeting.
• read the run diagnostic comment (quality_rationale) before deciding next steps.

Underpowered

Meaning:

• study setup likely too weak to detect realistic effects.

Action:

• increase signal strength, include more history/geos, rerun.

Data Issues

Meaning:

• input quality problems limit trust in output.

Action:

• resolve data issues first; do not make rollout decisions from this run.

No Results

Meaning:

• technical/model execution failure.

Action:

• inspect run logs/dependencies and rerun.

---

Non-significant result comments (important)

When a run is Inconclusive or Underpowered, GeoLift should be read with its diagnostic comment, not by p-value alone.

Where to read it:

• run summary comments in the result details view
• narrative lines in the generated PDF summary

Common diagnostic comments you may see:

• No similar markets found for the selected test market(s)
  Meaning: available controls do not resemble treatment markets enough for a reliable causal comparison.
• Weak test-control similarity / poor balance
  Meaning: controls exist, but they are not close enough on pre-period behavior.
• Insufficient pre-period history
  Meaning: not enough historical signal to build a stable counterfactual.
• Underpowered design (MDE not reliable)
  Meaning: current setup is too weak to detect realistic effects.

Recommended action when this appears:

1. Revisit test and control market selection
2. Extend pre-period history where possible
3. Remove contaminated controls (spillover/promo overlap)
4. Rerun and compare quality status + rationale before decisioning

---

Key terms

Relative Lift

• Percentage-style incremental change.
• Example: 0.07 means roughly 7% incremental effect.

Total Lift

• Absolute incremental value in KPI units.
• Example: +18,000 units or +$120,000 revenue.

Confidence Interval

• Plausible range for true impact.
• Narrower ranges usually indicate more stable evidence.

p-value

• Strength of evidence against "no effect."
• Lower usually means stronger evidence, but should never be read alone.

MDE (Minimum Detectable Effect)

• Smallest effect size the current setup can reliably detect.
• If real effect is below MDE, a non-significant result is expected.

Power

• Probability of detecting a true effect if it exists.
• Low power means "absence of evidence" is not "evidence of absence."

---

How to turn results into decisions

Decision matrix

Result pattern	Interpretation	Recommended move
Significant + strong agreement + acceptable uncertainty	Strong positive signal	Scale with guardrails
Significant + weak agreement	Possible signal with model sensitivity	Pilot expansion, monitor closely
Inconclusive + adequate power	Likely low impact	Re-evaluate intervention design
Inconclusive + low power	Design too weak	Improve setup, rerun
Underpowered	Insufficient detectability	Increase sample/scope/history
Data Issues / No Results	Unreliable run	Fix and rerun before decisions

---

Reporting recommendations

Use this format in reviews:

1. Objective and intervention context
2. GeoLift status and confidence level
3. Estimated impact range (not only point estimate)
4. Risks and assumptions
5. Recommended next action with owner and timeline

Avoid:

• "We saw lift, so let's scale" without uncertainty context
• sharing p-values without plain-language implications
• treating one run as universal truth for all market conditions

---

Practical examples of executive phrasing

Strong result

"The intervention shows reliable incremental impact with acceptable uncertainty across methods. Recommend phased scale-up to the next geo tier."

Mixed result

"Signals are directionally positive but method agreement is mixed. Recommend controlled extension and rerun before full rollout."

Weak setup

"Current run is underpowered, so this is not a conclusive test of impact. Recommend strengthening design and rerunning before investment decisions."

---

Output surfaces

Business users typically consume:

• summary/status from job details,
• PDF report for narrative + visuals,
• governance metrics for audit/review workflows.