GeoLift: Results and Interpretation
What results you get
A typical GeoLift run provides:
- quality status and rationale,
- lift estimates (absolute and relative),
- uncertainty bounds,
- method-level comparisons,
- report artifacts for review.
Read results in this order
- Quality status
- Method agreement
- Relative lift and confidence interval
- Power/MDE signals
- Plausibility
This order prevents overreacting to a single large point estimate.
Interpretation of status labels
Significant
Meaning:
- reliable evidence of intervention impact.
Action:
- consider controlled scale-up, budget reallocation, or geo expansion.
Inconclusive
Meaning:
- no method produced reliable significance, but run completed.
Action:
- refine intervention design, extend observation window, or adjust targeting.
- read the run diagnostic comment (
quality_rationale) before deciding next steps.
Underpowered
Meaning:
- study setup likely too weak to detect realistic effects.
Action:
- increase signal strength, include more history/geos, rerun.
Data Issues
Meaning:
- input quality problems limit trust in output.
Action:
- resolve data issues first; do not make rollout decisions from this run.
No Results
Meaning:
- technical/model execution failure.
Action:
- inspect run logs/dependencies and rerun.
Non-significant result comments (important)
When a run is Inconclusive or Underpowered, GeoLift should be read with its diagnostic comment, not by p-value alone.
Where to read it:
- run summary comments in the result details view
- narrative lines in the generated PDF summary
Common diagnostic comments you may see:
- No similar markets found for the selected test market(s)
Meaning: available controls do not resemble treatment markets enough for a reliable causal comparison. - Weak test-control similarity / poor balance
Meaning: controls exist, but they are not close enough on pre-period behavior. - Insufficient pre-period history
Meaning: not enough historical signal to build a stable counterfactual. - Underpowered design (MDE not reliable)
Meaning: current setup is too weak to detect realistic effects.
Recommended action when this appears:
- Revisit test and control market selection
- Extend pre-period history where possible
- Remove contaminated controls (spillover/promo overlap)
- Rerun and compare quality status + rationale before decisioning
Key terms
Relative Lift
- Percentage-style incremental change.
- Example:
0.07means roughly 7% incremental effect.
Total Lift
- Absolute incremental value in KPI units.
- Example: +18,000 units or +$120,000 revenue.
Confidence Interval
- Plausible range for true impact.
- Narrower ranges usually indicate more stable evidence.
p-value
- Strength of evidence against "no effect."
- Lower usually means stronger evidence, but should never be read alone.
MDE (Minimum Detectable Effect)
- Smallest effect size the current setup can reliably detect.
- If real effect is below MDE, a non-significant result is expected.
Power
- Probability of detecting a true effect if it exists.
- Low power means "absence of evidence" is not "evidence of absence."
How to turn results into decisions
Decision matrix
| Result pattern | Interpretation | Recommended move |
|---|---|---|
| Significant + strong agreement + acceptable uncertainty | Strong positive signal | Scale with guardrails |
| Significant + weak agreement | Possible signal with model sensitivity | Pilot expansion, monitor closely |
| Inconclusive + adequate power | Likely low impact | Re-evaluate intervention design |
| Inconclusive + low power | Design too weak | Improve setup, rerun |
| Underpowered | Insufficient detectability | Increase sample/scope/history |
| Data Issues / No Results | Unreliable run | Fix and rerun before decisions |
Reporting recommendations
Use this format in reviews:
- Objective and intervention context
- GeoLift status and confidence level
- Estimated impact range (not only point estimate)
- Risks and assumptions
- Recommended next action with owner and timeline
Avoid:
- "We saw lift, so let's scale" without uncertainty context
- sharing p-values without plain-language implications
- treating one run as universal truth for all market conditions
Practical examples of executive phrasing
Strong result
"The intervention shows reliable incremental impact with acceptable uncertainty across methods. Recommend phased scale-up to the next geo tier."
Mixed result
"Signals are directionally positive but method agreement is mixed. Recommend controlled extension and rerun before full rollout."
Weak setup
"Current run is underpowered, so this is not a conclusive test of impact. Recommend strengthening design and rerunning before investment decisions."
Output surfaces
Business users typically consume:
- summary/status from job details,
- PDF report for narrative + visuals,
- governance metrics for audit/review workflows.