Market Matching: Results and Interpretation
What results you get
A typical Market Matching run provides:
- overall quality score,
- ranked controls for each test market,
- per-control similarity scores,
- balance statistics,
- and report artifacts for review.
Read results in this order
- Overall quality assessment
- Test market -> control ranking
- Similarity levels
- Balance diagnostics
- Warnings/comments
This order helps prevent selecting controls based only on one metric.
Key output sections
Primary results
- One row per test market
- Lists selected control markets
- Includes similarity and balance summary
Quality assessment
- Overall quality score
- Per-test-market quality signals
- Balance audit details
Similarity chart
- Visual ranking of candidate controls
- Helps detect sharp quality drop-offs
Treatment drilldown
- Detailed per-control statistics for a specific test market
- Useful for final control acceptance decisions
How to interpret quality levels
High quality
Meaning:
- selected controls are close and stable enough for reliable downstream analysis.
Action:
- proceed with confidence to causal measurement.
Medium quality
Meaning:
- usable controls exist, but match quality is mixed.
Action:
- consider sensitivity rerun (different method/thresholds) before finalizing.
Low quality
Meaning:
- controls are weak or poorly balanced.
Action:
- revise test markets, improve data quality, and rerun.
Common diagnostic comments (important)
When quality is weak, read comments before deciding.
Where to read:
- run summary comments in result details
- narrative lines in the generated PDF summary
Comments you may see:
- No similar markets found for selected test market(s)
- Meaning: candidate controls do not match pre-period behavior closely enough.
- Weak similarity / poor balance
- Meaning: controls were found, but comparability is low.
- Insufficient history for stable matching
- Meaning: the pre-period window is too short/noisy for reliable pairing.
- Quality gate fallback applied
- Meaning: strict filters removed all candidates, so best-available fallback pair was kept.
Recommended follow-up:
- Revisit test market selection
- Increase historical window
- Tighten data quality checks
- Rerun with alternate matching method and compare results
Key terms
Similarity score
- Numeric measure of closeness between test and control markets.
- Higher is generally better.
Balance
- How comparable test and control markets are on key behavior features.
SMD (Standardized Mean Difference)
- Scale-free balance diagnostic.
- Lower absolute values indicate better balance.
Quality score
- Composite indicator summarizing match reliability.
- Should be read with warnings/comments, not alone.
Decision matrix
| Result pattern | Interpretation | Recommended move |
|---|---|---|
| High quality + strong balance | Reliable control set | Accept and proceed |
| Medium quality + mixed balance | Usable but uncertain | Run sensitivity checks |
| Low quality + weak balance | Unreliable controls | Redesign and rerun |
| No similar markets found | No credible controls | Re-scope test markets |
Reporting recommendations
Use this format in reviews:
- Objective and selected test markets
- Quality score and key warnings
- Final control selection rationale
- Risks and assumptions
- Next step (accept/refine/rerun)
Avoid:
- selecting controls from similarity alone,
- skipping balance diagnostics,
- ignoring warning comments in low-quality runs.
Output surfaces
Typical output surfaces:
- job details summary
- primary results table
- quality assessment panel
- similarity chart and drilldown
- PDF report and downloadable artifacts