Back to Articles
PRODUCT SCIENCE AUG 2023 6 MIN READ

The $114M Friction Fix: Employment Verification Optimization

How causal inference and surgical A/B testing challenged a core risk assumption, removing friction for high-quality borrowers and unlocking $114M in annualized volume.

Context

At a major financial product company, the loan origination flow is a balance between risk mitigation and conversion. Crucially, Employment Verification (EV) happened at the very end of the funnel—after the user had already invested time and effort.

The Problem

We were seeing massive drop-offs at this comprehensive final step.

The Hypothesis: These weren't just "lazy" users. They were high-intent applicants who, faced with a friction-heavy verification step, were likely taking offers from competitors who approved them faster.

The bank assumed EV was a necessary shield. But if the shield isn't stopping bad actors—only slowing down good ones—it's not a shield; it's a leak.

The Solution

Part 1: Causal Discovery

Before suggesting we remove a risk control, I needed proof. I applied causal inference techniques (using XGBoost feature importance and correlation analysis) on historical loan performance data.

The finding was stark: For applicants with FICO > 720, the Employment Verification status had zero correlation with default rates (p > 0.05). The signal was noise.

Part 2: Stratified Randomization Test

We couldn't just turn it off globally (too risky). I designed a stratified randomization A/B test. We isolated the "High FICO / Low Risk" segment and split them:

  • Control: Standard flow (EV Required)
  • Treatment: Frictionless flow (EV Skipped)
Used presplit methodology to ensure statistical power with a 30% smaller sample size to minimize risk exposure during the test.

Results

  • ➜ Conversion: Treatment group saw a massive lift, capturing 300+ additional customers monthly.
  • ➜ Risk: No statistically significant increase in early payment defaults (EPD) in the treatment group.
  • ➜ Impact: $50M in realized loan volume during rollout, with a $114M annualized projection.

Technologies

Python • XGBoost • Causal Inference • A/B Testing • SQL

Lessons

  1. Rigorous testing challenges "obvious" assumptions. "More verification = Less risk" seems intuitive, but data proved it wrong for specific segments.
  2. Stratified randomization is surgical. It allowed us to innovate in a high-risk environment without exposing the entire portfolio.
  3. Translate statistics into operations. The XGBoost insight wasn't just a chart; it became a decision tree rule that automatically routed users.

Get More Case Studies Like This

Deep dives into experimentation, causal inference, and AI systems.
2 emails/month max. No spam, unsubscribe anytime.