Dealing with Zeros: Zero-Inflated and Two-Part Models

TL;DR

Many product metrics have lots of zeros: revenue (non-purchasers), engagement (inactive users), support tickets (no issues). Standard methods struggle because zeros represent a distinct population, not just small values. Two-part models split the problem: model probability of any positive value, then model the amount given positive. Zero-inflated models treat some zeros as "structural" (will never be positive) and others as "sampling" (could be positive but weren't). Often, simpler approaches work: analyze positives separately, or use robust methods that handle the variance.

Why Zeros Are Different

Two Types of Zeros

Structural zeros: User fundamentally won't have this behavior

• Free users who can't purchase (paywall)
• Users in regions without the feature
• Bot accounts (no real engagement)

Sampling zeros: User could have the behavior but didn't in this period

• Purchaser who didn't buy this week
• Active user who didn't engage today
• Customer who had no support issues

Why This Matters

Standard models (linear regression, t-tests) treat all zeros the same—as small values. But:

• Structural zeros shouldn't be "converted" by treatment
• Combining populations inflates variance
• Mean is pulled toward zero, masking effects among actives

The Math Problem

For revenue with 80% zeros:

• Mean = 0.2 × (mean among buyers)
• Variance is huge (zero vs. non-zero dominates)
• Standard error is enormous → low power

Approaches to Zero-Heavy Data

Approach 1: Analyze Separately

Split into two analyses:

1. Conversion: Did they have any positive value? (logistic regression)
2. Amount: Among positives, what was the value? (standard methods)

import numpy as np
import pandas as pd
from scipy import stats
import statsmodels.api as sm


def separate_analysis(data, value_col, group_col):
    """
    Analyze zeros and positives separately.
    """
    results = {}

    # Part 1: Conversion (any positive value)
    data['is_positive'] = (data[value_col] > 0).astype(int)

    groups = sorted(data[group_col].unique())
    conv_control = data.loc[data[group_col] == groups[0], 'is_positive']
    conv_treatment = data.loc[data[group_col] == groups[1], 'is_positive']

    # Z-test for proportions
    n_c, n_t = len(conv_control), len(conv_treatment)
    p_c, p_t = conv_control.mean(), conv_treatment.mean()

    p_pooled = (conv_control.sum() + conv_treatment.sum()) / (n_c + n_t)
    se = np.sqrt(p_pooled * (1 - p_pooled) * (1/n_c + 1/n_t))
    z_stat = (p_t - p_c) / se
    p_value_conv = 2 * (1 - stats.norm.cdf(abs(z_stat)))

    results['conversion'] = {
        'control_rate': p_c,
        'treatment_rate': p_t,
        'lift': (p_t - p_c) / p_c if p_c > 0 else np.inf,
        'p_value': p_value_conv
    }

    # Part 2: Amount among positives
    pos_control = data.loc[(data[group_col] == groups[0]) & (data[value_col] > 0), value_col]
    pos_treatment = data.loc[(data[group_col] == groups[1]) & (data[value_col] > 0), value_col]

    if len(pos_control) > 1 and len(pos_treatment) > 1:
        t_stat, p_value_amount = stats.ttest_ind(pos_control, pos_treatment)

        results['amount_given_positive'] = {
            'control_mean': pos_control.mean(),
            'treatment_mean': pos_treatment.mean(),
            'lift': (pos_treatment.mean() - pos_control.mean()) / pos_control.mean(),
            'p_value': p_value_amount
        }

    # Combined effect (for context)
    all_control = data.loc[data[group_col] == groups[0], value_col]
    all_treatment = data.loc[data[group_col] == groups[1], value_col]

    results['overall'] = {
        'control_mean': all_control.mean(),
        'treatment_mean': all_treatment.mean(),
        'lift': (all_treatment.mean() - all_control.mean()) / all_control.mean() if all_control.mean() > 0 else np.inf
    }

    return results


# Example
np.random.seed(42)
n = 5000

# Simulate revenue data
data = pd.DataFrame({
    'group': np.random.choice(['control', 'treatment'], n),
    'revenue': np.where(
        np.random.binomial(1, 0.2, n),  # 20% purchasers
        np.random.lognormal(3, 1, n),    # Revenue among purchasers
        0
    )
})

# Treatment increases conversion slightly
treatment_mask = data['group'] == 'treatment'
data.loc[treatment_mask, 'revenue'] = np.where(
    np.random.binomial(1, 0.22, treatment_mask.sum()),  # 22% conversion
    np.random.lognormal(3, 1, treatment_mask.sum()),
    0
)

results = separate_analysis(data, 'revenue', 'group')

print("Separate Analysis Results")
print("=" * 50)
print(f"\nConversion (any purchase):")
print(f"  Control: {results['conversion']['control_rate']:.1%}")
print(f"  Treatment: {results['conversion']['treatment_rate']:.1%}")
print(f"  Lift: {results['conversion']['lift']:.1%}")
print(f"  p-value: {results['conversion']['p_value']:.4f}")

if 'amount_given_positive' in results:
    print(f"\nAmount (among purchasers):")
    print(f"  Control: ${results['amount_given_positive']['control_mean']:.2f}")
    print(f"  Treatment: ${results['amount_given_positive']['treatment_mean']:.2f}")
    print(f"  Lift: {results['amount_given_positive']['lift']:.1%}")
    print(f"  p-value: {results['amount_given_positive']['p_value']:.4f}")

print(f"\nOverall (all users):")
print(f"  Control: ${results['overall']['control_mean']:.2f}")
print(f"  Treatment: ${results['overall']['treatment_mean']:.2f}")
print(f"  Lift: {results['overall']['lift']:.1%}")

Approach 2: Two-Part (Hurdle) Model

Formally model both parts together: