The Only Metric That Matters: A Simple Guide

Most startups track too many numbers. Downloads, page views, sign-ups, revenue. But there's really only one question that matters: Are people actually using your product?

Josh Elman, who helped build Twitter, Facebook, and LinkedIn, explains why most metrics are distractions and how to focus on what actually matters.

Table of Contents

1. The Problem with Most Metrics
2. The Only Question That Matters
3. Understanding User Behavior
4. The Three Types of Users
5. How to Find Your Core Users
6. The State Diagram Method
7. What Makes Users Stick
8. Real Examples from Twitter
9. How to Track the Right Things
10. Common Metric Mistakes
11. Templates You Can Use

The Problem with Most Metrics

Most startups track the wrong things:

• Downloads: How many people installed your app
• Page views: How many times people looked at your website
• Sign-ups: How many people created accounts
• Revenue: How much money you're making

These numbers look good in presentations, but they don't tell you if your product is actually working.

The problem: You can have millions of downloads but zero real users.

The Only Question That Matters

There's really only one question you need to answer:

Are people using your product?

More specifically:

1. Are they using it how you expect? (performing the core action)
2. Are they using it at the frequency you expect? (coming back regularly)

Everything else is just noise.

Understanding User Behavior

To understand if your product is working, you need to look at user behavior, not just numbers.

Good behavior looks like:

• People come back regularly
• They use your product the way you designed it
• They get value from it
• They tell others about it

Bad behavior looks like:

• People try it once and never come back
• They use it differently than you expected
• They don't get value from it
• They don't recommend it to others

The Three Types of Users

Josh Elman found that users fall into three categories:

1. Cold Users

• Try your product once
• Never come back
• Don't get value from it
• These users don't matter for growth

2. Casual Users

• Use your product sometimes
• Might come back, might not
• Get some value but not consistently
• These users are unpredictable

3. Core Users

• Use your product regularly
• Come back consistently
• Get real value from it
• These users drive your growth

The goal: Turn as many users as possible into core users.

How to Find Your Core Users

Here's how to identify which users are core:

Step 1: Calculate Return Probability

Look at how likely someone is to return in the second month based on how many days they used your product in the first month.

Create a grid:

• X-axis: Days used in first month
• Y-axis: Probability of returning in second month

Step 2: Find the Breaking Point

Look for the point where users are 90% or more likely to return. This is your "core user threshold."

Example: If users who visit 8+ days in the first month are 90% likely to return, then 8 days is your threshold.

Step 3: Categorize Your Users

• Cold: Below the threshold, unlikely to return
• Casual: Near the threshold, might return
• Core: Above the threshold, highly likely to return

The State Diagram Method

Once you know your user types, map how users move between categories:

User State Transitions

• New → Cold: Most new users become cold
• New → Casual: Some become casual users
• New → Core: Few become core users
• Casual → Core: Casual users can become core
• Core → Casual: Core users can become casual (bad!)

What You Want to See

• High conversion from New to Core: Most new users should become core users quickly
• Low churn from Core: Core users should stay core
• Growth in Core users: Your core user base should grow over time

What Makes Users Stick

To understand what makes users stick, analyze what core users do differently:

Key Actions That Matter

Look for actions that predict whether someone becomes a core user:

Examples from Twitter:

• Following 30+ people
• Having 10+ followers
• Tweeting regularly
• Engaging with others' tweets

The pattern: Core users do specific actions that casual users don't do.

The "Aha Moment"

Find the moment when users understand your product's value:

• Twitter: When you follow enough people to see interesting content
• Facebook: When you connect with friends and see their updates
• LinkedIn: When you build a professional network

Real Examples from Twitter

Josh Elman shared specific examples from Twitter:

Example 1: Following People

• Users who follow 30+ people are much more likely to become core users
• This makes sense: you need content to see when you open Twitter

Example 2: Getting Followers

• Users with 10+ followers are more likely to stick around
• This makes sense: you need an audience to feel like tweeting matters

The Pattern

Core users do things that make the product more valuable to them. Casual users don't.

How to Track the Right Things

Instead of tracking vanity metrics, track these:

1. Core User Conversion Rate

• What percentage of new users become core users?
• This is your most important metric

2. Core User Retention

• How many core users stay core users?
• This tells you if your product is sustainable

3. Core User Growth

• Is your core user base growing?
• This tells you if your product is scaling

4. Time to Core

• How long does it take for users to become core?
• Faster is better

Common Metric Mistakes

Here are mistakes to avoid:

1. Tracking Vanity Metrics

• Downloads, page views, sign-ups
• These don't tell you if your product is working

2. Comparing to Benchmarks

• "Our retention is better than average"
• Every product is different

3. Focusing on Growth Over Engagement

• Getting more users who don't use your product
• Better to have fewer engaged users

4. Not Understanding User Behavior

• Not knowing why users stick around
• Not knowing why users leave

Templates You Can Use

Core User Analysis Template

Step 1: Calculate Return Probability

Days used in Month 1 → Probability of returning in Month 2
1 day → 20%
2 days → 35%
3 days → 50%
...
8 days → 90%

Step 2: Find Your Threshold

• Look for 90%+ return probability
• This is your core user threshold

Step 3: Categorize Users

• Below threshold = Cold
• Near threshold = Casual
• Above threshold = Core

User State Tracking Template

Track how users move between states:

Weekly Core User Report

Track these every week:

1. Core User Count: How many core users do you have?
2. New to Core Rate: What % of new users become core?
3. Core User Retention: What % of core users stay core?
4. Time to Core: How long does it take to become core?

Core User Actions Checklist

For each product, identify:

• What actions do core users take?
• What's the minimum threshold for each action?
• How can you help new users reach these thresholds?
• What happens if users don't reach these thresholds?

The Bottom Line

Most metrics are distractions. The only thing that matters is whether people are actually using your product.

Focus on:

• Core user conversion rate
• Core user retention
• Core user growth
• What makes users stick

Ignore:

• Vanity metrics like downloads
• Benchmarks from other companies
• Metrics that don't predict behavior

Remember: It's better to have 1,000 core users than 100,000 users who never come back.

Key Takeaways

1. Focus on usage, not vanity metrics
2. Identify your core users and what makes them stick
3. Track core user conversion, retention, and growth
4. Help new users become core users quickly
5. Understand why users leave and fix those problems

The only metric that matters: Are people actually using your product?

This article is based on insights from Josh Elman's "The Only Metric That Matters" talk at Mind The Product conference, drawing from his experience at Twitter, Facebook, and LinkedIn.

Sources and further reading:

• Josh Elman's original article: The Only Metric That Matters
• Josh Elman's Medium profile: @joshelman
• Mind The Product conference: mindtheproduct.com
• Lean Analytics by Alistair Croll and Benjamin Yoskovitz
• The Lean Startup by Eric Ries