Subscription Metrics 101: How to Set Up a Basic Metrics Hierarchy

I would argue that 99% of companies that are really good at developing tech products do these three things:

1. They have clearly defined metrics that they are trying to move
2. They review those metrics regularly
3. They try to determine the link between the work that they release and how those metrics move.

Let me also acknowledge that I have never seen a company that has 100% of its tracking set up correctly.

Something is always broken, out of date, or missing. It’s just the reality of tech products, unfortunately.

If this topic fills you with dread, it’s likely that your tracking infrastructure is broken somewhere.  That’s fine, literally everyone’s is to some degree.

That said, every subscription company should have a basic hierarchy of metrics, understand what those metrics mean and try to improve them.

Why are Metrics Important

Regardless of the type of product that you run, there are 4 core things that happen in product development:

1. Deciding where to focus
2. Deciding what exactly to ship
3. Measuring the impact
4. Using this to inform what to do next

Needless to say, if you aren’t measuring the impact of your work, you can’t move to step 3 or 4 and just play whack-a-mole with new ideas and bugs.

Very early on in a startup's development, product development is guided by the instincts of the founders.

The founders have some sort of unique insight in the market and they use this insight to guide product development and ship a product. They are probably only looking at user sign ups, revenue and trying to move as fast as they can.

Over time, however, as the company grows larger, the founders need to spend more of their focus building the company and the market evolves, this instinctive style of development stops being as effective.

Companies at this point need to switch to a more structured approach, where they have product teams who set goal metrics and then try to ship projects to move these goals.

If you don’t have a clear hierarchy of the metrics that govern your product, it’s too easy to change your goals every quarter or let teams set goals that don’t scale up to the company’s goals.

The biggest danger that I see in growing $1M - $5M ARR companies, is that they have something that works, they mistakenly break it and don’t realize it for many months.

Best case scenario is that they figure it out and just lose out on many months of growth. The Worst case is that this kills them.

What Subscription Products Should Be Measuring

This is a basic hierarchy of metrics that the majority of subscription products can use, which breaks into 3 core categories.

1. How well the product is attracting new users
2. How long they stay in the product
3. How they leave the product

Note here that I am only going to lightly touch on the core product metrics (activation, retention, etc) as those are highly dependent on the product that you’re creating.

I am going to focus instead on the “business” metrics and the revenue that the product produces.

I am going to break each group into:

• Primary metrics - these are your top-level metrics that are likely reported across your company and/or to your board. These are things like “Revenue” or “New User Sign Ups”
• Secondary metrics - sometimes called “operational metrics” these are the metrics that ladder up to drive your primary metrics. So if you were trying to improve your “New User Sign Ups” you might be working on your home page’s conversion rate.

1. How Well the Product is Attracting New Users

This bucket is effectively attracting new users to your product and getting them in the door.

Primary Metrics:

• Volume of New Sign-Ups - e.g how many people/companies are paying you for the first time this day/week/month, etc
• Revenue from New Sign-Ups - how much money you made from these sign-ups this day/week/month.
• Conversion Rate to Paid - How effectively the non-paid stuff converts people to paying customers.
• If this is a direct purchase product, this will the conversion rate of people landing on your site to paying users
• If have a free product, then this is the conversion rate between your free and paid products
• If you have a trial - This is the end-to-end conversion rate, meaning of everyone on your site, how many both start a trial and convert from that trial.

Secondary/Operational Metrics:

They start to vary from product to product, but I’ll list some ones here that are commonly useful.

• Trial Start Rate - this is the number of users that you have registered who see a trial and start it. It’s helpful to cut this metric by time window, so how many do it within 7, 14, 30, 60+ days of seeing your products
• Trial Conversion Rate - this is the conversion rate to paying within the trial experience, I would also cut this by the same time window as the trial start rate.
• Checkout Page Conversion Rate - Of everyone who lands on your checkout page, how many successfully pay you? You should segment this data by device size, country, and payment method
• Payment Button Conversion Rate - of all of the users who click “pay” on your checkout page, what % of these users actually successfully pay you? You want the highest conversion rate you can here. Segment this by payment method (credit cards, PayPal, etc)
• Plan Mix - If you have monthly and annual plans, what % of users choose each? Monitor this per month and try to increase the number of people in longer-term plans.

2. How Long Users Stay With the Product

This bucket is how well the product is retaining users. As you’re a subscription product, you always want to increase these numbers.

As noted in other ​articles​, many products have a natural “cap” on retention based on the problem they are solving for users. That said, you should try to figure out that limit and get retention to that point.

Primary Metrics:

• Lifetime Value - As noted in other ​posts​, this is a very important metric to know, however a very hard metric to use operationally. You want to be moving this number up, but you’ll likely have to define operation metrics to actually try to move this metric.
• Length of Usage Retention - How long are users coming back to the product to perform the core action, typically measured in weeks or months?
• Length of Payments Retention - How long are users paying for the product, also typically measured in weeks, months, or (if you’re lucky) years?

Ideally, you are looking at this data on a cohort level. So for every month of new paying users, you should see how many months they stay around.

Usage retention will likely drive payment retention, so when someone stops using something they will eventually stop paying.

It’s your job to figure out how to keep driving usage of the product.

You should look at both forms of retention both as a table and as a chart. They’ll look something like this.

Operational Metrics

• Payment retention - this is pretty simple, what % of the people who should pay you, do pay you
• Usage retention - is a lot more complex and I’ll go into it below.

Most of these metrics are governed by the core product that your problem is solving. This is a framework that I learned in Regorge’s “​Engagement and Retention​” course, which is great.

I can’t go nearly into the depth that this course can, but on a high level, you’re trying to measure the path that users have to “retained”, which means they will keep coming back to your product.

This path goes like this:

• Signed up: User joins the product
• Setup moment: User is prepared to experience the core value prop
• Aha moment: User has experienced the core value prop for the first time
• Habit moment: User has established the habit around the core value prop
• Retained moment: When the user is reliably coming back to the product

Improving engagement and retention is the core job of product teams and is essentially a never-ending process.

3. How They Leave the Product

Users will leave your product in a few ways and it’s important to measure cancellation according to these groups.

Before we get into metrics, there are two ways that users will leave your product

• Happily - I bought your product to solve a problem. That problem is now solved and I don’t need this anymore
• Unhappily - This product didn’t solve my problem and I am leaving.

In the example of language learning, if I was paying for a product like Babble to learn Spanish, I either

• Learned enough Spanish that I am happy and no longer need it
• Didn’t learn Spanish and am giving up

Within the “Unhappy” bucket, I would break this again into two additional buckets

• Intentional Cancellation - I went into settings and canceled my account
• Unintentional Cancellation - Something went wrong with payments, tech or another blocker that prevented me from using this service.

Primary Metrics

• Month Over Month Churn Rate - Of the users that you had last month, how many canceled. When most people talk about “churn” this is what they are talking about and this allows you to compare to benchmarks.
• Time Boxed Churn Rate - Of the users who signed up in x period, how many are left y months later. Typically this is 3, 6 or 12 months.

Operational Metrics

• Payment Processing Failure Rate ** Of all of users with recurring transactions, what % are failing their first payment attempt and reaching your retry process?
• Payment Failure Win Back Rate - Of all of the users that fail their initial payment, how many can you recover with revenue recovery strategies?
• Churn Appeasement Rate - Of all of the users that enter your cancellation flow, how many of them take an offer that is presented and don’t churn?
• “Happy” Cancellation Rate - How many users did this product work for and they no longer have the need for this product? Note, that to do this, you’ll need to add this option in your cancellation flow.

So what do you do with this information?

There is a lot in this post, so don’t get overwhelmed and try to implement everything at once.

1. Define Your Core Sources of Truth

As metrics always break, it helps to anchor to the sources of truth for the thing that you care about the most. These are going to be the key lifecycle events that are captured by other systems.

For the average subscription company, these are:

• Page views - for the core page of your site, like home, onboarding, checkout, etc
• Sign Ups - The free (if that exists) sign-ups, the best source of truth is typically email data
• Payments - Typically your payment system, such as Stripe/Recurly/Braintree/Paddle, etc
• Cancellations - Either the app itself or your subscription-manager

This is especially useful when your tracking data breaks or is suspect. It allows you to quickly hand calculate numbers like your free to paid conversion rate by taking your email sign-ups/ your new payments.

2. Validate Data for Primary Metrics

Pick the primary metrics that you want to track and ensure that whatever feeds your reporting systems roughly equals the SOT data.

These won’t be exactly equal, but they should be close.

So when you look at your free to paid conversion data, it should be roughly equal to hand calculated conversion rate.

Then, pick 1-2 operation metrics and do the same gut checks. You will likely find things that are broken and it’s time to start prioritizing things to fix.

3. Build Basic Dashboards & Monitor Daily

Even though most of these numbers won’t move daily, you’ll be able to start to draw a relationship between the projects that you see shipping and the way that numbers move.

You’ll also be able to build your intuition for the seasonality of your business.

The End

Typically when a company is not monitoring metrics, it is driven by one or more of these reasons

1. You have no tracking set up and/or it is too broken to be usable
2. You don’t have a self-service analytics tool
3. Your team doesn’t have the skills to look at numbers

Figure out which of these are your core problems and work from there.