- How To Measure Product Success
- Product Success Manager Job
- Customer Success Product Manager
- Product Success Definition
To be a successful product manager of a SaaS product, you need a good understanding of the fundamental metrics of the trade. From specific metrics that help you understand user engagement or the resonance of a new feature, to high level metrics that indicate the health of the overall business, understanding these fundamentals will make you a better product manager. In this post, we’ll cover 19 product management metrics across a handful of categories:
- Customer / user acquisition
- Trial-Customer Conversion Rate
- Retention Rate
- Churn Rate
- Product or user engagement
- DAU, WAU, MAU
- Cohort Retention
- Time in App
- Number of User Actions / Session
- User or customer satisfaction
- Support Tickets
- Feature resonance
- In app response
- Feature sentiment
- Roadmap Prioritization
- SaaS company metrics
- Cash Burn Rate
For the product to be a long term success, he needs to envision how the product, industry and competition is going to evolve and develop a long term roadmap. Passion for products Product managers. A good product manager perfectly understands the product, the industry, the customers, and the development team. This may be a lot of things to juggle at the same time, but that’s the thrill of the job.
Customer / User Acquisition
Trial-Customer Conversion Rate
The Customer Success Manager must establish policies the entire staff can adhere to so all customers receive the same quality of service. Provide Technical and Product Support. Customer Success Managers must know the company’s products inside and out. To be successful, product managers need a clear understanding of their roles and responsibilities. They also need a clear sense of the required skills and competencies. An appreciation of the roles, responsibilities, and requisite capabilities of this important position is also beneficial for stakeholders and team members who need to work closely with product managers.
What it is: Measures the rate at which free trial users convert to paying customers.
How to use it: Understanding your trial to customer conversion rate will help you forecast how many paying customers you’ll generate from a given number of trial users. Analyzing this rate will also help you understand how changes that you make to your trial experience impact the yield of paying customers. As with many other metrics, it can be valuable to compare the rate across different cohorts of users. You may also want to look at your trial as a funnel, and understand the conversion rates from stage to stage (e.g. account created, active user, engaged user, customer).
How to calculate it: # of paying customers / # of trial users (or a given cohort) = % customers converted
What it is: Retention rate is the percentage of users or customers that continue to be active, or continue to pay, from one period to the next.
How to use it: It is far more expensive to acquire new customers than to retain existing ones, so measuring retention will help you understand the health of your business. Retention rate can be used to predict the lifetime value of a customer and to understand the impact of changes on retention. Look at retention across different user and customer cohorts to see how retention rate is changes over time or how it varies across different types of users. Your time period can be anything from days to years, depending on your business and the specific questions you’re trying to answer.
How to calculate it: # users or customers in period 2 / # of users of customers in period 1 = % retention
Read more: https://www.profitwell.com/blog/youre-calculating-retention-wrong
What it is: Churn rate is the inversion of retention rate. It’s the percentage of users or customers that stop being active, or stop paying, from one period to the next.
How to use it: The applications are the same as for retention rate, although some prefer to look at one or the other metric.
How to calculate it: # of users or customers lost in period 2 / # of users of customers in period 1 = % churn
Product Engagement and User Engagement Metrics
DAU, WAU, MAU
What it is: These acronyms stand for Daily Active Users (DAU), Weekly Active Users (WAU) and Monthly Active Users (MAU). They measure active users within a certain time period.
How to use it: On their own, these metrics provide a high-level view of how your user base is changing over time and user activity in your product. However they’re more useful as components of other metrics, which we’ll discuss below.
How to calculate it (formula): The definition of an active user will depend on the nature of your product. For example Instagram may define an active user as one who logs in and spends at least 60 seconds viewing posts, whereas an accounting platform may define an active user as one who makes an accounting entry or runs a report. Using your definition of active, sum up all of the unique (counting only once) users within each time period.
What it is: Stickiness measures how often users engage with your product.
How to use it: Stickiness helps you understand whether users value your product (i.e. whether they’re returning or not) and is also an indicator of the growth rate of your business. A more advanced way to calculate it is to look at cohort retention, which provides a better understanding of whether a particular group of users is continuing to engage with your product.
How to calculate it: The exact calculation you use will depend on the nature of your product. For example, if your product is intended to be used on a daily basis (e.g. Slack), you may calculate the stickiness ratio as:
Daily Active Users (DAU) / Monthly Active Users (MAU) = Stickiness Ratio
A ratio of 20% would tell you that for every 100 monthly active users you have, 20 of them use your product daily. While there’s no absolute number that is good or bad, monitoring how the stickiness ratio changes over time will help you understand trends in how your users value your product.
To get deeper insights, you can measure this ratio across different user cohorts. Cohorts can be based on user or customer attributes (e.g. free vs high value users) or time-based (users who were acquired in Q1 vs users acquired in Q2).
What it is: Cohort retention measures whether a particular group of users or customers continues to use your product or pay for your service over time
How to use it: At a high level, retention helps you understand whether users continue to value your product over time. Cohort retention can be used to:
- Understand whether different types of users (e.g. light users vs power users, regular customers vs high paying customers) continue to value your product over time
- Understand how retention varies based on when particular users or customers were acquired. For example, your sales team introduced significant discounts in Q4 to close more business. Do the customers won during that discount period show the same retention rates as other customers?
- Test the impact of product or service changes on retention rates over time. For example, you could introduce a new feature to a test set of users to see how it impacts their retention rate.
How to calculate it: The retention calculation that you use will depend on the nature of your product. You may choose to measure using DAU, WAU or MAU as defined above. Alternatively, for a subscription product you may measure it as renewal of paying customers.
Let’s assume that we have a product where it makes sense to measure using monthly active users (MAU). We want to see how retention changes based on when (which month) we first acquired the users. So we consider the acquisition month time period 0, and we measure retention for each of the subsequent months. This might result in retention data that looks like this:
|Month 0||Month 1||Month 2||Month 3||Month 4||Month 5||Month 6|
What it is: Time in app measures how long a user spent in your app over a period of time.
How to use it: This metric helps uncover trends in usage time, which is a proxy for how valuable users find your app.
How to calculate it (formula): As with many metrics, the way that you calculate it will depend on how your app is used. For example, if your app is intended to be used daily, you’ll measure time in app by adding up all of the active time users spend in your app each day. Then you can see how the daily time in app is trending over time, to understand how different user cohorts’ app engagement is trending. As you identify different usage patterns, you’ll want to dig deeper to understand what’s driving them.
Number of User Actions Per Session
What it is: Tracks how many, and which, actions users take when using your app.
How to use it: This metric can be used to understand how usage patterns change as you make feature or functionality changes within your app. As with other metrics, you can compare across different cohorts to get deeper understanding.
How to calculate it: First, define the actions that are relevant to track for your particular app or product. Then you can simply count the actions taken per session or, for a deeper understanding, you can bucket actions into different groups and count actions within each group.
User or customer satisfaction metrics
Net Promoter Score (NPS)
What is it: The NPS was created by Bain and Company to measure customer loyalty. We have a much deeper dive into NPS here.
How to use it: We don’t think that NPS should be used to measure product success, because it’s simply too broad. NPS is useful for understanding your users and customers overall satisfaction with your product, and how that changes over time.
How to calculate it: Again, the post linked above goes into much greater detail. Briefly, calculating NPS involves collecting responses to a simple question “how likely are you to recommend [company X] to a friend or colleague?” and then aggregating responses into three different groups to determine your detractors, neutral responses and promoters.
Customer Satisfaction Score (CSAT)
What is it: CSAT is a proxy of customer happiness or satisfaction, typically derived from a short survey.
How to use it: CSAT can be used as an overall health indicator of the satisfaction of your user base (or individual cohorts). By targeting questions to a specific part of the experience (e.g. onboarding or the product itself), you can get more insight into which aspects of the experience need to be improved. CSAT responses can also help you determine which customers are unhappy, and thus needing some extra attention from your team, as well as which customers are very happy, and can be advocates for your company or product.
How to calculate it: There are a number of different ways that you can phrase your survey questions, depending on your product and the goal of running the survey. For example “How would you rate your onboarding experience?” or “How would you rate your experience using our product?”. You then take the number of positive responses and divide them by total responses to get your CSAT percentage.
# positive responses / # total responses X 100 = CSAT %
Number of Support Tickets Created
What is is: An overall measure of the volume of support tickets created in a given period of time.
How to use it: This metric can give an overall indication of trends in your customers’ need for support, as well as how well your team is keeping up with support volume. Making sure to adjust for changes in the size of your customer or user base, the overall volume of support tickets will help you understand if changes that you make to your product and support resources are improving the customer experience. The overall volume also helps you understand if your support staffing or process needs to change to keep up with inbound support requests.
How to calculate it: A simple calculation is just adding up the number of support tickets created within a given period. To get more insight, segment your support tickets by customer cohort or severity.
In-app feedback response rates
What it is: A measure of user participation rates in your in-app surveys or other feedback requests.
How to use it: This metric gives you an indication of how engaged your user base is, however it is impacted by a number of factors. So you can measure how response rates vary over time and also test different types of surveys and feedback mechanisms to see which ones generate the highest response rates.
How to calculate it: # of survey responses / # of surveys delivered or shown = feedback response %
Feature Sentiment Ratings / Feature Fit Index (FFI)
What it is: Feature sentiment ratings measure how users feel about an existing or new feature. A sentiment rating that works really well is Feature Fit Index, which is a way to gauge feature sentiment with one simple question. We wrote an in-depth post on Feature Fit Index here.
How to use it: Use FFI after launching a new feature to understand how well it resonates with your users. We recommend scheduling this survey to be delivered in-app 30 – 90 days after launching the new feature, so that users have time to understand it and provide meaningful feedback. The survey is then as simple as “Would you be disappointed if this feature disappeared” and asks for a yes or no response.
The gold standard for Feature Fit Index is Slack, which has a product-wide FFI of 60%. But we recommend focusing FFI on a single feature or functionality and targeting a score of ~40% or better.
How to calculate it: # of respondents who would be disappointed if the feature were removed / total # respondents = FFI score
Roadmap Validation and Scoring
What it is: A methodology for prioritizing work in your product roadmap
How to use it: There are a number of different prioritization models and frameworks. Most are designed to get to a cost / benefit weighting, which helps product teams decided what to work on next. A popular framework, developed at Intercom, is RICE, which stands for Reach, Impact, Confidence and Effort.
How to calculate it:
Reach – How many people you estimate the initiative or feature will impact within a given timeframe.
Impact – Intercom uses a five tiers of impact:
- 3 = massive impact
- 2 = high impact
- 1 = medium impact
- .5 = low impact
- .25 = minimal impact
Confidence – Represents how confident you are in your impact assessment. If you have historical data to back up your impact assessment, you may be highly confident. On the other hand, you may have very little basis for your assessment, which would lower your confidence. Confidence is expressed as a percentage.
Effort – To quantify effort, estimate the total resources (including from product, engineering, testing and design) to complete the initiative. Often this effort is expressed in “person weeks” or “person months”.
Once you have all of your inputs together, you can calculate your RICE score. Reach, Impact and Confidence are the numerators and Effort is the denominator.
Reach X Impact X Confidence / Effort
Different companies will use different definitions for each of the inputs. The key is that you’re internally consistent when evaluating different initiatives, which will allow you to weigh them against each other.
Other SaaS Metrics
Monthly Recurring Revenue (MRR)
What is it: Measures the value of monthly subscription (hence, recurring) revenue.
How to use it: One of the fundamental characteristics of SaaS businesses is the predictable, recurring revenue. MRR is a core metric to measure how subscription revenue is changing over time, which facilitates revenue forecasting and helps understand how successfully the business is (or isn’t) growing.
How to calculate it:
A quick way to estimate MRR is: # of customers * average monthly contract value
This approach is useful for forecasting
Actual MRR is most easily calculated from within your accounting software.
Customer Lifetime Value (CLV, LTV)
What is it: CustomerLifetime Value, also called Lifetime Value, measures the full value of revenue from a customer over the entire time that they remain a customer.
How to use it: By understanding the lifetime value of a customer, companies can make smart decisions about how much they can profitably spend to acquire a customer. Thinking in terms of lifetime value, rather than just initial contract value, helps make better decisions for the long term health of the business. Lifetime value is impacted by factors such as churn / retention, contract value and upsell / cross sell revenue.
How To Measure Product Success
How to calculate it: annual contract value * average customer lifetime (in years) = customer lifetime value
Customer Acquisition Cost (CAC)
Product Success Manager Job
What it is: Measures the total of all costs spent acquiring a customer, including marketing, sales and onboarding support.
How to use it: CAC is an important metric to understand the cost of customer growth. There’s no absolute CAC number that is healthy or unhealthy, rather what you can profitably afford to spend is driven by the lifetime value of your customers.
How to calculate it: Marketing cost + sales cost + new customer support cost = Customer Acquisition Cost
Average Revenue Per Unit (ARPU)
What is is: A measure of the average revenue associated with each customer or contract. ProfitWell has a great in depth post on calculating and optimizing ARPU.
Customer Success Product Manager
How to use it: Use it to understand the value of an average customer within a given time period (e.g. a year) and to see how pricing, product and other changes impact ARPU over time.
How to calculate it: Total annual recurring revenue / total # of paying customers = ARPU
Cash Burn Rate
What is is: A measure of how much net cash the entire company consumes in a given period, typically monthly or yearly. OpenView Partners has a good overview that includes benchmark cash burn rates for different growth phases.
How to use it: There are a lot of metrics that help understand the health and growth of a SaaS business. But since many SaaS businesses aren’t profitable during the growth phase, knowing how many months or years of runway you have is very important. Cash burn is also evaluated by investors in fundraising rounds and is an important metric to look at as you model various growth and investment scenarios for the business.
Product Success Definition
How to calculate it: All period cash in – all period cash out = cash burn rate