Simplification and common sense are surprisingly rare these days. It’s not because companies don’t want these things but because it is really easy to overcomplicate how you approach things. Metrics and KPIs are no different and this is why a north star metric can is such a popular topic right now.
In this article, I will show you what I have learned from working with hundreds of companies across multiple industries and 5 continents. Choosing the right KPIs can be tricky but I’ll share a few ideas and frameworks for making this decision easier and how a north star metric (NSM) can help.
Let’s start by covering the basics of NSMs and the unintended incentives it could create internally.
What Is It and How It Creates Internal Incentives
North Star Metrics are meant to global KPIs that help you understand the most important user behaviors of your product or business. They could be company-wide (for smaller companies) or limited to specific products (typically a better fit).
For example, let’s look at a few popular products or business types and what kinds of NSMs you end up with:
- Ecommerce = # of purchases or % of repeated purchases
- Meditation app = # of meditation sessions
- B2B SaaS = quick ratio
It can be hard to find THE number to use but the goal here is to unite everyone in your team under the same goal. If everyone is focusing on the same metric, all efforts should be optimized towards improving it.
I love the simplicity of NSMs but I also worry about the incentives it can create. As it turns out, companies (and people) will work to optimize whatever number they are given.
I remember working with a company that was spending a few million dollars per month on advertising and optimizing for page views and views of their ads. The agencies they worked with simply focused on finding the cheapest traffic sources to get the most views but it was unclear if this was the most effective way to get conversions which in their case was bookings of hotels.
What are the incentives behind popular KPIs like pageviews?
This is an example of negative incentives and what can happen with NSM. It’s not that people are malicious but everyone is trying to do their best. If you make your NSM user acquisition-related like signups, you may discover that the retention of those users is quite poor.
Finding the right balance between incentives is difficult but that’s where the identification process of the right NSM can be helpful. Let’s look at a few factors to consider to ensure you’re selecting the right north star metric.
Identifying Your NSM and How it Changes Over Time
Let’s look at a few factors for selecting a north star metric for your product.
Size of Analysis
The first factor to understand is the size of what you’re trying to analyze. As I mentioned in the previous section, I believe north star metrics can be helpful for specific products or smaller startups. It is too simplistic to try to boil down an entire company into just one number.
The ideal size is something that is new (new product, new startup) or something that needs a new focus due to degrading performance (a poor performing product, a misaligned company). If you ever been in a situation where someone says “we should only focus on X during the next 6 months”, that’s likely a good scenario for a north star metric.
If you’re working with something that is stable, growing and more mature, a north star metric may miss the subtleties of what you’re doing.
User Behaviors Desired
The second factor is understanding what user behaviors you’re trying to optimize for. You want users to adopt a certain feature or you may be interested in new user signups. For every behavior, you should be considering connect behaviors that could also be important.
For example, if we want more user signups (because we have a new product), do we also care about those users staying and using the product? The answer is probably yes and we should consider retention as an important behavior in our NSM.
This would mean that instead of looking at “# of new signups”, we could look at something like “# of weekly active users” since the second number would include new users but also existing retained users. We may even break that number down into active users that are new within the last 30 days or not to see how where the growth is coming from.
Next, we want to understand what kind of internal incentives are we created by choosing a north star metric. To do this properly, take your metric and brainstorm who different people or teams in your company will act to optimize around it.
Be reasonable here instead of assuming that people will go out of their way and find the most complicated way of optimizing this number. Let’s take our example of “# of weekly active users” and look at different teams:
- Product: depending on the definition of “active”, this team might set up communication journeys to bring users back into the app. If the “active” definition is to loose (e.g. just open the app), any communication with users will bump up the number since a certain % of users will always open and click-through messages. Fundamentals in product analytics will here as well.
- Marketing: this team may focus on marketing channels that drive lots of user signups but may skip the quality of these users or simply blame “product” for not retaining them.
- Sales: this team may focus on closing the largest clients as their sheer size could have a significant impact on the NSM.
Getting the incentives right is tricky and will be more of an iterative process. You will also need to adjust your expectations for targets as your team learns what is doable and what isn’t while still keep the quality of your north star metric high.
Formula Best Practices
The fourth factor to consider is the underlying formula for your north star metric. You want to make sure that every element of your NSM is clear and is aligned with your company goals. For example, NSMs involving “active” users are quite popular since they tend to balance incentives quite well.
However, companies will then drop the ball by making their “active” definition extremely broad. Basically anyone who breathes and opens the app will count as an active user. Instead, you should make your definition tight and closely connected to the core value of your product.
Tools like Amplitude will let you define what “active” means.
The same could apply to user signups. Instead of counting anyone with an email address, you could tweak it to only count qualified leads (by demographics) or users who onboarded successfully.
Fit Into Company Strategy
The fifth factor is to tie your north star metric into your overall company strategy. This varies depending on the size of your company but if you’re trying to raise the next round of funding, your NSM should be one of the KPIs that your investors would want to see.
If you’re expanding into new markets, the NSM for your product should be tied to what success should look like within that market.
How You Can Visualize & Track Your NSM
Defining and choosing the NSM is the trickiest part here. Once you do that, you can figure out how to visualize and track it. You can use pretty much any tool that you want (and there are hundreds of options) but keep in mind the following things.
- Can you easily send the latest report on the NSM to everyone in your team in different formats? This could be email, Slack messages, SMS, TV kpi dashboard or simply a bookmark but make it easy for everyone to be notified of the latest number.
- Are you able to drill down and segment the different elements of your NSM? This can help diagnose unexpected changes (positive or negative). You will need to properly track other elements of your product
- Can you tie your NSM with other long metrics? If your NSM is “# of weekly active users”, you may also be interested in seeing the LTV and CAC of these users.
Here are some example of popular tools:
Amplitude can easily create widgets and dashboards.
Mixpanel can also help you create complex NSMs formulas.
Domo lets you combine different sources of data to get one single view or dashboard.
Case Studies from Amazon, Airbnb, Spotify & Gojek
Amazon is a large company and wouldn’t benefit from having a single NSM but its individual products could. An example could be the Prime membership which could have an NSM like “# of active prime members” (active meaning a certain amount of purchases or activity within a time period.
Prime is an established product so pure growth of new members wouldn’t the most important item. They would also want to see how many members keep renewing (like me) and how many members are actually using it (taking advantage of Prime-only items, watching online video, etc).
Airbnb is another established company and has recently started to promote “Adventures” which are described as “multi-day trips led by local experts”. For this product, Airbnb could consider an NSM of % of satisfaction with adventures or how happy people were with the trip. This leading indicator can help uncover systematic issues (poor scheduling, incorrect choice of locations, misaligned expectations).
Unlike Amazon Prime, Adventures is a relatively new product (launched in mid-2019 at the time of this writing) and they are likely still working out the kinks. Ensuring that every experience is free of issues is on the high priority items for any new product before scaling the promotion of this product.
Spotify has been in the music business for a while but they have also started pushing into the growing podcast space. For this product (or part of a product), they could explore an NSM like “average # of followed podcasts”. This would look at how many podcasts a user follows which has an assumption that following a podcast is a stronger show of interest than simply playing an episode.
Notice that this NSM isn’t for the entire product which is quite complex now with different features. Instead, we focus on one part of the product and use north star metrics to measure it initial success.
Gojek is an Indonesian-based company that offers over 20 services from food delivery to taxi services. This is a fascinating example of how companies in other parts of the world grow and operate. Let’s take one of its products, Go-Food, which handles food deliveries. An NSM for this product could be “% of repeat orders”. Go-Food is an established product so focusing on retention and bringing customers back takes on an important part of the product instead of just driving new customers (which can easily be done with discounts).
Every KPI has consequences and you need to aware of them. Simplifying your business or product to one metric can be a great way to get everyone on the same page but you also need to understand it’s limitations.
Remember that KPIs are meant to help you make better decisions and let that be your guiding light.