North Star Metric Guide: Lessons from Amazon, Airbnb, Spotify & Gojek

Simplification and common sense are surprisingly rare these days. It’s not because companies don’t want these things but because it is really easy to overcomplicate how you approach things. Metrics and KPIs are no different and this is why a north star metric can is such a popular topic right now.

In this article, I will show you what I have learned from working with hundreds of companies across multiple industries and 5 continents. Choosing the right KPIs can be tricky but I’ll share a few ideas and frameworks for making this decision easier and how a north star metric (NSM) can help.

Let’s start by covering the basics of NSMs and the unintended incentives it could create internally.

What Is It and How It Creates Internal Incentives

North Star Metrics are meant to global KPIs that help you understand the most important user behaviors of your product or business. They could be company-wide (for smaller companies) or limited to specific products (typically a better fit).

For example, let’s look at a few popular products or business types and what kinds of NSMs you end up with:

  • Ecommerce = # of purchases or % of repeated purchases
  • Meditation app = # of meditation sessions
  • B2B SaaS = quick ratio

It can be hard to find THE number to use but the goal here is to unite everyone in your team under the same goal. If everyone is focusing on the same metric, all efforts should be optimized towards improving it.

I love the simplicity of NSMs but I also worry about the incentives it can create. As it turns out, companies (and people) will work to optimize whatever number they are given. 

I remember working with a company that was spending a few million dollars per month on advertising and optimizing for page views and views of their ads. The agencies they worked with simply focused on finding the cheapest traffic sources to get the most views but it was unclear if this was the most effective way to get conversions which in their case was bookings of hotels.

What are the incentives behind pageviews?

What are the incentives behind popular KPIs like pageviews?

This is an example of negative incentives and what can happen with NSM. It’s not that people are malicious but everyone is trying to do their best. If you make your NSM user acquisition-related like signups, you may discover that the retention of those users is quite poor. 

Finding the right balance between incentives is difficult but that’s where the identification process of the right NSM can be helpful. Let’s look at a few factors to consider to ensure you’re selecting the right north star metric.

Identifying Your NSM and How it Changes Over Time

Let’s look at a few factors for selecting a north star metric for your product. 

Size of Analysis

The first factor to understand is the size of what you’re trying to analyze. As I mentioned in the previous section, I believe north star metrics can be helpful for specific products or smaller startups. It is too simplistic to try to boil down an entire company into just one number.

The ideal size is something that is new (new product, new startup) or something that needs a new focus due to degrading performance (a poor performing product, a misaligned company). If you ever been in a situation where someone says “we should only focus on X during the next 6 months”, that’s likely a good scenario for a north star metric.

If you’re working with something that is stable, growing and more mature, a north star metric may miss the subtleties of what you’re doing. 

User Behaviors Desired

The second factor is understanding what user behaviors you’re trying to optimize for. You want users to adopt a certain feature or you may be interested in new user signups. For every behavior, you should be considering connect behaviors that could also be important.

For example, if we want more user signups (because we have a new product), do we also care about those users staying and using the product? The answer is probably yes and we should consider retention as an important behavior in our NSM.

This would mean that instead of looking at “# of new signups”, we could look at something like “# of weekly active users” since the second number would include new users but also existing retained users. We may even break that number down into active users that are new within the last 30 days or not to see how where the growth is coming from. 

Internal Incentives

Next, we want to understand what kind of internal incentives are we created by choosing a north star metric. To do this properly, take your metric and brainstorm who different people or teams in your company will act to optimize around it.

Be reasonable here instead of assuming that people will go out of their way and find the most complicated way of optimizing this number. Let’s take our example of “# of weekly active users” and look at different teams:

  • Product: depending on the definition of “active”, this team might set up communication journeys to bring users back into the app. If the “active” definition is to loose (e.g. just open the app), any communication with users will bump up the number since a certain % of users will always open and click-through messages. Fundamentals in product analytics will here as well.
  • Marketing: this team may focus on marketing channels that drive lots of user signups but may skip the quality of these users or simply blame “product” for not retaining them.
  • Sales: this team may focus on closing the largest clients as their sheer size could have a significant impact on the NSM. 

Getting the incentives right is tricky and will be more of an iterative process. You will also need to adjust your expectations for targets as your team learns what is doable and what isn’t while still keep the quality of your north star metric high.

Formula Best Practices

The fourth factor to consider is the underlying formula for your north star metric. You want to make sure that every element of your NSM is clear and is aligned with your company goals. For example, NSMs involving “active” users are quite popular since they tend to balance incentives quite well.

However, companies will then drop the ball by making their “active” definition extremely broad. Basically anyone who breathes and opens the app will count as an active user. Instead, you should make your definition tight and closely connected to the core value of your product. 

Defining active users in tools like Amplitude

Tools like Amplitude will let you define what “active” means.

The same could apply to user signups. Instead of counting anyone with an email address, you could tweak it to only count qualified leads (by demographics) or users who onboarded successfully

Fit Into Company Strategy

The fifth factor is to tie your north star metric into your overall company strategy. This varies depending on the size of your company but if you’re trying to raise the next round of funding, your NSM should be one of the KPIs that your investors would want to see. 

If you’re expanding into new markets, the NSM for your product should be tied to what success should look like within that market.

How You Can Visualize & Track Your NSM

Defining and choosing the NSM is the trickiest part here. Once you do that, you can figure out how to visualize and track it. You can use pretty much any tool that you want (and there are hundreds of options) but keep in mind the following things.

  1. Can you easily send the latest report on the NSM to everyone in your team in different formats? This could be email, Slack messages, SMS, TV kpi dashboard or simply a bookmark but make it easy for everyone to be notified of the latest number.
  2. Are you able to drill down and segment the different elements of your NSM? This can help diagnose unexpected changes (positive or negative). You will need to properly track other elements of your product
  3. Can you tie your NSM with other long metrics? If your NSM is “# of weekly active users”, you may also be interested in seeing the LTV and CAC of these users.

Here are some example of popular tools:

Amplitude NSM example

Amplitude can easily create widgets and dashboards.

Mixpanel NSM example

Mixpanel can also help you create complex NSMs formulas.

Domo NSM example

Domo lets you combine different sources of data to get one single view or dashboard.

Databox NSM example

Databox can also make it easy to combine different data sources to calculate your NSM. Source for image.

Case Studies from Amazon, Airbnb, Spotify & Gojek

Amazon

Amazon is a large company and wouldn’t benefit from having a single NSM but its individual products could. An example could be the Prime membership which could have an NSM like “# of active prime members” (active meaning a certain amount of purchases or activity within a time period.

Amazon Prime an an example of NSM.

Prime is an established product so pure growth of new members wouldn’t the most important item. They would also want to see how many members keep renewing (like me) and how many members are actually using it (taking advantage of Prime-only items, watching online video, etc). 

Airbnb

Airbnb is another established company and has recently started to promote “Adventures” which are described as “multi-day trips led by local experts”. For this product, Airbnb could consider an NSM of % of satisfaction with adventures or how happy people were with the trip. This leading indicator can help uncover systematic issues (poor scheduling, incorrect choice of locations, misaligned expectations). 

Airbnb Adventures an an example of NSM.

Unlike Amazon Prime, Adventures is a relatively new product (launched in mid-2019 at the time of this writing) and they are likely still working out the kinks. Ensuring that every experience is free of issues is on the high priority items for any new product before scaling the promotion of this product.

Spotify

Spotify has been in the music business for a while but they have also started pushing into the growing podcast space. For this product (or part of a product), they could explore an NSM like “average # of followed podcasts”. This would look at how many podcasts a user follows which has an assumption that following a podcast is a stronger show of interest than simply playing an episode. 

Spotify Podcasts an an example of NSM.

Notice that this NSM isn’t for the entire product which is quite complex now with different features. Instead, we focus on one part of the product and use north star metrics to measure it initial success.

Gojek

Gojek is an Indonesian-based company that offers over 20 services from food delivery to taxi services. This is a fascinating example of how companies in other parts of the world grow and operate. Let’s take one of its products, Go-Food, which handles food deliveries. An NSM for this product could be “% of repeat orders”. Go-Food is an established product so focusing on retention and bringing customers back takes on an important part of the product instead of just driving new customers (which can easily be done with discounts).

Go-Food an an example of NSM.

Conclusion

Every KPI has consequences and you need to aware of them. Simplifying your business or product to one metric can be a great way to get everyone on the same page but you also need to understand it’s limitations.

Remember that KPIs are meant to help you make better decisions and let that be your guiding light.

One more thing before you go! Do you know how to get more insights out of your data? 

All companies are sitting on a goldmine of data that they haven't fully explored. It's not about technology or capturing more data. The key is to learn how to make the most of your current data and convert it into actionable insights. This is the main idea behind my first book, The Data Miage: Why Companies Fail to Actually Use Their Data

I'm excited to announce the release of the book through all major retailers. If you're interested, you can download the first chapter for free using the form below. You'll learn what the best data-driven companies do differently and how to make sure you're playing the right data game.

Leave a Comment