Product Analytics Framework: Finding Your Everest and Inner Karate Kid

Let me make this clear. If you’re not using a product analytics framework to guide your decisions, you’re wasting yours and your team’s time. Building great products is hard and you need all the help you can get.

Teams tend to get in their own way when designing products. Egos clashed, customer feedback is taken too literally and teams get lost in a sea of software bugs.

A product analytics framework is like a lighthouse shining a guiding light towards the shore. You don’t have to sail through the darkness hoping that you come across land. Let me show you how to build your own lighthouse.

What’s Your Everest? Determining a Strategy

Climbing Mount Everest is a common bucket list goal for many people. There’s something about the mountain and the challenge that lures people in every day. It’s not something that you can attempt on a whim. You need to be physically and mentally fit. 

Nonetheless, thousands of people attempt this every year. The great thing about this goal is that it is crystal clear. You know what outcome you want and you can reverse engineer it to create a plan of attack for how you should train your body and mind. If you show up unprepared to climb Mount Everest, that’s on you. There was nothing secret about climbing this mountain.

You need to think about your product in the same way. Figure out what outcome you’re working towards and reverse engineer it into tangible milestones. This idea isn’t revolutionary but it can be forgotten in the fog of war. 

There are 4 questions that you should consider when trying to determine your “Everest”.

What stage of growth are you in?

Have you achieved product-market fit or are you still searching? Is your product ready to scale through paid acquisition? Perhaps you’re a mature company that is struggling with stagnation. Wherever you are, that’s ok. You simply need to acknowledge it and use that as your starting point. 

What’s the dream if you could do anything?

Don’t hold yourself back here. Forget about the endless backlog of issues or the list of customer complaints that you have yet to solve. Think offensively instead of defensively. Do you want to build a consumer product used on a daily basis? Achieve low levels of retention? Build a cross-platform experience?

What constraints are you dealing with?

I’m a big fan of ambitious goals but life is also about being practical. If you have limited funding and resources, you might need to scale back your goal or choose another one. You also need to keep in mind other priorities like raising the next round of funding or trying to secure a key hire. Don’t forget about data privacy either.

What’s after this specific mountain?

Climbing Mount Everest is a fantastic achievement but what’s after it? This isn’t going to the only goal you will ever tackle. Take some time to think through future mountains and what could be done in the future.

Wax On, Wax Off: Product Analytics Techniques

Any good product analytics framework will have several tools in the bag. These are the things that you will use to analyze data on a regular basis. You don’t need all of them at once or even in the beginning but it’s important to have a high-level understanding of what’s possible.

Trend Analysis

This is the classic report that you see everywhere. You can use this to see trends in signups, user behaviors, and more. For example, you could see if the number of cancellations is increasing or decreasing over time and break it down by the reason people canceled. You do need the right data to be able to run these kinds of granular analysis.

Active User Breakdown

Almost all companies use the term “active user” though the actual definitions vary by a mile. This report can show you metrics like DAU, WAU, MAU and North Star broken down by user attributes like marketing source and geographical location.

Don't assume that "active user" means what you think it does.
Don’t assume that “active user” means what you think it does.

Funnel Analysis

Use this report to understand where users are dropping off in a funnel. For example, you could analyze your onboarding flow to see what parts seem to confuse the user and prevent them from actually getting started with the product.

Mixpanel Funnels

Feature Adoption

If you spend 3 months on a new feature, do you know exactly how many users are actually using it? This is what Feature adoption reports are meant to answer and they can also give you feedback on what is actually working in your product strategy.

Retention Analysis

You need to keep users around long enough and that’s what Retention Analysis helps you understand. For example, you want to know how many users are using specific features on a weekly basis and if this is improving with recent users (or cohorts).

Stickiness

This is more relevant for consumer companies but it answers the question on how frequent customers are actually engaging with your product. There are different formulas for stickiness but the reports are similar.

Amplitude Lifecycle Report

A/B Testing

This isn’t a report but it will be one of the ways in which you can test hypotheses and ideas. Make sure that you can track experiment variations within your data.

Bringing It Together Into a Product Analytics Framework

Let’s bring it all together in a coherent product analytics framework. Chefs call this “mise in place” which is where they gather all the relevant food ingredients before they start cooking. This saves you from having to frantically search for something as your food burns.

First, you’ll need to communicate the strategy (your Everest) to the rest of the company. You don’t need feedback from everybody but you should be aware of what key people think e.g. CEO and if it aligns with the overall company strategy.

Two, you’ll need to get the relevant resources to execute on this plan. Do you need product managers? Data Analysts? Engineering time? Whatever is, this is the time to get it. Just like with our lists for Santa as kids, you won’t get everything you ask for.

Third, what data and tools (technology), do you need? Don’t go crazy here with tools but an ability to easily query and visualize data will be helpful. Look at product analytics tools like Mixpanel, Amplitude, Heap and Segment.com

Fourth, how will you evaluate progress? Are you going to check in every two weeks? How can you make the relevant adjustments on the fly? It’s important to know the route towards your destination but it’s just as important to know how to navigate without GPS.

Save all your answers into a document and share with the company. This is your guiding lighthouse that you can refer back to whenever you feel out of sync in your product.

Don’t copy what big companies are doing. Find your own path by learning what worked for them and combining it with what is unique about your team. That’s how you can end up with a product analytics framework to support growth at different company stages.

One more thing before you go! Do you know how to get more insights out of your data? 

All companies are sitting on a goldmine of data that they haven't fully explored. It's not about technology or capturing more data. The key is to learn how to make the most of your current data and convert it into actionable insights. This is the main idea behind my first book, The Data Miage: Why Companies Fail to Actually Use Their Data

I'm excited to announce the release of the book through all major retailers. If you're interested, you can download the first chapter for free using the form below. You'll learn what the best data-driven companies do differently and how to make sure you're playing the right data game.