Mixpanel vs Heap: Should You AutoTrack or Manually Instrument?

Choosing tools is hard. I sometimes forget how complex the process can seem especially when the options are quite similar. The question of Mixpanel vs Heap is one of those situations.

At a glance, they do roughly the same thing. So how do you choose one tool to commit for the next 12 months? That’s what I’m here to help with.

I’ll take both tools through 4 rounds, showing you the pros and cons of each tool. This won’t cover every single detail or report offered but instead, will focus on the key areas that every company should consider.

Let’s first start by understanding the point of these tools and why companies should even bother implementing either option.

What’s the Point of Either Tool?

Before we even discuss either tool, we need to understand why we are even bothering to go through all this effort. Mixpanel and Heap are product analytics tools designed to help teams understand their product.

Products in this context typically mean digital products like web apps, mobile apps, SaaS, ecommerce stores, etc. Being a “technology company” might mean that you qualify but you need to have a digital product. 

I have also seen a rise in traditional companies like The Guardian newspaper benefiting from these tools. In this case, The Guardian is able to understand how users are engaging with their content and then becoming paid subscribers (their primary revenue model now). If they were just offering to advertise, the need for product analytics might not be that clear.

Let’s say you have a digital product. What can you learn from tools like Mixpanel and Heap? For starters, you could find patterns among your best users. Why do they become paying customers? How do they use your product? Which features are being used and which ones are being ignored?

The answers to these questions will help you build a better product that is more engaging, more profitable, and more useful.

Both of these tools have reports specially designed to answer these questions. You have likely seen reports like Segmentation, Funnels, and Cohort Analysis which are the fundamental reports that you need to start with.

Finally, both of these tools will then make it easy to report on these product metrics through email, Slack, and other means. 

In summary, if you have a digital product and you need to understand how your users (or customers) are using it, then you’re in the right place. If not, you need to exit stage right. While you could “hack” these tools to do other things, it might not be worth the effort.

Let’s now go through a few rounds to see which tools “win” my conditions. You may rank them differently and that’s ok. You need to look at this comparison through the lens of your company.

Round 1: Implementation

We need to start by looking at implementation. If you can’t get the right data into the tool, this party won’t even get started.

Heap has popularized the idea of “auto-tracking”. This means that you add the snippet to your website or app and then Heap will start collecting data out of the box. The data itself is things like page views, user clicks, user form submissions, and more.

All this data could be enough to answer the questions that you care about. For example, you may want to see onboarding performance. You could build a report using the page views to get a look at where users are dropping off.

Heap also has the ability to capture custom events that could be used for tracking anything really. You will also need to manually instrument the capturing of user information like name, email, and user ID.

Mixpanel on the other hand tracks very little out of the box. You will need to manually instrument every event with some exceptions (e.g. mobile events like app open). 

This manual instrumentation can be quite time consuming but you need to go through it, otherwise, you won’t have any Mixpanel data. You will also need to manually instrument user information like name, email, and user ID.

Mixpanel actually had an auto track functionality at some point but they have moved away from it. They either couldn’t get it to work from a technical perspective or it didn’t make sense for their target market. If you’re exploring implementing through Segment.com, check out this post.

On implementation, Heap wins. The auto-track means that you will get data out of the box while still being able to track custom events like in Mixpanel.

Heap 1, Mixpanel 0.

Round 2: Report Building

Next, we have reporting. We have some data in the tool and now we want to build those sweet reports.

Let’s first look at the fundamental reports which to me are segmentation, funnels, and retention. On these 3 reports, both tools are quite similar. They will do roughly the same thing but Mixpanel gets an edge on details.

Mixpanel Insights Report
Fundamental Report #1: Segmentation
Fundamental Report #2: Funnels
Mixpanel Retention Report
Fundamental Report #3: Retention

I think the Mixpanel tool is more polished when it comes to doing advanced things like formulas, calculating DAU, etc If you’re looking flexible dashboards, both tools do quite well here. If you need more advanced options, check out Tableau or Data Studio.

If we look at advanced reports like path analysis, machine learning reports, and more, then Mixpanel also wins. There are several reports that Mixpanel has which Heap just simply doesn’t have. Signal and Impact are two examples.

Depending on your sophistication, you may or may not care about these advanced reports.

On report building, Mixpanel wins. Getting those details right takes time and Mixpanel has put in the time here.

Heap 1, Mixpanel 1.

Round 3: Messaging, A/B Testing, etc

Finally, let’s look at everything else. Reporting is the bread and butter for both of these tools but they do offer other things.

Let’s start by looking at connections with data warehouses. Both of them have options to automatically export the data to databases like Redshift, and BigQuery. I think both solutions are good enough.

Next, we can look at messaging. Mixpanel offers the ability to send notifications like email, SMS, push, and in-app. Heap doesn’t have this functionality. Marketing automation has a lot of hidden benefits so this could be a big win for the Mixpanel camp.

Finally, we can look at A/B testing. Mixpanel offers A/B testing for mobile apps only (which is a bit weird) and Heap doesn’t offer it at all.

On the extra features, Mixpanel wins. However, keep in mind that you may not need these features. You may already have a dedicated messaging platform and an A/B testing tool.

Heap 1, Mixpanel 2

Round 4: Mixpanel vs Heap in Pricing

Our final category is pricing. The solution needs to be fit in the budget.

Mixpanel has public pricing up to 25,000 MTUs which is quickly becoming the standard in how these tools charge for usage. MTU stands for Monthly Tracked Users and it includes known users and anonymous traffic. 

If you want things like messaging and data warehousing, you’ll need to add those as add ons. A fully loaded plan for 20K MTUs will run you around $1,100 per month. They also offer a free plan for up to 1,000 MTUs.

Heap on the other hand uses sessions for their billing. Their free plan gives you up to 5,000 sessions and all the analytics reports. Data warehousing (what they call Heap Connect) is an add on like Mixpanel. Any other pricing plans have to be negotiated with a sales team.

Mixpanel pricing has a more clear way of progressing for companies that are just getting started. If you’re beyond the public limits, you’ll likely pay similar amounts for Mixpanel and Heap. Their enterprise pricing doesn’t differ as much but you would need to get a custom quote.

On pricing, I would have to give it to Mixpanel.

Heap 1, Mixpanel 3.

Conclusion

While it may seem that Mixpanel is a “better”  tool, I wouldn’t jump to this conclusion just quite. The pain of implementation can be enough of a win that this justifies going with Heap. 

You may not need all the extra features like Messaging and A/B testing which leaves you with just reporting. Finally, you may be beyond their public pricing and looking at similar quotes.

Both tools have a solid product, clear philosophy, and strong growth behind them. Choosing either of them would be a great choice.

Finally, if you’re looking for other tool comparisons, here are a few relevant links:

One more thing before you go! Do you know how to get more insights out of your data? 

All companies are sitting on a goldmine of data that they haven't fully explored. It's not about technology or capturing more data. The key is to learn how to make the most of your current data and convert it into actionable insights. This is the main idea behind my first book, The Data Miage: Why Companies Fail to Actually Use Their Data

I'm excited to announce the release of the book through all major retailers. If you're interested, you can download the first chapter for free using the form below. You'll learn what the best data-driven companies do differently and how to make sure you're playing the right data game.