by Calvin French-Owen
Co-Founder and CTO @ Segment @ Segment
Ever since the early days of Segment, customers have asked us for advice on what sorts of data they should collect and which metrics they should use to track progress. Even after helping thousands of companies, the answers to these questions are still surprisingly hard to prescribe.
The main challenge here lies in the fact that no business is exactly the same. Whether you share the same marketplace dynamics as others, use the same channels for growth, or sell to a similar audience, there’s no exact playbook to follow that will guarantee product adoption or business growth.
However, there are proven methods you can use to arrive at your key success metrics as well as frameworks for tracking those metrics in a scalable way.
In this chapter, we’ll share how to do just that, along with a handful of tips and techniques we’ve seen work when it comes to understanding user behavior and drivers of growth.
Whether you’re just getting started or refreshing your existing metrics, we recommend limiting the number of key metrics to track—even if it’s just one. Paring down to a small set of key metrics provides an unparalleled focus for teams working towards a common goal.
As a way of narrowing down your metrics, a “future self” test will give you confidence that you’re on the right path. To do so, propose a timeline (typically three to six months out) and ask yourself questions like the following:
If you can give an emphatic “YES” to these future self questions, then you’re on the right track to identifying your key metrics! If not, we suggest pressure testing alternative metrics until you find a couple that create happiness for your future self.
Note: In our experience, it’s easy to fall into the trap of a red herring metric, also known as the vanity metric. These are metrics that at first glance seem directionally accurate, but after closer inspection, don’t actually deliver the results that you need. A red herring metric might be something like page views to your homepage, when what you actually want are paying, engaged users.
A poorly defined metric will cause a lot of trouble. At best, you’ll spend unnecessary energy explaining yourself in meetings to get all parties aligned. At worst, teams and individuals will assume their own definitions and optimize for a metric that could set your business on a disastrous path.
To avoid confusion and get everyone aligned toward a common goal, you’ll want to provide crisp definitions of your metric(s).
Here’s an example of how we define our “habit moment” at Segment:
You’ll notice this metric is time-bound, clearly defined, and makes sense to anyone at Segment reading it. What’s more, our habit moment is made up of a combination of various leading events and data points as inputs—sending data, two integrations, and signed-up within a specific time window. We chose this metric because its behavior most indicative of a customer retaining long term.
Because this definition is clearly defined, it makes it easy to spot where we should focus our efforts—on experiments that encourage users to send data or connect a new integration. To move our habit moment in the right direction, we could try experiments that encourage users to add multiple integrations during signup, suggest new integrations to try via email, or surface tutorials for sending data within our app.
Once you’ve clearly defined a key metric, you’ll want to start tracking events (user actions) that make up that metric. For example, if your key metric is active users, you’ll need to instrument events across your product that define an active user such as
User Logged In or
To do that, we recommend creating an implementation spec or a tracking plan. A tracking plan will help minimize the time and resources necessary to get your metrics set up quickly and effectively.
A tracking plan clarifies what events to track, where those events live in your code base, and why those events are necessary from a business perspective.
Tracking plans not only help everyone stay on the same page during implementation, but they also make it easy for engineering, product, marketing, and analytics teams to use and understand the data you collect.
Ideally, your tracking plan should be in a format like a Google Sheet or Wiki that's easily shared and updated by multiple stakeholders.
A good tracking plan does the following:
Check out our tracking plan if you want to see what this should look like.
Activation, or the process of turning a new user into an engaged user, is a key step in the conversion funnel. The activation rate, therefore, is the percentage of users that become activated or engaged after they sign up.
To get to an activation rate, we need to divide "active" users by total users. "Active" here depends on your business, but should suggest a user is engaged and on the way to becoming a paid customer.
We can break down the definition of "active" by understanding the key customer behaviors that indicate when a user gains considerable value from your product.
For example, for community forums, some activation events can be "Voted" or "Commented," as they indicate participation from the user in the forum rather than "lurkerdom."
At Segment, we consider a customer to be active if they are sending data to two or more integrations.
First, the customer must create an account. We use
Account Created for this event.
From there, one of our key activation events is
Integration Enabled because it means they have set up an integration using Segment. The other key activation event is Data Sent because it means they are successfully using Segment to send data to a different tool—the first step towards getting value.
With clearly defined key metrics and a tracking plan mapped out, you should be able to start piecing together a funnel of your customer journey.
A funnel is really just a series of user actions that roll up into a desired end result. Measuring the core stages in your funnel is incredibly helpful as it allows you to easily pinpoint which areas have the biggest drop off in conversion. And funnel stages with the worst conversion are usually your biggest areas of opportunity.
Putting this into practice, let’s imagine that a simplified version of your funnel goes from 1) a web visit to 2) a free trial sign-up to 3) an activated user and ends with 4) a paid customer. If your conversion rate from sign-up to an active user is 5%, and your conversion from an active user to a paid customer is 40%, it’s pretty obvious where to focus your efforts—incentivizing new users to activate.
Breaking down each part of your funnel into more granular subsets will provide even more clarity on where to focus. If we zoom into the activation stage of our funnel at Segment, it looks something like this:
Using the example above, we want users to successfully send data through Segment to an integration. But there’s an additional set of steps that lead up to that point...
Whew! Suddenly we have a more realistic picture of the steps that drive our activation metric. And we know exactly where to focus to improve user activation.
While measuring exactly what a user did is critical, it doesn’t always paint the full picture of what’s happening inside your funnel. Contextual user data points like business size, role, industry, and location will also have an impact on the conversion rate of your funnel.
If your product is built to serve a mom-and-pop retail business, then your conversion metrics will have a noticeable difference depending on a user’s industry and company size.
So how do you go about finding who is using your product?
First, you’ll want to make sure the metrics you track are associated with some sort of user ID—a unique identifier that lets you tie together events across user traits to give you a full picture of your customer and their journey.
Pro Tip: Generally we recommend making this ID a unique identifier which never changes such as a userId in your database. If you use email as a unique identifier for a user ID, you’ll run into issues when a user changes their email.
Additionally, you’ll want to assign a unique, temporary identifier for any user who isn’t logged in. You may do this via localstorage, a first-party cookie, or a device identifier. But you need some way of tying together a user’s path from first engaging with your product to when they actually convert.
At Segment, we solve this problem via the
identify call. An
identify lets you associate a user to their actions and record traits about them. It includes a unique User ID and any optional traits available like email, name, location, etc.
So when should you use an identify call? We recommend making an identify at at key moments such as:
The first three examples are pretty self-explanatory, but you might ask: why you would call identify on every page load if we’re storing the userId in the cookie?
Imagine this scenario: A user logs into your app. Identify is called. For whatever reason, the user closes the browser and does not return until later. There’s no way of knowing where that user will re-enter your app from. They could start my session from anywhere. And because there are many tools out there that require an initial identify call for certain features (e.g. Intercom chat widget), it’s important to tell your end tools who the user is when they first start their session.
Once you establish what actions to track and can identify who is taking those actions, you can combine the two to get a real sense of what is driving your core metrics.
To help you get started, we wanted to share a few tracking examples you can use. Below, you’ll find top events that we see today at Segment. Some of these are simple funnel metrics (how many users viewed a page), while others are transactional (who completed an order).
Events worth tracking by industry:
In addition to tracking events, you’ll want to also capture properties (the what) associated with your tracked events. Here’s an example of properties for a Signed Up event which is applicable across pretty much all industries.
And an example of a track call for iOS:
Because event tracking and creating a tracking plan from scratch can be a somewhat new concept for teams, we've developed sample tracking plans for a variety of industries and use cases.
Basic Tracking Plan This is a simplified version of our tracking plan to help get you started. We recommend starting with a plan like this before digging into more complicated tracking.
Full Tracking Plans We have also created industry specs to help our customers get started (and also for our own Segment tracking).
This may seem like a lot, but creating a tracking plan is worth it. The more you plan today, the better your data will be in the future!