Go back to Blog

Growth & Marketing

Sherry Huang, Caitlyn Sullivan on March 31st 2021

It’s time to say hello to the next era of digital advertising, powered by first-party data.

All Growth & Marketing articles

Katrina Wong on June 10th 2021

Joy Mangano shares her journey from QVC to omnichannel retail.

Katrina Wong on May 13th 2021

Personalized campaigns and service require customer data plus tools to leverage that information. Sara Varni, Twilio CMO, explains how businesses can do this.

Kevin Garcia on May 11th 2021

Increase loyalty and revenue by personalizing in-store pickup experience.

Andrew Jesien on April 28th 2021

Prepare for the world of first-party data with Facebook Conversions API and Segment.

Enzo Avigo on April 5th 2021

How do product managers get started with analytics? In this guest post, learn how to track the most important product metrics using Segment.

Sherry Huang, Caitlyn Sullivan on March 31st 2021

It’s time to say hello to the next era of digital advertising, powered by first-party data.

Shirley Javier on February 26th 2021

This a guest post from Shirley Javier, Product Manager at Taplytics, the mobile optimization platform. Try Taplytics out on the Segment platform today.

Building an effective experiment is one of the most powerful strategies you have at your disposal. So much of your company’s success depends on how you evolve alongside the market—being able to test out what works for your target audience is critical.

But creating effective A/B tests requires a deep understanding of both your product experience and your customer expectations. That understanding comes from a strong base of customer data and team insights. Without those two things, striking a balance between optimizing the customer experience and furthering your business goals is impossible.

Following A/B testing best practices helps you create experiments to improve conversion rates, connect with customers, and build out an engaging experience with your product, without negatively impacting customers or your business along the way. That makes it easy to learn more about what drives user engagement while adding value for your business.

1. Figure out what to test

Useful A/B tests are specific. Before you start building experiments, your team needs to determine what aspects of the product or user experience benefit most from experimentation. Defining the test parameters first helps you move forward, knowing you’ll make a real impact on customers.

Map out the customer journey to help your team get started. A customer journey map enables you to identify areas of improvement based on what aspects of the experience cause the most friction for users. Categorize what happens at each stage of the customer journey into:

  • Activities: what people do at various stages

  • Motivations: why people perform those actions

  • Emotions: how people feel as they perform those actions

  • Barriers: what stops them from taking action

Barriers will be the first thing you want to look at, because they are the most suitable for effective A/B testing. Removing these barriers makes it easier for users to accomplish their tasks, which leads to a better overall product experience. The easiest way to define these elements is to use your customer analytics. This data helps you pinpoint touchpoints along the journey, based on real-world customer experiences.

Understanding the typical paths customers take to learn more about your product, make a purchase, and engage with your team is the best way to identify what aspects of their experience need work.

2. Tie experiments to specific KPIs

Once you have a handle on the aspects of the product/user experience you want to test, define the key performance indicators (KPIs) you’ll use to track and analyze the impact of these experiments. A/B testing is only as powerful as the metrics you use to track your success. Without specific metrics, it’s impossible to categorize your results.

Choosing the right metrics helps set a baseline for your KPIs as well, which you’ll use to track the evolution of subsequent experiments over time. Outlining changes to these metrics make it easy to understand the impact a specific testing variant has on your hypothesis.

Analyzing these KPIs is one of the most high-leverage things you can do to refine the experimentation process because it helps you nail down exactly what works and what doesn’t. Especially when you pair that analysis with a well-defined A/B testing goal, these goals help you connect the impact of each experiment to your overarching business objectives.

The best way to do this is to formulate your testing hypothesis as a SMART goal. These goals are specific, measurable, attainable, relevant, and time-bound, so they provide your team all the context they need to understand results.

3. Leverage good data

Data integrity is a fundamental element of any successful experiment. Without good data, it’s impossible to track the real-world results of your A/B tests. Customer data platforms like Segment help your team build out their experiments with a solid basis of clean and actionable data.

With the Taplytics/Segment integration, you ensure parity between both tools and save your team a lot of time analyzing results. Information automatically syncs between the two platforms, so any data you’ve collected with either tool can be used to build out your experiments.

Example conversion rate experiment in Taplytics

Let’s say you’re running an experiment to track the conversion rate for your newsletter subscription landing page. If you have two plans set up, basic and premium, you can use that information to build out your test variants and manage these tests in Taplytics. That helps you narrow down specific metrics and KPIs to track on a per-experiment basis.

4. Target the right audience

A/B tests need to target a specific set of customers. Proper targeting not only makes it easier to define what changes to make your testing variants, it also gives you the ability to tailor the experience to specific sets of users and their needs.

Using Taplytics, you can easily define custom user attributes based on a variety of characteristics between Taplytics and Segment. As a bi-directional integration, it’s easy to pass these parameters between the two as well. Let’s say you have a subset of users who signed up for an account using the same landing page. By analyzing your customer data in Segment, it’s easy to identify what similarities these people share and how those characteristics impact their website experience.

Using that information, you can create a targeted testing variant for your landing page and serve it directly to website visitors that share these characteristics. A properly targeted A/B test has a better potential for positive results and makes it easy for you to see the impact of your experiments on a deeper level.

Targeting gives your team the context they need to validate certain assumptions about your target audience and what encourages them to take the desired actions on your site.

5. Create unique test variants

A/B testing takes time and resources to set up correctly, so it’s important that each test variant you make has the potential to make a significant impact on user behavior. While minute changes can help you refine experiences over time, more substantial changes make for more straightforward analysis and tracking at scale.

Unique test variants are also easier for users to differentiate. For example, changing the placement or size of a CTA on your site is potentially more impactful than merely changing the shade or color. Keep in mind that any A/B test should only change one aspect of the experience at a time. That’s not to say you can’t have multiple variants, just that it’s essential to keep variations to a single element. Otherwise, it’s difficult to nail down your results.

A/B test variant example via Taplytics

Keeping test variants unique also helps you keep track of which aspects of the experience have the most impact on your customers. Let’s say you’re testing out a transactional push notification. Changing the headline text in each variant is an A/B testing best practice because it gives you an easy method of tracking the impact of different languages on your target audience.

Refining this language over time helps you create a better overall notification. It helps you refine the language you can use on the rest of your site as well.

6. Schedule tests for the right time

A/B tests need time to gather enough data to either prove or disprove your hypothesis—two weeks is the recommended A/B testing best practice. As you build out experimentation processes at your company, include guidelines for both the scheduling and duration of tests. This standardizes workflows for your team and helps them build individual tests more efficiently.

Use customer data to understand the best timing for your target audience. If there’s an upcoming product release, sale, or promotion that will increase website traffic, scheduling your tests to coincide with those times is a great way to increase engagement with specific test variants.

As you schedule these tests, it’s also important to consider what aspects of the experience you plan to change. You don’t want to run into issues by changing primary product functionality during high usage times. And make sure you don’t run concurrent tests on individual aspects of the product or user experience either. This could skew your results and confuse your user base with too many changes in a short period.

7. Understand the statistical significance

Statistical significance tells your team that the outcome of each A/B test is actually a result of the experiment you created, and not just a random shift in customer behavior. These complex calculations look at the experiment’s result and measure how confident you can be that the data was a direct result of whatever you changed during the experiment itself.

Statistical significance in the Taplytics/Segment integration

Segment’s Taplytics integration calculates statistical significance automatically, to a confidence level of 95%. This takes a lot of the complex mathematical workload off your team and helps them gain a better understanding of the results. By proactively removing any uncertainty in the process, your team can act on the outcomes and make decisions faster.

An A/B testing dashboard keeps track of each experiment’s long-term impact in reference to other tests as well, which helps you refine the way your team builds tests with each iteration.

8. Share results with your team

Tracking the overall impact of various experiments is helpful only if you communicate testing results with your team. Highlighting successes and areas of improvement in this way boosts engagement with the A/B testing process and helps increase visibility into important aspects of the product and user experience.

When you increase visibility into these experiments, it holds the team accountable for the work they do to create and run your A/B tests. That helps everyone involved feel connected to the results and invested in the experiments they create. When you get multiple perspectives on the test results, it also builds a shared sense of commitment to the process and helps you brainstorm more diverse solutions to customer problems.

The bi-directional integration between Taplytics and Segment makes sharing those results easy by housing data on either platform. If you have a dashboard set up in Segment, you can share those results directly, or vice versa. This data parity ensures that, regardless of your team’s preferred tool, all A/B testing results are clear and easy to understand.

9. Apply changes through a phased rollout

Once you’ve run your A/B tests and determined which of the variants is more successful, it’s time to roll out those changes to your entire user base. A phased rollout mitigates the risk of implementing those changes by providing more control for your team.

Phased rollouts are especially important for changes that have a direct impact on the product experience. Let’s say your experiments were designed to make it easier for new customers to move through the onboarding process. Once you’ve determined what changes to make, pushing those updates live to the customer is the crucial next step

Using a phased rollout schedule helps you make these changes slowly for your target audience and decreases the potential strain on your infrastructure as well as your team. Whenever you make a change to core functionality, it’s important that you ensure the smoothest transition possible for your users. And bumps along the way can have a negative impact on your relationship with them.

A/B testing best practices help you build experiments at scale

Designing, implementing, and analyzing your A/B tests is a complex process involving a number of moving parts. As your company grows and your team expands, your approach to experimentation needs to mature as well. Following A/B testing best practices helps you build experiments more efficiently as your company needs evolve and helps you create and analyze experiments that have a real impact on your users as well as your business goals.

Kelly Kirwan on February 24th 2021

How StackCommerce used Segment to improve attribution accuracy

Geoffrey Keating on February 23rd 2021

Marketing automation software is quickly becoming a priority for competitive companies—businesses that are looking to raise customer satisfaction, boost ROI, and work more efficiently. 

But not all marketing automation software is created equally. Before you choose a tool, assess your software needs and the goals you want to accomplish. Then, evaluate the marketing automation software landscape so you can choose the best tool to help you reach your marketing goals.

For your marketing automation efforts to be successful, your data needs to be well-managed. Bad data can lead to misguided engagement efforts and frustrated customers. A customer data platform like Segment will power your marketing automation software with clean, organized data.

Table of Contents:

Familiarize yourself with the basics of marketing automation software

Before choosing marketing automation software, you’ll need to understand the basics of these tools and why you might want to use one. Then you can decide if it’s time for your team to invest in a marketing automation tool.

What is marketing automation software?

Marketing automation software is a tool that streamlines your digital marketing activities and delivers targeted messages to your customers and prospects based on their behavior and preferences. It helps marketers acquire new customers, nurture existing relationships, and analyze campaign performance across channels.

Marketing automation software functions by tracking user actions on your website, app, digital product, email campaigns, and other events. Then, this behavioral data triggers targeted content via email, SMS, push notification, in-app messaging, social, web, and more.

Using customer behavioral data, marketing automation software can help you create curated experiences—displaying a different landing page to a returning customer, delivering ads based on recent browsing history, or sending an email introducing products related to the customer’s interests, to name just a few.

Personalizing the customer experience can get complicated with so many customer interactions across channels and devices. But, with the right marketing automation software, you can track customer behavior and then deliver targeted customer experiences at scale while reducing the amount of time you spend on marketing processes.

Why use marketing automation software?

Customers want personalized experiences. According to Epsilon, “80% of consumers are more likely to make a purchase when brands offer personalized experiences.”

But, it’s not as simple as sending a follow-up email after a purchase to advertise products that are similar to the purchased product. You need to be able to anticipate what a customer would like to buy, even if they haven’t shown a direct interest in that product.

In a consumer survey by McKinsey, one shopper noted that after buying a puffy jacket, she received an email recommending similar jackets. “‘Regarding this product, you only need one. Why send an email for other similar coats?’ she asked.”

This company would be better off recommending winter boots or a scarf that reflects the customer’s style based on their first purchase. To deliver this level of personalization, companies need a tool that uses complex algorithms to make smarter campaigns and foster relationships by providing value to customers.

Through marketing automation, you can give a personalized and relevant experience to your customers while saving time by automating the process. You can put the time gained back into your marketing strategy development and leave repetitive tasks to the robots.

Assess your marketing automation software needs

Marketing automation platforms are not a cheap investment, so you want to make sure you pick one that meets your company’s needs. Consider these three questions to develop your criteria to find the best marketing automation software for your business.

What are your goals?

By knowing your goals, you can make sure the software you choose has the marketing automation features, functionality, and reporting you need to succeed. Some goals that marketing automation software can help with are to:

  • Boost customer LTV

  • Acquire new customers via inbound marketing

  • Increase upsells and cross-sells

  • Raise customer satisfaction

  • Conversion rate optimization

  • Elevate ROI

Mention your goals to marketing automation software sales reps when you take a product demo. They can share relevant success stories and show you features that will help you reach your objectives.

Look for reporting features in tools, so you can find one that provides the insights you need to track your progress toward your goals and improve your marketing workflows.

What other marketing tools do you need to integrate with?

While many marketing automation platforms will cover similar channels, each will likely integrate with different marketing tools. Find a marketing automation tool that connects with your existing marketing and engagement tools, such as your:

When you take a demo, ask to see these integrations in action, so you can envision how they will work for your use cases.

How much control do you want over your marketing automation flows?

If you already have marketing flows in mind that you want to automate, make sure the tool you choose will be able to react to specific user actions or a series of actions in the way you want it to.

You’ll likely want to develop multiple series of complex triggers based on different customer journeys and actions. Bring these flows up in the demos as well.

Most marketing automation software can handle complex marketing flows for SMBs. Enterprise businesses will benefit from the greater flexibility provided by more comprehensive marketing automation tools that help companies fine-tune their marketing orchestration.

Review the best marketing automation software in 2021

Now that you know what you need from marketing automation software, consider these top marketing automation tools. Bonus: They all integrate with Segment, so you can power your marketing efforts with clean, standardized data.

1. HubSpot

HubSpot is an all-in-one SaaS marketing automation tool that will help you attract new leads and turn them into customers.


Features include email marketing automation, landing page creation, analytics, lead scoring, and a built-in customer relationship management (CRM) system. HubSpot also has plenty of educational materials and strong customer support.

Best for: CRM and lead generation

2. Autopilot

Autopilot is a marketing automation tool focused on helping small businesses create engaging customer journeys.


Build lifecycle marketing campaigns with an easy drag-and-drop interface. Autopilot includes A/B testing, customizable CTAs, landing pages, and lead scoring. Autopilot even offers annotation and collaboration features right on the platform, so you can add context to your customer journeys and review with your marketing team. Dozens of integrations mean you can create a powerful marketing tech stack.

Best for: Lifecycle marketing

3. Iterable

Iterable is a marketing platform that prides itself on integrating your customer data and then using it to create targeted campaigns.


A/B test campaigns with up to 50 variations and automatically implement the most successful variant. Iterable also helps you easily test messages on different email clients, which can be tricky to get right. Plus, they offer a clean visual interface for creating campaigns.

Best for: Lead nurturing and customer retention

4. Intercom

Intercom allows you to analyze customer behavior on your site and communicate with customers 1-to-1 in real time or via automated campaigns.


Use Intercom to guide new users through an onboarding process and streamline customer support. In addition to robust customer service features, Intercom also has powerful prospecting tools for lead generation and nurturing, A/B testing, and the ability to customize CTAs.

Best for: Customer service, lead generation, and acquisition

5. Pardot

Pardot by Salesforce is a comprehensive marketing automation platform ideal for helping B2B enterprise businesses identify and engage high-quality leads.


Track pageviews, form fill out, downloads, and social interactions. Create dynamic emails right from the platform. More features include the ability to deploy landing pages, manage social media marketing, and integrate with Google Ads to track ROI and keyword performance.

Best for: Lead generation and engagement

6. Taplytics

Taplytics lets you target users across platforms to ensure a consistent experience.


Design complex user journey campaigns to engage customers with the intuitive drag-and-drop interface. Perfect your campaigns with A/B testing and roll out new features and digital products with feature flags. Taplytics prides itself on working with businesses 1-on-1 to make sure you get the most out of the platform.

Best for: Lifecycle marketing for digital products

7. Extole

Extole is a referral marketing tool to turn your customers into brand evangelists.


Encourage your current fans or subscribers to bring in new customers by rewarding them with Extole’s wide range of advocacy products. Enterprise businesses will benefit from advanced features like fraud protection, A/B testing, APIs and webhooks, and the sophisticated reward engine.

Best for: Referral and loyalty programs

8. Customer.io

Customer.io is designed to help companies reach customer activation and retention goals with a focus on automated messaging.


With Customer.io, you can create multiple branches in a single campaign to customize messages based on conditional statements, user attributes, or segments. Enhance your efforts with comprehensive reporting and A/B testing for more relevant and personalized customer interactions.

Best for: Customer activation and retention

9. Drip

Drip is primarily a B2C ecommerce marketing automation platform that allows you to send personalized messages to customers and leads.


Segment customers by tag, event, and even custom fields, so you can understand how different customers are interacting with your company. Create messages with dynamic content to personalize the customer experience with things like product recommendations and unique discount codes.

Best for: Customer acquisition and retention

10. Marketo

Marketo is another all-in-one solution ideal for B2B marketing that helps align marketing and sales teams.


Use Marketo for lead management, email marketing, and predictive web content. Marketo organizes leads for sales reps and provides context for them to understand how a lead has interacted with your website and marketing campaigns. Offering a variety of pricing plans, Marketo is a viable option for nearly any B2B company.

Best for: Lead generation and engagement

11. WebEngage

WebEngage is a multichannel B2C engagement platform.


WebEngage supports real-time segments, so you can engage customers based on their behavior. They offer pre-built templates for easy campaign creation. Use funnel and behavior analysis to better understand the effectiveness of cross-channel marketing campaigns throughout the customer lifecycle.

Best for: Lifecycle marketing

12. Blueshift

Blueshift automates behavior-based messaging across many marketing channels, including email, push notifications, Facebook, and display ads.


Use Blueshift’s behavioral segmentation to identify users who are more likely to perform actions like a repeat purchase, activation, or churn. Then, create automation workflows based on those segments to get more customers on a successful conversion or retention path. Campaigns in Blueshift are self-optimizing, so you’re always getting the best results from your efforts.

Best for: Segmenting and multichannel messaging

Supercharge your marketing automation with a customer data platform

A marketing automation tool will provide synergy between your marketing efforts and your customers’ actions and preferences. Amplify your marketing automation platform’s capabilities even further with a customer data platform like Segment.

Segment gathers, cleans, and standardizes data from all your data sources and sends it to your marketing automation platform to help you create hyper-personalized campaigns.

Segment even has pre-built infrastructure, so you can test different marketing automation tools with just a few clicks before choosing one. See how Halp used Segment to test and select a marketing automation solution and then improved their customer onboarding flow and increased activation by 4x.

Become a data expert.

Get the latest articles on all things data, product, and growth delivered straight to your inbox.