Connect Algolia A/B tests to your data warehouse and analytics tools

The search-as-a-service platform, Algolia, provides out-of-the-box analytics for A/B testing, offering insight into conversions, click-through rates, and the average click position. But, you often need to visualize this search data in other tools to gain more information on average cart size, customer context, user behavior, and so on.  

In this recipe, you’ll learn how to pipe Algolia’s A/B test data to your data warehouses and analytics tools to measure different test metrics and run further analysis.

Note: while this recipe is focused on retail and e-commerce, it can be applied to any industry. 

What you’ll learn

This guide will teach you how to expose the variant test ID and test number from the Algolia response (either through the API client or InstantSearch React library) and pass that information as a property to your track calls. 

(This recipe assumes you’ve already set up Algolia analytics tracking through Segment – here’s how.)  

Step 1: Enable getRankingInfo in the Algolia search request

Algolia’s getRankingInfo parameter provides more detailed information about the search request in the search response payload. It exposes deeper contextual information, like details about the ranking formula as well as the abTestID and abTestVariantID. 

abTestID (integer): In case of A/B testing, returns the ID of the A/B test.

abTestVariantID (integer): In case of A/B testing, returns the ID of the variant used. The variant ID is the position in the array of variants (starting at 1).

When an A/B test is enabled on the Algolia index, these two properties will be visible on any search to that index when getRankingInfo:true. Algolia will lock that specific user into the variant test group for the duration of the A/B test based on the user’s IP or userToken (if provided). 

The IP and userToken will not persist across sessions or devices, and is subject to how the Algolia implementation was conducted. For that reason, it may be prudent to only pass the test variant through track calls rather than identify calls (as the latter would assume the user is permanently part of the test variant across sessions and devices). 

There are a few ways to turn on getRankingInfo in Algolia, but the simplest, low-code approach is to leverage Algolia’s Rules feature. 

Using Rules:

1. Enter the Rules interface to add a new manual rule. (Make sure you’re on the index that you’ll be running the A/B test on.)

2. Select the trash can icon in the top right corner to remove the condition. This will create a condition-less rule that will be applied across all searches to that index. 

3. Select “Add Consequence” and then “Add Query Parameter.” (Adding a query parameter will append the object to every search request.)

4. Add the following code block to enable getRankingInfo:

{

  "getRankingInfo":true

}

5. Hit save.  NOTE: turning on getRankingInfo can increase the size of the payload by roughly 10%. Double check your usage capacity or reach out to Algolia support (support@algolia.com) to ensure this won’t be an issue.

If you want to append getRankingInfo through code, you can simply add it into the search request method (if using the API client) or through the Configure widget if using InstantSearch React.

Here is an example of passing in the getRankingInfo through the search method using the Javascript API client. 

results = index.search('query', {
  getRankingInfo: true
})

Step 2: Enable Algolia’s A/B test

Algolia has a great guide on how to set up and run an A/B test here. 

The one thing to keep in mind is that if you’re running an A/B test with different indices, “getRankingInfo:true” needs to be enabled for every applicable index. 

Step 3: Find the A/B test variant in search state or search response and pass through Segment (low code)

If you’re using the API client directly, you can likely conduct a search request and use the response object to build your front-end components. Once getRankingInfo is enabled, you will see the A/B test variant information in the root of that search response: 

{
  [...],
  abTestID: 123,
  abTestVariantID: 1
}
}

The two listed above are what you’ll be passing into all your track events as a property. 

For example:

{
  "type": "track",
  "event": "Product Clicked",
  "properties": {
    [...],
    "abTestID": 123,
    "abTestVariantID" : 1
  }
}

Notes: 

• Mind your types here! Algolia will return the values as integers. 

• For future proofing, pass in the A/B test variables conditionally to avoid errors when the test ends. 

This step should be done for every track call relevant to the A/B test.

(If you’re using React InstantSearch, you can find this information in the StateResults widget or by digging into the InstantSearch props.) 

Step 4: Use the data

You can either use this new event data within Twilio Engage (formerly Segment Personas) by creating audiences and custom traits, or in your downstream data warehouses and destinations.

The track calls being sent will contain the A/B test number and variant ID within the properties. 

Wrapping up

Here’s a quick recap of what we've done in this recipe:

• Exposed Algolia’s A/B test variant information in the search payload using Algolia Rules.

• Initialized an Algolia A/B test.

• Streamed that information through Segment track calls.