How Pomelo delivers one-to-one recommendations at scale with Twilio Segment and Amazon Personalize

Results

The big win

With over 8,000 products Pomelo needed to improve product discoverability and deliver relevant product recommendations to its increasing number of daily visitors. However, current infrastructure was rigid and outdated and would need a large engineering effort to build one-to-one personalization it needed at scale.

Pomelo turned to Segment and Amazon Personalize to deliver near real-time personalized content within iOS, Android, web, and via Braze. Personalized content resulted in increase in gross revenue and overall product engagement.

Founded in 2013, Pomelo Fashion is the leading omnichannel fashion brand in Southeast Asia and offers a range of stylish, affordable clothing for women globally. Pomelo strives to provide innovative experiences for customers across channels, including its website, mobile applications, brick and mortar locations, and in-store electronic kiosks that offer a self-serve ‘click-and-collect’ in-store service.

Rebuilding a personalization engine for scale

Pomelo’s global e-commerce business experienced rapid growth over the last four years. With nearly 8,000 different products on their site, the team struggled to manage product discoverability and provide relevant recommendations to the increasing number of new visitors. Shane Leese, Business Intelligence Director at Pomelo, explains, “Nearly six years after Pomelo started, we were still using virtually the same algorithm we built in-house. It ranked the products shown to customers based on overall page views and purchases by geo, delivering the same generic product recommendations to every customer in a given country.”

It would require significant engineering effort to update the algorithm and rebuild the infrastructure to collect, transform, and stream clean event data to power the models. With a small team and limited resources, Pomelo started researching external solutions to deliver more scalable one-to-one personalization.

The requirements: 

1. Rich, clean customer data to train models 

2. Live event streams to deliver real-time recommendations

3. Reliable and customizable algorithms

4. Track and attribute results to show quick, demonstrable gains 

5. Toolset to enable frequent experimentation to serve the short attention span of the market

Establishing a foundation with clean event data and reliable testing environments

Before tackling the algorithm updates, Pomelo started by building the foundation of its personalization engine. “We chose Segment to power our models with clean and reliable event data. Segment not only gives us confidence in our testing environment as we tweak and train our models but provides a better understanding of our customer journey and behavior,” explained Shane.

With a solid foundation, the team chose to couple Segment’s Customer Data Platform with Amazon’s machine learning service, Amazon Personalize, which enables developers to build real-time personalization into applications with the same machine learning technology used by Amazon.com. “We were already running on AWS when Segment had just announced the integration with Amazon Personalize, so it was an opportunity to try something new and test it out quickly,” said Shane.

To test, Pomelo started with a personalized-ranking model focused on a single product category. Pomelo powered the model with funnel events collected by Segment like product clicks, add to bag, wishlist, and purchase. 

After the initial test proved successful, Pomelo started leveraging larger data sets combining both product and customer metadata such as product tags, prices, discounts, quantity, and customer segments engaged. By powering the models with additional product and customer data, Pomelo could more accurately map relevant products to customers. With more confidence in the outputs, Pomelo expanded to all 350+ categories such as “Blouses”, “Tees”, and “Sleepwear”. Pomelo displayed these near-real-time recommendations within the ‘Just for You’ carousels on the app feed and individual product pages.

Adam Kirk, Pomelo’s Director of Product Management, explains the transformation, “We went from a system of batched, general recommendations, to a system of one-to-one recommendations based on your actions in near real-time. It’s so fast that you can even see your recommendations changing during the same session. Your category pages and recommendation layers could look different based on the behavior you’re showing during that session, such as clicks, your wishlist, or what you’re adding to the bag.”  

Balancing recommendation relevancy with data streaming costs 

Shane explains how the team trained and tweaked the new predictive models to improve effectiveness, “At first there was a period where we were trying to get as much data in as possible. We would do that by either adding to event payloads coming in from the apps and website via Segment and making sure the metadata attached to each event was robust or making changes to the lambda code to make sure it was all streaming into Personalize.” 

As the team continued to test, they experimented with the number of refreshes to the algorithm as they tried to balance relevancy with retraining costs. “We learned there was a sweet spot to retraining the models with diminishing returns of consistently streaming data. We decided upon a twice-a-week refresh on the model, timing it with new product drops, which are the most trafficked days. By doing these refreshes, it would always give a slightly different output to the user, even for those not showing new behavior between the refresh,” explained Adam. 

Pomelo is now testing new models for the 'Just for You' carousel. As it uncovers new learnings, the team will apply them to category pages to see additional revenue gains. Pomelo has also rolled out a new recommendation on iOS, Android, and Web showcasing relevant products within ‘Shop this Style’. “We’re using a Personalize model to build a more comprehensive look recommendation. So if you're viewing a top, you might see earrings, shoes, and pants to complete the look,” said Adam. After optimizing the model, the team saw a 1.5% lift in overall revenue, the highest revenue-generating recommendation on the product page.  

Building the architecture: Segment, AWS Lambda, AWS Personalize, Braze, Amazon Redshift, and Looker

Customer data from Segment and product data in AWS Redshift power Pomelo's personalization engine. Segment sends real-time customer event data to AWS Lambda into Amazon Personalize. Then recommended merchandise and personalized experiences display across Pomelo’s digital channels based on each visitor's profile and actions. Customer data is also sent to downstream tools like Pomelo’s email provider, Braze, to deliver personalized emails. Last, Pomelo uses Looker on top of Amazon Redshift to analyze the relevance of its models and report on their business impact.

The future of personalization at Pomelo

Using Segment, Amazon Redshift, and Looker as its attribution framework, the team could prove enough ROI to expand testing across the product. “Using Looker to monitor the robust data we were sending with our events from Segment was pivotal to our success, especially during the early days of COVID. We were able to report to executives on the effectiveness of our models and directly attribute them to an increase in revenue,” explained Shane.   

Pomelo plans to test using the recommendation model to support merchandising challenges due to COVID. “A large number of our products don’t have the most common sizes in stock right now, which is a frustrating user experience. We’re looking at how we can take previous customer events and predict their size to prioritize relevant in-stock products first. We hypothesize that while this will lower overall click-through rates, each click will be more valuable,” said Adam.

Increasing revenue and product engagement with predictive recommendations

Pomelo saw up to a 15% increase in revenue on personalized content and an 8% gain in incremental gross revenue after implementing Segment and Amazon Personalize. “We’ve seen a massive improvement in not only revenue generated, but also customer loyalty,” said Adam. Pomelo also benefits from improved customer engagement, with 60% of all product clicks now coming from personalized content. And most recently,  ‘Just for You’ and ‘Shop this style’ recommendations delivered a 50% uplift in overall product engagement.

With the flexibility and scalability of Segment, Pomelo has future-proofed its data architecture and can quickly stand up new solutions and use cases. Shane explains, “I can’t imagine having gotten this project off the ground without Segment. By using Segment to stream data, we saved significant developer time as we didn’t have to build a lot of infrastructure to collect, clean, schematize, and load the data into Personalize ourselves.” 

Segment allows Pomelo to leverage best-of-breed tools such as Amazon Personalize, Braze, and Looker to deliver a differentiated customer experience across channels.

The benefits of Segment and AWS Personalize

• Increased revenue up to 15% by displaying personalized content on category pages.

• Unlocked an 8% increase in incremental gross revenue after implementing Segment and Amazon Personalize on all category pages. 

• Saw a 50% uplift in overall product engagement after implementing predictive product recommendations on category pages, ‘Just for You’, and ‘Shop this style’ 

• Increased add-to-cart clicks from category pages by up to 16%.

• Increased return on investment by 400% within one month of using Amazon Personalize and Segment to power the ‘Just for You’ recommendation carousel.

• 60% of all product clicks are generated from Segment and Amazon Personalize–fueled recommendations.