Protocols creates clean, reliable data that Creative Market can trust

Results

The big win

Creative Market, an online marketplace for community-generated design assets, had inconsistent data proliferating across downstream marketing and analytics tools which often rendered measurement and analysis inaccurate.

The company turned to Segment Protocols to develop an actionable Tracking Plan that can validate and enforce customer data before it is distributed downstream. With Segment, Creative Market reduced detection time of data issues by 93% and preserved engineering resources for core product development.

Founded in 2012 in San Francisco, Creative Market is an online marketplace for community-generated design assets. Web creatives use Creative Market to source graphics, stock photography, fonts, WordPress themes, and other digital goods for personal and commercial use. Creative Market has over 250,000 purchasable items designed by independent creators around the world and submitted through user “shops.”

Spending too much time detecting and resolving data quality issues

Creative Market found its engineering teams bogged down by reactively detecting and resolving data quality issues rather than spending their time building and innovating. Additionally, when dirty data made it to production, it took up to two weeks to realize there was an error. This meant that two full weeks or more of data could be inaccurate, invalidating any tests the growth or data science team was running. As a result, Creative Market’s only option was to convince a data engineer to manually clean the data or start the entire analysis from scratch.

In addition, the Creative Market teams had no shareable, easy-to-update tracking spec. In searching for a solution, it was important that Creative Market could govern their data at the point of collection instead of reactively addressing issues in every tool they were using.

Akshay Singh, a Growth and Product Analyst for Creative Market, said, “Finding implementation issues really slows down our growth team. For instance, we’ve implemented events incorrectly and then used them to measure the results of A/B tests. Once we identified the data was wrong, we had to go back, fix the data, QA test the code again, and rerun the test. It’s a huge waste of resources.” Singh felt like there had to be a better way.

Preventing and detecting data quality issues with Segment

Creative Market has been successfully using Segment's data integration product, Connections, to capture, unify, and integrate customer data for years. Creative Market was thrilled to learn that Segment had a technology-based solution to help automate their data quality processes: Protocols.

Protocols is a product from Segment that can prevent and detect data quality issues before they hit marketing or analytics tools and warehouses. First, Protocols enables companies to align their customer data strategy around a centralized and actionable Tracking Plan stored in Segment. Next, Protocols automatically validates that all data sent to Segment adheres to the Tracking Plan. Finally, Protocols helps enforce the Tracking Plan by automatically blocking non-conforming events to keep the analytics tools and warehouses clean.

Protocols offers Creative Market the opportunity to develop an actionable Tracking Plan that can validate and enforce their customer data before it is distributed downstream. “Protocols has become the place that I can maintain a centralized data store. This means our Tracking Plan can actually be used as a reference by other teams,” said Singh.

Protocols reduces the detection time of data issues by 93%

Prior to Protocols, it took Creative Market weeks to detect data quality issues in their marketing and analytics tools. With Protocols, Creative Market receives alerts whenever invalid or unplanned data is detected, so they can quickly resolve any issues before they impact production. With Protocols, Creative Market reduced detection time of data issues from two weeks to 24 hours or less.

Using Segment Protocols, Creative Market has successfully preserved three engineering teams’ time for core product development instead of troubleshooting and manual implementation. Protocols’ Data Validation Report automatically flags implementation issues, ensuring clean, reliable data.

By using Protocols to create a centralized and enforceable Tracking Plan in Segment, all teams at Creative Market benefit from having a clearly documented spec that is easy to utilize and follow. That means they can create their own analyses without double-checking what data means across teams.

“Having data we can trust thanks to Protocols benefits the data science, analytics, product, and marketing teams because we all use Segment data for our analyses,” said Singh.