By Brennan Gamwell
Engineering Product Manager @ Segment
In a phrase, a lot.
Understanding user behavior via event tracking is a complex, choreographed dance among Product, Analytics, Marketing, and Engineering teams.
Product sees data as a means to innovate. For Analytics, the sky’s the limit when they can derive data-driven insights. Marketing’s benchmarks vary from optimizing the customer lifecycle to return on ad spend. And Engineering (perhaps being overworked) look to circumvent data integrations altogether to focus efforts on building the product.
When data collection is done right, each of these teams reap the benefits. Doing so, however, is much easier said than done.
After surveying Segment customers, we’ve found that gathering customer data is no small task. On average, a mid-sized company (200-1000 employees) dedicates 6-8 weeks per year to instrument a new data integration and 2-3 additional weeks to maintain it.
On top of that, there’s rarely a use case where data is collected from a single source and routed to a single tool. So every new source of data collection (web, mobile, server-side, etc.) and every tool where that data needs to be delivered (analytics, email marketing, ad platforms, data warehouses, CRMs, etc.) compounds in time and resources needed for instrumentation.
As you can imagine, this is not ideal when your team is trying to innovate and grow as fast as possible.We’ll lay out a better way to collect, unify, and act on your data in this chapter. But first, let’s take a closer into the obstacles that come up time and time again.
Implementing side-by-side data integrations with unique APIs introduces two unnecessary challenges:
On the whole, this doesn’t make much sense. Each new tool you implement relies on the same data, so why would you instrument tracking for each tool separately?
Imagine spending 6-8 weeks setting up a new tool, only to learn that the initial implementation included a crucial logic error.
Mistakes like this require not only more time to fix the issue, but also necessitate that code be re-deployed, and potentially, past data to be sanitized. Further, if your point of data collection is a mobile app, you may be cursed with bad data forever — there’s no way to force a user to upgrade to the latest version of your app with the correct SDK.
Opportunity costs pile up. Instead of working on new features, engineers spend an increasing amount of time building and maintaining data integrations.
Let’s think through the math. If an engineering team implemented 5 new data integrations, they would need to allocate between 30 and 40 weeks and between 10 and 30 additional weeks per year to maintain it. In the first year, that’s between 40 and 70 weeks of employee time. Imagine hiring a team member and dedicating a substantial portion of another team member’s time just to writing and maintaining data integrations.
The negative effects of opportunity costs inevitably cascade to other teams:
Each team in your company relies on a different source of truth for customer data.
Your sales team’s go to is a customer relationship management system (CRM), your success team to a help desk, your marketing team to a data management platform (DMP) or customer data platform (CDP), and your analytics team to a data warehouse.
When data flowing to and from these tools is not consistent and updated it leads to an incomplete picture of your customer and variance in key metrics driving your business. On top of that, it often results in teams spending more time arguing about whose data is right than they do putting it to use.
Clearly, there’s a problem afoot — either allocate valuable human resources to maintaining data integrations or force teams that rely on that data to operate on gut feeling rather than being data informed.
So what’s the alternative?
It just so happens that, many of the unique APIs needed to instrument tools for analytics, A/B testing, advertising, and other categories covered above, operate using much of the same data — clicks, page views, video views, purchases, etc. So it doesn’t really make sense to instrument each and every tool you use individually on your website or app.
The benefits of leveraging a solution like Segment are many:
Let’s dig in with even more specifics of how a customer data infrastructure (CDI) solution like Segment can provide out-of-the-box solutions to common intra- and cross-team challenges.
Instead of addressing data collection, connectivity, and access issues in a case-by-case fashion, a CDI addresses data collection at the source. A CDI unifies APIs across the tools your teams use, allowing Engineering teams to collect data once and route it to many downstream tools. Segment, for example, routes data to 200+ downstream connections via our standardized API. Further, gathering data using a single API guarantees data consistency when sent downstream to a source-of-truth repository such as an Amazon S3 bucket or a raw data warehouse. Immediately, Product and Analytics teams enjoy access to perfectly formatted data.
CDIs also offer a single point of control in the form of a user interface. Product, Analytics and Marketing teams, as a result, don’t need to request Engineering time for small tweaks to settings, names, and other config issues that arise. Instead, they can leverage the interface to adjust settings on the fly, and watch their updates take effect right away.
CDIs empower users to create new audiences on the fly, and populate these audiences to one or many data integrations at a time. In particular, our Personas offering allows Marketing teams to build custom audiences that combine user traits and activity — all from a user friendly GUI that doesn’t require knowledge of SQL.
Finally, CDIs collect data not only from their own API, but from other popular tools as well. Segment fetches data from 28 downstream tools, routing it to a raw data warehouse so your Product, Analytics and Marketing teams have access to not only Segment-collected data, but data originally stored or segmented elsewhere.
In short, leveraging a CDI will empower teams to innovate and optimize and focus on impact, not on the minutiae of implementing and maintaining data integrations. Moreover, the availability of CDIs should make the answer to at least one question crystal clear: Buy, don’t build, when it comes to data integration solutions.