How to use a Tracking Plan to block unplanned events or schema violations and send a Slack notification

Segment Protocols provides a governance layer across all your event data to ensure that only clean, consistent events (as defined by your organization) are sent to Twilio Engage, your destinations, or data warehouse.

Protocols works in real-time, validating every event against your Tracking Plan. For each Source in Segment, you can set up schema controls to block violating or incomplete events at the source and send them to another location for quarantine. We’ll walk through how to set this up and how you can forward violating events to a Slack channel, so your data engineering team can be proactive about preventing and resolving inconsistencies in event tracking.

Note: Segment Protocols may not be enabled on your workspace. If you cannot access the Protocols page, contact your sales representative for assistance.

Step 1 - Create a Protocols Tracking Plan.

Tracking plans ensure that events schemas across multiple sources are tracked consistently, so teams using this data can trust that it’s accurate. 

(If you’re not familiar with how to create a Tracking Plan, add it to a Source and block Schema Violations, follow step 1 of this recipe for a step-by-step guide.)

Add the Tracking Plan to a source if you’ve not done so already.

Now that the Tracking Plan has been added to the source, Segment will start validating the incoming events against that Tracking Plan, and any unplanned events or schema violations will be flagged in the Schema view of the source and in the Violations tab of Protocols.

Now if we want to ensure that all incoming data does in fact match our tracking plan, then we can use Schema Controls on the source, to block violating events and forward them to a new source. This acts like any other source so the events can be synced with storage destinations like data lakes, data warehouses, and any other destination. 

In this example, we’ll connect Slack as a destination and forward violating events into a Slack Channel.

Step 2 - Block schema violations and send them to a source

If you’re not familiar with blocking and forwarding violations to a source, follow the instructions in Steps 2 and 3 of this recipe.

Step 3 - Connect Slack as a destination

Within the Segment Connections screen, add a new destination, choose Slack from the catalog and press configure Slack to proceed.

Select the source we just created for violations. On the next page, give your Slack Destination a name, choose the Actions Destination Framework and continue. 

We now have a new Slack Destination for your workspace. To continue the setup, navigate to the Mappings tab, this is where we will configure which channel to send the events to and what the body of the message looks like. 

If you don’t have a slack webhook url for your Slack channel, follow the instructions from Slack here to configure a Slack App to process the webhooks and post them to your channel. Make note of your webhook URL, as we’ll need this in the mapping step. 

Create a new mapping

The first step in defining the mapping is to choose which events we want to send to Slack. We could configure our mapping to forward every event based on the type of event it is, such as Track, Identify, Page call etc. But for this example, we’re going to use the event name ‘Violation Generated’ as this in an event created by Protocols when a Schema Violation occurs.

If you’ve seen some violations to your new violations source, you can load a test event to preview the fields in the step 3 of the mapping.

In ‘Select Mappings’, paste your Webhook URL and define the message body using text and properties available from the event payload. In the example below, I've added some properties of the violation event, such as the source name, violating field, violation type, and a violation description. The violating field, type, and description are added to the Violation Event by Protocols to help us quickly understand what caused the violation. I’ve also added the message ID, so we can quickly find the exact violation in the original source.

At the bottom of the page, we can fire a test event to the Slack channel. For my configuration above, the message in Slack is as follows:

Once we're happy with the mapping, we can save and enable the mapping for our Slack channel. 

Wrapping up

What have we achieved? We:

• Built a tracking plan for our data in Segment Protocols.

• Used the tracking plan to ensure that the data we ingest was good.

• Configured a real-time notification service for schema violations in a Slack channel.