Building a Destination

Segment Destinations enable you to receive incoming data for our mutual customers to your service’s HTTPS endpoint in realtime.

A destination gives you complete control over how you want to store, transform and process the data. It means that our mutual customers can immediately start sending you data from any one of Segment’s sources, including a web browser, mobile apps, or from our mutual customer’s servers — with no added work. Segment Business Tier customers can also replay historical data, which means you can demonstrate the value of your tool rapidly.

Becoming a Destination Partner is a quick five-step process. By starting or continuing this process, you agree to the Segment Platform Partners Agreement.

Five-step Process:

  1. Request access to the Segment Developer Center.
  2. Build an endpoint to receive customer data.
  3. Test your destination.
  4. Submit your destination for review.
  5. Launch!

1. Request access to the Segment Developer Center.

Get in touch so we know you’re interested in building an destination. We’ll grant you access to our (Currently Private) Developer Center and agree on a launch date so that we can provide a focused window of deep guidance and support.

2. Build an endpoint to receive customer data

The following sections outline what kind of requests and data we send to you, and what we expect in return.

Important: A common best practice that we encourage is submitting your documentation to Segment via email before you get started. This allows us to call out any red flags and increase the likelihood of a smooth submission and launch process.

Accepting Segment Data

In order to receive data from Segment, you will need a server ready to accept HTTPS requests. You’ll give us a static endpoint to send data to, and that endpoint must:

  • Accept POST requests. Segment will send customer data to the endpoint you designate in POST requests.
  • Accept JSON data. This is the format Segment sends data to you in.
  • Use HTTPS. Segment transmits potentially sensitive data on behalf of customers, and using HTTPS goes a long way toward making sure their data stays safe.

Authorization

Segment will send your user’s API key with requests, which you can then use to authenticate requests. Note: This is the API key you give to your users; it is not a Segment API key.

Segment will send the key in the Authorization header using the Basic authentication type. It will be Base64 encoded, with your user’s API key as the username and an empty password.

For example, if your user’s API key was segment, Segment would Base64 encode the string 'segment:' and prepend the string 'Basic '.

Note: The colon is always present, even when the password is absent.

This would result in a final string of 'Basic c2VnbWVudDo='. This is what is contained in the Authorization header. You will need to decode the string when you receive it, just like any Authorization header.

See the headers section for more details.

Headers

Segment will send you the following HTTP headers with all requests:

HeaderDescriptionExample
AcceptSegment accepts any content type, but ignores responses unless this header is set to application/json.Accept: */*
AuthorizationSegment will send you your user’s API token in this header, with the Basic authentication type.Authorization: Basic c2VnbWVudDo=
Cache-ControlEach request Segment sends you is a new event, and so Segment expects you to not implement caching on your end.Cache-Control: no-cache
ConnectionSegment uses HTTP/1.1’s keep-alive functionality whenever possible (but you don’t have to).Connection: Keep-Alive
Content-LengthSegment will always send you the length of the request in bytes.Content-Length: 348
Content-TypeSegment indicates the type of data sent to you (this will always be JSON), along with Segment’s vendor type.Content-Type: application/json
User-AgentSegment will send you this field every time. You can count on us!User-Agent: Segment

Request Body

Segment’s spec standardizes the data that you can expect from Segment. You can choose to implement four types types of calls:

  • Who is this? .identify(userId, traits)
  • What are they doing? .track(userId, event, properties)
  • Where are they doing it? .page(userId, pageName, properties)
  • What group are they part of? .group(userId, groupId, groupTraits)

For example, you may want to implement the .identify(userId, traits) call to create contacts in an email marketing application. You can expect the following customer information as a JSON object in the call body:

{
  "anonymousId": "1234",
  "context": {
    "ip": "8.8.8.8",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36"
  },
  "messageId": "022bb90c-bbac-11e4-8dfc-aa07a5b093db",
  "receivedAt": "2015-02-23T22:28:55.387Z",
  "sentAt": "2015-02-23T22:28:55.111Z",
  "traits": {
    "name": "John Doe",
    "email": "john.doe@email.com",
    "plan": "premium",
    "logins": 5
  },
  "type": "identify",
  "userId": "5678",
  "version": "1.1"
}

Important: Please note that the casing on these fields will vary by customer, so please be ready to accept any casing.

The Segment Spec

To learn about the semantics of the five supported API calls, and the semantic event names and properties we recognize, please read the Segment spec.

The spec is a critical component of preserving semantics between disparate writers and readers of data. If you encourage customers to break the spec, you are breaking the promise of Segment, which is grounds for removal from the catalog.

Important: If there are any functional elements of your tool that have mappings to the spec and you are not adhering to it (eg. asking customers to send “Purchase” instead of “Order Completed” or “install” instead of “Application Installed”), we will reject your application.

If there is something unique about your tool that requires specific data points that are not included in the spec, get in touch. We love partner suggestions for augmentations to the spec!

Responding to Segment

This section defines how you should respond to Segment requests.

Status Code

Segment uses standard HTTP status code conventions to help us diagnose problems quickly and give better insight into how the destination is working.

Upon receiving data, your endpoint should reply with one of the following status codes:

CodeReason
200You’ve accepted and successfully processed the message.
202You’ve accepted the message, but have not yet processed it.
400The message is malformed, or otherwise contains an error that is the client’s fault.
401The client’s API key is malformed, or has expired or is otherwise no longer valid.
403The client’s API key is valid but has been rejected due to inadequate permissions.
500If you encounter an internal error when processing the message, reply with this code. (Hopefully you don’t have to send too many of these.)
501If Segment sends you a call type (indicated by the type property included on all messages) you don’t support, reply with this code. Read more about the API call types Segment supports here.
503Send Segment this code when your endpoint is temporarily down for maintenance or otherwise not accepting messages. This will help Segment avoid dropping users’ messages during your downtime.

Response Body

You can normally send back an empty body, but when sending back a 4xx- or 5xx-class error, you may optionally send Segment a diagnostic message that explains the error. This message will be displayed to the user in the Segment debugger and will be used in our Event Delivery summaries.

Be sure to send JSON (and set your Content-Type header to application/json), and send your message in the message property.

Here’s an example of a 401 response that would help a user track down why their calls aren’t appearing in your tool’s UI:

{
  "message": "API token expired"
}

Or, if your tool requires an email address in order to accept calls, this 400 reply:

{
  "message": "Missing email address"
}

Handling deletions

In addition to the five primary spec methods, Segment forwards partners a 6th message type for customer-requested deletions. Destination Partners with access to the Developer Center are required to implement and document support for this federated user deletion.

Here’s what a payload for that will look like.

{
    "type": "delete",
    "channel": "server",
    "messageId": "delete-022bb90c-bbac-11e4-8dfc-aa07a5b093db",
    "projectId": "abcd123",
    "userId": "5678",
    "context": [],
    "integrations": [],
    "receivedAt": "2019-02-19T23:58:54.387Z",
    "sentAt": "2019-02-19T21:58:54.387Z",
    "originalTimestamp": "2019-02-19T23:58:54.387Z",
    "timestamp": "2019-02-19T23:58:54.387Z"
}

3. Test your API endpoint and Destination

Once you’ve implemented against the Segment data spec and the requirements above, you’re ready to start testing your Destination! Segment requires all Destinations to be of highest quality, our best Destinations work exactly as they are described in the docs, have highest reliability (up-time etc.) and our Partners are quick when it comes to fixing customer issues.

Testing your Destination is a two-step process:

  1. Test your API endpoint to make sure it can receive Segment data at scale
  2. Test your Destination by connecting it to a Source in Segment and logging into your own tool in order to mimic how a mutual customer would send data between the two to achieve their desired use-cases.

Testing your API endpoint

In order to test your API endpoint, you’ll want to download either the Mac binary or the Linux binary. This program will test your endpoint across a wide range of fake customer data. We include all recommended call types (track, identify, group, page, screen), as well as a wide variety of logged in and logged out users.

From there, you’ll want to run the program from the command line, passing in your API key

chmod +x ./direct-endpoint-tester
./direct-endpoint-tester --api-key <YOUR_KEY> --endpoint <YOUR_ENDPOINT>

You should see the output say something along the lines of which indicates that your endpoint is properly processing all of the various Segment data.

0 failures, 305 successes

If, however, you receive 500 errors there might be a code issue or a heavier than expected load on your servers that you should take a quick look at.

Warn: error hitting endpoint The server did not return with a 200, got: 500 Internal Server Error
errors: 1, successes: 304

Testing your Destination

Once you have successfully tested your API endpoint, you can login to our Developer Center (request access here) and create your Destination. The Developer Center allows Partners like yourselves to create and publish high quality Destinations to the Segment Catalog. You can login to Dev Center using your existing Segment App login credentials. Your Segment account now doubles as a Sandbox account to test your Destinations which are under “Building” state.

Here’s how to create a Destination via the Developer Center:

Once you have saved the draft of your Destination, it is visible to you in your Segment workspace. You can access the Destination via this URL: https://app.segment.com/WORKSPACE-SLUG/destinations/catalog/DESTINATION-SLUG. Make sure you replace with your own workspace-slug from your Segment account as well as the auto-generated destination-slug from the Developer Center. You can also enable your Destination by connecting it with a Source.

Once you have successfully connected a Source, you can use our Event Tester feature to generate a sample Event payload and send events to your Destination. You can specify the event you want to test using the JSON text editor or the Event Builder. Learn more about this here.

PRO TIP: If you are already a Segment customer and have an active Source, that collects real-time customer data. You can also use our Event Delivery feature to easily identify and resolve data delivery issues. More on this here.

4. Submit your Destination for Approval

After successfully testing your Destination, it’s time to submit the Destination for approval. Please remember, you will not be able to edit your Destination information once you submit for approval, so please review all the details before hitting the “Submit for Review” button in the upper right corner of your Destination tile.

Once you have submitted your Destination for review, Segment will review your submission and get back to you within 2 business days. Once Segment approves your submission, we just need few docs and marketing material in order for your Destination to go live on the Catalog.

5. Launch Requirements

Once your Destination submission is approved by Segment, we need a few things from you in order for us to launch your Destination. By launching, your Destination will be live on the Catalog, which means it’s discoverable by Customers and Prospects of Segment. Here are the launch requirements:

  1. Docs on your website that explain how Segment <> Your Integration works, example - https://success.clearbrain.com/connections/import-connections/connect-to-your-data-in-segment
  2. Docs on Segment website, example, https://segment.com/docs/destinations/clearbrain/. Please share these docs by making a copy of this template https://hackmd.io/t7amLXluS7-39rg7ARZgSA?both=
  3. Details catalog material including screenshots by making a copy of this template: https://docs.google.com/document/d/1kvAvAHLyM3pOq-lBcZJhP_X_KivHlk1eiFy-5ERWDXc/edit
  4. A blog post on your website announcing Segment <> Your Integration, example - https://moosend.com/blog/segment-integration/

You can reach out to us at partner-support@segment.com once you have all these 3 docs ready for review, once we approve them your Destination goes live on Catalog in Public Beta 🎉

We’ll be promoting your Destination from Public Beta to Public once we see at-least 25 customers enable your Destination via Segment.

FAQ

How do customers collect data?

A mutual customer will use analytics.js (our client-side javascript library), a server-side library, or one of our mobile SDK’s to implement our api methods. For more information on Segment libraries, you can refer to our source documentation.

Does Segment automatically collect any data?

Only our analytics.js and mobile SDK’s collect contextual information from the device. Our server-side libraries do not collect contextual information, and a user is responsible for sending additional context themselves.

For more info on our automatically collected fields, please refer to this document.

How does Segment handle unique users?

For known users, a Segment customer will implement an identify method to collect info on the user. This can be a moment in the user flow where a user logs in, registers, updates their info, or provides any type of identifiable information. A known user will have a userId, which is up to the customer to create and send.

For unknown users, Segment will handle generating a unique anonymousId via our client-side libraries: analytics.js, analytics-android and analytics-ios, and pass this in through all of Segment’s api calls. This value is determined by the client cookie on our analytics.js library, and via the deviceId in our mobile SDKs.

Segment handles cacheing these values on our mobile SDKs and client-side analytics.js library and sending the values on subsequent calls. Our server-side libraries rely on the customer creating either the anonymousId or userId and passing this in on each call.

Read more about our unique Id’s here.

Do you have semantic events?

Yes!

To start, a Segment customer will track their user actions using our track method. Segment has industry specs to define semantic naming to follow, so when sending events in for a particular event flow, such as Ecommerce, Live Chat, Video and Email events, Segment can translate these semantic event names into other downstream tools.

It is essential that the destination follows the relevant industry specs when translating a Segment event into how the destination tool understands the event. That way, customers can enable any new integration and specced events, such as “Order Completed”, and it will automatically work with the new downstream destination.

Are the events guaranteed to send in order?

No. Since Segment queues events, Segment cannot guarantee the order in which the event is delivered to your endpoint.

Does Segment de-dupe messages?

Yes! Segment de-dupes messages by messageId.

Segment maintains a sliding window of all messageIds received for each source, only allowing messageIds through that do not already appear within the window.

Segment guarantees this window to be at least 24 hours of messages (meaning any message sent twice within 24 hours will be de-duped), but in practice, this window is significantly larger(currently sitting at around 170 days).

You can read more here.

What is a replay?

Segment supports replaying historical data to new tools for Business Tier customers. This can greatly increase your activation and lower time to our customers getting value out of your tool.

Generally, the conditions required to replay data are:

  1. The partner API respects the timestamp field for event time ordering.
  2. Order of events does not matter.
  3. The API has reasonably high rate-limit events.

Be sure to let us know if you are able to accept replays and what your rate limits look like.

What are Segment’s delivery guarantees?

Segment provides excellent data deliverability by employing API layer scalability and durability, data backup and replay, partner API monitoring, and library and integration cloud retries. Segment’s API processes 170B+ billion calls per month across over a billion of monthly tracked users, is rate performant (avg. load 100,000 msg/sec), fully automated and scalable, can tolerate massive data spikes.

Segment monitors hundreds of partner APIs for 500s, success rate, and end-to-end latency to help our customers proactively achieve the best data deliverability possible.

You can subscribe to updates here.

Does Segment retry data?

Segment retries nine times over the course of four hours. This will increase the number of attempts for messages, so we’ll try and re-deliver them another 4 times after some backoff.

We don’t retry anywhere which is the sign of an expired API key or failed payment. However, if we push bad code that results in a malformed payload and a 400 or 422 response from an endpoint, we also won’t retry given that the call would not ever succeed.


If you have any questions, or see anywhere we can improve our documentation, please let us know!