What really happened after the GDPR?

TL;DR: in the year since the GDPR, we’ve seen millions of deletion requests. In this post, we dig into the trends we’re seeing: more companies becoming increasingly privacy conscious.

May 25th, 2018 marked a completely unprecedented event: the enforcement of the General Data Protection Regulation (GDPR). 

For the first time in history, a major governing body (the EU) rolled out new privacy regulations that made hiring a data protection officer mandatory and controlling personal data a fundamental right for all citizens.

The new laws didn’t just stop with European corporations. A core tenet of the legislation was that EU citizens could request that any business around the world delete their personal data.

For companies running online banks, social networks, and e-commerce sites, the message was clear: honor the privacy rights of your users, or face a fine of 4% of your global revenue. 

As it turns out, implementing a solution for the GDPR also turns out to be a fairly hard technology problem. Instead of reading and writing data, you now have to focus on purging and removing it. According to a recent report, only 4 in 10 executives feel ‘ready’ for new privacy regulations like the GDPR.

At Segment, we found ourselves in a fairly unique position: we already help thousands of businesses manage their customer data… why not help them comply with this new legislation as well?

As a result, we started building a product to automate user deletion and suppression of new data. We wanted to make it easier for companies to comply with the GDPR not only by deleting data in Segment, but also by having us federate those deletion requests to the other SaaS tools our customers might be using. 

The question was… how many of these ‘requests to be forgotten’ would end users actually make? Would businesses receive tens of deletion requests… hundreds… millions? Would it all spike around the time of the new regulation? Or would everyone ignore it until the first lawsuits started cropping up?

One year later, we now have our answers. 

We analyzed how hundreds of companies have made millions of user deletion requests to our GDPR product. 

In this post, we’ll share the trends we’re seeing across the industry and the product improvements we’ve made to make it even easier to respect your end users’ privacy.

Here’s what really happened in the aftermath of the GDPR.

How does deletion work?

To give you some quick background, our GDPR product works like this:

1. A customer gets a notice from one of their users to delete or halt collection of their user data

2. The customer sends Segment a given userId for one of their users

3. Based on (1), the customer can suppress new data from being collected for that userId, delete existing data, or both

4. We purge and/or suppress that user’s data from our internal systems, the customer’s data warehouse, and forward it along to all GDPR-supported integrations

The end value for customers is that they can streamline GDPR compliance across supported integrations. Instead of managing data in half a dozen tools, they can make a single API call to handle GDPR data subject requests.

How many requests did we see?

To date, we’ve helped customers delete nearly 7 million users from Segment and other tools. The magnitude of that number surprised us. 

Translating that number to more concrete terms, it means that we helped delete the equivalent of nearly 1 out of every 80 EU Citizens’ data. Chances are that you know someone who has either requested that their data be deleted, or deleted their accounts on one of these services.

To put that in perspective: if the GDPR were a destination, it'd be one of our most popular tools (in the top 10% actually). Not bad for a tool which removes data instead of promising to give you additional insights.

A significant portion of these requests came from B2C companies who have large numbers of users visiting their site. Many of these companies offer a ‘one-click’ solution for customers to submit deletion requests directly, which translates to a high volume of deletion requests each week coming through Segment.

Surprisingly, not all of these requests were issued strictly due to the GDPR. We’ve also seen an increasing number of customers remove all data as part of account deletion flows. 

It seems that this is an unintended benefit of the GDPR–it’s also encouraging companies to implement better data practices when it comes to privacy and data.

Which way are requests trending?

It’s clear that we’re seeing more GDPR requests coming from more customers, in both deletions and suppressions.

The overall request volume has grown by about 50% since the first few months after we released our GDPR endpoint. Not only that, but volume in April 2019 was about 3x higher than in June 2018 (the first full month after GDPR). 

The increase in request volume wasn’t just driven by existing customers.

Below you can see a graph of the number of customers issuing deletion requests in a given month, from May 2018 through April 2019. Year over year, the number of customers making deletions has grown by 45%.

A second question we had was whether there would be an initial flurry of deletions just after the May 25th enforcement, followed by a period of relative silence. 

Here, we’ve graphed the total number of user IDs deleted each month:

In July and August, we saw a number of large bulk deletions, followed by a lull. When we asked customers how they were using the deletion API, these early customers told us they wanted to purge all data from previously deleted accounts. 

However, since then, we’ve seen a steady increase in the number of users deleted. The trend only seems to be continuing month-over-month, with a record 1 million users deleted in April 2019. 

The growing privacy-conscious climate has clearly spurred more and more businesses to take action when it comes respecting their customers’ right to privacy. Not only that, but it appears that end consumers and companies are continuing to think more critically about their data as well. 

What sorts of companies care about the GDPR?

In short… everyone. 

Since launching a year ago, one in fifteen customers across both our business tier and self-service have made some sort of deletion request. 

Interestingly enough, there was no correlation between the Segment customer’s plan and whether they made deletion requests. Customers who signed up with a credit card (typically smaller startups) and ones who had negotiated annual plans (typically bigger enterprises) both issued deletions at similar rates.

When we looked across customer’s geographies, we also started seeing more correlations. Europe and Australia lead the pack in deletions. Nearly 1 in 10 customers deleted data in both Europe and Australia.

• Europe: 9.7% of customers have issued requests

• Australia: 9.4% of customers

• North America: 5.8% of customers

• Asia: 0.63% of customers

Notably, Asian companies lag behind by a significant margin, less than one percent. We’ve graphed the raw counts below, split by country. 

Where did the data get deleted?

Of course, Segment is just one place which helps you gather and manage your customer data. We then help federate and adapt that data to more than 250 different downstream destinations. 

Some of these destinations also support our deletion spec, where we forward deletion requests we receive to be automatically purged in the downstream tool. 

To date, we’ve made more than 7.28 million deletion requests to other SaaS apps amongst our GDPR-supported destinations. For every individual user deletion request we’ve received, we’ve forwarded it along to roughly 1 other destination. 

In the coming months, we plan to work with partners to further expand the number of tools supporting deletion requests.

Making it easier to respect privacy

Given these learnings, we’re continuing to build features that make it easier for our customers to comply with the GDPR and future privacy regulations such as the California Consumer Privacy Act (CCPA) and beyond.

At launch, users only had the ability to delete a single user at a time. After we saw the sheer volume of requests, we made significant product updates to make it easier for customers to manage their deletions at scale. 

We now offer batching for all requests, where customers can submit up to 5,000 user IDs in a single deletion or suppression request. This helps customers more easily manage large volumes of deletion, as well as efficiently process deletions in batches for tools such as warehouses which are costly to scan if you’re only going one user at a time.

We’ve also added more visibility into the status returned by destinations to help when troubleshooting any issues that might arise when sending deletion requests to another tool. If your warehouse is unavailable for deletions, we will retry and update the status for the user with the reason the data hasn’t been deleted.

Additionally, these features are all supported via the Config API, which makes it easy for customers to programmatically control their deletion, suppression, and more.

Coming soon, we plan to give you the ability to set limited data retention periods for different sources of data right in the app. This will give customers direct control over how long their data is retained within Segment. Sign up here to be added to the beta for this feature.

Takeaways

Based on our role in handling the GDPR process for companies, here are our key takeaways:

The GDPR is here, and businesses are respecting it

The strict fines of the GDPR seem to be having their desired effect. This new regulation is fundamentally altering the privacy landscape.

In particular, the actual volume of incoming requests far exceeded our expectations. Originally we had scaled our deletion system to handle tens of thousands of deletion requests. Instead, we saw 100x that.

Upon seeing the count of deleted users climb to the hundreds of thousands, and millions, we invested even more work into creating bulk deletions, and further scaling our pipeline.

Business models are becoming privacy-forward

One popular refrain from our customers was that they weren’t using these GDPR endpoints for just compliance. Many of them were also using it to get their data to a more privacy-conscious, clean-room state. 

Additionally, many customers were treating all account deletion (regulated or not) as a driver to purge data from their systems. Even without the threat of a fine, businesses have decided that handling removing unused data is the responsible thing to do. 

Products must be built ‘deletion-first’

In readying ourselves for the GDPR, we spent six months updating our systems. We originally built Segment to accommodate terabytes of new incoming data, instead of focusing more on the sorts of ‘needle in a haystack’ searches demanded by the GDPR. 

Going forward, every new part of our pipeline is designed to support GDPR deletion from day one. 

The age of building data pipelines which are ‘write-only’ is dead. Instead, it’s clear that pipelines have to be engineered from the ground up not only to write new data, but to allow deleting old data as well. 

It’s clear that the GDPR has fundamentally altered how businesses think about data. For any business operating at scale, one thing is now clear: we’ve entered a new privacy-first age of the internet, where every user will soon have the inherent right to manage their own data.