Collecting the Right Data to Serve your Audience and your Advertisers

Kailey Raymond: In the golden age of streaming, advertising is more important than ever, but reaching customers based on their interests and consuming habits remains a challenge. Even more so, the ways in which advertisers collect customer data is changing drastically. It's a challenge Eric Mies knows all too well. Eric is the Director of Product, Video Experience, Ads and Ad Operations at Crunchyroll, a 360 anime brand offering streaming, games, eCommerce and news. In this episode, I sit down with Eric to discuss personalization in the age of GDPR, contextual targeting and unifying data with the help of humans and machines. Walk me through how'd you get to Crunchyroll?

Eric Mies: So out of college, I'm from the Bay Area originally wanted to get back this way and so I started looking for jobs and found a great role at Healthline.com doing ad operations. And so that's really how I started to get into the advertising world and into the tech side of things. And so spent about two years at Healthline and then moved over to a startup called Revision3 where I was also doing ad operations and getting more involved in video. And so that's where I really started to learn more and more about streaming and video on demand.

That company actually got bought by Discovery Communications and so then spent some time there, it spun off into Group Nine where I continued to really be ad focused. In that transition I moved over from ad operations to product management and that's where I am today. Actually coming to Crunchyroll, went full circle and also now oversee ad operations as well as ad product. And so how I got to Crunchyroll was some of the leadership team that I worked with at Discovery Communications moved over to Sony and Crunchyroll, formerly Funimation, and they brought me in. And so it's been a good three years now, it's been really exciting times here with the Funimation and Crunchyroll merging together along with a few international anime streaming brands and working with the whole Sony family.

Kailey Raymond: That's awesome. So you've kind of been in ads, advertising adjacent at the very least for the vast majority of your career. So you've certainly seen a lot of the evolution, I am sure. Before we get into some of those specifics, tell us more about Crunchyroll for people that might not be so familiar? In your own words, what do y'all do?

Eric Mies: Primarily we are an anime streaming service. We offer thousands of different titles, the big ones like Demon Slayer, Attack on Titan, My Hero Academia, but really working to bring anime to the mainstream, to the global market. So we are in every country, we stay out of some of the APAC regions because that's really where anime premieres, is even stronger. More recently we've expanded outside of streaming, working on games, eCommerce, more news and really trying to be a 360 brand for anime.

Kailey Raymond: That is so cool. And so do you have a favorite anime show that you watch?

Eric Mies: That's a tough question. I really like Attack on Titan, I think that's my favorite. It's very exciting and the last movie was awesome so looking forward to the next new season.

Kailey Raymond: I love it. It's something that I need to get into, I have so many friends and family that are anime fans and it's never been something that I've delved into, but it's so massive across the globe that maybe I'm going to take you up on your favorite and check it out. So you are in the product management group leading a whole host of different things, but tell me a little bit more about your role specifically as it relates to driving customer engagement?

Eric Mies: Yeah. So I'd say there's two sides to that. I think in the ad world, and especially when you're a B2C company, it's helping your advertisers, which are your customers and then it's also helping your users creating a good experience while they're viewing your content. And so for advertisers really looking to help promote their brands, bring their advertisements, their products to the right audience and helping them drive scale, drive the actual engagement on their ads and driving conversions. And then for our users, it's really making sure that when they're watching, they feel like they're having a premium viewing experience and they're not getting repetitive ads, they're seeing ads that they think are relevant and they continue to come back to our service and spend long viewing times there.

Kailey Raymond: I feel like in the early days of streamer ads, you would get the same one over and over again. So glad to hear that the engagement and making sure that you're diversifying it is key for y'all. Being in this space for a long time, I'm sure you've seen a lot of things come and go. So what are some of the top trends that you are currently seeing as it relates to customer engagement, customer experience in the streaming industry?

Eric Mies: Yeah. So I think it's interesting, I think as things evolve, they always come full circle. So I think the obvious one is data privacy and over the last 10 years, maybe even a little bit longer, there's been a big push towards behaviorally targeted ads. So showing people ads based on what they've looked at on the internet, what they've done, but with data privacy coming in, that needed to change. Outside the laws, there's also the platforms really asking users and trying to be user-centric, iOS is the obvious, whether they want to be tracked.

And so I think that's really had to change how we look at targeting our ads to users and bringing brands to the right demographic. When I first started in ad trafficking, it was very contextually targeted ads and very heavy there. And then that started to change, but we're starting to see that trend come back. I think outside ad specific things, a new trend we're starting to see are ad paid tiers and there was a big split between AVOD, ad video on demand and, SVOD, subscription video on demand. But now we're seeing people like Netflix and Disney Plus offer this third option, and Hulu, it was obviously the original one, of paying a small subscription and also having some ads run on that. And again, going full circle to more the cable model I guess. So yeah, it's interesting.

Kailey Raymond: So interesting. And for the privacy one, do you have any examples of just some of the ways that, that's been impacting some of your strategies? Any things that you've kind of seen in real life as somebody that's been working and advertising for this long?

Eric Mies: It has us needing to consider different things with our users and how to serve ads that we used to.

Kailey Raymond: You're going from third to first?

Eric Mies: Exactly. And so we need to do more of that data collection, we need to ask for their approval before we start that data collection. And so there's more technology we need to implement on our platform to get that consent from users, which as a user I'm very supportive of. But it does change our approach, our priorities of the technology we need to implement on our platforms. So as GDPR rolls out, there's a lot more work we need to do across our websites to go and collect that consent.

As Apple or other platforms implement their own asking users to opt-in or opt-out, we have to make sure we're responding to those signals properly and changing our technology stack to make sure we're respecting those user opt-ins or opt-outs and then downstream making sure we pass those properly to our Adtech partners and they're respecting them. And so between the legal side, the technology side, it just creates new challenges and new priorities we need to do to continue to serve ads and create that customer engagement for our advertisers and still drive high conversions, click throughs and make sure we're showing the ads to the right demographic.

Kailey Raymond: Yeah, totally. I think that the shift from third party data over into first party data and all the privacy implications coming across from both GDPR or CCPA, government entities as well as the big platforms is so shifting the abilities for anybody that's sitting in your seat of what they're thinking about every day of what data to collect, how to collect it in order to increase conversions, in order to increase their return on ad spend. And I know it's been the cookie apocalypse that has been coming for years and years now and so as somebody sitting in that seat, I can empathize with some of the challenges that you're going through. Besides privacy, are there any other customer behaviors that you're seeing in the last couple of years, especially that you're tracking to see where they go in terms of how they're going to shape trends in the future?

Eric Mies: Yeah. And I think that speaks to that ad paid tier. I think we're seeing that users want different options of how to consume content and so we've been lucky as a streaming service to continue to grow that side of the business and our overall subscription, but we're seeing other streaming services and as competition comes in that plateau. So we need to find ways to give our users options of how they consume that content, whether it's just ad supported, whether it's a combination, it's just subscription. Outside that, what I was talking about with the 360, the flywheel business, finding ways to also support our streaming service and diversify our revenue streams with games, eCommerce, theatrical, we've done a really good job releasing movies in the US over the past couple years. We've had two of the biggest movies, I think Dragon Ball Z was number one in the box office last month.

Kailey Raymond: That's awesome.

Eric Mies: Which is really exciting. So again, it's finding other ways to build our brand as just streaming gets more competitive.

Kailey Raymond: That's so interesting, yeah. And I think really a lot of what you're talking about right now in some ways is just the need for personalization. I want to see content in this way, whether it's a video game or some sort of different subscription service than you're used to. I think that folks are just so used to the Amazons of the world creating these highly personalized touchpoints for every single person and expecting that from every single company in the world, which is a massive challenge of course to try to keep up with. So speaking of which, do you have any challenges as it relates to building that customer engagement journey?

Eric Mies: Yeah, definitely. I mean, I think you spoke to some of them, there's a high bar with some of these other tech companies that have put tons of money into their personalization. And so broadly that's definitely something we're continuing to work on is making sure we're recommending the right content when users come to the site, understand what users watch and really make sure we're putting our best foot forward in presenting our whole catalog. We have hundreds of thousands of titles, I think our catalog's bigger than most people probably realize and it's important that we're really showing them everything and things we think they're interested in.

I think some of the other challenges we've had is more on a granular, me personally, is just around our content itself and getting all the data against it. Our company is 20 years old and so there hasn't always been the best human tagging of content and personalization all starts with understanding your content along with your users, it's the dual side of it. And so really making sure we have a robust metadata against all of our content. And so recently we've been finding various solutions to help us tag our content outside just our content operations team. And so I think even on the ad side, that's something that we'll continue to really lean in on is making sure we have robust contextual data so we can do cool things there.

Kailey Raymond: That's so interesting. I'm so excited to dig in there a little bit more. So you're talking about building contextual ads for folks based off of their behavior, what they're liking, what they're watching. I'm assuming that building a lot of that metadata in is going to take a combination of both human and machines to do so. Walk me through how you're doing that? How are you actually serving up those ads? What are some of the ways that your marketing teams and tactics have been evolving with both humans and machines in the world? Walk me through what's going on?

Eric Mies: We actually use a few different ML services to help run all of our video content through and making sure we're properly tagging that content. We then have the humans come in and look at how we're tagging that content and making sure that each service is bucketing it into the same terms. Every ML service has its own naming conventions, way of identifying things, but then after we can bucket that in, we can pump that into our ad server to build genres, ratings, thematic scenes and really target ads that way.

And then really look at that data of how those ads and those buckets are performing on our platform and start building packages for our advertisers. If there's a certain advertiser that is promoting a video game that might be a little bit more action centric, we can look at our action content and see if that performs well. So building these kind of contextual packages, using our ad ops team to test campaigns against it where the advertisers is giving us certain kind of requirements of where they think their ads will work on certain types of content and testing these various ones, looking at click through rates and understanding where our users are responding well and then reapplying those learnings to future campaigns.

Kailey Raymond: I have 20 questions off the back of this. So in terms of the categories, if you're working with a lot of different ML programs and they all have different labels to all these different categories, how are you unifying those?

Eric Mies: Yeah. So this is where we rely on our content operations team to really help us map-

Kailey Raymond: Humans always.

Eric Mies: Humans, yes.

Kailey Raymond: Totally, cool.

Eric Mies: Map those terms together and make sense of it all. Rely on them to really do that work and then after they give their approval, push that information downstream. So we have different layers of metadata that we're saving in our system because we still want to understand what was originally captured, but we have a layer three, layer two and layer one and humans are that layer one piece on it.

Kailey Raymond: Makes total sense. And then of those different categories, are there ratings or themes? First of all, I would love to learn a little bit more about all of those. How many are? What does that look like? And then are there ones that are more predictive of whether or not an ad is going to be really high in conversion for users? More than anything, is the rating the best predictive indicator or is it the theme of somebody that they've watched before?

Eric Mies: So it really depends on the campaign KPI coming in. And again, what we do is we track all the performance of our campaigns and we have a database that allows us to understand the original KPIs or performance that the campaigns were looking for and then we have different buckets. So there's probably about 20 genres, hundreds of themes, and we try to connect the dots and document, again, because people change at the company. I think a lot of our ad ops team has a feel of what works, what doesn't work, but mid campaign and really every day we're looking at those performance of the data and optimizing it.

So if we have a certain genre targeted to a campaign and it doesn't seem to be working, if it's a Valentine's Day ad campaign for chocolates or something like that and we think, "Oh, it's obvious to target this to romantic content," because that feels like that would make sense, but it's not performing well, we'll review that and tweak the targeting on the campaign real-time. So it's really important for our ad ops team to everyday be looking at the campaign performance, the data and pulling the contextual performance and tweaking and pushing impressions where they are seeing it well. So sometimes they'll start broad, see the click through rates, how they're performing in a certain category and push more impressions there over time to see if that performance continues there.

Kailey Raymond: I love that example of Valentine's Day, perhaps you were surprised by that. Do you have any examples of when maybe you expected one thing and then the data was showing you something completely different?

Eric Mies: Well, I think for us we have a lot of gaming campaigns and gamer centric, whether it's mobile, whether it's a platform like PlayStation, Xbox games. And a lot of our mobile advertisers will think they should only deliver ads or deliver their campaign on mobile platforms and want to stay off of web. And this was surprising to me is that mobile campaigns actually perform just as well, if not better on web sometimes, they're really focused on driving click through and this is outside just being on the website on your phone, just even on desktop. But we've seen that users really still convert really well on downloads even if they're not on their phone. So I think as long as we're giving that impact, still giving them the ability to click through, we've seen good performance of driving conversions. So that was surprising to me and it's something we're constantly trying to educate some of our advertisers on because we do have a lot of mobile game campaigns that come to our platform.

Kailey Raymond: That's so interesting. And so are you saying that folks will pick up their remote control and interact with the ad just as readily as they would tap their finger on something?

Eric Mies: Yeah, exactly. Or I'd say more on web so use their mouse, but click through rates are just as good. And again, you'd think driving directly to the Play Store through the mobile phone would help that, but people are able to click through this site to the website when they see those ads, they get excited about it and then they'll still go to the store and download the apps through their phones. So I think that's where it's really been interesting to see.

Kailey Raymond: That's very cool. I'm sure as somebody who deals in hundreds of thousands and millions of impressions, you're looking at tiny little conversion rates all day, every single day, you probably have a perspective on what good data is so I'd love for you to try to define that. What does good data mean to y'all at Crunchyroll?

Eric Mies: Most importantly, it's accurate and so we're not sending duplicate impressions, bad clicks. And even on the product side with the video player, really just making sure we're firing everything correctly. I think accurate and clean data are important and then making sure it's impactful and driving high conversions, high CTR.

Kailey Raymond: What does accurate mean?

Eric Mies: In the ad world, the IAB defines certain ways to read VMAP responses and fire beacons for impressions, clicks. So just at the real basic level, it's really following those guidelines and making sure that we're doing that correctly. There're standards with that, IAB helps all publishers have that standard. And so to me at this lowest level, that's really how we can ensure we have accurate data.

Kailey Raymond: It's nice when you can all have a common definition. I feel like everywhere I go, it's always like, "All right, what's our definition of an MQL?" Everybody does it differently, everybody does all their modeling differently and so thank you IAB for at least making one thing relatively unanimous. We talked about some of the machine learning things that you're doing and tagging content for contextual targeting. Are there other ways that you're using data when you're building your marketing strategies and tactics?

Eric Mies: Going back to relying on working closely with our content operations team, another thing that we sometimes struggle with is having natural cue points in all of our content and knowing where to run our advertisements. So machine learning is another tool we use along with the humans to make sure all of our content has the proper cue points we want in there to have the ad load we feel is most appropriate for our platform. And just making sure that it's always a full set of ads so we can also have the best revenue opportunity. And so that's a project we've been working hard on because again, we have a catalog that has had spotty metadata and so really running all of our content back through these tools to get our proper cue points and making sure we're making as much revenue as possible with the various lengths. So whether it's a movie and it's an hour and a half long, different kind of an ad load than a 30 minute video or even a 15 minute video, let's say.

Kailey Raymond: That's so interesting. So you're telling me as somebody who grew up watching sitcoms where you would actually have that break that's so clear of what's happening with, here are your advertisements served up to you, you're using computers to tell you exactly when you should serve up an ad to an audience based off of the context?

Eric Mies: Yep. And really making sure we're not running ads in the middle of someone's sentence. And so we want to give that good user experience, that good viewing opportunity for our actual customers. So yeah, we've been using machines to help us find those breaks.

Kailey Raymond: That's so interesting and something that hasn't yet been applied to the podcasting industry. I was just thinking about that whenever I listen to podcasts, it's like, "Okay and now we're going to hear..." They introduce their advertising segment still, that's such an interesting new problem and use case to solve. So cool. I mean, videos are such rich data though. Is audio as rich as video? Probably not, right?

Eric Mies: No, it's not at all. And some of the media companies I've worked for, we dabbled a little bit in podcasting and it is interesting to see, tracking is a lot harder, how you serve ads. I think that's actually expanded a lot where it doesn't have to necessarily be someone reading off a piece of paper and it can't be more inserted in to get the volume for different marketers that want to spread against various publishers and track it all back in one place like an ad server does on the video or the display side. So I think it's an evolving medium in that sense for advertising, but it's definitely not quite as robust as video. And then I think display is probably still the most robust because it's the original.

Kailey Raymond: Definitely makes sense. Do you have anybody that you look to for inspiration, that you think is doing it right as it relates to leveraging all of this rich data to build customer experiences?

Eric Mies: To me, YouTube's really the gold star in the video business probably as a whole, but definitely in video advertising. I think they've taken it another step and with ad load they've really taken the time to figure out the best ad load per user. And so however you consume your content on YouTube and let's say you skip a lot of ads or you sometimes watch the long ads before you skip them, if you keep those little overlays open or you close them, they'll send you a different mixture of ads based on your behavior there. And so that's always been the gold star and something I want to get to because everyone has a different way or different types of ads within video they like and so figuring out that mixture to keep them on site as long as possible so they can serve the most ads to those users as possible without deterring the ad experience. So don't run a pre-roll in the beginning of a video because they'll watch longer overall. So I think YouTube's definitely the best at that.

Kailey Raymond: Do you have a favorite piece of data that y'all collect and you act on or perhaps a campaign or a strategy or a program that you're really excited about?

Eric Mies: I'm really obsessed with making sure our user experience is prime. And so really looking at viewership, minutes watched and slicing some of that data together to make sure that we're keeping users engaged on our actual players because to me, then everything trickles down. And also it's with the content on every media site and layering and advertising is a piece of that. From an advertiser KPI performance standpoint, I mean I think the classic CTR and impressions and you're helping companies drive brand awareness. I think since there are limitations in CTV of clicking through, that's where CTR gets a little bit weaker. And as you know, I think streaming services continue to grow impressions, viewability and really making sure people can see your ads at a high volume.

Kailey Raymond: That's interesting. The subscription services continue to grow is just such a massive industry trend, I feel like everybody has some sort of subscription service now. I wonder when it's going to begin and end, but I'm sure advertisers are so stoked for it, it's just more eyeballs and more platforms. Or maybe it's not, do you see any budget shifting between really niche and granular platforms now with that?

Eric Mies: I think with the rise of paid tiers, advertisers are getting excited about that. Advertisers I'm sure would love to be on Netflix and on Disney Plus and places like that and there's been maybe a little bit of void outside, I'd say more user generated content with YouTube. And so I think having more premium content and the ability to advertise against that gets advertisers excited. So I think video and streaming advertising will continue to grow and I think we're starting to see a dip in linear and that shift. And so over the past 15 or probably 20 years, digital advertising has grown year over year and it's going to continue to do that. And I think where you start seeing the big change is off of linear, which is cable basically. But cable still has big concurrent viewership and still big audiences so I think it'll take a little bit more time to have the full change. It's pretty close though to flipping, where there's more advertising spend on streaming services versus linear.

Kailey Raymond: That's interesting. I mean, you're talking to a cord-cutter right now. So I fully haven't subscribed to cable ever in my adult life, actually maybe, so I definitely imagine that, that's the case. But you're right, I do think that sports, maybe the ESPNs of the world, do they have a streaming service? I don't even know, but I know that they do, but is it paid? Is it subscriber based?

Eric Mies: I think if it's live, then there might be ads. But what you're starting to see is people like YouTube TV, Hulu, which is part of Disney and ESPN as well, they have live sports. So there is more live sports getting on streaming so I think that is helping the shift.

Kailey Raymond: That's got to be the big anchor for sure. I mean, football season in the States is just beginning so the way that they have built that business is absolutely incredible with the fact that they spread games from Thursday to Monday. And then of course throughout the week they have you picking your team on fantasy, I am like, "Y'all really figured it out." What changes do you see coming in the horizon and the next six to 12 months as it relates to customer data in the ways that you're using data to power a lot of your programs?

Eric Mies: This might be longer than six to 12, but I think we'll continue to see more laws, more platforms, like you said, the death of the cookie and so more of those types of changes and limiting certain types of user tracking and giving users their power back to opt-in or opt-out of that type of tracking. So I think we will continue to see that. I think we will continue to see the shift into streaming and the competition heat up. So back to football and streaming, Amazon just spent a lot of money to be able to host all of the Thursday Night Football games this year. And so that's new for them, they're spending a lot of money on that. Prime spent a lot of money on the new Lord of the Ring series, so the streaming companies are really heating up in the content they're buying as they roll out their paid tiers, there'll be more opportunities for advertisers to hit there.

So I think the paid tier is something I'm definitely interested in. Our company doesn't have that today, it's very split SVOD, AVOD, so I think that will be another continued change we see. Data-wise, I think the rise of contextual, I think that's going to be a big thing. There's companies like IRIS TV that are coming out and really helping publishers like us add another layer into contextual targeting and they're working with the ad server companies to also bake in their solution. And so I think the rise of contextual targeting and using those abilities. And I think Apple has really increased their ad revenue by offering and proving that marketers can use contextual targeting to hit their demographic. And so there was a recent study with them giving information about that and why limited ad tracking isn't the end of the world for advertisers. So yeah, I think that will be the other big thing.

Kailey Raymond: Yeah. They definitely need to do some damage control around a lot of that for advertisers, but I do think it is the only way to go. Protecting consumer privacy I think is incredibly important and valuable. And actually that's what we see too, when people implement Segment, leveraging first party data, their conversions go up. So it's like over time when you're actually investing in this really rich source of information and then channeling it to every downstream application, you are spending less time, you're increasing your conversions, it's all working together. Interested to see how those trends continue to shift and shape, but I think consumers are always going to win on this one. And the privacy implications that have been coming down for the past five, 10 years, keep getting stronger across every country in the world.

Eric Mies: Absolutely. And I think we'll see more states within the US roll out their own policies like CCPA. So it'll be interesting, but we're ready for it.

Kailey Raymond: You're ready for it, you're like, "I've got the technology to go, here we are." My last question for you, Eric, before I let you off the hook is, do you have any recommendations for anybody that might be stepping into your shoes that would help them to up level some of their advertising strategies?

Eric Mies: It's three things. I think it's understanding your users and back to that first party data, what they like, who they are, what they're interested in. I think it's understanding your advertisers when they come to you for running a campaign and what their KPIs are. And then it's working internally with your team to help them drive those KPIs. Looking at the data constantly, responding, optimizing and making sure you're having an impact.

Kailey Raymond: Listen to people, listen to your customers on both sides of the house, make sure you understand what those goal posts are and how to hit them. Really appreciate it, Eric. Thanks so much for being here.

Eric Mies: Yeah. Thank you.