If you want more details on what’s discussed below, check out Lex’s Growth Masters lesson.
As an early-stage startup, our team's focus has been building the functionality of our product to satisfy existing customers. Because of this, the focus to date has been on output. Now we need to transition to outcomes to hit KPIs and grow. How have you seen teams navigate this shift in thinking?
I've helped several teams through this transition. Starting with KPIs is a good approach. It's easy in the early stages of a company to work off assumptions about customers and their behavior. But as your product and userbase grow, it's more and more important to build up evidence for your decision-making and work towards this transition.
A few tactics you could try:
Defining goals together so everyone is bought into them.
Establishing a working group to help figure out how you'll approach measuring goals (choosing tools, instrumenting events, data capture).
Running some exercises to map out what you know (even if only qualitative customer feedback) and what you need to learn.
Leaning on customer development (interviewing customers) as an input into product output while you build up your quantitative measurement.
How do you build cross-team alignment on naming principles for tracking plans? And, once that's done, how do you migrate from existing (and horribly misaligned) data tracking infrastructure to a more modern one?
One way to approach this is to bring folks together in a working group. Invite a small group of stakeholders from across your team and define some goals. You can assign a couple members of the group to try some naming conventions out and bring back ideas for discussion. Then, try the naming conventions out on your existing events. See how they work. As you go, you can share out progress with your wider team to invite participation from any passionate contributors.
As far as migration, I would recommend not doing it all at once. I tried that and it nearly cost us several teammates. Start with the events that are most used AND most poorly instrumented. Just work in one area of your product at a time. When renaming events, I like to send duplicates in for a period of time to ensure accuracy (aka send the old name and the new one). Then, after a couple of weeks if data looks good, document the change in your taxonomy.
What recommendations do you have for development/growth teams that are trying to scale analytics while their company is growing fast?
I see this all the time. As new products or features keep popping up, it's really challenging to make sure everything's aligned across tools. One way to do this is to ensure that everything has an owner. One thing that happens at large companies is that someone instruments something, like a new tool, and then no one owns it. I see it happen over and over again, where teams are paying a ton of money for tools that no one owns inside their companies. Then, they end up sending garbage data to it and then they end up not being able to use it at all. Ownership is key.
Lex’s data ownership model. Download the worksheet to fill out yours.
What are some ways you’ve seen teams develop internal training on how to use data?
I often point to Airbnb's Data University as the gold standard of internal training:
Airbnb’s Data University Curriculum
Even though most teams are not at that scale, there's a lot to learn from their approach. I especially loved that they met their teammates where they were on data literacy.
I've worked with some tiny teams and we've been able to achieve a lot of knowledge sharing through:
Creating a single place with all data documentation.
Holding monthly trainings to help new teammates get up and running on our tools.
Establishing a Slack channel for data questions.
Pulling together a list of the most relevant documentation from the internet (for example, I always tell people to use Segment's Analytics Academy)
Making it clear who to go to for help
What are some good practices for increasing data literacy on your team?
This answer is similar to the above question, but I would add two things.
Make it really really easy for colleagues to get help with data. Offer office hours. DM people when they ask for data access and offer to help them make a visualization. Be patient with folks. When you lower the bar to using data, more people will use it.
Show your work. When you are writing a query or making a chart, explain what went into it. Or walk through it live in a working session. Teach your team by example.
How do you address data battles where people mistrust or misuse data?
I've seen a lot of mistrust of data. And I've definitely seen some folks share some sloppy charts.
On the mistrust side:
Document their concerns and see if you can investigate them.
Then, go back to them with what you learned and see if you can build more trust.
On the misuse side
Sometimes, you can gently ask questions to ensure data is being presented fairly. "Walk me through how you created this view" or "Can I ask where this number came from?" are good places to start.
Other times, it might just be worth letting it go depending on the stakes. Relationships with your teammates are often more important than bad data.
How do you get designers and engineers more involved in understanding and acting on data?
I actually answered this in detail in Lenny Rachitsky's growth newsletter, but I will summarize it here:
Define metrics together
Review metrics together
Reflect on releases together
I rarely see teams reviewing their goals every week or so when they plan work. Doing this will really help your team get comfortable with what they are looking at and why. It will also encourage questions and interest in learning more.
From a UX perspective, what have you seen some of the biggest mistake teams make in terms of developing user experiences for customers? How do you get ahead of them?
The biggest mistake is not talking to your customers. I see this over and over from founders and from designers. Being able to accurately source and build relationships with your customers to understand their true behavior, situations and attitudes is the key to solving for them.
If you are talking with customers often and truly understanding where they are coming from *before* creating solutions, you will be much closer towards an experience that is meaningful and valuable.
Could you provide some examples of when you set up a specific test, examined the results (which were not optimal), and then had to switch up your following tactic/strategy?
All the time – how much time do you have? : )
Most of the A/B tests I've run are inconclusive which is not optimal and can make decision-making really tough. I've also had several tests fail spectacularly. Those are more fun but also demoralizing for the team.
Here's one of my favorites.
At an e-commerce company I worked at, we ran a test to add a shopping cart into checkout.
Usually, when you buy stuff online, you see what you're buying during checkout. This company wasn't previously showing that and it was resulting in a lot of issues with orders. We also had a strong hypothesis that it was causing people to abandon checkout.
I did several design iterations and ran usability interviews with them. I was pretty close to customers so I took a lot of their input into account.
But when we ran the test, it tanked. Within hours, which was incredibly fast for the size of this cohort, it lost by a landslide.
The team was heartbroken but we picked up the pieces and deconstructed what went wrong.
First, I dug into the analytics. I went through every single user path to see if I could find errors people hit. I also wanted to understand where they went after abandoning checkout.
Second, I reached out to interview the test cohort. I didn't have much luck but I got a couple interviews and some email replies that helped clarify the issue.
What we learned, in the end, was that this new feature was actually highlighting issues that needed to be addressed earlier in the user flow. With this information, we were able to properly prioritize fixing some of the earlier issues with the cart.
Lastly, I'll offer one piece of advice from a data scientist I'm working with now.
As you write your test plan, play forward the results. What will you do if it wins? If it loses? If it's inconclusive? Thinking through this early can help avoid bias and assumption-based decisions later on.
What is Growth UX? And where does a Growth UX team or person fall in small to medium-sized businesses?
Growth UX is a UX practitioner focused on growth. Some folks say all UX is focused on growth and that's absolutely not true. Some UX practitioners focus only on users and may not solve for business growth.
It's important to call out that growth UX is not practiced alone. To achieve growth, you need a cross-functional team of some kind, of which UX is a piece.
Regarding how this fits into teams, it's still unsolved. Some companies establish a growth team. Some establish more than one growth team. Some spread growth responsibility onto all teams.
On a small team, I've seen UX practitioners work on a mix of general product work as well as some growth initiatives. In this case, they usually sit on a product team.
What are the biggest changes you see in the field of UX? If you had to create a new job title/position to prepare for this change what would it be and what would the skillsets look like?
The change I'm paying attention to is designers getting more involved on the business side.
Growth design is a part of that, though there is plenty of design work that is business-focused but not growth-focused. For example, a money-saving project is business-focused but won't grow the business.
For UX practitioners who want to get more involved in business, they need to learn more about:
business models, how businesses make money
strategy (both product but also overall market strategy)
I hesitate to give this a name. My hope is that design as a field becomes more business aware and that it's less of a specialization in the future. Regardless of whether or not a designer has all the skills I listed above, I believe all design should solve for both users and the business. I think we are moving rapidly toward that future.
Focus teams on explicit, singular business goals. Source.
So, you've been hired as a Growth UX person at a new company who is clearly having data issues. What do you tackle first? Second? What are the lower priorities?
The first thing I'd do is get the lay of the land. Talk to everyone you can. Document everything you learn. Diagram how the data is flowing and where any issues might be
You have to do this thoroughly and thoughtfully to build trust and truly understand what's going on.
Then, I would review what you learned with your key stakeholders (probably data or product leaders). Ask them to confirm that your read is accurate. Make a recommendation on where to start.
As far as what to tackle, I would tackle any red flags first. Red flags are data outages, broken tools, broken events, broken pipelines and sensitive user information flowing into tools.
Second, I would tackle any major points of confusion. Cut unnecessary/unused tools. Remove or hide unused events or data being captured.
Consider writing some clear documentation on what's reliable now and the plan for making the rest reliable. Break it into small pieces of work. Include your team along the way.
What are some trends or shifts do you see for the upcoming year in the growth UX space?
It's super exciting that growth UX is a thing now! The first thing I'm seeing is much more awareness that design is a critical component of growth. The amount of roles I see for designers and design leaders focused on growth has tripled this year.
Another trend is an increasing focus on growth in product teams. Designers working in growth can work on marketing, product or even sales. I am seeing more product teams focus on growth and as a result more product designers are becoming growth designers.
Lastly, I'm seeing a lot more designers involved in data. I've even seen roles for designers on data teams. UX designers can offer their human-centric approach to data infrastructure and operations. Previously, data has been left to those with PhDs and strong math affinity, but when UX practitioners can get in and make data more accessible, it can transform how a team uses data to make decisions.
What companies do you admire in terms of Growth UX? Are there prime examples that you encourage companies to look at when talking about examples?
In my initial blog post defining growth design, I highlighted Airbnb and Headspace, both of which have some fantastic growth design.
I think Instagram does a pretty great job of marrying growth and product. Their core growth action is "following" and that's also something most users want to do.
Anytime a product can build a growth loop that functions off its core value proposition, that's when you've really nailed growth design. When you don't need to create a referral program because your growth loop is naturally part of the product experience, that's when you know you've got something.
If you want to access more AMAs like this and ask hosts questions live, sign up for Growth Masters now.