If you want to first test this Experience in a staging environment, you can add an additional URL condition on the first Step (only), for example, URL contains “staging” (or whatever subdomain represents your staging environment.) Once you’ve created and previewed, and are happy with it, you can go back to the Tour to complete the targeting and get ready to go live.
Now that we have our content and placement ready, we can define who this Experience should display to based on the user data sent by Segment to Chameleon.
In our example above, we want to only show the New Reports announcement to users that have completed their onboarding and are also on the Gold plan (as that feature is only available to that plan level.)
Within Chameleon, we can set the Target Audience to be a combination of two filters, for example:
Filter 1: User event = “setup_completed” has occurred
Filter 2: Company attribute = “plan_current” is “Gold”
Here we are assuming that an event “setup_completed” is being sent, but this could be replaced by any other event that indicates a user is done with their onboarding stage or has undertaken activity that qualifies them to see this Experience. We are also showing the “plan_current” as a company (group) property but again this will depend on what user/company data you would like to target this feature announcement around.
Once you’ve set this up, then you can first test this Experience live, either on staging (using the URL rules defined above) or directly in production. To test in production, you can add an additional filter to target only you or your teammates, e.g. “User attribute = “email” contains “yourdomain.com”. This will limit the Experience only showing to users identified with your company’s email address.
You can then set the Experience live, exit the Builder, log in to your app (with the right data, such as “setup_completed” has happened and “plan_current” is “Gold”) and see your Experience in the wild!
Once you’ve seen and dismissed/completed the Experience you will not see it again by default. If you want to test repeatedly, you can either clear your history from the Chrome Extension, or set the Tour to repeat (e.g. on each page load) to allow you to test multiple times. However when you are ready to deploy to your user base then remove the Recurrence setting.
Once you’ve gone live with your Experience you will start to see results and you can measure impact easily. There are multiple ways to see and get data from Chameleon:
View results in the Chameleon dashboard
Download CSV of all events to analyze
Use our APIs to download data in batch or in real-time
Push data to other services via the Segment integration
We recommend you leverage an event analytics tool such as Mixpanel, Amplitude, or Heap, for your product analytics data broadly, and combine your Chameleon data with this.
The benefit of this is that you get a holistic view of product success and can link interactions users are having with your Chameleon Experiences with their other activity and metrics from within the product and outside.
For example, you should ideally track a couple of events around adoption of this new feature; maybe something to indicate discovery (could be a page load for that feature homepage, or a click into a key CTA) and then retention (is that feature used on a recurring basis?)
Then in your analytics tool, you can create some charts to assess performance:
Funnel analysis: to compare users where “Step seen” occurred (for this announcement Experience) followed by the activation event, compared to where “Step seen” did not occur. This will indicate the impact of users seeing the announcement and then completing the feature activation.
Retention analysis: you can compare feature retention between users who saw the Chameleon Experience vs. those who didn’t.
It’s also possible to run more sophisticated control group tests with Chameleon where you randomly sample a proportion of the target audience, and Chameleon sends “Experiment” events to help you run comparative analysis. Read more about that here.
Here’s what we’ve done in this growth recipe:
Created customized in-product announcements that trigger on set pages
Launched in-product announcements to the right users
Measured the impact of in-product feature announcements on retention
You can learn more about using Chameleon and Segment here.