What makes attribution so hard? Leads don’t always originate from a single source or even from a source that’s clearly identified. Customers can find you offline, either by word-of-mouth or events—does that mean any online engagement you have isn’t worth the investment? Of course not.
Moreover, how do you ensure that measuring attribution and the worth of each acquisition channel does not succumb to the observer effect (or the Hawthorne effect)? A questionnaire is no good if it causes customers to turn their back on your business.
As hard as proper revenue attribution can be, we think it can make the difference between your marketing team being seen as a cost-center or a revenue generation machine.
To prove that hypothesis, we tested the idea ourselves. Here's how we did it.
Launching a survey that won’t annoy customers
With the goal of providing a survey that wouldn’t turn customers away, our team turned to the power of Fullstory integrations. We launched an experiment to redirect half of our customers to an optional customer survey right after signing up (and before activating their account).
The rationale for the placement was that we wanted the survey to show up early enough to be close to the point of conversion (“signing up,” in our case) but not disrupt the onboarding experience after account activation.
In terms of our tech stack, we used Optimizely, an experimentation platform that lets marketing and product teams test, learn and deploy digital experiences. Once integrated, it gives you powerful insights into how users interact with your tests as well as provides ideas on what new tests to run.
Then, we used Jotform, a free online form builder, to create 12 response choices (a user can select as many as they like) and a field for respondents to provide additional information.
Here's what the user flow looked like:
Sign up: The user enters their email and hits the “go” button.
Survey page: The user is prompted by a call-to-action to check their email for an activation link, but also has the opportunity to fill out the survey if they would like.
Submission confirmation page: If the user decides to fill out and submit the survey, we thank them for taking the time to do so.
Results that surprise and delight
The results far exceeded our expectations, with over 3 out of 4 signups who received the survey submitting a response. Pretty impressive!
At first, we thought the result was too good to be true for an optional poll. However, we created a segment for these users, watched several session replays, and realized they interacted with the form as we'd hoped.
The survey helped us understand several things about our customer base:
Our intuitions about word-of-mouth were correct. The largest percentage of respondents indicated that they learned about Fullstory through a colleague or friend.
Similar to a Net Promoter Score (NPS) survey, the open-response helped us discover several digital channels we hadn’t attributed leads to, including Ran Segall’s YouTube channel and Neil Patel’s Marketing School Podcast.
We also discovered other interesting lead sources including previous employers, industry events, and even software engineers exploring a website’s JavaScript console.
Want to take your digital experience to the next level? Let’s chat.
Request your personalized demo of the Fullstory Digital Experience Intelligence platform.
Despite revealing useful insights, the real question is—do surveys detract from users’ experience or prevent sign-ups?
Here, we turned to Fullstory’s DXI platform to analyze digital experience data. Our team searched for sessions in which the survey was answered, and observed real user behavior.
Although users generally navigated our survey seamlessly, we found some potential friction points that could potentially sidetrack some users.
Luckily, the conversion rate differential between our test (those who saw the survey) and control groups (those who did not see the survey) was not statistically significant, leading us to conclude that the survey did not have a meaningful impact on our conversion rates.
Keep in mind, a DXI solution lets you track and search for almost all UX metrics via OmniSearch and conversion funnels. We further checked to see if users exposed to the survey had a lower rate of recording and playing sessions, and again to our relief, they did not.
Useful insights via Digital Experience Intelligence
Ideally, we wanted to incorporate the survey results into our broader business metrics in order to:
Optimize marketing resources to the right channels where people are most receptive
Generate new ideas and test new channels as our company scales
Use the survey data to calibrate quantitative models
At Fullstory, we are heavy users of Data Export. We use it to pipe indexed user event data from our site into Google BigQuery. From there, we query the data and match survey results with the costs and revenue earned for each channel.
With BigQuery, we’ve built a handy dashboard that can tell us the annual recurring revenue and customer acquisition cost for our main digital acquisition channels, combined with some of the categories selected on the survey.
Among other useful insights, identify sponsored events that lead to big accounts—so you’ll be sure to sponsor and attend them again in the years to come.
Determining which product features offer the most value to certain audience segments can be a bit of a challenge. That said, DXI helps bridge the gap. By double-clicking into the why behind users’ decision to visit your site, teams can get useful insights for next time.
