Mailchimp’s Iterative Process for Support Videos

This post was originally published in MailChimp’s fantastic UX Newsletter. Subscribe for more user experience stories!

MailChimp makes it easy to send emails, but we also pack in some powerful features. Some folks learn how to use these features by reading instructions in our Knowledge Base, while others like to see and hear their lessons in a video.

The video side of our documentation team caters to that latter group by creating brief, step-by-step tutorials that walk users through our app. The average video runs about a minute and a half — short enough to hold someone’s attention and concise enough to provide the information they need.

So far, this approach has proved effective — our top 20 most recent videos have 21,252 total plays, which is huge considering that a majority of those videos are less than a month old. Those same videos show an average engagement rate of 77%, which means that roughly 77% of our viewers watch all the way through to the end of a video.

We thought we’d open the curtains a bit and explain how we identify video subject matter, how we map out our video content, and how we iterate based on viewership and feedback.

Our Challenge: An Overview

Because we cater to a diverse audience with a wide range of needs, our video documentation is a balancing act.

We strive for simplicity in covering complex material

We want to get our customers up the magic escalator of acquired knowledge. We try to make it easy to get in, get acclimated, and move on up, and we do this by teaching the basics with the assumption that a little bit of powerful knowledge will inspire our customers to learn more. To achieve this, we look for the path of least resistance to achieve a goal (and the fewest clicks to get there).

We cater to various learning styles

Everyone learns in different ways — by seeing, by hearing, or by doing. Considering these styles, we show (visually) and describe (audibly) while keeping video “pausers” in mind. The pausers can pause and re-watch to try the actions on their own and then continue on.

We also aren’t too prescriptive, as not everyone likes to be directed. Each video uses first-person phrasing such as “I’ll click” or “I can.” This lets users see how we perform a task, but leaves the steps open to experimentation or customization. This phrasing also comes across as if we’re showing them a lesson one-on-one. We took inspiration from This Old House — those guys use plain language spectacularly to explain what a doohickey is, how it’s used, and what they’ve accomplished by using doohickey A vs. thingamabob B.

We work within time-constraints based on releases and customer interest

Every few weeks, MailChimp is updated — something gets added, refined, or removed. We prioritize our video workload by what has changed the most and what videos indicate the highest viewership.

We strive for accessibility to a global audience

MailChimp is growing around the world. When we create a video, we keep this in mind by using common — not slang — language, adhering to the naming conventions used throughout our documentation (including in closed captions in each video), and being deliberate and specific in each description of action.

Selecting a Topic

We choose topics from the top 250 to 500 searched Knowledge Base articles. There’s a bit of subjectivity involved here, as we rely on our tech support experience to correlate the top searched topics to familiar pain points for our customers. A high number of searches for an article could mean people are really interested in a feature, or they’re really confused by the feature. We also review specific article steps and screenshots to determine if the topic requires a walk-through to better understand it.

We work with our Analytics team to determine which high-traffic articles, like Getting Started with Merge Tags, would benefit from having additional instructional content. For instance, we learned that folks have trouble with formatting merge tags since the “pipe” key is not frequently used, which made us decide to show in our video the “merge tag drop down” function in the editor, which automatically adds the formatted merge tag to the content.

Process

Once we select a topic, we think about how customers ask about a feature. That leads us to the specific angle we adopt for a topic — we list out the points to cover and draft a script for each topic. It’s always helpful for us to state the video’s goals at the start of a tutorial, and again at the end to reinforce the message — the clearer the message in a video, the easier for users to follow it. Our intention is to empower users to figure things out on their own, and reduce the number of customers entering our Customer Support queue.

For our scripts, we walk through the ways each process could be performed in the app. We then identify the path that is most sensible and involves the least amount of steps. This helps us figure out where the transitions should be and how to explain the process. There may be multiple ways to use a feature, so we look for the path that gets customers to each step the fastest with the fewest number of clicks. The longer a set of directions is, the more likely we are to lose someone.

With our path determined, we get to the storyboard phase, map out the critical scenes, and begin making our videos with Screenflow.

Using Video Analytics to Gain Insight

Wistia’s analytics give us an unbiased view of how our videos are performing with customers all over the world. As people watch, we gain insight into how people are interacting with each video.

We look for folks who scrub back through the video — they might be struggling with specific words or visuals. Once we discover a trend, we discuss making adjustments for clarification.

When people exit the video around the same point, it’s often an indicator that they just wanted to see where to access a feature. Once they’ve seen where it is, they’ll figure out how to use it on their own.

However, common exit points can also indicate the video is too long or delivers too much information. Short videos hold users attention well, but long videos can explain complicated concepts in detail. We look for a balance when we create new videos.

When we see people are repeatedly watching passages of a video, we listen to the voice-over for words or phrases that might confuse people, and check the visuals to make sure the transition clearly displays how to reach the current step. Before the redesign of MailChimp, we found the phrase “gear icon” (a reference to an old settings menu) received an exceptional number of re-watches from viewers in the United Kingdom and Canada, but in other countries where English is not the primary language it was passed over.

When we see a lot of skipping around in videos it’s a signal that people are looking for a speedy overview of a topic. To solve the problem we look at cutting frequently skipped content to focus on what’s most relevant.

Engagement

Engagement is based on many factors. If customers aren’t engaged with a video, we look into possible explanations and ask ourselves a lot of questions.

What was difficult to grasp when we first learned MailChimp?

Terminology is always tricky. When we describe buttons and actions in MailChimp, we think of how the name will be perceived by customers and try to use intuitive terms. As mentioned previously, we encountered measurable confusion with the term “gear icon.”

What type of tutorial content could have helped us better understand concepts?

To better understand our own products, it helps to learn new features ourselves with personal MailChimp accounts. This way we learn our product features and discover where we need further explanation.

Are we too instructional at times? Are we not instructional enough?

We try to avoid a large amount of “click here to make this happen” without additional explanation in our videos. Each video opens with an explanation of a feature and how we’ll use it. From there, we show how it functions and reiterate what we just did. We also have to ensure we’re not spending too much time on all the feature can do versus explaining how to best utilize it.

Is our language too American?

MailChimp is growing internationally, and we’re cognizant of this. Sometimes it’s all too easy to slip into familiar turns of phrase that don’t translate across borders.

If the timeline heatmap shows a drop off, was it because the user got the info they needed so quickly that they left early?

We tend to see more drop-offs when we move features around. With the recent redesign, we found a high number of people accessing the videos to see how to find a feature, but as soon as we showed them where, they left. This indicated that they were already familiar with how the feature works.

Are people more informed than we think?

We have to assume some level of understanding of our app when we create videos. However, we also structure the videos to accommodate new users who might only be familiar with concepts and not specifics.

The Job Doesn’t End

If a video shows really high engagement, we pat ourselves on the back for a job well done. At the same time, we wonder if the topic is fairly difficult to grasp, which could lead to a higher percentage of viewers watching to the end. Each video’s stats come from complex conditions, ranging from language barriers to level of previous knowledge of a topic.

We’re constantly learning and iterating based on our viewership. We dissect each video’s stats from multiple angles to figure out if our script and visuals need improvement, or if the content is doing what we intended. And this goes on daily to keep our videos relevant and our customers empowered.