The Dangers of Data-Driven Marketing

Marketing has gone digital, and we can now measure our efforts like never before. As a result, marketers have fallen in love with data. Head over heels in love — to the point where we want data to drive our marketing, instead of people, like you and me. I think this has gone too far.

To be clear, I love data. I have an engineering degree and an MBA. Data is my security blanket. But I don’t want data to be the sole driver of marketing at Wistia.

I believe marketing should be data-informed, rather than data-driven.

Data-informed marketing

Instead of focusing on data alone, data-informed marketing considers data as just one factor in making decisions. We then combine relevant data, past experiences, intuition, and qualitative input to make the best decisions we can.

Instead of poring over data hoping to find answers, we develop a theory and a hypothesis first, then test it out. We force ourselves to make more gut calls, but we validate those choices with data wherever possible so that our gut gets smarter with time.

Common problems with data

The issues with data-driven marketing arise because there are a number of problems with how we generally use data.

The ease of measurement trap

When we let data drive our marketing, we all too often optimize for things that are easy to measure, not necessarily what matters most.

Some results are very easy to measure. Others are significantly harder. Click-through rate on an email? Easy. Brand feelings evoked by a well-designed landing page? Hard. Conversion rate of visitors who touch your pricing page? Easy. Word-of-mouth generated from a delightful video campaign? Hard.

While some benefits are easier to measure than others, both the easily quantifiable and the nearly immeasurable are critical for evaluating how your marketing is performing.

The local optimization trap

The local optimization trap typically rears its head when we try to optimize a specific part of the marketing funnel. We face this challenge routinely at Wistia when we try increase the conversion rate of new visitors. In isolation, improving the signup rate is a relatively straightforward optimization problem that can be “solved” with basic testing.

The problem is, we don’t just want visitors to sign up for our Free Plan. We want them to sign up for our Free Plan, then use their account, then tell others how great Wistia is, then eventually purchase one of our paid plans (and along the way generate more and more positive feelings toward our brand).

This is a much harder optimization problem, because we have to track website visitors as they become app users and then customers. We have to track a process that can last weeks or months, instead of a sign-up process that usually happens in minutes.

Local optimization makes data easier to manage, but in our haste to find “quick wins,” we often oversimplify our marketing objectives and end up optimizing for intermediary results that don’t necessarily match our larger business goals.

The data quality trap

We are rarely as critical of our data as we ought to be. Consider, for example, A/B tests, which have become the gold standard for marketing experimentation. In theory, these tests should produce repeatable and accurate results, since website visitors are assigned randomly to each page variant.

“We are rarely as critical of our data as we ought to be.”

In practice, however, there are lots of ways even the simplest A/B tests can produce misleading results. If your website traffic is anything like ours, visitors come from a variety of sources: organic, direct, referral, paid search, and beyond. If one of those sources converts at a much higher rate than others, it’s easy to get skewed results by treating your traffic as a single, uniform audience.

Don’t believe me? Try running an A/A test, where you create two identical variants. Given a large enough sample, you’ll be able to share the great news with your boss that you’ve increased conversion rate by 4% without changing a thing!

Data vigilance

As marketers, we should continue to explore new and better ways to harness the power of data, but we also must remain vigilant about becoming overly reliant on data.

Data can be a tremendous source of insight. Harness that. But don’t pretend it’s something more. And definitely don’t put it in charge of your marketing team.