Learning to Experiment

This is part of a series of reflections inspired by my courses at HBX, an online business school cohort powered by Harvard Business School. With Business Analytics, Economics for Managers, and Financial Accounting, I'm learning the fundamentals of business. Find the whole series here.

No one walks around every day setting out to prove they're wrong. With the pressure on to perform better with less resources in every industry, there doesn't feel like room for failure. 

Or is there?

We need to shift our mindset from "doers" to "experimenters." We should want to fail, to prove ourselves wrong. When it comes to assessing how much someone will pay for our product, solution, or service, we can't just guess and hope we're right. We need to know. We need to know not just from the people currently buying from us, but from those who aren't. Why choose a competitor? More importantly, how do we win them over (or back?)

Ask Them

Well, one way is to ask. Companies do this a lot--if our inboxes indicate anything--as surveys have become easier and easier to implement. Surveys help us determine large scale data points for a broad pool of customers, giving us aggregate preferences or sentiment. Best of all, it's cheap. You can tweet it.

As I'm sure you've already seen, though, people lie (sometimes maliciously, sometimes ignorantly). If you're truly innovating, people won't know what to make of your product and wouldn't know where to start pricing it. For something brand new, there's no anchor. You have to ask the right questions, but also to the right people.

If I had asked people what they wanted, they would have said ‘Faster horses.’
— Henry Ford

The other problem with surveys is that it may tell you what's going on, but not why. That's why marketers often turn to the focus group. Get a cross-section of customers (or more helpfully, non-customers) in a room and figure out what they're thinking by interviewing them. You can dig much deeper into their responses, ask follow up questions, and ultimately, find the why. That's the goal, at least.

Focus groups provide a rich depth of qualitative data, but they're expensive and time-intensive (and...people still lie.) It might help us know about the overall product, but not specific features--particularly which features make or break the consumer's decision. 

Build An Experiment

To really know, you need to find another angle. In digital marketing, we never run a hero banner or feature on something as widely trafficked as the home page without running an A/B test. Which is better? Why? It's not enough just to sell the product and make the transaction happen. 

One way to assess what people are willing to pay for each attribute, rather than the whole, is a method called the conjoint analysis.  Respondents decide which item they would purchase based on what's most important to them. HBX used the example of a television set: would you rather have higher quality screen resolution, or a larger screen, or both? Rather than ask them to rank every single possible combination, the conjoint analysis asks the respondent to rank 2 or 3 different combinations (creating pair-wise rankings) and then infers the relative values (called part-worths, or utilities) from those original rankings.

Willingness to pay comes down to the sum of each attribute you're willing to pay for; by adding these together in the conjoint analysis, we can more easily see which features matter and which don't. Think of it this way: do you really care if an airline will serve you food if you never get off the runway? Didn't think so.

The Power of the Marketer

Tools like the ones describe above ultimately allow the marketer to better understand their audience. If you don't know who you're talking to and what they care about, then why bother pick up your pen or run that ad? 

With something as sophisticated as a conjoint analysis, marketers can look at customer segments, creating specific personas and building their preferences. We can look for patterns and find the why behind what makes the product so special. And then, of course, tweet about it.

Most importantly, segmentation builds your business model. If you know which parts of your products people will pay more for, then you know how to price. In the event of something disruptive--be it the next Uber, iPhone, or Google--you can adjust your model based on the "market," the people behind every purchase each day. 

But you'll never know about them and what they care about if you don't get curious enough to experiment.

Find the whole series here. You can learn more about HBX, the platform, and what it has to offer here.