This is part of a series of reflections inspired by my courses at HBX, an online business school cohort powered by Harvard Business School. With Business Analytics, Economics for Managers, and Financial Accounting, I'm learning the fundamentals of business. Find the whole series here.
Once you know the problem, you have to find a way to solve it. In business, we can't always rely on the cold, hard numbers to tell us what we need to know. Sometimes, we have to actually get out there and talk to people. And ask them things.
People introduce a lot of problems into the data, because, well, we're people.
Good marketing relies on data (buzzword: data-driven). That's not easy. To really learn which experience people respond to, how our customer service is performing, or what product they'll actually buy, we have to eliminate bias that comes with being a person.
This gets tricky, because what people say and what they mean is often different. Are you leading them in one direction or another? What non-verbal cues or bias are you introducing into the conversation? How you're asking the question matters as much as the question itself. Take these examples from HBX:
- "Do you believe we should enforce stricter gun control laws to prevent school shootings?"
- "Most people purchase an extended warranty with their car. Would you purchase one?"
- "Are you one of those annoying people who likes musicians like Taylor Swift?"
Each question introduces extra information into that person's decision. Whether that's an image of something horrific, the idea that others are doing something, or providing anchors for the "right" answer, each of these examples would result in skewed data. Consider instead:
- "Do you think gun laws should be stricter, looser, or remain the same?"
- "Would you purchase an extended warranty with the purchase of a new car?"
- "Do you like Taylor Swift?"
Open-ended enough to invite debate, these answers prompt conversation and don't automatically put the respondent in a defensive position. Should the respondent be truthful, we won't have skewed data.
The second part of asking the right questions: asking the right amount of people. No matter what, we'll never be able to ask every single person every burning question on our minds. To approximate what most people would say, we take a sample.
What makes a good sample? Asking one person their opinion on gun control and using that as your data certainly doesn't provide the whole picture. Typically, over 30 responses makes a robust sample size so we can actually analyze the data. Notice that it's not the amount of people you ask that matters, but the number of people who actually say something back.
Put this all together and you can visualize the data with the normal distribution, or the bell curve. This is a distribution of data centered at the mean (or average) that shows the probability of the value occurring. It allows us to determine how confident we can be estimating the real values we look for from the sample. Now, your data might not be "perfect" like this--it might be skewed one side or the other, or take a different shape entirely---but this is the most common shape.
Whether you're out there knocking on doors or tweeting out Survey Monkey links, ask the right questions to ensure that you're not leading your sample in one way or another. Gather enough responses to form a confident estimate that what you've got is worth studying. This is becoming more and more important for marketers so that we find the truth, rather than what we want to tell our bosses is the "truth."