Can You Predict The Future?

This is part of a series of reflections inspired by my courses at HBX, an online business school cohort powered by Harvard Business School. With Business Analytics, Economics for Managers, and Financial Accounting, I'm learning the fundamentals of business. Find the whole series here.

Would you ever make a serious life decision on the roll of a dice or the sayings of a fortune teller? Probably not. But that's basically what companies do for most of the decisions they make. An executive makes a hunch, so they go for it.

That can work or it can completely backfire on you. The trick is, you don't know which until it's too late.

Though data can't always give you the whole picture, it can help you make a better decision by showing the relationship between two factors. Sometimes, you don't need to test to find the solution. If you have data that has a strong relationship, you can forecast.

How It Works

Determining a point forecast requires regression analysis. Regression helps you understand the structure of a relationship between two variables, but also its magnitude. Based on the relationship found in the regression, we can determine what another point might be, or forecast.

To keep it simple, let's stick with two variables for a linear regression. (Multiple regression to come, based on the syllabus!)

Source: Wikipedia

Source: Wikipedia

With two variables on the graph, we can take a look at what the data points are and determine a best fit line that most accurately averages the points in the set (that would be the red line above). The equation for that best fit line then determines our forecasting equation, y=ab+x.

To forecast, we plug in the new data point (b) we want to know into the equation. Excel has a beautiful regression function so we don't have to actually plot it all out. This allows you to use all of the possible data to make your forecast, rather than use one data point to make a guess.

To become more confident that our answer is correct, we can construct a prediction interval around the point forecast. Just like a confidence interval, this shows us the likelihood that, assuming the equation holds true, our point would fall within that range. Prediction intervals vary based on standard deviation (or error possibility) of the regression, how confident you want to be, and the location of the value itself. If it's well outside of the historical data (otherwise known as what we know) than we can't be totally sure.

In sum, forecasting takes two variables that you know are related in some way and makes an educated guess based on the data as to what a second data point would be. We can't be 100% confident that it's the right answer, but it's much closer than what it would be if we randomly guessed.

So...Can you?

With the rise of big data, predicting the future will only become easier. The more data you have, the more confident you can be about your decision. Keeping an experimental mindset with any data set is critical, though, particularly if factors exist outside of what you can measure. 

Hidden variables pop up in unexpected places. We can't forget the qualitative story behind the quantitative numbers. Regression analysis shows a relationship between two factors and can help us determine if that relationship really exists and how strong it is, but we can still miss the whole picture. Sometimes there's more than one variable at play, or variables we don't know about.

You don't have to put away your magic 8 ball just yet, but this analysis might help clarify your decision making.

Find the whole series here. You can learn more about HBX, the platform, and what it has to offer here.