Being a market researcher, I found the article “Good to Great, or Just Good?” to be a fabulous read. It reminded me that we always have to be careful about what we read. Particularly when it relates to “facts” found in supposedly sound research studies—specifically those that are published in widely popular business books.
Based on the new findings, I think that Niendorf and Beck are right. The Collins study was flawed. Though I would argue that it wasn’t the data mining specifically that caused the flaw, that it was more a problem of not using a generally accepted scientific method; compounded by what appears to be a weak causal link between the independent and dependent variables. Because Collins used regression, some variables (that could have been the true causal factors) may have been through out of the model due to multicollinearity. And the world will never know.
Data mining is an interesting exercise. It can truly provide insights into your own data that you never thought possible. In my opinion, the problem with data mining is that, while it is a science, there is a lot of bias in the set up of the data mining programming that could make the actual data mining flawed. The worst part is that you may never know the impact the bias had. And, it’s more than just garbage in garbage out. It is more like slightly moldy in to stinky and putrid out (but in a rose colored package).
Another thing that I liked about the study conducted by Niendorf and Beck is that they identify the potential weakness of their own study. That said, I would have to agree that the chances that the companies jettisoned their GTG ways after the Collins study was concluded is probably pretty low. Particularly since a popular book stating how wonderful their management practices were had just been published!