Getting the Real Facts: How Industry Analyst Reports Can Trick Readers - page 3

Start from the Beginning

It also makes a big difference whether the data was collected from the real-world experiences of interviewees or from their opinion. When analysts ask survey particpants about a hypothetical situation, the answers they get can only describe perceptions on the topic, not reality on the ground. For example, while it is a common perception that Linux administration is more expensive than Windows administration, the actual reliable data suggests otherwise. Therefore a survey asking participants without significant Linux deployments about this topic will yield a suspect result.

As you read the questions and responses of a survey, think about whether the answers are based on speculation or experience. Speculation may be interesting--it can be a good barometer of industry-wide thought on a topic--but it doesn't give you an accurate picture of what's happening in real-world settings.

Lesson: real-world data is more valuable than opinion.