Yesterday, I attended a Science Fair at my kids' elementary school. I learned some interesting things:
- Plants do *not* thrive on a diet of vinegar and water. Plain or sugared water is much better (the young researchers also highly recommended chocolate milk as an alternative).
- A ping pong ball floats on water, while a golf ball sinks, although they're both approximately the same size.
- Today's elementary school children understand and can explain the difference in environmental impact between burning coal to generate electricity ("it pollutes the air we breathe") vs using a Windmill to do the same.
- Any experiment that combines dry ice with oozing, bubbling liquids inherently attracts a much larger crowd than one that studies the growth of plants.
The whole school participated in the fair, from Grades K-4, and the projects got progressively more complex at the higher grades. Interestingly, though, the main approach was uniform across all the grades; since these young students are learning the Scientific method , every experiment included the following steps:
- Purpose
- Hypothesis
- Procedure
- Observation
- Results
- Conclusion
As I watched the participants repeatedly going through these steps with minor variations, it suddenly struck me that this recipe was equally applicable, and tremendously useful, to the world of web design. Too often, I've seen web site design decisions being made relatively arbitrarily (often influenced purely by the opinions of high-level managers, without any objective support), rather than based on an analysis of hard data obtained through testing. Since user behavior on web sites is very amenable to clickstream/action tracking and statistical analysis, it is perfectly practical to actually test various hypotheses on user subsets and let the data show you the way.
Some of the Science Fair experiments effectively demonstrated outcomes that were non-intuitive; in the same way, the analysis of user behavior can highlight for us the sometimes unexpected and non-intuitive impact of design choices.
A good scientist assumes nothing and tests everything! Should a good web designer do any less?
----------------
Update: One of my favorite blogs on this topic, Occam's Razor by Avinash Kaushik, has a wonderful post on the topic of web analytics and testing: Experimentation and Testing: A Primer . I highly recommend reading this post if you're interested in this area.
