Realities of web analytics – Apply the 3 layers of “So What Test”

Reading one of my favorite authors (and of course blogger too) Avinash Kaushik, I came across his most recent blog post: Kill Useless Web Metrics: Apply The “Three Layers Of So What” Test

Avinash appropriately describes the misconception that certain (what I call noisy) popular web metrics are as important as they seem. Even though a favorable change in one of these metrics looks good, it doesn’t always mean that it is a benefit to your business (and of course your bottom line).

Asking “So What?” three times to every key metric should eventually end in an actionable item.

If at the third “so what” you don’t get a recommendation for an action you should take, you have the wrong metric. Kill it.

You need to face up to the reality that just because it sounds good, or looks good on paper during a board meeting, doesn’t mean that the actual statistic is relevant or worth anything. If you can not act upon it, who cares?

This brutal recommendation is to force you to confront this reality: If you can’t take action, some action (any action!), based on your analysis, why are you reporting data?

While Mr. Kaushik gives some great examples on his blog post, one of the most traditional and possibly easiest to understand references of this goes way back to the origin of the internet in the mid 1990’s. Remember when “Hits” actually mattered, or so some executives were persuaded to think? Not only was it discovered that hits could easily be exploited, but they actually were not very relevant whatsoever.

Obviously one size does not fit all when it comes to web metrics and there may be instances where certain rules do not apply, but they will be the exception and not the norm. Apply your 3 tests and see if you can actually do something with that wonderful key statistic you have been bragging about for the past 6 months.

Categorized in: Miscellaneous

Published On: Mar 11, 2010