Proof

You are the web analytics expert at your company. On any given day, you expect 4% of your visitors to purchase merchandise.

Yesterday, your reporting suggests that the conversion rate was 6%. You wouldn't expect that to happen, the outcome is not what is normally observed.

If you shared this finding with leadership, what would their response be?

• Something is wrong. Did you screw up? Is something wrong with our reporting system? Go find out what went wrong and fix the problem.
  
• This is great! I accept the results.
• This is great! No go find out why this happened, can we do it again?

Of course, some context is required. If the employee routinely makes mistakes, if reporting is frequently wrong, or if the employee has an agenda to hype, the response from leadership is predictable.

But if the employee is trustworthy, and the source of the data is trustworthy, what response would you typically hear from your leadership team?

I am continually surprised that we, as honest, hard working individuals, error on the side of something being wrong. If findings don't match our worldview, or don't fall into a pre-defined set of best practices established by industry veterans, we humans are often more likely to reject the findings than to accept them.

We look for all the reasons the findings could be wrong. When we exhaust those reasons, we demand that the findings be replicated via controlled tests. We advise folks not to listen to the individual who produced the unusual findings until the results can be validated. If the results are threatening to an industry, we'll discredit the individual who announced the findings, or block the access that individual might have to audiences who might want to hear about the findings.

The older the industry is, the more likely we are to hold on to what we knew to be true. Maybe we need to do the opposite, to openly consider other options. Maybe we can act before we have proof.