Conversion XL 2017: How Do You Find $100M of Lost Revenue Without Creating a Single Experiment?

Conversion XL 2017, San Antonio, TX, USA

Key Themes:

This presentation emphasizes the importance of functionality testing in the optimization and conversion process.

Bored people = dull topic: Sounds like a boring talk, but you won’t be when you see how much revenue you can make

What is functionality?: It refers to the correct, seamless operation of all elements on a site across devices and browsers — from buttons to navigation.

• Real-life testing:
  A case study revealed serious issues through physical testing (not just relying on digital tools).
  Testing across two tablets, one mobile, and six desktop browsers uncovered a broken button that was impacting sales.

• The revenue impact:
  This small functionality flaw led to massive financial loss

  • £65.7k per year from one broken button

  • Potentially £24 million in broader revenue risk if issues scale.

• Message: It’s an epidemic:
  Functional problems are widespread because companies often rely solely on analytics or assume “no news is good news” when users don’t report bugs.

• SPINACH analogy:
  Just because users don’t complain doesn’t mean problems don’t exist. One person reporting a bug often represents many who simply leave silently.

• The main lesson:

  • Functionality testing must be integrated into every optimization process.

  • Test across all browsers, devices, and real conditions.

  • Fix hidden issues before chasing new experiments or fancy features.

​