Experimentation Works: The Surprising Power of Business Experiments
Stefan H. Thomke
Harvard Business Review Press (February 2020)
How to fully deploy the scientific method in almost any organization
The potential power and value of experimentation in business would not be a surprise to anyone who has any experience with that process. For at least several thousand years, “success” in experimentation has achieved (if not exceeded) the desired results. Consider Odysseus’ development of a large wooden horse on the plains of Troy. More recently, Thomas Edison explained to a research assistant, “I have not failed. I’ve just found 10,000 ways that don’t work.”
As Stefan Thomke explains, “I’ve spent over twenty-five years studying experimentation in business and, along the way, benefited tremendously from the work of many scholars and practicioners who are referenced throughout this book. I think they would all agree with me: [begin italics[ Experimentation works! [end italics] But to fully benefit from the surprising power of business experiments, companies have to invest in an [begin italics[ experimentation works [end italics] — the systems, tools, organizing principles, values, and behaviors that enable today’s managers to think and act scientifically, with high velocity, with precision, and at large scale. This book will show you how it’s done.”
Thomke suggests that there are two immensely important questions that everyone involved in experimentation should constantly ask: “What can we learn from this experiment?” and “What can we learn from the next?” One of Thomke’s most important points is that an experiments should not be conducted and evaluated in isolation. Rather, in [begin italics] context [end italics], as part of an on-going process.
Francis Bacon provided the foundation for all this when he introduced the scientific method in his classic work Novum Organum in 1620. As Thomke notes, “Thinking and acting scientifically has had an enormous impact on the world. For centuries, we’ve built and organized scientific and technological knowledge through testable explanations and predictions.”
I was especially interested in Thomke’s discussion of the power of “high-velocity incrementalism” that enables experimentation organizations to develop “a laser-like focus on the things that truly matter and don’t rely as much on experience, past data, intuition, imitation, or so-called best practices.”
These are among the passages of greatest interest and value to me, also listed to suggest the scope of Thomke’s coverage:
o Team New Zealand’ continuous experimentation (Pages 7-8, 29-36, 42-45, and 47-49)
o Questions to ask/answer when planning an experiment (8-9 and 55)
o Learning from experiments, including failure (20-22 and 120-124)
o Thomas Alva Edison (20-22 and 38-39)
o Francis Bacon (25-26 and 132-135)
o Experimentation tools (25-27, 139-141, 189-190, and 206-212)
o Testable hypotheses (31-32 and 54-60)
o High-velocity learning (34-49)
o Characteristics of good business experimentation (51-80)
o Reliability of results (64-70)
o Big data (66-72)
o Online experimentation (81-113)
o Large-scale experimentation (92-95, 147-149, 188, 216-219, and 225-229)
o Building trust in experimentation (101-108 and 209-210)
o Understanding results (102-106)
o Experimentation culture (115-151)
o Learning mindset (117-124)
o Booking.com (153-185)
o Lukas Vermeer (159-160, 166-169, and 174-176)
o David Vismans (163-164, 173-174, and 182-183
In the Epilogue, I was delighted to encounter once again another passage of special significance, from Shakespeare’s last great play, The Tempest:
“O wonder!
How many goodly creatures are there here!
How beauteous mankind is! O brave new world,
That has such people in’t.”
With heartfelt passion as well as exceptional intellect, Stefan Thomke urges his reader to embark on a journey into the future and help to build an experimental organization. The rules and practices for how they go about it are thoroughly examined in this book. Another brave new world awaits. Bon voyage!