Don’t Test This on Your Children

Experimentation lies at the heart of science. You’d be hard pressed to find a scientist not keen to run some kind of experiment. A few scientists have even been known to experiment on their own children - a psychologist interested in the laugh response to being tickled comes to mind*.  Experiments are performed to make discoveries, test hypotheses or demonstrate known facts. That may sound fairly mundane, and something that is generally confined to science laboratories around the globe. But experiments bring with them excitement, wonder and surprise and have over the course of history changed our view of the world.

In a world of increasing consumer data volumes, experiments are crucial to business success. You may wonder why we need experiments if we have so much observed data. Experiments are a way of interrogating patterns and response in a specific way. We can set up and create  experiments with set conditions and in ways that allow us to draw more conclusive conclusions about the response of interest, such as how a certain type of consumer responds to a promotional offer. Experiments also allow us to access things we might normally not be able to see or measure. For example, how do daily credit card transactions change when we run a rewards program vs when we don’t?

Amazon is one of the world’s most innovative companies and follows a strategy of continuous evolution, driven by experimentation. It runs hundreds of experiments, from product recommendation engines, testing new products and new markets through to experiments in workforce management. Jeff Bezos, Amazon founder and chairman, had this to say: “Given a 10% chance of a 100x payout, you should take that bet every time. Failure and invention are inseparable twins. To invent you have to experiment, and if you know in advance that it’s going to work, it’s not an experiment”.

In an African context, the development and launch of M-Shwari, the savings and loans product linked to an M-Pesa mobile money account is one of the best examples of research and experimentation leading to success in business. Initial product development was undertaken with the support of FSD Kenya to first understand the needs of clients within their mass market focus. Once the product was designed, the team set out on a cautious testing approach with an initial pilot of just 100 000 clients. After a very successful launch, and with data beginning to accumulate, the team focused on segmenting consumers to understand uptake and usage to better serve consumer’s needs. Segment-specific messages which followed behavioural economics principles were developed to stimulate activity. Message effectiveness was measured against a control group who received no messages at all and was ~95% across all of their experiments **. Today, there are more than 10 million M-Shwari accounts with tens of thousands of loans disbursed every day.

Effectiveness is measured as change in the outcome of interest e.g. increased activity would be measured by deposits in this instance. The process of experimentation with messages directly contributed to increased activity and success of the product.

Experiments aren’t just a defining hallmark of science, but are a special case of something we all do everyday - observe. We observe our customers, our businesses, their successes and the world around us and experiments are a particular manifestation of how we observe.

---------------------------------

* Psychologist Clarence Leuba wanted to find out whether laughing while being tickled is an instinct or whether we learn it from watching other people do it. To test this, he used his infant son as the subject and banned all laughing while touching the child. Leuba would then tickle the child while wearing a cardboard mask  (to mask any facial expressions) in “controlled tickling” treatments. The child did start laughing while being tickled, but the validity of the experiment was questionable as his wife had laughed while bouncing their son on her knee.

** The Growth of M-Shwari in Kenya – A Market Development Story. Going digital and getting to scale with banking services. Case Study Series, November 2016