Article: The Surprising Power of Online Experiments: Getting the Most Out of A/B and Other Controlled Tests (Kohavi & Thomke, 2017 - Book chapter Innovate by rapid experimentation
9 important questions on Article: The Surprising Power of Online Experiments: Getting the Most Out of A/B and Other Controlled Tests (Kohavi & Thomke, 2017 - Book chapter Innovate by rapid experimentation
Innovation = any change to a business product, service or process that adds value.
- This change can range from an incremental improvement to
the creation of something totally new and unprecedented.
- For example, For Google, an innovation may be launching a
completely new product such as Gmail, Google Maps, or its
Chromebook laptop line.
- But innovation at Google also includes the continuous process
of refining, adding and subtracting features, and evolving the
user interface and experience.
Seven principles of experimentation
To create the most value for your innovation efforts, a few principles are critical. These seven principles apply for any business experiment, whether convergent or divergent:
- Learn early
- Be fast and iterate
- fall in love with the problem, not the solution
- Get credible feedback
- Measure what matters now
- Test your assumptions
- Fail smart
Seven principles of experimentation
2. Be fast and iterate
- The second key principle of experimentation is speed
- The primary goals as a leader is to get his teams to learn faster—in iterative cycles of days rather than weeks or months – as this can become a competitive advantage.
- Increasing the speed of experimentation may require infrastructure, too
- Higher grades + faster learning
- Never study anything twice
- 100% sure, 100% understanding
For my 1st exam I got a C. Then, I started looking for ways to learn better and I came across Study Smart. Since then I have scored a A+, A, A- and another A. I highly recommend it to everyone who wants to hear it. I am so happy with it!!
I was struggling to finish all my first-year subjects for 3 years. Then I discovered Study Smart, which helped me to finish all of them within 3 months.
Because of StudySmart, things are going well in many ways. Before, I always had study stress and no time for anything. Now I feel calm and confident (because of the program). I used to score a B, now I usually score an A or A+.
Hey Chris, I really want to thank you very much for developing the platform, I usually got around C for my exams, now I scored an A+, an A and a A- !!
Since using Study Smart, my average has increased significantly. I even completed a statistics course with a A+ while I'm bad at statistics. I took on extra courses this year, because of Study Smart. Thanks so much!
Ever since I started studying with StudySmart I passed all my exams !!! No more re-sits!! At first, I was happy with a pass and now I only get good grades. Many thanks!
I aced all the courses I studied with Study Smart. For law, I first scored a D, and now an A! I’m 100% sure that I’ll pass all the courses I study with Study Smart.
I always thought I studied well. My grades were fine. I could never have imagined that applying these study skills would have such a huge and direct impact on my grades, my speed and the pleasure I have in learning.
For the first time, I finally feel totally confident about the way I learn. I have the perfect overview and make great progress in terms of speed and the grades I get. Thanks, Chris!
I immediately started to enjoy learning again, and my grades soared. All of a sudden, everything went better. I highly recommend Study Smart to all students..
Being able to create my materials online made the process of studying so much smoother and quicker. I no longer have to spend a huge amount of my time on creating my summaries.
At first, my notes were a mess. Now I use Study Smart to take my notes in a perfectly structured way. It helps me to save a lot of time and do other things I enjoy.
Students that study with Study Smart get better grades. The system applies all the learning methods in such a way that both well-performing students and students who struggle perform much better.
Hi Chris, I would like to thank you very much. Last year I scored C on a course, and this year I got an A+ thanks to your method! Thank you very much!
The largest effect is that my child is more enthusiastic about school! Every parent wants their children to be happy at school, Study Smart made it happen.
In high school, I failed several exams, and that has really changed now. I actually needed this tool very much back then. Since I started studying with StudySmart, I haven't had any D's. My average score is now an A and those are results I would have never had in the past
With my old method, I passed only 3 out of 8 courses. Since I started taking my notes digitally in Study Smart, I have passed all my courses on the first try. For me, StudySmart takes away the stress of wondering if I will pass or not.
Thanks to StudySmart, I passed all my exams, and with better grades than before! On top of that, I have mastered a very good study method now, which I am confident will help me earn my degree.
Seven principles of experimentation
3. Fall in love with the problem, not the solution
- This keeps you focused on the customer and their needs => Customer value
- Focusing on the problem you to consider more than one possible solution.
- If your goal is the solution itself, you stop generating new ideas.
- You become attached emotionally to a creative solution. If you think you only have one idea, you are unwilling to let it go. Thus, you continue driving up the costs of that one idea, trying to make it work. Therefore, don’t focus on the solution, but on the problem.
Seven principles of experimentation
4. Get credible feedback
Credibility starts with the people you speak to: they need to be real (potential) customers, not colleagues who give feedback.
Seven principles of experimentation
5. Measure what matters now
- As interactions become more digitized, the number of things that can be measured is growing, and it is easy to get distracted by all the numbers you could be tracking.
- Solution: try to identify the most important single metric for the success of your innovation (= “the one metric that matters”)
Seven principles of experimenting
6. Test your assumptions
- It is essential to test assumptions to eliminate risk, especially for innovations that take your business into unknown territory.
- If your idea is based on assumptions about customers, their interests, and their willingness to pay for such an innovation, test your assumptions in a series of experiments!
Seven principles of experimenting
7. Fail smart
- Failure is inevitable. We can define failure as trying
something that doesn’t work
- If you try to avoid any failures, you will retreat into whatever
seems most sage and you will never innovate.
- But when you fail, you should fail smart. We can think of
smart failure as one that passes these four tests:
1. Did you learn from the failed test?
2. Did you apply that learning to change your strategy?
3. Did you fail as early and cheap as possible?
4. Did you share your learning (so that others in your organization won’t
make the same mistake)?
Tool: the convergent experimental method
7 steps
This experimental method is particularly useful for innovating on existing products, services, and processes; for optimizing and continually improving them; and for comparing versions in the later stages of an innovation process
The question on the page originate from the summary of the following study material:
- A unique study and practice tool
- Never study anything twice again
- Get the grades you hope for
- 100% sure, 100% understanding
