It’s always easy to say data-driven, or to claim to foster a culture in which people are empowered as change-makers and mad scientists.
Mark Zuckerberg’s advice was to “Move fast and break things.”
But the question is how? How do you integrate a growth mindset into your organizational DNA?
In my experience, the most powerful approach is to embrace the “1% experiment” philosophy. I’ve been practicing data-driven approaches in my previous startup experiences — including my current one, which is up close and personal with data-driven approaches.
A data-driven approach is at the heart of Twitter’s success, and you can see a similar approach in other agile, highly innovative companies.
Become the core of product innovation
At Twitter, the 1% experiment gives product developers, engineers, marketers, or others the opportunity to test new ideas in 1% of user sessions, as long as the experiment is well designed and the results can be accurately measured.
Done right, it will be a major growth driver, which is a great idea to improve Twitter’s user experience from the start, not a top-down mandate.
Traditional testing tends to reinforce traditional silos and bureaucracy, and taking the 1% approach is all about democracy for users and employees, shifting power downward from decision makers.
It encourages people to take initiative, to work together, to break down barriers, and to align individual ingenuity with teamwork in the best way possible.
It reinforces solidarity and shared responsibility, and enables people to work proudly together on the basis of innovation and initiative.
This is what you see in companies like Netflix, which promotes a culture of “freedom and responsibility” in order to attract self-motivated workers who expect to constantly test new ways to optimize consumer value.
“Testing our product ideas allowed us to try radical or unpopular ideas without having to make a massive investment,” said John Ciancutti, former VP of personalization technology at Netflix. This allows the best product thinkers to build a track record based on real customer value, allows us to build consensus from debates and implement our best ideas, and helps us avoid the risk of ‘OR’ because we can test many approaches to solving our toughest challenges.”
1% process
This is the classic scientific method, which is used in the laboratory.
First, establish clear assumptions: what is the specific theory or idea being tested? What do you consider a successful outcome? When you create a hypothesis, make sure it’s user-centric.
This kind of empathy allows you to test your ideas without being on the sidelines to benefit your users or your business.
Define the success metrics of the idea, what are the most specific and relevant metrics to use, and what results will demonstrate its feasibility or value?
To be clear: at Twitter, forcing experimenters to lock on to these metrics prevents the “cherry-picking” and “HARKing” from picking only the metrics that support their assumptions, or even changing them to fit the results.
Demonstrate it by building a working test framework to measure results. Ideally, when different teams test their ideas, apply the same framework across the organization to ensure cost effectiveness and consistency,
You can create your own custom testing framework, but there are plenty of existing platforms, such as Appadhoc.
Learning and iteration: It is highly likely that your experimental results will not have any meaningful changes in metrics. Most experiments might yield these results, but that’s the learning process itself.
So be prepared to repeat the experiment several times, and you may find that the initial thought variable works well, or that different levers change your metrics. All good, because this is all the data.
Remember: Keep testing even if the metrics have changed. Continue testing until you find limitations or constraints that drive your idea.
Ship: If you get meaningful positive results, sell your innovation to the organization and hopefully see it unleashed into the wider world.
Repeat the process: Create more ideas and build new hypotheses using what you learned from the last round of experiments.
The experiment case
Twitter’s 1% experiment included a feature called “Quote Tweet” that tested a series of variables after creating the initial hypothesis before identifying the most successful variable — the characteristics users use today.
There are a lot of other changes on Twitter that start with 1% and then push out to all users. Such as moving from “Favorites” (stars) to “likes” (hearts), and introducing larger innovations like states.
Testing will almost always attract some anger or confusion from users or the press, but the basic rule of innovation is that someone will complain about it. This kind of negative feedback is good and helps you understand the limits of change.
Instill the 1% mentality
So what is the key to fostering a 1% experimental mindset in your organization?
Commitment:
If you’re in software or services, where innovation is a survival tool, your team is your best innovation asset. So give them a 1% experimental philosophy as a way to survive and thrive
Set expectations:
Make sure everyone knows that doing the experiment is part of their job description
The delegation:
Delegate creativity to the entire team
Education:
Make sure they are trained in the right methods
Tools:
Give them the resources and equipment they need
Admit:
Reward people for experimenting (even if it doesn’t work) — and spread the culture to others.
Ready for the AB test to leverage X% growth with 1% traffic?
By SparkPost CMO Josh
In this paper, by Shouting technology compiled from: http://www.cmswire.com/digital-workplace/experiment-your-way-to-data-driven-success/