5 tips when hiring a strategic experimentation partner
Emerging practices can be a minefield when it comes to finding a conversion rate optimization partner, and the field of experimentation is no different.
There is a cyclical nature to frontier practices where a few early adopters dedicate themselves to new channels or methods. In this period, there’s no formal training because the field is so new. Instead, a process of refining techniques and sharing best practice with the community eventually leads to mainstream recognition and uptake. Following this comes formalisation of the practice. The people at the forefront are notable experts and the next wave learns from them.
Just after mainstream recognition is a period where commoditization of a specialisation occurs. This means for people new to the practice, expertise is hard to distinguish from people who can talk the talk and provide the basics. It’s this uncertainty that makes it difficult for clients to discern who has learnt the craft over years and knows what they are doing, versus someone whose practice is comparatively immature. We’ve seen this recently with conversion rate optimisation, and we’re seeing it now with experimentation at the tipping point from early adoption to mainstream uptake.
The market has woken up to the fact that experimentation is a game-changer. Many organisations have started experimentation programs and are looking for agencies and consultancies to strengthen their internal practice. The split is likely to be between media agencies and UX practitioners both claiming they have experimentation expertise due to media versus experience-led CRO in the past.
So, how can organisations hire the right conversion rate optimization partner to lead their experimentation practice?
- Check for experience in experimentation
History, experience and a strong track record are important. While the practice may only be a few years old, it’s always comforting to know the partner has been involved for most of its life, growing their skillset along with the discipline.
It is also helpful if the partner has a track record of integrating frontier practices when they were previously new. This indicates that they are attuned to emerging methodologies and have demonstrated that they can develop them before successfully bringing them into practice for clients.
- Pay attention to partnerships
Being associated with a technology partner is a good sign because it means the experimentation partner is adept at using the tools and indicates that the tech company thinks the business has a significant chance of traction with clients. A partnership can be seen as an endorsement of the business and confirmation of their ability to demonstrate new methodologies.
- Confirm their experimentation process
Ask for an overview or demonstration of their process, or use a probationary period to see how they operate. This will give you an insight into the methods and processes of the strategic partner and expose any bad habits. Take note of how they embed experimentation in your organisation and find out how willing they are to educate you, the client.
This is less about running tests and more about how they will work with your organisation to set it up to experiment more effectively, which is both a cultural change and a question of having the right infrastructure to run tests and manage the knowledge gained from the results.
- Discover their purpose
If the optimization partner talks about creating a hypothesis or designing tests as the core of their process, move on. The most important part of a test is defining the problem you intend to solve. Spending a little time doing this can save you a lot of time and resources in the long run.
In new methodologies, the ‘why’ behind the practice is the key driver for learning and development. A optimization partner who can demonstrate this first-principle stance is incredibly valuable, and it helps if they have a track record of doing so.
- Ask to set up a culture of experimentation
A partner who just wants to give you the answer is a short-term solution. Instead, find a partner who wants to teach you how to run it yourself. At the end of the day, experimentation is a cultural change and it needs all parties in the organisation to be involved. If you’re going to have someone running it for you and not willing to teach you how to do it yourself, you don’t have a sustainable, long-term solution.
Finding the right optimization partner in the phase of experimentation we’re about to enter is going to be challenging because there are already cowboys on the scene. The best course of action is to know where you are on the maturity curve so you can see what you need from a skills, culture and technology standpoint. Consider how different engagement models might work for your organisation and use these five tips to discover if you’re ready to work together.
Learn how your organisation compares to others in The Pulse Report.
