Podcast 5: Key takeouts on preparing your experimentation program for scaling
As the practice of experimentation matures, New Republique has gone from fielding questions about how to start (although we still get those, of course), to how to scale. Once you’ve moved past the initial teething problems of setting up, it can be really difficult to know where to go to next.
In Epsode 5 of the New Republique podcast (listen here) I asked my co-hosts Richard Taylor (Optimizely) and Jessica Vandenbruggen (Nib) about the obstacles to scaling and what they think organisations need to scale effectively. Here are three key takeouts:
Scaling challenges
Jess was upfront when she said talent was hard to find in this space (see Episode 4 of the podcast for further background). But even before that, she admitted it was difficult because it was unclear which roles she was actually looking to fill. “What does the ideal team look like?” she asked Richard.
She also predicted that although she had the buy-in to run an experimentation program within the business (“we’re reaching a point where experimentation is being taken on board by a lot of areas within the business”), as it scales she would need to justify the higher headcount and the extra resources she was requesting.
Richard’s suggestion addressed all three issues. He believes experimentation can advance by stealth.
“What I’ve seen work really well is organisations that, instead of taking on that headcount themselves for the particular team – which can be expensive, especially in an unproven process – actually take portions of those roles from the other teams, which then gets those other teams bought in.”
In keeping with that, I see education as the piece of the puzzle that can really fill in some gaps. If we can educate those ‘borrowed’ team members it could really lift the maturity of experimentation within organisations as well as introduce these skills to a whole range of people who may be part of our next cohort of talent.
He also says there are cases where teams are “winning in a vacuum”, which is to say “you’re testing to find solutions to customer problems, and you’re finding those solutions, but the testing team is detached from the BAU team, and nothing ever gets changed or implemented.” Ouch! To me this speaks to a lack of organisational readiness, which is reflected in our Maturity Index Report.
Begin with the end in mind?
One of the difficulties with scaling is also to do with the question of what an experimentation program looks like once it has scaled. In Jess’ case, her team started with what they had and then worked to build trust in the experimentation process throughout the business before asking for more resources to scale. Richard says it’s “almost impossible… to begin with where you’re going to end. The space is just too new to be able to get that level of buy-in and investment from the business.”
We see a lot of organisations move from a centralised to a decentralised model once they realise their needs. But here’s the key thing: it’s very difficult to go from fully centralised to fully decentralised in one go, especially while you’re scaling.
We advise our clients to divide the experimentation process. Perhaps they can decentralise and scale test ideation while still keeping development centralised. That way the change management side of scaling doesn’t fall apart.
Foundations to scale
So what do you need to scale? I believe very heavily in education and mentorship to get the right talent. Jess adds that connecting with other teams in the community and sharing lessons is also vital. “In order for us to grow, we need to share, we need to start talking about what we’re doing in this space and how we’re doing it… Finding people who have done it before has been a very good learning curve for us.”
I’m lucky that I meet so many people at events like Conversion Jam and Opticon. I also invite myself over to the offices of a Go Group member for a week every year just to watch what they’re doing, which is a practice I think everyone should get into. Rich says one of his customers met the team from the BBC and went on an excursion to the UK to check out what they were doing. Exposure to different ways of doing things, learning from each other, is so important.
Listen to Episode 5 to find out how to document your experimentation journey so you can secure the buy-in you need to justify more resources and scale effectively.
The New Republique Podcast is Australia’s first podcast dedicated to all things CRO (conversion rate optimisation), experimentation and personalisation.
If you are interested in being a guest or would like to suggest a topic for discussion please get in touch with me by sending me an email at hello@newrepublique.com
