Three ways to structure your experimentation team
The set-up an organisation chooses for its experimentation team has a significant influence on how its program will run. According to the recently released Australian Experimentation Maturity Index 2019, more than 47% of respondents use a centralised model. The remainder employ a decentralised model (25.7%) or work with external consultancies.
This is not unusual. Most organisations start with a centralised model and then as they mature may choose to transition to a hybrid centralised-decentralised version, or work within a decentralised structure. In the US, digital businesses are more likely to run decentralised teams whereas it’s very uncommon in more traditional businesses.
So how does structure affect experimentation programs?
A centralised team
Centralisation is the most common structure in the market. This format enables one team to run experiments on behalf of the entire organisation.
Having a central team is much simpler, which is why organisations choose this model to initiate their experimentation program – it’s much easier to secure buy-in for a small team. A lot of people understand the concept of experimentation and the value it brings but they don’t understand the impact of experimentation on the organisation so they start small, deliver benefits and prove value before spreading experimentation culture throughout the business.
While organisations that are experimentally immature will almost always have a centralised structure, it doesn’t mean centralised teams can’t mature. A mature centralised team will still run experiments at a high velocity as well as have the outcomes of those tests implemented in the business. The mark of a mature organisation in relation to experimentation is when they’re running 10 or more experiments a month, which then turns into effective changes and improvements on the development side.
The trouble with a centralised model is that it’s difficult for the program to scale to service the entire organisation. Of the brands that are ready to scale (13%), nearly all of them blame a centralised model for their inability to do so. This is because centralisation means the program has become unwieldy, so big that to reformat it would mean a loss of IP and process.
Decentralised teams
Decentralised teams are less common and are largely confined to bigger and/or more mature organisations. Decentralisation is where the business has several experimentation squads located in multiple departments, usually attuned to different parts of the sales and marketing funnel – for example top of funnel, first transaction, repeat transactions etc – all of which have distinct objectives.
It is common for each of these teams to build expertise in the type of testing that best suits their part of the funnel, such as A/B testing for lead generation and personalisation for customer experience, which is one of the reasons a business might decentralise.
However, the biggest risk with moving from a centralised to a decentralised model is loss of inter-team communication. To avoid silos, ensure that your experimentation teams meet regularly, share and discuss findings and collaborate so you experience the full benefits experimentation can bring. You will know the decentralised structure is working when you see the funnel becoming more and more efficient with fewer exit points.
Hybrid models
Some entities operate with a hybrid model, either to transition from a centralised to a decentralised structure or to accrue the benefits of both. Hybrid models may also use external consultancies to support and build skills and expertise in internal teams.
One way a hybrid works is when a decentralised team focuses on idea generation, while the centralised team focuses on test prioritisation, test development, deployment and reporting. It means each part of the business can be focused on test ideas relating to their part of the funnel, while the centralised team can focus on the function components of experimentation. This format provides both focus and scale.
Which model?
The structure you choose really comes down to buy-in. Experimentation is a change program and will only work so long as the organisation understands how it will fundamentally change the business from the experimentation team through to the development team through to the company’s bottom line.
A lot of CMOs buy into experimentation as a conceptual idea of ‘test, learn and implement’, but they don’t understand that it should have a wider impact on the business like human resources has through KPIs etc. I talked to one of my clients who thought they were quite mature about this. I said to the CMO, “name one person in your organisation who would get fired if they didn’t put two experiments up per month”. The answer was “no one”. That meant they were never going to get more than one experiment up each month because the KPIs of the individual didn’t reflect a transition towards creating a culture of experimentation.
Maturity is not about the structure of your teams or the experiments, it’s about implementing the outcomes of the experiments or capturing lessons.
Whether a centralised, decentralised or hybrid structure will help your organisation achieve those benefits is up to the existing culture of the business and how you intend to shift the mindset towards experimentation.
Selecting your model will be the key to how your organisation will take up a culture of experimentation. Each have their pros and cons, but the model you chose will have the greatest impact on scale. If you need help, remember external consultancies have the skills to bridge the gaps. Good luck!
ENDS
This article is one of a five part series breaking down the key findings of the 2019 Australian Experimentation Maturity Index. To read more about why we’re tracking experimentation maturity in Australia please click here
For a copy of the report or for further information please don’t hesitate to get in touch via nyassini@newrepublique.com or leave a comment in the section below.
About New Republique
New Republique is an award winning, experience optimisation consultancy specialising in brand, user experience design (UX) and conversion rate optimisation (CRO) strategies, to dramatically improve the performance of websites for brands like BMW Australia, Fantastic Furniture, Chemist Warehouse, HSBC, Employsure, General Pants, Bendigo Bank, Kogan, Retail Apparel Group and OFX.
In 2018 New Republique was awarded the Optimizely APAC Agency of the Year.
