How A/B Testing & Optimisation can reveal valuable insights into brand affinities and the home page experience.
About Robert Dyas
Legendary hardware retailer Robert Dyas has an impressive history that stems back over 140 years. From the innovative to the everyday, the retailer offers a wide variety of home and garden needs: from cleaning solutions and kitchen essentials to the latest DIY gadgets and gardening must-haves. Their extensive selection of products range from some of the world’s most famous brands to exciting new start-ups, accessible to consumers via their 90+ retail locations as well as their robust online store.
Robert Dyas began using Nosto in 2019 and has since deployed a variety of commerce experience strategies across their online store: spanning from Product Recommendations to Segmentation & Insights and Content Personalisation.
Most merchants are familiar with or are using some form of multivariate testing to optimise their ecommerce experiences, but many don’t always get the most out of their test results. Results are often taken at face value without digging deeper to see if there are valuable insights that can be derived by reading between the lines.
In the case of Robert Dyas, they wanted to optimise both their home page experience and their product detail page experiences by testing a different content personalisation strategy and a different product recommendation filtering strategy respectively. The hardware retailer wanted to see how personalising their home page experience using category affinity data affected customer behaviour at their first touch point when returning to the site. Meanwhile, the retailer wanted to also learn how using brand affinity data affects the effectiveness of product recommendations on product detail pages. By uncovering these insights, the retailer would be able to optimise their onsite experience to attract and convert more customers in some of the most consequential moments of their shopping journey.
Robert Dyas chose to use Nosto’s A/B Testing & Optimisation to uncover valuable insights into how different onsite experiences affect the way customers behave onsite. To find out how category and brand affinity affects the way customers engage and convert, they deployed tests on their homepage and product detail pages in order to directly compare these variations in experiences against their status quo.
Test 1: Recommending Products Based on Brand Affinity on Product Description Pages
The first test, which was aimed at discovering how brand affinity affects the effectiveness of product recommendations on PDPs, was set up to compare product recommendations that matched the brand of the product whose page the shopper was currently on to recommendations that varied in brand but focused on similar products. The purpose of the brand affinity test was to determine if brand affinity is a strong indicator of how effective product recommendations will be on a PDP. To yield the highest test engagement, the test was conducted on the YMAL block on the product description page—a high-traffic area of the retailer’s website. Two variations were set:
- Variation A which showcased standard YMAL recommendations
- Variation B which displayed the YMAL filtered by brand of the viewed product which allowed the customer to view alternative products from the same brand.
The tests results would answer the critical question of whether brand affinity or product utility is more important when shoppers are shopping at a multi-brand homeware retailer.
‘Salter’ pan, sold by merchant:
Test #2: Personalising the Home Page with Category Specific Banners
For the second test, Robert Dyas tested how personalised homepage experiences affect shopper engagement compared to their standard homepage experience. Robert Dyas wanted to test if personalised homepage experiences caused different audience segments to engage more. To form the test, the retailer created different variations of their homepage content that had messaging and visuals directed toward specific categories that were tied to the customer’s previous behaviour.
Homepage general banner:
Homepage deals segmented banner:
Test #1 Results: Brand Affinity on Product Description Pages
Overall, customers were 28% more likely to convert when exposed to a recommendation block containing products that were the same brand as the product whose detail page the shopper was currently on. Each segment saw an increase in conversion performance with the new variation and the prospect segment had an exceptional 30% increase in conversion compared to the standard variation. Incidentally, the retailer did not think that this would be the case—which is a critical example of the impact testing can have on a retailer’s performance expectations.
Test # 2 Results: Category Specific Banners on the Home Page Results
Introducing the same kind of campaign test for multiple different segments can be a strong indicator that not all groups of shoppers behave in the same way—which allows for more streamlined content segmentation. Personalised messaging won out for the category ‘Kitchen Electricals’ by a margin of 88%, whereas the shoppers in the ‘Kitchenware’ segment were 78% more likely to convert on the default content. This shows a successful testing strategy involves more than simply setting up and running a test.
‘Kitchenware’ banner test results:
‘Kitchen Electricals’ Banner Results:
“Nosto has enabled us to easily deploy personalised recommendations across the Robert Dyas website, improving customer engagement and conversion. The ability for us to target customer segments and provide a more personalised experience, coupled with the Nosto testing functionality has really enabled us to take huge steps forward in providing a better online experience for our customers.”
Simon Spencelayh, Ecommerce Director at Robert Dyas