A/B-test was started on 09.10.2017 and was stopped on 20.10.2017. As it was implement for both Mobile, Desktop and Tablets users, we analyzed results for them separately, because user behaviour can vary on different types of devices. Due to segmented post-test analysis we got interesting results:
Skipped Cart logic even slightly decreased Revenue and Ecommerce Conversion Rate for Mobile segment, but it increased the same metrics by 13.1% and 12.6% for Desktop and Tablets users. As a result of this A/B-test, we increased:
1. Revenue by 13.1% for Desktop and Tablet users
2. Ecommerce Conversion Rate by 12.6% for Desktop and Tablet usersSplit-testing rules:
- A/B test was active within more than 2 Business Cycles (as 95% of all transactions happens on the first visit)
- Both variations had more than 100 ConversionsLearnings for further A/B tests:
1. Skipped cart doesn't decrease AOV because our target audience buys only 1 product per purchase, so we can implement it for 100% of Desktop and Tablet traffic
2. Skipped cart increased Conversion Rate and Revenue only for Desktop and Tablet devices, while on Mobile Conversion rate slightly decreased.Next steps:
- Redistribute all traffic to testing variation for Desktop and Tablet users and create task for Dev Team to implement it in Production.
- Remain standard Add-to-Cart process for Mobile users.