Description / Hypothesis
In our continuation of SEO-friendly A/B tests on the Timeshare Rentals page, this test will incorporate an FAQ section towards the bottom of the page. The hypothesis is that adding this section will improve SEO performance. We maintain that these changes will not have any significant impact on CVR.
Variations
Post-Mortem Analysis
Control CVR: 2.57%
Variant CVR: 1.97%
Lift: -23.3%
Stat sig? YES – 95% confidence
Even those these changes were implemented toward the bottom of the page (and viewed by only ~2% of users on mobile devices), there was a statistically significant drop in Traveler Offer conversion rate. This is likely due to a disparate number of users in the control segment submitting multiple offers in the same session, but this theory requires deeper analysis to validate.