Here’s a wonderfully geeky question from a forum I’m on, related to lean startup:
I have a product that I want to test. I’ve built the one-pager website and set up some Facebook advertising to figure out the messaging / features that makes people sign up to determine the direction I will eventually build towards.
I have three ads: A, B, A+B
and three website versions: A, B, A+B
My question is whether I should test the advertising AND the website versions at the same time. I want to test quickly, but I’m worried that if someone clicks advertisement A and then lands on website B, it won’t be relevant and it will skew my data.
Is this a warranted concern? Should I test the advertising first, then the website version? Or should I do both and just figure that, with enough users, the data will smooth itself out faster then if I test one at a time? Any insight would be greatly appreciated.