Trending with CSE’s

One thing I have noticed recently is the variations in comparison shopping engines on a day-to-day basis.

With any marketing campaign, you will notice daily fluctuations and it is extremely important to bring some levity to the situation and not overreact. Especially with CSE use, you will notice daily fluctuations in performance. The exact amount of fluctuation will vary depending on the size of your account and the channels being used. However, this does not obscure the point which is to be patient.

Time Range

When judging performance, it is best to measure things on a weekly basis, at a bare minimum. On a recent webinar I was listening to, Tom Leung, Product Manager for Google, recommended making site tests within 2 week time intervals as a minimum. Ideally, he recommended performing site tests over a month long period.

It’s my belief that working with comparison shopping engines is extremely similar. Do not judge results over a few days time as definitive proof something should be done.

Seasonal Variations

One additional thing to take into account when judging performance is seasonal variations. Since we are looking at month long time ranges here, take a look at seasonal differences. For example, if your leprechaun and Easter bunny outfit was not performing well in January, perhaps you should take another look at it.

Seasonal variations contribute greatly to the rise and fall in popularity of products. So, be aware of these variations, even when not as obvious as holiday fluctuations. I recommend being acutely aware if your usual yearly sales decline 5% in April, so you can be aware that if you notice a drop in CSE revenue or return on ad spend, there is most likely a correlation.

CSE Seasonality and Testing : Putting It All Together

So, in conclusion, make sure you are providing enough time to test variations on comparison shopping engines. At the very least, schedule 7 days for a test, but preferably schedule 14 days or more. In testing in general, a statistically significant amount of traffic must pass through the test to make it meaningful. Even more importantly, it has to be done at the right time. Keep in mind that a Christmas ornament will not sell well in July, and if you are testing home improvement items right now, your year to year numbers will be slightly skewed. Give enough time for statistically meaningful information to build and be aware of external conditions. Then, you should be able to squeze a profit out of all that time you are investing.

March 11th, 2008