Loading ...
Sorry, an error occurred while loading the content.

Re: How to determine sample size for test

Expand Messages
  • Zachariah Freitas
    Hi Philip, I m also curious about what your testing. Because you talked about being concerned that sample sizes needed might be too small. You might be
    Message 1 of 9 , Dec 5, 2007
    • 0 Attachment
      Hi Philip,

      I'm also curious about what your testing. Because you talked about
      being concerned that sample sizes needed might be too small. You
      might be right, if your customers are biased on how they utilize your
      website. Some examples of this might be customers don't purchase on
      weekdays but do purchase on weekends. Customer do one kind of
      behavior at the beginning of a month and an entrirely different type
      of behavior at the end of the month. You have business & individual
      customers at different intervals.

      Basically, testing is complicated if the behavior your trying to
      influence varies over time? Website are transiantly visited unlike
      mailings. This makes it harder to analyze.

      My experience has shown that testing longer is more accurate than
      larger sample sizes when the reason for the behavior changes with
      time. Understand the behavior your trying to influence to understand
      the timing for the test.


      Zach




      --- In webanalytics@yahoogroups.com, "nevertrustab" <patriccc@...>
      wrote:
      >
      > Hi Philip,
      >
      > just out of curiousity: What are you trying to test?
      >
      > Patrick
      >
      >
      >
      > --- In webanalytics@yahoogroups.com, "Philip Shannon"
      > <philip.shannon@> wrote:
      > >
      > > Basic stats' problem:
      > >
      > > I will be running a test version of my homepage (2,000,000 monthly
      > > visits). How long (or how many visits) will I need to test to be
      > > statistically significant (confidence level 95%)? I will be
      sending
      > > 100% of my visitors to the test (I won't be running an a/b test -
      I'm
      > > using historical data as my baseline).
      > >
      > > According to the online calculators and equations I have found, I
      only
      > > need a few days of testing. I find that hard to believe...
      > >
      > > Thanks a lot!
      > >
      >
    Your message has been successfully submitted and would be delivered to recipients shortly.