Loading ...
Sorry, an error occurred while loading the content.

Holes in stats

Expand Messages
  • JA
    Hello,Since we installed webalizer on our cobalt raq3 we have noticed that webalizer is showing erratic behaviour which shows itself in days where the stats
    Message 1 of 4 , Aug 13 3:13 AM
    • 0 Attachment
      Hello,

      Since we installed webalizer on our cobalt raq3 we have noticed that webalizer is showing erratic behaviour which shows itself in
      days where the stats are quite clearly wrong. For example the stats were regulary around the 1000 visits per day mark but on the
      10/19/28th day the visits would drop to 90?

      This has been getting more frequent as the sites have become more popular so that it occurs every 4 days!

      Some background info: Webalizer is set to incremental. TIt is run each night from cron.daily before the logs are rotated by the
      server.

      How can I stop this?

      thanks

      jon

      __________________________________________________
      Do You Yahoo!?
      Everything you'll ever need on one web page
      from News and Sport to Email and Music Charts
      http://uk.my.yahoo.com
    • Lillith Lesanges
      ... Blame Cobalt. :) What happens is this: Every night, logrotate splits out the main apache.log file into the siteXX/logs/web.log files. This is what your
      Message 2 of 4 , Aug 13 8:50 AM
      • 0 Attachment
        On Tuesday, August 13, 2002, at 04:13 AM, JA wrote:
        > Since we installed webalizer on our cobalt raq3 we have noticed that
        > webalizer is showing erratic behaviour which shows itself in
        > days where the stats are quite clearly wrong. For example the stats were
        > regulary around the 1000 visits per day mark but on the
        > 10/19/28th day the visits would drop to 90?

        Blame Cobalt. :)

        What happens is this: Every night, logrotate splits out the main
        apache.log file into the siteXX/logs/web.log files. This is what your
        webalizer is running against.

        The catch is that, after your siteXX/logs/web.log file gets too big,
        logrotate hits -it- too. Before webalizer can run again.

        So you've got:

        logrotate(apache.log); webalizer; logrotate(apache.log); webalizer....
        until web.log gets big and then you have:

        logrotate(apache.log); logrotate(web.log); webalizer;

        So, about a day's worth of stuff gets lost.

        The only solution I've come up with is a PITA to implement. for every
        /etc/logrotate.d/siteXX file, in the web.log section, add a prerotate
        section that runs webalizer. This tells logrotate that, before it can
        rotate a web.log file, it needs to run webalizer on it.

        As far as I know, you'll have to do this manual change for every site when
        you add it, and I won't guarantee that changes in the GUI won't mess it up.

        Lillith K. Lesanges
        Sysadmin/Programmer, MIS, Inc.
      • Bradford L. Barrett
        ... Bad solution.. the Webalizer should _never_ be run against a live log when using incremental mode. Doing so can cause data loss. -- Bradford L. Barrett
        Message 3 of 4 , Aug 13 9:01 AM
        • 0 Attachment
          > The only solution I've come up with is a PITA to implement. for every
          > /etc/logrotate.d/siteXX file, in the web.log section, add a prerotate
          > section that runs webalizer. This tells logrotate that, before it can
          > rotate a web.log file, it needs to run webalizer on it.

          Bad solution.. the Webalizer should _never_ be run against a live log when
          using incremental mode. Doing so can cause data loss.

          --
          Bradford L. Barrett brad@...
          A free electron in a sea of neutrons DoD#1750 KD4NAW

          The only thing Micro$oft has done for society, is make people
          believe that computers are inherently unreliable.
        • Lillith Lesanges
          ... /home/sites/siteXX/logs/web.log is not a live log. apache s /var/log/httpd/access.log is the live log. the daily cron runs logrotate and, only once a
          Message 4 of 4 , Aug 13 9:25 AM
          • 0 Attachment
            On Tuesday, August 13, 2002, Bradford L. Barrett wrote:
            >> The only solution I've come up with is a PITA to implement. for every
            >> /etc/logrotate.d/siteXX file, in the web.log section, add a prerotate
            >> section that runs webalizer. This tells logrotate that, before it can
            >> rotate a web.log file, it needs to run webalizer on it.
            >
            > Bad solution.. the Webalizer should _never_ be run against a live log
            > when using incremental mode. Doing so can cause data loss.

            /home/sites/siteXX/logs/web.log is not a live log.

            apache's /var/log/httpd/access.log is the live log. the 'daily' cron runs
            logrotate and, only once a day, updates the
            /home/sites/siteXX/logs/web.log file. The rest of the time, the web.log
            file is -not- live.

            I do -not- suggest running webalizer against the access.log directly.
            Beyond the badness of running on a live log, it includes all the accesses
            for -all- the sites.

            That's why I suggested adding a prerotate section in the /etc/logrotate.d/
            siteXX files and -not- in the /etc/logrotate.d/apache file.

            (FWIW, I've a raq4, not a raq3, but it should be close enough.)

            Lillith K. Lesanges
            Sysadmin/Programmer, MIS, Inc.
          Your message has been successfully submitted and would be delivered to recipients shortly.