Loading ...
Sorry, an error occurred while loading the content.
 

Good Script for unresolved rotating logs?

Expand Messages
  • jeepdriver
    Does anyone have a descent script to handle unresolved logs rotated by Apache s rotatelog? Because apache may need restarting (off the weekly rotation
    Message 1 of 3 , Dec 4, 2002
      Does anyone have a descent script to handle unresolved logs rotated by
      Apache's rotatelog?

      Because apache may need restarting (off the weekly rotation schedule)
      I'm currently doing something like -

      # grab all files generated in the past week, but no the active file.
      for log in `find <logdir> -mtime +1 -mtime -8 -name access_log.*`
      do
      logresolve < $log | webalizer -c somesite.conf
      done

      My problem is that this seems to hang. My files are large enough
      (weekly) that it takes about 2 hours to resolve them. Could this
      cause webalizer to hang up while waiting for a log entry?
    • Waitman C. Gobble, II
      ... I usually do this: Create a logrotate configuration script, and save it as something like /www/web/_logrotate/site.conf... /var/logs/www/site-access_log
      Message 2 of 3 , Dec 4, 2002
        On Wed, 2002-12-04 at 08:46, jeepdriver wrote:
        > Does anyone have a descent script to handle unresolved logs rotated by
        > Apache's rotatelog?
        >


        I usually do this:

        Create a logrotate configuration script, and save it as something like
        /www/web/_logrotate/site.conf...


        "/var/logs/www/site-access_log" /var/logs/www/site-error_log {daily
        olddir /var/logs/archive
        rotate 30
        compress
        sharedscripts
        postrotate
        /usr/bin/killall -HUP httpd
        endscript
        }



        Next I create a shell script that is run from cron, that resolves the
        logs, rotates them and runs webalzer:


        usr/local/httpd/bin/logresolve -s /var/logs/apache/httpd-resolver-stats
        < /var/logs/www/site-access_log >
        /var/logs/www/resolved/site-access_log
        /usr/sbin/logrotate -f /www/web/_logrotate/site.conf
        /usr/local/bin/webalizer -c /www/web/_webalizer/site.conf



        I do this with a couple hundred sites on different servers, many sites
        have massive log files. It seems to run without trouble.

        Also, you probably want to do it on a _daily_ ( _nightly_ ?) basis, not
        weekly.

        Best,

        --
        Waitman Gobble EMK Design Buena Park, California
        http://emkdesign.com +1.7145222528 waitman@...
        Public Key http://pgp.emkdesign.com
        Find an example http://freakinexample.com
      • jeepdriver
        I agree that s a good idea, except that I m not in control of the webserver I m generating stats for. Logs are rsync d to my system. The guy that runs the
        Message 3 of 3 , Dec 4, 2002
          I agree that's a good idea, except that I'm not in control of the
          webserver I'm generating stats for. Logs are rsync'd to my system.
          The guy that runs the webserver pipes the log entries to Apache's
          rotatelog program, so I'm kinda stuck with that. I've tweaked my
          script to use an intermediate file for the resolved log, and webalizer
          seems to like that better than a |'d file.

          Thanks.

          --- In webalizer@y..., "Waitman C. Gobble, II" <waitman@e...> wrote:
          > On Wed, 2002-12-04 at 08:46, jeepdriver wrote:
          > > Does anyone have a descent script to handle unresolved logs rotated by
          > > Apache's rotatelog?
          > >
          >
          >
          > I usually do this:
          >
          > Create a logrotate configuration script, and save it as something like
          > /www/web/_logrotate/site.conf...
          >
          >
          > "/var/logs/www/site-access_log" /var/logs/www/site-error_log {daily
          > olddir /var/logs/archive
          > rotate 30
          > compress
          > sharedscripts
          > postrotate
          > /usr/bin/killall -HUP httpd
          > endscript
          > }
          >
          >
          >
          > Next I create a shell script that is run from cron, that resolves the
          > logs, rotates them and runs webalzer:
          >
          >
          > usr/local/httpd/bin/logresolve -s /var/logs/apache/httpd-resolver-stats
          > < /var/logs/www/site-access_log >
          > /var/logs/www/resolved/site-access_log
          > /usr/sbin/logrotate -f /www/web/_logrotate/site.conf
          > /usr/local/bin/webalizer -c /www/web/_webalizer/site.conf
          >
          >
          >
          > I do this with a couple hundred sites on different servers, many sites
          > have massive log files. It seems to run without trouble.
          >
          > Also, you probably want to do it on a _daily_ ( _nightly_ ?) basis, not
          > weekly.
          >
          > Best,
          >
          > --
          > Waitman Gobble EMK Design Buena Park, California
          > http://emkdesign.com +1.7145222528 waitman@e...
          > Public Key http://pgp.emkdesign.com
          > Find an example http://freakinexample.com
        Your message has been successfully submitted and would be delivered to recipients shortly.