Loading ...
Sorry, an error occurred while loading the content.

Re: webalizer help

Expand Messages
  • enventa2000
    Errr, you re using IIS, so you can t of course run the script I told you. Ejem. Try this: webalizer [your_options] c: rotated_logs access_log* If rotated logs
    Message 1 of 5 , Apr 16, 2004
    • 0 Attachment
      Errr, you're using IIS, so you can't of course run the script I told
      you. Ejem. Try this:

      webalizer [your_options] c:\rotated_logs\access_log*

      If rotated logs have proper names that get the files ordered correctly
      when listing, then webalizer will work well. For names like
      "access_log20040102" (a log rotated in 02/Jan/2004) it will work fine.

      If logs are entered not in order, then all entries should be lumped
      together in a single day (200000 visits in day x and 0 visits from day
      x to today, sometimes with days before day x appearing correctly,
      meaning that some entry from day x has appeared incorrectly after day
      x+1 has been processed)


      NOTE: Remember to delete everything in the output directory before
      trying what I told you, so the old history information is not used
      accidentally.



      Have a nice day.



      --- In webalizer@yahoogroups.com, Enric Naval <enventa2000@y...>
      wrote:
      > --- Michael Stewart <mstewartga@y...> wrote:
      > > I cant get the moderator of the yahoo group to
      > > accept my membership.
      > > I have been pending membership for a week.
      > >
      > > I have webalizer, recently installed, running on IIS
      > > 6 processing 3
      > > websites.
      > >
      > > For some reason, after each process I get only 1
      > > days worth of info
      > > and it deletes the previous days.
      > >
      > >
      > http://www2.mstewart.net:8080/stats/usage_200404.html
      > >
      > > The history file always contains only one line and
      > > the file is time
      > > stamped at the time of processing.
      > >
      > > Any ideas?
      > >
      > > I appreciate your help.
      > >
      > > Regads,
      > >
      > > Michael Stewart
      > >
      > >
      > >
      >
      > Hello:
      >
      > (Please next time post also your webalizer.conf,
      > cutting thigs for privacy if you need to, so we can
      > help better.)
      >
      > Let's me guess, you are rotating logs daily, and in
      > your webalizer.conf you have a line saying:
      >
      > #Incremental no
      >
      > or perhaps:
      >
      > Incremental no
      >
      > (the default value is 'no').
      >
      > You are also of course processing only the logfile you
      > have just rotated that day, not all past rotated logs.
      >
      >
      > Because webalizer has not been told with "Incremental
      > yes" to remember past log files then it will only
      > display statistics about the single daily file it has
      > just processed.
      >
      > OK, you have two solutions:
      >
      > solution 1:
      >
      > Put a line saying "Incremental yes" in your
      > webalizer.conf. This will create a file called
      > webalizer.current in the output directory.
      >
      > Then process all old rotated log files in order.
      > You'll see how webalizer will add them to the
      > statistics. If the incremental info gets corrupted,
      > you'll get to delete webalizer.current and reprocess
      > everything again (you can make it fastly with a
      > script).
      >
      > #!/bin/bash
      > for logfile in $(ls -r /var/log/httpd/access_log.*);
      > do
      > webalizer [some options] $logfile
      > done
      >
      > You may also process all old files and then save the
      > output directory, writing down which was the last file
      > processed (or simply looking at what is the last day
      > appearing in the stats). This way you can recover the
      > directory and reprocess only from that day.
      >
      > NOTE: if you delete old logfiles, and incremental
      > becomes corrupted, you'll lose the data!
      >
      >
      > solution 2:
      >
      > cat all logfiles together in a single file and then
      > add every night the rotated file to the end. Make
      > webalizer re-process that file every night. Prepare to
      > buy more RAM for webalizer so it doesn't resort to
      > swap.
      >
      >
      >
      >
      > To prevent problems:
      >
      > With incremental on, just let webalizer go on
      > analysing. Before it process 1 Jan 2004, save a copy
      > of the stats in a different folder, so you'll have a
      > nice summary page for all 12 months of 2003. Do the
      > same before it process 1 Jan 2004.
      >
      >
      >
      > Last: Your server is not doing reverse-resolving for
      > visitors'IPs. The webalizer is not resolving either,
      > that's why you have no countries stats. To solve this,
      > uncoment the DNSChildren line, so it will attempt to
      > resolve all IPs it finds (expect your DNS server to be
      > hammered for good if you process lots of entries! You
      > can limit the "children" number, "0" disables
      > resolving, all sites are IPs and no countries are
      > found. Of course "0" is the default value... Normally,
      > people relies in Apache to resolve addresses).
      >
      >
      > Buf, sorry for being so verbose. Hope I'm useful.
      >
      > =====
      > Enric Naval
      > Estudiante de Informática de Gestión en la Udl (Lleida)
      >
      >
      >
      >
      > __________________________________
      > Do you Yahoo!?
      > Yahoo! Tax Center - File online by April 15th
      > http://taxes.yahoo.com/filing.html
    • enventa2000
      Then webalizer only uses more RAM when you add more entries to the webalizer.conf file? Entries like SearchEngine xxx x, GroupSite/HideSite,
      Message 2 of 5 , Apr 16, 2004
      • 0 Attachment
        Then webalizer only uses more RAM when you add more entries to the
        webalizer.conf file? Entries like SearchEngine xxx x,
        GroupSite/HideSite, GroupAgent/HideAgent, etc?

        How much RAM would every extra entry add on average? 100 bytes? 64
        bytes/each plus overhead for hash tables?


        --- In webalizer@yahoogroups.com, "Bradford L. Barrett" <brad@m...>
        wrote:
        >
        > [...]
        > > solution 2:
        > >
        > > cat all logfiles together in a single file and then
        > > add every night the rotated file to the end. Make
        > > webalizer re-process that file every night. Prepare to
        > > buy more RAM for webalizer so it doesn't resort to
        > > swap.
        >
        > FYI: There is no difference in RAM usage between using one big log
        file
        > or processing lots of smaller files using incremental mode. The
        same
        > memory is consumed regardless of the method used.
        >
        > Also, Option #3: Just rotate your logs once a month. Process the
        log
        > whenever you want to get an updated report, and once at the end of
        the
        > month after rotation to get the final monthly report. This is the
        > preferred and _recommended_ method to use.
        >
        > --
        > Bradford L. Barrett brad@m...
        > A free electron in a sea of neutrons DoD#1750 KD4NAW
        >
        > The only thing Micro$oft has done for society, is make people
        > believe that computers are inherently unreliable.
      • Bradford L. Barrett
        ... The use of entries such as the above have a minimal effect on the amount of RAM used, but a huge impact on processing time since each entry must be
        Message 3 of 5 , Apr 16, 2004
        • 0 Attachment
          > Then webalizer only uses more RAM when you add more entries to the
          > webalizer.conf file? Entries like SearchEngine xxx x,
          > GroupSite/HideSite, GroupAgent/HideAgent, etc?

          The use of entries such as the above have a minimal effect on the amount
          of RAM used, but a huge impact on processing time since each entry must
          be compared against each log record. If you have lots and lots of
          PageType/IndexAlias/Hide/Group/Ignore/Include/SearchEngine entries, they
          will have a large impact on processing time since each one must be
          processed for each log record, and the impact is linear depending on the
          number of such entries.

          > How much RAM would every extra entry add on average? 100 bytes? 64
          > bytes/each plus overhead for hash tables?

          Depends. Some just keep the match string in a linked list, so it only
          takes however many bytes required for the string + linkage. Others
          will also create a hash table entry of whatever size required for the
          object being referenced. The size depends on the object, since each
          object maintains different metrics particular to the object.

          --
          Bradford L. Barrett brad@...
          A free electron in a sea of neutrons DoD#1750 KD4NAW

          The only thing Micro$oft has done for society, is make people
          believe that computers are inherently unreliable.
        Your message has been successfully submitted and would be delivered to recipients shortly.