Loading ...
Sorry, an error occurred while loading the content.

RE: [webalizer] Re: Question for Win32 Webalizer users?

Expand Messages
  • Nick Heinemans
    Hi Gareth, You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm. Could you please email me a copy of the
    Message 1 of 5 , Oct 10, 2001
    • 0 Attachment
      Hi Gareth,
       
      You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm.
       
      Could you please email me a copy of the multi-threaded reverse DNS code?
       
      Nick
       
      -----Original Message-----
      From: gisaac@... [mailto:gisaac@...]
      Sent: dinsdag 9 oktober 2001 19:07
      To: webalizer@yahoogroups.com
      Subject: [webalizer] Re: Question for Win32 Webalizer users?

      Thanks,

      That is interesting as I was wondering whether or not it was worth
      compressing either the DNS files or the 'webalizer.current' file.

      Actually I have good news and bad news for you! I ported the UNIX
      version to Windows just a test to see how close I could keep them. It
      basically ported very well and easily, I then worked on the DNS part
      which didnt compile in by default.

      And I have a multithreaded DNS lookup mechanism that uses a
      configurable thread pool (dnschildren) to perform the lookups (max 64
      concurrent threads). In fact this turned out to be reasonably simple.
      However at that time I didnt have access to the Win32 port, so while
      I did things in a similar way as the current Win32 version it is not
      a drop in replacement.

      So in essence the threaded DNS code exists, and I am going to release
      a modified version of Webalizer with a couple of new features:

      * Multi-threaded DNS pool
      * IIS W3C parsing no longer requires the date to be used
      * 404 error report
      * Denied URL report
      * Squid proxy status (HITS vs MISSES vs DENIED etc)

      -- The following are work in progress

      * Ensure all changes are portable to Linux
      * Additional data collection for 304's
      * and a couple of other Squid related reports.
      * Combine the languages into a single executable
      * Potentially compressing 'webalizer.current' to save space on the
      server

      However the release wont be for a month or two.

      I have a particular interest in parsing Squid logs at the moment. If
      someone wants to reintegrate in the multithreaded DNS code into the
      offical Win32 version I can email them a copy. It shouldnt be too
      hard.

      Also do you have any information on the webtrends addin? I havnt seen
      this one before. Do you have a URL for it?

      Thanks,

      Gareth

      --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
      > Well, that what I thought when I first setup webalizer. I'm running
      several
      > servers with Webalizer, some of them have 200+ sites running. These
      sites
      > all create separate logs in separate directories. I'm running
      multiple
      > instances of Webalizer in order to have the stats updated every
      night. At
      > first I setup all these webalizers to use the same dns.db file, for
      I
      > thought it would speed things up (no duplicate DNS lookups). I
      noticed the
      > dns.db file never grew very big, sometime it was around 2MB, but
      never much
      > bigger. Sometimes is was 100-200KB. So I got the impression at some
      point in
      > the webalizer process, the dns.db is rebuilt, discarding existing
      > information. So then I figured it would be better to have a
      separate dns.db
      > for each webalizer. This doesn't seem to have a big impact on the
      speed.
      >
      > I really would like to see some sort of a multi-threaded reverse DNS
      > function in win32 Webalizer, since this is the most time consuming
      part of
      > the whole webalizer process. Is it so much different from Unix,
      where there
      > is some sort of a webazolver running?
      >
      > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
      names
      > instead of IP adresses in the normal IIS extended logfiles? This
      could speed
      > up webalizer, but I can imagine it results in a bigger load on the
      server
      > since it has to do real-time reverse DNS. I haven't had time to try
      this
      > plug-in in a real environment, but I hope to do so soon.
      >
      > Nick
      >   -----Original Message-----
      >   From: gisaac@m... [mailto:gisaac@m...]
      >   Sent: dinsdag 9 oktober 2001 4:50
      >   To: webalizer@y...
      >   Subject: [webalizer] Question for Win32 Webalizer users?
      >
      >
      >   A curiosity question for those using the Win32 version of
      Webalizer
      >   and are using the reverse DNS lookups. How big are your DNS
      files? I
      >   would think these files get pretty big as they are text, but was
      >   curious how big they got in a live environment.
      >
      >   Many thanks,
      >
      >   Gareth
      >
      >
      >         Yahoo! Groups Sponsor
      >
      >
      >
      >   To unsubscribe from this group, send an email to:
      >   webalizer-unsubscribe@egroups.com
      >   Webalizer homepage: http://www.webalizer.org
      >   Webalizer for NT: http://www.medasys-lille.com/webalizer/
      >
      >
      >
      >
      >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of
      Service.



      To unsubscribe from this group, send an email to:
      webalizer-unsubscribe@egroups.com
      Webalizer homepage: http://www.webalizer.org
      Webalizer for NT: http://www.medasys-lille.com/webalizer/




      Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
    • Jean-Noel HAYART
      hello, I am delighted by learn that you have working on webalizer win 32 version, i can if you want add your Multi-threaded code to current port. You can also
      Message 2 of 5 , Oct 10, 2001
      • 0 Attachment
        hello,

        I am delighted by learn that you have working on webalizer win 32 version, i
        can if you want add your Multi-threaded code to current port.

        You can also download source of this port :

        http://www.medasys-lille.com/webalizer/VersionR04/default.htm

        I send all my code to Brad somes months ago.
        I think that the win32 release is not its priority.

        regards

        -----Message d'origine-----
        De : gisaac@... [mailto:gisaac@...]
        Envoye : mardi 9 octobre 2001 19:07


        A : webalizer@yahoogroups.com
        Objet : [webalizer] Re: Question for Win32 Webalizer users?


        Thanks,

        That is interesting as I was wondering whether or not it was worth
        compressing either the DNS files or the 'webalizer.current' file.

        Actually I have good news and bad news for you! I ported the UNIX
        version to Windows just a test to see how close I could keep them. It
        basically ported very well and easily, I then worked on the DNS part
        which didnt compile in by default.

        And I have a multithreaded DNS lookup mechanism that uses a
        configurable thread pool (dnschildren) to perform the lookups (max 64
        concurrent threads). In fact this turned out to be reasonably simple.
        However at that time I didnt have access to the Win32 port, so while
        I did things in a similar way as the current Win32 version it is not
        a drop in replacement.

        So in essence the threaded DNS code exists, and I am going to release
        a modified version of Webalizer with a couple of new features:

        * Multi-threaded DNS pool
        * IIS W3C parsing no longer requires the date to be used
        * 404 error report
        * Denied URL report
        * Squid proxy status (HITS vs MISSES vs DENIED etc)

        -- The following are work in progress

        * Ensure all changes are portable to Linux
        * Additional data collection for 304's
        * and a couple of other Squid related reports.
        * Combine the languages into a single executable
        * Potentially compressing 'webalizer.current' to save space on the
        server

        However the release wont be for a month or two.

        I have a particular interest in parsing Squid logs at the moment. If
        someone wants to reintegrate in the multithreaded DNS code into the
        offical Win32 version I can email them a copy. It shouldnt be too
        hard.

        Also do you have any information on the webtrends addin? I havnt seen
        this one before. Do you have a URL for it?

        Thanks,

        Gareth

        --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
        > Well, that what I thought when I first setup webalizer. I'm running
        several
        > servers with Webalizer, some of them have 200+ sites running. These
        sites
        > all create separate logs in separate directories. I'm running
        multiple
        > instances of Webalizer in order to have the stats updated every
        night. At
        > first I setup all these webalizers to use the same dns.db file, for
        I
        > thought it would speed things up (no duplicate DNS lookups). I
        noticed the
        > dns.db file never grew very big, sometime it was around 2MB, but
        never much
        > bigger. Sometimes is was 100-200KB. So I got the impression at some
        point in
        > the webalizer process, the dns.db is rebuilt, discarding existing
        > information. So then I figured it would be better to have a
        separate dns.db
        > for each webalizer. This doesn't seem to have a big impact on the
        speed.
        >
        > I really would like to see some sort of a multi-threaded reverse DNS
        > function in win32 Webalizer, since this is the most time consuming
        part of
        > the whole webalizer process. Is it so much different from Unix,
        where there
        > is some sort of a webazolver running?
        >
        > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
        names
        > instead of IP adresses in the normal IIS extended logfiles? This
        could speed
        > up webalizer, but I can imagine it results in a bigger load on the
        server
        > since it has to do real-time reverse DNS. I haven't had time to try
        this
        > plug-in in a real environment, but I hope to do so soon.
        >
        > Nick
        > -----Original Message-----
        > From: gisaac@m... [mailto:gisaac@m...]
        > Sent: dinsdag 9 oktober 2001 4:50
        > To: webalizer@y...
        > Subject: [webalizer] Question for Win32 Webalizer users?
        >
        >
        > A curiosity question for those using the Win32 version of
        Webalizer
        > and are using the reverse DNS lookups. How big are your DNS
        files? I
        > would think these files get pretty big as they are text, but was
        > curious how big they got in a live environment.
        >
        > Many thanks,
        >
        > Gareth
        >
        >
        > Yahoo! Groups Sponsor
        >
        >
        >
        > To unsubscribe from this group, send an email to:
        > webalizer-unsubscribe@egroups.com
        > Webalizer homepage: http://www.webalizer.org
        > Webalizer for NT: http://www.medasys-lille.com/webalizer/
        >
        >
        >
        >
        > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
        Service.



        To unsubscribe from this group, send an email to:
        webalizer-unsubscribe@egroups.com
        Webalizer homepage: http://www.webalizer.org
        Webalizer for NT: http://www.medasys-lille.com/webalizer/




        Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
      Your message has been successfully submitted and would be delivered to recipients shortly.