Loading ...
Sorry, an error occurred while loading the content.
 

Re: Question for Win32 Webalizer users?

Expand Messages
  • gisaac@mediaone.net
    Thanks, That is interesting as I was wondering whether or not it was worth compressing either the DNS files or the webalizer.current file. Actually I have
    Message 1 of 5 , Oct 9, 2001
      Thanks,

      That is interesting as I was wondering whether or not it was worth
      compressing either the DNS files or the 'webalizer.current' file.

      Actually I have good news and bad news for you! I ported the UNIX
      version to Windows just a test to see how close I could keep them. It
      basically ported very well and easily, I then worked on the DNS part
      which didnt compile in by default.

      And I have a multithreaded DNS lookup mechanism that uses a
      configurable thread pool (dnschildren) to perform the lookups (max 64
      concurrent threads). In fact this turned out to be reasonably simple.
      However at that time I didnt have access to the Win32 port, so while
      I did things in a similar way as the current Win32 version it is not
      a drop in replacement.

      So in essence the threaded DNS code exists, and I am going to release
      a modified version of Webalizer with a couple of new features:

      * Multi-threaded DNS pool
      * IIS W3C parsing no longer requires the date to be used
      * 404 error report
      * Denied URL report
      * Squid proxy status (HITS vs MISSES vs DENIED etc)

      -- The following are work in progress

      * Ensure all changes are portable to Linux
      * Additional data collection for 304's
      * and a couple of other Squid related reports.
      * Combine the languages into a single executable
      * Potentially compressing 'webalizer.current' to save space on the
      server

      However the release wont be for a month or two.

      I have a particular interest in parsing Squid logs at the moment. If
      someone wants to reintegrate in the multithreaded DNS code into the
      offical Win32 version I can email them a copy. It shouldnt be too
      hard.

      Also do you have any information on the webtrends addin? I havnt seen
      this one before. Do you have a URL for it?

      Thanks,

      Gareth

      --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
      > Well, that what I thought when I first setup webalizer. I'm running
      several
      > servers with Webalizer, some of them have 200+ sites running. These
      sites
      > all create separate logs in separate directories. I'm running
      multiple
      > instances of Webalizer in order to have the stats updated every
      night. At
      > first I setup all these webalizers to use the same dns.db file, for
      I
      > thought it would speed things up (no duplicate DNS lookups). I
      noticed the
      > dns.db file never grew very big, sometime it was around 2MB, but
      never much
      > bigger. Sometimes is was 100-200KB. So I got the impression at some
      point in
      > the webalizer process, the dns.db is rebuilt, discarding existing
      > information. So then I figured it would be better to have a
      separate dns.db
      > for each webalizer. This doesn't seem to have a big impact on the
      speed.
      >
      > I really would like to see some sort of a multi-threaded reverse DNS
      > function in win32 Webalizer, since this is the most time consuming
      part of
      > the whole webalizer process. Is it so much different from Unix,
      where there
      > is some sort of a webazolver running?
      >
      > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
      names
      > instead of IP adresses in the normal IIS extended logfiles? This
      could speed
      > up webalizer, but I can imagine it results in a bigger load on the
      server
      > since it has to do real-time reverse DNS. I haven't had time to try
      this
      > plug-in in a real environment, but I hope to do so soon.
      >
      > Nick
      > -----Original Message-----
      > From: gisaac@m... [mailto:gisaac@m...]
      > Sent: dinsdag 9 oktober 2001 4:50
      > To: webalizer@y...
      > Subject: [webalizer] Question for Win32 Webalizer users?
      >
      >
      > A curiosity question for those using the Win32 version of
      Webalizer
      > and are using the reverse DNS lookups. How big are your DNS
      files? I
      > would think these files get pretty big as they are text, but was
      > curious how big they got in a live environment.
      >
      > Many thanks,
      >
      > Gareth
      >
      >
      > Yahoo! Groups Sponsor
      >
      >
      >
      > To unsubscribe from this group, send an email to:
      > webalizer-unsubscribe@egroups.com
      > Webalizer homepage: http://www.webalizer.org
      > Webalizer for NT: http://www.medasys-lille.com/webalizer/
      >
      >
      >
      >
      > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
      Service.
    • Nick Heinemans
      Hi Gareth, You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm. Could you please email me a copy of the
      Message 2 of 5 , Oct 10, 2001
        Hi Gareth,
         
        You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm.
         
        Could you please email me a copy of the multi-threaded reverse DNS code?
         
        Nick
         
        -----Original Message-----
        From: gisaac@... [mailto:gisaac@...]
        Sent: dinsdag 9 oktober 2001 19:07
        To: webalizer@yahoogroups.com
        Subject: [webalizer] Re: Question for Win32 Webalizer users?

        Thanks,

        That is interesting as I was wondering whether or not it was worth
        compressing either the DNS files or the 'webalizer.current' file.

        Actually I have good news and bad news for you! I ported the UNIX
        version to Windows just a test to see how close I could keep them. It
        basically ported very well and easily, I then worked on the DNS part
        which didnt compile in by default.

        And I have a multithreaded DNS lookup mechanism that uses a
        configurable thread pool (dnschildren) to perform the lookups (max 64
        concurrent threads). In fact this turned out to be reasonably simple.
        However at that time I didnt have access to the Win32 port, so while
        I did things in a similar way as the current Win32 version it is not
        a drop in replacement.

        So in essence the threaded DNS code exists, and I am going to release
        a modified version of Webalizer with a couple of new features:

        * Multi-threaded DNS pool
        * IIS W3C parsing no longer requires the date to be used
        * 404 error report
        * Denied URL report
        * Squid proxy status (HITS vs MISSES vs DENIED etc)

        -- The following are work in progress

        * Ensure all changes are portable to Linux
        * Additional data collection for 304's
        * and a couple of other Squid related reports.
        * Combine the languages into a single executable
        * Potentially compressing 'webalizer.current' to save space on the
        server

        However the release wont be for a month or two.

        I have a particular interest in parsing Squid logs at the moment. If
        someone wants to reintegrate in the multithreaded DNS code into the
        offical Win32 version I can email them a copy. It shouldnt be too
        hard.

        Also do you have any information on the webtrends addin? I havnt seen
        this one before. Do you have a URL for it?

        Thanks,

        Gareth

        --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
        > Well, that what I thought when I first setup webalizer. I'm running
        several
        > servers with Webalizer, some of them have 200+ sites running. These
        sites
        > all create separate logs in separate directories. I'm running
        multiple
        > instances of Webalizer in order to have the stats updated every
        night. At
        > first I setup all these webalizers to use the same dns.db file, for
        I
        > thought it would speed things up (no duplicate DNS lookups). I
        noticed the
        > dns.db file never grew very big, sometime it was around 2MB, but
        never much
        > bigger. Sometimes is was 100-200KB. So I got the impression at some
        point in
        > the webalizer process, the dns.db is rebuilt, discarding existing
        > information. So then I figured it would be better to have a
        separate dns.db
        > for each webalizer. This doesn't seem to have a big impact on the
        speed.
        >
        > I really would like to see some sort of a multi-threaded reverse DNS
        > function in win32 Webalizer, since this is the most time consuming
        part of
        > the whole webalizer process. Is it so much different from Unix,
        where there
        > is some sort of a webazolver running?
        >
        > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
        names
        > instead of IP adresses in the normal IIS extended logfiles? This
        could speed
        > up webalizer, but I can imagine it results in a bigger load on the
        server
        > since it has to do real-time reverse DNS. I haven't had time to try
        this
        > plug-in in a real environment, but I hope to do so soon.
        >
        > Nick
        >   -----Original Message-----
        >   From: gisaac@m... [mailto:gisaac@m...]
        >   Sent: dinsdag 9 oktober 2001 4:50
        >   To: webalizer@y...
        >   Subject: [webalizer] Question for Win32 Webalizer users?
        >
        >
        >   A curiosity question for those using the Win32 version of
        Webalizer
        >   and are using the reverse DNS lookups. How big are your DNS
        files? I
        >   would think these files get pretty big as they are text, but was
        >   curious how big they got in a live environment.
        >
        >   Many thanks,
        >
        >   Gareth
        >
        >
        >         Yahoo! Groups Sponsor
        >
        >
        >
        >   To unsubscribe from this group, send an email to:
        >   webalizer-unsubscribe@egroups.com
        >   Webalizer homepage: http://www.webalizer.org
        >   Webalizer for NT: http://www.medasys-lille.com/webalizer/
        >
        >
        >
        >
        >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of
        Service.



        To unsubscribe from this group, send an email to:
        webalizer-unsubscribe@egroups.com
        Webalizer homepage: http://www.webalizer.org
        Webalizer for NT: http://www.medasys-lille.com/webalizer/




        Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
      • Jean-Noel HAYART
        hello, I am delighted by learn that you have working on webalizer win 32 version, i can if you want add your Multi-threaded code to current port. You can also
        Message 3 of 5 , Oct 10, 2001
          hello,

          I am delighted by learn that you have working on webalizer win 32 version, i
          can if you want add your Multi-threaded code to current port.

          You can also download source of this port :

          http://www.medasys-lille.com/webalizer/VersionR04/default.htm

          I send all my code to Brad somes months ago.
          I think that the win32 release is not its priority.

          regards

          -----Message d'origine-----
          De : gisaac@... [mailto:gisaac@...]
          Envoye : mardi 9 octobre 2001 19:07


          A : webalizer@yahoogroups.com
          Objet : [webalizer] Re: Question for Win32 Webalizer users?


          Thanks,

          That is interesting as I was wondering whether or not it was worth
          compressing either the DNS files or the 'webalizer.current' file.

          Actually I have good news and bad news for you! I ported the UNIX
          version to Windows just a test to see how close I could keep them. It
          basically ported very well and easily, I then worked on the DNS part
          which didnt compile in by default.

          And I have a multithreaded DNS lookup mechanism that uses a
          configurable thread pool (dnschildren) to perform the lookups (max 64
          concurrent threads). In fact this turned out to be reasonably simple.
          However at that time I didnt have access to the Win32 port, so while
          I did things in a similar way as the current Win32 version it is not
          a drop in replacement.

          So in essence the threaded DNS code exists, and I am going to release
          a modified version of Webalizer with a couple of new features:

          * Multi-threaded DNS pool
          * IIS W3C parsing no longer requires the date to be used
          * 404 error report
          * Denied URL report
          * Squid proxy status (HITS vs MISSES vs DENIED etc)

          -- The following are work in progress

          * Ensure all changes are portable to Linux
          * Additional data collection for 304's
          * and a couple of other Squid related reports.
          * Combine the languages into a single executable
          * Potentially compressing 'webalizer.current' to save space on the
          server

          However the release wont be for a month or two.

          I have a particular interest in parsing Squid logs at the moment. If
          someone wants to reintegrate in the multithreaded DNS code into the
          offical Win32 version I can email them a copy. It shouldnt be too
          hard.

          Also do you have any information on the webtrends addin? I havnt seen
          this one before. Do you have a URL for it?

          Thanks,

          Gareth

          --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
          > Well, that what I thought when I first setup webalizer. I'm running
          several
          > servers with Webalizer, some of them have 200+ sites running. These
          sites
          > all create separate logs in separate directories. I'm running
          multiple
          > instances of Webalizer in order to have the stats updated every
          night. At
          > first I setup all these webalizers to use the same dns.db file, for
          I
          > thought it would speed things up (no duplicate DNS lookups). I
          noticed the
          > dns.db file never grew very big, sometime it was around 2MB, but
          never much
          > bigger. Sometimes is was 100-200KB. So I got the impression at some
          point in
          > the webalizer process, the dns.db is rebuilt, discarding existing
          > information. So then I figured it would be better to have a
          separate dns.db
          > for each webalizer. This doesn't seem to have a big impact on the
          speed.
          >
          > I really would like to see some sort of a multi-threaded reverse DNS
          > function in win32 Webalizer, since this is the most time consuming
          part of
          > the whole webalizer process. Is it so much different from Unix,
          where there
          > is some sort of a webazolver running?
          >
          > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
          names
          > instead of IP adresses in the normal IIS extended logfiles? This
          could speed
          > up webalizer, but I can imagine it results in a bigger load on the
          server
          > since it has to do real-time reverse DNS. I haven't had time to try
          this
          > plug-in in a real environment, but I hope to do so soon.
          >
          > Nick
          > -----Original Message-----
          > From: gisaac@m... [mailto:gisaac@m...]
          > Sent: dinsdag 9 oktober 2001 4:50
          > To: webalizer@y...
          > Subject: [webalizer] Question for Win32 Webalizer users?
          >
          >
          > A curiosity question for those using the Win32 version of
          Webalizer
          > and are using the reverse DNS lookups. How big are your DNS
          files? I
          > would think these files get pretty big as they are text, but was
          > curious how big they got in a live environment.
          >
          > Many thanks,
          >
          > Gareth
          >
          >
          > Yahoo! Groups Sponsor
          >
          >
          >
          > To unsubscribe from this group, send an email to:
          > webalizer-unsubscribe@egroups.com
          > Webalizer homepage: http://www.webalizer.org
          > Webalizer for NT: http://www.medasys-lille.com/webalizer/
          >
          >
          >
          >
          > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
          Service.



          To unsubscribe from this group, send an email to:
          webalizer-unsubscribe@egroups.com
          Webalizer homepage: http://www.webalizer.org
          Webalizer for NT: http://www.medasys-lille.com/webalizer/




          Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
        Your message has been successfully submitted and would be delivered to recipients shortly.