Loading ...
Sorry, an error occurred while loading the content.

Question for Win32 Webalizer users?

Expand Messages
  • gisaac@mediaone.net
    A curiosity question for those using the Win32 version of Webalizer and are using the reverse DNS lookups. How big are your DNS files? I would think these
    Message 1 of 5 , Oct 8, 2001
    • 0 Attachment
      A curiosity question for those using the Win32 version of Webalizer
      and are using the reverse DNS lookups. How big are your DNS files? I
      would think these files get pretty big as they are text, but was
      curious how big they got in a live environment.

      Many thanks,

      Gareth
    • Nick Heinemans
      Well, that what I thought when I first setup webalizer. I m running several servers with Webalizer, some of them have 200+ sites running. These sites all
      Message 2 of 5 , Oct 9, 2001
      • 0 Attachment
        Well, that what I thought when I first setup webalizer. I'm running several servers with Webalizer, some of them have 200+ sites running. These sites all create separate logs in separate directories. I'm running multiple instances of Webalizer in order to have the stats updated every night. At first I setup all these webalizers to use the same dns.db file, for I thought it would speed things up (no duplicate DNS lookups). I noticed the dns.db file never grew very big, sometime it was around 2MB, but never much bigger. Sometimes is was 100-200KB. So I got the impression at some point in the webalizer process, the dns.db is rebuilt, discarding existing information. So then I figured it would be better to have a separate dns.db for each webalizer. This doesn't seem to have a big impact on the speed.
         
        I really would like to see some sort of a multi-threaded reverse DNS function in win32 Webalizer, since this is the most time consuming part of the whole webalizer process. Is it so much different from Unix, where there is some sort of a webazolver running?
         
        Did anybody try the IIS log plug-in from Webtrends, which logs DNS names instead of IP adresses in the normal IIS extended logfiles? This could speed up webalizer, but I can imagine it results in a bigger load on the server since it has to do real-time reverse DNS. I haven't had time to try this plug-in in a real environment, but I hope to do so soon.
         
        Nick
        -----Original Message-----
        From: gisaac@... [mailto:gisaac@...]
        Sent: dinsdag 9 oktober 2001 4:50
        To: webalizer@yahoogroups.com
        Subject: [webalizer] Question for Win32 Webalizer users?

        A curiosity question for those using the Win32 version of Webalizer
        and are using the reverse DNS lookups. How big are your DNS files? I
        would think these files get pretty big as they are text, but was
        curious how big they got in a live environment.

        Many thanks,

        Gareth



        To unsubscribe from this group, send an email to:
        webalizer-unsubscribe@egroups.com
        Webalizer homepage: http://www.webalizer.org
        Webalizer for NT: http://www.medasys-lille.com/webalizer/




        Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
      • gisaac@mediaone.net
        Thanks, That is interesting as I was wondering whether or not it was worth compressing either the DNS files or the webalizer.current file. Actually I have
        Message 3 of 5 , Oct 9, 2001
        • 0 Attachment
          Thanks,

          That is interesting as I was wondering whether or not it was worth
          compressing either the DNS files or the 'webalizer.current' file.

          Actually I have good news and bad news for you! I ported the UNIX
          version to Windows just a test to see how close I could keep them. It
          basically ported very well and easily, I then worked on the DNS part
          which didnt compile in by default.

          And I have a multithreaded DNS lookup mechanism that uses a
          configurable thread pool (dnschildren) to perform the lookups (max 64
          concurrent threads). In fact this turned out to be reasonably simple.
          However at that time I didnt have access to the Win32 port, so while
          I did things in a similar way as the current Win32 version it is not
          a drop in replacement.

          So in essence the threaded DNS code exists, and I am going to release
          a modified version of Webalizer with a couple of new features:

          * Multi-threaded DNS pool
          * IIS W3C parsing no longer requires the date to be used
          * 404 error report
          * Denied URL report
          * Squid proxy status (HITS vs MISSES vs DENIED etc)

          -- The following are work in progress

          * Ensure all changes are portable to Linux
          * Additional data collection for 304's
          * and a couple of other Squid related reports.
          * Combine the languages into a single executable
          * Potentially compressing 'webalizer.current' to save space on the
          server

          However the release wont be for a month or two.

          I have a particular interest in parsing Squid logs at the moment. If
          someone wants to reintegrate in the multithreaded DNS code into the
          offical Win32 version I can email them a copy. It shouldnt be too
          hard.

          Also do you have any information on the webtrends addin? I havnt seen
          this one before. Do you have a URL for it?

          Thanks,

          Gareth

          --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
          > Well, that what I thought when I first setup webalizer. I'm running
          several
          > servers with Webalizer, some of them have 200+ sites running. These
          sites
          > all create separate logs in separate directories. I'm running
          multiple
          > instances of Webalizer in order to have the stats updated every
          night. At
          > first I setup all these webalizers to use the same dns.db file, for
          I
          > thought it would speed things up (no duplicate DNS lookups). I
          noticed the
          > dns.db file never grew very big, sometime it was around 2MB, but
          never much
          > bigger. Sometimes is was 100-200KB. So I got the impression at some
          point in
          > the webalizer process, the dns.db is rebuilt, discarding existing
          > information. So then I figured it would be better to have a
          separate dns.db
          > for each webalizer. This doesn't seem to have a big impact on the
          speed.
          >
          > I really would like to see some sort of a multi-threaded reverse DNS
          > function in win32 Webalizer, since this is the most time consuming
          part of
          > the whole webalizer process. Is it so much different from Unix,
          where there
          > is some sort of a webazolver running?
          >
          > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
          names
          > instead of IP adresses in the normal IIS extended logfiles? This
          could speed
          > up webalizer, but I can imagine it results in a bigger load on the
          server
          > since it has to do real-time reverse DNS. I haven't had time to try
          this
          > plug-in in a real environment, but I hope to do so soon.
          >
          > Nick
          > -----Original Message-----
          > From: gisaac@m... [mailto:gisaac@m...]
          > Sent: dinsdag 9 oktober 2001 4:50
          > To: webalizer@y...
          > Subject: [webalizer] Question for Win32 Webalizer users?
          >
          >
          > A curiosity question for those using the Win32 version of
          Webalizer
          > and are using the reverse DNS lookups. How big are your DNS
          files? I
          > would think these files get pretty big as they are text, but was
          > curious how big they got in a live environment.
          >
          > Many thanks,
          >
          > Gareth
          >
          >
          > Yahoo! Groups Sponsor
          >
          >
          >
          > To unsubscribe from this group, send an email to:
          > webalizer-unsubscribe@egroups.com
          > Webalizer homepage: http://www.webalizer.org
          > Webalizer for NT: http://www.medasys-lille.com/webalizer/
          >
          >
          >
          >
          > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
          Service.
        • Nick Heinemans
          Hi Gareth, You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm. Could you please email me a copy of the
          Message 4 of 5 , Oct 10, 2001
          • 0 Attachment
            Hi Gareth,
             
            You can check out the Webtrends plug-in at http://www.webtrends.com/site_download/plugins.htm.
             
            Could you please email me a copy of the multi-threaded reverse DNS code?
             
            Nick
             
            -----Original Message-----
            From: gisaac@... [mailto:gisaac@...]
            Sent: dinsdag 9 oktober 2001 19:07
            To: webalizer@yahoogroups.com
            Subject: [webalizer] Re: Question for Win32 Webalizer users?

            Thanks,

            That is interesting as I was wondering whether or not it was worth
            compressing either the DNS files or the 'webalizer.current' file.

            Actually I have good news and bad news for you! I ported the UNIX
            version to Windows just a test to see how close I could keep them. It
            basically ported very well and easily, I then worked on the DNS part
            which didnt compile in by default.

            And I have a multithreaded DNS lookup mechanism that uses a
            configurable thread pool (dnschildren) to perform the lookups (max 64
            concurrent threads). In fact this turned out to be reasonably simple.
            However at that time I didnt have access to the Win32 port, so while
            I did things in a similar way as the current Win32 version it is not
            a drop in replacement.

            So in essence the threaded DNS code exists, and I am going to release
            a modified version of Webalizer with a couple of new features:

            * Multi-threaded DNS pool
            * IIS W3C parsing no longer requires the date to be used
            * 404 error report
            * Denied URL report
            * Squid proxy status (HITS vs MISSES vs DENIED etc)

            -- The following are work in progress

            * Ensure all changes are portable to Linux
            * Additional data collection for 304's
            * and a couple of other Squid related reports.
            * Combine the languages into a single executable
            * Potentially compressing 'webalizer.current' to save space on the
            server

            However the release wont be for a month or two.

            I have a particular interest in parsing Squid logs at the moment. If
            someone wants to reintegrate in the multithreaded DNS code into the
            offical Win32 version I can email them a copy. It shouldnt be too
            hard.

            Also do you have any information on the webtrends addin? I havnt seen
            this one before. Do you have a URL for it?

            Thanks,

            Gareth

            --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
            > Well, that what I thought when I first setup webalizer. I'm running
            several
            > servers with Webalizer, some of them have 200+ sites running. These
            sites
            > all create separate logs in separate directories. I'm running
            multiple
            > instances of Webalizer in order to have the stats updated every
            night. At
            > first I setup all these webalizers to use the same dns.db file, for
            I
            > thought it would speed things up (no duplicate DNS lookups). I
            noticed the
            > dns.db file never grew very big, sometime it was around 2MB, but
            never much
            > bigger. Sometimes is was 100-200KB. So I got the impression at some
            point in
            > the webalizer process, the dns.db is rebuilt, discarding existing
            > information. So then I figured it would be better to have a
            separate dns.db
            > for each webalizer. This doesn't seem to have a big impact on the
            speed.
            >
            > I really would like to see some sort of a multi-threaded reverse DNS
            > function in win32 Webalizer, since this is the most time consuming
            part of
            > the whole webalizer process. Is it so much different from Unix,
            where there
            > is some sort of a webazolver running?
            >
            > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
            names
            > instead of IP adresses in the normal IIS extended logfiles? This
            could speed
            > up webalizer, but I can imagine it results in a bigger load on the
            server
            > since it has to do real-time reverse DNS. I haven't had time to try
            this
            > plug-in in a real environment, but I hope to do so soon.
            >
            > Nick
            >   -----Original Message-----
            >   From: gisaac@m... [mailto:gisaac@m...]
            >   Sent: dinsdag 9 oktober 2001 4:50
            >   To: webalizer@y...
            >   Subject: [webalizer] Question for Win32 Webalizer users?
            >
            >
            >   A curiosity question for those using the Win32 version of
            Webalizer
            >   and are using the reverse DNS lookups. How big are your DNS
            files? I
            >   would think these files get pretty big as they are text, but was
            >   curious how big they got in a live environment.
            >
            >   Many thanks,
            >
            >   Gareth
            >
            >
            >         Yahoo! Groups Sponsor
            >
            >
            >
            >   To unsubscribe from this group, send an email to:
            >   webalizer-unsubscribe@egroups.com
            >   Webalizer homepage: http://www.webalizer.org
            >   Webalizer for NT: http://www.medasys-lille.com/webalizer/
            >
            >
            >
            >
            >   Your use of Yahoo! Groups is subject to the Yahoo! Terms of
            Service.



            To unsubscribe from this group, send an email to:
            webalizer-unsubscribe@egroups.com
            Webalizer homepage: http://www.webalizer.org
            Webalizer for NT: http://www.medasys-lille.com/webalizer/




            Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
          • Jean-Noel HAYART
            hello, I am delighted by learn that you have working on webalizer win 32 version, i can if you want add your Multi-threaded code to current port. You can also
            Message 5 of 5 , Oct 10, 2001
            • 0 Attachment
              hello,

              I am delighted by learn that you have working on webalizer win 32 version, i
              can if you want add your Multi-threaded code to current port.

              You can also download source of this port :

              http://www.medasys-lille.com/webalizer/VersionR04/default.htm

              I send all my code to Brad somes months ago.
              I think that the win32 release is not its priority.

              regards

              -----Message d'origine-----
              De : gisaac@... [mailto:gisaac@...]
              Envoye : mardi 9 octobre 2001 19:07


              A : webalizer@yahoogroups.com
              Objet : [webalizer] Re: Question for Win32 Webalizer users?


              Thanks,

              That is interesting as I was wondering whether or not it was worth
              compressing either the DNS files or the 'webalizer.current' file.

              Actually I have good news and bad news for you! I ported the UNIX
              version to Windows just a test to see how close I could keep them. It
              basically ported very well and easily, I then worked on the DNS part
              which didnt compile in by default.

              And I have a multithreaded DNS lookup mechanism that uses a
              configurable thread pool (dnschildren) to perform the lookups (max 64
              concurrent threads). In fact this turned out to be reasonably simple.
              However at that time I didnt have access to the Win32 port, so while
              I did things in a similar way as the current Win32 version it is not
              a drop in replacement.

              So in essence the threaded DNS code exists, and I am going to release
              a modified version of Webalizer with a couple of new features:

              * Multi-threaded DNS pool
              * IIS W3C parsing no longer requires the date to be used
              * 404 error report
              * Denied URL report
              * Squid proxy status (HITS vs MISSES vs DENIED etc)

              -- The following are work in progress

              * Ensure all changes are portable to Linux
              * Additional data collection for 304's
              * and a couple of other Squid related reports.
              * Combine the languages into a single executable
              * Potentially compressing 'webalizer.current' to save space on the
              server

              However the release wont be for a month or two.

              I have a particular interest in parsing Squid logs at the moment. If
              someone wants to reintegrate in the multithreaded DNS code into the
              offical Win32 version I can email them a copy. It shouldnt be too
              hard.

              Also do you have any information on the webtrends addin? I havnt seen
              this one before. Do you have a URL for it?

              Thanks,

              Gareth

              --- In webalizer@y..., "Nick Heinemans" <spam@h...> wrote:
              > Well, that what I thought when I first setup webalizer. I'm running
              several
              > servers with Webalizer, some of them have 200+ sites running. These
              sites
              > all create separate logs in separate directories. I'm running
              multiple
              > instances of Webalizer in order to have the stats updated every
              night. At
              > first I setup all these webalizers to use the same dns.db file, for
              I
              > thought it would speed things up (no duplicate DNS lookups). I
              noticed the
              > dns.db file never grew very big, sometime it was around 2MB, but
              never much
              > bigger. Sometimes is was 100-200KB. So I got the impression at some
              point in
              > the webalizer process, the dns.db is rebuilt, discarding existing
              > information. So then I figured it would be better to have a
              separate dns.db
              > for each webalizer. This doesn't seem to have a big impact on the
              speed.
              >
              > I really would like to see some sort of a multi-threaded reverse DNS
              > function in win32 Webalizer, since this is the most time consuming
              part of
              > the whole webalizer process. Is it so much different from Unix,
              where there
              > is some sort of a webazolver running?
              >
              > Did anybody try the IIS log plug-in from Webtrends, which logs DNS
              names
              > instead of IP adresses in the normal IIS extended logfiles? This
              could speed
              > up webalizer, but I can imagine it results in a bigger load on the
              server
              > since it has to do real-time reverse DNS. I haven't had time to try
              this
              > plug-in in a real environment, but I hope to do so soon.
              >
              > Nick
              > -----Original Message-----
              > From: gisaac@m... [mailto:gisaac@m...]
              > Sent: dinsdag 9 oktober 2001 4:50
              > To: webalizer@y...
              > Subject: [webalizer] Question for Win32 Webalizer users?
              >
              >
              > A curiosity question for those using the Win32 version of
              Webalizer
              > and are using the reverse DNS lookups. How big are your DNS
              files? I
              > would think these files get pretty big as they are text, but was
              > curious how big they got in a live environment.
              >
              > Many thanks,
              >
              > Gareth
              >
              >
              > Yahoo! Groups Sponsor
              >
              >
              >
              > To unsubscribe from this group, send an email to:
              > webalizer-unsubscribe@egroups.com
              > Webalizer homepage: http://www.webalizer.org
              > Webalizer for NT: http://www.medasys-lille.com/webalizer/
              >
              >
              >
              >
              > Your use of Yahoo! Groups is subject to the Yahoo! Terms of
              Service.



              To unsubscribe from this group, send an email to:
              webalizer-unsubscribe@egroups.com
              Webalizer homepage: http://www.webalizer.org
              Webalizer for NT: http://www.medasys-lille.com/webalizer/




              Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
            Your message has been successfully submitted and would be delivered to recipients shortly.