Loading ...
Sorry, an error occurred while loading the content.
 

KB stats in logs

Expand Messages
  • Boulytchev, Vasiliy
    Ladies and Gents, We would llike to utilize the KB counter options in webalizer. The fact that it is not working is most likely due to apache not logging the
    Message 1 of 6 , Apr 19, 2004
      Ladies and Gents,
          We would llike to utilize the KB counter options in webalizer.  The fact that it is not working is most likely due to apache not logging the bytes transferred.  What module do we need to install?  Log file options?
       
      THANKS!
       

      Vasiliy Boulytchev
      Colorado Information Technologies, Inc.
      http://www.coinfotech.com

       


      From: simon benedict [mailto:sbdcunha@...]
      Sent: Monday, April 19, 2004 12:51 PM
      To: webalizer@yahoogroups.com
      Subject: [webalizer] queries abt webalizer

      hi all,
       
      i have installed webalizer and wanna use it to generate graphs with IIS  generated log files
       
      what is the log format i have to choose in IIS and is there any special setting in the webalizer conf files
       
      tappreciate ur help
       
      regards
       
       
      simon


       


      Yahoo! Groups Sponsor
      ADVERTISEMENT
      click here


      Yahoo! Groups Links


      Do you Yahoo!?
      Yahoo! Photos: High-quality 4x6 digital prints for 25ยข

      Webalizer homepage: http://www.webalizer.org
      Webalizer for NT: http://www.medasys-lille.com/webalizer/


      ttp://www.webalizer.org
      Webalizer for NT: http://www.medasys-lille.com/webalizer/




    • enventa2000
      ... The fact ... bytes ... This is strange, because both the common and the combined format log include the bytes sent. There is probably a problem with
      Message 2 of 6 , Apr 20, 2004
        --- In webalizer@yahoogroups.com, "Boulytchev, Vasiliy"
        <vboulytchev@c...> wrote:
        > Ladies and Gents,
        > We would llike to utilize the KB counter options in webalizer.
        The fact
        > that it is not working is most likely due to apache not logging the
        bytes
        > transferred. What module do we need to install? Log file options?


        This is strange, because both the common and the combined format log
        include the bytes sent. There is probably a problem with LogFormat


        You have a line like this in httpd.conf? (The %b part means "bytes
        sent excluding HTTP headers")

        LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}
        i\"" combined

        You should have the line above somewhere in your httpd.conf, and then
        define the different log files using this:

        CustomLog my_log_file.log combined

        I also just posted a message here discussing the use of TransferLog.




        If you still have trouble, you should post your LogFormat lines, and
        some CustomLog and/or TransferLog lines if you have them, as well as
        some VirtualHost container, if you are writing one log for each
        virtual host. Probably there is some problem with the LogFormat.

        (Notice that apache won't log the bytes actually sent, but rather the
        size of the file being sent. If the visitor starts downloading a large
        file buts changes his opinion and stops the download, the log will
        still indicate the whole file size.)


        You have all the LogFormat docs here (for apache 2):

        http://httpd.apache.org/docs-2.0/mod/mod_log_config.html


        >
        > THANKS!
        >
        >
        > Vasiliy Boulytchev
        > Colorado Information Technologies, Inc.
        > http://www.coinfotech.com <http://www.coinfotech.com/>
      • Boulytchev, Vasiliy
        THANKS!!!!!!!!!!! Just for your information, we are using virtual hosting. Access Log snip: www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:48:42 -0600]
        Message 3 of 6 , Apr 21, 2004
          THANKS!!!!!!!!!!!
          Just for your information, we are using virtual hosting.




          Access Log snip:

          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:48:42 -0600] "GET
          /usage/usage.png HTTP/1.1" 200 2938 "http://boulytcheva.com/usage/"
          "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040115"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:52:54 -0600] "GET
          /english_images/english_main_r6_c2.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:04 -0600] "GET
          /english_images/english_main_r8_c6.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:03 -0600] "GET
          /english_images/english_main_r9_c6.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:03 -0600] "GET
          /english_images/english_main_r2_c3.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:03 -0600] "GET
          /english_images/english_main_r2_c3_f2.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:03 -0600] "GET
          /english_images/english_main_r4_c5.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
          www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:55:03 -0600] "GET
          /english_images/english_main_r4_c5_f2.gif HTTP/1.1" 304 -
          "http://boulytcheva.com/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT
          5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"



          LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
          combined
          LogFormat "%h %l %u %t \"%r\" %>s %b" common
          LogFormat "%{Referer}i -> %U" referer
          LogFormat "%{User-agent}i" agent



          Ok, as I can see, 2938 is a byte count in the first
          instance..........However, I don't see bytecount in the later ones. Does it
          put bytecount in every log entry, or are they added on?


          THANKS!


          Vasiliy Boulytchev
          Colorado Information Technologies, Inc.
          http://www.coinfotech.com

          -----Original Message-----
          From: enventa2000 [mailto:enventa2000@...]
          Sent: Tuesday, April 20, 2004 3:50 PM
          To: webalizer@yahoogroups.com
          Subject: [webalizer] Re: KB stats in logs


          --- In webalizer@yahoogroups.com, "Boulytchev, Vasiliy"
          <vboulytchev@c...> wrote:
          > Ladies and Gents,
          > We would llike to utilize the KB counter options in webalizer.
          The fact
          > that it is not working is most likely due to apache not logging the
          bytes
          > transferred. What module do we need to install? Log file options?


          This is strange, because both the common and the combined format log include
          the bytes sent. There is probably a problem with LogFormat


          You have a line like this in httpd.conf? (The %b part means "bytes
          sent excluding HTTP headers")

          LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}
          i\"" combined

          You should have the line above somewhere in your httpd.conf, and then
          define the different log files using this:

          CustomLog my_log_file.log combined

          I also just posted a message here discussing the use of TransferLog.




          If you still have trouble, you should post your LogFormat lines, and
          some CustomLog and/or TransferLog lines if you have them, as well as
          some VirtualHost container, if you are writing one log for each
          virtual host. Probably there is some problem with the LogFormat.

          (Notice that apache won't log the bytes actually sent, but rather the
          size of the file being sent. If the visitor starts downloading a large
          file buts changes his opinion and stops the download, the log will
          still indicate the whole file size.)


          You have all the LogFormat docs here (for apache 2):

          http://httpd.apache.org/docs-2.0/mod/mod_log_config.html


          >
          > THANKS!
          >
          >
          > Vasiliy Boulytchev
          > Colorado Information Technologies, Inc.
          > http://www.coinfotech.com <http://www.coinfotech.com/>






          Webalizer homepage: http://www.webalizer.org
          Webalizer for NT: http://www.medasys-lille.com/webalizer/


          ttp://www.webalizer.org
          Webalizer for NT: http://www.medasys-lille.com/webalizer/



          Yahoo! Groups Links
        • waldo kitty
          ... that really doesn t matter... as long as the log format structure is correct, things should work just fine... [trim] ... this one and the rest are 304
          Message 4 of 6 , Apr 21, 2004
            Boulytchev, Vasiliy wrote:
            > THANKS!!!!!!!!!!!
            > Just for your information, we are using virtual hosting.

            that really doesn't matter... as long as the log format structure is correct, things should work just fine...

            [trim]

            > Access Log snip:
            >
            > www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:48:42 -0600] "GET
            > /usage/usage.png HTTP/1.1" 200 2938 "http://boulytcheva.com/usage/"
            > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040115"
            > www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:52:54 -0600] "GET
            > /english_images/english_main_r6_c2.gif HTTP/1.1" 304 -

            this one and the rest are 304 codes... 304 is "Not Modified" which means that that data is /not/ sent to the remote since it already
            exists in a cache between the requester and the server... those won't get a size value set because there's not one...

            >
            > Ok, as I can see, 2938 is a byte count in the first
            > instance..........However, I don't see bytecount in the later ones. Does it
            > put bytecount in every log entry, or are they added on?

            see above

            --
            _\/
            (@@) Waldo Kitty, Waldo's Place USA
            __ooO_( )_Ooo_____________________ telnet://bbs.wpusa.dynip.com
            _|_____|_____|_____|_____|_____|_____ http://www.wpusa.dynip.com
            ____|_____|_____|_____|_____|_____|_____ ftp://ftp.wpusa.dynip.com
            _|_Eat_SPAM_to_email_me!_YUM!__|_____|_____ wkitty42 -at- alltel.net
          • Bradford L. Barrett
            ... Unsupported log format... ... Bogus %v in front of each record. You cannot use this format with the Webalizer. You can if you move the %v to the end
            Message 5 of 6 , Apr 21, 2004
              > Access Log snip:
              >
              > www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:48:42 -0600] "GET
              > /usage/usage.png HTTP/1.1" 200 2938 "http://boulytcheva.com/usage/"
              > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040115"

              Unsupported log format...

              > LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
              > combined

              Bogus '%v' in front of each record. You cannot use this format with the
              Webalizer. You can if you move the '%v' to the end of the record however.

              --
              Bradford L. Barrett brad@...
              A free electron in a sea of neutrons DoD#1750 KD4NAW

              The only thing Micro$oft has done for society, is make people
              believe that computers are inherently unreliable.
            • enventa2000
              ... GET ... com/usage/ ... GET ... means that that data is /not/ sent to the remote since it already ... won t get a size value set because there s not
              Message 6 of 6 , Apr 26, 2004
                > > Access Log snip:
                > >
                > > www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:48:42 -0600]
                "GET
                > > /usage/usage.png HTTP/1.1" 200 2938 "http://boulytcheva.
                com/usage/"
                > > "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040115"
                > > www.boulytcheva.com 63.247.192.30 - - [21/Apr/2004:13:52:54 -0600]
                "GET
                > > /english_images/english_main_r6_c2.gif HTTP/1.1" 304 -
                >
                > this one and the rest are 304 codes... 304 is "Not Modified" which
                means that that data is /not/ sent to the remote since it already
                > exists in a cache between the requester and the server... those
                won't get a size value set because there's not one...


                Hello.

                I once had a problem with webalizer and managed to solve it, I put
                this here in case someone else has the same problem.

                In my webalizer stats, Internet Explorer would not reload the pictures
                for the stats, just the html page. For some reason Apache would not
                detect that the png file had changed and kept returning 304 codes,
                causing IE to use the pictures in its cache, instead of the new ones.
                This caused the deilusion that stats were not really being generated
                daily.

                To solve it, I added this to the webalizer.conf:

                HTMLHead <HTTP-EQUIV="Pragma" CONTENT="no-cache">

                This is a HTML tag, instructing browsers that they should not really
                cache this file. Browsers will normally obey this, and reload every
                file in the page every visit and every reload. Robots & spiders will
                ignore it if their mission is to copy webs to some hard disk, of
                course.

                It is normally used for dinamic pages, because it will cause browsers
                to always load the page directly from the server and to tell servers
                that they have never ever had a cached copy of the file, causing the
                servers to send the whole file every time.

                In pages without this tag, browsers will keep a copy of the file in
                the cache AND the date they saved the file in the cache. When
                requesting the same file again, the browser will fill a small field in
                the request headers with the save date. The server will compare this
                date in the header with the change date of the file in the server. If
                the server date is younger, the file has obviously changed, and the
                server will answer 200 OK, set the data field size to the file size,
                and fill it with the file data. Otherwise, it will answer 304 NOT
                CHANGED and will set the data field to zero bytes size.

                If the server keeps not noticing the changes in your files, you may
                put a pragma no-cache tag, so the browsers will never send a save
                date, even if they really have a cache copy for, say, offline
                browsing.

                It also helps to solve annoyances like "proxycaches" set by some evil
                and bad ISP like Telefonica de Espana. These "proxycaches" or
                "transparent proxies" will cache every file requested by the ISP
                clients, causing these clients to see many times outdated pages
                because the proxycache won't always bother to really check the
                existance of a new version, or will do so only once a day or every n
                requests.

                If you reload your browser while pressing the Ctrl key, then your
                browser will send a pragma no-cache petition in your request. This
                should cause both your browser cache and your "transparent proxies"
                and "proxycaches" to ignore their cache and request all files again
                from the server.

                (The combined log format usually does not count the bytes in the
                headers. Because a 304 response is composed just by headers, it gets
                counted as a zero bytes response, which is kind of imposible. Better
                say that the data part of the response has zero bytes size. The
                headers must weight by themselves a little less than a kilobyte. This
                varies with the type of response and the info on it.)
              Your message has been successfully submitted and would be delivered to recipients shortly.