Loading ...
Sorry, an error occurred while loading the content.

search stats problem

Expand Messages
  • Iain Pople
    Hi, I have been using webalizer to generate statistics on search terms used on our local search engine. We are using ultraseek and the search cgi variable is
    Message 1 of 6 , Jun 18, 2003
    • 0 Attachment
      Hi,

      I have been using webalizer to generate statistics on search terms used
      on our local search engine. We are using ultraseek and the search cgi
      variable is qt. The search engine runs on the host
      websearch.its.unimelb.edu.au so I have the following in my webalizer config:

      SearchEngine websearch.its.unimelb.edu.au qt=

      However I am finding that webalizer is missing a lot of entries. E.g.
      when I do a manual count on a particular term using grep I get a much
      higher value (approx 10x ) than in the webalizer report. Here is a
      typical entry from our web logs:

      128.250.139.35 - - [08/Jun/2003:00:01:07 +1000] "GET
      /ExtRels/Media/UN/archive/1999/414/newsmakers.html HTTP/1.1" 200 14076
      "http://websearch.its.unimelb.edu.au/query.html?qp2=&col2=unimelb&ht2=0&qs2=&qc2=
      &pw2=100%25&ws2=0&qm2=0&st2=1&nh2=25&lk2=1&rf2=0&rq2=0&si2=1&qt=Dr+Franz+Grieser&submit2.x=10&submit2.y=13"
      "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; .NET CLR
      1.0.3705)"

      Could it be that the request is too long for webalizer to process?

      cheers, Iain.
    • Iain Pople
      Hi, I have been using webalizer to generate statistics on search terms used on our local search engine. We are using ultraseek and the search cgi variable is
      Message 2 of 6 , Jun 23, 2003
      • 0 Attachment
        Hi,

        I have been using webalizer to generate statistics on search terms used
        on our local search engine. We are using ultraseek and the search cgi
        variable is qt. The search engine runs on the host
        websearch.its.unimelb.edu.au so I have the following in my webalizer config:

        SearchEngine websearch.its.unimelb.edu.au qt=

        However I am finding that webalizer is missing a lot of entries. E.g.
        when I do a manual count on a particular term using grep I get a much
        higher value (approx 10x ) than in the webalizer report. Here is a
        typical entry from our web logs:

        128.250.139.35 - - [08/Jun/2003:00:01:07 +1000] "GET
        /ExtRels/Media/UN/archive/1999/414/newsmakers.html HTTP/1.1" 200 14076
        "http://websearch.its.unimelb.edu.au/query.html?qp2=&col2=unimelb&ht2=0&qs2=&qc2=
        &pw2=100%25&ws2=0&qm2=0&st2=1&nh2=25&lk2=1&rf2=0&rq2=0&si2=1&qt=Dr+Franz+Grieser&submit2.x=10&submit2.y=13"

        "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; .NET CLR
        1.0.3705)"

        Could it be that the request is too long for webalizer to process?

        cheers, Iain.
      • Iain Pople
        Well, I received no replies to my original question so i ll try and simplify it. What is the maximum size of a referrer log entry that webalizer can handle?
        Message 3 of 6 , Jul 9, 2003
        • 0 Attachment
          Well, I received no replies to my original question so i'll try and
          simplify it. What is the maximum size of a referrer log entry that
          webalizer can handle?

          cheers, Iain.

          Iain Pople wrote:

          > Hi,
          >
          > I have been using webalizer to generate statistics on search terms used
          > on our local search engine. We are using ultraseek and the search cgi
          > variable is qt. The search engine runs on the host
          > websearch.its.unimelb.edu.au so I have the following in my webalizer config:
          >
          > SearchEngine websearch.its.unimelb.edu.au qt=
          >
          > However I am finding that webalizer is missing a lot of entries. E.g.
          > when I do a manual count on a particular term using grep I get a much
          > higher value (approx 10x ) than in the webalizer report. Here is a
          > typical entry from our web logs:
          >
          > 128.250.139.35 - - [08/Jun/2003:00:01:07 +1000] "GET
          > /ExtRels/Media/UN/archive/1999/414/newsmakers.html HTTP/1.1" 200 14076
          > "http://websearch.its.unimelb.edu.au/query.html?qp2=&col2=unimelb&ht2=0&qs2=&qc2=
          > &pw2=100%25&ws2=0&qm2=0&st2=1&nh2=25&lk2=1&rf2=0&rq2=0&si2=1&qt=Dr+Franz+Grieser&submit2.x=10&submit2.y=13"
          >
          > "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; .NET CLR
          > 1.0.3705)"
          >
          > Could it be that the request is too long for webalizer to process?
          >
          > cheers, Iain.
          >
          >
        • Bradford L. Barrett
          From the source code (webalizer.h): #define MAXREF 1024 /* Max referrer field size */ -- ... -- Bradford L. Barrett
          Message 4 of 6 , Jul 10, 2003
          • 0 Attachment
            From the source code (webalizer.h):

            #define MAXREF 1024 /* Max referrer field size */

            --
            On Thu, 10 Jul 2003, Iain Pople wrote:

            > Well, I received no replies to my original question so i'll try and
            > simplify it. What is the maximum size of a referrer log entry that
            > webalizer can handle?
            >
            > cheers, Iain.
            >
            > Iain Pople wrote:
            >
            > > Hi,
            > >
            > > I have been using webalizer to generate statistics on search terms used
            > > on our local search engine. We are using ultraseek and the search cgi
            > > variable is qt. The search engine runs on the host
            > > websearch.its.unimelb.edu.au so I have the following in my webalizer config:
            > >
            > > SearchEngine websearch.its.unimelb.edu.au qt=
            > >
            > > However I am finding that webalizer is missing a lot of entries. E.g.
            > > when I do a manual count on a particular term using grep I get a much
            > > higher value (approx 10x ) than in the webalizer report. Here is a
            > > typical entry from our web logs:
            > >
            > > 128.250.139.35 - - [08/Jun/2003:00:01:07 +1000] "GET
            > > /ExtRels/Media/UN/archive/1999/414/newsmakers.html HTTP/1.1" 200 14076
            > > "http://websearch.its.unimelb.edu.au/query.html?qp2=&col2=unimelb&ht2=0&qs2=&qc2=
            > > &pw2=100%25&ws2=0&qm2=0&st2=1&nh2=25&lk2=1&rf2=0&rq2=0&si2=1&qt=Dr+Franz+Grieser&submit2.x=10&submit2.y=13"
            > >
            > > "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; .NET CLR
            > > 1.0.3705)"
            > >
            > > Could it be that the request is too long for webalizer to process?
            > >
            > > cheers, Iain.
            > >
            > >
            >
            >
            >
            > To unsubscribe from this group, send an email to:
            > webalizer-unsubscribe@egroups.com
            > Webalizer homepage: http://www.webalizer.org
            > Webalizer for NT: http://www.medasys-lille.com/webalizer/
            >
            >
            >
            >
            > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
            >
            >
            --
            Bradford L. Barrett brad@...
            A free electron in a sea of neutrons DoD#1750 KD4NAW

            The only thing Micro$oft has done for society, is make people
            believe that computers are inherently unreliable.
          • Iain Pople
            Hi, thanks for the reply (and for writing great software!). The referrer field generated by ultraseek are much less than this. Can you think of any other
            Message 5 of 6 , Jul 10, 2003
            • 0 Attachment
              Hi,

              thanks for the reply (and for writing great software!). The referrer
              field generated by ultraseek are much less than this. Can you think of
              any other reason why webalizer is only picking up a fraction of the
              search terms?

              cheers, Iain.

              Bradford L. Barrett wrote:

              >>From the source code (webalizer.h):
              >
              > #define MAXREF 1024 /* Max referrer field size */
              >
              > --
              > On Thu, 10 Jul 2003, Iain Pople wrote:
              >
              >
              >>Well, I received no replies to my original question so i'll try and
              >>simplify it. What is the maximum size of a referrer log entry that
              >>webalizer can handle?
              >>
              >>cheers, Iain.
              >>
              >>Iain Pople wrote:
              >>
            • Bradford L. Barrett
              The referrer portion is limited to 1024 bytes in total.. after parsing out the hostname/url portion, it is restricted to 128 bytes (which does not include the
              Message 6 of 6 , Jul 11, 2003
              • 0 Attachment
                The referrer portion is limited to 1024 bytes in total.. after parsing out
                the hostname/url portion, it is restricted to 128 bytes (which does not
                include the cgi portion). The cgi portion itself (everything after the
                first '?' character is limited to the value defined as MAXSRCH in the
                header file:

                #define MAXSRCH 256 /* Max size of search string buffer */

                and the actual search string, after being parsed out of the above is:

                #define MAXSRCHH 64 /* Max size of search str in htab */

                .. all defined in webalizer.h

                --
                On Fri, 11 Jul 2003, Iain Pople wrote:

                > Hi,
                >
                > thanks for the reply (and for writing great software!). The referrer
                > field generated by ultraseek are much less than this. Can you think of
                > any other reason why webalizer is only picking up a fraction of the
                > search terms?
                >
                > cheers, Iain.
                >
                > Bradford L. Barrett wrote:
                >
                > >>From the source code (webalizer.h):
                > >
                > > #define MAXREF 1024 /* Max referrer field size */
                > >
                > > --
                > > On Thu, 10 Jul 2003, Iain Pople wrote:
                > >
                > >
                > >>Well, I received no replies to my original question so i'll try and
                > >>simplify it. What is the maximum size of a referrer log entry that
                > >>webalizer can handle?
                > >>
                > >>cheers, Iain.
                > >>
                > >>Iain Pople wrote:
                > >>
                >
                >
                >
                > To unsubscribe from this group, send an email to:
                > webalizer-unsubscribe@egroups.com
                > Webalizer homepage: http://www.webalizer.org
                > Webalizer for NT: http://www.medasys-lille.com/webalizer/
                >
                >
                >
                >
                > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                >
                >
                --
                Bradford L. Barrett brad@...
                A free electron in a sea of neutrons DoD#1750 KD4NAW

                The only thing Micro$oft has done for society, is make people
                believe that computers are inherently unreliable.
              Your message has been successfully submitted and would be delivered to recipients shortly.