Loading ...
Sorry, an error occurred while loading the content.
 

Search Query Log Analysis

Expand Messages
  • jamjam76
    Hi there, Anyone have any tools or best practices that help you analyze search query logs? Thanks! Pete
    Message 1 of 8 , Nov 11, 2008
      Hi there,
      Anyone have any tools or best practices that help you analyze search
      query logs?

      Thanks!
      Pete
    • Lee Romero
      Hey Pete - I think to be specific, it would help to know your search engine. Can you share that? How are the logs maintained? Are they text? Stored in a
      Message 2 of 8 , Nov 11, 2008
        Hey Pete - I think to be specific, it would help to know your search
        engine. Can you share that?

        How are the logs maintained? Are they text? Stored in a data
        repository of some sort that's queryable?

        Some general thoughts:

        * The primary tool for analyzing logs I would recommend is any kind of
        reporting capability provided by the engine (if any). That will
        probably provide things like top searches, total searches, searches by
        index (if your engine supports multiple indexes), perhaps some
        representation of searches per hour/day/whatever to get a sense for
        peak usage, etc.

        * If you have a text file only, the format of that will dictate tools
        - I've grown used to using things like grep, awk, etc. Standard text
        manipulation tools.

        * In terms of best practices - focusing on top searches is useful, but
        if you can get a view of the whole set or possibly break the searches
        down into constituent words, I've found that to be very useful. The
        combinations of words people use will vary widely but if you can see
        the individual words, that, in my experience, helps provide a very
        simple way to categorize searches without too much effort.

        * Also, I have found that looking at behavior over time is more useful
        than individual snapshots (which is what I get from the search engine
        we use). In my case it requires some manual maintenance but it's
        valuable to be able to contrast what's popular today with what was
        popular last month, 6 months ago, etc., to look for trends.

        * You should also check out
        http://www.searchtools.com/guide/index.html - I think that provides a
        good overview of search log analysis.

        Lee

        On Tue, Nov 11, 2008 at 3:51 PM, jamjam76 <jamjam76@...> wrote:
        > Hi there,
        > Anyone have any tools or best practices that help you analyze search
        > query logs?
        >
        > Thanks!
        > Pete
        >
      • Dave Lemen
        Hi Pete, Lee s recommendations are great. I d just add these points: 1. If your logs include the number of hits returned for each query, you should spend some
        Message 3 of 8 , Nov 12, 2008
          Hi Pete,

          Lee's recommendations are great. I'd just add these points:

          1. If your logs include the number of hits returned for each query,
          you should spend some time looking at no-hit queries. See if you can
          figure out what the searcher was looking for, and if such content
          exists in your corpus, why the search engine didn't return relevant
          results.

          2. If your system captures click-throughs for a query, you can do
          similar analysis on queries for which the searcher doesn't click a
          link.

          3. Search is like a conversation between a searcher and the search
          engine, so it's helpful to analyze sessions, rather than individual
          queries. How did the searcher start the session, and how did she
          refine her query over time. If you have a spell checker, for example,
          did misspellings get corrected in a subsequent query? (Misspellings
          may result in no-hit or no-click queries.)

          Dave Lemen

          On Tue, Nov 11, 2008 at 3:51 PM, jamjam76 <jamjam76@...> wrote:
          > Hi there,
          > Anyone have any tools or best practices that help you analyze search
          > query logs?
          >
          > Thanks!
          > Pete
          >
          >
        • Lee Romero
          Great adds, Dave. ... [LR] This one, especially, can be enlightening and high-value analysis (especially if you are supporting e-commerce with your search).
          Message 4 of 8 , Nov 12, 2008
            Great adds, Dave.

            On Wed, Nov 12, 2008 at 10:58 AM, Dave Lemen <dlemen@...> wrote:
            > Hi Pete,
            >
            > Lee's recommendations are great. I'd just add these points:
            >
            > 1. If your logs include the number of hits returned for each query,
            > you should spend some time looking at no-hit queries. See if you can
            > figure out what the searcher was looking for, and if such content
            > exists in your corpus, why the search engine didn't return relevant
            > results.
            >

            [LR] This one, especially, can be enlightening and high-value analysis
            (especially if you are supporting e-commerce with your search).

            One example from our intranet that I can share in this regard that I
            thought was interesting was when I finally was able to really dig into
            the individual words of *all* searches performed in our query logs
            (not just the top searches or looking at whole search terms), I found
            that among the words most commonly included in searches that returned
            no results were a large variety of misspellings of the word
            "bereavement" - trying those myself, about 1/3 of them came up with a
            suggested spelling correction but not all of them. These various
            misspellings probably accounted for about 15% of all "no results
            found" searches.

            So I ended up simply defining a set of synonyms (though potentially an
            abuse of this, it seems like a reasonable solution) for bereavement
            that included the most-commonly used misspellings.

            Had I not been able to see all of the queries (these were each very
            deep into the "long tail" in any given reporting period) or if I had
            not been able to break the search terms into individual words, I doubt
            that I would have been able to see the connection here and so would
            not have been able to "correct" this for searchers.
          • jamjam76
            Hi Dave and Lee, Thank s a bunch for the excellent insights. Are there any analytics products that support this type of analysis? SEO applications perhaps?
            Message 5 of 8 , Nov 12, 2008
              Hi Dave and Lee,
              Thank's a bunch for the excellent insights.

              Are there any analytics products that support this type of analysis?
              SEO applications perhaps?

              Identify top poor performing queries
              - zero results
              - search result pages served that have zero clickthroughs
              - next page

              Ad hoc reporting:
              - Show me how many times the term 'x' appears in user queries for the
              months.
              - Allow me to drill down and see all the queries that contain the term
              'x' for this month.
              - Allow me to drill down and see a session(s) that contain the term
              'x' for this month.

              Thanks!
              Pete
            • Dave Lemen
              Hi Pete, ... Depending on your search product, some of this may be included. We created our own tools for log analysis. Dave
              Message 6 of 8 , Nov 12, 2008
                Hi Pete,

                > Are there any analytics products that support this type of analysis?
                > SEO applications perhaps?
                >

                Depending on your search product, some of this may be included. We
                created our own tools for log analysis.

                Dave
              • Lee Romero
                Hi Pete - Actually, if you re looking for some great ideas on particular questions or classes of questions to try to answer through log analysis, you should
                Message 7 of 8 , Nov 13, 2008
                  Hi Pete - Actually, if you're looking for some great ideas on
                  particular questions or classes of questions to try to answer through
                  log analysis, you should check out Louis Rosenfeld's presentation:

                  http://www.slideshare.net/lrosenfeld/site-search-analytics-workshop-presentation?src=embed

                  and a few additional presentations he has available under his user
                  name on that site:

                  http://www.slideshare.net/lrosenfeld

                  As for products - again, your search engine might provide the details
                  but if web analytics tools can also provide the same types of data,
                  though it might take a bit of configuring.

                  Regards
                  Lee

                  On Wed, Nov 12, 2008 at 1:44 PM, jamjam76 <jamjam76@...> wrote:
                  > Hi Dave and Lee,
                  > Thank's a bunch for the excellent insights.
                  >
                  > Are there any analytics products that support this type of analysis?
                  > SEO applications perhaps?
                  >
                  > Identify top poor performing queries
                  > - zero results
                  > - search result pages served that have zero clickthroughs
                  > - next page
                  >
                  > Ad hoc reporting:
                  > - Show me how many times the term 'x' appears in user queries for the
                  > months.
                  > - Allow me to drill down and see all the queries that contain the term
                  > 'x' for this month.
                  > - Allow me to drill down and see a session(s) that contain the term
                  > 'x' for this month.
                  >
                  > Thanks!
                  > Pete
                  >
                • jamjam76
                  Awesome! Thanks for the pointers Lee. Pete
                  Message 8 of 8 , Nov 13, 2008
                    Awesome! Thanks for the pointers Lee.
                    Pete
                  Your message has been successfully submitted and would be delivered to recipients shortly.