Loading ...
Sorry, an error occurred while loading the content.

Re: [webanalytics] Web Trends not filtering out visits from spiders

Expand Messages
  • Wes Contreras
    ... You can create a Webtrends filter for excluding spider and bot traffic. To make one, you ll want to create a hit filter based on Browser, then select
    Message 1 of 7 , Nov 7, 2011
    • 0 Attachment
      On Fri, Nov 4, 2011 at 3:46 AM, Dan <dan@...> wrote:
      >
      > We have been having an issue with a client who's using WebTrends. It appears
      > that WebTrends has been reporting site traffic including spider visits and inflating
      > the figures.
      >
      > In some cases reports are showing that spiders visits are inflating traffic by over 100%.
      >
      > Is this a known problem with WebTrends and is there anything we can do to address
      > the problem, or do we recommend a switch to a tool that does strip out these visits?

      You can create a Webtrends filter for excluding spider and bot
      traffic. To make one, you'll want to create a "hit filter" based on
      Browser, then select "All Spiders & Robots" from the drop-down list.
      If you're using the On Premise (software) version of the product,
      you'll want to keep your local definition file for spiders and bots
      current for this to work best:
      http://kb.webtrends.com/support/solutiondetail.aspx?Id=501400000009whvAAA

      As noted, most spiders and bots don't execute JavaScript, but a few
      do, so this filter can still be useful. This is doubly true if you're
      analyzing server logfiles.


      --
      Wes Contreras
      Disclaimer: I am a Webtrends employee
    • jing.suk@thomsonreuters.com
      To be fair to WebTrends, I don’t think what your client encountered is a vendor issue. I’m assuming your client has the software version of WebTrends
      Message 2 of 7 , Nov 7, 2011
      • 0 Attachment
        To be fair to WebTrends, I don’t think what your client encountered is a vendor issue. I’m assuming your client has the software version of WebTrends instead of using WebTrends ondemand services. When using the software version of WebTrends, your client would need to manage the bots/spiders filtering rules within the WebTrends admin console. If they’re using the ondemand version, then WebTrends’s technical team would manage this. At my previous position, we used WebTrends ondemand for 5+ years, and we audit our site traffic, WebTrends didn’t report any spider/bots traffic.

        Best,
        Jing



        From: webanalytics@yahoogroups.com [mailto:webanalytics@yahoogroups.com] On Behalf Of Bob Willis
        Sent: Monday, November 07, 2011 9:33 AM
        To: webanalytics@yahoogroups.com
        Subject: Re: [webanalytics] Re: Web Trends not filtering out visits from spiders



        they are probably analyzing web log files and not the WebTrends js created log files.
        I think you can just update the bot/spider definition and filter this out.

        --- On Sun, 11/6/11, Rubix <rubix606@...<mailto:rubix606%40yahoo.com>> wrote:

        From: Rubix <rubix606@...<mailto:rubix606%40yahoo.com>>
        Subject: [webanalytics] Re: Web Trends not filtering out visits from spiders
        To: webanalytics@yahoogroups.com<mailto:webanalytics%40yahoogroups.com>
        Date: Sunday, November 6, 2011, 12:34 PM



        I'm actually really surprised to hear this. I'm not familiar with WebTrends, but Omniture data tends to be much more reliable than my company's log data as a reflection of actual human activity. The reason for this is that spiders/scripters/bots/etc. are most likely not using JavaScript-enabled clients so that activity simply never gets recorded in the first place.

        What is WebTrends' tracking mechanism?

        --- In webanalytics@yahoogroups.com<mailto:webanalytics%40yahoogroups.com>, "Dan" <dan@...> wrote:
        >
        > Hi,
        >
        > We have been having an issue with a client who's using WebTrends. It appears that WebTrends has been reporting site traffic including spider visits and inflating the figures.
        >
        > In some cases reports are showing that spiders visits are inflating traffic by over 100%.
        >
        > Is this a known problem with WebTrends and is there anything we can do to address the problem, or do we recommend a switch to a tool that does strip out these visits?
        >
        > Any help much appreciated
        >
        > Thanks
        >
        > Dan
        >

        [Non-text portions of this message have been removed]


        This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters.

        [Non-text portions of this message have been removed]
      • Matthias Bettag
        Hi, Webtrends can filter spiders, it is an optional setting of the profile configuration. However, I experienced some rare issues where a not-yet blacklisted
        Message 3 of 7 , Nov 7, 2011
        • 0 Attachment
          Hi,

          Webtrends can filter spiders, it is an optional setting of the profile
          configuration.
          However, I experienced some rare issues where a not-yet blacklisted
          spider faked it's user agent and by this passed the filter.

          best,
          Matthias
        Your message has been successfully submitted and would be delivered to recipients shortly.