Re: Splunk and Webtrends 9.2 On Premise / Web Analytics Tools
- Hi Paul,
Thanks for the reply.
I would be interested in knowing more detail around your comparison....
"We did a brief comparison, and found that it's not nearly as
comprehensive or extensible as Webtrends... but it had a few distinct
in some areas as well, such as the ad hoc re-"analysis" of data using
I guess I am in a unique position managing analytics for an application
that requires authentication. I have been able to replicate every report
produced in Webtrends that is used by the business and in nearly all
cases enhance them.
I can understand if you use web logs you will collect a large volume of
data. In our On Premise setup we do use web logs for a few sites, these
go through the pre-processing and post-processing stages and the volumes
are large. I am not responsible for these sites and will not be looking
at using Splunk with them.
With the SDC logs you are only capturing the tags fired from the site so
the data volume is smaller. I have predicted I could manage on the
minimum 500meg a day licence from Splunk. The application I manage is
running at around 15,000 users but this will rise over time to around
The SDC logs are in the iis format, after spending a little time
learning the Splunk search language and getting a bit of help I had no
issues with indexing or searching in Splunk. I was able to produce the
same data as in Webtrends.
I think it is important to note that I extract all the Webtrends report
data and add it into a CUBE like tool, it is then enriched, other data
added and Qlikview sits on top of the data. This is more for reporting
but is also used for analysis. Therefore Webtrends or Splunk will never
be my final solution but a good addition to a set of tools used for
My next challenges in my Proof of Concept are first looking a sessions
and secondly extracting data from Splunk.
The idea of a session has its own complexities as different business
users are expected to behave completely differently and thus require
different session lengths to be considered a visit. Some users will be
the standard 30 min no activity while other maybe 3 hours etc.
I'll give an update as to how things move along.
Feel free to contact me in the future with any questions you may have.
--- In firstname.lastname@example.org, resonancevibe-analytics@... wrote:
> Hi, Dan. We use Webtrends On Premise as well, and I share your
> Visitor Data Mart product does a good job of turning the SDC data into
> relational DB and orienting toward the visitor rather than the
> visits. But it's a big investment that I don't think I could recommend
> the stagnation of the On-Prem offering.
> We also use SPLUNK, but not so much for web analytics but other
> purposes. We did a brief comparison, and found that it's not nearly as
> comprehensive or extensible as Webtrends... but it had a few distinct
> in some areas as well, such as the ad hoc re-"analysis" of data using
> criteria. But it does results in a ton of data, which makes report
> bit of a problem. Our typical SPLUNK index only has data for the last
> in most cases due to the storage required.
> I wasn't deeply involved in the research, so can't give too much
> one of the things that proved tricky was configuring SPLUNK to mirror
> for a given set of logs. (We were using web server logs in this case,
> It took some configuration of the SPLUNK indexing to get it close, and
> was apparently a problem getting SPLUNK to do session tracking on
> than the IP address field. I am willing to consider the chance that
> very well do that, and maybe our internal SPLUNK guys just didn't know
> We are considering the possibility, if we need to migrate to Webtrends
> that we might migrate only certain sites/apps to OnDemand and rely on
> the internal and/or non-strategic or uncomplicated analysis.
> I'd like to possibly reach out to you in the future as we spelunk in
> ~ Paul
> ----- Original Message -----
> From: Daniel Ford
> Sent: 05/24/13 12:01 PM
> To: email@example.com
> Subject: [webanalytics] Splunk and Webtrends 9.2 On Premise / Web
> Due to the lack of updates for the Webtrends On Premise product I am
> leveraging up on the data already collected by our Webtrends On
> 9.2 setup.
> I have been running a Proof of Concept over the last week or so and I
> been pleasantly surprised by Splunk.
> One of the largest costs associated with running our Webtrends On
> has been the development time in tagging and collecting data. I would
> to start from scratch with a new tool.
> The application I oversee is completely written in Flex and requires
> user to login, thus providing a difficult implementation but once the
> collected it has strong session identification.
> Webtrends On Premise is a fantastic product but I find it has its
> the lack of future updates and how it processes the SDC logs into
> (table limits, drilldown limits, no REST etc). Webtrends will always
be used for
> the API data collection function and for some reporting. Getting down
> level reporting has been achieved but challenging!
> Installing Spunk and pointing it at your SDC logs essentially gives
> extra UI and turns your data collected into a completely relational
> While I continue with my proof of concept I am reaching out to the
> Premise community to see if this has been done before??
> I am looking for potential drawbacks and benefits experience by other
> Any other Web Analytics practitioners that are using Splunk then
please add your
> I would like to stress this is an addition to the Webtrends setup
rather than a
> Look forward to your thoughts,
[Non-text portions of this message have been removed]