Re: [webanalytics] Historical Data
- Braden and Matt,
Excellent points, all. The fundamental problem with placing too much
emphasis on historical data is that you often lose sight of the forest
for the trees. In my experience, companies that have a "must have"
policy for historical data and claim that they "manage by trends"
(meaning long-term trends) don't appreciate nearly the full value of
even the most rudimentary analytics applications.
Case in point. In a previous job I worked with a company that would
have a hissy fit (yes, a hissy fit) whenever slight improvements were
made to the tracking system. Even though the system was becoming more
efficient and ultimately accurate, certain senior stakeholders would
blow gaskets (yes, gaskets) whenever this would happen, essentially
saying, "We'd rather have less accurate information and preserve our
precious trends ..."
Because this is common, I lean towards encouraging companies to track
year-over-year data where highly appropriate - examining gross
seasonal sales trends, guesstimating growth in key metrics (page
views, visits), etc. - but letting go of the strict requirement to
track long term trends for Web analytic data. Obviously, Web analytic
data is only as useful as you allow it to be.
My $0.02 ...
----- Original Message -----
From: Braden Hoeppner <braden.hoeppner@...>
Date: Wed, 14 Jul 2004 08:33:24 -0700
Subject: RE: [webanalytics] Historical Data
Matt & all,
Let me make sure that I"m clear. I totally agree with you that big
wins with web analytics will never come from historical info. No
question, that real-time tools, campaigns, A/b testing etc, add more
value than historical information.
Your point on the data warehouse - again, agreed. Most information
needed for forecasting can be available in this structure and often
the level of detail that an analytics tool gives you is not needed.
However, let me pose this scenario. Lets say you were an
information/news site & knew historical data (1 to 2 years old) for a
section of your site that was removed 1 year ago. Now, the question is
being raised about if it would be beneficial to bring that section
back. Having all the data from an analytics tool will help with this
decision. Not to mention, as companies grow in their interpretation of
analytics, they may be able to mine nuggets of knowledge that they
didn't realize at the time. Maybe now, using the historical info, you
could point to trends, affinities etc that may support your decision
to bring the section back or not.
I realize that I'm being simplistic in my example. I'm just trying to
say that there are business decisions that can be aided by the use of
historical data, albeit they may only provide small gains. I would not
suggest to companies to invest a lot of time looking at older data,
but rather focus on the present - and when needed, understand what has
happened in the past when appropriate for business decisions. what's
the quote?: "those who forget the past are likely to repeat it?"
From: Matt Belkin [mailto:mbelkin@...]
Sent: Wednesday, July 14, 2004 8:10 AM
Subject: RE: [webanalytics] Historical Data
Braden, thanks for perspectives. I think many of your points are
valid. But in my experience, the big wins we've had with Web
Analytics have never come from analyzing reams of historical data.
That doesn't mean historical data isn't important. Most major
companies retain multiple years of financial data, broken down by
channel (every multi-channel company I've worked at keeps this in a
data warehouse). While the warehouse doesn't support precision
measurement like campaign optimization/AB testing/visitor conversion
analysis, etc, it more than supports forecasting and competitive
analysis with multi-dimensional analysis. It doesn't make sense to
use a real-time web analytics tool for this – it's like using a
microscope to look at the stars.
From: Braden Hoeppner [mailto:braden.hoeppner@...]
Sent: Tuesday, July 13, 2004 4:51 PM
Subject: [webanalytics] Historical Data
Eric et al,
Sorry for the delay in response to this thread, but a few other things
On the topic of historical data... I agree that using recent data and
focusing on A/B testing tools, adopting a continuous improvement
process etc. is most important when assessing and improving a site's
performance. However, I have to stand up and defend historical data!
Let me explain. :-)
I think there are a few reasons why historical data is significant for
1. Companies that are not primarily on the web. These organizations
have oft treated their web site as the 'little sister' of the
retail/marketing machine. Working on the 'web team' in an organization
like this requires that a case be made to show the increasing
performance of the web. Often enough,
that performance is compared YoY. Here is where it gets interesting.
Let's say a company sells products/services in retail stores and
online. The e-commerce piece of the business is only 5 years old. Most
businesses will want to assess the uptake on the web, and in some
areas want to push sales onto
the web (being a lower cost channel). Being able to track YoY results,
and analyze these in light of web user trends can provide useful
information. For example, if my site has grown in terms of
users/conversions over the past 3 years at a rate of 2%/year, that's
fine. But, if average online
sales/usage has grown by 5%/year, in some way I'm missing out on 3% of
the growth in the market. I'm only going to know this if I have some
reliable historical data.
2. Forecasting. When it comes to business planning time, most
companies forecast usage statistics, sales, revs etc to make up their
outlooks & budgets. My experience has been that forecasting is usually
yearly, so to plan ahead, it is helpful to use YoY stats to forecast
the coming year.
Additionally, looking at these stats each year can help you understand
your business's high/low seasons.
3. Developing a Business Cases. By looking over the past couple of
years of statistics, it may be possible to identify some impact on
your site as a result of external factors (ie. A new competitor's site
slowly eating away your market share) These stats may not be clearly
apparent if you were only
reporting on a week over week basis, but investigating a 12 month or
24 month average may show some downward trending that needs to be
explained. If you are able to identify some external factors that have
affected traffic, you can then build some cases and an argument to
invest in avoiding these
I'm curious on other people's take on this & if they use historical
data for anything; particularity if you work in a multi-channel
organization where the web is only a part of what the company does.
From: Eric Peterson [mailto:eric@...]
Sent: Friday, July 02, 2004 3:04 PM
Subject: RE: [webanalytics] What applications are other people using?
Excellent comments! While I cannot share specific data regarding the
cost of switching, these costs generally emerge from three specific
areas: IT resources, training and relationship development.
IT resources is pretty obvious - if you're moving from a log-based
solution, ala WebTrends, to a tag-based method there is the cost of
tagging pages. I mention this since this is a pretty common direction
for companies to move, away from log files. Some of the hidden costs
here include the time it
takes to devise a hierarchy and implementation plan for data
collection, establishment of custom variables, segmentation strategy
(if the application does not allow ad hoc, historical segmentation).
Training is also fairly obvious - your company is likely pretty used
to the kinds of reports they've been getting and now you need to
retrain them to use different reports/language/concepts/etc.
Providing your new provider has an experienced team to provide
training support this is less of an issue
but I caution my clients to be careful when examining training
options. Analytics training is not a "one size fits all" endeavour -
different levels of training should be offered, from "basic training"
on the interface all the way up to "expert user" and "business
objective" focused training
programs (typically delivered by analysts, less often by CSR or training teams).
Relationship development is the cost that is usually overlooked - the
time it will take you to establish a strong relationship with your new
vendor of choice. The top-tier players in analytics all have a
slightly different approach to customer support and some of these
approaches work better than
others. While I obviously cannot advise anyone about which vendor has
a strategy that would work for them, I can say this. It's not a bad
idea to ask to talk to/meet the person or team that will be
responsible for your relationship after the ink dries DURING the
pre-sale process. Time and time
again we see that companies that have a strong relationship with their
vendor - software or services, does not matter - are more likely to
make good use of the application.
Regarding your comment about portability of data, this is tricky. If
you're tied to historicals then I recommend you examine "why" this is
the case. Companies that place too heavy a reliance on historical
data are often not taking advantage of the tactical value of said
data collection devices - even from tags to tags or logs to logs -
almost always concern about comparison to historical data. My advice
is to focus more on recently collected data, take advantage of
emerging tools for A/B testing and site/marketing optimization, and
select the best tool for the job
NOW, not year over year.
P.S. I do hear rumors that one of the vendors has devised a way to
either import data from other applications/data formats and/or co-opt
competitors tags and collect partial data without massive re-tagging.
If anyone has more details about this rumor I'd love to know!
Web Metrics Discussion Group
Moderated by Eric T. Peterson
Author, Web Analytics Demystified
Yahoo! Groups Sponsor
Yahoo! Groups Links
To visit your group on the web, go to:
To unsubscribe from this group, send an email to:
Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
Eric T. Peterson
Author, Web Analytics Demystified
Have you joined the Metrics Discussion Group? Email
email@example.com to join today!