Loading ...
Sorry, an error occurred while loading the content.
 

Re: [webanalytics] Historical Data

Expand Messages
  • Eric Peterson
    Braden and Matt, Excellent points, all. The fundamental problem with placing too much emphasis on historical data is that you often lose sight of the forest
    Message 1 of 5 , Jul 15, 2004
      Braden and Matt,

      Excellent points, all. The fundamental problem with placing too much
      emphasis on historical data is that you often lose sight of the forest
      for the trees. In my experience, companies that have a "must have"
      policy for historical data and claim that they "manage by trends"
      (meaning long-term trends) don't appreciate nearly the full value of
      even the most rudimentary analytics applications.

      Case in point. In a previous job I worked with a company that would
      have a hissy fit (yes, a hissy fit) whenever slight improvements were
      made to the tracking system. Even though the system was becoming more
      efficient and ultimately accurate, certain senior stakeholders would
      blow gaskets (yes, gaskets) whenever this would happen, essentially
      saying, "We'd rather have less accurate information and preserve our
      precious trends ..."

      Crazy, huh?

      Because this is common, I lean towards encouraging companies to track
      year-over-year data where highly appropriate - examining gross
      seasonal sales trends, guesstimating growth in key metrics (page
      views, visits), etc. - but letting go of the strict requirement to
      track long term trends for Web analytic data. Obviously, Web analytic
      data is only as useful as you allow it to be.

      My $0.02 ...


      ----- Original Message -----
      From: Braden Hoeppner <braden.hoeppner@...>
      Date: Wed, 14 Jul 2004 08:33:24 -0700
      Subject: RE: [webanalytics] Historical Data
      To: webanalytics@yahoogroups.com


      Matt & all,

      Let me make sure that I"m clear. I totally agree with you that big
      wins with web analytics will never come from historical info. No
      question, that real-time tools, campaigns, A/b testing etc, add more
      value than historical information.

      Your point on the data warehouse - again, agreed. Most information
      needed for forecasting can be available in this structure and often
      the level of detail that an analytics tool gives you is not needed.
      However, let me pose this scenario. Lets say you were an
      information/news site & knew historical data (1 to 2 years old) for a
      section of your site that was removed 1 year ago. Now, the question is
      being raised about if it would be beneficial to bring that section
      back. Having all the data from an analytics tool will help with this
      decision. Not to mention, as companies grow in their interpretation of
      analytics, they may be able to mine nuggets of knowledge that they
      didn't realize at the time. Maybe now, using the historical info, you
      could point to trends, affinities etc that may support your decision
      to bring the section back or not.

      I realize that I'm being simplistic in my example. I'm just trying to
      say that there are business decisions that can be aided by the use of
      historical data, albeit they may only provide small gains. I would not
      suggest to companies to invest a lot of time looking at older data,
      but rather focus on the present - and when needed, understand what has
      happened in the past when appropriate for business decisions. what's
      the quote?: "those who forget the past are likely to repeat it?"


      ________________________________
      From: Matt Belkin [mailto:mbelkin@...]
      Sent: Wednesday, July 14, 2004 8:10 AM
      To: webanalytics@yahoogroups.com
      Subject: RE: [webanalytics] Historical Data







      Braden, thanks for perspectives. I think many of your points are
      valid. But in my experience, the big wins we've had with Web
      Analytics have never come from analyzing reams of historical data.
      That doesn't mean historical data isn't important. Most major
      companies retain multiple years of financial data, broken down by
      channel (every multi-channel company I've worked at keeps this in a
      data warehouse). While the warehouse doesn't support precision
      measurement like campaign optimization/AB testing/visitor conversion
      analysis, etc, it more than supports forecasting and competitive
      analysis with multi-dimensional analysis. It doesn't make sense to
      use a real-time web analytics tool for this – it's like using a
      microscope to look at the stars.



      ________________________________


      From: Braden Hoeppner [mailto:braden.hoeppner@...]
      Sent: Tuesday, July 13, 2004 4:51 PM
      To: webanalytics@yahoogroups.com
      Subject: [webanalytics] Historical Data



      Eric et al,

      Sorry for the delay in response to this thread, but a few other things
      took priority.

      On the topic of historical data... I agree that using recent data and
      focusing on A/B testing tools, adopting a continuous improvement
      process etc. is most important when assessing and improving a site's
      performance. However, I have to stand up and defend historical data!
      Let me explain. :-)

      I think there are a few reasons why historical data is significant for
      companies.

      1. Companies that are not primarily on the web. These organizations
      have oft treated their web site as the 'little sister' of the
      retail/marketing machine. Working on the 'web team' in an organization
      like this requires that a case be made to show the increasing
      performance of the web. Often enough,
      that performance is compared YoY. Here is where it gets interesting.
      Let's say a company sells products/services in retail stores and
      online. The e-commerce piece of the business is only 5 years old. Most
      businesses will want to assess the uptake on the web, and in some
      areas want to push sales onto
      the web (being a lower cost channel). Being able to track YoY results,
      and analyze these in light of web user trends can provide useful
      information. For example, if my site has grown in terms of
      users/conversions over the past 3 years at a rate of 2%/year, that's
      fine. But, if average online
      sales/usage has grown by 5%/year, in some way I'm missing out on 3% of
      the growth in the market. I'm only going to know this if I have some
      reliable historical data.

      2. Forecasting. When it comes to business planning time, most
      companies forecast usage statistics, sales, revs etc to make up their
      outlooks & budgets. My experience has been that forecasting is usually
      yearly, so to plan ahead, it is helpful to use YoY stats to forecast
      the coming year.
      Additionally, looking at these stats each year can help you understand
      your business's high/low seasons.

      3. Developing a Business Cases. By looking over the past couple of
      years of statistics, it may be possible to identify some impact on
      your site as a result of external factors (ie. A new competitor's site
      slowly eating away your market share) These stats may not be clearly
      apparent if you were only
      reporting on a week over week basis, but investigating a 12 month or
      24 month average may show some downward trending that needs to be
      explained. If you are able to identify some external factors that have
      affected traffic, you can then build some cases and an argument to
      invest in avoiding these
      external influences.

      I'm curious on other people's take on this & if they use historical
      data for anything; particularity if you work in a multi-channel
      organization where the web is only a part of what the company does.

      Cheers,
      Braden

      -----Original Message-----
      From: Eric Peterson [mailto:eric@...]
      Sent: Friday, July 02, 2004 3:04 PM
      To: webanalytics@yahoogroups.com
      Subject: RE: [webanalytics] What applications are other people using?

      Braden,

      Excellent comments! While I cannot share specific data regarding the
      cost of switching, these costs generally emerge from three specific
      areas: IT resources, training and relationship development.

      IT resources is pretty obvious - if you're moving from a log-based
      solution, ala WebTrends, to a tag-based method there is the cost of
      tagging pages. I mention this since this is a pretty common direction
      for companies to move, away from log files. Some of the hidden costs
      here include the time it
      takes to devise a hierarchy and implementation plan for data
      collection, establishment of custom variables, segmentation strategy
      (if the application does not allow ad hoc, historical segmentation).

      Training is also fairly obvious - your company is likely pretty used
      to the kinds of reports they've been getting and now you need to
      retrain them to use different reports/language/concepts/etc.
      Providing your new provider has an experienced team to provide
      training support this is less of an issue
      but I caution my clients to be careful when examining training
      options. Analytics training is not a "one size fits all" endeavour -
      different levels of training should be offered, from "basic training"
      on the interface all the way up to "expert user" and "business
      objective" focused training
      programs (typically delivered by analysts, less often by CSR or training teams).

      Relationship development is the cost that is usually overlooked - the
      time it will take you to establish a strong relationship with your new
      vendor of choice. The top-tier players in analytics all have a
      slightly different approach to customer support and some of these
      approaches work better than
      others. While I obviously cannot advise anyone about which vendor has
      a strategy that would work for them, I can say this. It's not a bad
      idea to ask to talk to/meet the person or team that will be
      responsible for your relationship after the ink dries DURING the
      pre-sale process. Time and time
      again we see that companies that have a strong relationship with their
      vendor - software or services, does not matter - are more likely to
      make good use of the application.

      Regarding your comment about portability of data, this is tricky. If
      you're tied to historicals then I recommend you examine "why" this is
      the case. Companies that place too heavy a reliance on historical
      data are often not taking advantage of the tactical value of said
      applications. Switching
      data collection devices - even from tags to tags or logs to logs -
      almost always concern about comparison to historical data. My advice
      is to focus more on recently collected data, take advantage of
      emerging tools for A/B testing and site/marketing optimization, and
      select the best tool for the job
      NOW, not year over year.

      Eric

      P.S. I do hear rumors that one of the vendors has devised a way to
      either import data from other applications/data formats and/or co-opt
      competitors tags and collect partial data without massive re-tagging.
      If anyone has more details about this rumor I'd love to know!



      ---------------------------------------
      Web Metrics Discussion Group
      Moderated by Eric T. Peterson
      Author, Web Analytics Demystified
      http://www.webanalyticsdemystified.com





      Yahoo! Groups Sponsor



      ADVERTISEMENT


      ________________________________
      Yahoo! Groups Links

      To visit your group on the web, go to:
      http://groups.yahoo.com/group/webanalytics/

      To unsubscribe from this group, send an email to:
      webanalytics-unsubscribe@yahoogroups.com

      Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.



      --
      Eric T. Peterson
      Author, Web Analytics Demystified
      www.webanalyticsdemystified.com

      Have you joined the Metrics Discussion Group? Email
      webanalytics-subscribe@yahoogroups.com to join today!
    Your message has been successfully submitted and would be delivered to recipients shortly.