Loading ...
Sorry, an error occurred while loading the content.

Client and/or Server-Side Data / Outsourced Service and/or Software Product

Expand Messages
  • jimmacintyreiv
    Matt, Happy Halloween! Let me guess, you re dressed as a Page Tag or a Third-Party Cookie :) Yes, most measurement data collected from most all types of
    Message 1 of 45 , Oct 31, 2004
    • 0 Attachment
      Matt,

      Happy Halloween! Let me guess, you're dressed as a Page Tag or a
      Third-Party Cookie :)

      Yes, most measurement data collected from most all types of
      applications, of most all sorts, end up in log files, we agree. The
      page tag generated requests that end up in log files are logged by
      the Web (HTTP) servers that receive the page tag requests.

      "Log Files" have never come to mean "server-side" data collection
      alone and certainly have never come to mean a class of vendors (a
      class that I don't think even exists) that can only process
      measurements generated on the "server-side." The folks that have
      been in the internet industry for more than 10 years know that such
      is just the marketing spin of a generation of ASPs; A group of
      vendors that according to Jupiter currently make up 29% of the web
      analytics market.

      Web measurement data has always been collected from the "client-
      side." In fact most all that web server log files contain is
      information generated from the "client-side;" by a user agent
      (browser or otherwise) which is then logged by a web server, whether
      that web server is owned by an ASP or anyone else for that matter.
      Client-side information gathered through the instrumentation of pages
      for specific measurement purposes is no exception.

      My comments are not playing semantics with you. Rather they focus on
      clarifying the confusing language that has been used, and you are
      using as marketing spin to seemingly confound the less technical in
      some hope of them believing that "client-side" data collection is
      somehow better than "client-side" and "server-side" data collection.
      This is logic I just can't seem to stomach, its kind of like saying
      that I would rather have one dollar instead of two because I can only
      spend one dollar today.

      The fact is, that the market has many vendors who can collect both
      client and server-side data and use the best of both to provide value
      to their customers. Some of those companies offer their services on
      an outsourced basis (as an "ASP" or a "Managed Service") and some
      offer software products that their clients can run themselves to gain
      this value, while still others offer both of these delivery options.

      Pretending that there is some special linkage between "client-side"
      data collection and ASPs is just marketing spin. Rather, ASPs only
      collect client-side information because that is what they are
      structured to do. So sure, it is clear why you are pushing this
      issue in this forum, if a company wants anything other than Page Tag
      generated "client-side" data then an ASP-only vendor doesn't provide
      it and that ASP-only vendor can not effectively compete for the that
      company's business by other than convincing that company that
      anything else is not important or won't produce a ROI.

      To pretend to the good folks here that companies have not found value
      and ROI in server-side data in addition to client-side data, and that
      companies have not found value in software in addition to ASP
      services is just marketing spin.

      Vendors that offer software products and outsourced services can
      collect both "client-side" and "server-side" data. They can do this
      as either an "outsourced/managed service or ASP" or provide software
      products so that a client can process their data themselves and keep
      their customer's information privately behind their own firewalls.

      So Matt, I think that the strategic question for many companies
      is, "if they can have the best of client-side measurement, in
      addition to the best of server-side measurement, and they can have it
      as an ASP or they can have it as a software product they can run,
      then why would they purchase a service that can only collect client-
      side information and is only available on an outsourced/ASP basis?"

      Given that Visual Sciences and many other vendors can provide both
      software and outsourced service and collect both client and server-
      side data, I would have to agree with you the real issue that should
      be focused on is how much Business Value can be created from client
      and server-side data through the analysis of it.

      Visual Sciences was proud to have Jupiter rate Visual Sciences as the
      vendor whose offerings provide the most "Overall Business Value" of
      the top web analytics vendors in their most recent report. So rather
      than provide the so many sometimes competitors that are in attendance
      here Visual Sciences' client ROI cases (as such are reserved for the
      clients whose business we will compete with you for), I would refer
      you and others to the due diligence of JupiterResearch in regard to
      value that Visual Sciences generates for its clients.

      Now what I won't claim to you tonight is that Visual Sciences or any
      data collection and analysis solution produces "exponential ROI,"
      because I and most of the people here know what "exponential"
      means.. But hey, if using page tag generated client-side data
      alone, and dropping the server-side data collection method, for some
      cosmic reason, produces "exponential ROI," then I and every other
      sane person here and in the investing world would be putting every
      available dollar into Page Tag-only/ASP-only data collection and
      reporting solutions rather buying stocks, bonds, commodities and
      derivatives.

      In any case, Happy Halloween and I hope a third-party doesn't eat
      your cookie ;)

      Jim MacIntyre
      Visual Sciences


      --- In webanalytics@yahoogroups.com, Matt Belkin <mbelkin@y...> wrote:
      > Jim, interesting comments as always...and yes, I actually read the
      whole thing :)
      >
      > First, let me say I'm certainly aware that page tags generate their
      own 'log file'; akin to server log files. However, since this
      industry began circa in the mid 1990's, the term 'log file' has been
      commonly accepted to mean a server log file. So sure, I agree with
      your observation that both data collection approaches generate logs,
      but that is simply a semantic discussion (or a silly one) - not a
      strategic one that would allow our mutual customers to be more
      successful. Perhaps the Web Analytics Association can help out in
      that area, but for time being, I'd rather focus on helping customers
      maximize their analytics ROI. And in my experience, server log files
      have never produced the exponential levels of ROI that I have seen
      from page tag collection methods. If you have seen otherwise, I'd
      love to hear about some case studies of such success amongst your
      customers.
      >
      > Finally, I agree that server logs are useful for tracking things
      like 'hits' on movies and mp3s - but again, where's the ROI? That's
      tactical stuff, good for blocking and tackling - but not the type of
      measurement that empowers organizations to make strategic decisions
      about their customers, marketing expenditures, and online strategy.
      >
      > Cheers, and thanks as always for the stimulating discussion - Matt.
      >
      > Matt Belkin
      > Vice President, Best Practices Group
      > Omniture
      >
      > jimmacintyreiv <jim@v...> wrote:
      >
      >
      > This thread/conversation about Page Tags and Log Files in general
      > and statements like "Page Tagging is more accurate, scalable,
      > reliable and actionable" remind me of the famous quote "Complex
      > problems have simple, easy-to-understand wrong answers."
      >
      > Like with most overreaching statements and promises it pays
      to "read
      > the fine print." And the fine print most often says "buyer
      > beware."
      >
      > The fine print in the case of web analytics is the technical detail
      > around measurement and data collection. It may be nice to avoid
      > this topic, but if you do you will someday pay the real price that
      > the sales person didn't want to tell you about. They might tell
      you
      > that their way is "best practice" but if that is all they can do
      > then you might take that as a clue to review the fine print right
      > away and at the very least consider it a "best practice" for them
      > rather than necesarily a "best practice" for you.
      >
      > I have no particular interest in one method of data collection or
      > the other (except that my clients choose the one that is right for
      > their situation) as Visual Sciences supports "Page Tagging" (Client-
      > Side Only), "Web and Application Server API" (Server-Side Only),
      > Traditional Web Server "Log File Processing", "Real-Time Client and
      > Server Side Combined" Data Collection and even some lesser used and
      > known methods. And because we do so I can confidently testify to
      > the group that every site has different issues and in many cases
      the
      > method that is right for one is not right for the other, as Stephen
      > suggests. The idea that Server-Side data collection is equivalent
      > to Log File processing is a long dead topic. It is not, and there
      > is absolutely no reason that a site must choose between having
      > server side and client side information, they don't have to, they
      > can have the best of both and eliminate the worst of both by doing
      > some thoughtful planning and choosing the right tools and services.
      >
      > So some background on the fine print . . . and I apologize in
      > advance to the good people that I am boring with long known
      > information.
      >
      > A "Page Tag" is the javascript or other client executable script
      > that is added to a web page to cause an embedded object request to
      > be made to web (http) servers. The embedded object request is
      kicked
      > off when the page is loaded and that script runs in the browser's
      > (client's) javascript interpreter and then causes the embedded
      > object request to be made. That Page Tag driven embedded object
      > request (no different than any other http request for a javascript
      > file, image, etc.) might be made to web servers owned by a web
      > analytics ASP (third-party request) or to web servers owned by the
      > site owner (first-party request).
      >
      > Those web servers that receive those Page Tag instigated embedded
      > requests generate log entries that are written to Log Files. Those
      > Log Files are processed by a web analytics ASP's or the site
      owner's
      > data extraction, transformation, loading (ETL) and management
      (DBMS)
      > systems for reporting and maybe analysis.
      >
      > It is silly to talk about Page Tags verses Log Files as Page Tags
      > produce Log Files when they cause HTTP requests and web servers at
      > an ASP log those HTTP requests into Log Files.
      >
      > The question is what is in the Log Files. For an ASP the Log Files
      > contain the information sent to them with the embedded object
      > request, typically one request per Page View, but sometimes more
      > than one.
      >
      > For a Log File created by a typical web server, all of the HTTP
      > requests made to the web server are placed in the Log File if the
      > server is configured to log them.
      >
      > So the ASP services' web servers log Page Views (through the proxy
      > of an embedded object request) and regular web servers log Page
      > Views (except those served from the browser's cache) as well as all
      > other types of requests (e.g. PDFs, MP3s, Images, Style Sheets,
      etc.
      > etc.) and they also track server errors, etc., where Page Tagging
      > services do not.
      >
      > Most ASP-only services use Page Tags because that's what they can
      > do. They use Page Tags to track Page Views on a third-party
      basis.
      > They set unique identifiers in cookies when the Page Tag's embedded
      > object request is made so that they can try to identify returning
      > visitors (browsers) and use the unique id to knit together sessions
      > when they process the Page Tag Log Files. They can do that in
      their
      > fullest form when the Page Tag code is actually run by the
      browser.
      > When javascript is turned off in the browser they run a "no script"
      > function to kick off an embedded object request, dropping and
      losing
      > the data that would have been collected from the HTML DOM by the
      > javascript and appended to that embedded object request.
      >
      > It is nonsense to talk about Page Tags as being "more accurate"
      > than "Log Files" or server-side data in general. Using data that
      can
      > collected with Page Tags alone is only more "accurate" than W3C log
      > files produced by web servers, in regard to tracking Page Views,
      > when the Page View is served from the browser's cache, and it is
      > much less accurate at tracking anything other than Page Views
      > because it is hard or impossible to Tag many of the other types of
      > objects served from sites on the Internet. Any site operator can
      > add a one-line "Page Tag" to their web pages to make the same Page
      > View that is served from a browser cache be tracked and logged in
      > their standard web server Log Files. There is no vendor at all
      > required to do this. The same is true for any thing that a Page
      Tag
      > ASP can do. Anyone can add a page tag to extract information from
      a
      > dynamic page for instance and cause it to be sent back to their
      > servers for logging and use.
      >
      > In fact using Page Tags alone is less accurate in many ways, and
      > here are just a short list of them: 1) Page Tags drop information
      > when javascript is turned off for the browser; 2) Page Tags do not
      > log Page Views when someone forgets to put the Page Tags in the
      Page
      > or puts them in wrong; 3) Page Tag requests served/collected by
      > third-parties (ASPs) cannot be identified as being from a
      particular
      > browser (visitor) when third-party cookies are not allowed by the
      > browser; and 4) Page Tag requests sometimes never make it to the
      > logs of the third-party ASP because of network congestion, server
      > saturation and other issues (have you ever seen an error in loading
      > a web page? Well then have you ever seen an error from a Page Tag?
      > If not, can you realistically believe that is because there aren't
      > any?
      >
      > In fact a growing number of browser settings and a Internet
      > population that is getting ever smarter about how to use them and
      > other privacy tools can make Page Tag only solutions increasingly
      > inaccurate as time goes by. The still worst thing about Page Tag
      > only solutions is that you will never even know when they are
      > inaccurate so you can fix them. This is very convenient for Page
      > Tag only ASP vendors. In many cases they seem to want you to turn
      > off your other server logging . . . if you do you will never know
      > how inaccurate they really are, especially if you get convinced, as
      > I saw someone trying to do the convincing in this thread, to never
      > even check.
      >
      > It is also silly to talk about Page Tag only solutions being "more
      > scalable." In fact very few if any sites can stop logging on their
      > web servers, for many good reasons, so rather than dealing with
      just
      > their log files and maybe a very small page tag added to the pages
      > to track browser cached requests, site operators not only have to
      > pay to collect and store their web server's actual log files, they
      > pay for an ASP to create still more log files from additional page
      > tags.
      >
      > It is a fantasy to believe that the web server log files can be
      > turned off because the log files contain all sorts of information
      > necessary for the technical operations of the site like what errors
      > are occurring on the site. Sure a Page Tag only solution might be
      > able to Page Tag "error pages" on a site, but it cannot track
      errors
      > on anything other than Page Views . . . and the web is a multi-
      > media environment where images, movies, songs, documents any number
      > of other types of content objects exist and need to be monitored by
      > someone unless you run a very simple site.
      >
      > If an ASP only solution could convince your technical team to turn
      > off your web server logging then the part of purchasing an ASP
      > service that is "more scalable" can all be summed up in the
      > statement that the ASP service is employing a data reduction
      > strategy and only collecting a small portion of the data that a
      site
      > operator might actually find useful if they had the tools to do so.
      >
      > The data reduction strategy is eliminating most of the data that
      has
      > gone into traditional web server log files by just logging Page
      > Views. So sure, because Page Views are typically 1/10th to 1/30th
      > of the total number of HTTP requests to a site then such a data
      > reduction strategy makes a web analytics system seem able to handle
      > more Page Views. This is not "more scalable" this is a it is more
      > scalable in regard to Page Views alone.
      >
      > Page Tagging only may be the right strategy for you site and it may
      > not, and by the way you can do this yourself if you like by just
      > Page Tagging your site and sending the Page Tags back to a
      dedicated
      > web server or two on your site and spare yourself the third-party
      > cookie and P3P issues, the chance that you traffic in your
      > customer's personal information improperly and other negatives.
      >
      > Further, many web analytics software tools can easily filter out
      non
      > Page View HTTP requests for the purposes of reporting analysis
      while
      > still leaving the rest of your log files available for other types
      > of technical analysis that your operations team might need to do.
      >
      > "More Reliable" OK so some software products that have existed in
      > the market over the last 10 years haven't been very reliable. That
      > does not mean that all of them are unreliable. In fact I regularly
      > hear stories about the irregular availability of reports and
      > analysis from ASP only vendors. Worse, any complication in the
      > ASP's data collection infrastructure can make data collection very
      > unreliable. The worse thing again here is you will likely never
      > know unless they tell you. What if the web analytics ASP has taken
      > on some very large clients and they have a busy day and swamp the
      > data collection servers for the ASP? What happens to those Page
      Tag
      > requests from your site. . .they disappear into the ether is what
      > happens, but will you ever know about it? Not likely.
      >
      > And "More Actionable?" I'm even going to address that now, its too
      > late and rather than address it I would refer you to the proprietor
      > of this forum's latest report and specifically the amount of actual
      > business value derived from the solutions reviewed. . . the
      business
      > value delivered by the solution rated highest in Business Value was
      > sometimes derived from Page Tags and sometimes not. . there is not
      > particular correlation between Page Tagging and Actionability, that
      > is just vendor marketing talk and has little to nothing to do with
      > Page Tagging verses other methods of data collection.
      >
      >
      > --- In webanalytics@yahoogroups.com, Matt Belkin <mbelkin@y...>
      > wrote:
      > > Stephen,
      > >
      > > Interesting comments. Omniture can work with log file data, so I
      > just want to be clear that my bias for page tagging isn't a
      function
      > of that :) It's a function of having dealt with logs for years and
      > struggled to justify the investment. I agree that there are
      > considerations that will be unique to each customer, but with so
      > many high- and low-end page tag products on the market, I can't
      > understand why folks would stick with log files.
      > >
      > > Anyway, one of your comments drew my curiosity - you said it is
      > much more useful to look at trends over time, or comparisons
      between
      > different groups of visitors. I completely agree with this
      > statement. But what I'd love more color on is how log files help
      > you achieve this? I've used log file solutions from several
      vendors
      > (who shall remain nameless), and found it perpetually challenging
      to
      > make reliable decisions from the data.
      > >
      > > For example, a customer segment I created would jump by 15% one
      > day, and upon further exploration, we discovered our program failed
      > to properly parse IP addresses. On another occasion, advertising
      > impressions (as measured by pageview counts) were inaccurate
      because
      > of proxy caching issues and some spidering activity. This didn't
      > sit well with our advertising partners. Another issue we ran into
      at
      > Macromedia specifically was Flash tracking - simply couldn't do it
      > with log files. Finally, I found it difficult to tie campaign
      > response and customer value to the individual, both from an
      absolute
      > dollar standpoint and within a dynamic app.
      > >
      > > Anyway, forgive me if your log file product has workarounds for
      > these issues, I've never used ClickTracks. But if it does, I'd
      love
      > to hear more about it and I'm sure the other members of the board
      > who still use log files would benefit from some clarity too :)
      > >
      > > In closing, I thought I'd share some commentary I found on the
      > NetIQ/WebTrends website. For those not familiar with WebTrends,
      > they are the largest, most established vendor of log file solutions
      > in the market. "The vast majority of the data captured in log files
      > has limited use in understanding visitor behavior. A related but
      > more serious issue with web server logs is the potential accuracy
      > problems they create. In many cases, web server log files do not
      > accurately represent the actual visitor interaction with a web
      > site."
      > >
      > > Here is the WebTrends page for reference -
      >
      http://www.netiq.com/webtrends/resources/web_analytics_101/clientside
      > tagging.asp
      > >
      > > - Matt.
      > >
      > > Stephen Turner <yahoo@a...> wrote:
      > >
      > >
      > > --- In webanalytics@yahoogroups.com, Matt Belkin <mbelkin@y...>
      > wrote:
      > > > Hi Fred,
      > > >
      > > > I've done this many times (unfortunately) and in many ways. The
      > > > good news is that you're moving in the right direction. Page
      > tags,
      > > > while not perfect, are a significant step forward in accuracy,
      > > > reliability, scaleability, and actionability.
      > > >
      > >
      > > Matt states this as if it were an accepted fact, but I don't think
      > > it's a simple question at all.
      > >
      > > At ClickTracks, we offer both a logfile and a page tag version of
      > our
      > > software, so perhaps we are less biased than some vendors. The
      > fact is
      > > that both methods have different advantages and disadvantages,
      > both in
      > > terms of ease of implementation and in terms of accuracy of the
      > data.
      > > I could happily argue either side.
      > >
      > > And in the end, I actually think that people worry about it too
      > much.
      > > The exact numbers are usually not the important thing in website
      > > analysis. What is much more useful is when you start to look at
      > trends
      > > over time, or comparisons between different groups of visitors.
      And
      > > for these sorts of questions, any small differences between
      > logfiles
      > > and page tagging are much less important than the differences
      > you're
      > > actually analysing.
      > >
      > > --
      > > Stephen Turner
      > > CTO, ClickTracks http://www.clicktracks.com/
      > > ClickTracks wins ClickZ Marketing Excellence Award again!
      > > WINNER: Best Web Analytics Tool 2003 & 2004
    • matthewjncroche
      In my experience, the companies that do analytics well: 1. Have a catalog background as a company 2. Hire (or are lucky to have) someone with a DM or financial
      Message 45 of 45 , Nov 4, 2004
      • 0 Attachment
        In my experience, the companies that do analytics well:
        1. Have a catalog background as a company
        2. Hire (or are lucky to have) someone with a DM or financial service
        analyst background
        3. Very hands-on approach to the site content

        Where it fails:
        1. When the project definition in the first phase includes tagging
        everything and tracking everything
        2. When IT is overloaded and cannot make changes to the site to
        respond to information learned from the data.

        Biggest issue with analytics is loss of interest by the
        marketing/merch. team when they cannot/will not make changes to
        influence the numbers.

        Matthew Roche
        Offermatica
        www.offermatica.com

        --- In webanalytics@yahoogroups.com, Jim Sterne <jsterne@t...> wrote:
        > At 05:51 AM 11/2/2004, Jim Novo wrote:
        > >I think it's the "culture" issue; as a company, you have to want
        to dig
        > >into data and really understand your business.
        >
        > >At a higher conceptual level, it is companies that support a
        culture that
        > >is not afraid of failure. Another way to say this is
        experimentation and
        > >testing are encouraged throughout the company.
        >
        > At 02:56 PM 11/2/2004, Justin England wrote:
        > >The companies I have seen that are truly succeeding with analytics
        > >fall into the following categories:
        >
        > >3. (though 3rd this is probably most important) There is
        strong
        > >SUPPORT and
        > >UNDERSTANDING of/for analytics from the **TOP EXECUTIVES** of the
        company -
        >
        > I just hate to do this - I mean I really *hate* to do this
        > but ... welllll.... Amazon! OK! I said it. I said it for about
        > the 4 millionth time. These people get it. They take risks.
        > They measure everything. Their fearless (of failure) leader
        > is rabid about numbers.
        >
        > Yesterday, I was lunching with fellow speakers at Jakob
        > Nielsen's Usability Week. One of them was Tamara Adlin
        > from Amazon who said that Jeff Bezos is a wild man of
        > ideas. He comes up with the wackiest stuff to try and if
        > it doesn't work, so what? Thomas Edison is credited with
        > saying how many ways he now knows not to make a
        > light bulb.
        >
        > Yes, Amazon get massive amounts of traffic so they can
        > test an idea an hour. Yes, they are a Web first-last-and-
        > always kind of company. But that's the culture it takes to
        > be successful.
        >
        > Ronny Kohavi of Amazon put this in a PowerPoint slide
        > at the last Emetrics Summit: Data Trumps Intuition
        >
        >
        > -------------------------------------------------------------
        > Jim Sterne Target Marketing of Santa Barbara
        > 805-965-3184 http://www.targeting.com
        > Consultant, Author, Speaker on Measuring Website Success
        > -------------------------------------------------------------
        > Subscribe to "Sterne Measures" at http://www.emetrics.org
      Your message has been successfully submitted and would be delivered to recipients shortly.