Client and/or Server-Side Data / Outsourced Service and/or Software Product
Happy Halloween! Let me guess, you're dressed as a Page Tag or a
Third-Party Cookie :)
Yes, most measurement data collected from most all types of
applications, of most all sorts, end up in log files, we agree. The
page tag generated requests that end up in log files are logged by
the Web (HTTP) servers that receive the page tag requests.
"Log Files" have never come to mean "server-side" data collection
alone and certainly have never come to mean a class of vendors (a
class that I don't think even exists) that can only process
measurements generated on the "server-side." The folks that have
been in the internet industry for more than 10 years know that such
is just the marketing spin of a generation of ASPs; A group of
vendors that according to Jupiter currently make up 29% of the web
Web measurement data has always been collected from the "client-
side." In fact most all that web server log files contain is
information generated from the "client-side;" by a user agent
(browser or otherwise) which is then logged by a web server, whether
that web server is owned by an ASP or anyone else for that matter.
Client-side information gathered through the instrumentation of pages
for specific measurement purposes is no exception.
My comments are not playing semantics with you. Rather they focus on
clarifying the confusing language that has been used, and you are
using as marketing spin to seemingly confound the less technical in
some hope of them believing that "client-side" data collection is
somehow better than "client-side" and "server-side" data collection.
This is logic I just can't seem to stomach, its kind of like saying
that I would rather have one dollar instead of two because I can only
spend one dollar today.
The fact is, that the market has many vendors who can collect both
client and server-side data and use the best of both to provide value
to their customers. Some of those companies offer their services on
an outsourced basis (as an "ASP" or a "Managed Service") and some
offer software products that their clients can run themselves to gain
this value, while still others offer both of these delivery options.
Pretending that there is some special linkage between "client-side"
data collection and ASPs is just marketing spin. Rather, ASPs only
collect client-side information because that is what they are
structured to do. So sure, it is clear why you are pushing this
issue in this forum, if a company wants anything other than Page Tag
generated "client-side" data then an ASP-only vendor doesn't provide
it and that ASP-only vendor can not effectively compete for the that
company's business by other than convincing that company that
anything else is not important or won't produce a ROI.
To pretend to the good folks here that companies have not found value
and ROI in server-side data in addition to client-side data, and that
companies have not found value in software in addition to ASP
services is just marketing spin.
Vendors that offer software products and outsourced services can
collect both "client-side" and "server-side" data. They can do this
as either an "outsourced/managed service or ASP" or provide software
products so that a client can process their data themselves and keep
their customer's information privately behind their own firewalls.
So Matt, I think that the strategic question for many companies
is, "if they can have the best of client-side measurement, in
addition to the best of server-side measurement, and they can have it
as an ASP or they can have it as a software product they can run,
then why would they purchase a service that can only collect client-
side information and is only available on an outsourced/ASP basis?"
Given that Visual Sciences and many other vendors can provide both
software and outsourced service and collect both client and server-
side data, I would have to agree with you the real issue that should
be focused on is how much Business Value can be created from client
and server-side data through the analysis of it.
Visual Sciences was proud to have Jupiter rate Visual Sciences as the
vendor whose offerings provide the most "Overall Business Value" of
the top web analytics vendors in their most recent report. So rather
than provide the so many sometimes competitors that are in attendance
here Visual Sciences' client ROI cases (as such are reserved for the
clients whose business we will compete with you for), I would refer
you and others to the due diligence of JupiterResearch in regard to
value that Visual Sciences generates for its clients.
Now what I won't claim to you tonight is that Visual Sciences or any
data collection and analysis solution produces "exponential ROI,"
because I and most of the people here know what "exponential"
means.. But hey, if using page tag generated client-side data
alone, and dropping the server-side data collection method, for some
cosmic reason, produces "exponential ROI," then I and every other
sane person here and in the investing world would be putting every
available dollar into Page Tag-only/ASP-only data collection and
reporting solutions rather buying stocks, bonds, commodities and
In any case, Happy Halloween and I hope a third-party doesn't eat
your cookie ;)
--- In email@example.com, Matt Belkin <mbelkin@y...> wrote:
> Jim, interesting comments as always...and yes, I actually read the
whole thing :)
> First, let me say I'm certainly aware that page tags generate their
own 'log file'; akin to server log files. However, since this
industry began circa in the mid 1990's, the term 'log file' has been
commonly accepted to mean a server log file. So sure, I agree with
your observation that both data collection approaches generate logs,
but that is simply a semantic discussion (or a silly one) - not a
strategic one that would allow our mutual customers to be more
successful. Perhaps the Web Analytics Association can help out in
that area, but for time being, I'd rather focus on helping customers
maximize their analytics ROI. And in my experience, server log files
have never produced the exponential levels of ROI that I have seen
from page tag collection methods. If you have seen otherwise, I'd
love to hear about some case studies of such success amongst your
> Finally, I agree that server logs are useful for tracking things
like 'hits' on movies and mp3s - but again, where's the ROI? That's
tactical stuff, good for blocking and tackling - but not the type of
measurement that empowers organizations to make strategic decisions
about their customers, marketing expenditures, and online strategy.
> Cheers, and thanks as always for the stimulating discussion - Matt.
> Matt Belkin
> Vice President, Best Practices Group
> jimmacintyreiv <jim@v...> wrote:
> This thread/conversation about Page Tags and Log Files in general
> and statements like "Page Tagging is more accurate, scalable,
> reliable and actionable" remind me of the famous quote "Complex
> problems have simple, easy-to-understand wrong answers."
> Like with most overreaching statements and promises it pays
> the fine print." And the fine print most often says "buyer
> The fine print in the case of web analytics is the technical detail
> around measurement and data collection. It may be nice to avoid
> this topic, but if you do you will someday pay the real price that
> the sales person didn't want to tell you about. They might tell
> that their way is "best practice" but if that is all they can do
> then you might take that as a clue to review the fine print right
> away and at the very least consider it a "best practice" for them
> rather than necesarily a "best practice" for you.
> I have no particular interest in one method of data collection or
> the other (except that my clients choose the one that is right for
> their situation) as Visual Sciences supports "Page Tagging" (Client-
> Side Only), "Web and Application Server API" (Server-Side Only),
> Traditional Web Server "Log File Processing", "Real-Time Client and
> Server Side Combined" Data Collection and even some lesser used and
> known methods. And because we do so I can confidently testify to
> the group that every site has different issues and in many cases
> method that is right for one is not right for the other, as Stephen
> suggests. The idea that Server-Side data collection is equivalent
> to Log File processing is a long dead topic. It is not, and there
> is absolutely no reason that a site must choose between having
> server side and client side information, they don't have to, they
> can have the best of both and eliminate the worst of both by doing
> some thoughtful planning and choosing the right tools and services.
> So some background on the fine print . . . and I apologize in
> advance to the good people that I am boring with long known
> that is added to a web page to cause an embedded object request to
> be made to web (http) servers. The embedded object request is
> off when the page is loaded and that script runs in the browser's
> object request to be made. That Page Tag driven embedded object
> file, image, etc.) might be made to web servers owned by a web
> analytics ASP (third-party request) or to web servers owned by the
> site owner (first-party request).
> Those web servers that receive those Page Tag instigated embedded
> requests generate log entries that are written to Log Files. Those
> Log Files are processed by a web analytics ASP's or the site
> data extraction, transformation, loading (ETL) and management
> systems for reporting and maybe analysis.
> It is silly to talk about Page Tags verses Log Files as Page Tags
> produce Log Files when they cause HTTP requests and web servers at
> an ASP log those HTTP requests into Log Files.
> The question is what is in the Log Files. For an ASP the Log Files
> contain the information sent to them with the embedded object
> request, typically one request per Page View, but sometimes more
> than one.
> For a Log File created by a typical web server, all of the HTTP
> requests made to the web server are placed in the Log File if the
> server is configured to log them.
> So the ASP services' web servers log Page Views (through the proxy
> of an embedded object request) and regular web servers log Page
> Views (except those served from the browser's cache) as well as all
> other types of requests (e.g. PDFs, MP3s, Images, Style Sheets,
> etc.) and they also track server errors, etc., where Page Tagging
> services do not.
> Most ASP-only services use Page Tags because that's what they can
> do. They use Page Tags to track Page Views on a third-party
> They set unique identifiers in cookies when the Page Tag's embedded
> object request is made so that they can try to identify returning
> visitors (browsers) and use the unique id to knit together sessions
> when they process the Page Tag Log Files. They can do that in
> fullest form when the Page Tag code is actually run by the
> function to kick off an embedded object request, dropping and
> the data that would have been collected from the HTML DOM by the
> It is nonsense to talk about Page Tags as being "more accurate"
> than "Log Files" or server-side data in general. Using data that
> collected with Page Tags alone is only more "accurate" than W3C log
> files produced by web servers, in regard to tracking Page Views,
> when the Page View is served from the browser's cache, and it is
> much less accurate at tracking anything other than Page Views
> because it is hard or impossible to Tag many of the other types of
> objects served from sites on the Internet. Any site operator can
> add a one-line "Page Tag" to their web pages to make the same Page
> View that is served from a browser cache be tracked and logged in
> their standard web server Log Files. There is no vendor at all
> required to do this. The same is true for any thing that a Page
> ASP can do. Anyone can add a page tag to extract information from
> dynamic page for instance and cause it to be sent back to their
> servers for logging and use.
> In fact using Page Tags alone is less accurate in many ways, and
> here are just a short list of them: 1) Page Tags drop information
> log Page Views when someone forgets to put the Page Tags in the
> or puts them in wrong; 3) Page Tag requests served/collected by
> third-parties (ASPs) cannot be identified as being from a
> browser (visitor) when third-party cookies are not allowed by the
> browser; and 4) Page Tag requests sometimes never make it to the
> logs of the third-party ASP because of network congestion, server
> saturation and other issues (have you ever seen an error in loading
> a web page? Well then have you ever seen an error from a Page Tag?
> If not, can you realistically believe that is because there aren't
> In fact a growing number of browser settings and a Internet
> population that is getting ever smarter about how to use them and
> other privacy tools can make Page Tag only solutions increasingly
> inaccurate as time goes by. The still worst thing about Page Tag
> only solutions is that you will never even know when they are
> inaccurate so you can fix them. This is very convenient for Page
> Tag only ASP vendors. In many cases they seem to want you to turn
> off your other server logging . . . if you do you will never know
> how inaccurate they really are, especially if you get convinced, as
> I saw someone trying to do the convincing in this thread, to never
> even check.
> It is also silly to talk about Page Tag only solutions being "more
> scalable." In fact very few if any sites can stop logging on their
> web servers, for many good reasons, so rather than dealing with
> their log files and maybe a very small page tag added to the pages
> to track browser cached requests, site operators not only have to
> pay to collect and store their web server's actual log files, they
> pay for an ASP to create still more log files from additional page
> It is a fantasy to believe that the web server log files can be
> turned off because the log files contain all sorts of information
> necessary for the technical operations of the site like what errors
> are occurring on the site. Sure a Page Tag only solution might be
> able to Page Tag "error pages" on a site, but it cannot track
> on anything other than Page Views . . . and the web is a multi-
> media environment where images, movies, songs, documents any number
> of other types of content objects exist and need to be monitored by
> someone unless you run a very simple site.
> If an ASP only solution could convince your technical team to turn
> off your web server logging then the part of purchasing an ASP
> service that is "more scalable" can all be summed up in the
> statement that the ASP service is employing a data reduction
> strategy and only collecting a small portion of the data that a
> operator might actually find useful if they had the tools to do so.
> The data reduction strategy is eliminating most of the data that
> gone into traditional web server log files by just logging Page
> Views. So sure, because Page Views are typically 1/10th to 1/30th
> of the total number of HTTP requests to a site then such a data
> reduction strategy makes a web analytics system seem able to handle
> more Page Views. This is not "more scalable" this is a it is more
> scalable in regard to Page Views alone.
> Page Tagging only may be the right strategy for you site and it may
> not, and by the way you can do this yourself if you like by just
> Page Tagging your site and sending the Page Tags back to a
> web server or two on your site and spare yourself the third-party
> cookie and P3P issues, the chance that you traffic in your
> customer's personal information improperly and other negatives.
> Further, many web analytics software tools can easily filter out
> Page View HTTP requests for the purposes of reporting analysis
> still leaving the rest of your log files available for other types
> of technical analysis that your operations team might need to do.
> "More Reliable" OK so some software products that have existed in
> the market over the last 10 years haven't been very reliable. That
> does not mean that all of them are unreliable. In fact I regularly
> hear stories about the irregular availability of reports and
> analysis from ASP only vendors. Worse, any complication in the
> ASP's data collection infrastructure can make data collection very
> unreliable. The worse thing again here is you will likely never
> know unless they tell you. What if the web analytics ASP has taken
> on some very large clients and they have a busy day and swamp the
> data collection servers for the ASP? What happens to those Page
> requests from your site. . .they disappear into the ether is what
> happens, but will you ever know about it? Not likely.
> And "More Actionable?" I'm even going to address that now, its too
> late and rather than address it I would refer you to the proprietor
> of this forum's latest report and specifically the amount of actual
> business value derived from the solutions reviewed. . . the
> value delivered by the solution rated highest in Business Value was
> sometimes derived from Page Tags and sometimes not. . there is not
> particular correlation between Page Tagging and Actionability, that
> is just vendor marketing talk and has little to nothing to do with
> Page Tagging verses other methods of data collection.
> --- In firstname.lastname@example.org, Matt Belkin <mbelkin@y...>
> > Stephen,
> > Interesting comments. Omniture can work with log file data, so I
> just want to be clear that my bias for page tagging isn't a
> of that :) It's a function of having dealt with logs for years and
> struggled to justify the investment. I agree that there are
> considerations that will be unique to each customer, but with so
> many high- and low-end page tag products on the market, I can't
> understand why folks would stick with log files.
> > Anyway, one of your comments drew my curiosity - you said it is
> much more useful to look at trends over time, or comparisons
> different groups of visitors. I completely agree with this
> statement. But what I'd love more color on is how log files help
> you achieve this? I've used log file solutions from several
> (who shall remain nameless), and found it perpetually challenging
> make reliable decisions from the data.
> > For example, a customer segment I created would jump by 15% one
> day, and upon further exploration, we discovered our program failed
> to properly parse IP addresses. On another occasion, advertising
> impressions (as measured by pageview counts) were inaccurate
> of proxy caching issues and some spidering activity. This didn't
> sit well with our advertising partners. Another issue we ran into
> Macromedia specifically was Flash tracking - simply couldn't do it
> with log files. Finally, I found it difficult to tie campaign
> response and customer value to the individual, both from an
> dollar standpoint and within a dynamic app.
> > Anyway, forgive me if your log file product has workarounds for
> these issues, I've never used ClickTracks. But if it does, I'd
> to hear more about it and I'm sure the other members of the board
> who still use log files would benefit from some clarity too :)
> > In closing, I thought I'd share some commentary I found on the
> NetIQ/WebTrends website. For those not familiar with WebTrends,
> they are the largest, most established vendor of log file solutions
> in the market. "The vast majority of the data captured in log files
> has limited use in understanding visitor behavior. A related but
> more serious issue with web server logs is the potential accuracy
> problems they create. In many cases, web server log files do not
> accurately represent the actual visitor interaction with a web
> > Here is the WebTrends page for reference -
> > - Matt.
> > Stephen Turner <yahoo@a...> wrote:
> > --- In email@example.com, Matt Belkin <mbelkin@y...>
> > > Hi Fred,
> > >
> > > I've done this many times (unfortunately) and in many ways. The
> > > good news is that you're moving in the right direction. Page
> > > while not perfect, are a significant step forward in accuracy,
> > > reliability, scaleability, and actionability.
> > >
> > Matt states this as if it were an accepted fact, but I don't think
> > it's a simple question at all.
> > At ClickTracks, we offer both a logfile and a page tag version of
> > software, so perhaps we are less biased than some vendors. The
> fact is
> > that both methods have different advantages and disadvantages,
> both in
> > terms of ease of implementation and in terms of accuracy of the
> > I could happily argue either side.
> > And in the end, I actually think that people worry about it too
> > The exact numbers are usually not the important thing in website
> > analysis. What is much more useful is when you start to look at
> > over time, or comparisons between different groups of visitors.
> > for these sorts of questions, any small differences between
> > and page tagging are much less important than the differences
> > actually analysing.
> > --
> > Stephen Turner
> > CTO, ClickTracks http://www.clicktracks.com/
> > ClickTracks wins ClickZ Marketing Excellence Award again!
> > WINNER: Best Web Analytics Tool 2003 & 2004
- In my experience, the companies that do analytics well:
1. Have a catalog background as a company
2. Hire (or are lucky to have) someone with a DM or financial service
3. Very hands-on approach to the site content
Where it fails:
1. When the project definition in the first phase includes tagging
everything and tracking everything
2. When IT is overloaded and cannot make changes to the site to
respond to information learned from the data.
Biggest issue with analytics is loss of interest by the
marketing/merch. team when they cannot/will not make changes to
influence the numbers.
--- In firstname.lastname@example.org, Jim Sterne <jsterne@t...> wrote:
> At 05:51 AM 11/2/2004, Jim Novo wrote:
> >I think it's the "culture" issue; as a company, you have to want
> >into data and really understand your business.
> >At a higher conceptual level, it is companies that support a
> >is not afraid of failure. Another way to say this is
> >testing are encouraged throughout the company.
> At 02:56 PM 11/2/2004, Justin England wrote:
> >The companies I have seen that are truly succeeding with analytics
> >fall into the following categories:
> >3. (though 3rd this is probably most important) There is
> >SUPPORT and
> >UNDERSTANDING of/for analytics from the **TOP EXECUTIVES** of the
> I just hate to do this - I mean I really *hate* to do this
> but ... welllll.... Amazon! OK! I said it. I said it for about
> the 4 millionth time. These people get it. They take risks.
> They measure everything. Their fearless (of failure) leader
> is rabid about numbers.
> Yesterday, I was lunching with fellow speakers at Jakob
> Nielsen's Usability Week. One of them was Tamara Adlin
> from Amazon who said that Jeff Bezos is a wild man of
> ideas. He comes up with the wackiest stuff to try and if
> it doesn't work, so what? Thomas Edison is credited with
> saying how many ways he now knows not to make a
> light bulb.
> Yes, Amazon get massive amounts of traffic so they can
> test an idea an hour. Yes, they are a Web first-last-and-
> always kind of company. But that's the culture it takes to
> be successful.
> Ronny Kohavi of Amazon put this in a PowerPoint slide
> at the last Emetrics Summit: Data Trumps Intuition
> Jim Sterne Target Marketing of Santa Barbara
> 805-965-3184 http://www.targeting.com
> Consultant, Author, Speaker on Measuring Website Success
> Subscribe to "Sterne Measures" at http://www.emetrics.org