Loading ...
Sorry, an error occurred while loading the content.
 

Re: [webanalytics] Analytics Packages - Build or Buy?

Expand Messages
  • Edward Vielmetti
    On Jan 31, 2008 2:03 PM, Digital Alex by Alex L. Cohen ... I m personally disappointed that the state of the art in open source analytics tools isn t better
    Message 1 of 11 , Jan 31, 2008
      On Jan 31, 2008 2:03 PM, Digital Alex by Alex L. Cohen
      <alex@...> wrote:

      > Analysts - Would you prefer a custom tool?
      > Vendors - How do you address the points in this article?
      > Marketers - Have you considered or are you building your own tool?

      I'm personally disappointed that the state of the art in
      open source analytics tools isn't better than it is now.

      With massively parallel compute infrastructures within
      reach (something like Hadoop + EC2 + S3 on Amazon)
      you should be able to put together ad hoc reports on web sized data that
      run in something resembling interactive exploratory research
      time without investing $600 m last quarter on capex like
      the goog did.

      One of the Yahoo Research team was here in Ann Arbor
      talking about their parallel environment and the languages
      they are working on for constructing ad hoc queries - and I
      know that although Yahoo hires PhDs to design stuff, they
      don't hire PhDs to do routine work, so they are no smarter
      than me at the new-hire level. (Alas, Yahoo Research appears
      oblivious to the need to actually make money with the tools
      they develop.)

      Vendors get away with building crappy tools that take forever
      to learn and cost a fortune to install because they have gotten
      into a mindset that says it all has to run on iron that you own,
      and thus they need to coddle the data in such a way that your
      queries run in finite time on your machine. If you build your
      analysis architecture so that you rent your CPU by the job
      and have a workbench with machine tools instead of some
      prefab plastic single purpose tool, you should be able to
      get a lot broader set of interesting things done faster.

      Ed

      Edward Vielmetti
      Pure Visibility
      Ann Arbor, MI

      analyzing internet data since 1988



      >
      > -Alex
      > www.alexlcohen.com
      >
      > --
      > Alex Cohen
      >
      > [Non-text portions of this message have been removed]
      >
      >



      --
      Edward Vielmetti +1 734 330 2465
    • Steve
      ... It s usually known as import , or open . :-D You don t need anything special to data gather log files for analysis. Mind you, only 64K lines of logfile
      Message 2 of 11 , Feb 1, 2008
        On Feb 1, 2008 11:16 AM, Nick Arnett <narnett@...> wrote:
        > On Jan 31, 2008 1:22 PM, Steve <nuilvows@...> wrote:
        > > Out of curiosity: How many people in this industry *don't* use excel
        > > or some other spreadsheet or like program to smash numbers together?
        > > Because if you do use excel and friends you have just created your own
        > > custom analysis tool.
        >
        > Hmmm.... last time I checked, Excel didn't have any data-gathering
        > capabilities. Analytics requires data gathering and analysis.

        It's usually known as "import", or "open". :-D
        You don't need anything special to "data gather" log files for analysis.

        Mind you, only 64K lines of logfile is going to be ... challenging for
        many systems.

        ;-)


        But, evil humour aside, that wasn't the direction of my original point.
        Which is that will we or nil we, we probably all DO create custom
        ***analysis*** tools all the time. One off spreadsheets. Pivot Tables.
        etc etc etc etc etc.

        So while I certainly don't disagree with the thrust of Tim's (and
        others) comments, there is a line in the sand where a simple tool like
        a home built spreadsheet is the perfect *additional* analysis tool.
        Don't confuse all home grown analysis with trying to build a Visual
        Sciences like interface. Sometimes all you need is to compare a few
        numbers over time.

        Perhaps it's simply I'm more used to that mindset? I would easily
        create and discard up to dozens of home grown tools each and every
        day. Need to do X, "glue" these tools together == answer. Do Y, use
        these tools slightly differently == different answer. And so on.
        Ha! It could be easily argued that it's my *job* to continuously
        create and discard these simple tools. BTW: I'm not talking just
        analysis/analytics on this creation/discard cycle either.


        Cheers!
        - Steve
      • Julien Coquet
        I once built my own WA tool from scratch with about 300 lines of Javascript and PHP, which took me about one day. The tool is actually still in use, along with
        Message 3 of 11 , Feb 1, 2008
          I once built my own WA tool from scratch with about 300 lines of Javascript
          and PHP, which took me about one day.
          The tool is actually still in use, along with Google Analytics on some of my
          sites.
          Which is all fun and games for a few days but in the end, you realize that
          unless you spend weeks to make the tool modular and adapted to business
          requirements, you're better off buying one of the off-the-shelves solutions.

          Just my 2 cents :-)

          Cheers,

          Julien Coquet
          OX2
          Analytics Country Manager - France

          On Fri, Feb 1, 2008 at 7:20 AM, Edward Vielmetti <edward.vielmetti@...>
          wrote:

          > On Jan 31, 2008 2:03 PM, Digital Alex by Alex L. Cohen
          >
          > <alex@... <alex%40alexlcohen.com>> wrote:
          >
          > > Analysts - Would you prefer a custom tool?
          > > Vendors - How do you address the points in this article?
          > > Marketers - Have you considered or are you building your own tool?
          >
          > I'm personally disappointed that the state of the art in
          > open source analytics tools isn't better than it is now.
          >
          > With massively parallel compute infrastructures within
          > reach (something like Hadoop + EC2 + S3 on Amazon)
          > you should be able to put together ad hoc reports on web sized data that
          > run in something resembling interactive exploratory research
          > time without investing $600 m last quarter on capex like
          > the goog did.
          >
          > One of the Yahoo Research team was here in Ann Arbor
          > talking about their parallel environment and the languages
          > they are working on for constructing ad hoc queries - and I
          > know that although Yahoo hires PhDs to design stuff, they
          > don't hire PhDs to do routine work, so they are no smarter
          > than me at the new-hire level. (Alas, Yahoo Research appears
          > oblivious to the need to actually make money with the tools
          > they develop.)
          >
          > Vendors get away with building crappy tools that take forever
          > to learn and cost a fortune to install because they have gotten
          > into a mindset that says it all has to run on iron that you own,
          > and thus they need to coddle the data in such a way that your
          > queries run in finite time on your machine. If you build your
          > analysis architecture so that you rent your CPU by the job
          > and have a workbench with machine tools instead of some
          > prefab plastic single purpose tool, you should be able to
          > get a lot broader set of interesting things done faster.
          >
          > Ed
          >
          > Edward Vielmetti
          > Pure Visibility
          > Ann Arbor, MI
          >
          > analyzing internet data since 1988
          >
          > >
          > > -Alex
          > > www.alexlcohen.com
          > >
          > > --
          > > Alex Cohen
          > >
          > > [Non-text portions of this message have been removed]
          > >
          > >
          >
          > --
          > Edward Vielmetti +1 734 330 2465
          >
          >



          --
          Julien Coquet
          julien.coquet@...


          [Non-text portions of this message have been removed]
        • Tim Wilson
          Upgrade, Steve! Excel 2007 goes up to 1 million rows (...said the guy who was on 2007 for 3 months...and then changed jobs to a company that s still on 2003).
          Message 4 of 11 , Feb 1, 2008
            Upgrade, Steve! Excel 2007 goes up to 1 million rows (...said the guy
            who was on 2007 for 3 months...and then changed jobs to a company that's
            still on 2003).

            I threw a raw log file with ~500K rows in it at Excel 2007 and then did
            some pivot tables to see how it handled it. I was running a
            run-of-the-mill business-grade laptop...and was impressed with the
            results. It didn't choke at all.

            Tim Wilson
            Vice President, Services
            Office: 512.538.0371
            Mobile: 512.431.8502
            Fax: 512.652.2558
            twilson@...

            Bulldog Solutions Inc.
            5608 Parkcrest Drive, Suite 300
            Austin, TX 78731
            http://www.bulldogsolutions.com <http://www.bulldogsolutions.com/>






            ________________________________

            From: webanalytics@yahoogroups.com [mailto:webanalytics@yahoogroups.com]
            On Behalf Of Steve
            Sent: Friday, February 01, 2008 3:32 AM
            To: webanalytics@yahoogroups.com
            Subject: Re: [webanalytics] Analytics Packages - Build or Buy?



            On Feb 1, 2008 11:16 AM, Nick Arnett <narnett@...
            <mailto:narnett%40mccmedia.com> > wrote:
            > On Jan 31, 2008 1:22 PM, Steve <nuilvows@...
            <mailto:nuilvows%40yahoo.com.au> > wrote:
            > > Out of curiosity: How many people in this industry *don't* use excel
            > > or some other spreadsheet or like program to smash numbers together?
            > > Because if you do use excel and friends you have just created your
            own
            > > custom analysis tool.
            >
            > Hmmm.... last time I checked, Excel didn't have any data-gathering
            > capabilities. Analytics requires data gathering and analysis.

            It's usually known as "import", or "open". :-D
            You don't need anything special to "data gather" log files for analysis.

            Mind you, only 64K lines of logfile is going to be ... challenging for
            many systems.

            ;-)

            But, evil humour aside, that wasn't the direction of my original point.
            Which is that will we or nil we, we probably all DO create custom
            ***analysis*** tools all the time. One off spreadsheets. Pivot Tables.
            etc etc etc etc etc.

            So while I certainly don't disagree with the thrust of Tim's (and
            others) comments, there is a line in the sand where a simple tool like
            a home built spreadsheet is the perfect *additional* analysis tool.
            Don't confuse all home grown analysis with trying to build a Visual
            Sciences like interface. Sometimes all you need is to compare a few
            numbers over time.

            Perhaps it's simply I'm more used to that mindset? I would easily
            create and discard up to dozens of home grown tools each and every
            day. Need to do X, "glue" these tools together == answer. Do Y, use
            these tools slightly differently == different answer. And so on.
            Ha! It could be easily argued that it's my *job* to continuously
            create and discard these simple tools. BTW: I'm not talking just
            analysis/analytics on this creation/discard cycle either.

            Cheers!
            - Steve





            [Non-text portions of this message have been removed]
          Your message has been successfully submitted and would be delivered to recipients shortly.