Loading ...
Sorry, an error occurred while loading the content.

Re: [jslint] Actual JavaScript Engine Performance (UNCLASSIFIED)

Expand Messages
  • Cheney, Edward A SSG RES USAR USARC
    Classification: UNCLASSIFIED ... It is not so simple. Computation time of JavaScript, and thus metrics of performance results, is extremely relative. It is
    Message 1 of 18 , Apr 20, 2011
    • 0 Attachment
      Classification: UNCLASSIFIED

      > That's kind of exciting to see. Chrome's bragging is so justified,
      > apparently.

      It is not so simple. Computation time of JavaScript, and thus metrics of performance results, is extremely relative. It is relative to a single machine with a somewhat static software configuration, the user-agent, and the state of the user-agent. In JavaScript performance benchmarks I never see any mention or measure of user-agent state. Consider the following:

      1) Open Firefox and after all application start up is complete verify the amount memory consumed by that application process thread, thread hierarchy, or thread collection is consuming as reported by your respective operating system.

      2) Do not open any additional tab in the browser. In the one default tab go to JSLint with the "Good Parts" option set and test a very large application, such the JSLint application itself. Observed the processing time.

      3) Then open 11 additional tabs. Point some of these tabs to various large PDF files. Point one other tab to Facebook and message some people you know and engage in their live chat for a few seconds. In yet another tab go to armorgames.com and play for two minutes in 5 different games. In yet another tab go view some of the HTML5 demos online and interact with them. In yet another tab go to a website with horrible amounts of advertisements, of which there are a few to pick from. Do not close any of these tabs.

      4) In the original tab run JSLint again. The processing time will have increased by more than 15%, which is outside an allowable threshold of variance.

      5) Observe the amount of memory the browser is now consuming compared to when it was originally opened. I gotten a single Firefox browser to consume more than 1.4gb of memory on a 2gb machine. The more memory the browser consumes the lower its processing performance of JavaScript, especially with regards to large JavaScript applications receiving extremely large input.

      This said you can make Firefox 3.6 out perform Chrome 10 on the same machine running the same JavaScript application with the same input. You could even make IE7 out perform Firefox 3.1 presuming you don't crash the browser or operating system attempting to push the memory threshold, which Firefox can and will do on Windows XP.

      Austin Cheney, CISSP
      http://prettydiff.com/
      Classification: UNCLASSIFIED
    • Mark Volkmann
      ... Am I reading the results wrong? I thought they show that Chrome has the slowest JavaScript engine. -- R. Mark Volkmann Object Computing, Inc.
      Message 2 of 18 , Apr 20, 2011
      • 0 Attachment
        On Wed, Apr 20, 2011 at 11:55 AM, Erik Eckhardt <erik@...> wrote:
        > That's kind of exciting to see. Chrome's bragging is so justified, apparently.

        Am I reading the results wrong? I thought they show that Chrome has
        the slowest JavaScript engine.

        --
        R. Mark Volkmann
        Object Computing, Inc.
      • Kent Davidson
        Ditto that sentiment. It appears that the Seconds column shows the number of seconds, then the last column is relative to IE10, which has the fastest
        Message 3 of 18 , Apr 20, 2011
        • 0 Attachment
          Ditto that sentiment. It appears that the "Seconds" column shows the number of seconds, then the last column is relative to IE10, which has the fastest parser/execution engine. In order of slow to fast:

          Chrome, IE 9, Opera 11, FF 4, Safari 5, IE 10

          Surprising about Chrome, wonder if it's some RegExp issue.

          -Kent.

          On Apr 20, 2011, at 3:20 PM, Mark Volkmann wrote:

          > On Wed, Apr 20, 2011 at 11:55 AM, Erik Eckhardt <erik@...> wrote:
          > > That's kind of exciting to see. Chrome's bragging is so justified, apparently.
          >
          > Am I reading the results wrong? I thought they show that Chrome has
          > the slowest JavaScript engine.
          >
          > --
          > R. Mark Volkmann
          > Object Computing, Inc.
          >



          [Non-text portions of this message have been removed]
        • Satyam
          As the subject line well says, this is the performance of the JavaScript engine, not the performance of the browser as a whole. How good is the interface to
          Message 4 of 18 , Apr 20, 2011
          • 0 Attachment
            As the subject line well says, this is the performance of the JavaScript
            engine, not the performance of the browser as a whole. How good is the
            interface to the DOM (usually several times slower than the JavaScript
            execution on its own) or how many XHR connections to the data servers it
            can handle might really make a difference in a whole application. So,
            read these results as what they are, as the subject line states.

            Satyam

            El 20/04/2011 21:39, Kent Davidson escribió:
            > Ditto that sentiment. It appears that the "Seconds" column shows the number of seconds, then the last column is relative to IE10, which has the fastest parser/execution engine. In order of slow to fast:
            >
            > Chrome, IE 9, Opera 11, FF 4, Safari 5, IE 10
            >
            > Surprising about Chrome, wonder if it's some RegExp issue.
            >
            > -Kent.
            >
            > On Apr 20, 2011, at 3:20 PM, Mark Volkmann wrote:
            >
            >> On Wed, Apr 20, 2011 at 11:55 AM, Erik Eckhardt<erik@...> wrote:
            >>> That's kind of exciting to see. Chrome's bragging is so justified, apparently.
            >> Am I reading the results wrong? I thought they show that Chrome has
            >> the slowest JavaScript engine.
            >>
            >> --
            >> R. Mark Volkmann
            >> Object Computing, Inc.
            >>
            >
            >
            > [Non-text portions of this message have been removed]
            >
            >
            >
            > ------------------------------------
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
            >
            > -----
            > No virus found in this message.
            > Checked by AVG - www.avg.com
            > Version: 10.0.1209 / Virus Database: 1500/3585 - Release Date: 04/20/11
            >
            >
          • Eric Goforth
            What s the second IE10 column, is that the performance of IE10 on same tests as the other browser? If so, why does its performance vary so much? -Eric On
            Message 5 of 18 , Apr 20, 2011
            • 0 Attachment
              What's the second "IE10" column, is that the performance of IE10 on same
              tests as the other browser? If so, why does its performance vary so
              much?

              -Eric

              On Wed, 20 Apr 2011 15:39:18 -0400
              Kent Davidson <kent@...> wrote:

              > Ditto that sentiment. It appears that the "Seconds" column shows the
              > number of seconds, then the last column is relative to IE10, which
              > has the fastest parser/execution engine. In order of slow to fast:
              >
              > Chrome, IE 9, Opera 11, FF 4, Safari 5, IE 10
              >
              > Surprising about Chrome, wonder if it's some RegExp issue.
              >
              > -Kent.
              >
              > On Apr 20, 2011, at 3:20 PM, Mark Volkmann wrote:
              >
              > > On Wed, Apr 20, 2011 at 11:55 AM, Erik Eckhardt
              > > <erik@...> wrote:
              > > > That's kind of exciting to see. Chrome's bragging is so
              > > > justified, apparently.
              > >
              > > Am I reading the results wrong? I thought they show that Chrome has
              > > the slowest JavaScript engine.
              > >
              > > --
              > > R. Mark Volkmann
              > > Object Computing, Inc.
              > >
            • Luke Page
              Real life performance measuring almost always involves dom.. not many computationally expensive things are written for browsers. I would class jslint as an
              Message 6 of 18 , Apr 20, 2011
              • 0 Attachment
                Real life performance measuring almost always involves dom.. not many
                computationally expensive things are written for browsers.

                I would class jslint as an exception to this. Ie9 has fast javascript
                execution but in some cases the same dom performance as ie8 making in the
                end, little real world performance benefit. Has anyone tried profiling
                Chrome to see what is slow?
                On 20 Apr 2011 21:13, "Satyam" <satyam@...> wrote:
                > As the subject line well says, this is the performance of the JavaScript
                > engine, not the performance of the browser as a whole. How good is the
                > interface to the DOM (usually several times slower than the JavaScript
                > execution on its own) or how many XHR connections to the data servers it
                > can handle might really make a difference in a whole application. So,
                > read these results as what they are, as the subject line states.
                >
                > Satyam
                >
                > El 20/04/2011 21:39, Kent Davidson escribió:
                >> Ditto that sentiment. It appears that the "Seconds" column shows the
                number of seconds, then the last column is relative to IE10, which has the
                fastest parser/execution engine. In order of slow to fast:
                >>
                >> Chrome, IE 9, Opera 11, FF 4, Safari 5, IE 10
                >>
                >> Surprising about Chrome, wonder if it's some RegExp issue.
                >>
                >> -Kent.
                >>
                >> On Apr 20, 2011, at 3:20 PM, Mark Volkmann wrote:
                >>
                >>> On Wed, Apr 20, 2011 at 11:55 AM, Erik Eckhardt<erik@...>
                wrote:
                >>>> That's kind of exciting to see. Chrome's bragging is so justified,
                apparently.
                >>> Am I reading the results wrong? I thought they show that Chrome has
                >>> the slowest JavaScript engine.
                >>>
                >>> --
                >>> R. Mark Volkmann
                >>> Object Computing, Inc.
                >>>
                >>
                >>
                >> [Non-text portions of this message have been removed]
                >>
                >>
                >>
                >> ------------------------------------
                >>
                >> Yahoo! Groups Links
                >>
                >>
                >>
                >>
                >>
                >> -----
                >> No virus found in this message.
                >> Checked by AVG - www.avg.com
                >> Version: 10.0.1209 / Virus Database: 1500/3585 - Release Date: 04/20/11
                >>
                >>


                [Non-text portions of this message have been removed]
              • Cheney, Edward A SSG RES USAR USARC
                Classification: UNCLASSIFIED ... That is a slippery slope. The depth of integration of the JavaScript interpreter into the core of the browser application is
                Message 7 of 18 , Apr 20, 2011
                • 0 Attachment
                  Classification: UNCLASSIFIED
                  > As the subject line well says, this is the performance of thejava_script
                  > engine, not the performance of the browser as a whole.

                  That is a slippery slope. The depth of integration of the JavaScript interpreter into the core of the browser application is wildly variant. As a result your point is less valid in regards to some user-agents than with others. Unfortunately, your point cannot be entirely valid unless you wish to suggest a user-agent that is little more than an API wrapping a JavaScript interpreter, such as Rhino or Node.js. With regard to measuring performance and responsiveness how do you separate the JavaScript interpreter from the browser?
                  Classification: UNCLASSIFIED
                • mathew
                  ... Looks to me like the factor by which IE 10 is faster than the other browser. e.g. IE 10 takes 2 seconds, other browser takes 6 seconds, second IE10 column
                  Message 8 of 18 , Apr 20, 2011
                  • 0 Attachment
                    On Wed, Apr 20, 2011 at 15:54, Eric Goforth <eric.goforth@...> wrote:

                    > What's the second "IE10" column, is that the performance of IE10 on
                    > same tests as the other browser?
                    >
                    Looks to me like the factor by which IE 10 is faster than the other browser.

                    e.g. IE 10 takes 2 seconds, other browser takes 6 seconds, second IE10
                    column says 3 because the other browser takes 3x as long as IE10.


                    mathew


                    [Non-text portions of this message have been removed]
                  • Erik Eckhardt
                    For what it s worth I meant to say Chrome s bragging is NOT so justified On Wed, Apr 20, 2011 at 12:04 PM, Cheney, Edward A SSG RES USAR USARC
                    Message 9 of 18 , Apr 20, 2011
                    • 0 Attachment
                      For what it's worth I meant to say "Chrome's bragging is NOT so justified"

                      On Wed, Apr 20, 2011 at 12:04 PM, Cheney, Edward A SSG RES USAR USARC <
                      austin.cheney@...> wrote:

                      >
                      >
                      > Classification: UNCLASSIFIED
                      >
                      > > That's kind of exciting to see. Chrome's bragging is so justified,
                      > > apparently.
                      >
                      > It is not so simple. Computation time of JavaScript, and thus metrics of
                      > performance results, is extremely relative. It is relative to a single
                      > machine with a somewhat static software configuration, the user-agent, and
                      > the state of the user-agent. In JavaScript performance benchmarks I never
                      > see any mention or measure of user-agent state. Consider the following:
                      >
                      > 1) Open Firefox and after all application start up is complete verify the
                      > amount memory consumed by that application process thread, thread hierarchy,
                      > or thread collection is consuming as reported by your respective operating
                      > system.
                      >
                      > 2) Do not open any additional tab in the browser. In the one default tab go
                      > to JSLint with the "Good Parts" option set and test a very large
                      > application, such the JSLint application itself. Observed the processing
                      > time.
                      >
                      > 3) Then open 11 additional tabs. Point some of these tabs to various large
                      > PDF files. Point one other tab to Facebook and message some people you know
                      > and engage in their live chat for a few seconds. In yet another tab go to
                      > armorgames.com and play for two minutes in 5 different games. In yet
                      > another tab go view some of the HTML5 demos online and interact with them.
                      > In yet another tab go to a website with horrible amounts of advertisements,
                      > of which there are a few to pick from. Do not close any of these tabs.
                      >
                      > 4) In the original tab run JSLint again. The processing time will have
                      > increased by more than 15%, which is outside an allowable threshold of
                      > variance.
                      >
                      > 5) Observe the amount of memory the browser is now consuming compared to
                      > when it was originally opened. I gotten a single Firefox browser to consume
                      > more than 1.4gb of memory on a 2gb machine. The more memory the browser
                      > consumes the lower its processing performance of JavaScript, especially with
                      > regards to large JavaScript applications receiving extremely large input.
                      >
                      > This said you can make Firefox 3.6 out perform Chrome 10 on the same
                      > machine running the same JavaScript application with the same input. You
                      > could even make IE7 out perform Firefox 3.1 presuming you don't crash the
                      > browser or operating system attempting to push the memory threshold, which
                      > Firefox can and will do on Windows XP.
                      >
                      > Austin Cheney, CISSP
                      > http://prettydiff.com/
                      > Classification: UNCLASSIFIED
                      >
                      >


                      [Non-text portions of this message have been removed]
                    • Jean-Charles Meyrignac
                      Other questions: how did the browsers have been benchmarked ? Is the time column a mean of several runs (10 or 100 times ?), or is it only one run ? Also, do
                      Message 10 of 18 , Apr 20, 2011
                      • 0 Attachment
                        Other questions:

                        how did the browsers have been benchmarked ?

                        Is the time column a mean of several runs (10 or 100 times ?), or is it only
                        one run ?

                        Also, do you count the time to open the browser or not ?

                        And finally, is everything in cache ?

                        JC

                        On Thu, Apr 21, 2011 at 3:00 AM, mathew <meta404@...> wrote:

                        >
                        >
                        > On Wed, Apr 20, 2011 at 15:54, Eric Goforth <eric.goforth@...>
                        > wrote:
                        >
                        > > What's the second "IE10" column, is that the performance of IE10 on
                        > > same tests as the other browser?
                        > >
                        > Looks to me like the factor by which IE 10 is faster than the other
                        > browser.
                        >
                        > e.g. IE 10 takes 2 seconds, other browser takes 6 seconds, second IE10
                        > column says 3 because the other browser takes 3x as long as IE10.
                        >
                        >


                        [Non-text portions of this message have been removed]
                      • Douglas Crockford
                        The JSLint benchmark is a measure of JavaScript performance. It does not attempt to measure overall browser performance. It does not look at network
                        Message 11 of 18 , Apr 21, 2011
                        • 0 Attachment
                          The JSLint benchmark is a measure of JavaScript performance. It does not
                          attempt to measure overall browser performance. It does not look at
                          network efficiency, marshalling, rendering, layout, or any factor other
                          than the execution of JavaScript. Most web applications are DOM limited.
                          Such applications benefit little if at all from improved JavaScript
                          performance. The benefit of the new faster JavaScript engines is the
                          enabling of new kinds of applications.

                          The JSLint benchmark is 6.6KLOC. Ordinary benchmarks tend to be much
                          smaller. It does a good mix or regular expressions, string building
                          with + and join, and prototypal and functional patterns.

                          I was very surprised to see Chrome score at the bottom. My expectation
                          was that it would score at the top. I don't know why that was the case.
                        • Jean-Charles Meyrignac
                          I m sorry, but this is not an answer to my question. How did you compute the benchmark ? Did you run JSLint 10 times on every browser, removed the slowest and
                          Message 12 of 18 , Apr 21, 2011
                          • 0 Attachment
                            I'm sorry, but this is not an answer to my question.

                            How did you compute the benchmark ?

                            Did you run JSLint 10 times on every browser, removed the slowest and faster
                            runs, and computed the average of the remaining runs ? (I think that this is
                            the standard method to compute benchmarks).

                            Or did you run the program once, and just displayed the time it took to run
                            the program ?

                            Browsers tend to be slow when you start them, and Chrome is very slow when
                            you open a window (see how much memory it consumes !).
                            I guess also that their JS engine may be slower to generate assembly code,
                            but probably does a better job at optimizing it.
                            On the other hand, and I hope I'm not wrong, JSLint doesn't heavily execute
                            loops, so converting it to assembly code should not be very efficient.

                            Another thing is that, in order to appear fast, the browsers tend to redraw
                            the loaded page frequently, and this slows down the Javascript engine.

                            In the case of Chrome, the 2.801 seconds may be an artefact.

                            JC
                            (Not trying to defend Chrome, since I prefer Firefox !)


                            On Fri, Apr 22, 2011 at 1:31 AM, Douglas Crockford <douglas@...>wrote:

                            >
                            >
                            > The JSLint benchmark is a measure of JavaScript performance. It does not
                            > attempt to measure overall browser performance. It does not look at
                            > network efficiency, marshalling, rendering, layout, or any factor other
                            > than the execution of JavaScript. Most web applications are DOM limited.
                            > Such applications benefit little if at all from improved JavaScript
                            > performance. The benefit of the new faster JavaScript engines is the
                            > enabling of new kinds of applications.
                            >
                            > The JSLint benchmark is 6.6KLOC. Ordinary benchmarks tend to be much
                            > smaller. It does a good mix or regular expressions, string building
                            > with + and join, and prototypal and functional patterns.
                            >
                            > I was very surprised to see Chrome score at the bottom. My expectation
                            > was that it would score at the top. I don't know why that was the case.
                            >
                            >


                            [Non-text portions of this message have been removed]
                          • Douglas Crockford
                            ... Try the experiment yourself.
                            Message 13 of 18 , Apr 21, 2011
                            • 0 Attachment
                              --- In jslint_com@yahoogroups.com, Jean-Charles Meyrignac <jcmeyrignac@...> wrote:
                              >
                              > I'm sorry, but this is not an answer to my question.
                              >
                              > How did you compute the benchmark ?
                              >
                              > Did you run JSLint 10 times on every browser, removed the slowest and faster
                              > runs, and computed the average of the remaining runs ? (I think that this is
                              > the standard method to compute benchmarks).
                              >
                              > Or did you run the program once, and just displayed the time it took to run
                              > the program ?
                              >
                              > Browsers tend to be slow when you start them, and Chrome is very slow when
                              > you open a window (see how much memory it consumes !).
                              > I guess also that their JS engine may be slower to generate assembly code,
                              > but probably does a better job at optimizing it.
                              > On the other hand, and I hope I'm not wrong, JSLint doesn't heavily execute
                              > loops, so converting it to assembly code should not be very efficient.
                              >
                              > Another thing is that, in order to appear fast, the browsers tend to redraw
                              > the loaded page frequently, and this slows down the Javascript engine.
                              >
                              > In the case of Chrome, the 2.801 seconds may be an artefact.

                              Try the experiment yourself.
                            • Jean-Charles Meyrignac
                              Here are my timings, I have an old laptop with Windows 7 64-bits: Firefox 4.0: 0.807 seconds IE 9: 1.113 seconds Chrome: 1.867 seconds I guess that the
                              Message 14 of 18 , Apr 21, 2011
                              • 0 Attachment
                                Here are my timings, I have an old laptop with Windows 7 64-bits:

                                Firefox 4.0: 0.807 seconds
                                IE 9: 1.113 seconds
                                Chrome: 1.867 seconds

                                I guess that the rendering is slow, because the result page is large.

                                JC

                                On Fri, Apr 22, 2011 at 2:17 AM, Douglas Crockford <douglas@...>wrote:

                                >
                                >
                                > --- In jslint_com@yahoogroups.com, Jean-Charles Meyrignac <jcmeyrignac@...>
                                > wrote:
                                > >
                                >
                                > Try the experiment yourself.
                                >
                                >


                                [Non-text portions of this message have been removed]
                              • Douglas Crockford
                                ... That is in line with my results. ... Rendering is not a factor. It only measures JavaScript execution. Chrome executes slower.
                                Message 15 of 18 , Apr 21, 2011
                                • 0 Attachment
                                  --- In jslint_com@yahoogroups.com, Jean-Charles Meyrignac <jcmeyrignac@...> wrote:

                                  > Here are my timings, I have an old laptop with Windows 7 64-bits:
                                  >
                                  > Firefox 4.0: 0.807 seconds
                                  > IE 9: 1.113 seconds
                                  > Chrome: 1.867 seconds

                                  That is in line with my results.

                                  > I guess that the rendering is slow, because the result page is large.

                                  Rendering is not a factor. It only measures JavaScript execution.
                                  Chrome executes slower.
                                Your message has been successfully submitted and would be delivered to recipients shortly.