Loading ...
Sorry, an error occurred while loading the content.

Re: [pcgen_developers] memory usage

Expand Messages
  • Tom Parker
    As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many
    Message 1 of 22 , Jan 28, 2013
    • 0 Attachment

      As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many cases it may save a method call... vs some cases where it's a useless null check... (which is the penalty of lazy instantiation)... but either way, you will be lucky to even be able to measure the difference since the things I'm talking about are a handful of CPU cycles, and not really material in the grand scheme.

      At load time, it will probably be faster, since it will be doing a lot less memory allocation.

      TP.
      --
      Tom Parker


      From: Chris Dolan <chris@...>
      To: pcgen_developers@yahoogroups.com
      Sent: Monday, January 28, 2013 5:04 PM
      Subject: Re: [pcgen_developers] memory usage

      Aha, awesome. I'll try that change this week and benchmark how much it
      helps memory vs. hurts speed. Thanks!
      Chris

    • Chris Dolan
      Do I need to worry about synchronization of the lazy instantiation? It looks like most of PCGen is two-threaded: one for loading and the UI thread, and the
      Message 2 of 22 , Jan 28, 2013
      • 0 Attachment
        Do I need to worry about synchronization of the lazy instantiation? It looks like most of PCGen is two-threaded: one for loading and the UI thread, and the flow of data is purely one-way. So I suspect the answer is no, synchronization is not important.

        Chris



        On Jan 28, 2013, at 4:53 PM, Tom Parker wrote:




        As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many cases it may save a method call... vs some cases where it's a useless null check... (which is the penalty of lazy instantiation)... but either way, you will be lucky to even be able to measure the difference since the things I'm talking about are a handful of CPU cycles, and not really material in the grand scheme.

        At load time, it will probably be faster, since it will be doing a lot less memory allocation.

        TP. 
        --
        Tom Parker


        From: Chris Dolan <chris@...>
        To: pcgen_developers@yahoogroups.com 
        Sent: Monday, January 28, 2013 5:04 PM
        Subject: Re: [pcgen_developers] memory usage

        Aha, awesome. I'll try that change this week and benchmark how much it
        helps memory vs. hurts speed. Thanks!
        Chris




      • Tom Parker
        Since the map creation should all happen at LST load, the answer *should* be no. That being said, I make no guarantees about UI behavior.  I was not involved
        Message 3 of 22 , Jan 28, 2013
        • 0 Attachment
          Since the map creation should all happen at LST load, the answer *should* be no.

          That being said, I make no guarantees about UI behavior.  I was not involved in that project.  James or Connor might be able to help with what the UI is doing during LST load.

          TP.
          --
          Tom Parker


          From: Chris Dolan <chris@...>
          To: pcgen_developers@yahoogroups.com
          Sent: Monday, January 28, 2013 7:32 PM
          Subject: Re: [pcgen_developers] memory usage



          Do I need to worry about synchronization of the lazy instantiation? It looks like most of PCGen is two-threaded: one for loading and the UI thread, and the flow of data is purely one-way. So I suspect the answer is no, synchronization is not important.

          Chris



          On Jan 28, 2013, at 4:53 PM, Tom Parker wrote:




          As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many cases it may save a method call... vs some cases where it's a useless null check... (which is the penalty of lazy instantiation)... but either way, you will be lucky to even be able to measure the difference since the things I'm talking about are a handful of CPU cycles, and not really material in the grand scheme.

          At load time, it will probably be faster, since it will be doing a lot less memory allocation.

          TP. 
          --
          Tom Parker


          From: Chris Dolan <chris@...>
          To: pcgen_developers@yahoogroups.com 
          Sent: Monday, January 28, 2013 5:04 PM
          Subject: Re: [pcgen_developers] memory usage

          Aha, awesome. I'll try that change this week and benchmark how much it
          helps memory vs. hurts speed. Thanks!
          Chris








        • James Dempsey
          Hi, After load, the UI can have multiple threads accessing the objects. These are the AWT event thread (used for most character updates) and the HTML output
          Message 4 of 22 , Jan 28, 2013
          • 0 Attachment
            Hi,

            After load, the UI can have multiple threads accessing the objects. These are the AWT event thread (used for most character updates) and the HTML output threads which keep the summary sheet, skills list and character sheet up to date.

            As Tom mentioned, these should not be updating the LST objects much though. Some cloning still remains though, which is where a small number of updates will occur. Tom is gradually eliminating those though.

            Cheers,
            James

            On 29 January 2013 11:41, Tom Parker <thpr@...> wrote:


            Since the map creation should all happen at LST load, the answer *should* be no.

            That being said, I make no guarantees about UI behavior.  I was not involved in that project.  James or Connor might be able to help with what the UI is doing during LST load.

            TP.
            --
            Tom Parker

            Sent: Monday, January 28, 2013 7:32 PM

            Subject: Re: [pcgen_developers] memory usage



            Do I need to worry about synchronization of the lazy instantiation? It looks like most of PCGen is two-threaded: one for loading and the UI thread, and the flow of data is purely one-way. So I suspect the answer is no, synchronization is not important.

            Chris



            On Jan 28, 2013, at 4:53 PM, Tom Parker wrote:




            As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many cases it may save a method call... vs some cases where it's a useless null check... (which is the penalty of lazy instantiation)... but either way, you will be lucky to even be able to measure the difference since the things I'm talking about are a handful of CPU cycles, and not really material in the grand scheme.

            At load time, it will probably be faster, since it will be doing a lot less memory allocation.

            TP. 
            --
            Tom Parker


            From: Chris Dolan <chris@...>
            To: pcgen_developers@yahoogroups.com 
            Sent: Monday, January 28, 2013 5:04 PM
            Subject: Re: [pcgen_developers] memory usage

            Aha, awesome. I'll try that change this week and benchmark how much it
            helps memory vs. hurts speed. Thanks!
            Chris



          • FerretDave
            Greetings, This sounds like it could be of benefit to the core/desktop version, never mind just the mobile viewer project ? Cheers D
            Message 5 of 22 , Jan 30, 2013
            • 0 Attachment
              Greetings,

              This sounds like it could be of benefit to the core/desktop version, never mind just the mobile viewer project ?

              Cheers
              D

              --- In pcgen_developers@yahoogroups.com, Tom Parker wrote:
              >
              >
              >
              > As a note, the speed I was referring to is at runtime [when a character is created/viewed/modified].  Though on second thought that may get faster as in many cases it may save a method call... vs some cases where it's a useless null check... (which is the penalty of lazy instantiation)... but either way, you will be lucky to even be able to measure the difference since the things I'm talking about are a handful of CPU cycles, and not really material in the grand scheme.
              >
              >
              > At load time, it will probably be faster, since it will be doing a lot less memory allocation.
              >
              >
              > TP.
              >
              > --
              > Tom Parker
              >
              >
              >
              > ________________________________
              > From: Chris Dolan
              > To: pcgen_developers@yahoogroups.com
              > Sent: Monday, January 28, 2013 5:04 PM
              > Subject: Re: [pcgen_developers] memory usage
              >
              > Aha, awesome. I'll try that change this week and benchmark how much it
              > helps memory vs. hurts speed. Thanks!
              > Chris
              >
            • Tom Parker
              Absolutely.  Like everything else it s been a matter of picking what I address first.   TP. -- Tom Parker ________________________________ From: FerretDave
              Message 6 of 22 , Jan 30, 2013
              • 0 Attachment
                Absolutely.  Like everything else it's been a matter of picking what I address first.
                 
                TP.
                --
                Tom Parker


                From: FerretDave <ferret.griffin@...>
                To: pcgen_developers@yahoogroups.com
                Sent: Wednesday, January 30, 2013 4:44 AM
                Subject: [pcgen_developers] Re: memory usage

                Greetings,

                This sounds like it could be of benefit to the core/desktop version, never mind just the mobile viewer project ?

                Cheers
                D


              • Henk Slaaf
                Hey all, It would be cool if Jenkins could run a memory benchmark of every commit for a few sample characters to see what effects commits have. Then we could
                Message 7 of 22 , Jan 30, 2013
                • 0 Attachment

                  Hey all,

                  It would be cool if Jenkins could run a memory benchmark of every commit for a few sample characters to see what effects commits have.

                  Then we could graph the results.

                  No idea if this is actually feasible, but just dreaming :-)

                  Best,

                  Henk

                  On Jan 25, 2013 9:25 AM, "Chris Dolan" <chris@...> wrote:
                  As a performance metric, I wrote a simple desktop app that loads a single (hard-coded) Pathfinder PCGen character and renders it with the CharacterSheetPanel. I measured time to load (about 40 seconds) and resulting memory usage (about 85 MB of heap when idle).

                  My test app is at the URL below (less than 200 lines of code): https://github.com/chrisdolan/pcgen-android/blob/acf2de4e874697329246b9378525f7fc4852f344/pcgenuitest/src/net/chrisdolan/pcgen/viewer/uitest/HtmlSheet.java

                  Then I went through the PCGen source looking at all of the Lst loading paths and I added lots of .intern() clauses to the tokenined input to detach them from the original file strings. This saved a notable amount of memory (now 790 MB instead of 85 MB), but much less of an improvement than I hoped. My intern changes are visible on this git branch commit:

                  I've attached a pretty chart of the heap dominators before and after my change, according the Eclipse MAT plugin. I believe that the heap associated with the AppClassLoader is the static fields, like in SystemCollections, but I'm not sure. It's not surprising that Equipment is the #1 user.

                  BEFORE: (note that "char[]" is the 4th biggest, not sure why the HTML classes don't show. Maybe I made a mistake?)
                  AFTER: (note that "char[]" has vanished, AppClassLoader has shrunk, and the others are about the same)


                  Chris

                  P.S. I had an interesting discussion on StackOverflow about String.intern(). Take a look:
                • Tom Parker
                  also... Number of java warnings Processing time per character regression test Load errors/warnings for key dataset combinations Unit test coverage ...probably
                  Message 8 of 22 , Jan 30, 2013
                  • 0 Attachment
                    also...

                    Number of java warnings
                    Processing time per character regression test
                    Load errors/warnings for key dataset combinations
                    Unit test coverage

                    ...probably a lot more, but there are a lot of interesting things we could graph over time if someone was up to the task.  Like many other things, just not my top priority.

                    TP.
                    --
                    Tom Parker


                    From: Henk Slaaf <henk@...>
                    To: pcgen_developers@yahoogroups.com
                    Sent: Wednesday, January 30, 2013 12:58 PM
                    Subject: Re: [pcgen_developers] memory usage



                    Hey all,
                    It would be cool if Jenkins could run a memory benchmark of every commit for a few sample characters to see what effects commits have.
                    Then we could graph the results.
                    No idea if this is actually feasible, but just dreaming :-)
                    Best,
                    Henk
                    On Jan 25, 2013 9:25 AM, "Chris Dolan" <chris@...> wrote:
                    As a performance metric, I wrote a simple desktop app that loads a single (hard-coded) Pathfinder PCGen character and renders it with the CharacterSheetPanel. I measured time to load (about 40 seconds) and resulting memory usage (about 85 MB of heap when idle).

                    My test app is at the URL below (less than 200 lines of code): https://github.com/chrisdolan/pcgen-android/blob/acf2de4e874697329246b9378525f7fc4852f344/pcgenuitest/src/net/chrisdolan/pcgen/viewer/uitest/HtmlSheet.java

                    Then I went through the PCGen source looking at all of the Lst loading paths and I added lots of .intern() clauses to the tokenined input to detach them from the original file strings. This saved a notable amount of memory (now 790 MB instead of 85 MB), but much less of an improvement than I hoped. My intern changes are visible on this git branch commit:

                    I've attached a pretty chart of the heap dominators before and after my change, according the Eclipse MAT plugin. I believe that the heap associated with the AppClassLoader is the static fields, like in SystemCollections, but I'm not sure. It's not surprising that Equipment is the #1 user.

                    BEFORE: (note that "char[]" is the 4th biggest, not sure why the HTML classes don't show. Maybe I made a mistake?)
                    AFTER: (note that "char[]" has vanished, AppClassLoader has shrunk, and the others are about the same)


                    Chris

                    P.S. I had an interesting discussion on StackOverflow about String.intern(). Take a look:
                    http://stackoverflow.com/questions/14516635/how-do-i-reclaim-memory-after-parsing-via-substrings-intern-or-new-string




                  • Chris Dolan
                    For most of those genetic metrics, I recommend Sonar. It needs maven pom.xml but otherwise is really easy to set up. Chris ... For most of those genetic
                    Message 9 of 22 , Jan 30, 2013
                    • 0 Attachment
                      For most of those genetic metrics, I recommend Sonar. It needs maven pom.xml but otherwise is really easy to set up.
                      Chris

                      Tom Parker <thpr@...> wrote:
                      also...

                      Number of java warnings
                      Processing time per character regression test
                      Load errors/warnings for key dataset combinations
                      Unit test coverage

                      ...probably a lot more, but there are a lot of interesting things we could graph over time if someone was up to the task.  Like many other things, just not my top priority.

                      TP.
                      --
                      Tom Parker


                      From: Henk Slaaf <henk@...>
                      To: pcgen_developers@yahoogroups.com
                      Sent: Wednesday, January 30, 2013 12:58 PM
                      Subject: Re: [pcgen_developers] memory usage



                      Hey all,
                      It would be cool if Jenkins could run a memory benchmark of every commit for a few sample characters to see what effects commits have.
                      Then we could graph the results.
                      No idea if this is actually feasible, but just dreaming :-)
                      Best,
                      Henk
                      On Jan 25, 2013 9:25 AM, "Chris Dolan" <chris@...> wrote:
                      As a performance metric, I wrote a simple desktop app that loads a single (hard-coded) Pathfinder PCGen character and renders it with the CharacterSheetPanel. I measured time to load (about 40 seconds) and resulting memory usage (about 85 MB of heap when idle).

                      My test app is at the URL below (less than 200 lines of code): https://github.com/chrisdolan/pcgen-android/blob/acf2de4e874697329246b9378525f7fc4852f344/pcgenuitest/src/net/chrisdolan/pcgen/viewer/uitest/HtmlSheet.java

                      Then I went through the PCGen source looking at all of the Lst loading paths and I added lots of .intern() clauses to the tokenined input to detach them from the original file strings. This saved a notable amount of memory (now 790 MB instead of 85 MB), but much less of an improvement than I hoped. My intern changes are visible on this git branch commit:

                      I've attached a pretty chart of the heap dominators before and after my change, according the Eclipse MAT plugin. I believe that the heap associated with the AppClassLoader is the static fields, like in SystemCollections, but I'm not sure. It's not surprising that Equipment is the #1 user.

                      BEFORE: (note that "char[]" is the 4th biggest, not sure why the HTML classes don't show. Maybe I made a mistake?)
                      AFTER: (note that "char[]" has vanished, AppClassLoader has shrunk, and the others are about the same)


                      Chris

                      P.S. I had an interesting discussion on StackOverflow about String.intern(). Take a look:
                      http://stackoverflow.com/questions/14516635/how-do-i-reclaim-memory-after-parsing-via-substrings-intern-or-new-string




                    Your message has been successfully submitted and would be delivered to recipients shortly.