Loading ...
Sorry, an error occurred while loading the content.
 

Java Object Serialization memory problems

Expand Messages
  • Brendan Macmillan
    Hi Craig, I m the author of JSX, I ve just approved your memship to the mailing list. ... of memory problems during serialization, and wondered whether XML
    Message 1 of 2 , Jul 11, 2004
      Hi Craig,

      I'm the author of JSX, I've just approved your memship to the mailing list.
      I'm answering your comment directly:

      > I'm writing a distributed artificial life program. I have been having out
      of memory problems during serialization, and wondered whether XML might
      provide a less memory intense solution.

      Probably, JSX is going to be the same, as it uses the same mechanisms.


      However, with XML you can see what is happening, and hopefully detect where
      all that memory is going. You can probably write writeObject() and
      readObject() methods in the problematic class to fix it - or use a memento.
      For example, the java.util.HashMap has a huge array for the hashtable, much
      of which is empty - so that table is transient, and only the actual entries
      are serialized.


      You can test it out JSX very easily - download, add to classpath, change
      calls to *constructors*:
      new ObjectOutputWriter(xxx) to new JSX.ObjectWriter(xxx)
      new ObjectInputReader(xxx) to new JSX.ObjectReader(xxx)

      You *only* need to change the constructor (the class of the variable holding
      the object can stay the same etc).

      See http://www.jsx.org/download.html for all the details.



      HTH

      Cheers,
      Brendan
    • crgmcc
      Thanks, I will look into it. I am an academic/hobbyist programmer, with too little time to research the matter properly, but it seems to me that too little
      Message 2 of 2 , Jul 11, 2004
        Thanks,
        I will look into it. I am an academic/hobbyist programmer, with too little time to
        research the matter properly, but it seems to me that too little attention has been
        paid to memory consumption in the default java serialization methods. I have made
        some progress by serializing some large child components within memory and then
        compressing them (also in memory) before finally serializing the parent object but
        that method is slow. It would be nice to serialize and write small sections of my large
        objects to the hard disc one section at a time, instead of facing a huge memory
        blowout all at once. Maybe JSX will enable me to do some sort of sequential
        serialization; I'm not sure. I could, of course, write some of my sub-objects to
        separate files, one at a time, and then recombine them manually but I was not
        wanting to write a lot of my own code to do it, and wanted everything saved in a
        single file.
        If I find a nice solution I will let you know.
        Cheers,
        Craig.
        PS. To give you an idea of the scale of the problem, the final file (even with some
        compression) is about 200 megabytes on the hard disc.
      Your message has been successfully submitted and would be delivered to recipients shortly.