Loading ...
Sorry, an error occurred while loading the content.

1988Re: [soaplite] Using SOAP::Lite and C++, memory consumption problem

Expand Messages
  • Paul Kulchenko
    Oct 29, 2002
    • 0 Attachment
      Hi Kimmo,

      > I have got the impression that Perl does release memory but it is
      > released to Perl's own memory pool, not to the operating system.

      I believe that's the correct assumption:

      "The way Perl handles memory is very simple. When it needs to
      allocate memory it will allocate everything it needs to perform it's
      tasks. For example, if a variable's content is 1MB, then Perl will
      allocate 1MB for the variable. When that variable goes out of scope
      or is catched by the garbage fairy, Perl will not release that 1MB to
      the kernel right away, it will reuse it in case it needs it later on
      during execution."

      Unless there is a memory leak somewhere in my or Perl's code, this
      memory pool should stop growing after some number of requests. I've
      done extensive tests on my windows and linux boxes chasing down
      memory leaks and tests showed that pool stops growing.

      > 1. Find some mysterious Perl trick which forces Perl to release
      > memory from
      > it's memory pool or makes Perl reuse memory in it's pool more
      > efficiently.
      > - I haven't found even if I have searched intensively
      It may exist, but I'm unaware of any such option.

      > 2. Re-compile Perl with usemultiplicity-option and create an
      > instance of the
      > Perl interpreter for each time a SOAP message is sent.
      > - I don't know the possible consequences and I'm not the only one
      > to use Perl. Others might not agree
      Will probably work, but may have performance impact. Can't say ho
      big.

      > 3. Shutdown and restart the persistent processes which use my
      > library. For example a script is run every Saturday night.
      > - Sounds quite quick and dirty
      It may be better than #2. You may think about adding file semaphore
      and graceful shutdown to your code.

      > 4. Find the bug in my code.
      > - I have tried to follow the instructions in Perlcall and Perlembed
      > documents but You never know
      From what you describe I don't there is any in memory handling piece
      (In other words, I would expect similar behavior from properly
      written code).

      5. it *may* be possible to restrict the amount of memory available
      for your process in your OS. Although the amount of required memory
      depends on the size of the message you want to process/generate, so
      you may not be able to handle all of the messages.

      Unfortunately I can't be more helpful on that. Some other options
      are: (1) reduce the SOAP::Lite's memory footprint, (2) provide
      streaming parsing as an option, (3) do something else. #1 and 2 are
      on my list, but they both only experiments for now and it's not clear
      when they will make into the code.

      Best wishes, Paul.

      --- "Andersson, Kimmo" <Kimmo.Andersson@...> wrote:
      > Hi !
      >
      > I have already posted this message to a Perl newsgroup but as
      > SOAP::Lite is
      > implemented with Perl I thought why not also to this mailing list.
      > This is more a Perl problem, not about SOAP::Lite as such. But I'
      > hoping
      > that some one might have met a similar problem and even solved it.
      >
      > I have written a library in C++ which acts as an wrapper for
      > sending SOAP
      > messages. It is done in HP-UX environment and the SOAP::Lite
      > toolkit is
      > used
      >
      > As SOAP::Lite is implemented with Perl I have written my own Perl
      > module
      > which acts as an intermediary between my C++ code and SOAP::Lite.
      > My own
      > Perl module calls SOAP::Lite and returns the SOAP response to my
      > C++ class
      > which has called it.
      >
      > Unfortunately it has been discovered that the processes that use my
      > library
      > seem consume more and more memory while running and several SOAP
      > messages
      > are sent via my library. This has been verified using several
      > processes and
      > I have also verified it using my own test client, we are absolutely
      > sure
      > that SOAP::Lite and Perl are to be blamed.
      >
      > A clear pattern hasn't however emerged, it is not that say 25 K per
      > each
      > SOAP message memory is used and not released, it varies.
      > Some time memory decreases and several SOAP messages are sent
      > before memory
      > again starts growing. I have used Purify but Purify does not
      > report memory
      > leaks. Actually there leaks from some system libraries but their
      > amount
      > remains constant whether I send one or hundred SOAP messages.
      >
      > Because the Perl we use is not compiled with
      > usemultiplicity-option, the
      > persistent processes which use my library must use just one
      > instance of a
      > class in my library and via it just one instance of the Perl
      > interpreter
      > during their lifetime.
      >
      > I have got the impression that Perl does release memory but it is
      > released
      > to Perl's own memory pool, not to the operating system.
      >
      > So I'm asking is there anything to be done ? I have just one
      > subroutine in
      > my Perl module and all the variables used are lexical variables
      > (my)
      > declared in that subroutine. They should be released when gone out
      > of scope.
      > In my C++ code I use perl_call_argv to call my Perl module and it
      > and
      > reading the Perl stack are surrounded by ENTER/SAVETMPS and
      > FREETMPS/LEAVE.
      > According to the Perlcall document that should get rid of any
      > values
      > returned by the Perl module.
      >
      > So possible solutions ?
      >
      > 1. Find some mysterious Perl trick which forces Perl to release
      > memory from
      > it's memory pool or makes Perl reuse memory in it's pool more
      > efficiently.
      > - I haven't found even if I have searched intensively
      >
      > 2. Re-compile Perl with usemultiplicity-option and create an
      > instance of the
      > Perl interpreter for each time a SOAP message is sent.
      > - I don't know the possible consequences and I'm not the only one
      > to use
      > Perl. Others might not agree
      >
      > 3. Shutdown and restart the persistent processes which use my
      > library. For
      > example a script is run every Saturday night.
      > - Sounds quite quick and dirty
      >
      > 4. Find the bug in my code.
      > - I have tried to follow the instructions in Perlcall and Perlembed
      > documents but You never know
      >
      >
      > Kimmo Andersson
      >
      > Yomi Solutions Ltd E-mail: Kimmo.Andersson@....O
      > Box 354, Piippukatu 11 Mobile: +358 50 384 3734
      > FIN-40101 JYV�SKYL� Fax : +358 14 240 4709FINLAND
      >
      >
      > ------------------------ Yahoo! Groups Sponsor
      >
      > To unsubscribe from this group, send an email to:
      > soaplite-unsubscribe@yahoogroups.com
      >
      >
      >
      > Your use of Yahoo! Groups is subject to
      > http://docs.yahoo.com/info/terms/
      >
      >


      __________________________________________________
      Do you Yahoo!?
      HotJobs - Search new jobs daily now
      http://hotjobs.yahoo.com/
    • Show all 2 messages in this topic