Loading ...
Sorry, an error occurred while loading the content.

memory leak

Expand Messages
  • Orlando Andico
    i have a SOAP::Lite perl client which makes a WS function call repeatedly (about 5X per second, with differing parameters). my problem is that it leaks memory
    Message 1 of 6 , Feb 20, 2006
    • 0 Attachment
      i have a SOAP::Lite perl client which makes a WS function call
      repeatedly (about 5X per second, with differing parameters).

      my problem is that it leaks memory rather spectacularly: after 1200
      function calls, the footprint has grown from 22MB to 88MB! that's
      about 37KB of leakage per SOAP call!

      my "solution" for now is for the daemon to restart every few hundred
      SOAP calls, but i was wondering if there's a more elegant solution.
    • Ollie Cook
      ... I am seeing similar behaviour and have tracked it down to an ever-growing { _serializer }- { _seen } object. I will debug this further tomorrow morning,
      Message 2 of 6 , Mar 13, 2006
      • 0 Attachment
        On Mon, 2006-02-20 at 11:31 +0000, Orlando Andico wrote:
        > i have a SOAP::Lite perl client which makes a WS function call
        > repeatedly (about 5X per second, with differing parameters).
        >
        > my problem is that it leaks memory rather spectacularly: after 1200
        > function calls, the footprint has grown from 22MB to 88MB! that's
        > about 37KB of leakage per SOAP call!
        >
        > my "solution" for now is for the daemon to restart every few hundred
        > SOAP calls, but i was wondering if there's a more elegant solution.

        I am seeing similar behaviour and have tracked it down to an
        ever-growing {'_serializer'}->{'_seen'} object.

        I will debug this further tomorrow morning, but in the meantime I wonder
        if the SOAP::Lite developers can shed any light on the purpose and use
        of this object?

        Ollie
        --
        Ollie Cook
        UNIX Scripter
        Information Systems

        Direct Line: 020 8834 8128
        Waterfront | Hammersmith Embankment | Chancellors Road London | W6 9HP
      • Byrne Reese
        That object is to track whether an object has been parsed or not to avoid circular/infinite loops in processes href references. Make sense?
        Message 3 of 6 , Mar 13, 2006
        • 0 Attachment
          That object is to track whether an object has been parsed or not to
          avoid circular/infinite loops in processes href references. Make sense?

          Ollie Cook wrote:
          > On Mon, 2006-02-20 at 11:31 +0000, Orlando Andico wrote:
          > > i have a SOAP::Lite perl client which makes a WS function call
          > > repeatedly (about 5X per second, with differing parameters).
          > >
          > > my problem is that it leaks memory rather spectacularly: after 1200
          > > function calls, the footprint has grown from 22MB to 88MB! that's
          > > about 37KB of leakage per SOAP call!
          > >
          > > my "solution" for now is for the daemon to restart every few hundred
          > > SOAP calls, but i was wondering if there's a more elegant solution.
          >
          > I am seeing similar behaviour and have tracked it down to an
          > ever-growing {'_serializer'}->{'_seen'} object.
          >
          > I will debug this further tomorrow morning, but in the meantime I wonder
          > if the SOAP::Lite developers can shed any light on the purpose and use
          > of this object?
          >
          > Ollie
          > --
          > Ollie Cook
          > UNIX Scripter
          > Information Systems
          >
          > Direct Line: 020 8834 8128
          > Waterfront | Hammersmith Embankment | Chancellors Road London | W6 9HP
          >
          >
          > SPONSORED LINKS
          > Protocol analyzer
          > <http://groups.yahoo.com/gads?t=ms&k=Protocol+analyzer&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=-vNi0F9sGxz3wJiUypOlsA>
          > Ssl protocol
          > <http://groups.yahoo.com/gads?t=ms&k=Ssl+protocol&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=QcUDWIbIdXkmz4QSFhqszw>
          > Protocol converter
          > <http://groups.yahoo.com/gads?t=ms&k=Protocol+converter&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=8MMcEvfzdk5saYSJv-2g3g>
          >
          > Sip protocol
          > <http://groups.yahoo.com/gads?t=ms&k=Sip+protocol&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=X7aZ6IACnEQUtAC57u829w>
          > Protocol analysis
          > <http://groups.yahoo.com/gads?t=ms&k=Protocol+analysis&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=SI1LEHwb69y6gddZAv_Fuw>
          > Protocol
          > <http://groups.yahoo.com/gads?t=ms&k=Protocol&w1=Protocol+analyzer&w2=Ssl+protocol&w3=Protocol+converter&w4=Sip+protocol&w5=Protocol+analysis&w6=Protocol&c=6&s=120&.sig=aCiMFP0b3kFn96wqrZacGA>
          >
          >
          >
          > ------------------------------------------------------------------------
          > YAHOO! GROUPS LINKS
          >
          > * Visit your group "soaplite
          > <http://groups.yahoo.com/group/soaplite>" on the web.
          >
          > * To unsubscribe from this group, send an email to:
          > soaplite-unsubscribe@yahoogroups.com
          > <mailto:soaplite-unsubscribe@yahoogroups.com?subject=Unsubscribe>
          >
          > * Your use of Yahoo! Groups is subject to the Yahoo! Terms of
          > Service <http://docs.yahoo.com/info/terms/>.
          >
          >
          > ------------------------------------------------------------------------
          >
        • Ollie Cook
          ... Should the hash be cleared, then, after each successful parse of an XML response? At the moment it is growing between requests. If the XML returned is
          Message 4 of 6 , Mar 13, 2006
          • 0 Attachment
            On Mon, 2006-03-13 at 12:58 -0800, Byrne Reese wrote:
            > That object is to track whether an object has been parsed or not to
            > avoid circular/infinite loops in processes href references. Make sense?

            Should the hash be cleared, then, after each successful parse of an XML
            response?

            At the moment it is growing between requests. If the XML returned is
            sufficiently different on each iteration, the memory consumption grows
            quite rapidly.

            Ollie
            --
            Ollie Cook
            UNIX Scripter
            Information Systems

            Direct Line: 020 8834 8128
            Waterfront | Hammersmith Embankment | Chancellors Road London | W6 9HP
          • Byrne Reese
            Yes. Yes it should. Damn. This could be the root cause of this long known issue. Wow.
            Message 5 of 6 , Mar 13, 2006
            • 0 Attachment
              Yes. Yes it should.

              Damn.

              This could be the root cause of this long known issue. Wow.

              Ollie Cook wrote:
              > On Mon, 2006-03-13 at 12:58 -0800, Byrne Reese wrote:
              >
              >> That object is to track whether an object has been parsed or not to
              >> avoid circular/infinite loops in processes href references. Make sense?
              >>
              >
              > Should the hash be cleared, then, after each successful parse of an XML
              > response?
              >
              > At the moment it is growing between requests. If the XML returned is
              > sufficiently different on each iteration, the memory consumption grows
              > quite rapidly.
              >
              > Ollie
              >
            • Ollie Cook
              ... Having instrumented this further today, I can say that the problem is not in SOAP::Lite but rather XML::SAX (which I use through XML::Simple) as a
              Message 6 of 6 , Mar 14, 2006
              • 0 Attachment
                On Mon, 2006-03-13 at 21:00 +0000, Ollie Cook wrote:
                > On Mon, 2006-03-13 at 12:58 -0800, Byrne Reese wrote:
                > > That object is to track whether an object has been parsed or not to
                > > avoid circular/infinite loops in processes href references. Make sense?
                >
                > Should the hash be cleared, then, after each successful parse of an XML
                > response?
                >
                > At the moment it is growing between requests. If the XML returned is
                > sufficiently different on each iteration, the memory consumption grows
                > quite rapidly.
                >
                > Ollie

                Having instrumented this further today, I can say that the problem is
                not in SOAP::Lite but rather XML::SAX (which I use through XML::Simple)
                as a deserializer. If I switch to using XML::Parser (my XML is generally
                quite small, so the trade-off is OK) rather than XML::SAX the resident
                size of the process does not grow.

                Byrne, you're off the hook! :)

                Ollie

                --
                Ollie Cook
                UNIX Scripter
                Information Systems

                Direct Line: 020 8834 8128
                Waterfront | Hammersmith Embankment | Chancellors Road London | W6 9HP
              Your message has been successfully submitted and would be delivered to recipients shortly.