Loading ...
Sorry, an error occurred while loading the content.

Performance problem large return values- best way to approach?

Expand Messages
  • Steve McCarthy
    Hello, We are using SOAP::Lite 0.52 (as CGI) to access one of our methods which returns a text string. This text string happens to contain XML, but I suppose
    Message 1 of 3 , Oct 29, 2002
    • 0 Attachment
      Hello,

      We are using SOAP::Lite 0.52 (as CGI) to access one of our methods
      which returns a text string. This text string happens to contain XML,
      but I suppose that it could be anything.

      We are noticing that when this string gets to be greater than 400K in
      size, performance begins to get very slow, and we notice high memory
      and CPU utilization on the server side. As the string size grows,
      performance gets disproportionately worse. From timings and print
      statements, we can see that the time is being spent somewhere inside
      SOAP::Lite after our method returns.

      What are some strategies that we could try to get the perfomance to a
      more reasonable level?

      I have seen some posts here on the list about writing custom
      Serializers/Deserializers, though I am not sure whether that would buy
      me anything for an XML payload.

      Other posts mention that XML parsing is the bottleneck. Are there any
      settings I can change to tell SOAP::Lite to just "trust me" and treat
      the return string as an opaque blob, and bypass XML parsing step?

      Background: we are using SOAP::Lite in just about the most basic way.
      For example, here is what our server-side code looks like:

      SOAP::Transport::HTTP::CGI
      -> dispatch_to('API::pkg1', 'API::pkg2', 'API::pkg3')
      -> handle;

      Any hints greatly appreciated,

      -Steve
    • Paul Kulchenko
      Hi Steve, ... Actually, as funny it may sound, you ll get much better results if it will be something else. If it s XML, then it s probably escaped using <,
      Message 2 of 3 , Oct 29, 2002
      • 0 Attachment
        Hi Steve,

        > We are using SOAP::Lite 0.52 (as CGI) to access one of our methods
        > which returns a text string. This text string happens to contain
        > XML, but I suppose that it could be anything.
        Actually, as funny it may sound, you'll get much better results if it
        will be something else. If it's XML, then it's probably escaped using
        <, & and the rest and the way XML::Parser works is that it
        will issue callback for every encoded entity, thus generating the
        massive number of callback. The logic in SOAP::Parser module will
        ocncatenate those in one element and for big strings that part
        consumes significant memory and CPU resources.

        You may consider v0.55 which uses arrays instead of string
        concatenation; it may significantly boost the performance in this
        particular case (the security patch is the additional reason to
        install v0.55). You may also use base64 encoding as suggested in the
        SOAP::Lite documentation
        (http://theoryx5.uwinnipeg.ca/CPAN/data/SOAP-Lite/SOAP/Lite.html#PERFORMANCE).
        I would love to hear about the difference (we will think about
        something else if it's not enough).

        Best wishes, Paul.

        --- Steve McCarthy <mccst04@...> wrote:
        > We are using SOAP::Lite 0.52 (as CGI) to access one of our methods
        > which returns a text string. This text string happens to contain
        > XML,
        > but I suppose that it could be anything.
        >
        > We are noticing that when this string gets to be greater than 400K
        > in
        > size, performance begins to get very slow, and we notice high
        > memory
        > and CPU utilization on the server side. As the string size grows,
        > performance gets disproportionately worse. From timings and print
        > statements, we can see that the time is being spent somewhere
        > inside
        > SOAP::Lite after our method returns.
        >
        > What are some strategies that we could try to get the perfomance to
        > a
        > more reasonable level?
        >
        > I have seen some posts here on the list about writing custom
        > Serializers/Deserializers, though I am not sure whether that would
        > buy
        > me anything for an XML payload.
        >
        > Other posts mention that XML parsing is the bottleneck. Are there
        > any
        > settings I can change to tell SOAP::Lite to just "trust me" and
        > treat
        > the return string as an opaque blob, and bypass XML parsing step?
        >
        > Background: we are using SOAP::Lite in just about the most basic
        > way.
        > For example, here is what our server-side code looks like:
        >
        > SOAP::Transport::HTTP::CGI
        > -> dispatch_to('API::pkg1', 'API::pkg2', 'API::pkg3')
        > -> handle;
        >
        > Any hints greatly appreciated,
        >
        > -Steve
        >
        >
        > ------------------------ Yahoo! Groups Sponsor
        >
        > To unsubscribe from this group, send an email to:
        > soaplite-unsubscribe@yahoogroups.com
        >
        >
        >
        > Your use of Yahoo! Groups is subject to
        > http://docs.yahoo.com/info/terms/
        >
        >


        __________________________________________________
        Do you Yahoo!?
        HotJobs - Search new jobs daily now
        http://hotjobs.yahoo.com/
      • Steve McCarthy
        It looks like I misspoke earlier... It turns out that our production environment is running version 0.51, and we tracked the immediate
        Message 3 of 3 , Oct 29, 2002
        • 0 Attachment
          <groan> It looks like I misspoke earlier... </groan>

          It turns out that our production environment is running version 0.51,
          and we tracked the immediate performance problem down to
          SOAP::Utils::encode_data(), which has been changed in 0.52 to
          something which appears to be much more efficient for large strings.
          The substitution operator had the 'e' expression flag (i.e.
          s/xxx/yyy/e) and the newer version is more of a straight substitution.
          What was taking 60 seconds is now down to about 1 second.

          Will be looking at 0.55 for our next release.

          Thanks for quick reply,
          -Steve


          --- In soaplite@y..., Paul Kulchenko <paulclinger@y...> wrote:
          > Hi Steve,
          >
          > > We are using SOAP::Lite 0.52 (as CGI) to access one of our methods
          > > which returns a text string. This text string happens to contain
          > > XML, but I suppose that it could be anything.
          > Actually, as funny it may sound, you'll get much better results if
          it
          > will be something else. If it's XML, then it's probably escaped
          using
          > <, & and the rest and the way XML::Parser works is that it
          > will issue callback for every encoded entity, thus generating the
          > massive number of callback. The logic in SOAP::Parser module will
          > ocncatenate those in one element and for big strings that part
          > consumes significant memory and CPU resources.
          >
          > You may consider v0.55 which uses arrays instead of string
          > concatenation; it may significantly boost the performance in this
          > particular case (the security patch is the additional reason to
          > install v0.55). You may also use base64 encoding as suggested in the
          > SOAP::Lite documentation
          >
          (http://theoryx5.uwinnipeg.ca/CPAN/data/SOAP-Lite/
          SOAP/Lite.html#PERFORMANCE).
          > I would love to hear about the difference (we will think about
          > something else if it's not enough).
          >
          > Best wishes, Paul.
          >
          > --- Steve McCarthy <mccst04@y...> wrote:
          > > We are using SOAP::Lite 0.52 (as CGI) to access one of our methods
          > > which returns a text string. This text string happens to contain
          > > XML,
          > > but I suppose that it could be anything.
          > >
          > > We are noticing that when this string gets to be greater than 400K
          > > in
          > > size, performance begins to get very slow, and we notice high
          > > memory
          > > and CPU utilization on the server side. As the string size grows,
          > > performance gets disproportionately worse. From timings and print
          > > statements, we can see that the time is being spent somewhere
          > > inside
          > > SOAP::Lite after our method returns.
          > >
          > > What are some strategies that we could try to get the perfomance
          to
          > > a
          > > more reasonable level?
          > >
          > > I have seen some posts here on the list about writing custom
          > > Serializers/Deserializers, though I am not sure whether that would
          > > buy
          > > me anything for an XML payload.
          > >
          > > Other posts mention that XML parsing is the bottleneck. Are there
          > > any
          > > settings I can change to tell SOAP::Lite to just "trust me" and
          > > treat
          > > the return string as an opaque blob, and bypass XML parsing step?
          > >
          > > Background: we are using SOAP::Lite in just about the most basic
          > > way.
          > > For example, here is what our server-side code looks like:
          > >
          > > SOAP::Transport::HTTP::CGI
          > > -> dispatch_to('API::pkg1', 'API::pkg2', 'API::pkg3')
          > > -> handle;
          > >
          > > Any hints greatly appreciated,
          > >
          > > -Steve
          > >
          > >
          > > ------------------------ Yahoo! Groups Sponsor
          > >
          > > To unsubscribe from this group, send an email to:
          > > soaplite-unsubscribe@y...
          > >
          > >
          > >
          > > Your use of Yahoo! Groups is subject to
          > > http://docs.yahoo.com/info/terms/
          > >
          > >
          >
          >
          > __________________________________________________
          > Do you Yahoo!?
          > HotJobs - Search new jobs daily now
          > http://hotjobs.yahoo.com/
        Your message has been successfully submitted and would be delivered to recipients shortly.