Loading ...
Sorry, an error occurred while loading the content.

Large XML

Expand Messages
  • Kyle Ellrott
    I ve been upgrading some of my servers, and as a result, ended up with the new version of SOAP::Lite on my system. One of my CGI scripts uses SOAP::Lite to
    Message 1 of 3 , Nov 9, 2004
    View Source
    • 0 Attachment
      I've been upgrading some of my servers, and as a result, ended up with
      the new version of SOAP::Lite on my system.
      One of my CGI scripts uses SOAP::Lite to package together some result
      files, and send them back to the system that requested them. The call
      is :
      return SOAP::Data->name("psiblast_results" => $str);
      Where $str contains the contents of the file. This file is XML. And
      it's 'very' large, about 6.9M. It's XML formated blast output.

      Previously (with older SOAP, I forget the version, but something like
      0.51), this worked just fine. It would insert the text without playing
      with it, and send it along. But it seems that SOAP::Lite 0.60 plays
      with the text, trying to make it XML compatible, mostly by turning all
      the '<' into '<'
      The problem is that how ever it is doing it, it is real slow, and it
      takes up a lot of memory. Essentially the soap.cgi script tries to
      allocate almost 200M of RAM and then the connection times out because
      it takes so long processes it.

      Is there way to get SOAP::Lite to stop messing around with the text and
      just stick it in and send it along?

      Thanks,

      Kyle
    • Duncan Cameron
      ... Hi Kyle I m a bit surprised that you think that the earlier version of SOAP::Lite didn t encode the data, but given no other information that is what
      Message 2 of 3 , Nov 9, 2004
      View Source
      • 0 Attachment
        On 2004-11-09 at 20:54:47 Kyle Ellrott wrote:

        >I've been upgrading some of my servers, and as a result, ended up with
        >the new version of SOAP::Lite on my system.
        >One of my CGI scripts uses SOAP::Lite to package together some result
        >files, and send them back to the system that requested them. The call
        >is :
        > return SOAP::Data->name("psiblast_results" => $str);
        >Where $str contains the contents of the file. This file is XML. And
        >it's 'very' large, about 6.9M. It's XML formated blast output.
        >
        >Previously (with older SOAP, I forget the version, but something like
        >0.51), this worked just fine. It would insert the text without
        >playing
        >with it, and send it along. But it seems that SOAP::Lite 0.60 plays
        >with the text, trying to make it XML compatible, mostly by turning all
        >the '<' into '<'
        >The problem is that how ever it is doing it, it is real slow, and it
        >takes up a lot of memory. Essentially the soap.cgi script tries to
        >allocate almost 200M of RAM and then the connection times out because
        >it takes so long processes it.
        >
        >Is there way to get SOAP::Lite to stop messing around with the text
        >and
        >just stick it in and send it along?
        >
        >Thanks,
        >
        Hi Kyle

        I'm a bit surprised that you think that the earlier version of
        SOAP::Lite didn't encode the data, but given no other information that
        is what SOAP::Lite has to do. You can tell SOAP::Lite that the string
        is XML by using the type() method, and then it will not be encoded

        return SOAP::Data->name("psiblast_results" => $str)->type('xml');

        Note that it is your responsibility to ensure that the string is well-
        formed XML.

        You might also want to look at returning the files as attachments. It
        is pretty easy using the support in the latest SOAP::Lite, 0.65, but
        may also depend on what language your clients are developed in.

        Duncan
      • Kyle Ellrott
        ... Previously, according to debug output, it would simply pass back my XML without changing the
        Message 3 of 3 , Nov 9, 2004
        View Source
        • 0 Attachment
          > I'm a bit surprised that you think that the earlier version of
          > SOAP::Lite didn't encode the data,

          Previously, according to debug output, it would simply pass back my XML
          without changing the '<' s to '<'s.
          That's really beside the point ;-) It used to work, and after the
          update, it didn't. (looks like it's working now, but I still have
          another question, see below)

          > return SOAP::Data->name("psiblast_results" => $str)->type('xml');

          Thank you, that little gem helps. Back up and almost running.
          But I do still have one concern, and mainly it is that the cgi script
          allocates close to 450M while sending the data back. Thanks to the
          type('xml') it doesn't get locked up now, and only takes a few seconds
          to complete the transfer. But this memory issue could hammer me bad if
          I get several requests simultaneously.
          Any thoughts? Would moving them to be attachments bring down the
          memory size?

          Kyle
        Your message has been successfully submitted and would be delivered to recipients shortly.