Loading ...
Sorry, an error occurred while loading the content.

Asynchronous messaging

Expand Messages
  • ryan_gammon
    Hi folks, I m trying to use SOAP::Lite for asynchronous messaging (ie - I want to send a message, and have the client side return right away). At the moment,
    Message 1 of 7 , Oct 23, 2002
    • 0 Attachment
      Hi folks,

      I'm trying to use SOAP::Lite for asynchronous messaging (ie - I want
      to send a message, and have the client side return right away). At the
      moment, I've accomplished this by using the TCP transport, and
      changing the handler function to close the connection before handling
      the request (details at the end of this message)

      I feel this is a bit of a hack, and I'm trying to figure out how I'd
      like this to work.

      If SOAP::Lite were modified to...
      1. Look at the method being called
      2. See if it returns a value
      3. If it does, handle it as it does currently
      4. If it doesn't, make the client return (close the connection in the
      case of TCP) and then handle it

      ... would this be considered appropriate behavior in the soap world?
      Would there be any interest in having this functionality in
      SOAP::Lite?

      I'm a little hazy on the syntax that would be involved in steps 1 and
      2. If all else fails, I suppose an api could be provided to mark
      certain functions as not returning values from the point of view of
      SOAP::Lite.

      Current hack for TCP transport asynchronous messages:
      Added to server:
      ----
      package SOAP::Transport::TCP::Server;

      sub handle {
      my $self = shift->new;
      my $sock = $self->{_socket};
      my $session_set = IO::SessionSet->new($sock);
      my %data;
      while (1) {
      my @ready = $session_set->wait($sock->timeout);
      for my $session (@ready) {
      my $data;
      if (my $rc = $session->read($data, 4096)) {
      $data{$session} .= $data if $rc > 0;
      } else {
      $session->close;
      $self->SUPER::handle(delete $data{$session});
      }
      }
      }
      }

      package main;
    • Keanan Smith
      Would it work to have your SOAP service simply fork off a new process to perform your asynchronous stuff, and then return? a la: package Soapservices; sub
      Message 2 of 7 , Oct 23, 2002
      • 0 Attachment
        Would it work to have your SOAP service simply fork off a new process to
        perform your asynchronous stuff, and then return?
        a la:

        package Soapservices;
        sub asynch
        {
        if (fork)
        {
        ... Do stuff that we don't want to wait for...
        exit();
        }
        }

        or would the child process become completely confused?
        (The 'exit();' is just a guess as to how to defuse the child processess's
        attempt to return data through SOAP, maybe it's not needed, or should be
        something else?)

        -----Original Message-----
        From: ryan_gammon [mailto:rggammon@...]
        Sent: Wednesday, October 23, 2002 12:43 PM
        To: soaplite@yahoogroups.com
        Subject: [soaplite] Asynchronous messaging


        Hi folks,

        I'm trying to use SOAP::Lite for asynchronous messaging (ie - I want
        to send a message, and have the client side return right away). At the
        moment, I've accomplished this by using the TCP transport, and
        changing the handler function to close the connection before handling
        the request (details at the end of this message)

        I feel this is a bit of a hack, and I'm trying to figure out how I'd
        like this to work.

        If SOAP::Lite were modified to...
        1. Look at the method being called
        2. See if it returns a value
        3. If it does, handle it as it does currently
        4. If it doesn't, make the client return (close the connection in the
        case of TCP) and then handle it

        ... would this be considered appropriate behavior in the soap world?
        Would there be any interest in having this functionality in
        SOAP::Lite?

        I'm a little hazy on the syntax that would be involved in steps 1 and
        2. If all else fails, I suppose an api could be provided to mark
        certain functions as not returning values from the point of view of
        SOAP::Lite.

        Current hack for TCP transport asynchronous messages:
        Added to server:
        ----
        package SOAP::Transport::TCP::Server;

        sub handle {
        my $self = shift->new;
        my $sock = $self->{_socket};
        my $session_set = IO::SessionSet->new($sock);
        my %data;
        while (1) {
        my @ready = $session_set->wait($sock->timeout);
        for my $session (@ready) {
        my $data;
        if (my $rc = $session->read($data, 4096)) {
        $data{$session} .= $data if $rc > 0;
        } else {
        $session->close;
        $self->SUPER::handle(delete $data{$session});
        }
        }
        }
        }

        package main;



        To unsubscribe from this group, send an email to:
        soaplite-unsubscribe@yahoogroups.com



        Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
      • Byrne Reese
        What are you going to do to correlate the response message with your request message? Were you thinking of using a callback, or some kind of polling mechanism?
        Message 3 of 7 , Oct 23, 2002
        • 0 Attachment
          What are you going to do to correlate the response message with your request message? Were you thinking of using a callback, or some kind of polling mechanism?
           
          Regardless, I can certainly help you quite a bit, as I have implemented many asynchronous perl services before. Granted, I have done them using my own company's protocol (Grand Central)... but I think I can help nonetheless.
           
          All modesty aside, I think Grand Central's model is very nice. The trick is for the service that will do the asynchronous messaging to return some kind of conversation or session id that is assigned to the incoming request so that down the road, when the response is returned, you have some way to correlate the two. I think SOAP toolkits kind of freak out when a SOAP message is not returned in an HTTP response... so it is important to do so. IOW, dropping the HTTP connection could have some unwelcome consequences (especially, if other toolkits access your service).
           
          First, let me recommend you take a look at a HOWTO I wrote for SOAP::Lite and Grand Central asynchronous messaging:
           
           
          Regardless of whether you use Grand Central or not, the article may give you some insight into how the async protocol works. On the other hand, feel free to sign up for a free developer's account on Grand Central and implement your services there... take a look at:
           
           
          If you have any questions regarding GC or even SOAP::Lite and asynchronous messaging feel free to drop me a line - I would be happy to help!
           
          Byrne Reese
          Developer Program Manager
          Grand Central Communications
          -----Original Message-----
          From: ryan_gammon [mailto:rggammon@...]
          Sent: Wednesday, October 23, 2002 11:43 AM
          To: soaplite@yahoogroups.com
          Subject: [soaplite] Asynchronous messaging

          Hi folks,

          I'm trying to use SOAP::Lite for asynchronous messaging (ie - I want
          to send a message, and have the client side return right away). At the
          moment, I've accomplished this by using the TCP transport, and
          changing the handler function to close the connection before handling
          the request (details at the end of this message)

          I feel this is a bit of a hack, and I'm trying to figure out how I'd
          like this to work.

          If SOAP::Lite were modified to...
          1. Look at the method being called
          2. See if it returns a value
          3. If it does, handle it as it does currently
          4. If it doesn't, make the client return (close the connection in the
          case of TCP) and then handle it

          ... would this be considered appropriate behavior in the soap world?
          Would there be any interest in having this functionality in
          SOAP::Lite?

          I'm a little hazy on the syntax that would be involved in steps 1 and
          2. If all else fails, I suppose an api could be provided to mark
          certain functions as not returning values from the point of view of
          SOAP::Lite.

          Current hack for TCP transport asynchronous messages:
          Added to server:
          ----
          package SOAP::Transport::TCP::Server;

          sub handle {  
            my $self = shift->new;
            my $sock = $self->{_socket};
            my $session_set = IO::SessionSet->new($sock);
            my %data;
            while (1) {
              my @ready = $session_set->wait($sock->timeout);
              for my $session (@ready) {
                my $data;
                if (my $rc = $session->read($data, 4096)) {
                  $data{$session} .= $data if $rc > 0;
                } else {
                  $session->close;
                  $self->SUPER::handle(delete $data{$session});
                }
              }
            }
          }

          package main;



          To unsubscribe from this group, send an email to:
          soaplite-unsubscribe@yahoogroups.com



          Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
        • Ryan Gammon
          ... In my case, I m using soap to send a notification and don t require a response. The function being called over soap takes some time to complete, and I
          Message 4 of 7 , Oct 23, 2002
          • 0 Attachment
            Byrne Reese wrote:

            > What are you going to do to correlate the response message with your
            > request message? Were you thinking of using a callback, or some kind
            > of polling mechanism?

            In my case, I'm using soap to send a notification and don't require a
            response. The function being called over soap takes some time to
            complete, and I don't want the soap client to have to wait for it to
            complete
            The tutorial at http://www.w3schools.com/wsdl/wsdl_ports.asp leads me to
            believe that I'm not totally barking up the wrong tree.

            There are several ways to do this:
            Keanan Smith suggested forking -- a good idea, but it has a performance
            cost, and I'm not aware of clean way to close the file descriptors
            opened by soap::lite.

            I notice Peter Fraenkel includes some forking server examples for the
            http transport
            SOAP-Lite-0.55/examples/SOAP/Transport/HTTP/Daemon/Fork*, and my
            overriding of the TCP transport handler is in a similar vein.

            I could also do something along the lines of the Grand Central method,
            and have a function similar to "search" that returns immediately, and
            (in my case) handle the notification in a different process.

            But wouldn't it be nice if soap::lite servers could see that the method
            being dispatched to isn't going to return anything, and allow the client
            to return immediately? Or alternatively, give a dispatched function the
            ability to close the transport, and continue processing? Would the
            soap::lite powers that be like to see such a beast?

            > All modesty aside, I think Grand Central's model is very nice. The
            > trick is for the service that will do the asynchronous messaging to
            > return some kind of conversation or session id that is assigned to the
            > incoming request so that down the road, when the response is returned,
            > you have some way to correlate the two.

            Makes sense -- I imagine that inside the "search" function, you have to
            fork a process (or send a message somewhere) to do the actual processing
            that produces the result for getMessage.

            > I think SOAP toolkits kind of freak out when a SOAP message is not
            > returned in an HTTP response... so it is important to do so. IOW,
            > dropping the HTTP connection could have some unwelcome consequences
            > (especially, if other toolkits access your service).

            This is one of the big questions I have -- is it proper SOAP behavior to
            not return a xml response? (as opposed to a response by the transport
            mechanism, like a 2xx message in http -- that is definitely required)

            > First, let me recommend you take a look at a HOWTO I wrote for
            > SOAP::Lite and Grand Central asynchronous messaging:
            >
            > http://soapenv.org/article.pl?sid=02/06/04/207222&mode=thread
            > <http://soapenv.org/article.pl?sid=02/06/04/207222&mode=thread>

            Nice howto!

            > Regardless of whether you use Grand Central or not, the article may
            > give you some insight into how the async protocol works. On the other
            > hand, feel free to sign up for a free developer's account on Grand
            > Central and implement your services there... take a look at:
            >
            > http://developer.grandcentral.com/
            >
            > If you have any questions regarding GC or even SOAP::Lite and
            > asynchronous messaging feel free to drop me a line - I would be happy
            > to help!
            >
            > Byrne Reese
            > Developer Program Manager
            > Grand Central Communications
            >
            > -----Original Message-----
            > *From:* ryan_gammon [mailto:rggammon@...]
            > *Sent:* Wednesday, October 23, 2002 11:43 AM
            > *To:* soaplite@yahoogroups.com
            > *Subject:* [soaplite] Asynchronous messaging
            >
            > Hi folks,
            >
            > I'm trying to use SOAP::Lite for asynchronous messaging (ie - I want
            > to send a message, and have the client side return right away). At the
            > moment, I've accomplished this by using the TCP transport, and
            > changing the handler function to close the connection before handling
            > the request (details at the end of this message)
            >
            > I feel this is a bit of a hack, and I'm trying to figure out how I'd
            > like this to work.
            >
            > If SOAP::Lite were modified to...
            > 1. Look at the method being called
            > 2. See if it returns a value
            > 3. If it does, handle it as it does currently
            > 4. If it doesn't, make the client return (close the connection in the
            > case of TCP) and then handle it
            >
            > ... would this be considered appropriate behavior in the soap world?
            > Would there be any interest in having this functionality in
            > SOAP::Lite?
            >
            > I'm a little hazy on the syntax that would be involved in steps 1 and
            > 2. If all else fails, I suppose an api could be provided to mark
            > certain functions as not returning values from the point of view of
            > SOAP::Lite.
            >
            > Current hack for TCP transport asynchronous messages:
            > Added to server:
            > ----
            > package SOAP::Transport::TCP::Server;
            >
            > sub handle {
            > my $self = shift->new;
            > my $sock = $self->{_socket};
            > my $session_set = IO::SessionSet->new($sock);
            > my %data;
            > while (1) {
            > my @ready = $session_set->wait($sock->timeout);
            > for my $session (@ready) {
            > my $data;
            > if (my $rc = $session->read($data, 4096)) {
            > $data{$session} .= $data if $rc > 0;
            > } else {
            > $session->close;
            > $self->SUPER::handle(delete $data{$session});
            > }
            > }
            > }
            > }
            >
            > package main;
            >
          • Paul Kulchenko
            Hi Ryan, ... How would you know it? There is no such information in method signature. ... Don t see an easy way to do it without forking a process. SOAP::Lite
            Message 5 of 7 , Oct 23, 2002
            • 0 Attachment
              Hi Ryan,

              --- Ryan Gammon <rggammon@...> wrote:
              > But wouldn't it be nice if soap::lite servers could see that the
              > method
              > being dispatched to isn't going to return anything, and allow the
              > client to return immediately?
              How would you know it? There is no such information in method
              signature.

              > Or alternatively, give a dispatched function the
              > ability to close the transport, and continue processing? Would the
              > soap::lite powers that be like to see such a beast?
              Don't see an easy way to do it without forking a process. SOAP::Lite
              does some optimization on server side (if dispatched call is done in
              void context response message will not be generated/serialized; used
              for POP3 server). Which means that you can overload HTTP transport
              and make a call in empty context, but still it will block before
              processing is complete.

              > This is one of the big questions I have -- is it proper SOAP
              > behavior to
              > not return a xml response? (as opposed to a response by the
              > transport
              > mechanism, like a 2xx message in http -- that is definitely
              > required)
              I would say yes. In fact, it was discussed several times and it seems
              like spec doesn't require response message to be present and it's ok
              to send empty response (for example only 200OK). SOAP::Lite supports
              this mode: you'll get undef instead of SOAP::SOM object in this case
              (unless fault was returned, but SOAP response is also optional in
              this case).

              Hope it helps. I would like to discuss proper API for async calls,
              because I want it to be implemented consistently for different
              transports.

              Best wishes, Paul.

              __________________________________________________
              Do you Yahoo!?
              Y! Web Hosting - Let the expert host your web site
              http://webhosting.yahoo.com/
            • Ryan Gammon
              ... The best answer I can think of here is to allow a soap::lite server to optionally parse the same wsdl file used for the client, and use that to determine
              Message 6 of 7 , Oct 23, 2002
              • 0 Attachment
                Paul Kulchenko wrote:

                >Hi Ryan,
                >
                >--- Ryan Gammon <rggammon@...> wrote:
                >
                >
                >>But wouldn't it be nice if soap::lite servers could see that the
                >>method
                >>being dispatched to isn't going to return anything, and allow the
                >>client to return immediately?
                >>
                >>
                >How would you know it? There is no such information in method
                >signature.
                >

                The best answer I can think of here is to allow a soap::lite server to
                optionally parse the same wsdl file used for the client, and use that to
                determine return types.

                >>Or alternatively, give a dispatched function the
                >>ability to close the transport, and continue processing? Would the
                >>soap::lite powers that be like to see such a beast?
                >>
                >>
                >Don't see an easy way to do it without forking a process.
                >

                The transports could be modified with a function that causes them to
                close on demand. I could call this function inside the method that has
                been dispatched to. On returning from the dispatched method, the handler
                of the transport in question would have to check to see if the transport
                had already been closed for it.

                >SOAP::Lite
                >does some optimization on server side (if dispatched call is done in
                >void context response message will not be generated/serialized; used
                >for POP3 server). Which means that you can overload HTTP transport
                >and make a call in empty context, but still it will block before
                >processing is complete.
                >
                >

                Pretty slick. That's the first time I've seen wantarray.

                >>This is one of the big questions I have -- is it proper SOAP
                >>behavior to
                >>not return a xml response? (as opposed to a response by the
                >>transport
                >>mechanism, like a 2xx message in http -- that is definitely
                >>required)
                >>
                >>
                >I would say yes. In fact, it was discussed several times and it seems
                >like spec doesn't require response message to be present and it's ok
                >to send empty response (for example only 200OK). SOAP::Lite supports
                >this mode: you'll get undef instead of SOAP::SOM object in this case
                >(unless fault was returned, but SOAP response is also optional in
                >this case).
                >

                That is indeed what I found with my TCP server with the redefined handler.

                >Hope it helps. I would like to discuss proper API for async calls,
                >because I want it to be implemented consistently for different
                >transports.
                >
                >Best wishes, Paul.
                >
                >__________________________________________________
                >Do you Yahoo!?
                >Y! Web Hosting - Let the expert host your web site
                >http://webhosting.yahoo.com/
                >
                >
                >
              • Keanan Smith
                To wit, there s no way to tell if a perl subroutine intends to return something, because even without an explicit return statement, a perl subroutine still
                Message 7 of 7 , Oct 24, 2002
                • 0 Attachment
                  To wit, there's no way to tell if a perl subroutine 'intends' to return
                  something, because even without an explicit 'return' statement, a perl
                  subroutine still returns the value of the last statement executed, in
                  effect, a perl subroutine always returns *something* it's just a question if
                  the author of the subroutine put something usefull there, and if the user of
                  the subroutine wants it. Neither of which can be examined from the method
                  itself.

                  WSDL or some kind of method-defining language of some sort would be required
                  for that kind of logic to be even possible.



                  There are several usefull combinations using fork, in addition to the
                  server-side one (Guessing at how to clean up after a spawned process from
                  the middle of a soap service is messy at best, admittedly)

                  If you as the client-writer, know you don't need whatever value the server
                  returns you could simply do:
                  if (fork)
                  {
                  $soap->soap_call;
                  exit;
                  }

                  in your client, resulting in a notification call to your server, and your
                  main program not even having to acknowledge that it sent it, this is
                  probably the fastest fork-based implementation of what you particularly want
                  to do.

                  I've also written a couple of fork-based catch-throw implementations,
                  similar to what the fellow from Grand Central was talking about, where your
                  'client' forks a 'listening' agent (Actually a soap server in it's own
                  right) and calls a service via soap, and ignores the return value as per
                  above, and the 'server' forks off a new soap object which calls the soap
                  service in your 'client' effectively turning both sides into client-servers,
                  then your client simply needs a way to collect the messages from the
                  'listening' agent to do something with them (IPC pipes work reasonably well,
                  although I haven't found anything that works truely satisfactorly on Win32)
                  both sides can implement a 'message session' to assossiate a given call with
                  a given return, in the event the calls need to be associated (The particular
                  implementation I wrote was a chat-client, so the client didn't need to know
                  where the message from the server came from, but with soap, it's just as
                  easy to send a lively little hash with message-id, authentication(which you
                  need in your client too, if your client is a client-server), and anything
                  else you want to stick in it :)

                  All that being said, yes fork has some overhead issues (particularly if you
                  try to do a bunch, a single fork that simply listens and passes message back
                  to your main program isn't too bad, overhead-wise. Still, it's a good Idea
                  to fork early to have less process state to duplicate) A more efficient
                  implementation would use Perl's multi-threaded model, however it only exists
                  in perl 5.8 and isn't entirely stable from what I've seen, and I have no
                  Idea if SOAP is thread-safe :)

                  Perhaps an alternative to forking on every SOAP call you want to disregard
                  the return value of, would be to fork once at the beginning and use some
                  kind of IPC messaging system between your main client process and your child
                  and have the child simply perform the soap call, and be blocked until it
                  returns but not have to worry about being blocked in the main program.

                  -----Original Message-----
                  From: Ryan Gammon [mailto:rggammon@...]
                  Sent: Wednesday, October 23, 2002 11:39 PM
                  To: soaplite@yahoogroups.com
                  Subject: Re: [soaplite] Asynchronous messaging


                  Paul Kulchenko wrote:

                  >Hi Ryan,
                  >
                  >--- Ryan Gammon <rggammon@...> wrote:
                  >
                  >
                  >>But wouldn't it be nice if soap::lite servers could see that the
                  >>method
                  >>being dispatched to isn't going to return anything, and allow the
                  >>client to return immediately?
                  >>
                  >>
                  >How would you know it? There is no such information in method
                  >signature.
                  >

                  The best answer I can think of here is to allow a soap::lite server to
                  optionally parse the same wsdl file used for the client, and use that to
                  determine return types.

                  >>Or alternatively, give a dispatched function the
                  >>ability to close the transport, and continue processing? Would the
                  >>soap::lite powers that be like to see such a beast?
                  >>
                  >>
                  >Don't see an easy way to do it without forking a process.
                  >

                  The transports could be modified with a function that causes them to
                  close on demand. I could call this function inside the method that has
                  been dispatched to. On returning from the dispatched method, the handler
                  of the transport in question would have to check to see if the transport
                  had already been closed for it.

                  >SOAP::Lite
                  >does some optimization on server side (if dispatched call is done in
                  >void context response message will not be generated/serialized; used
                  >for POP3 server). Which means that you can overload HTTP transport
                  >and make a call in empty context, but still it will block before
                  >processing is complete.
                  >
                  >

                  Pretty slick. That's the first time I've seen wantarray.

                  >>This is one of the big questions I have -- is it proper SOAP
                  >>behavior to
                  >>not return a xml response? (as opposed to a response by the
                  >>transport
                  >>mechanism, like a 2xx message in http -- that is definitely
                  >>required)
                  >>
                  >>
                  >I would say yes. In fact, it was discussed several times and it seems
                  >like spec doesn't require response message to be present and it's ok
                  >to send empty response (for example only 200OK). SOAP::Lite supports
                  >this mode: you'll get undef instead of SOAP::SOM object in this case
                  >(unless fault was returned, but SOAP response is also optional in
                  >this case).
                  >

                  That is indeed what I found with my TCP server with the redefined handler.

                  >Hope it helps. I would like to discuss proper API for async calls,
                  >because I want it to be implemented consistently for different
                  >transports.
                  >
                  >Best wishes, Paul.
                  >
                  >__________________________________________________
                  >Do you Yahoo!?
                  >Y! Web Hosting - Let the expert host your web site
                  >http://webhosting.yahoo.com/
                  >
                  >
                  >





                  To unsubscribe from this group, send an email to:
                  soaplite-unsubscribe@yahoogroups.com



                  Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                Your message has been successfully submitted and would be delivered to recipients shortly.