Loading ...
Sorry, an error occurred while loading the content.

Possible solution to session-oriented TCP server pool?

Expand Messages
  • Ron Murphy
    Hi, ... I hadn t mentioned it, but this was assuming the TCP server. I got some code working which is enclosed below. Any comments would be greatly
    Message 1 of 1 , Oct 25, 2002
    • 0 Attachment
      Hi,

      A while back I asked about the following design problem:

      > I would like to have my SOAP-enabled server do forking so it is
      > more scalable, but not on a per-request basis as (apparently) in
      > ForkOnAccept and ForkAfterProcessing.
      >
      > Instead, I would like to make an explicit server-dispatched function
      > like login() which forks, and then have the client talk to the
      > child process for all remaining calls in this "session". I'd also
      > like to do this for not just one SOAP object, but several different
      > ones (representing different server capabilities).

      I hadn't mentioned it, but this was assuming the TCP server.

      I got some code working which is enclosed below. Any comments would be
      greatly appreciated. Also, perhaps others will benefit from the
      design discussion and code.

      Some notes on the approach:
      1. The basic trick involves the client first doing a login call which causes
      a fork
      and produces a unique port number returned to the client; then, additional
      SOAP
      objects are allocated in the client using this port number.
      2. The server parent allocates the ports and processes out of a fixed range.
      Children are forked with the first available port number, and when a child
      exits
      (usually because the user asked for "logout"), this is seen by the parent
      and the port
      is made available again.
      3. Forking is triggered by a "spawn" method on the new server object.
      Again, the idea
      is that a particular server-side API function like login() will call spawn()
      and give a port
      number back. I found that doing the fork() *before* the request loop in
      handle() has finished,
      causes an unknown transport fault in the client. The response made it back
      to the
      client, but might have been garbled...possibly parent and child were both
      responding
      and I should have been closing something in one or the other. Rather than
      sort this out,
      I shifted to setting a "reinit" flag instead, and doing the fork later at a
      safe point in handle().
      4. Since this code relies on getting SIGCHLD signals, I found that Lincoln
      Stein's
      IO::SessionSet->wait() call does not handle interrupted system calls. The
      signals bomb
      out the whole parent process. I had to override the wait() function to
      tolerate EINTR.

      Anyway:

      package SOAP::Transport::TCP::ServerPool; # TODO Adjust name/hierarchy
      appropriately

      use strict;
      use vars qw(@ISA);

      use constant MAXSLOTS => 20;
      use constant PORTSTART => 4560;

      @ISA = qw(SOAP::Transport::TCP::Server);

      my $HandlerObject;

      sub new {
      my $self = shift;
      my @args = @_;

      unless (ref $self) {
      my $class = ref($self) || $self;

      # save a copy for socket reinit
      my @init1;
      foreach my $i (@args) {push(@init1, $i);}
      $self = $class->SUPER::new(@args);
      # Duplicate the logic of the base class to get the parameters used
      # to construct the socket.
      while (@init1)

      if ($class->can($init1[0]) || $init1[0] eq 'LocalPort') {
      shift @init1; shift @init1;
      } else {
      # TODO the LocalAddr might have a port number in it...overridden?
      push(@{$self->{socketparms}}, shift @init1);
      }
      }

      $self->{children} = ();

      # Parent sets signal handler to cleanup children. Note that children
      # might set this up differently for their own purposes.
      $SIG{CHLD} = \&cleanup_children;
      $HandlerObject = $self;
      }
      return $self;
      }

      sub handle {
      my $self = shift->new;
      while (1) {
      my $sock = $self->{_socket};
      my $session_set = IO::SessionSet->new($sock);
      my %data;
      undef $self->{reinit};
      while (!defined($self->{reinit})) {
      my @ready = $session_set->wait($sock->timeout);
      for my $session (@ready) {
      my $data;
      if (my $rc = $session->read($data, 4096)) {
      $data{$session} .= $data if $rc > 0;
      } else {
      $session->write($self->SOAP::Server::handle(delete
      $data{$session}));
      $session->close;
      }
      if (defined($self->{reinit})) {
      my $slot = $self->{nextslot};
      if (my $pid = fork) { # parent
      $self->{children}->[$slot] = $pid;
      print "parent spawned $pid\n";
      # TODO...what cleanup is done in parent?
      undef $self->{reinit};
      } else {
      # TODO...any other logic?
      }
      }
      }
      }
      $self->{_socket}->close;
      print "PID $$: creating new socket with parms LocalPort $self->{port} "
      . join(' ', @{$self->{socketparms}}) . "\n";
      my $socket = $self->io_socket_class;
      $self->{_socket} = $socket->new(Proto => 'tcp',
      LocalPort => $self->{port},
      @{$self->{socketparms}})
      or Carp::croak "Can't open socket: $!";
      }
      }

      sub find_empty_slot {
      my $self = shift;

      my $slot=-1;
      foreach my $ix (0..MAXSLOTS) {
      if (!exists($self->{children}->[$ix])) {
      $slot = $ix;
      last;
      }
      }

      return $slot;
      }

      sub spawn {
      my $self = shift;

      my $slot = $self->find_empty_slot();
      if ($slot == -1) {return -1;}
      $self->{reinit} = 1;
      $self->{nextslot} = $slot;
      $self->{port} = PORTSTART + $slot;
      return $self->{port};
      }

      sub remove_pid {
      my ($self, $pid) = @_;
      my $slot=-1;
      foreach my $ix (0..$#{$self->{children}}) {
      if (exists($self->{children}->[$ix]) &&
      $self->{children}->[$ix] == $pid) {
      $slot = $ix;
      last;
      }
      }
      if ($slot == -1) {
      warn "Child PID $pid not found!\n";
      }

      delete $self->{children}->[$slot];
      }

      sub cleanup_children {
      my $self = $HandlerObject;
      use POSIX qw(:sys_wait_h);
      my $pid = waitpid(-1, WNOHANG);
      while ($pid > 0) {
      print "Child $pid returned status $?\n";
      $self->remove_pid($pid);
      $pid = waitpid(-1, WNOHANG);
      }
      }

      1;
    Your message has been successfully submitted and would be delivered to recipients shortly.