Loading ...
Sorry, an error occurred while loading the content.

Re: "ocaml_beginners"::[] Will Unix.wait stop a processor's threads?

Expand Messages
  • Hugo Ferreira
    Hi Cedric, ... Agreed. ... I was questioning if this was true. I had imagined that if the lock were not released then a wait wouldn t cut it. ... I am. But I
    Message 1 of 9 , Jun 17, 2010
    • 0 Attachment
      Hi Cedric,

      rixed@... wrote:
      >> In the slave process I split the application into two threads:
      >>
      >> 1) Reads messages (requests) that then fork jobs/tasks.
      >> It also records the pid of process that was launched.
      >>
      >> 2) The main loop checks if any child process has terminated
      >> via Unix.wait. It records the pid that terminated and removes
      >> it from the pending list.
      >>
      >> I have noticed that many times the Unix.wait returns a pid before
      >> thread (1) gets a chance to record it.
      >
      > I suppose this scenario happens very frequently, where
      > the ocaml thread scheduler switch thread after a fork() and
      > the forked process runs so quickly that the CPU still
      > belongs to thread 2 in the ocaml process :
      >

      Agreed.

      >
      >> Also it seems like every time I wait on a child pid, the thread (1)
      >> doesn't receive messages.
      >
      > I never looked how it actually works but I always though that ocaml
      > thread implemenation using system threads uses a global giant mutex
      > for ensuring that no two threads run simultaneously. So I imagine this
      > lock is released before all waiting system call like wait() (and
      > reaquired one the wait returns). So I can't figure why you would not
      > receive messages during the wait().
      >

      I was questioning if this was true. I had imagined that if the lock
      were not released then a wait wouldn't cut it.

      > Or maybe you are using the VM based implementation for threads ?
      >

      I am. But I am assuming that the behaviour of both is similar.

      >> If so what would be the best way to know when a
      >> child process has finished without stopping thread (1)?
      >
      > Quick fix :
      > Instead of recording the pid after the fork() and removing it
      > after the wait(), why not record both creation and termination,
      > for instance in a hash with key = pid and value = 1 if created, -1
      > if terminated (0 means created + terminated, delete the entry).
      >

      Actually I am doing this. I wait on a process and record its pid.
      I then check if any such pid are forked processes that I am supposed
      to be waiting on. If so I acknowledge their completion to the master
      process. I do this because as I said the wait may signal an end of
      process before it gets recorded.

      Now that I think of it, all processes I fork I acknowledge. So may
      no list/hash of pids is required.

      > More conventional way of doing this :
      > Do not use another thread for waiting pids but ask for a signal
      > when one of your child terminates, then you will quit the select on
      > EINTR and can then wait (non-blocking variant) to collect the pids.
      >
      >

      I tried the non-blocking variant which also did not work. This led me
      to the real issue: I was not releasing a mutex that both threads share,
      hence the alternating behaviour. The solution that uses blocking "wait"
      works. "wait" therefore does _not_ pre-empt the other threads and works
      as expected.

      Out of curiosity when in the above paragraph you say "select" are you
      referring to the actual select call?

      Thanks for the input,
      Hugo F.


      >
      > ------------------------------------
      >
      > Archives up to December 31, 2009 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners/
      > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
      > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
      >
      >
      >
    • rixed@happyleptic.org
      ... Then wait() being a blocking syscall this is not surprising that your OCaml process can t receive anything during the wait().
      Message 2 of 9 , Jun 17, 2010
      • 0 Attachment
        > > Or maybe you are using the VM based implementation for threads ?
        >
        > I am. But I am assuming that the behaviour of both is similar.

        Then wait() being a blocking syscall this is not surprising
        that your OCaml process can't receive anything during the wait().
      • Hugo Ferreira
        ... It can. While one thread blocks on in/out the others work. Blocking on wait does seem to allow other threads to work. This in the vm. Should be the same
        Message 3 of 9 , Jun 17, 2010
        • 0 Attachment
          rixed@... wrote:
          >>> Or maybe you are using the VM based implementation for threads ?
          >> I am. But I am assuming that the behaviour of both is similar.
          >
          > Then wait() being a blocking syscall this is not surprising
          > that your OCaml process can't receive anything during the wait().
          >

          It can. While one thread blocks on in/out the others work.
          Blocking on wait does seem to allow other threads to work.
          This in the vm. Should be the same for opt.


          >
          >
          > ------------------------------------
          >
          > Archives up to December 31, 2009 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners/
          > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
          > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
          >
          >
          >
        • rixed@happyleptic.org
          -[ Thu, Jun 17, 2010 at 02:40:11PM +0100, Hugo Ferreira ]---- ... Are you certain you are using the VM implementation of threads ? Ie are you compiling with
          Message 4 of 9 , Jun 17, 2010
          • 0 Attachment
            -[ Thu, Jun 17, 2010 at 02:40:11PM +0100, Hugo Ferreira ]----
            > rixed@... wrote:
            > >>> Or maybe you are using the VM based implementation for threads ?
            > >> I am. But I am assuming that the behaviour of both is similar.
            > >
            > > Then wait() being a blocking syscall this is not surprising
            > > that your OCaml process can't receive anything during the wait().
            >
            > It can. While one thread blocks on in/out the others work.
            > Blocking on wait does seem to allow other threads to work.
            > This in the vm. Should be the same for opt.

            Are you certain you are using the VM implementation of threads ?
            Ie are you compiling with -thread or -vmthread ?
          • Hugo Ferreira
            ... Fairly certain. I am using ocamlbuild and generating the application with the *.d.byte extension.
            Message 5 of 9 , Jun 17, 2010
            • 0 Attachment
              rixed@... wrote:
              > -[ Thu, Jun 17, 2010 at 02:40:11PM +0100, Hugo Ferreira ]----
              >> rixed@... wrote:
              >>>>> Or maybe you are using the VM based implementation for threads ?
              >>>> I am. But I am assuming that the behaviour of both is similar.
              >>> Then wait() being a blocking syscall this is not surprising
              >>> that your OCaml process can't receive anything during the wait().
              >> It can. While one thread blocks on in/out the others work.
              >> Blocking on wait does seem to allow other threads to work.
              >> This in the vm. Should be the same for opt.
              >
              > Are you certain you are using the VM implementation of threads ?
              > Ie are you compiling with -thread or -vmthread ?
              >

              Fairly certain. I am using ocamlbuild and generating the
              application with the *.d.byte extension.

              >
              >
              >
              > ------------------------------------
              >
              > Archives up to December 31, 2009 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners/
              > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
              > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
              >
              >
              >
            • rixed@happyleptic.org
              -[ Thu, Jun 17, 2010 at 05:41:03PM +0100, Hugo Ferreira ]---- ... You probably use the pthread implementation. Byte code programs can use both.
              Message 6 of 9 , Jun 17, 2010
              • 0 Attachment
                -[ Thu, Jun 17, 2010 at 05:41:03PM +0100, Hugo Ferreira ]----
                > > Are you certain you are using the VM implementation of threads ?
                > > Ie are you compiling with -thread or -vmthread ?
                > >
                >
                > Fairly certain. I am using ocamlbuild and generating the
                > application with the *.d.byte extension.

                You probably use the pthread implementation. Byte code programs can
                use both.
              • Hugo Ferreira
                ... Ok. I stand corrected. Looking at the ocamlbuild file I see the use of -thread flag and threads.cma library.
                Message 7 of 9 , Jun 18, 2010
                • 0 Attachment
                  rixed@... wrote:
                  > -[ Thu, Jun 17, 2010 at 05:41:03PM +0100, Hugo Ferreira ]----
                  >>> Are you certain you are using the VM implementation of threads ?
                  >>> Ie are you compiling with -thread or -vmthread ?
                  >>>
                  >> Fairly certain. I am using ocamlbuild and generating the
                  >> application with the *.d.byte extension.
                  >
                  > You probably use the pthread implementation. Byte code programs can
                  > use both.
                  >

                  Ok. I stand corrected. Looking at the ocamlbuild file I
                  see the use of -thread flag and threads.cma library.


                  >
                  >
                  > ------------------------------------
                  >
                  > Archives up to December 31, 2009 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners/
                  > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
                  > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
                  >
                  >
                  >
                Your message has been successfully submitted and would be delivered to recipients shortly.