Loading ...
Sorry, an error occurred while loading the content.

Re: [dansguardian] Max Number of Processes

Expand Messages
  • Scott Mayo
    ... I am curious to this 250 number. I decided to bump mine up to 350 this morning and so far it is doing fine. This is on a Fedora box. ... down, yet so far
    Message 1 of 15 , Apr 27 7:11 AM
    View Source
    • 0 Attachment
      On Mon, Feb 27, 2012 at 10:46 AM, Philip Pearce <e2bntech@...> wrote:

      > **
      >
      >
      > In the standard dg build it's set at a max of around 1020.
      >
      > However, as the comment suggests it depends on the OS you are using.
      >
      > On Linux it is around 250, it may be different for other OS's.
      >
      >
      I am curious to this 250 number. I decided to bump mine up to 350 this
      morning and so far it is doing fine. This is on a Fedora box.


      > Note that when you set maxchildren to a higher value than supported by the
      > OS, it will only fail (possibly with core dump) when the system limit on
      > the number of file_ids that can be handled by a select call is reached.
      >
      > To test what the system limit is set the minchildren to a high value say
      > (maxchildren - preforkchildren). DG will then attempt to create that number
      > of child processes on start-up, so if it's too large it will fail
      > immediately. Once you have found the limit you can then reduce the
      > minchildren to a more sensible figure.
      >
      > I set my minchildren to 240 and it failed on startup so I set it back
      down, yet so far the machine is still running with maxchildren at 350. I
      am curious to know for sure how high I can set my maxchildren.

      --
      Scott


      [Non-text portions of this message have been removed]
    • Philip Pearce
      maxchildren just sets the upper limit of the number of child filter processes dg will attempt to spawn. If the OS limit is lower than maxchildren, this will
      Message 2 of 15 , Apr 27 8:25 AM
      View Source
      • 0 Attachment
        maxchildren just sets the upper limit of the number of child filter processes dg will attempt to spawn.

        If the OS limit is lower than maxchildren, this will not stop dg starting, but when dg tries to spawn over the OS limit it will will fail. Since it failed with minchildren=240 this would indicate that your OS limit is around 250 (or less). To see how many processes dg is actually using do a 'ps -ef|grep dans|wc -l' or something similar.

        Philip
        ----- Original Message -----

        From: "Scott Mayo" <scotgmayo@...>
        To: dansguardian@yahoogroups.com
        Sent: Friday, 27 April, 2012 3:11:08 PM
        Subject: Re: [dansguardian] Max Number of Processes





        On Mon, Feb 27, 2012 at 10:46 AM, Philip Pearce < e2bntech@... > wrote:

        > **
        >
        >
        > In the standard dg build it's set at a max of around 1020.
        >
        > However, as the comment suggests it depends on the OS you are using.
        >
        > On Linux it is around 250, it may be different for other OS's.
        >
        >
        I am curious to this 250 number. I decided to bump mine up to 350 this
        morning and so far it is doing fine. This is on a Fedora box.

        > Note that when you set maxchildren to a higher value than supported by the
        > OS, it will only fail (possibly with core dump) when the system limit on
        > the number of file_ids that can be handled by a select call is reached.
        >
        > To test what the system limit is set the minchildren to a high value say
        > (maxchildren - preforkchildren). DG will then attempt to create that number
        > of child processes on start-up, so if it's too large it will fail
        > immediately. Once you have found the limit you can then reduce the
        > minchildren to a more sensible figure.
        >
        > I set my minchildren to 240 and it failed on startup so I set it back
        down, yet so far the machine is still running with maxchildren at 350. I
        am curious to know for sure how high I can set my maxchildren.

        --
        Scott

        [Non-text portions of this message have been removed]





        [Non-text portions of this message have been removed]
      • Scott Mayo
        ... I did that this morning and it was using 347 at one time. That was the largest number that I saw. Everything was still working then and still is so I
        Message 3 of 15 , Apr 27 8:54 AM
        View Source
        • 0 Attachment
          On Fri, Apr 27, 2012 at 10:25 AM, Philip Pearce <e2bntech@...> wrote:

          > **
          >
          >
          > maxchildren just sets the upper limit of the number of child filter
          > processes dg will attempt to spawn.
          >
          > If the OS limit is lower than maxchildren, this will not stop dg starting,
          > but when dg tries to spawn over the OS limit it will will fail. Since it
          > failed with minchildren=240 this would indicate that your OS limit is
          > around 250 (or less). To see how many processes dg is actually using do a
          > 'ps -ef|grep dans|wc -l' or something similar.
          >
          >
          I did that this morning and it was using 347 at one time. That was the
          largest number that I saw. Everything was still working then and still is
          so I assume I can bump it on up if I need to? Until I get some sort of
          error?

          Thanks.

          --
          Scott


          [Non-text portions of this message have been removed]
        • Philip Pearce
          ... Yes, you can increase it - If you are getting 347 then it s likely that your OS limit is 1024 - But keep an eye on memory usage as performance will sharply
          Message 4 of 15 , Apr 30 2:12 AM
          View Source
          • 0 Attachment
            >> If the OS limit is lower than maxchildren, this will not stop dg starting,
            >> but when dg tries to spawn over the OS limit it will will fail. Since it
            >> failed with minchildren=240 this would indicate that your OS limit is
            >> around 250 (or less). To see how many processes dg is actually using do a
            >> 'ps -ef|grep dans|wc -l' or something similar.
            >>
            >
            >I did that this morning and it was using 347 at one time. That was the
            >largest number that I saw. Everything was still working then and still is
            >so I assume I can bump it on up if I need to? Until I get some sort of
            >error?

            Yes, you can increase it - If you are getting 347 then it's likely that your OS limit is 1024 - But keep an eye on memory usage as performance will sharply degrade if the systems starts to swap.

            Regards
            Philip

            ----- Original Message -----

            From: "Scott Mayo" <scotgmayo@...>
            To: dansguardian@yahoogroups.com
            Sent: Friday, 27 April, 2012 4:54:50 PM
            Subject: Re: [dansguardian] Max Number of Processes





            On Fri, Apr 27, 2012 at 10:25 AM, Philip Pearce < e2bntech@... > wrote:

            > **
            >
            >
            > maxchildren just sets the upper limit of the number of child filter
            > processes dg will attempt to spawn.
            >
            > If the OS limit is lower than maxchildren, this will not stop dg starting,
            > but when dg tries to spawn over the OS limit it will will fail. Since it
            > failed with minchildren=240 this would indicate that your OS limit is
            > around 250 (or less). To see how many processes dg is actually using do a
            > 'ps -ef|grep dans|wc -l' or something similar.
            >
            >
            I did that this morning and it was using 347 at one time. That was the
            largest number that I saw. Everything was still working then and still is
            so I assume I can bump it on up if I need to? Until I get some sort of
            error?

            Thanks.

            --
            Scott

            [Non-text portions of this message have been removed]





            [Non-text portions of this message have been removed]
          • Scott Mayo
            ... Thanks for the info. I have been watching TOP off and on. Looks like my CPU usage is somewhere around 1-5% on average and jumps up to 8-12 once in a bit.
            Message 5 of 15 , May 1, 2012
            View Source
            • 0 Attachment
              On Mon, Apr 30, 2012 at 4:12 AM, Philip Pearce <e2bntech@...> wrote:

              > **
              >
              >
              > >> If the OS limit is lower than maxchildren, this will not stop dg
              > starting,
              > >> but when dg tries to spawn over the OS limit it will will fail. Since
              > it
              > >> failed with minchildren=240 this would indicate that your OS limit is
              > >> around 250 (or less). To see how many processes dg is actually using do
              > a
              > >> 'ps -ef|grep dans|wc -l' or something similar.
              > >>
              > >
              > >I did that this morning and it was using 347 at one time. That was the
              > >largest number that I saw. Everything was still working then and still is
              > >so I assume I can bump it on up if I need to? Until I get some sort of
              > >error?
              >
              > Yes, you can increase it - If you are getting 347 then it's likely that
              > your OS limit is 1024 - But keep an eye on memory usage as performance will
              > sharply degrade if the systems starts to swap.
              >


              Thanks for the info. I have been watching TOP off and on. Looks like my
              CPU usage is somewhere around 1-5% on average and jumps up to 8-12 once in
              a bit. Memory usage is around 950,000k out of 2,065,084k on average, so it
              is not having to use any Swap yet.

              Those numbers came when it was using 250-301 processes.

              --
              Scott


              [Non-text portions of this message have been removed]
            Your message has been successfully submitted and would be delivered to recipients shortly.