Loading ...
Sorry, an error occurred while loading the content.
 

Re: high-speed postfix configuration

Expand Messages
  • Stan Hoeppner
    ... Of course. And it should be noted that even though this is rather soft, the OP was tripping it and seeing a back off delay of around 6 seconds. ... Which
    Message 1 of 21 , Aug 29 5:36 AM
      On 8/28/2012 1:13 PM, Viktor Dukhovni wrote:
      > On Tue, Aug 28, 2012 at 11:48:19AM -0500, Stan Hoeppner wrote:
      >
      >> So maybe for this particular application a ramdisk isn't a horrible
      >> idea. But he still has the problem of implementing parallel submission
      >> in his java application, otherwise it won't matter if it's a ramdisk or
      >> an old 3600 RPM MFM drive on an 8 bit controller.
      >
      > It should be noted that parallel submission is not a work-around
      > for Postfix per-client rate limits (the default in_flow_delay is
      > a rather soft barrier in any case).

      Of course. And it should be noted that even though this is rather soft,
      the OP was tripping it and seeing a back off delay of around 6 seconds.

      > Rather, parallel submission
      > is a way to ammortize SMTP latency across multiple connections.

      > In the same time that it takes to process a single incoming message,
      > it is possible to accept multiple parallel messages. For clients
      > with a low-latency (LAN or campus network) connection to the SMTP
      > server, concurrency of ~20 connections is often sufficient to
      > saturate the server's disk bandwidth (or CPU bandwidth if moderately
      > expensive content filtering is in place).

      Which is why I recommended the OP do two things in his java app:

      1. Implement parallel delivery _and_
      2. Implement submission rate control on each thread, e.g. 20 msgs/sec

      > Strictly serialized SMTP submission is slow because it fails to
      > fully utilize idle CPU, network and disk cycles.

      Usually disk, as will be the case here once the OP gets his submission
      rate up where he wants it. Which how we wound up talking about the
      typically insane idea of using a ramdisk for the queue. ;)

      --
      Stan
    • DN Singh
      ... The thread seems very interesting. We use threading for concurrent submissions, and were able to submit around 40k mails per hour on a VM. We had 10 smtp
      Message 2 of 21 , Sep 5, 2012
        On Wed, Aug 29, 2012 at 6:06 PM, Stan Hoeppner <stan@...> wrote:
        On 8/28/2012 1:13 PM, Viktor Dukhovni wrote:
        > On Tue, Aug 28, 2012 at 11:48:19AM -0500, Stan Hoeppner wrote:
        >
        >> So maybe for this particular application a ramdisk isn't a horrible
        >> idea.  But he still has the problem of implementing parallel submission
        >> in his java application, otherwise it won't matter if it's a ramdisk or
        >> an old 3600 RPM MFM drive on an 8 bit controller.
        >
        > It should be noted that parallel submission is not a work-around
        > for Postfix per-client rate limits (the default in_flow_delay is
        > a rather soft barrier in any case).

        Of course.  And it should be noted that even though this is rather soft,
        the OP was tripping it and seeing a back off delay of around 6 seconds.

        > Rather, parallel submission
        > is a way to ammortize SMTP latency across multiple connections.

        > In the same time that it takes to process a single incoming message,
        > it is possible to accept multiple parallel messages. For clients
        > with a low-latency (LAN or campus network) connection to the SMTP
        > server, concurrency of ~20 connections is often sufficient to
        > saturate the server's disk bandwidth (or CPU bandwidth if moderately
        > expensive content filtering is in place).

        Which is why I recommended the OP do two things in his java app:

        1.  Implement parallel delivery _and_
        2.  Implement submission rate control on each thread, e.g. 20 msgs/sec

        > Strictly serialized SMTP submission is slow because it fails to
        > fully utilize idle CPU, network and disk cycles.

        Usually disk, as will be the case here once the OP gets his submission
        rate up where he wants it.  Which how we wound up talking about the
        typically insane idea of using a ramdisk for the queue. ;)


        The thread seems very interesting. We use threading for concurrent submissions, and were able to submit around 40k mails per hour on a VM. We had 10 smtp connections with each connection pushing around 1000 mails. The threads would work until the submission got completed.
        We had to throttle it down, because we were facing delivery issues.
         
        --
        Stan


      Your message has been successfully submitted and would be delivered to recipients shortly.