Loading ...
Sorry, an error occurred while loading the content.

Adjust smtp to limitations of a host

Expand Messages
  • Ultrabug
    Dear list, I m facing a problem where I have to adapt and optimize my smtp servers to a host s constraints which are as follow : - maximum 3 connections to
    Message 1 of 16 , Mar 31, 2011
    • 0 Attachment
      Dear list,

      I'm facing a problem where I have to adapt and optimize my smtp servers
      to a host's constraints which are as follow :

      - maximum 3 connections to each MX of the host (he has 10 MX so
      potentially I should be able to make 30 connections)
      - maximum 1000 connections per MX per hour
      - maximum 100 emails sent per connection

      Today I'm using a special transport to match the host's domains and I
      limited the transport to 3 smtp processes so I don't break the first
      rule and get blocked.
      In order to catch and balance between the 10 MX, I also use
      smtp_mx_address_limit = 10 instead of the default 5.

      My problem is that this setup is far from optimal compared to the
      limitations and it slows down a lot my email delivery rate to these domains.

      Anyone have any tips on how I could do this better please ?

      Thanks a lot,

      Regards.
    • Victor Duchovni
      ... What happens if you happen to exceed the limit on particular host among the 10? If it just quickly returns a 4XX code, and does not penalize future
      Message 2 of 16 , Mar 31, 2011
      • 0 Attachment
        On Thu, Mar 31, 2011 at 10:15:55AM +0200, Ultrabug wrote:

        > Dear list,
        >
        > I'm facing a problem where I have to adapt and optimize my smtp servers
        > to a host's constraints which are as follow :
        >
        > - maximum 3 connections to each MX of the host (he has 10 MX so
        > potentially I should be able to make 30 connections)

        What happens if you happen to exceed the limit on particular host
        among the 10? If it just quickly returns a 4XX code, and does not
        penalize future connections, ignore this limit and let Postfix do
        what it does by default.

        > - maximum 1000 connections per MX per hour

        Connection caching should help if volume is high enough to worry about
        this. Note this is just less than one connection every 3 seconds, but
        Postfix caches idle connections for 2 seconds, so if your output rate
        is 1200 messages spaced perfectly 3 seconds apart, you lose, but this
        is fairly unlikely.

        > - maximum 100 emails sent per connection

        Postfix has no such limit, instead a connection is retired if idle
        for more than 2 seconds, or after 300s (tunable). Again the site
        should just return a 4XX response to RSET, and Postfix will drop
        the connection and build a new connection, probably to another host.

        > My problem is that this setup is far from optimal compared to the
        > limitations and it slows down a lot my email delivery rate to these domains.
        >
        > Anyone have any tips on how I could do this better please ?

        The receiving sites policies are stupid if they don't implement
        them sensibly by just returning 4XX responses without penalizing
        subsequent transactions.

        Have you in fact observed that default Postfix settings run into trouble
        with this site? Have you considered the less aggressive concurrency
        feedback controls in Postfix 2.5?

        slow_initial_concurrency = 2
        slow_destination_concurrency_limit = 15
        slow_destination_concurrency_failed_cohort_limit = 5
        slow_destination_concurrency_positive_feedback = 1/5
        slow_destination_concurrency_negative_feedback = 1/8

        and if absolutely necessary, in master.cf:

        slow unix - - n - - smtp
        -o smtp_connection_reuse_time_limit=30s

        (the remote side starts rejecting traffic consistently instead of
        sending 421 for the 100th RSET over a given connection).

        --
        Viktor.
      • Mark Alan
        On Thu, 31 Mar 2011 12:39:20 -0400, Victor Duchovni ... I am sorry to hijack this thread but we have what seems to be the same problem. While using the default
        Message 3 of 16 , Mar 31, 2011
        • 0 Attachment
          On Thu, 31 Mar 2011 12:39:20 -0400, Victor Duchovni
          <Victor.Duchovni@...> wrote:

          > The receiving sites policies are stupid if they don't implement
          > them sensibly by just returning 4XX responses without penalizing
          > subsequent transactions.

          I am sorry to hijack this thread but we have what seems to be the
          same problem.

          While using the default Postfix settings (v.2.8.1 on Ubuntu 10.10), we
          do have trouble to connect with several MTA's (usually
          smtp1.min-saude.pt and smtp2.min-saude.pt, but sometimes others
          at .min-saude.pt).
          The server at smtp3.min-saude.pt never complains, nor do any of
          the other email MTA at .min-saude.pt whose name do not start with
          smtpNN.

          When they refuse our connections, they seem to start shutting down at
          25 to 30 RCPT commands, with:
          "...mx postfix-slow/smtp[4907]: 36BB7818B:
          to=<some_subscriber@...-saude.pt>,
          relay=smtp1.min-saude.pt[194.65.151.38]:25, delay=415,
          delays=414/0.25/0.41/0, dsn=4.0.0, status=deferred (host
          smtp1.min-saude.pt[194.65.151.38] refused to talk to me: 421 #4.4.5 Too
          many connections from your host.) "

          To deal with this we are currently using:

          /etc/postfix/transport
          .min-saude.pt slow:

          /etc/postfix/master.cf
          slow unix - - - - - smtp
          -o syslog_name=postfix-slow
          -o smtp_connection_cache_on_demand=no
          EOT

          /etc/postfix/main.cf
          slow_destination_concurrency_failed_cohort_limit = 3 # we give up
          after getting three 421
          slow_destination_recipient_limit = 20 # keep it bellow 25
          slow_destination_rate_delay = 1 # do not know if we really need this

          > Have you considered the less aggressive
          > concurrency feedback controls in Postfix 2.5?

          Do you think that the following would be a more elegant approach than
          the above described setting?

          /etc/postfix/master.cf
          slow unix - - - - - smtp
          -o syslog_name=postfix-slow
          -o smtp_connection_reuse_time_limit=30s
          EOT

          /etc/postfix/main.cf
          slow_initial_destination_concurrency = 2
          slow_destination_concurrency_limit = 15
          slow_destination_concurrency_failed_cohort_limit = 5
          slow_destination_concurrency_positive_feedback = 1/5
          slow_destination_concurrency_negative_feedback = 1/8

          Thank you,

          M.
        • Victor Duchovni
          ... Why would this be a response to too many recipient commands , a single message with many recipients is sent over a single connection, unless you have set
          Message 4 of 16 , Mar 31, 2011
          • 0 Attachment
            On Thu, Mar 31, 2011 at 07:41:41PM +0100, Mark Alan wrote:

            > On Thu, 31 Mar 2011 12:39:20 -0400, Victor Duchovni
            > <Victor.Duchovni@...> wrote:
            >
            > > The receiving sites policies are stupid if they don't implement
            > > them sensibly by just returning 4XX responses without penalizing
            > > subsequent transactions.
            >
            > I am sorry to hijack this thread but we have what seems to be the
            > same problem.
            >
            > While using the default Postfix settings (v.2.8.1 on Ubuntu 10.10), we
            > do have trouble to connect with several MTA's (usually
            > smtp1.min-saude.pt and smtp2.min-saude.pt, but sometimes others
            > at .min-saude.pt).
            > The server at smtp3.min-saude.pt never complains, nor do any of
            > the other email MTA at .min-saude.pt whose name do not start with
            > smtpNN.
            >
            > When they refuse our connections, they seem to start shutting down at
            > 25 to 30 RCPT commands, with:
            > "...mx postfix-slow/smtp[4907]: 36BB7818B:
            > to=<some_subscriber@...-saude.pt>,
            > relay=smtp1.min-saude.pt[194.65.151.38]:25, delay=415,
            > delays=414/0.25/0.41/0, dsn=4.0.0, status=deferred (host
            > smtp1.min-saude.pt[194.65.151.38] refused to talk to me: 421 #4.4.5 Too
            > many connections from your host.) "

            Why would this be a response to "too many recipient commands", a single
            message with many recipients is sent over a single connection, unless
            you have set an ill-advised destination recipient limit.

            > To deal with this we are currently using:
            >
            > /etc/postfix/transport
            > .min-saude.pt slow:
            >
            > /etc/postfix/master.cf
            > slow unix - - - - - smtp
            > -o syslog_name=postfix-slow
            > -o smtp_connection_cache_on_demand=no
            > EOT
            >
            > /etc/postfix/main.cf
            > slow_destination_concurrency_failed_cohort_limit = 3 # we give up
            > after getting three 421
            > slow_destination_recipient_limit = 20 # keep it bellow 25

            This increases the number of connections, which is unlikely what you
            want, provided of course you have messages with a large recipient count.

            > slow_destination_rate_delay = 1 # do not know if we really need this

            This limits you to one connection at-a-time.

            > > Have you considered the less aggressive
            > > concurrency feedback controls in Postfix 2.5?
            >
            > Do you think that the following would be a more elegant approach than
            > the above described setting?
            >
            > /etc/postfix/master.cf
            > slow unix - - - - - smtp
            > -o syslog_name=postfix-slow
            > -o smtp_connection_reuse_time_limit=30s
            > EOT
            >
            > /etc/postfix/main.cf
            > slow_initial_destination_concurrency = 2
            > slow_destination_concurrency_limit = 15
            > slow_destination_concurrency_failed_cohort_limit = 5
            > slow_destination_concurrency_positive_feedback = 1/5
            > slow_destination_concurrency_negative_feedback = 1/8

            That depends on how determined the remote site is to damage the
            SMTP eco-system by imposing counter-productive punitive mechanisms
            on legitimate senders. You can certainly try, and report your findings.

            --
            Viktor.
          • Mark Alan
            On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni ... All _recipient_limit parameters are all at their defaults. With the exception of things related to
            Message 5 of 16 , Mar 31, 2011
            • 0 Attachment
              On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni
              <Victor.Duchovni@...> wrote:
              > Why would this be a response to "too many recipient commands", a
              > single message with many recipients is sent over a single connection,
              > unless you have set an ill-advised destination recipient limit.

              All _recipient_limit parameters are all at their defaults. With the
              exception of things related to ciphers and TLS, we try hard to keep the
              default Postfix settings.

              > > /etc/postfix/main.cf
              > > slow_destination_concurrency_failed_cohort_limit = 3 # we give up
              > > after getting three 421
              > > slow_destination_recipient_limit = 20 # keep it bellow 25
              >
              > This increases the number of connections, which is unlikely what you
              > want, provided of course you have messages with a large recipient
              > count.

              It was not obvious to us. The idea was simply to put a limit on each
              burst of messages sent to the slow transport MTA's.

              These messages are related to a low traffic (2-3 messages a month), low
              volume (280 subscribers) mailing list, managed with mlmmj and using
              VERP tagging.
              We have exactly 142 subscribers from subdomains at .min-saude.pt.
              Hardly huge numbers.

              > > slow_destination_rate_delay = 1 # do not know if we really need this
              > This limits you to one connection at-a-time.

              The idea was to have a 1s delay between each message delivered. But, of
              course not knowing if this helped or not.

              > > /etc/postfix/master.cf
              > > slow unix - - - - - smtp
              > > -o syslog_name=postfix-slow
              > > -o smtp_connection_reuse_time_limit=30s

              Should we use only those 2 lines, or should we also add
              -o smtp_connection_cache_on_demand=no

              > > /etc/postfix/main.cf
              > > slow_initial_destination_concurrency = 2
              > > slow_destination_concurrency_limit = 15
              > > slow_destination_concurrency_failed_cohort_limit = 5
              > > slow_destination_concurrency_positive_feedback = 1/5
              > > slow_destination_concurrency_negative_feedback = 1/8
              >
              > That depends on how determined the remote site is to damage the
              > SMTP eco-system by imposing counter-productive punitive mechanisms
              > on legitimate senders.

              Being it the health ministry bureaucracy, I am pretty sure that they
              have the time and resources to be creative at it.
              We know for sure that up until now they did not answer any emails regarding their strange
              mail server policies.

              > You can certainly try, and report your

              We will wait for your opinion on the above
              -o smtp_connection_cache... parameter, to try to those new settings.


              Thank you,

              M.
            • Victor Duchovni
              ... The actual effect is to drive up the number of messages, this parameter limits the number of recipients per message delivery, thus a 100-recipient message
              Message 6 of 16 , Mar 31, 2011
              • 0 Attachment
                On Thu, Mar 31, 2011 at 10:18:52PM +0100, Mark Alan wrote:

                > On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni
                > <Victor.Duchovni@...> wrote:
                > > Why would this be a response to "too many recipient commands", a
                > > single message with many recipients is sent over a single connection,
                > > unless you have set an ill-advised destination recipient limit.
                >
                > All _recipient_limit parameters are all at their defaults. With the
                > exception of things related to ciphers and TLS, we try hard to keep the
                > default Postfix settings.
                >
                > > > /etc/postfix/main.cf
                > > > slow_destination_concurrency_failed_cohort_limit = 3 # we give up
                > > > after getting three 421
                > > > slow_destination_recipient_limit = 20 # keep it bellow 25
                > >
                > > This increases the number of connections, which is unlikely what you
                > > want, provided of course you have messages with a large recipient
                > > count.
                >
                > It was not obvious to us. The idea was simply to put a limit on each
                > burst of messages sent to the slow transport MTA's.

                The actual effect is to drive up the number of messages, this parameter
                limits the number of recipients per message delivery, thus a 100-recipient
                message now gets sent 5 times instead of the default 2.

                > > > /etc/postfix/master.cf
                > > > slow unix - - - - - smtp
                > > > -o syslog_name=postfix-slow
                > > > -o smtp_connection_reuse_time_limit=30s
                >
                > Should we use only those 2 lines, or should we also add
                > -o smtp_connection_cache_on_demand=no

                Connection caching can help reduce connection setup cost, especially
                when using rate delays, though you are then keeping an open connection
                needlessly idle every other second. If connectivity to the MX hosts
                in question is reliable (say they are using load balancers, ...)
                connection caching could be turned off...

                > > > /etc/postfix/main.cf
                > > > slow_initial_destination_concurrency = 2
                > > > slow_destination_concurrency_limit = 15
                > > > slow_destination_concurrency_failed_cohort_limit = 5
                > > > slow_destination_concurrency_positive_feedback = 1/5
                > > > slow_destination_concurrency_negative_feedback = 1/8
                > >
                > > That depends on how determined the remote site is to damage the
                > > SMTP eco-system by imposing counter-productive punitive mechanisms
                > > on legitimate senders.
                >
                > Being it the health ministry bureaucracy, I am pretty sure that they
                > have the time and resources to be creative at it.

                :-)

                > We know for sure that up until now they did not answer any emails regarding their strange
                > mail server policies.
                >
                > > You can certainly try, and report your
                >
                > We will wait for your opinion on the above
                > -o smtp_connection_cache... parameter, to try to those new settings.

                With rate delays in place, disabling connection caching may spread
                the load more evenly, but will reduce performance when one of the
                MX hosts is down. It's a trade-off.

                If you can take a small risk, you may be able to achieve reasonable
                throughput with less hand-tuning, by letting the more advanced feedback
                mechanisms in Postfix adjust the site's negative replies. This depends
                on the site's feedback being reasonably timely and the consequences of
                "overshoot" being transient.

                --
                Viktor.
              • Ultrabug
                ... The result is pretty much that they deny any further email with 4XX codes AND penalize further connections. It just stops accepting any new mail. ... Ok,
                Message 7 of 16 , Apr 1 12:20 AM
                • 0 Attachment
                  On 31/03/2011 18:39, Victor Duchovni wrote:
                  > On Thu, Mar 31, 2011 at 10:15:55AM +0200, Ultrabug wrote:
                  >
                  >> Dear list,
                  >>
                  >> I'm facing a problem where I have to adapt and optimize my smtp servers
                  >> to a host's constraints which are as follow :
                  >>
                  >> - maximum 3 connections to each MX of the host (he has 10 MX so
                  >> potentially I should be able to make 30 connections)
                  >
                  > What happens if you happen to exceed the limit on particular host
                  > among the 10? If it just quickly returns a 4XX code, and does not
                  > penalize future connections, ignore this limit and let Postfix do
                  > what it does by default.
                  >

                  The result is pretty much that they deny any further email with 4XX
                  codes AND penalize further connections. It just stops accepting any new
                  mail.

                  >> - maximum 1000 connections per MX per hour
                  >
                  > Connection caching should help if volume is high enough to worry about
                  > this. Note this is just less than one connection every 3 seconds, but
                  > Postfix caches idle connections for 2 seconds, so if your output rate
                  > is 1200 messages spaced perfectly 3 seconds apart, you lose, but this
                  > is fairly unlikely.
                  >

                  Ok, connection caching I do already and I don't think to fall into this
                  limitation.

                  >> - maximum 100 emails sent per connection
                  >
                  > Postfix has no such limit, instead a connection is retired if idle
                  > for more than 2 seconds, or after 300s (tunable). Again the site
                  > should just return a 4XX response to RSET, and Postfix will drop
                  > the connection and build a new connection, probably to another host.
                  >
                  >> My problem is that this setup is far from optimal compared to the
                  >> limitations and it slows down a lot my email delivery rate to these domains.
                  >>
                  >> Anyone have any tips on how I could do this better please ?
                  >
                  > The receiving sites policies are stupid if they don't implement
                  > them sensibly by just returning 4XX responses without penalizing
                  > subsequent transactions.
                  >
                  > Have you in fact observed that default Postfix settings run into trouble
                  > with this site?

                  Oh yes, it ends up with thousands of mails deferred which in turn would
                  try again at a too high rate so the pile remains deferred etc etc

                  Have you considered the less aggressive concurrency
                  > feedback controls in Postfix 2.5?
                  >
                  > slow_initial_concurrency = 2
                  > slow_destination_concurrency_limit = 15
                  > slow_destination_concurrency_failed_cohort_limit = 5
                  > slow_destination_concurrency_positive_feedback = 1/5
                  > slow_destination_concurrency_negative_feedback = 1/8
                  >

                  This sounds interesting indeed, I didn't understand fully these
                  cohort/feedback options, I'll give them a try !

                  > and if absolutely necessary, in master.cf:
                  >
                  > slow unix - - n - - smtp
                  > -o smtp_connection_reuse_time_limit=30s
                  >
                  > (the remote side starts rejecting traffic consistently instead of
                  > sending 421 for the 100th RSET over a given connection).
                  >

                  Many tanks for your time and help
                • Victor Duchovni
                  ... So instead of gently nudging you to the right sending rate, the DoS their service and require the whole planet to hand-tune transports for their domain,
                  Message 8 of 16 , Apr 1 9:50 AM
                  • 0 Attachment
                    On Fri, Apr 01, 2011 at 09:20:39AM +0200, Ultrabug wrote:

                    > > What happens if you happen to exceed the limit on particular host
                    > > among the 10? If it just quickly returns a 4XX code, and does not
                    > > penalize future connections, ignore this limit and let Postfix do
                    > > what it does by default.
                    > >
                    >
                    > The result is pretty much that they deny any further email with 4XX
                    > codes AND penalize further connections. It just stops accepting any new
                    > mail.

                    So instead of gently nudging you to the right sending rate, the DoS
                    their service and require the whole planet to hand-tune transports for
                    their domain, this is insane, and as much as possible, senders should
                    not play-along with this insanity. I know that some senders will have
                    to jump through hoops for business reasons, but if possible apply some
                    spine.

                    > > Have you considered the less aggressive concurrency
                    > > feedback controls in Postfix 2.5?
                    > >
                    > > slow_initial_concurrency = 2
                    > > slow_destination_concurrency_limit = 15
                    > > slow_destination_concurrency_failed_cohort_limit = 5
                    > > slow_destination_concurrency_positive_feedback = 1/5
                    > > slow_destination_concurrency_negative_feedback = 1/8
                    > >
                    >
                    > This sounds interesting indeed, I didn't understand fully these
                    > cohort/feedback options, I'll give them a try !

                    This really won't help if the remote sites response to exceeding their
                    maximum rate is a sticky refusal to accept further mail.

                    If that's the case, you must psychically (i.e. prior hand-tuning) stay
                    under their limits. The feedback controls assume that the feedback is in
                    the form of transient 421 responses when the connection concurrency or
                    re-use limits are reached. No simple feedback algorithm will dynamically
                    adjust to a feedback control that goes from open to sticky-closed.

                    > > and if absolutely necessary, in master.cf:
                    > >
                    > > slow unix - - n - - smtp
                    > > -o smtp_connection_reuse_time_limit=30s
                    > >
                    > > (the remote side starts rejecting traffic consistently instead of
                    > > sending 421 for the 100th RSET over a given connection).
                    >
                    > Many tanks for your time and help

                    You probably can't do much with my advice, when the receiving system
                    is fubared, your options are limited.

                    --
                    Viktor.
                  • Wietse Venema
                    ... You ll have to use a combination of /etc/postfix/main.cf: slow_destination_rate_delay=x (this forces concurrency == 1) slow_destination_recipient_limit=y
                    Message 9 of 16 , Apr 1 10:39 AM
                    • 0 Attachment
                      Victor Duchovni:
                      > > Many tanks for your time and help
                      >
                      > You probably can't do much with my advice, when the receiving system
                      > is fubared, your options are limited.

                      You'll have to use a combination of

                      /etc/postfix/main.cf:
                      slow_destination_rate_delay=x (this forces concurrency == 1)
                      slow_destination_recipient_limit=y

                      /etc/postfix/master.cf:
                      slow unix - - n - - smtp
                      -o smtp_connection_cache_on_demand=no

                      that stays within the stated limits (which of course may not bear
                      any relationship to reality, but that is a different matter).

                      And these clowns will have to get used to abnormal email delays.
                      I hope this isn't my tax money at work.

                      Wietse
                    • Victor Duchovni
                      ... Of course, if their policy is that they only accept a trickle of mail from sites that are not their important business partners, the correct place to make
                      Message 10 of 16 , Apr 1 10:51 AM
                      • 0 Attachment
                        On Fri, Apr 01, 2011 at 01:39:41PM -0400, Wietse Venema wrote:

                        > Victor Duchovni:
                        > > > Many tanks for your time and help
                        > >
                        > > You probably can't do much with my advice, when the receiving system
                        > > is fubared, your options are limited.
                        >
                        > You'll have to use a combination of
                        >
                        > /etc/postfix/main.cf:
                        > slow_destination_rate_delay=x (this forces concurrency == 1)
                        > slow_destination_recipient_limit=y
                        >
                        > /etc/postfix/master.cf:
                        > slow unix - - n - - smtp
                        > -o smtp_connection_cache_on_demand=no
                        >
                        > that stays within the stated limits (which of course may not bear
                        > any relationship to reality, but that is a different matter).
                        >
                        > And these clowns will have to get used to abnormal email delays.
                        > I hope this isn't my tax money at work.

                        Of course, if their policy is that they only accept a trickle of
                        mail from sites that are not their important business partners,
                        the correct place to make adjustments is in their white-list.

                        If their systems support a whitelist, the OP should find a way to
                        get whitelisted.

                        --
                        Viktor.
                      • Ultrabug
                        ... I see indeed, I shall try this out on monday at work. ... It s not an american public host thanksfully (but french host, maybe that s why) :) ... Many
                        Message 11 of 16 , Apr 1 2:07 PM
                        • 0 Attachment
                          On 01/04/2011 19:39, Wietse Venema wrote:
                          > Victor Duchovni:
                          >>> Many tanks for your time and help
                          >> You probably can't do much with my advice, when the receiving system
                          >> is fubared, your options are limited.
                          > You'll have to use a combination of
                          >
                          > /etc/postfix/main.cf:
                          > slow_destination_rate_delay=x (this forces concurrency == 1)
                          > slow_destination_recipient_limit=y
                          >
                          > /etc/postfix/master.cf:
                          > slow unix - - n - - smtp
                          > -o smtp_connection_cache_on_demand=no
                          >
                          I see indeed, I shall try this out on monday at work.
                          > that stays within the stated limits (which of course may not bear
                          > any relationship to reality, but that is a different matter).
                          >
                          > And these clowns will have to get used to abnormal email delays.
                          > I hope this isn't my tax money at work.
                          It's not an american public host thanksfully (but french host, maybe
                          that's why) :)
                          > Wietse
                          Many thanks to you both for taking time to look on this matter. I'll
                          keep this thread updated if/when I get this working better.

                          Regards
                        • Mark Alan
                          On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni ... Tried the above setup. It does not help. We have much more 421 with this approach than we had with our
                          Message 12 of 16 , Apr 2 10:59 AM
                          • 0 Attachment
                            On Thu, 31 Mar 2011 14:53:11 -0400, Victor Duchovni
                            <Victor.Duchovni@...> wrote:

                            > > /etc/postfix/master.cf
                            > > slow unix - - - - - smtp
                            > > -o syslog_name=postfix-slow
                            > > -o smtp_connection_reuse_time_limit=30s
                            > > EOT
                            > >
                            > > /etc/postfix/main.cf
                            > > slow_initial_destination_concurrency = 2
                            > > slow_destination_concurrency_limit = 15
                            > > slow_destination_concurrency_failed_cohort_limit = 5
                            > > slow_destination_concurrency_positive_feedback = 1/5
                            > > slow_destination_concurrency_negative_feedback = 1/8


                            > (...) You can certainly try, and report your findings.

                            Tried the above setup. It does not help.
                            We have much more 421 with this approach than we had with our former
                            setup.

                            We will now try the following transport settings (based on a
                            recent Wietse sugestion):

                            slow unix - - - - - smtp
                            -o syslog_name=postfix-slow
                            -o default_destination_rate_delay=1s
                            -o default_destination_recipient_limit=20
                            -o smtp_connection_cache_on_demand=no

                            As usual we will be reporting here the results.


                            Thank you for your time and good will.


                            M.
                          • Wietse Venema
                            ... THAT DOES NOT WORK. Please follow the instructions. As documented, the following parameters: xxx_destination_rate_delay xxx_destination_recipient_limit are
                            Message 13 of 16 , Apr 2 3:03 PM
                            • 0 Attachment
                              Mark Alan:
                              > We will now try the following transport settings (based on a
                              > recent Wietse sugestion):
                              >
                              > slow unix - - - - - smtp
                              > -o syslog_name=postfix-slow
                              > -o default_destination_rate_delay=1s
                              > -o default_destination_recipient_limit=20
                              > -o smtp_connection_cache_on_demand=no

                              THAT DOES NOT WORK.

                              Please follow the instructions.

                              As documented, the following parameters:

                              xxx_destination_rate_delay
                              xxx_destination_recipient_limit

                              are implemented by the QUEUE MANAGER not SMTP CLIENT.

                              See my earlier email for what parameters go into main.cf and what
                              go into master.cf.

                              Wietse
                            • Mark Alan
                              On Sat, 2 Apr 2011 18:03:29 -0400 (EDT), Wietse Venema ... I keep forgetting the inner workings of the multiple inner postfix modules. So here it is the
                              Message 14 of 16 , Apr 3 2:26 AM
                              • 0 Attachment
                                On Sat, 2 Apr 2011 18:03:29 -0400 (EDT), Wietse Venema
                                <wietse@...> wrote:

                                > > slow unix - - - - - smtp
                                > > -o syslog_name=postfix-slow
                                > > -o default_destination_rate_delay=1s
                                > > -o default_destination_recipient_limit=20
                                > > -o smtp_connection_cache_on_demand=no
                                > THAT DOES NOT WORK.
                                > Please follow the instructions.
                                > As documented, the following parameters:
                                > xxx_destination_rate_delay
                                > xxx_destination_recipient_limit
                                > are implemented by the QUEUE MANAGER not SMTP CLIENT.

                                I keep forgetting the inner workings of the multiple inner postfix
                                modules.

                                So here it is the revised setting:

                                /etc/postfix/master.cf
                                slow unix - - - - - smtp
                                -o syslog_name=postfix-slow
                                -o smtp_connection_cache_on_demand=no

                                /etc/postfix/main.cf
                                slow_destination_rate_delay = 1s
                                postconf -e 'slow_destination_recipient_limit = 20


                                I will be reporting the results.

                                Thank you very much for your guidance.

                                M.
                              • Mark Alan
                                On Sat, 2 Apr 2011 18:03:29 -0400 (EDT), Wietse Venema ... I keep forgetting the inner workings of the multiple inner postfix modules. So here it is the
                                Message 15 of 16 , Apr 3 2:32 AM
                                • 0 Attachment
                                  On Sat, 2 Apr 2011 18:03:29 -0400 (EDT), Wietse Venema
                                  <wietse@...> wrote:
                                  > > slow unix - - - - - smtp
                                  > > -o syslog_name=postfix-slow
                                  > > -o default_destination_rate_delay=1s
                                  > > -o default_destination_recipient_limit=20
                                  > > -o smtp_connection_cache_on_demand=no
                                  > THAT DOES NOT WORK.
                                  > Please follow the instructions.
                                  > As documented, the following parameters:
                                  > xxx_destination_rate_delay
                                  > xxx_destination_recipient_limit
                                  > are implemented by the QUEUE MANAGER not SMTP CLIENT.

                                  I keep forgetting the inner workings of the multiple inner postfix
                                  modules.

                                  So here it is the revised setting:

                                  /etc/postfix/master.cf
                                  slow unix - - - - - smtp
                                  -o syslog_name=postfix-slow
                                  -o smtp_connection_cache_on_demand=no

                                  /etc/postfix/main.cf
                                  slow_destination_rate_delay = 1s
                                  slow_destination_recipient_limit = 20


                                  I will be reporting the results.

                                  Thank you very much for your guidance.

                                  M.
                                • Victor Duchovni
                                  ... The number of 421 responses is not a good metric for success. The real metric is the resulting average time in the queue for messages that are finally
                                  Message 16 of 16 , Apr 3 12:40 PM
                                  • 0 Attachment
                                    On Sat, Apr 02, 2011 at 06:59:42PM +0100, Mark Alan wrote:

                                    > > > /etc/postfix/master.cf
                                    > > > slow unix - - - - - smtp
                                    > > > -o syslog_name=postfix-slow
                                    > > > -o smtp_connection_reuse_time_limit=30s
                                    > > > EOT
                                    > > >
                                    > > > /etc/postfix/main.cf
                                    > > > slow_initial_destination_concurrency = 2
                                    > > > slow_destination_concurrency_limit = 15
                                    > > > slow_destination_concurrency_failed_cohort_limit = 5
                                    > > > slow_destination_concurrency_positive_feedback = 1/5
                                    > > > slow_destination_concurrency_negative_feedback = 1/8
                                    >
                                    > > (...) You can certainly try, and report your findings.
                                    >
                                    > Tried the above setup. It does not help.
                                    > We have much more 421 with this approach than we had with our former
                                    > setup.

                                    The number of 421 responses is not a good metric for success. The real
                                    metric is the resulting average time in the queue for messages that are
                                    finally delivered.

                                    A dymamic feedback mechanism that is able to go faster, will necessarily
                                    elicit 421 responses at a stead rate, to keep it from reaching peak
                                    capacity. Naturally, a statically hand-tuned configuration that stays
                                    under the remote caps will not elicit 421 responses, but it is likely
                                    to stay below the optimal throughput.

                                    With sites whose 421 responses are NOT "sticky", the feedback controls
                                    will in a variety of cases lead to a nearly optimal throughput with
                                    some messages retried at a second MX or deferred, but the overall
                                    delay can be smaller than conservative upper-bounds on concurrency
                                    and message rates.

                                    --
                                    Viktor.
                                  Your message has been successfully submitted and would be delivered to recipients shortly.