Loading ...
Sorry, an error occurred while loading the content.

Re: rate limit issue

Expand Messages
  • Wietse Venema
    ... This DOES NOT limit your delivery RATE!! This limits only the delivery CONCURRENCY. To limit the delivery RATE, see
    Message 1 of 11 , May 1, 2010
    • 0 Attachment
      Gary Smith:
      > Lately I have found that my outgoing queues are getting a little clogged for yahoo and sbcglobal.net. This usually coincides with a bulk set of news letters sent out from a couples clients. Typically we are seeing that they dump about 2000msg/per batch, with no more than one batch per week (usually on Friday nights). The problem is they do it in one fatal swoop. When this happens, we seem to get rate limited from yahoo/sbcglobal for a few hours, which directly affects our other users.
      >
      > Some time ago I put a rate limiter in place for AOL/yahoo/sbcglobal/gmail so we wouldn't be bombarding but I don't think that it's being honored, probably because I missed something. When I do a postconf I don't see my rate limiter, which is in main.cf, listed there. I'm not sure if that's by design or part of a misconfiguration on my part. Mail goes out from different sets of servers, with the same public IP, so rate limiting will only affect the bulk mail queue.
      >
      > Is there something obvious that I missed?
      >
      > master.cf:
      > ratelimit unix - - n - 3 smtp
      >
      > rate_limit_transport:
      > aol.com ratelimit:
      > yahoo.com ratelimit:
      > sbcglobal.net ratelimit:
      > gmail.com ratelimit:
      >
      > main.cf:
      > ratelimit_destination_concurrency_failed_cohort_limit = 100
      > ratelimit_destination_concurrency_limit = 10

      This DOES NOT limit your delivery RATE!!

      This limits only the delivery CONCURRENCY.

      To limit the delivery RATE, see
      http://www.postfix.org/postconf.5.html#transport_destination_rate_delay.


      Wietse
    • Gary Smith
      ... Looking into it now. Thanks for the pointer Wietse. If I m running multiple outgoing relays, does it make sense to have some type of concurrency rate for
      Message 2 of 11 , May 1, 2010
      • 0 Attachment
        > This DOES NOT limit your delivery RATE!!
        >
        > This limits only the delivery CONCURRENCY.
        >
        > To limit the delivery RATE, see
        > http://www.postfix.org/postconf.5.html#transport_destination_rate_delay.
        >

        Looking into it now. Thanks for the pointer Wietse. If I'm running multiple outgoing relays, does it make sense to have some type of concurrency rate for the outgoing messages in this case?
      • Noel Jones
        ... As a general rule, reducing concurrency to the usual problem domains is enough to keep mail flowing at a reasonable rate (which is one reason I didn t
        Message 3 of 11 , May 1, 2010
        • 0 Attachment
          On 5/1/2010 9:43 PM, Gary Smith wrote:
          >> This DOES NOT limit your delivery RATE!!
          >>
          >> This limits only the delivery CONCURRENCY.
          >>
          >> To limit the delivery RATE, see
          >> http://www.postfix.org/postconf.5.html#transport_destination_rate_delay.
          >>
          >
          > Looking into it now. Thanks for the pointer Wietse. If I'm running multiple outgoing relays, does it make sense to have some type of concurrency rate for the outgoing messages in this case?
          >
          >

          As a general rule, reducing concurrency to the usual problem
          domains is enough to keep mail flowing at a reasonable rate
          (which is one reason I didn't address the concurrency vs. rate
          delay issue). It's common for them to temporarily 4xx some of
          the mail but eventually it all should go through, even with
          default postfix settings.

          You should sign up for "feedback loops" at each mail provider
          if you haven't already. This may or may not give preference
          to your mail, but is a necessary first step if you're doing
          any kind of bulk mail.

          If that still doesn't help, then implement the rate delay as
          explained in the link Wietse gave you above. Read the link
          carefully; the intent of the rate delay feature is to
          significantly slow down mail delivery to the target domain.
          And some domains will *still* 4xx some of the mail.

          As far as multiple outgoing relays is concerned, the answer is
          "maybe". The only way to know what works for *your* situation
          is to try; first with the default settings, then reduce
          concurrency to see if long-term delivery is any better. If the
          average throughput of a mail run is significantly less than 1
          delivery per second, implementing rate delays *may* allow you
          to send more mail by not triggering evasive action at the
          target mail server.

          -- Noel Jones
        • Gary Smith
          ... Just to make sure I m reading the docs correctly, in my case where I use the transport of ratelimit (as per the former email) I would be adding
          Message 4 of 11 , May 1, 2010
          • 0 Attachment
            > >> This DOES NOT limit your delivery RATE!!
            > >>
            > >> This limits only the delivery CONCURRENCY.
            > >>
            > >> To limit the delivery RATE, see
            > >> http://www.postfix.org/postconf.5.html#transport_destination_rate_delay.
            > >>
            ...
            > If that still doesn't help, then implement the rate delay as
            > explained in the link Wietse gave you above. Read the link
            > carefully; the intent of the rate delay feature is to
            > significantly slow down mail delivery to the target domain.
            > And some domains will *still* 4xx some of the mail.
            >

            Just to make sure I'm reading the docs correctly, in my case where I use the transport of ratelimit (as per the former email) I would be adding ratelimit_destination_rate_delay to main.cf. And if my understanding is correct, these are global settings so I shouldn't set them on the transport in master.cf.

            Can you guys confirm that this assumption is correct?
          • Noel Jones
            yes. -- Noel Jones
            Message 5 of 11 , May 1, 2010
            • 0 Attachment
              yes.
              -- Noel Jones

              "Gary Smith" <gary.smith@...> wrote:

              >> >> This DOES NOT limit your delivery RATE!!
              >> >>
              >> >> This limits only the delivery CONCURRENCY.
              >> >>
              >> >> To limit the delivery RATE, see
              >> >> http://www.postfix.org/postconf.5.html#transport_destination_rate_delay.
              >> >>
              >...
              >> If that still doesn't help, then implement the rate delay as
              >> explained in the link Wietse gave you above. Read the link
              >> carefully; the intent of the rate delay feature is to
              >> significantly slow down mail delivery to the target domain.
              >> And some domains will *still* 4xx some of the mail.
              >>
              >
              >Just to make sure I'm reading the docs correctly, in my case where I use the transport of ratelimit (as per the former email) I would be adding ratelimit_destination_rate_delay to main.cf. And if my understanding is correct, these are global settings so I shouldn't set them on the transport in master.cf.
              >
              >Can you guys confirm that this assumption is correct?
            Your message has been successfully submitted and would be delivered to recipients shortly.