Loading ...
Sorry, an error occurred while loading the content.

Re: Balancing destination concurrency + rate delay

Expand Messages
  • Viktor Dukhovni
    ... Yes, they are willing to cripple SMTP and expect everyone to cope, because they are too big to ignore. :-) Rumour has it that established proffesional
    Message 1 of 11 , Jan 18, 2013
    • 0 Attachment
      On Fri, Jan 18, 2013 at 09:49:34AM -0800, Steve Jenkins wrote:

      > Agreed - but Yahoo is really the only one we're having issues with (even
      > after complying with all their guidelines here):
      >
      > http://help.yahoo.com/kb/index?page=content&y=PROD_MAIL_ML&locale=en_US&id=SLN3435
      >

      Yes, they are willing to cripple SMTP and expect everyone to cope,
      because they are too big to ignore. :-)

      Rumour has it that established proffesional bulk-senders typically
      don't turn up new IP addresses (no existing good reputation) to
      full capacity overnight. Rather, their sending software gradually
      ramps-up load to a new address allowing its reputation to build-up
      over time (the first few weeks may be at a low load, then add more,
      ...).

      Once your IP reputation is good, you should be able to use reasonable
      concurrency... You may want disable the demand connection cache
      for Yahoo. This cache is primarily useful for destinations that
      sporadically have some of their MX records unreachable at the TCP
      layer.

      Yahoo's MX SMTP pool essentially never has an IP address that is
      completely down. Instead, you can set a low "smtp_connect_timeout",
      just in case.

      master.cf:
      yahoo unix - - n - - smtp
      -o smtp_connect_timeout=$yahoo_connect_timeout
      -o smtp_connection_cache_on_demand=$yahoo_connection_cache_on_demand

      main.cf:
      # Good enough for light RTT to the moon
      yahoo_connect_timeout=3s

      # Don't annoy them with connection reuse.
      yahoo_connection_cache_on_demand=no

      At that point you may not even need rate delays, just set a modest
      concurrency, and typical SMTP transaction latency of 0.2-0.5s (
      with spam checks, RBL lookups, ...) will give you at most 2-5
      messages per unit concurrency per second.

      --
      Viktor.
    • Steve Jenkins
      On Fri, Jan 18, 2013 at 11:36 AM, Viktor Dukhovni
      Message 2 of 11 , Jan 18, 2013
      • 0 Attachment
        On Fri, Jan 18, 2013 at 11:36 AM, Viktor Dukhovni <postfix-users@...> wrote:
        Yes, they are willing to cripple SMTP and expect everyone to cope,
        because they are too big to ignore. :-)

        Sad, but true.
         
        At that point you may not even need rate delays, just set a modest
        concurrency, and typical SMTP transaction latency of 0.2-0.5s (
        with spam checks, RBL lookups, ...) will give you at most 2-5
        messages per unit concurrency per second.

        Would this be considered "modest?"

        yahoo_initial_destination_concurrency = 1
        yahoo_destination_concurrency_limit = 4
        yahoo_destination_recipient_limit = 2
        #yahoo_destination_rate_delay = 2s
        yahoo_connect_timeout=3s
        yahoo_connection_cache_on_demand=no

        Thx,

        SteveJ
         
      • Viktor Dukhovni
        ... You ll have to figure that out for yourself mostly, my gut reaction is that the recipient limit is too low, if you ever did have a multi-recipient message,
        Message 3 of 11 , Jan 18, 2013
        • 0 Attachment
          On Fri, Jan 18, 2013 at 07:46:45PM -0800, Steve Jenkins wrote:

          > > At that point you may not even need rate delays, just set a modest
          > > concurrency, and typical SMTP transaction latency of 0.2-0.5s (
          > > with spam checks, RBL lookups, ...) will give you at most 2-5
          > > messages per unit concurrency per second.
          > >
          >
          > Would this be considered "modest?"
          >
          > yahoo_initial_destination_concurrency = 1
          > yahoo_destination_concurrency_limit = 4
          > yahoo_destination_recipient_limit = 2
          > #yahoo_destination_rate_delay = 2s
          > yahoo_connect_timeout=3s
          > yahoo_connection_cache_on_demand=no

          You'll have to figure that out for yourself mostly, my gut reaction
          is that the recipient limit is too low, if you ever did have a
          multi-recipient message, you'd split it into too many pieces.
          Rather, set something in the 10-20 range (the RFC requires
          servers to support at least 100, and Postfix by default sends
          up to 50 and takes in up to 1000).

          After burning in your IP at low volume, with mail typically not
          delayed by Yahoo at all, I would try to grow the (destination aka
          connection) concurrency limit to something in the 5-10 range.

          The main thing that will matter is "burning in" a new IP address,
          once they've learned you're not evil, I would hope their rate limits
          won't get in your way at all.

          --
          Viktor.
        Your message has been successfully submitted and would be delivered to recipients shortly.