Loading ...
Sorry, an error occurred while loading the content.

The Yahoo trickle

Expand Messages
  • Joey Prestia
    Hi all, I am familiar with yahoo being difficult to send email to, as i m sure most all of us here are. I am now faced with needing to increase my server s
    Message 1 of 6 , Sep 2, 2012
    • 0 Attachment
      Hi all,

      I am familiar with yahoo being difficult to send email to, as i'm sure
      most all of us here are. I am now faced with needing to increase my
      server's throughput to yahoo's MTA's. I was scouring the net and what
      caught my attention on the postfix mailing list archives was a reply
      from *Victor Duchovni*
      http://tech.groups.yahoo.com/group/postfix-users/message/244843?threaded=1&p=3

      I am trying to figure out how to plugin my values to get the equation
      correct so I could calculate the what maximal throughput per unit of
      concurrency msgs/sec? It seems like I am only sending roughly 800 -
      1,000 emails an hour to yahoo. Settings for my yahoo transport are


      yahoo_destination_concurrency_limit = 20

      yahoo_destination_rate_delay = 1s

      yahoo_destination_recipient_limit = 20



      I found this cool one liner thanks to Victor


      $ perl -lne '
      m{\A\S+T(\S+).{5} \S+ \S+: \w+: to=<\S+\@yahoo\.com>, (?:orig_to=\S+,
      )?relay=\S+, (?:conn_use=\d+, )?delay=\S+,
      delays=[\d.]+/[\d.]+/([\d.]+)/([\d.])+,} or next;
      $c = 0.95 * $c + 0.05 * $2;
      $d = 0.95 * $d + 0.05 * $3;
      if (++$i % 100 == 0) {
      printf "%s %5.2f %5.2f\n", $1, $c, $d;
      }' maillog



      18:43:28 0.07 4.43

      18:47:20 0.52 3.71

      18:51:01 0.13 3.13

      18:54:21 0.05 4.73

      18:57:49 0.08 3.32

      19:01:06 0.05 4.63

      19:04:32 0.06 4.84

      19:07:55 0.09 3.91


      Can anyone offer any guidance on what direction I need to go?

      Thanks in advance,

      Joey
    • Stan Hoeppner
      ... [snip] ... Start here: http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html -- Stan
      Message 2 of 6 , Sep 3, 2012
      • 0 Attachment
        On 9/2/2012 10:07 PM, Joey Prestia wrote:
        > Hi all,
        >
        > I am familiar with yahoo being difficult to send email to

        [snip]

        > Can anyone offer any guidance on what direction I need to go?

        Start here:
        http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html

        --
        Stan
      • Joey Prestia
        ... Hi Stan, I did that some time ago. I am white listed with them and have published SPF Records and use DKIM to sign all outgoing mail. I am not receiving
        Message 3 of 6 , Sep 3, 2012
        • 0 Attachment
          On 9/3/2012 3:50 AM, Stan Hoeppner wrote:
          > On 9/2/2012 10:07 PM, Joey Prestia wrote:
          >> Hi all,
          >>
          >> I am familiar with yahoo being difficult to send email to
          >
          > [snip]
          >
          >> Can anyone offer any guidance on what direction I need to go?
          >
          > Start here:
          > http://help.yahoo.com/l/us/yahoo/mail/postmaster/bulkv2.html
          >

          Hi Stan,

          I did that some time ago. I am white listed with them and have published
          SPF Records and use DKIM to sign all outgoing mail. I am not receiving
          any error codes only (250 ok dirdel) in my logs.

          I will recheck with yahoo on my bulk sending status and validate it
          indeed as it should be.

          Joey
        • Viktor Dukhovni
          ... You have serialized deliveries to Yahoo, they happen one at a time. Given that each delivery takes ~5s, there is not much point in doing that, you can
          Message 4 of 6 , Sep 3, 2012
          • 0 Attachment
            On Sun, Sep 02, 2012 at 08:07:21PM -0700, Joey Prestia wrote:

            > yahoo_destination_concurrency_limit = 20

            This setting is trumpted by the setting below:

            > yahoo_destination_rate_delay = 1s

            You have serialized deliveries to Yahoo, they happen one at a time.
            Given that each delivery takes ~5s, there is not much point in
            doing that, you can instead set a low concurrency, and get a bunch
            more throughput by not setting an explicit rate limit. (perhaps
            2 deliveries per second with a concurrency of 10, rather than 1
            delivery every 5 seconds).

            For more throughput, you need more IP addresses, perhaps even in
            distint address blocks, ...

            Sadly, Yahoo discriminates the Postfix connection cache which limits
            connection re-use by time rather than delivery count. Limiting by
            delivery count behaves poorly when one or more of the MX hosts for
            a site is slower than the rest, it becomes a connection "attractor",
            so Postfix uses a better strategy.

            You can just disable connection caching with Yahoo, they rarely
            have unreachable MX hosts, your deliveries are just as slow whether
            connections are cached or not.

            --
            Viktor.
          • Joey Prestia
            ... So what I would need then is in main.cf and in master.cf in my transport would be this? yahoo_destination_concurrency_limit = 10 -o
            Message 5 of 6 , Sep 3, 2012
            • 0 Attachment
              On 9/3/2012 10:43 AM, Viktor Dukhovni wrote:
              > On Sun, Sep 02, 2012 at 08:07:21PM -0700, Joey Prestia wrote:
              >
              >> yahoo_destination_concurrency_limit = 20
              >
              > This setting is trumpted by the setting below:
              >
              >> yahoo_destination_rate_delay = 1s
              >
              > You have serialized deliveries to Yahoo, they happen one at a time.
              > Given that each delivery takes ~5s, there is not much point in
              > doing that, you can instead set a low concurrency, and get a bunch
              > more throughput by not setting an explicit rate limit. (perhaps
              > 2 deliveries per second with a concurrency of 10, rather than 1
              > delivery every 5 seconds).

              So what I would need then is in main.cf and in master.cf in my transport
              would be this?

              yahoo_destination_concurrency_limit = 10


              -o smtp_connection_cache_on_demand=no


              >
              > For more throughput, you need more IP addresses, perhaps even in
              > distint address blocks, ...
              >
              > Sadly, Yahoo discriminates the Postfix connection cache which limits
              > connection re-use by time rather than delivery count. Limiting by
              > delivery count behaves poorly when one or more of the MX hosts for
              > a site is slower than the rest, it becomes a connection "attractor",
              > so Postfix uses a better strategy.
              >

              I thought we were making good use of our connection caching?

              > You can just disable connection caching with Yahoo, they rarely
              > have unreachable MX hosts, your deliveries are just as slow whether
              > connections are cached or not.
              >
            • Stan Hoeppner
              ... Connection caching needs two good dance partners to work effectively. In this case Yahoo is stepping on Postfix toes. -- Stan
              Message 6 of 6 , Sep 3, 2012
              • 0 Attachment
                On 9/3/2012 5:44 PM, Joey Prestia wrote:
                > On 9/3/2012 10:43 AM, Viktor Dukhovni wrote:

                >> Sadly, Yahoo discriminates the Postfix connection cache which limits
                >> connection re-use by time rather than delivery count. Limiting by
                >> delivery count behaves poorly when one or more of the MX hosts for
                >> a site is slower than the rest, it becomes a connection "attractor",
                >> so Postfix uses a better strategy.

                > I thought we were making good use of our connection caching?

                Connection caching needs two good dance partners to work effectively.
                In this case Yahoo is stepping on Postfix' toes.

                --
                Stan
              Your message has been successfully submitted and would be delivered to recipients shortly.