Loading ...
Sorry, an error occurred while loading the content.

Re: Problem with slow transport map

Expand Messages
  • Wietse Venema
    ... As documented -D attempts to start a debugger process when the SMTP client process is created. This process will handle subsequent delivery requests
    Message 1 of 7 , Sep 23, 2013
    • 0 Attachment
      Matt - Opem Solutions:
      > [MASTER.CF FILE]
      >
      > smtpslow unix - - n - 1 smtp -D
      >
      > -o syslog_name=postfix-smtp-slow
      >
      > -o max_use=1
      >
      >
      >
      > The -D (debug) is a bit of a dirty hack as it basically calls a sleep for 6
      > seconds between messages to ensure it doesn't hit the external hosts message
      > per hour limit.

      As documented -D attempts to start a debugger process when the SMTP
      client process is created. This process will handle subsequent
      delivery requests without attempting to start another debugger
      process. Therefore -D is not a good way to force delays between
      deliveries.

      To force delays between deliveries to the same destination, see
      http://www.postfix.org/postconf.5.html#transport_destination_rate_delay
      (requires Postfix 2.5 or later).

      Wietse
    • Matt - Opem Solutions
      Hi Noel/ Wietse, I was just about to reply i had worked it out when your email came in changed it to 2 and it all worked: smtpslow_destination_recipient_limit
      Message 2 of 7 , Sep 23, 2013
      • 0 Attachment
        Hi Noel/ Wietse,

        I was just about to reply i had worked it out when your email came in
        changed it to 2 and it all worked:

        smtpslow_destination_recipient_limit = 2

        # set this to 2 will take care of the domain. If you set it to 1, throttling
        will work only if you send mails to the same recipient and not for the same
        recipient domain.

        The debugger sleep 6 is overall for both normal smtp relay and smtpslow
        relay to ensure the server never hits the messages per hour limit of the
        external relay server.

        Ive removed the max use (i think i had stuck it in there when testing the
        server previously)

        As per Wietse

        smtp_destination_rate_delay = 6 as an alternative to using the debug
        command.

        Am i right in thinking as the smtpslow then uses smtp for delivery it will
        effect both, as in ensure all emails get a 6 second delay.

        Thank you both for your quick reply.

        Best Regards

        Matt.

        -----Original Message-----
        From: owner-postfix-users@...
        [mailto:owner-postfix-users@...] On Behalf Of Noel Jones
        Sent: 23 September 2013 2:26 PM
        To: postfix-users@...
        Subject: Re: Problem with slow transport map

        On 9/23/2013 7:17 AM, Matt - Opem Solutions wrote:
        > Hi,
        >
        >
        >
        > Ive a postfix server which is used to relay emails to an external smtp
        > server, this was done to prevent the receiving smtp server from being
        > flooeded by to many messages per hour which i did by using the debug
        > command and a sleep 6, in addition to this i added a transport map to
        > slow down delivery even more for certain hosts/recipient domains:
        >
        >
        >
        > [MAIN.CF FILE]
        >
        > transport_maps = hash:/etc/postfix/transport
        >
        > smtpslow_destination_rate_delay = 310s
        >
        > smtpslow_destination_concurrency_limit = 1
        >
        > smtpslow_destination_recipient_limit = 1
        >

        When you set the recipient limit to 1, the delay is applied to deliveries to
        the SAME RECIPIENT rather than deliveries to the SAME DOMAIN. This is
        probably not what you intend. See:
        http://www.postfix.org/postconf.5.html#default_destination_rate_delay

        >
        >
        > debugger_command = /bin/sleep 6

        This shouldn't be necessary after you increase the recipient limit.

        > -o max_use=1

        This shouldn't be necessary either.



        -- Noel Jones



        >
        >
        >
        > [TRANSPORT FILE]
        >
        > yahoo.com smtpslow:[external.server.com]:25
        >
        > yahoo.co.uk smtpslow:[external.server.com]:25
        >
        > googlemail.com smtpslow:[external.server.com]:25
        >
        > gmail.com smtpslow:[external.server.com]:25
        >
        > hotmail.com smtpslow:[external.server.com]:25
        >
        > hotmail.co.uk smtpslow:[external.server.com]:25
        >
        > outlook.com smtpslow:[external.server.com]:25
        >
        > outlook.co.uk smtpslow:[external.server.com]:25
        >
        > aol.co.uk smtpslow:[external.server.com]:25
        >
        > aol.com smtpslow:[external.server.com]:25
        >
        > comcast.com smtpslow:[external.server.com]:25
        >
        > live.com smtpslow:[external.server.com]:25
        >
        > live.co.uk smtpslow:[external.server.com]:25
        >
        > msn.com smtpslow:[external.server.com]:25
        >
        > msn.co.uk smtpslow:[external.server.com]:25
        >
        > sbcglobal.net smtpslow:[external.server.com]:25
        >
        > verizon.net smtpslow:[external.server.com]:25
        >
        > bellsouth.net smtpslow:[external.server.com]:25
        >
        > yahoo.ca smtpslow:[external.server.com]:25
        >
        > cox.net smtpslow:[external.server.com]:25
        >
        > ymail.com smtpslow:[external.server.com]:25
        >
        > btinternet.com smtpslow:[external.server.com]:25
        >
        > btinternet.co.uk smtpslow:[external.server.com]:25
        >
        > btopenworld.co.uk smtpslow:[external.server.com]:25
        >
        > btopenworld.com smtpslow:[external.server.com]:25
        >
        > mail.com smtpslow:[external.server.com]:25
        >
        >
        >
        > The smtpslow is set to use one thread only in master.cf
        >
        >
        >
        > [MASTER.CF FILE]
        >
        > smtpslow unix - - n - 1 smtp -D
        >
        > -o syslog_name=postfix-smtp-slow
        >
        > -o max_use=1
        >
        >
        >
        > The -D (debug) is a bit of a dirty hack as it basically calls a sleep
        > for 6 seconds between messages to ensure it doesn't hit the external
        > hosts message per hour limit.
        >
        >
        >
        > This all works fine, however ive noticed a big problem
        >
        >
        >
        > If i send 3 emails to my hotmail.com address they all rightly have a
        > 5 minute (310s) delay between them, however if i send an email to 2
        > different hotmail.com address and send 3 messages per recipient it
        > does correctly insert the 310s delay between the recipients messages.
        > However the issue is its trying to deliver to both recipients at the
        > same time, ie
        >
        >
        >
        > Emails spooled
        >
        > Email 1 to both reclipents sent (with the 6 second debug gap)
        >
        > Wait 310s
        >
        > Email 2 to both recipients sent (with the 6 second debug gap)
        >
        > Wait 310s
        >
        > Email 3 to both recipients sent (with the 6 second debug gap)
        >
        >
        >
        > Where as i expected it to instead work like:
        >
        >
        >
        > Emails spooled
        >
        > Email 1 to recipient 1 sent
        >
        > Wait 310s
        >
        > Email 1 to recipient 2 sent
        >
        > Wait 310s
        >
        > Email 2 to recipient 1 sent
        >
        > Etc
        >
        >
        >
        > How can i adjust the config so that instead of matching the recipients
        > its matching the domains as if 50 emails go out to hotmail.com at the
        > same time to different recipients it doesn't try and delvier them all
        > at the same time.
        >
        >
        >
        > Best Regards
        >
        >
        >
        > Matt.
        >
      • Wietse Venema
        ... Use smtpslow_destination_rate_delay = 6 , and get rid of the debugger hack. Wietse
        Message 3 of 7 , Sep 23, 2013
        • 0 Attachment
          Matt - Opem Solutions:
          > As per Wietse
          >
          > smtp_destination_rate_delay = 6 as an alternative to using the debug
          > command.

          Use "smtpslow_destination_rate_delay = 6", and get rid of the
          debugger hack.

          Wietse
        • Matt - Opem Solutions
          The problem is i need to 6 second slow down for both smtp and smtpslow, whilst smtpslow would slow down using this it wouldn t slow down smtp as well. The
          Message 4 of 7 , Sep 23, 2013
          • 0 Attachment
            The problem is i need to 6 second slow down for both smtp and smtpslow,
            whilst smtpslow would slow down using this it wouldn't slow down smtp as
            well.

            The debug hack although dirty was the only way i could to enforce it on both
            transits and all domains, i set:

            smtp_destination_rate_delay = 60s
            smtp_destination_concurrency_limit = 1
            smtp_destination_recipient_limit = 1

            which is for the standard (non slow delivery) and spooled in two messages
            (that are not in the smtpslow transport map) and both got delivered
            instantly.

            Regards

            Matt.


            -----Original Message-----
            From: owner-postfix-users@...
            [mailto:owner-postfix-users@...] On Behalf Of Wietse Venema
            Sent: 23 September 2013 2:48 PM
            To: Postfix users
            Subject: Re: Problem with slow transport map

            Matt - Opem Solutions:
            > As per Wietse
            >
            > smtp_destination_rate_delay = 6 as an alternative to using the debug
            > command.

            Use "smtpslow_destination_rate_delay = 6", and get rid of the debugger hack.

            Wietse
          • Wietse Venema
            ... Then, use both, and set each delay to an appropriate value. Note that setting _rate_delay on one transport WILL NOT affect deliveries made with the other
            Message 5 of 7 , Sep 23, 2013
            • 0 Attachment
              Matt - Opem Solutions:
              > The problem is i need to 6 second slow down for both smtp and smtpslow,
              > whilst smtpslow would slow down using this it wouldn't slow down smtp as
              > well.

              Then, use both, and set each delay to an appropriate value.
              Note that setting _rate_delay on one transport WILL NOT affect
              deliveries made with the other transport).

              > smtp_destination_recipient_limit = 1

              That is a mistake. As documented that controls the delay
              for deliveries to the same recipient only.

              Wietse
            Your message has been successfully submitted and would be delivered to recipients shortly.