Loading ...
Sorry, an error occurred while loading the content.

strange delay in pre-queue phase

Expand Messages
  • Oliver Schonrock
    I have a postfix system which is running fine. However the log reports a constant pre-queue (incl message transmission) delay (the a component of the log
    Message 1 of 4 , Jul 1, 2011
    • 0 Attachment
      I have a postfix system which is running fine.

      However the log reports a constant pre-queue (incl message transmission)
      delay (the "a" component of the log entry) of ~0.1s even for a tiny
      message zero load and local 1GB connection.

      This might be "normal" except that the delay reduces to ~0.01s when the
      same mail is sent by a different client?!

      To reduce to the simplest possible I am sending the mail like so:

      echo "hello" | sendmail user@...

      When I do this from the machine that postfix is running on (ie using the
      postfix sendmail binary replacement) or another local machine that has
      the postfix binary replacement intalled, then delay is very very small
      0.01s.

      However...
      When sending it from a machine which has Sendmail installed and I use
      the sendmail "client" binary which comes with that the delay is 10x
      greater ~0.1s.

      I also get the larger delay when sending using a userland php library
      (my actually use case).

      When using my php library I was able to isolate the delay to the
      END_OF_DATA command ie

      <CR><LF>.<CR>LF>

      Postfix's response to that command takes 0.1s if using the "Sendmail"
      client or the php library. but takes only 0.01s when using the postfix
      sendmail replacement library (note this is true when run from the host
      that postfix server is on or a machine which is on local Gbit LAN).

      Feels like some buffer is hanging waiting for more bytes and then times
      out..?

      I increased the verbosity of the log but could see no difference between
      the use cases.

      Help please!? Ideas for narrowing it down further..

      Oliver
    • Victor Duchovni
      ... It the sending system s TCP stack that delays . after the message body, when the sending system does not implement SMTP PIPELINING, or does not make use
      Message 2 of 4 , Jul 1, 2011
      • 0 Attachment
        On Fri, Jul 01, 2011 at 06:12:09PM +0100, Oliver Schonrock wrote:

        > When using my php library I was able to isolate the delay to the
        > END_OF_DATA command ie
        >
        > <CR><LF>.<CR>LF>
        >
        > Postfix's response to that command takes 0.1s if using the "Sendmail"
        > client or the php library. but takes only 0.01s when using the postfix
        > sendmail replacement library (note this is true when run from the host that
        > postfix server is on or a machine which is on local Gbit LAN).

        It the sending system's TCP stack that delays "." after the message body,
        when the sending system does not implement SMTP PIPELINING, or does not
        make use of all available SMTP PIPELINING opportunities. The Nagle algorithm
        in TCP will delay a second (small) write when a previous write is not yet
        acknowledged.

        Use Postfix, it will not suffer unnecessary Nagle delays.

        --
        Viktor.
      • Wietse Venema
        ... This means that some client implementation does small writes back-to-back, and therefore the client suffers from Nagle delays. ... The client
        Message 3 of 4 , Jul 1, 2011
        • 0 Attachment
          Oliver Schonrock:
          > However...
          > When sending it from a machine which has Sendmail installed and I use
          > the sendmail "client" binary which comes with that the delay is 10x
          > greater ~0.1s.

          This means that some client implementation does small writes
          back-to-back, and therefore the client suffers from Nagle delays.

          > I also get the larger delay when sending using a userland php library
          > (my actually use case).
          >
          > When using my php library I was able to isolate the delay to the
          > END_OF_DATA command ie
          >
          > <CR><LF>.<CR>LF>

          The client implementation does small writes back-to-back, i.e. it
          sends the last portion of the message in one write operation and
          then it sends <CR><LF>.<CR>LF> in another write operation.

          This means that the client implementation does small writes
          back-to-back, and therefore the client suffers from Nagle delays.

          The fix is for the client to append <CR><LF>.<CR>LF> to the message
          content before writing it to the network.

          The workaround is for the client to turn off Nagle delays.

          Wietse
        • Oliver Schonrocks
          ... Thanks. That was it for sure. Just for the benefit of others: - it was hard for me to change the php client library, although I have posted a bug for Swift
          Message 4 of 4 , Jul 4, 2011
          • 0 Attachment
            On 01/07/11 19:24, Wietse Venema wrote:
            > The fix is for the client to append<CR><LF>.<CR>LF> to the message
            > content before writing it to the network.
            >
            > The workaround is for the client to turn off Nagle delays.

            Thanks. That was it for sure.

            Just for the benefit of others:

            - it was hard for me to change the php client library, although I have
            posted a bug for Swift Mailer
            http://swiftmailer.lighthouseapp.com/projects/21527-swift-mailer/tickets/204-nagle-delays-affect-mail-sending-throughput

            - it was also hard to change the client to turn off Nagle delays,
            because Swift Mailer is using the wrong php library to allow that.

            So I changed the system wide Nagle behaviour on the server, which is a
            freebsd system.

            http://lists.freebsd.org/pipermail/freebsd-stable/2008-June/043325.html

            sysctl net.inet.tcp.delacktime=10
            # (default 100ms)

            Which reduced 90% of the delay.

            Not ideal, but fine in this case, since most of the traffic to that
            hosts is LAN or very high speed WAN.

            Thanks again.

            --
            Oliver Schonrock
          Your message has been successfully submitted and would be delivered to recipients shortly.