Loading ...
Sorry, an error occurred while loading the content.
 

Postfix performance issue

Expand Messages
  • Teh Kim Chooi
    Hi guys, i recently just setup a high volume postfix server, still in testing mode before the server go for live, OS rhel 5.5 and postfix version 2.3.3 server
    Message 1 of 7 , May 31, 2010
      Hi guys,

      i recently just setup a high volume postfix server, still in testing mode before the server go for live, OS rhel 5.5 and postfix version 2.3.3

      server with 1 quad core, 8gb ram OS on mirror disk, /var/spool/postfix in 1+0 6 disks, all is SAS 15k disk.

      my postfix configuration file will be all default setting due to still on testing.

      for my testing, i  run the smtp-source for the stress test

      time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t user1@... localhost:25

      abc.com is the local server, cde.com which i configure on another server to receive the message within same lan.

      result is 0.5 second which looks  fine,

      BUT when i use the server IP instead of localhost, my performance drop

      time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t user1@... 192.168.1.10:25

      result is 5 seconds, which almost 10 times more.

      the server now only run smtp postfix service, i stop mostly unwanted services from the server, and i never enable any spam filter, cause this server will be mainly sending email from a single source.

      what can cause weird behavior ? please advice....

      Regards,
      Teh
    • Jeroen Geilman
      ... It s hard to say without knowing postfix s attitude towards localhost vs. 192.168.1.10. Have you restarted postfix between these tests, to clear smtpd
      Message 2 of 7 , May 31, 2010
        On 05/31/2010 08:50 PM, Teh Kim Chooi wrote: Hi guys,

        i recently just setup a high volume postfix server, still in testing mode before the server go for live, OS rhel 5.5 and postfix version 2.3.3

        server with 1 quad core, 8gb ram OS on mirror disk, /var/spool/postfix in 1+0 6 disks, all is SAS 15k disk.

        my postfix configuration file will be all default setting due to still on testing.

        for my testing, i  run the smtp-source for the stress test

        time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t user1@... localhost:25

        abc.com is the local server, cde.com which i configure on another server to receive the message within same lan.

        result is 0.5 second which looks  fine,

        BUT when i use the server IP instead of localhost, my performance drop

        time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t user1@... 192.168.1.10:25

        result is 5 seconds, which almost 10 times more.

        the server now only run smtp postfix service, i stop mostly unwanted services from the server, and i never enable any spam filter, cause this server will be mainly sending email from a single source.

        what can cause weird behavior ? please advice....

        Regards,
        Teh

        It's hard to say without knowing postfix's attitude towards localhost vs. 192.168.1.10.

        Have you restarted postfix between these tests, to clear smtpd connection slots and any caches ?

        J.
      • Wietse Venema
        Teh Kim Chooi: [ Charset ISO-8859-1 unsupported, converting... ] ... Which is no longer maintained. The last release was postfix-2.3.19 in August 2009. ...
        Message 3 of 7 , May 31, 2010
          Teh Kim Chooi:
          [ Charset ISO-8859-1 unsupported, converting... ]
          > Hi guys,
          >
          > i recently just setup a high volume postfix server, still in testing mode
          > before the server go for live, OS rhel 5.5 and postfix version 2.3.3

          Which is no longer maintained. The last release was postfix-2.3.19
          in August 2009.

          > time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t
          > user1@... localhost:25
          >
          > result is 0.5 second which looks fine,

          Here, your client IP address is 127.0.0.1 or ::1.

          > BUT when i use the server IP instead of localhost, my performance drop
          >
          > time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t
          > user1@... 192.168.1.10:25
          >
          > result is 5 seconds, which almost 10 times more.

          Here your client IP address is 192.168.1.10.

          The Postfix SMTP server looks up the host name. Apparently this
          times out after 5 seconds when looking up 192.168.1.10.

          Wietse
        • Teh Kim Chooi
          It sounds weird to me, the 192.168.1.10 is on server eth0 network interface, and it will be on my local etc host file, will postfix still do NSlookup ? since i
          Message 4 of 7 , May 31, 2010
            It sounds weird to me, the 192.168.1.10 is on server eth0 network interface, and it will be on my local etc host file, will postfix still do NSlookup ?

            since i try inject 100 msgs, there is not time out 5 secs then only start injecting for the 100 msgs, the 5 secs is the time for 100 msgs to inject to the queue.

            i have tested this example on the other server ( the cde.com ) which also running the DNS ( temporary for this test ) it took 1.5 secs for localhost and 3 secs for the eth0 interface on this server, for this server the time only increase 2 times, but on the high end server it tooks 10 times ?

            dns lookup can really have such a big impact ? i will try using dig command later to find out the  ns lookup time between the server.



            On Tue, Jun 1, 2010 at 3:19 AM, Wietse Venema <wietse@...> wrote:
            Teh Kim Chooi:
            [ Charset ISO-8859-1 unsupported, converting... ]
            > Hi guys,
            >
            > i recently just setup a high volume postfix server, still in testing mode
            > before the server go for live, OS rhel 5.5 and postfix version 2.3.3

            Which is no longer maintained. The last release was postfix-2.3.19
            in August 2009.

            > time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t
            > user1@... localhost:25
            >
            > result is 0.5 second which looks  fine,

            Here, your client IP address is 127.0.0.1 or ::1.

            > BUT when i use the server IP instead of localhost, my performance drop
            >
            > time smtp-source -s 100 -l 409600 -m 100 -c -f user1@... -t
            > user1@... 192.168.1.10:25
            >
            > result is 5 seconds, which almost 10 times more.

            Here your client IP address is 192.168.1.10.

            The Postfix SMTP server looks up the host name.  Apparently this
            times out after 5 seconds when looking up 192.168.1.10.

                   Wietse

          • Wietse Venema
            ... The Postfix SMTP server looks up the client hostname with the getnameinfo() system library routine. I have attached a test program and recommend that you
            Message 5 of 7 , May 31, 2010
              Teh Kim Chooi:
              > It sounds weird to me, the 192.168.1.10 is on server eth0 network interface,
              > and it will be on my local etc host file, will postfix still do NSlookup ?

              The Postfix SMTP server looks up the client hostname with the
              getnameinfo() system library routine.

              I have attached a test program and recommend that you run it
              on the machine where sending mail was slow.

              ./getnameinfo 192.168.1.10

              See source code for instructions.

              Wietse
            • Teh Kim Chooi
              i test the command today, and found out that it only takes 1.5 secs, nothing change from the 5 secs result. I add the 192.168.1.10 to my /etc/hosts file, and
              Message 6 of 7 , Jun 1 8:00 AM
                i test the command today, and found out that it only takes 1.5 secs, nothing change from the 5 secs result. I add the 192.168.1.10 to my /etc/hosts file, and it drop to 0.5 secs to inject 100 msgs.

                1. Question here, if the sender IP is not in my /etc/hosts, will postfix do a reverse lookup on the IP ? If yes, means it will affect my performance, any way to help to increase without adding to the /etc/hosts file.

                although it took only 0.5 secs to inject the 100 msgs, but from the /var/log/maillog, i noticed that for the 100 msgs to fully sent to the cde.com from start injecting to successfully sent it took almost 9 secs. There only a cable direct connect between 2 servers and is gigabits connection. So from the calculation it send almost 11 msgs / sec and the is 400 KB per msg so almost 4.5MB  transfer between the 2 server, thats was way under 1 gigabits connection throughput. i check that the user maildir from the cde.com, the time to create the 100 msgs is only 2 secs, wheres go the other 7 secs in between ? is it cause by the low end cde.com receiving server ?

                kindly please advice again, thanks..




                On Tue, Jun 1, 2010 at 9:52 AM, Wietse Venema <wietse@...> wrote:
                Teh Kim Chooi:
                > It sounds weird to me, the 192.168.1.10 is on server eth0 network interface,
                > and it will be on my local etc host file, will postfix still do NSlookup ?

                The Postfix SMTP server looks up the client hostname with the
                getnameinfo() system library routine.

                I have attached a test program and recommend that you run it
                on the machine where sending mail was slow.

                ./getnameinfo 192.168.1.10

                See source code for instructions.

                       Wietse

              • Wietse Venema
                ... No surprise. ... These lookups are made by the SYSTEM LIBRARY not Postfix. Postfix invokes the system library functions getnameinfo() (to look up the SMTP
                Message 7 of 7 , Jun 1 8:36 AM
                  Teh Kim Chooi:
                  > i test the command today, and found out that it only takes 1.5 secs, nothing
                  > change from the 5 secs result. I add the 192.168.1.10 to my /etc/hosts file,
                  > and it drop to 0.5 secs to inject 100 msgs.

                  No surprise.

                  > 1. Question here, if the sender IP is not in my /etc/hosts, will postfix do
                  > a reverse lookup on the IP ? If yes, means it will affect my performance,
                  > any way to help to increase without adding to the /etc/hosts file.

                  These lookups are made by the SYSTEM LIBRARY not Postfix.

                  Postfix invokes the system library functions getnameinfo() (to look
                  up the SMTP client IP address) and getaddrinfo() (to verify that
                  the name thus found actually resolves to the client IP address).

                  The getwhateverinfo() functions use a mechanism called nsswitch or
                  equivalent to choose between lookups from local file, DNS, NIS,
                  LDAP or whatever your system has configured.

                  Apparently, your system looks in /etc/hosts first then DNS.
                  Apparently, your DNS is mis-configured, and that was responsible
                  for the lookup delays.

                  Wietse
                Your message has been successfully submitted and would be delivered to recipients shortly.