Loading ...
Sorry, an error occurred while loading the content.

Re: Our postfix works fine, but it is very slow when we send newsletter

Expand Messages
  • Ansgar Wiechers
    ... Since the message gets delivered there might be nothing wrong with your configuration. Im wondering, though, why you chose so large values for ... Anyway,
    Message 1 of 6 , Feb 20, 2013
    View Source
    • 0 Attachment
      On 2013-02-20 Vince Wang wrote:
      > We have a configured postfix email server worked well when we had it
      > on the public IP. After we moved it behind our firewall on a intranet
      > with ip 192.168.xxx.xxx, we found it is very slow when we send
      > newsletter.
      >
      > Server info: Ubuntu 10.4 32 bit running on 4cpus + 8GB memory VM (
      > VMware host )
      > postfix: 2.70
      >
      > As I just start learning about postfix so I tried to figure how it
      > works. I sent a newsletter to 1100 members last week and monitored
      > the queue in the webmin and mailq, and the postfix log. After I
      > clicked the "send" button on our web page, I found that the messages
      > are added into the queue for 15 minutes and then I saw messages are
      > sent out from the log file for around 15 minutes.
      >
      > Is there anything wrong in our configure?

      Since the message gets delivered there might be nothing wrong with your
      configuration. Im wondering, though, why you chose so large values for
      $queue_run_delay and $minimal_backoff_time:

      > queue_run_delay = 4000s
      > minimal_backoff_time = 4000s

      Anyway, there isn't much we could tell without seeing the logs. Please
      post a log excerpt showing a full transaction of a delayed mail (from
      the point where the mail enters postfix to the point where it's
      delivered). You can get that data by greping for the queue ID of such a
      message.

      Also, always post the output of "postconf -n", not the content of
      main.cf. The latter isn't guaranteed to be the active configuration. The
      former is.

      Regards
      Ansgar Wiechers
      --
      "Abstractions save us time working, but they don't save us time learning."
      --Joel Spolsky
    • Wietse Venema
      ... Please define slow , and don t forget to include your outbound speed and message size. If you are sending mail to Yahoo and other large mailers, they will
      Message 2 of 6 , Feb 20, 2013
      View Source
      • 0 Attachment
        Vince Wang:
        > Hello,
        >
        > We have a configured postfix email server worked well when we had
        > it on the public IP. After we moved it behind our firewall on a
        > intranet with ip 192.168.xxx.xxx, we found it is very slow when
        > we send newsletter.

        Please define "slow", and don't forget to include your outbound
        speed and message size. If you are sending mail to Yahoo and other
        large mailers, they will slow you down intentionally.

        Wietse
      • Ralf Hildebrandt
        ... Logs? ... It could be that the process injecting the mails into the queue is stalling the queuemanager, thus sending out can only begin AFTER the injection
        Message 3 of 6 , Feb 21, 2013
        View Source
        • 0 Attachment
          * Vince Wang <vwang@...>:
          > Hello,
          >
          > We have a configured postfix email server worked well when we had it on the public IP. After we moved it behind our firewall on a intranet with ip 192.168.xxx.xxx, we found it is very slow when we send newsletter.

          Logs?

          > As I just start learning about postfix so I tried to figure how it
          > works. I sent a newsletter to 1100 members last week and monitored
          > the queue in the webmin and mailq, and the postfix log. After I
          > clicked the "send" button on our web page, I found that the messages
          > are added into the queue for 15 minutes and then I saw messages are
          > sent out from the log file for around 15 minutes.

          It could be that the process injecting the mails into the queue is
          stalling the queuemanager, thus sending out can only begin AFTER the
          injection period.

          > Here is the main.cf:
          > # amavis loop
          > content_filter = smtp-amavis:[127.0.0.1]:10024

          You're filtering the mail? I hope not.
          --
          [*] sys4 AG

          http://sys4.de, +49 (89) 30 90 46 64
          Franziskanerstraße 15, 81669 München

          Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
          Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer
          Aufsichtsratsvorsitzender: Joerg Heidrich
        • Jeroen Geilman
          ... How is DNS set up in comparison with the previous server ? Badly configured DNS can certainly slow things down, especially on outgoing mail. Any even
          Message 4 of 6 , Feb 21, 2013
          View Source
          • 0 Attachment
            On 02/20/2013 07:16 PM, Vince Wang wrote:

            Hello,

             

            We have a configured postfix email server worked well when we had it on the public IP.
            After we moved  it behind our firewall on a intranet with ip 192.168.xxx.xxx, we found it is very slow when we send newsletter.


            How is DNS set up in comparison with the previous server ?
            Badly configured DNS can certainly slow things down, especially on outgoing mail.
            Any even moderately busy mailserver should have a local DNS cache.

              Server info:     Ubuntu 10.4 32 bit running on 4cpus + 8GB memory VM ( VMware host )


            A 32-bit OS with 8GB of memory ? only 3.5GB of that will be used, ever.
            Regardless, postfix hardly uses any memory, unless you are receiving hundreds of 10MB messages concurrently.
            That is much more relevant for mail performance is storage I/O - and you don't mention anything related to storage.

            As I just start learning about  postfix so  I tried to figure how it works.  I sent a newsletter to 1100 members last week

            How many *messages* did you send ?

            and monitored  the queue in the webmin and mailq, and the postfix log.  After I clicked the "send" button on our web page, I  found that the messages are added into the queue for 15 minutes and then I saw messages are sent out  from the log file for around 15 minutes.   


            So you are seeing an average processing speed of 1.2 messages per second before queue, and another average 1.2 messages per second during delivery ?
            Show logs that exhibit these delays; postfix logs detailed delay statistics for each message delivered.

              content_filter = smtp-amavis:[127.0.0.1]:10024


            If you're submitting via smtpd(8) then all locally submitted mail will be scanned, which is patently useless in this case.

            smtpd_recipient_limit = 100000

            That is insane.

            qmgr_message_active_limit = 50000

            line_length_limit = 204800

            maximal_queue_lifetime = 2d

            queue_run_delay = 4000s

            minimal_backoff_time = 4000s


            Do not mess with these values unless you know exactly what they do.

            No logs, so how do you expect us to deduce what is happening here ?


            -- 
            J.
            
          • Jeroen Geilman
            ... ... how ? Either pickup(8) or smtpd(8) do the queueing; the qmgr only SENDS mail. There could be disk I/O contention, sure, but that would never translate
            Message 5 of 6 , Feb 21, 2013
            View Source
            • 0 Attachment
              On 02/21/2013 03:34 PM, Ralf Hildebrandt wrote:
              > It could be that the process injecting the mails into the queue is
              > stalling the queuemanager, thus sending out can only begin AFTER the
              > injection period.

              ... how ?

              Either pickup(8) or smtpd(8) do the queueing; the qmgr only SENDS mail.
              There could be disk I/O contention, sure, but that would never translate
              into a scenario where no mail could be de-queued before all mail was
              finished queueing.
              These are wholly separate processes after all, and the only point of
              contact is the mail queue, which is concurrent read-write by design.
              By default, there may be many simultaneous processes accessing the queue
              (100 each of smtpd and smtp, for starters.)

              Of course, it could be that he really is sending every single submitted
              message through amavisd and then re-injecting into postfix, thus
              effectively forcing every single message through the pipeline twice.

              This would be inane no matter what kind of IP address it has, but the
              cause of the delays would be the content_filter, nothing else.

              There are settings in amavisd-new that govern what to do when a message
              originates from a trusted or untrusted IP range, offering the option to
              pass it through without scanning.
              If this was impacted by the IP change, that could easily explain the
              delays - but they would still never be sequential.

              Of course, you did ask for logs as well :)

              --
              J.
            Your message has been successfully submitted and would be delivered to recipients shortly.