Loading ...
Sorry, an error occurred while loading the content.

performance of postfix

Expand Messages
  • Selcuk Yazar
    Hi, we have Postfix with LDAP backend , everything is working good but i think we have some performance issues , but i can t sure :/ ( Our mailbox folders are
    Message 1 of 9 , May 22 3:53 AM
    • 0 Attachment
      Hi,

      we have Postfix with LDAP backend , everything is working good but i think we have some performance issues , but i can't sure :/  ( Our mailbox folders are located Storage Drive  mapped at Redhat Enterprise)


      Every 5.0s:  ./qshape.pl                                                                                                                                                                                Wed May 22 13:37:53 2013

                                               T          5 10 20 40 80 160 320 640 1280 1280+
                                        TOTAL        25 25  0  0  0  0   0   0   0    0     0
                                our domain name 24 24  0  0  0  0   0   0   0    0     0
                                u-picardie.fr          1  1  0  0  0  0   0   0   0    0     0

      above is watch command results of qshape command  after  approx. 5 minutes later 
      results are below

      Every 5.0s:  ./qshape.pl                                                                                                                                                                                Wed May 22 13:41:20 2013

                                               T  5 10 20 40 80 160 320 640 1280 1280+
                                        TOTAL  0  0  0  0  0  0   0   0   0    0     0
                                                                                      
      our server have 45 GB of RAM, and 12 CPU .

      how can i learn any compare or opinion for our mail server performance.
      (i think it can work more fast ??  )


      thanks in advance

      --
      Selçuk YAZAR

    • Mike
      ... Have you looked at your logs to determine where and if your perceived delays are taking place? -- Looking for (employment|contract) work in the Internet
      Message 2 of 9 , May 22 3:56 AM
      • 0 Attachment
        On 13-05-22 07:53 AM, Selcuk Yazar wrote:
        > Hi,
        >
        > we have Postfix with LDAP backend , everything is working good but i
        > think we have some performance issues , but i can't sure :/ ( Our
        > mailbox folders are located Storage Drive mapped at Redhat Enterprise)
        >

        Have you looked at your logs to determine where and if your perceived
        delays are taking place?



        --
        Looking for (employment|contract) work in the
        Internet industry, preferably working remotely.
        Building / Supporting the net since 2400 baud was
        the hot thing. Ask for a resume! ispbuilder@...
      • Selcuk Yazar
        Hi, actually i forgot write additional info about our server, also we have policyd deamon (cluebringer) , amavis(SpamAssasin, clamav) and dovecot . when i
        Message 3 of 9 , May 22 4:20 AM
        • 0 Attachment
          Hi,

          actually  i forgot write additional info about our server, also we have policyd deamon (cluebringer) , amavis(SpamAssasin, clamav) and dovecot . when i looked up logs everything looking good for me :) . it flows like waterfall . sometime when one e-mail comes from yahoo groups, it takes minutes to delivery. What is the term do i look up the logs ? as i said i can't sure our performance is good or slow. 


          thanks.

           


          On Wed, May 22, 2013 at 1:56 PM, Mike <ispbuilder@...> wrote:
          On 13-05-22 07:53 AM, Selcuk Yazar wrote:
          Hi,

          we have Postfix with LDAP backend , everything is working good but i think we have some performance issues , but i can't sure :/  ( Our mailbox folders are located Storage Drive  mapped at Redhat Enterprise)


          Have you looked at your logs to determine where and if your perceived delays are taking place?



          --
          Looking for (employment|contract) work in the
          Internet industry, preferably working remotely.
          Building / Supporting the net since 2400 baud was
          the hot thing. Ask for a resume! ispbuilder@...




          --
          Selçuk YAZAR
          http://www.selcukyazar.blogspot.com
        • Viktor Dukhovni
          ... Running qshape every 5s is too often. Qshape is disk intensive. Run it every 5 minutes or so. ... If your content filter is not very fast, bursts of mail
          Message 4 of 9 , May 22 5:58 AM
          • 0 Attachment
            On Wed, May 22, 2013 at 01:53:15PM +0300, Selcuk Yazar wrote:

            > we have Postfix with LDAP backend , everything is working good but i think
            > we have some performance issues , but i can't sure :/ ( Our mailbox
            > folders are located Storage Drive mapped at Redhat Enterprise)
            >
            >
            > Every 5.0s: ./qshape.pl

            Running qshape every 5s is too often. Qshape is disk intensive.
            Run it every 5 minutes or so.

            >
            > Wed May 22 13:37:53 2013
            >
            > T 5 10 20 40 80 160 320 640 1280 1280+
            > TOTAL 25 25 0 0 0 0 0 0 0 0 0
            > our domain name 24 24 0 0 0 0 0 0 0 0 0
            > u-picardie.fr 1 1 0 0 0 0 0 0 0 0 0
            >
            > above is watch command results of qshape command after approx. 5 minutes
            > later results are below
            >
            > Every 5.0s: ./qshape.pl
            >
            > Wed May 22 13:41:20 2013
            >
            > T 5 10 20 40 80 160 320 640 1280 1280+
            > TOTAL 0 0 0 0 0 0 0 0 0 0 0

            If your content filter is not very fast, bursts of mail will accumulate
            while they are waiting to be scanned. Then the queue becomes empty.

            You may also have deferred mail that is retried periodically. You logs
            have a more complete picture.

            To improve content filter performance, eliminate remote DNS lookups
            in the filter, or increate concurrency. If the problem is lack of
            sufficient CPU resources, try to find a more performant scanner or
            turn off optional scanning features you don't need.

            Since mail is not delayed for very long, there is no problem (certainly
            not with Postfix itself, but scanning could perhaps be tuned).

            --
            Viktor.
          • Selcuk Yazar
            On Wed, May 22, 2013 at 3:58 PM, Viktor Dukhovni
            Message 5 of 9 , May 22 6:45 AM
            • 0 Attachment
              On Wed, May 22, 2013 at 3:58 PM, Viktor Dukhovni <postfix-users@...> wrote:
              On Wed, May 22, 2013 at 01:53:15PM +0300, Selcuk Yazar wrote:

              > we have Postfix with LDAP backend , everything is working good but i think
              > we have some performance issues , but i can't sure :/  ( Our mailbox
              > folders are located Storage Drive  mapped at Redhat Enterprise)
              >
              >
              > Every 5.0s:  ./qshape.pl

              Running qshape every 5s is too often.  Qshape is disk intensive.
              Run it every 5 minutes or so.

              >
              >                              Wed May 22 13:37:53 2013
              >
              >                            T   5 10 20 40 80 160 320 640 1280 1280+
              >               TOTAL        25 25  0  0  0  0  0    0   0    0    0
              >      our domain name       24 24  0  0  0  0   0   0   0    0    0
              >        u-picardie.fr        1  1  0  0  0  0   0   0   0    0    0
              >
              > above is watch command results of qshape command  after  approx. 5 minutes
              > later results are below
              >
              > Every 5.0s:  ./qshape.pl
              >
              >                              Wed May 22 13:41:20 2013
              >
              >                      T  5 10 20 40 80 160 320 640 1280 1280+
              >               TOTAL  0  0  0  0  0  0   0   0   0    0    0

              >If your content filter is not very fast, bursts of mail will accumulate<
              >while they are waiting to be scanned.  Then the queue becomes empty.

              >You may also have deferred mail that is retried periodically. You logs
              >have a more complete picture.

              >To improve content filter performance, eliminate remote DNS lookups
              >in the filter, or increate concurrency.  If the problem is lack of
              >sufficient CPU resources, try to find a more performant scanner or
              >turn off optional scanning features you don't need.

              >Since mail is not delayed for very long, there is no problem (certainly
              >not with Postfix itself, but scanning could perhaps be tuned).

              --
                      Viktor.


              i found sctipt for log analyze (sourceforge), result are like below. i think we have some queue problem, as i understand, %95 e-mails wait in queue 132 seconds ? 


              postfix logwatch

              === Delivery Delays Percentiles ============================================================
                                  0%       25%       50%       75%       90%       95%       98%      100%
              --------------------------------------------------------------------------------------------
              Before qmgr       0.01      0.33      1.00      3.30      5.30      8.10     65.00   2372.00
              In qmgr           0.00      0.00      0.01     21.00    110.00    132.00    158.00    180.00
              Conn setup        0.00      0.00      0.00      0.00      0.85      9.43     51.00    226.00
              Transmission      0.00      0.11      4.70      6.20     13.00     18.00     22.00     73.00
              Total             0.04      1.10      9.00     46.00    123.00    154.00    180.00   2373.00
              ============================================================================================


               
            • Viktor Dukhovni
              ... No, less than 5% of messages spend more than 132s in the active queue. Most messages spend less than 21s, with 50%s delivered immediately. ... To
              Message 6 of 9 , May 22 8:13 AM
              • 0 Attachment
                On Wed, May 22, 2013 at 04:45:42PM +0300, Selcuk Yazar wrote:

                > > >If your content filter is not very fast, bursts of mail will accumulate<
                > > >while they are waiting to be scanned. Then the queue becomes empty.
                > > >
                > > >You may also have deferred mail that is retried periodically. You logs
                > > >have a more complete picture.
                > > >
                > > >To improve content filter performance, eliminate remote DNS lookups
                > > >in the filter, or increate concurrency. If the problem is lack of
                > > >sufficient CPU resources, try to find a more performant scanner or
                > > >turn off optional scanning features you don't need.
                > > >
                > > >Since mail is not delayed for very long, there is no problem (certainly
                > > >not with Postfix itself, but scanning could perhaps be tuned).
                >
                > I found a script for log analyze (sourceforge), result are like below. I
                > think we have some queue problem, as I understand, %95 e-mails wait in
                > queue 132 seconds ?

                No, less than 5% of messages spend more than 132s in the active
                queue. Most messages spend less than 21s, with 50%s delivered
                immediately.

                > postfix logwatch
                >
                > === Delivery Delays Percentiles
                > ============================================================
                > 0% 25% 50% 75% 90% 95%
                > 98% 100%
                > --------------------------------------------------------------------------------------------
                > In qmgr 0.00 0.00 0.01 21.00 110.00 132.00
                > 158.00 180.00
                > Conn setup 0.00 0.00 0.00 0.00 0.85 9.43
                > 51.00 226.00
                > Transmission 0.00 0.11 4.70 6.20 13.00 18.00
                > 22.00 73.00
                > Total 0.04 1.10 9.00 46.00 123.00 154.00
                > 180.00 2373.00
                > ============================================================================================

                To understand what is actually going on, you'll have to *read* the
                logs, not just look at summaries. You'll probably find occasional
                latency sending messages through the content filter. If that's a
                problem, tune the content filter to remove DNS lookups or raise
                its concurrency. If the content filter is using all available CPU
                resources, tune it to do less, or find a more efficient one.

                Before any of that, locate the log entries showing delayed deliveries,
                read them, and figure out the reasons for the delay.

                --
                Viktor.
              • Stan Hoeppner
                ... I ve been using logwatch for quite some time and I ve found the Delivery Delay Percentiles 100% column to be seemingly pulled from thin air. Don t rely
                Message 7 of 9 , May 22 1:00 PM
                • 0 Attachment
                  On 5/22/2013 10:13 AM, Viktor Dukhovni wrote:
                  > On Wed, May 22, 2013 at 04:45:42PM +0300, Selcuk Yazar wrote:
                  >
                  >>>> If your content filter is not very fast, bursts of mail will accumulate<
                  >>>> while they are waiting to be scanned. Then the queue becomes empty.
                  >>>>
                  >>>> You may also have deferred mail that is retried periodically. You logs
                  >>>> have a more complete picture.
                  >>>>
                  >>>> To improve content filter performance, eliminate remote DNS lookups
                  >>>> in the filter, or increate concurrency. If the problem is lack of
                  >>>> sufficient CPU resources, try to find a more performant scanner or
                  >>>> turn off optional scanning features you don't need.
                  >>>>
                  >>>> Since mail is not delayed for very long, there is no problem (certainly
                  >>>> not with Postfix itself, but scanning could perhaps be tuned).
                  >>
                  >> I found a script for log analyze (sourceforge), result are like below. I
                  >> think we have some queue problem, as I understand, %95 e-mails wait in
                  >> queue 132 seconds ?
                  >
                  > No, less than 5% of messages spend more than 132s in the active
                  > queue. Most messages spend less than 21s, with 50%s delivered
                  > immediately.
                  >
                  >> postfix logwatch
                  >>
                  >> === Delivery Delays Percentiles
                  >> ============================================================
                  >> 0% 25% 50% 75% 90% 95%
                  >> 98% 100%
                  >> --------------------------------------------------------------------------------------------
                  >> In qmgr 0.00 0.00 0.01 21.00 110.00 132.00
                  >> 158.00 180.00
                  >> Conn setup 0.00 0.00 0.00 0.00 0.85 9.43
                  >> 51.00 226.00
                  >> Transmission 0.00 0.11 4.70 6.20 13.00 18.00
                  >> 22.00 73.00
                  >> Total 0.04 1.10 9.00 46.00 123.00 154.00
                  >> 180.00 2373.00
                  >> ============================================================================================
                  >
                  > To understand what is actually going on, you'll have to *read* the
                  > logs, not just look at summaries.

                  I've been using logwatch for quite some time and I've found the Delivery Delay Percentiles '100%' column to be seemingly pulled from thin air. Don't rely on it.

                  === Delivery Delays Percentiles ============================================================
                  0% 25% 50% 75% 90% 95% 98% 100%
                  --------------------------------------------------------------------------------------------
                  Before qmgr 0.02 0.03 0.06 0.28 0.42 0.88 2.48 9.20
                  In qmgr 0.00 0.02 0.02 0.03 0.03 0.03 0.04 0.06
                  Conn setup 0.00 0.00 0.00 0.00 0.00 0.00 0.55 2.70
                  Transmission 0.03 0.07 0.62 3.10 4.12 5.36 9.75 232.00
                  Total 0.08 0.12 1.50 3.60 4.82 6.90 11.28 232.00
                  ============================================================================================

                  For instance this summary of yesterday shows 232s for Transmission. Yet when I search my last ~3 days of logs with:

                  ~$ grep local /var/log/mail.log|mawk '{ print($10) }'|grep "delays"
                  ~$ grep smtp /var/log/mail.log|mawk '{ print($10) }'|grep "delays"

                  the largest value I see is 3.1s, in smtp. For local all delays are less than one second.

                  > You'll probably find occasional
                  > latency sending messages through the content filter. If that's a
                  > problem, tune the content filter to remove DNS lookups or raise
                  > its concurrency. If the content filter is using all available CPU
                  > resources, tune it to do less, or find a more efficient one.
                  >
                  > Before any of that, locate the log entries showing delayed deliveries,
                  > read them, and figure out the reasons for the delay.

                  I'm using spamc/spamd via pipe so it doesn't add to delays in postfix/local log stamps. To see the spamd delays I use:

                  ~$ grep scantime /var/log/mail.log|mawk '{ print($12) }'|cut -f1 -d,

                  This shows the largest spamd time is 37.7s, the next largest 13.0s. Some 95% appear to be less than 6s. Summing the largest of these with corresponding postfix/local delays doesn't come close to 232s, but less than 40s.

                  --
                  Stan
                • Viktor Dukhovni
                  ... When the scanner throughput is too low, the delay shows up in the active queue of the pre-scan Postfix instance, not in the scanner time to scan a message
                  Message 8 of 9 , May 22 2:04 PM
                  • 0 Attachment
                    On Wed, May 22, 2013 at 03:00:44PM -0500, Stan Hoeppner wrote:

                    > > You'll probably find occasional
                    > > latency sending messages through the content filter. If that's a
                    > > problem, tune the content filter to remove DNS lookups or raise
                    > > its concurrency. If the content filter is using all available CPU
                    > > resources, tune it to do less, or find a more efficient one.
                    > >
                    > > Before any of that, locate the log entries showing delayed deliveries,
                    > > read them, and figure out the reasons for the delay.
                    >
                    > I'm using spamc/spamd via pipe so it doesn't add to delays in postfix/local log stamps. To see the spamd delays I use:
                    >
                    > ~$ grep scantime /var/log/mail.log|mawk '{ print($12) }'|cut -f1 -d,

                    When the scanner throughput is too low, the delay shows up in the
                    active queue of the pre-scan Postfix instance, not in the scanner
                    time to scan a message logs. Messages sitting in active wating to
                    be scheduled for scanning are not seen by the non-telepathic scanner.

                    --
                    Viktor.
                  • Stan Hoeppner
                    ... The only point I was making is that some of the logwatch summary values may not be accurate, providing him a heads up as he had apparently never used
                    Message 9 of 9 , May 22 11:35 PM
                    • 0 Attachment
                      On 5/22/2013 4:04 PM, Viktor Dukhovni wrote:
                      > On Wed, May 22, 2013 at 03:00:44PM -0500, Stan Hoeppner wrote:
                      >
                      >>> You'll probably find occasional
                      >>> latency sending messages through the content filter. If that's a
                      >>> problem, tune the content filter to remove DNS lookups or raise
                      >>> its concurrency. If the content filter is using all available CPU
                      >>> resources, tune it to do less, or find a more efficient one.
                      >>>
                      >>> Before any of that, locate the log entries showing delayed deliveries,
                      >>> read them, and figure out the reasons for the delay.
                      >>
                      >> I'm using spamc/spamd via pipe so it doesn't add to delays in postfix/local log stamps. To see the spamd delays I use:
                      >>
                      >> ~$ grep scantime /var/log/mail.log|mawk '{ print($12) }'|cut -f1 -d,
                      >
                      > When the scanner throughput is too low, the delay shows up in the
                      > active queue of the pre-scan Postfix instance, not in the scanner
                      > time to scan a message logs. Messages sitting in active wating to
                      > be scheduled for scanning are not seen by the non-telepathic scanner.

                      The only point I was making is that some of the logwatch summary values
                      may not be accurate, providing him a heads up as he had apparently never
                      used logwatch prior to installing it and posting his summary table. I
                      was not attempting to troubleshoot his larger issue in this post.

                      --
                      Stan
                    Your message has been successfully submitted and would be delivered to recipients shortly.