Loading ...
Sorry, an error occurred while loading the content.

postscreen_dnsbl_whitelist_threshold

Expand Messages
  • Wietse Venema
    After travel and several deadlines I started work this weekend on the idea to allow SMTP clients to skip postscreen tests based on their postscreen_dnsbl_sites
    Message 1 of 25 , May 12, 2013
    • 0 Attachment
      After travel and several deadlines I started work this weekend on
      the idea to allow SMTP clients to skip postscreen tests based on
      their postscreen_dnsbl_sites score.

      This required a little code reorganization (for the better) so that
      I could whitelist tests thusly:

      for (n = 0; n < TESTCOUNT; n++)
      if test[n] not already completed
      mark test[n] as completed

      This code reorganization allowed me to clean up parts of postscreen
      and replace N almost-identical blocks of code with a loop.

      I simplified the user interface. Below is a draft manpage.

      Wietse

      postscreen_dnsbl_whitelist_threshold (default: 0)
      The inclusive upper bound for whitelisting a remote SMTP client, based
      on its combined DNSBL score as defined with the postscreen_dnsbl_sites
      parameter. This allows a client to skip the pregreet test and the
      "after 220 greeting" protocol tests.

      Specify a negative value to enable this feature. When a client passes
      the postscreen_dnsbl_whitelist_threshold without having failed other
      tests, all pending tests are flagged as completed with a time-to-live
      value equal to postscreen_dnsbl_ttl. When a test was already com-
      pleted, its time-to-live value is updated if it was less than
      postscreen_dnsbl_ttl.

      This feature is available in Postfix 2.11.
    • Wietse Venema
      Before someone points out the obvious, here is a fixed version. Wietse ... Allow a remote SMTP client to skip before and after 220 greeting protocol
      Message 2 of 25 , May 12, 2013
      • 0 Attachment
        Before someone points out the obvious, here is a fixed version.

        Wietse

        Wietse Venema:
        > After travel and several deadlines I started work this weekend on
        > the idea to allow SMTP clients to skip postscreen tests based on
        > their postscreen_dnsbl_sites score.
        >
        > This required a little code reorganization (for the better) so that
        > I could whitelist tests thusly:
        >
        > for (n = 0; n < TESTCOUNT; n++)
        > if test[n] not already completed
        > mark test[n] as completed
        >
        > This code reorganization allowed me to clean up parts of postscreen
        > and replace N almost-identical blocks of code with a loop.
        >
        > I simplified the user interface. Below is a draft manpage.
        >
        > Wietse
        >
        > postscreen_dnsbl_whitelist_threshold (default: 0)
        Allow a remote SMTP client to skip "before" and "after 220 greeting"
        protocol tests, based on its combined DNSBL score as defined with the
        postscreen_dnsbl_sites parameter.
        >
        > Specify a negative value to enable this feature. When a client passes
        > the postscreen_dnsbl_whitelist_threshold without having failed other
        > tests, all pending tests are flagged as completed with a time-to-live
        > value equal to postscreen_dnsbl_ttl. When a test was already com-
        > pleted, its time-to-live value is updated if it was less than
        > postscreen_dnsbl_ttl.
        >
        > This feature is available in Postfix 2.11.
        >
        >
      • Wietse Venema
        A lightly-tested version is available as postfix-2.11-20130512. Wietse
        Message 3 of 25 , May 12, 2013
        • 0 Attachment
          A lightly-tested version is available as postfix-2.11-20130512.

          Wietse
        • /dev/rob0
          ... Woohoo! Thanks! I installed it, set postscreen_dnsbl_whitelist_threshold=-1 followed by a reload. Two seconds later I think it is working. May 13 00:59:50
          Message 4 of 25 , May 12, 2013
          • 0 Attachment
            On Sun, May 12, 2013 at 08:47:38PM -0400, Wietse Venema wrote:
            > A lightly-tested version is available as postfix-2.11-20130512.

            Woohoo! Thanks!

            I installed it, set postscreen_dnsbl_whitelist_threshold=-1
            followed by a reload. Two seconds later I think it is working.

            May 13 00:59:50 harrier postfix/postfix-script[12251]: starting the Postfix mail system
            May 13 00:59:50 harrier postfix/master[12253]: daemon started -- version 2.11-20130512, configuration /etc/postfix
            May 13 01:02:23 harrier postfix/postfix-script[12502]: refreshing the Postfix mail system
            May 13 01:02:23 harrier postfix/master[12253]: reload -- version 2.11-20130512, configuration /etc/postfix
            May 13 01:02:25 harrier postfix/postscreen[12508]: CONNECT from [66.220.144.151]:57808 to [207.223.116.211]:25
            May 13 01:02:25 harrier postfix/dnsblog[12509]: addr 66.220.144.151 listed by domain list.dnswl.org as 127.0.9.1
            May 13 01:02:25 harrier postfix/smtpd[12518]: connect from outmail017.snc4.facebook.com[66.220.144.151]
            May 13 01:02:26 harrier postfix/smtpd[12518]: 3b83fB2KJ4z3B92: client=outmail017.snc4.facebook.com[66.220.144.151]

            I don't see any PASS OLD in there, so I guess the whitelist did the
            trick? Would anything else be logged?
            --
            http://rob0.nodns4.us/ -- system administration and consulting
            Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
          • /dev/rob0
            ... Hmm, I m not sure what that was; maybe 66.220.144.151 was due for retesting in some tests? Here are some from a bit later, which get PASS NEW without any
            Message 5 of 25 , May 13, 2013
            • 0 Attachment
              On Sun, May 12, 2013 at 08:11:14PM -0500, /dev/rob0 wrote:
              > On Sun, May 12, 2013 at 08:47:38PM -0400, Wietse Venema wrote:
              > > A lightly-tested version is available as postfix-2.11-20130512.
              >
              > Woohoo! Thanks!
              >
              > I installed it, set postscreen_dnsbl_whitelist_threshold=-1
              > followed by a reload. Two seconds later I think it is working.
              >
              > May 13 00:59:50 harrier postfix/postfix-script[12251]: starting the Postfix mail system
              > May 13 00:59:50 harrier postfix/master[12253]: daemon started -- version 2.11-20130512, configuration /etc/postfix
              > May 13 01:02:23 harrier postfix/postfix-script[12502]: refreshing the Postfix mail system
              > May 13 01:02:23 harrier postfix/master[12253]: reload -- version 2.11-20130512, configuration /etc/postfix
              > May 13 01:02:25 harrier postfix/postscreen[12508]: CONNECT from [66.220.144.151]:57808 to [207.223.116.211]:25
              > May 13 01:02:25 harrier postfix/dnsblog[12509]: addr 66.220.144.151 listed by domain list.dnswl.org as 127.0.9.1
              > May 13 01:02:25 harrier postfix/smtpd[12518]: connect from outmail017.snc4.facebook.com[66.220.144.151]
              > May 13 01:02:26 harrier postfix/smtpd[12518]: 3b83fB2KJ4z3B92: client=outmail017.snc4.facebook.com[66.220.144.151]
              >
              > I don't see any PASS OLD in there, so I guess the whitelist did the
              > trick? Would anything else be logged?

              Hmm, I'm not sure what that was; maybe 66.220.144.151 was due for
              retesting in some tests? Here are some from a bit later, which get
              "PASS NEW" without any after-220 tests:

              May 13 01:15:09 harrier postfix/postscreen[13360]: CONNECT from [98.136.219.129]:36682 to [207.223.116.211]:25
              May 13 01:15:09 harrier postfix/dnsblog[13365]: addr 98.136.219.129 listed by domain list.dnswl.org as 127.0.5.0
              May 13 01:15:09 harrier postfix/postscreen[13360]: PASS NEW [98.136.219.129]:36682
              May 13 01:15:10 harrier postfix/smtpd[13371]: connect from ng10-vm12.bullet.mail.gq1.yahoo.com[98.136.219.129]
              May 13 01:15:10 harrier postfix/smtpd[13371]: 3b83wt3SgQz3B99: client=ng10-vm12.bullet.mail.gq1.yahoo.com[98.136.219.129]

              May 13 02:22:50 harrier postfix/postscreen[18837]: CONNECT from [98.138.214.175]:46014 to [207.223.116.211]:25
              May 13 02:22:50 harrier postfix/dnsblog[18943]: addr 98.138.214.175 listed by domain list.dnswl.org as 127.0.5.0
              May 13 02:22:50 harrier postfix/postscreen[18837]: PASS NEW [98.138.214.175]:46014
              May 13 02:22:50 harrier postfix/smtpd[18952]: connect from ng19-vm1.bullet.mail.ne1.yahoo.com[98.138.214.175]
              May 13 02:22:51 harrier postfix/smtpd[18952]: 3b85Qz1WQfz3BMc: client=ng19-vm1.bullet.mail.ne1.yahoo.com[98.138.214.175]

              May 13 07:45:06 harrier postfix/postscreen[9497]: CONNECT from [144.160.128.166]:38244 to [207.223.116.211]:25
              May 13 07:45:06 harrier postfix/dnsblog[9502]: addr 144.160.128.166 listed by domain list.dnswl.org as 127.0.5.0
              May 13 07:45:06 harrier postfix/postscreen[9497]: PASS NEW [144.160.128.166]:38244
              May 13 07:45:07 harrier postfix/smtpd[9507]: connect from egssmtp02.att.com[144.160.128.166]
              May 13 07:45:07 harrier postfix/smtpd[9507]: 3b8DZq6bcpz38Bm: client=egssmtp02.att.com[144.160.128.166]

              May 13 07:48:54 harrier postfix/postscreen[9811]: CONNECT from [54.240.15.13]:45225 to [207.223.116.211]:25
              May 13 07:48:54 harrier postfix/dnsblog[9812]: addr 54.240.15.13 listed by domain list.dnswl.org as 127.0.5.1
              May 13 07:48:54 harrier postfix/postscreen[9811]: PASS NEW [54.240.15.13]:45225
              May 13 07:48:54 harrier postfix/smtpd[9821]: connect from a15-13.smtp-out.amazonses.com[54.240.15.13]
              May 13 07:48:55 harrier postfix/smtpd[9821]: 3b8DgC17cnz38D6: client=a15-13.smtp-out.amazonses.com[54.240.15.13]

              This next one is very interesting. Whitelisted and blacklisted,
              coming in with a score of +1, so not reaching either of the
              thresholds. This host hits the lower priority MX .214 before the
              DISCONNECT on the main address of .211, and gets a WHITELIST VETO.

              May 13 11:53:27 harrier postfix/postscreen[28908]: CONNECT from [200.11.173.11]:46875 to [207.223.116.211]:25
              May 13 11:53:27 harrier postfix/dnsblog[28910]: addr 200.11.173.11 listed by domain b.barracudacentral.org as 127.0.0.2
              May 13 11:53:27 harrier postfix/dnsblog[28913]: addr 200.11.173.11 listed by domain list.dnswl.org as 127.0.5.0
              May 13 11:53:27 harrier postfix/dnsblog[28909]: addr 200.11.173.11 listed by domain dnsbl.sorbs.net as 127.0.0.6
              May 13 11:53:33 harrier postfix/tlsproxy[28928]: CONNECT from [200.11.173.11]:46875
              May 13 11:53:33 harrier postfix/tlsproxy[28928]: Anonymous TLS connection established from [200.11.173.11]:46875: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)
              May 13 11:53:33 harrier postfix/postscreen[28908]: NOQUEUE: reject: RCPT from [200.11.173.11]:46875: 450 4.3.2 Service currently unavailable; from=<officefile8184@...>, to=<1001@...>, proto=ESMTP, helo=<10ibl20ser04.datacenter.cha.cantv.net>
              May 13 11:53:34 harrier postfix/postscreen[28908]: CONNECT from [200.11.173.11]:54443 to [207.223.116.214]:25
              May 13 11:53:34 harrier postfix/postscreen[28908]: WHITELIST VETO [200.11.173.11]:54443
              May 13 11:53:34 harrier postfix/dnsblog[28913]: addr 200.11.173.11 listed by domain list.dnswl.org as 127.0.5.0
              May 13 11:53:34 harrier postfix/dnsblog[28912]: addr 200.11.173.11 listed by domain b.barracudacentral.org as 127.0.0.2
              May 13 11:53:34 harrier postfix/dnsblog[28911]: addr 200.11.173.11 listed by domain dnsbl.sorbs.net as 127.0.0.6
              May 13 11:53:40 harrier postfix/tlsproxy[28928]: CONNECT from [200.11.173.11]:54443
              May 13 11:53:40 harrier postfix/tlsproxy[28928]: Anonymous TLS connection established from [200.11.173.11]:54443: TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)
              May 13 11:53:41 harrier postfix/postscreen[28908]: NOQUEUE: reject: RCPT from [200.11.173.11]:54443: 450 4.3.2 Service currently unavailable; from=<officefile8184@...>, to=<1001@...>, proto=ESMTP, helo=<10ibl20ser04.datacenter.cha.cantv.net>
              May 13 11:54:25 harrier postfix/postscreen[28908]: PASS NEW [200.11.173.11]:46875
              May 13 11:54:25 harrier postfix/postscreen[28908]: DISCONNECT [200.11.173.11]:46875
              May 13 11:54:25 harrier postfix/tlsproxy[28928]: DISCONNECT [200.11.173.11]:46875
              May 13 11:54:27 harrier postfix/postscreen[28908]: DISCONNECT [200.11.173.11]:54443
              May 13 11:54:27 harrier postfix/tlsproxy[28928]: DISCONNECT [200.11.173.11]:54443

              Sadly, this host which was definitely carrying spam got a PASS NEW.
              But this is not the sort of spam which postscreen can safely block.
              --
              http://rob0.nodns4.us/ -- system administration and consulting
              Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
            • Wietse Venema
              ... It may well be that PASS OLD logging has broken. I spent most development time looking at verbose output to verify that postscreen makes the right
              Message 6 of 25 , May 13, 2013
              • 0 Attachment
                /dev/rob0:
                > > I don't see any PASS OLD in there, so I guess the whitelist did the
                > > trick? Would anything else be logged?
                >
                > Hmm, I'm not sure what that was; maybe 66.220.144.151 was due for
                > retesting in some tests? Here are some from a bit later, which get
                > "PASS NEW" without any after-220 tests:

                It may well be that PASS OLD logging has broken. I spent most
                development time looking at verbose output to verify that postscreen
                makes the right decisions. I'll check out the logging next.

                Wietse
              • /dev/rob0
                ... Not entirely, as I do have numerous PASS OLD in the logs: $ egrep ^May 13 .* PASS OLD /var/log/maillog | wc 73 584 5947 (Less 12 from before the
                Message 7 of 25 , May 13, 2013
                • 0 Attachment
                  On Mon, May 13, 2013 at 09:12:57AM -0400, Wietse Venema wrote:
                  > /dev/rob0:
                  > > > I don't see any PASS OLD in there, so I guess the whitelist
                  > > > did the trick? Would anything else be logged?
                  > >
                  > > Hmm, I'm not sure what that was; maybe 66.220.144.151 was due
                  > > for retesting in some tests? Here are some from a bit later,
                  > > which get "PASS NEW" without any after-220 tests:
                  >
                  > It may well be that PASS OLD logging has broken.

                  Not entirely, as I do have numerous PASS OLD in the logs:

                  $ egrep "^May 13 .* PASS OLD" /var/log/maillog | wc
                  73 584 5947

                  (Less 12 from before the upgrade at ~01:00 UTC.) Would it be useful
                  to see some of those? I suppose they're meaningless without the
                  whitelist database showing when the various tests were passed.

                  > I spent most development time looking at verbose output to
                  > verify that postscreen makes the right decisions. I'll check
                  > out the logging next.

                  I'll be ready to try another snapshot when available. :)
                  --
                  http://rob0.nodns4.us/ -- system administration and consulting
                  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                • Wietse Venema
                  ... With whitelisting turned on I see no missing PASS logging PASS NEW New client. PASS OLD Client reconnects after cached test results expire. PASS OLD
                  Message 8 of 25 , May 13, 2013
                  • 0 Attachment
                    /dev/rob0:
                    > On Mon, May 13, 2013 at 09:12:57AM -0400, Wietse Venema wrote:
                    > > /dev/rob0:
                    > > > > I don't see any PASS OLD in there, so I guess the whitelist
                    > > > > did the trick? Would anything else be logged?
                    > > >
                    > > > Hmm, I'm not sure what that was; maybe 66.220.144.151 was due
                    > > > for retesting in some tests? Here are some from a bit later,
                    > > > which get "PASS NEW" without any after-220 tests:
                    > >
                    > > It may well be that PASS OLD logging has broken.
                    >
                    > Not entirely, as I do have numerous PASS OLD in the logs:
                    >
                    > $ egrep "^May 13 .* PASS OLD" /var/log/maillog | wc
                    > 73 584 5947

                    With whitelisting turned on I see no missing "PASS" logging

                    PASS NEW New client.
                    PASS OLD Client reconnects after cached test results expire.
                    PASS OLD Client reconnects after cached test results expire.

                    And Postscreen behavior does not change when DNS whitelisting is disabled.

                    Wietse
                  • Wietse Venema
                    ... Cut-and-paste error. The correct text is: PASS NEW New client. PASS OLD Client reconnects before cached test results expire. PASS OLD Client
                    Message 9 of 25 , May 13, 2013
                    • 0 Attachment
                      Wietse Venema:
                      > /dev/rob0:
                      > > On Mon, May 13, 2013 at 09:12:57AM -0400, Wietse Venema wrote:
                      > > > /dev/rob0:
                      > > > > > I don't see any PASS OLD in there, so I guess the whitelist
                      > > > > > did the trick? Would anything else be logged?
                      > > > >
                      > > > > Hmm, I'm not sure what that was; maybe 66.220.144.151 was due
                      > > > > for retesting in some tests? Here are some from a bit later,
                      > > > > which get "PASS NEW" without any after-220 tests:
                      > > >
                      > > > It may well be that PASS OLD logging has broken.
                      > >
                      > > Not entirely, as I do have numerous PASS OLD in the logs:
                      > >
                      > > $ egrep "^May 13 .* PASS OLD" /var/log/maillog | wc
                      > > 73 584 5947
                      >
                      > With whitelisting turned on I see no missing "PASS" logging
                      >
                      > PASS NEW New client.
                      > PASS OLD Client reconnects after cached test results expire.
                      > PASS OLD Client reconnects after cached test results expire.

                      Cut-and-paste error. The correct text is:

                      PASS NEW New client.
                      PASS OLD Client reconnects before cached test results expire.
                      PASS OLD Client reconnects after cached test results expire.

                      In any case I will roll out a new snapshot. I forgot to include
                      some other fixes that have been tested since 20130405.

                      Wietse

                      > And Postscreen behavior does not change when DNS whitelisting is disabled.
                      >
                      > Wietse
                      >
                    • Wietse Venema
                      ... Found it. With postscreen_dnsbl_whitelist_threshold turned on, postscreen raised the pregreet test is passed flag even when that test was disabled. This
                      Message 10 of 25 , May 13, 2013
                      • 0 Attachment
                        /dev/rob0:
                        > On Sun, May 12, 2013 at 08:11:14PM -0500, /dev/rob0 wrote:
                        > > On Sun, May 12, 2013 at 08:47:38PM -0400, Wietse Venema wrote:
                        > > > A lightly-tested version is available as postfix-2.11-20130512.
                        > >
                        > > Woohoo! Thanks!
                        > >
                        > > I installed it, set postscreen_dnsbl_whitelist_threshold=-1
                        > > followed by a reload. Two seconds later I think it is working.
                        > >
                        > > May 13 00:59:50 harrier postfix/postfix-script[12251]: starting the Postfix mail system
                        > > May 13 00:59:50 harrier postfix/master[12253]: daemon started -- version 2.11-20130512, configuration /etc/postfix
                        > > May 13 01:02:23 harrier postfix/postfix-script[12502]: refreshing the Postfix mail system
                        > > May 13 01:02:23 harrier postfix/master[12253]: reload -- version 2.11-20130512, configuration /etc/postfix
                        > > May 13 01:02:25 harrier postfix/postscreen[12508]: CONNECT from [66.220.144.151]:57808 to [207.223.116.211]:25
                        > > May 13 01:02:25 harrier postfix/dnsblog[12509]: addr 66.220.144.151 listed by domain list.dnswl.org as 127.0.9.1
                        > > May 13 01:02:25 harrier postfix/smtpd[12518]: connect from outmail017.snc4.facebook.com[66.220.144.151]
                        > > May 13 01:02:26 harrier postfix/smtpd[12518]: 3b83fB2KJ4z3B92: client=outmail017.snc4.facebook.com[66.220.144.151]
                        > >
                        > > I don't see any PASS OLD in there, so I guess the whitelist did the
                        > > trick? Would anything else be logged?

                        Found it. With postscreen_dnsbl_whitelist_threshold turned on,
                        postscreen raised the "pregreet test is passed" flag even when that
                        test was disabled. This led to a mis-match between what tests were
                        required versus what tests were passed, resulting in no "PASS NEW"
                        logging.

                        The error is only cosmetic and has no effect on mail deliveries.

                        Wietse
                      • Noel Jones
                        ... Just installed the 20130512 snapshot... getting a panic: psc_dnsbl_retrieve: no blocklist score , /seems to/ happen after a PREGREET from a dnsbl listed
                        Message 11 of 25 , May 13, 2013
                        • 0 Attachment
                          On 5/13/2013 4:04 PM, Wietse Venema wrote:
                          > /dev/rob0:
                          >> On Sun, May 12, 2013 at 08:11:14PM -0500, /dev/rob0 wrote:
                          >>> On Sun, May 12, 2013 at 08:47:38PM -0400, Wietse Venema wrote:
                          >>>> A lightly-tested version is available as postfix-2.11-20130512.
                          >>>
                          >>> Woohoo! Thanks!
                          >>>
                          >>> I installed it, set postscreen_dnsbl_whitelist_threshold=-1
                          >>> followed by a reload. Two seconds later I think it is working.
                          >>>
                          >>> May 13 00:59:50 harrier postfix/postfix-script[12251]: starting the Postfix mail system
                          >>> May 13 00:59:50 harrier postfix/master[12253]: daemon started -- version 2.11-20130512, configuration /etc/postfix
                          >>> May 13 01:02:23 harrier postfix/postfix-script[12502]: refreshing the Postfix mail system
                          >>> May 13 01:02:23 harrier postfix/master[12253]: reload -- version 2.11-20130512, configuration /etc/postfix
                          >>> May 13 01:02:25 harrier postfix/postscreen[12508]: CONNECT from [66.220.144.151]:57808 to [207.223.116.211]:25
                          >>> May 13 01:02:25 harrier postfix/dnsblog[12509]: addr 66.220.144.151 listed by domain list.dnswl.org as 127.0.9.1
                          >>> May 13 01:02:25 harrier postfix/smtpd[12518]: connect from outmail017.snc4.facebook.com[66.220.144.151]
                          >>> May 13 01:02:26 harrier postfix/smtpd[12518]: 3b83fB2KJ4z3B92: client=outmail017.snc4.facebook.com[66.220.144.151]
                          >>>
                          >>> I don't see any PASS OLD in there, so I guess the whitelist did the
                          >>> trick? Would anything else be logged?
                          >
                          > Found it. With postscreen_dnsbl_whitelist_threshold turned on,
                          > postscreen raised the "pregreet test is passed" flag even when that
                          > test was disabled. This led to a mis-match between what tests were
                          > required versus what tests were passed, resulting in no "PASS NEW"
                          > logging.
                          >
                          > The error is only cosmetic and has no effect on mail deliveries.
                          >
                          > Wietse
                          >


                          Just installed the 20130512 snapshot...

                          getting a "panic: psc_dnsbl_retrieve: no blocklist score", /seems
                          to/ happen after a PREGREET from a dnsbl listed client. Anyway,
                          valid mail is sill arriving with both PASS NEW and PASS OLD, dnsbl
                          listed clients that don't pregreet are being rejected without error.

                          The following was logged after a postfix restart with an empty
                          postscreen_cache database.


                          May 13 16:12:11 mgate3 postfix/master[9707]: daemon started --
                          version 2.11-20130512, configuration /etc/postfix
                          May 13 16:12:12 mgate3 postfix/postscreen[9711]: cache
                          btree:/var/lib/postfix/postscreen_cache full cleanup: retained=0
                          dropped=0 entries
                          May 13 16:12:12 mgate3 postfix/postscreen[9711]: CONNECT from
                          [186.83.226.229]:1480 to [192.168.70.43]:25
                          May 13 16:12:12 mgate3 postfix/dnsblog[9714]: addr 186.83.226.229
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:12 mgate3 postfix/dnsblog[9714]: addr 186.83.226.229
                          listed by domain zen.spamhaus.org as 127.0.0.11
                          May 13 16:12:13 mgate3 postfix/postscreen[9711]: PREGREET 42 after
                          0.72 from [186.83.226.229]:1480: HELO
                          Dynamic-IP-18683226229.cable.net.co\r\n
                          May 13 16:12:13 mgate3 postfix/postscreen[9711]: panic:
                          psc_dnsbl_retrieve: no blocklist score for 186.83.226.229
                          May 13 16:12:14 mgate3 postfix/master[9707]: warning: process
                          /usr/libexec/postfix/postscreen pid 9711 killed by signal 6
                          May 13 16:12:16 mgate3 postfix/postscreen[9715]: CONNECT from
                          [173.44.230.38]:15114 to [192.168.70.43]:25
                          May 13 16:12:17 mgate3 postfix/postscreen[9715]: CONNECT from
                          [61.70.82.57]:2124 to [192.168.70.43]:25
                          May 13 16:12:17 mgate3 postfix/dnsblog[9712]: addr 61.70.82.57
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:18 mgate3 postfix/postscreen[9715]: PREGREET 44 after
                          0.82 from [61.70.82.57]:2124: HELO
                          host-61-70-82-57.static.kbtelecom.net\r\n
                          May 13 16:12:18 mgate3 postfix/postscreen[9715]: panic:
                          psc_dnsbl_retrieve: no blocklist score for 61.70.82.57
                          May 13 16:12:19 mgate3 postfix/master[9707]: warning: process
                          /usr/libexec/postfix/postscreen pid 9715 killed by signal 6
                          May 13 16:12:19 mgate3 postfix/postscreen[9716]: CONNECT from
                          [178.125.147.190]:4660 to [192.168.70.43]:25
                          May 13 16:12:19 mgate3 postfix/dnsblog[9713]: addr 178.125.147.190
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:19 mgate3 postfix/dnsblog[9713]: addr 178.125.147.190
                          listed by domain zen.spamhaus.org as 127.0.0.11
                          May 13 16:12:19 mgate3 postfix/postscreen[9716]: CONNECT from
                          [89.114.17.136]:3427 to [192.168.70.43]:25
                          May 13 16:12:19 mgate3 postfix/postscreen[9716]: PREGREET 22 after
                          0.65 from [178.125.147.190]:4660: HELO 178.125.147.190\r\n
                          May 13 16:12:19 mgate3 postfix/postscreen[9716]: panic:
                          psc_dnsbl_retrieve: no blocklist score for 178.125.147.190
                          May 13 16:12:19 mgate3 postfix/dnsblog[9713]: addr 89.114.17.136
                          listed by domain zen.spamhaus.org as 127.0.0.10
                          May 13 16:12:19 mgate3 postfix/dnsblog[9713]: addr 89.114.17.136
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:20 mgate3 postfix/master[9707]: warning: process
                          /usr/libexec/postfix/postscreen pid 9716 killed by signal 6
                          May 13 16:12:20 mgate3 postfix/postscreen[9719]: CONNECT from
                          [173.14.106.45]:4693 to [192.168.70.43]:25
                          May 13 16:12:21 mgate3 postfix/postscreen[9719]: CONNECT from
                          [220.134.174.161]:62439 to [192.168.70.43]:25
                          May 13 16:12:21 mgate3 postfix/dnsblog[9713]: addr 220.134.174.161
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:22 mgate3 postfix/postscreen[9719]: PREGREET 41 after
                          0.82 from [220.134.174.161]:62439: HELO
                          220-134-174-161.HINET-IP.hinet.net\r\n
                          May 13 16:12:22 mgate3 postfix/postscreen[9719]: panic:
                          psc_dnsbl_retrieve: no blocklist score for 220.134.174.161
                          May 13 16:12:23 mgate3 postfix/master[9707]: warning: process
                          /usr/libexec/postfix/postscreen pid 9719 killed by signal 6
                          May 13 16:12:24 mgate3 postfix/postscreen[9720]: CONNECT from
                          [93.158.11.233]:56107 to [192.168.70.43]:25
                          May 13 16:12:25 mgate3 postfix/dnsblog[9712]: addr 93.158.11.233
                          listed by domain zen.spamhaus.org as 127.0.0.11
                          May 13 16:12:25 mgate3 postfix/dnsblog[9712]: addr 93.158.11.233
                          listed by domain zen.spamhaus.org as 127.0.0.4
                          May 13 16:12:25 mgate3 postfix/postscreen[9720]: PREGREET 23 after
                          0.39 from [93.158.11.233]:56107: HELO concordances.com\r\n
                          May 13 16:12:25 mgate3 postfix/postscreen[9720]: panic:
                          psc_dnsbl_retrieve: no blocklist score for 93.158.11.233



                          # postconf | grep postscreen
                          postscreen_access_list = permit_mynetworks,
                          cidr:$mapdir/postscreen_access.cidr
                          postscreen_bare_newline_action = enforce
                          postscreen_bare_newline_enable = no
                          postscreen_bare_newline_ttl = 30d
                          postscreen_blacklist_action = drop
                          postscreen_cache_cleanup_interval = 12h
                          postscreen_cache_map = btree:$data_directory/postscreen_cache
                          postscreen_cache_retention_time = 7d
                          postscreen_client_connection_count_limit = 2
                          postscreen_command_count_limit = 20
                          postscreen_command_filter =
                          postscreen_command_time_limit = ${stress?10}${stress:300}s
                          postscreen_disable_vrfy_command = $disable_vrfy_command
                          postscreen_discard_ehlo_keyword_address_maps =
                          $smtpd_discard_ehlo_keyword_address_maps
                          postscreen_discard_ehlo_keywords = $smtpd_discard_ehlo_keywords
                          postscreen_dnsbl_action = enforce
                          postscreen_dnsbl_reply_map =
                          postscreen_dnsbl_sites = zen.spamhaus.org*1 list.dnswl.org*-1
                          swl.spamhaus.org*-1
                          postscreen_dnsbl_threshold = 1
                          postscreen_dnsbl_ttl = 1h
                          postscreen_dnsbl_whitelist_threshold = -1
                          postscreen_enforce_tls = $smtpd_enforce_tls
                          postscreen_expansion_filter = $smtpd_expansion_filter
                          postscreen_forbidden_commands = $smtpd_forbidden_commands
                          postscreen_greet_action = drop
                          postscreen_greet_banner = mgate3.vbhcs.org ESTMP -- validating
                          connection
                          postscreen_greet_ttl = 1d
                          postscreen_greet_wait = 6s
                          postscreen_helo_required = $smtpd_helo_required
                          postscreen_non_smtp_command_action = drop
                          postscreen_non_smtp_command_enable = no
                          postscreen_non_smtp_command_ttl = 30d
                          postscreen_pipelining_action = enforce
                          postscreen_pipelining_enable = no
                          postscreen_pipelining_ttl = 30d
                          postscreen_post_queue_limit = $default_process_limit
                          postscreen_pre_queue_limit = $default_process_limit
                          postscreen_reject_footer = \c; Contact postmaster@... for
                          assistance. Include this data: servertime=($localtime)
                          client=([$client_address]:$client_port) server=($server_name)
                          (postscreen)
                          postscreen_tls_security_level = $smtpd_tls_security_level
                          postscreen_upstream_proxy_protocol =
                          postscreen_upstream_proxy_timeout = 5s
                          postscreen_use_tls = $smtpd_use_tls
                          postscreen_watchdog_timeout = 10s
                          postscreen_whitelist_interfaces = !192.168.70.44 !12.107.221.44
                          static:all




                          -- Noel Jones
                        • Wietse Venema
                          ... Thanks for finding this. Easy fix: prepend this: if (state- dnsbl_score == NO_DNSBL_SCORE) before: (void) psc_dnsbl_retrieve(state- smtp_client_addr,...
                          Message 12 of 25 , May 13, 2013
                          • 0 Attachment
                            Noel Jones:
                            > May 13 16:12:13 mgate3 postfix/postscreen[9711]: PREGREET 42 after
                            > 0.72 from [186.83.226.229]:1480: HELO
                            > Dynamic-IP-18683226229.cable.net.co\r\n
                            > May 13 16:12:13 mgate3 postfix/postscreen[9711]: panic:
                            > psc_dnsbl_retrieve: no blocklist score for 186.83.226.229

                            Thanks for finding this.

                            Easy fix: prepend this:

                            if (state->dnsbl_score == NO_DNSBL_SCORE)

                            before:

                            (void) psc_dnsbl_retrieve(state->smtp_client_addr,...

                            and:

                            (void) psc_dnsbl_retrieve(state->smtp_client_addr,...

                            That is, there are two places where the guard is needed.

                            Wietse
                          • Noel Jones
                            ... Works, thanks. The botherder/spammer conveniently sent me another run just after patching; no more errors. -- Noel Jones
                            Message 13 of 25 , May 13, 2013
                            • 0 Attachment
                              On 5/13/2013 4:55 PM, Wietse Venema wrote:
                              > Noel Jones:
                              >> May 13 16:12:13 mgate3 postfix/postscreen[9711]: PREGREET 42 after
                              >> 0.72 from [186.83.226.229]:1480: HELO
                              >> Dynamic-IP-18683226229.cable.net.co\r\n
                              >> May 13 16:12:13 mgate3 postfix/postscreen[9711]: panic:
                              >> psc_dnsbl_retrieve: no blocklist score for 186.83.226.229
                              >
                              > Thanks for finding this.
                              >
                              > Easy fix: prepend this:
                              >
                              > if (state->dnsbl_score == NO_DNSBL_SCORE)
                              >
                              > before:
                              >
                              > (void) psc_dnsbl_retrieve(state->smtp_client_addr,...
                              >
                              > and:
                              >
                              > (void) psc_dnsbl_retrieve(state->smtp_client_addr,...
                              >
                              > That is, there are two places where the guard is needed.
                              >
                              > Wietse
                              >


                              Works, thanks. The botherder/spammer conveniently sent me another
                              run just after patching; no more errors.



                              -- Noel Jones
                            • Wietse Venema
                              ... Also uploaded as snapshot 20130513. Wietse
                              Message 14 of 25 , May 13, 2013
                              • 0 Attachment
                                Noel Jones:
                                > Works, thanks. The botherder/spammer conveniently sent me another
                                > run just after patching; no more errors.

                                Also uploaded as snapshot 20130513.

                                Wietse
                              • /dev/rob0
                                In the time since I ve been running this, I saw the first thing that might be seen as a problem: dnsblog timing out on one of the DNSBL lookups: May 16
                                Message 15 of 25 , May 16, 2013
                                • 0 Attachment
                                  In the time since I've been running this, I saw the first thing that
                                  might be seen as a problem: dnsblog timing out on one of the DNSBL
                                  lookups:

                                  May 16 21:51:44 harrier postfix/postscreen[29502]: CONNECT from [208.66.205.36]:53814 to [207.223.116.211]:25
                                  May 16 21:51:44 harrier postfix/dnsblog[29507]: addr 208.66.205.36 listed by domain list.dnswl.org as 127.0.15.0

                                  This gives it a -2 so far, but when the greet pause is finished,
                                  postscreen proceeds anyway:

                                  May 16 21:51:51 harrier postfix/postscreen[29502]: NOQUEUE: reject: RCPT from [208.66.205.36]:53814: 450 4.3.2 Service currently unavailable; from=<newsletter@...>, to=<mungeduser@...>, proto=ESMTP, helo=<smtp36.elabs8.com>
                                  May 16 21:51:54 harrier postfix/postscreen[29502]: warning: dnsblog reply timeout 10s for psbl.surriel.com
                                  May 16 21:51:56 harrier postfix/postscreen[29502]: PASS NEW [208.66.205.36]:53814
                                  May 16 21:51:56 harrier postfix/postscreen[29502]: DISCONNECT [208.66.205.36]:53814

                                  To avoid this, I guess I'd need postscreen_greet_wait to be longer
                                  than the 10-second dnsblog reply timeout? (Is that reply timeout
                                  configurable?)
                                  --
                                  http://rob0.nodns4.us/ -- system administration and consulting
                                  Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                                • Wietse Venema
                                  ... All postscreen versions work that way. When the DNSBL score is not final before the pregreet test completes, the DNSBL test remains undecided, and the test
                                  Message 16 of 25 , May 16, 2013
                                  • 0 Attachment
                                    /dev/rob0:
                                    > In the time since I've been running this, I saw the first thing that
                                    > might be seen as a problem: dnsblog timing out on one of the DNSBL
                                    > lookups:
                                    >
                                    > May 16 21:51:44 harrier postfix/postscreen[29502]: CONNECT from [208.66.205.36]:53814 to [207.223.116.211]:25
                                    > May 16 21:51:44 harrier postfix/dnsblog[29507]: addr 208.66.205.36 listed by domain list.dnswl.org as 127.0.15.0
                                    >
                                    > This gives it a -2 so far, but when the greet pause is finished,
                                    > postscreen proceeds anyway:

                                    All postscreen versions work that way. When the DNSBL score is not
                                    final before the pregreet test completes, the DNSBL test remains
                                    undecided, and the test will be repeated the next time the client
                                    connects.

                                    Increasing the greet-wait to 10+ seconds could result in legitimate
                                    clients hanging up, so I would not recommend that.

                                    You can try to change the DNS resolver timeout/retry behavior:

                                    /etc/resolv.conf:
                                    # Typical default settings shown here. See resolver(5).
                                    options timeout:5 attempts:2 ...

                                    However, this changes all DNS lookups of every program on the system,
                                    and that may be undesirable.

                                    You can instead specify these settings for Postfix only by setting
                                    the RES_OPTIONS environment variable.

                                    /etc/postfix/main.cf:
                                    import_environment = ... RES_OPTIONS=timeout:3 ...

                                    Unfortunately main.cf does not support RES_OPTIONS values that
                                    contain spaces (there is no support for quotes) and multiple
                                    RES_OPTIONS=whatever settings don't add up, so you can override
                                    only one of "timeout" or "attempts" but not both.

                                    From here on things only gets worse. The following information is
                                    only for completeness. I would not recommend anyone to take this
                                    path. To override RES_OPTIONS with spaces and all you would have
                                    to set it in $daemon_directory/postfix-script.

                                    /usr/libexec/postfix/postfix-script:
                                    export RES_OPTIONS; RES_OPTIONS="xxx yyy zzz"

                                    etc/postfix/main.cf:
                                    import_environment = ... RES_OPTIONS ...

                                    This will import an environment setting literally. But it will break
                                    the next time Postfix is updated.

                                    Wietse
                                  • /dev/rob0
                                    ... Do we have any testing to validate this? I m pretty sure I recall from a few years back on the old original SPAM-L list that some Sendmail people[1] were
                                    Message 17 of 25 , May 17, 2013
                                    • 0 Attachment
                                      On Thu, May 16, 2013 at 07:48:24PM -0400, Wietse Venema wrote:
                                      > /dev/rob0:
                                      > > In the time since I've been running this, I saw the first thing
                                      > > that might be seen as a problem: dnsblog timing out on one of
                                      > > the DNSBL lookups:
                                      > >
                                      > > May 16 21:51:44 harrier postfix/postscreen[29502]: CONNECT from [208.66.205.36]:53814 to [207.223.116.211]:25
                                      > > May 16 21:51:44 harrier postfix/dnsblog[29507]: addr 208.66.205.36 listed by domain list.dnswl.org as 127.0.15.0
                                      > >
                                      > > This gives it a -2 so far, but when the greet pause is finished,
                                      > > postscreen proceeds anyway:
                                      >
                                      > All postscreen versions work that way. When the DNSBL score is not
                                      > final before the pregreet test completes, the DNSBL test remains
                                      > undecided, and the test will be repeated the next time the client
                                      > connects.
                                      >
                                      > Increasing the greet-wait to 10+ seconds could result in
                                      > legitimate clients hanging up, so I would not recommend that.

                                      Do we have any testing to validate this? I'm pretty sure I recall
                                      from a few years back on the old original SPAM-L list that some
                                      Sendmail people[1] were saying they used greet pauses in excess of 30
                                      seconds.

                                      > You can try to change the DNS resolver timeout/retry behavior:

                                      Thanks for all that. As it happens, I have a quick fix for this:

                                      $ grep 'dnsblog.*timeout' /var/log/maillog | wc
                                      35 420 3731
                                      $ grep 'dnsblog.*timeout' /var/log/maillog | grep -v surriel | wc
                                      0 0 0

                                      PSBL seems to be a bit slow for me. I've taken it out of my
                                      postscreen_dnsbl_sites; I had only recently added it.

                                      What this shows is that there's no good, risk-free way to test
                                      potential new DNSBLs. No great harm done: at the most, 35 delayed
                                      mails. But could a site which is consistently timing out cause
                                      positive scores to be ignored? Apparently not here:

                                      May 12 05:05:39 harrier postfix/postscreen[17895]: CONNECT from [24.227.47.42]:1362 to [207.223.116.211]:25
                                      May 12 05:05:39 harrier postfix/postscreen[17895]: PREGREET 21 after 0.03 from [24.227.47.42]:1362: EHLO [192.168.2.33]\r\n
                                      May 12 05:05:39 harrier postfix/dnsblog[17901]: addr 24.227.47.42 listed by domain dnsbl.sorbs.net as 127.0.0.7
                                      May 12 05:05:39 harrier postfix/dnsblog[17897]: addr 24.227.47.42 listed by domain b.barracudacentral.org as 127.0.0.2
                                      May 12 05:05:40 harrier postfix/dnsblog[17900]: addr 24.227.47.42 listed by domain zen.spamhaus.org as 127.0.0.4
                                      May 12 05:05:45 harrier postfix/postscreen[17895]: DNSBL rank 6 for [24.227.47.42]:1362
                                      May 12 05:05:45 harrier postfix/postscreen[17895]: NOQUEUE: reject: RCPT from [24.227.47.42]:1362: 550 5.7.1 Service unavailable; client [24.227.47.42] blocked using zen.spamhaus.org; from=<test@...>, to=<therichsheickc@...>, proto=ESMTP, helo=<[192.168.2.33]>
                                      May 12 05:05:45 harrier postfix/postscreen[17895]: DISCONNECT [24.227.47.42]:1362
                                      May 12 05:05:49 harrier postfix/postscreen[17895]: warning: dnsblog reply timeout 10s for psbl.surriel.com
                                      May 12 05:05:59 harrier postfix/dnsblog[17902]: warning: dnsblog_query: lookup error for DNS query 42.47.227.24.psbl.surriel.com: Host or domain name not found. Name service error for name=42.47.227.24.psbl.surriel.com type=A: Host not found, try again

                                      I guess this says that postscreen_dnsbl_action fires at the end of
                                      the greet pause when postscreen_dnsbl_threshold is met, but
                                      postscreen_dnsbl_whitelist_threshold is not calculated. Here's the
                                      same botnet from a different zombie, which does not meet the
                                      threshold, rejected for protocol error:

                                      May 12 05:43:09 harrier postfix/postscreen[19787]: CONNECT from [80.24.21.133]:23652 to [207.223.116.211]:25
                                      May 12 05:43:09 harrier postfix/dnsblog[19790]: addr 80.24.21.133 listed by domain bl.spameatingmonkey.net as 127.0.0.2
                                      May 12 05:43:09 harrier postfix/postscreen[19787]: PREGREET 21 after 0.22 from [80.24.21.133]:23652: EHLO [192.168.2.33]\r\n
                                      May 12 05:43:19 harrier postfix/postscreen[19787]: warning: dnsblog reply timeout 10s for psbl.surriel.com
                                      May 12 05:43:20 harrier postfix/postscreen[19787]: NOQUEUE: reject: RCPT from [80.24.21.133]:23652: 550 5.5.1 Protocol error; from=<test@...>, to=<therichsheickc@...>, proto=ESMTP, helo=<[192.168.2.33]>
                                      May 12 05:43:21 harrier postfix/postscreen[19787]: DISCONNECT [80.24.21.133]:23652

                                      Here's one without the pregreet:

                                      May 13 06:21:09 harrier postfix/postscreen[3805]: CONNECT from [89.121.129.184]:43448 to [207.223.116.211]:25
                                      May 13 06:21:09 harrier postfix/dnsblog[3807]: addr 89.121.129.184 listed by domain b.barracudacentral.org as 127.0.0.2
                                      May 13 06:21:09 harrier postfix/dnsblog[3813]: addr 89.121.129.184 listed by domain zen.spamhaus.org as 127.0.0.11
                                      May 13 06:21:09 harrier postfix/dnsblog[3813]: addr 89.121.129.184 listed by domain zen.spamhaus.org as 127.0.0.4
                                      May 13 06:21:09 harrier postfix/dnsblog[3808]: addr 89.121.129.184 listed by domain bl.mailspike.net as 127.0.0.12
                                      May 13 06:21:15 harrier postfix/postscreen[3805]: DNSBL rank 6 for [89.121.129.184]:43448
                                      May 13 06:21:16 harrier postfix/postscreen[3805]: NOQUEUE: reject: RCPT from [89.121.129.184]:43448: 550 5.7.1 Service unavailable; client [89.121.129.184] blocked using zen.spamhaus.org; from=<watcheslz@...>, to=<mungeduser@...>, proto=ESMTP, helo=<89-121-129-184.romtelecom.net>
                                      May 13 06:21:16 harrier postfix/postscreen[3805]: HANGUP after 0.68 from [89.121.129.184]:43448 in tests after SMTP handshake
                                      May 13 06:21:16 harrier postfix/postscreen[3805]: DISCONNECT [89.121.129.184]:43448
                                      May 13 06:21:19 harrier postfix/postscreen[3805]: warning: dnsblog reply timeout 10s for psbl.surriel.com


                                      [Snip all the good resolver(5) information]


                                      [1] Specifically I am thinking of the late Bruce Gingery, a true
                                      master spamfighter. I will ask about this on SDLU[2] also.
                                      [2] http://spammers.dontlike.us/
                                      --
                                      http://rob0.nodns4.us/ -- system administration and consulting
                                      Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                                    • Viktor Dukhovni
                                      ... It creates a lot of needless congestion on legitimate sending systems even if they don t hang up. Now every message (from a small MTA that does not visit
                                      Message 18 of 25 , May 17, 2013
                                      • 0 Attachment
                                        On Fri, May 17, 2013 at 12:26:13PM -0500, /dev/rob0 wrote:

                                        > > Increasing the greet-wait to 10+ seconds could result in
                                        > > legitimate clients hanging up, so I would not recommend that.
                                        >
                                        > Do we have any testing to validate this? I'm pretty sure I recall
                                        > from a few years back on the old original SPAM-L list that some
                                        > Sendmail people[1] were saying they used greet pauses in excess of 30
                                        > seconds.

                                        It creates a lot of needless congestion on legitimate sending
                                        systems even if they don't hang up.

                                        Now every message (from a small MTA that does not visit often)
                                        starts to take 30s to make a delivery. Queue throughput collapses
                                        and Patrick Raq's MTA can't deliver new mail in a timely fashion.
                                        On the plus side, Wietse and Patrick may finally consider my
                                        "concurrency balooning" suggestion. :-)

                                        Much of the damage to the SMTP infrastructure is done by well-meaning
                                        anti-spam measures. Let's not take it too far.

                                        --
                                        Viktor.
                                      • /dev/rob0
                                        ... snip ... I understand all this and agree. I m not advocating a 30+ second greet pause. My original goal was to reduce delays. Most of those who manage
                                        Message 19 of 25 , May 17, 2013
                                        • 0 Attachment
                                          On Fri, May 17, 2013 at 05:53:47PM +0000, Viktor Dukhovni wrote:
                                          > On Fri, May 17, 2013 at 12:26:13PM -0500, /dev/rob0 wrote:
                                          > Wietse:
                                          > > > Increasing the greet-wait to 10+ seconds could result in
                                          > > > legitimate clients hanging up, so I would not recommend that.
                                          > >
                                          > > Do we have any testing to validate this? I'm pretty sure I
                                          > > recall from a few years back on the old original SPAM-L list
                                          > > that some Sendmail people[1] were saying they used greet
                                          > > pauses in excess of 30 seconds.
                                          >
                                          > It creates a lot of needless congestion on legitimate sending
                                          > systems even if they don't hang up.
                                          >
                                          snip
                                          >
                                          > Much of the damage to the SMTP infrastructure is done by
                                          > well-meaning anti-spam measures. Let's not take it too far.

                                          I understand all this and agree. I'm not advocating a 30+ second
                                          greet pause. My original goal was to reduce delays.

                                          Most of those who manage really busy outbounds will have gone to the
                                          trouble of getting listed on DNS whitelists. And for these outbounds,
                                          an occasional 10-second greet pause is better than "Service currently
                                          unavailable" and PASS NEW.

                                          But I think this is all moot, and my quick fix, to stop querying
                                          psbl.surriel.com, was the best. The moral of the story being, use
                                          DNSBL sites with adequate response times and five nines. It's
                                          probably also moot if the postscreen_dnsbl_threshold score is only
                                          calculated when in excess thereof in case of DNS timeouts.
                                          --
                                          http://rob0.nodns4.us/ -- system administration and consulting
                                          Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                                        • Wietse Venema
                                          ... [begin background material] I mis-understood how postscreen works (I do not constantly stare at Postfix source code, having other things to work on that
                                          Message 20 of 25 , May 17, 2013
                                          • 0 Attachment
                                            /dev/rob0:
                                            >
                                            > I guess this says that postscreen_dnsbl_action fires at the end of
                                            > the greet pause when postscreen_dnsbl_threshold is met, but
                                            > postscreen_dnsbl_whitelist_threshold is not calculated. Here's the

                                            [begin background material]

                                            I mis-understood how postscreen works (I do not constantly stare
                                            at Postfix source code, having other things to work on that pay the
                                            bills).

                                            I thought that the whitelist will be applied only when DNS lookups
                                            complete *before* the pregreet timer expires. That is,

                                            - When some DNS lookup is taking too long, no DNS score is available.

                                            This is consistent with how postscreen whitelisting works for non-DNS
                                            tests. It applies the whitelist threshold only when DNS lookup
                                            completes before the pregreet timer expires.

                                            However, the bullet above is incorrect. When soe DNS lookup takes
                                            too long, a DNS score is available, and the postscreen DNS blocking
                                            code uses that partial score.

                                            This is safe when there are only positive scores (if the partial
                                            client is already over the threshold then the client should be
                                            blocked even if some DNS results are not yet in).

                                            This is less safe when there may also be exculpatory evidence (in
                                            the form of DNSWL lookups). But, sites are usually not listed in
                                            both white and block lists.

                                            [end background material]

                                            I can change postscreen to also use partial scores for whitelisting
                                            of non-DNS tests, and thereby make whitelisting of non-DNS tests
                                            consistent with DNS-based blocking (that's one less WTF factor).
                                            This requires minor code duplication.

                                            Wietse
                                          • Wietse Venema
                                            ... Released as snapshot 20130517. Wietse
                                            Message 21 of 25 , May 17, 2013
                                            • 0 Attachment
                                              Wietse Venema:
                                              > I can change postscreen to also use partial scores for whitelisting
                                              > of non-DNS tests, and thereby make whitelisting of non-DNS tests
                                              > consistent with DNS-based blocking (that's one less WTF factor).
                                              > This requires minor code duplication.

                                              Released as snapshot 20130517.

                                              Wietse
                                            • /dev/rob0
                                              ... For testing I reenabled PSBL, and I ll see what comes in overnight. I thought I could make my own pseudo-DNSBL on a random IP address with blocked ports
                                              Message 22 of 25 , May 17, 2013
                                              • 0 Attachment
                                                On Fri, May 17, 2013 at 10:06:38PM -0400, Wietse Venema wrote:
                                                > Wietse Venema:
                                                > > I can change postscreen to also use partial scores for
                                                > > whitelisting of non-DNS tests, and thereby make whitelisting
                                                > > of non-DNS tests consistent with DNS-based blocking (that's one
                                                > > less WTF factor). This requires minor code duplication.
                                                >
                                                > Released as snapshot 20130517.

                                                For testing I reenabled PSBL, and I'll see what comes in overnight.
                                                I thought I could make my own pseudo-DNSBL on a random IP address
                                                with blocked ports 53, but I need to set up an NS record to point to
                                                that. I'll do that tomorrow if results tonight are inconclusive.
                                                --
                                                http://rob0.nodns4.us/ -- system administration and consulting
                                                Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                                              • Wietse Venema
                                                ... For whitelisting I used a wild-card A record, and for timeout testing I used an NS record that resolves to a firewalled port (a black hole). This
                                                Message 23 of 25 , May 18, 2013
                                                • 0 Attachment
                                                  /dev/rob0:
                                                  > On Fri, May 17, 2013 at 10:06:38PM -0400, Wietse Venema wrote:
                                                  > > Wietse Venema:
                                                  > > > I can change postscreen to also use partial scores for
                                                  > > > whitelisting of non-DNS tests, and thereby make whitelisting
                                                  > > > of non-DNS tests consistent with DNS-based blocking (that's one
                                                  > > > less WTF factor). This requires minor code duplication.
                                                  > >
                                                  > > Released as snapshot 20130517.
                                                  >
                                                  > For testing I reenabled PSBL, and I'll see what comes in overnight.
                                                  > I thought I could make my own pseudo-DNSBL on a random IP address
                                                  > with blocked ports 53, but I need to set up an NS record to point to
                                                  > that. I'll do that tomorrow if results tonight are inconclusive.

                                                  For whitelisting I used a wild-card "A" record, and for timeout
                                                  testing I used an NS record that resolves to a firewalled port (a
                                                  black hole).

                                                  This confirmed that postscreen will now use partial scores to
                                                  whitelist pending non-dnbsbl tests.

                                                  I can make those domain names available for general testing (but
                                                  not now as I am in the middle of a copper-to-fiber conversion).

                                                  Wietse
                                                • /dev/rob0
                                                  Still watching logs, this one just passed by. Probably unrelated to the changes in 20130517, but I was curious about it: May 19 13:24:20 harrier
                                                  Message 24 of 25 , May 19, 2013
                                                  • 0 Attachment
                                                    Still watching logs, this one just passed by. Probably unrelated to
                                                    the changes in 20130517, but I was curious about it:

                                                    May 19 13:24:20 harrier postfix/postscreen[3533]: CONNECT from [188.42.15.19]:48706 to [207.223.116.211]:25
                                                    May 19 13:24:26 harrier postfix/postscreen[3533]: NOQUEUE: reject: RCPT from [188.42.15.19]:48706: 450 4.3.2 Service currently unavailable; from=<bounce@...>, to=<munged@...>, proto=ESMTP, helo=<mail18.consumer-news123.com>
                                                    May 19 13:24:26 harrier postfix/postscreen[3533]: PASS NEW [188.42.15.19]:48706
                                                    May 19 13:24:26 harrier postfix/postscreen[3533]: DISCONNECT [188.42.15.19]:48706

                                                    All is well and good for a non-whitelisted host, but apparently it
                                                    was too quick in coming back to the secondary MX IP address ...

                                                    May 19 13:24:26 harrier postfix/postscreen[3533]: CONNECT from [188.42.15.9]:33610 to [207.223.116.214]:25
                                                    May 19 13:24:26 harrier postfix/postscreen[3533]: WHITELIST VETO [188.42.15.9]:33610

                                                    ... all in the same second, but according to syslog, sequentially
                                                    after having earned whitelist status.

                                                    May 19 13:24:32 harrier postfix/postscreen[3533]: NOQUEUE: reject: RCPT from [188.42.15.9]:33610: 450 4.3.2 Service currently unavailable; from=<bounce@...>, to=<munged@...>, proto=ESMTP, helo=<mail8.consumer-news123.com>
                                                    May 19 13:24:32 harrier postfix/postscreen[3533]: DISCONNECT [188.42.15.9]:33610

                                                    Another six seconds pass before this one is turned away, which
                                                    suggests that the greet pause was repeated. Makes sense, because
                                                    "WHITELIST VETO" means it was not seen before.
                                                    --
                                                    http://rob0.nodns4.us/ -- system administration and consulting
                                                    Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                                                  • Wietse Venema
                                                    ... postscreen does not find the client IP address in the permanent postscreen_access_list, does not find client the IP address in the temporary
                                                    Message 25 of 25 , May 19, 2013
                                                    • 0 Attachment
                                                      /dev/rob0:
                                                      > Still watching logs, this one just passed by. Probably unrelated to
                                                      > the changes in 20130517, but I was curious about it:
                                                      >
                                                      > May 19 13:24:20 harrier postfix/postscreen[3533]: CONNECT from [188.42.15.19]:48706 to [207.223.116.211]:25
                                                      > May 19 13:24:26 harrier postfix/postscreen[3533]: NOQUEUE: reject: RCPT from [188.42.15.19]:48706: 450 4.3.2 Service currently unavailable; from=<bounce@...>, to=<munged@...>, proto=ESMTP, helo=<mail18.consumer-news123.com>
                                                      > May 19 13:24:26 harrier postfix/postscreen[3533]: PASS NEW [188.42.15.19]:48706
                                                      > May 19 13:24:26 harrier postfix/postscreen[3533]: DISCONNECT [188.42.15.19]:48706

                                                      postscreen does not find the client IP address in the permanent
                                                      postscreen_access_list, does not find client the IP address in the
                                                      temporary postscreen_cache_map, logs the "all tests passed" status,
                                                      updates the temporary postscreen_cache_map with the expiration time
                                                      for each test, and forgets the test results.

                                                      > All is well and good for a non-whitelisted host, but apparently it
                                                      > was too quick in coming back to the secondary MX IP address ...
                                                      >
                                                      > May 19 13:24:26 harrier postfix/postscreen[3533]: CONNECT from [188.42.15.9]:33610 to [207.223.116.214]:25
                                                      > May 19 13:24:26 harrier postfix/postscreen[3533]: WHITELIST VETO [188.42.15.9]:33610
                                                      >
                                                      > ... all in the same second, but according to syslog, sequentially
                                                      > after having earned whitelist status.

                                                      postscreen logs "CONNECT from", does not find the client IP address
                                                      in the permanent postscreen_access_list, and does not find the
                                                      client IP address in the temporary postscreen_cache_map. Therefore
                                                      this is handled as a non-whitelisted client that connects to the
                                                      "wrong" IP address.

                                                      Why wasn't the client IP address found in the temporary
                                                      postscreen_cache_map? Maybe silent corruption of the cache database.

                                                      Wietse
                                                    Your message has been successfully submitted and would be delivered to recipients shortly.