Loading ...
Sorry, an error occurred while loading the content.

Re: [feature request] Subzero postscreen/dnsblog score to bypass after-220 tests?

Expand Messages
  • /dev/rob0
    On Fri, Apr 12, 2013 at 04:39:29AM -0500, Stan Hoeppner wrote ... I did think of this, and yes, it would save us the pain which seems to hit every 30 days, as
    Message 1 of 9 , Apr 12, 2013
    • 0 Attachment
      On Fri, Apr 12, 2013 at 04:39:29AM -0500, Stan Hoeppner wrote
      Re: scripting a list of Google outbound CIDRs:
      > This seems quite a bit less effort than Wietse adding the feature
      > you requested. The end result is nearly identical, at least for
      > the Google case, and can easily be extended to cover other domains.

      I did think of this, and yes, it would save us the pain which seems
      to hit every 30 days, as the after-220 tests for gmail expire. But
      extending it to cover other domains would not scale well. Which
      domains? What's the structure of their SPF records?

      When you "easily extend" this idea it becomes much more onerous. And
      still sitting out there are those unused DNSWL scores.

      Yes, unused. As it stands I could drop those checks from my config
      without noticing a change. There is very little overlap between the
      DNSWLs (I currently use SWL and dnswl.org) and reasonable, well-run
      DNSBLs. In my experience a few of the spamtrap-driven automated
      DNSBLs occasionally list a dnswl.org whitelisted host, but I don't
      recall having seen an instance where whitelisting prevented a
      rejection. And I have never found a blacklist entry for the (much
      smaller, I think) SWL zone.

      A DNSWL entry says two things:
      1. This is a real MTA, not a zombie
      2. At one point someone trustworthy thought it was not
      spammer-controlled

      Case 1 mostly entitles it to speak to smtpd, unless of course
      offsetting DNSBL scores overcome the whitelist score. By continuing
      on to check DNSBLs, Case 2 is addressed.

      I believe that DNS-based whitelisting will grow in importance,
      especially in the IPv6 world. I expect to move into IPv6 with a
      default-deny policy, where non-whitelisted hosts are rejected.

      > And with this method the Google outbounds skip all Postscreen
      > processing entirely, not just the after 220 tests.

      I wouldn't want that. :) If one of these providers is seriously
      compromised, they'll be blacklisted, and I would want to check for
      that. I don't give Google my absolute trust. I think they may have
      improved, but I know they're not infallible.
      --
      http://rob0.nodns4.us/ -- system administration and consulting
      Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
    • Reindl Harald
      ... how do you imagine this working? in this case it would be better you stay at ipv4 at all instead answer AAA dns-requests which may be preferred from
      Message 2 of 9 , Apr 12, 2013
      • 0 Attachment
        Am 12.04.2013 16:52, schrieb /dev/rob0:
        > I believe that DNS-based whitelisting will grow in importance,
        > especially in the IPv6 world. I expect to move into IPv6 with a
        > default-deny policy, where non-whitelisted hosts are rejected

        how do you imagine this working?

        in this case it would be better you stay at ipv4 at all instead
        answer AAA dns-requests which may be preferred from dual-stack
        machines try to deliver to your customer

        it does not work that anybody who wants to send you e-mail he
        must prove that he is no spammer, really this does not work
      • Wietse Venema
        ... On second consideration, this can be done as follows: - One parameter with the (negative) postscreen_dnsbl_sites score that is needed to allow the client
        Message 3 of 9 , Apr 23, 2013
        • 0 Attachment
          On Fri, Apr 12, 2013 at 06:34:24AM -0400, Wietse Venema wrote:
          > /dev/rob0:
          > > I finally got around to my upgrade to 2.11-20130405 and was watching
          > > logs. A gmail message fell afoul of the after-220 tests; each time it
          > > came from a different host. Each one got a "PASS NEW" and of course
          > > the "450 4.3.2 Service currently unavailable" rejection.
          > >
          > > These gmail outbounds are all listed in list.dnswl.org as 127.0.5.1,
          > > and I give that a negative score in my postscreen_dnsbl_sites. So
          > > with no offsetting DNSBL scores, these hosts all got a subzero score.
          > > It would be nice if we could put those whitelist scores to work, and
          > > not have to maintain so big of a postscreen_access_list whitelist.
          >
          > Disabling tests based on DNSWL score would make sense (currently
          > they "disable" DNSBL tests only). Perhaps this needs a "disable"
          > flag in the postscreen cache.

          On second consideration, this can be done as follows:

          - One parameter with the (negative) postscreen_dnsbl_sites score
          that is needed to allow the client to skip tests.

          - One parameter with the names of tests that are skipped (using
          !name to exclude a name, and static:all to match everything).
          This may include "greet" to cancel a "greet wait" in progress.

          The procedure is: postscreen does a postscreen_dnsbl_sites query
          for the client IP address. If the score satifies the threshold in
          the first parameter, then all tests with a name that matches the
          second parameter will be skipped until the next postscreen_dnsbl_sites
          query for that client IP address (i.e. after postscreen_dnsbl_ttl).

          Wietse
        • /dev/rob0
          ... postscreen_skip_tests_threshold , or should there be a _dnsbl or _dnswl in there? postscreen_dnsbl_skip_tests_threshold is good because it lumps it in
          Message 4 of 9 , Apr 23, 2013
          • 0 Attachment
            On Tue, Apr 23, 2013 at 08:05:34PM -0400, Wietse Venema wrote:
            > On Fri, Apr 12, 2013 at 06:34:24AM -0400, Wietse Venema wrote:
            > > /dev/rob0:
            > > > I finally got around to my upgrade to 2.11-20130405 and was
            > > > watching logs. A gmail message fell afoul of the after-220
            > > > tests; each time it came from a different host. Each one got
            > > > a "PASS NEW" and of course the "450 4.3.2 Service currently
            > > > unavailable" rejection.
            > > >
            > > > These gmail outbounds are all listed in list.dnswl.org as
            > > > 127.0.5.1, and I give that a negative score in my
            > > > postscreen_dnsbl_sites. So with no offsetting DNSBL scores,
            > > > these hosts all got a subzero score. It would be nice if we
            > > > could put those whitelist scores to work, and not have to
            > > > maintain so big of a postscreen_access_list whitelist.
            > >
            > > Disabling tests based on DNSWL score would make sense (currently
            > > they "disable" DNSBL tests only). Perhaps this needs a "disable"
            > > flag in the postscreen cache.
            >
            > On second consideration, this can be done as follows:
            >
            > - One parameter with the (negative) postscreen_dnsbl_sites score
            > that is needed to allow the client to skip tests.

            "postscreen_skip_tests_threshold", or should there be a _dnsbl or
            _dnswl in there? "postscreen_dnsbl_skip_tests_threshold" is good
            because it lumps it in with the other postscreen_dnsbl_* settings
            and makes it clear that this is associated with the DNSBL lookups.
            OTOH _dnswl almost does this too, and it's more accurate. To see how
            it looks: "postscreen_dnswl_skip_tests_threshold".

            > - One parameter with the names of tests that are skipped (using
            > !name to exclude a name, and static:all to match everything).
            > This may include "greet" to cancel a "greet wait" in progress.

            As for "type:name", like postscreen_access_list, you're going to want
            to discourage lookups which might slow this down. I guess only pcre,
            regexp, static, and texthash would be suitable for this? I did not
            include such a warning in the proposed documentation below.

            I think you might want to consider an "all", because that's used in
            numerous other places and meets the "minimum surprise" ideal.


            * Postscreen Skippable Test Names *

            before-220 after-220
            ========== =========
            bare_newline
            blacklist
            greet
            mx_policy
            non_smtp_command
            pipelining
            static:all


            I don't think there's much point in skipping the blacklist test. It
            would be very strange if the postscreen_access_list lookup came in
            after the DNSBL lookups. Furthermore, if I listed something there
            that I don't want to see, it should never pass. But maybe someone
            would want this? A test can be "skipped" even if already completed.
            (But what if the postscreen_access_list result is reject? Shouldn't
            that be done immediately, before the DNSBL lookups are in?)

            Likewise, I think the mx_policy (postscreen_whitelist_interfaces)
            test should also be absolute. If a client is not connecting on the
            proper IP address, this should be cause for at least having it talk
            to postscreen and try again later. But again, maybe someone would
            trust the DNS whitelists' judgment?

            I like the idea of two umbrella categories, before-220 and after-220,
            in the spirit of inet_interfaces' "all" and "loopback-only". But
            there's only three per category, so this is not major at this point.
            (I suppose in the future more tests could be added.)

            I think the default should be either "after-220" or "greet,
            after-220". Typically the result would come in during a greet pause,
            and even though it's only a few seconds, it can add up in the Big
            Scheme of Things.

            > The procedure is: postscreen does a postscreen_dnsbl_sites query
            > for the client IP address. If the score satifies the threshold
            > in the first parameter, then all tests with a name that matches
            > the second parameter will be skipped until the next
            > postscreen_dnsbl_sites query for that client IP address (i.e.
            > after postscreen_dnsbl_ttl).

            """
            postscreen_skip_tests (default: greet, after-220)

            Allow a remote SMTP client with a score less than or equal to
            postscreen_skip_tests_threshold based on its combined DNSBL
            score as defined with the postscreen_dnsbl_sites parameter,
            to skip the listed tests, if enabled. Specify zero or more of
            blacklist, greet, mx_policy (these three collectively can be
            "before-220"), bare_newline, non_smtp_command, pipelining
            (these three collectively can be "after-220"), or
            "static:all" to skip all postscreen tests except for the
            DNSBL test itself. Specify "!pattern" to exclude a test from
            the list.

            Example:

            /etc/postfix/main.cf:
            postscreen_dnsbl_sites = dnsbl.example.org,
            whitelist.example.com*-1
            postscreen_skip_tests = !blacklist, !mx_policy,
            static:all

            This feature is available in Postfix 2.11.

            postscreen_skip_tests_threshold (default: -1)

            The inclusive upper bound for allowing a remote SMTP client,
            based on its combined DNSBL score as defined with the
            postscreen_dnsbl_sites parameter, to bypass the tests listed
            in the postscreen_skip_tests parameter.

            Note: this typically would be a negative number, and it only
            makes sense when using DNS whitelists with negative weights
            in the postscreen_dnsbl_sites list. See the example at
            postscreen_skip_tests.

            This feature is available in Postfix 2.11.
            """

            I hope this is getting closer? The only point of confusion about it
            in my mind is whether/how to skip the blacklist test. Should
            postscreen, knowing "blacklist" is in the postscreen_skip_tests list,
            await the dnsblog results for a blacklisted client? Why? It's
            certainly not going to hold up a postscreen_access_list "permit"
            client.

            Thanks again for considering this.
            --
            http://rob0.nodns4.us/ -- system administration and consulting
            Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
          • /dev/rob0
            Here s a proposed diff for the POSTSCREEN_README: rob0@harrier:~/stuff/postscreen.dnswl$ diff -Nru POSTSCREEN_README* ... +++ POSTSCREEN_README.new
            Message 5 of 9 , Apr 24, 2013
            • 0 Attachment
              Here's a proposed diff for the POSTSCREEN_README:

              rob0@harrier:~/stuff/postscreen.dnswl$ diff -Nru POSTSCREEN_README*
              --- POSTSCREEN_README 2013-04-12 03:34:16.000000000 +0000
              +++ POSTSCREEN_README.new 2013-04-24 21:04:06.155395154 +0000
              @@ -245,6 +245,7 @@

              * Pregreet test
              * DNS White/blacklist test
              + * Skipping other tests for whitelisted clients
              * When tests fail before the 220 SMTP server greeting

              Pregreet test
              @@ -315,6 +316,17 @@
              the combined DNSBL score is equal to or greater than the threshold. See "When
              tests fail before the 220 SMTP server greeting" below.

              +Skipping other tests for whitelisted clients
              +
              +The postscreen_skip_tests parameter lists the short names of tests which will
              +be skipped if a client's combined DNSBL score is less than or equal to
              +postscreen_skip_tests_threshold. This only makes sense when using whitelists
              +with negative weights in the postscreen_dnsbl_sites list.
              +
              +The tests which can be skipped are all but the DNSBL test itself. The default
              +is to perform the blacklist and MX policy tests, but skip the greet test and
              +all the "deep protocol" tests, described below.
              +
              When tests fail before the 220 SMTP server greeting

              When the client address matches the permanent blacklist, or when the client
              @@ -612,6 +624,7 @@
              postscreen_dnsbl_threshold = 2
              postscreen_dnsbl_sites = zen.spamhaus.org*2
              bl.spamcop.net*1 b.barracudacentral.org*1
              + list.dnswl.org*-1 swl.spamhaus.org*-1

              Note: if your DNSBL queries have a "secret" in the domain name, you must
              censor this information from the postscreen(8) SMTP replies. For example:
              --
              http://rob0.nodns4.us/ -- system administration and consulting
              Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
            • Wietse Venema
              ... This will not skip (or otherwise overrule) the *static* access list that is queried before DNS lookup. Normally a client must pass *dynamic* test X to
              Message 6 of 9 , Apr 24, 2013
              • 0 Attachment
                /dev/rob0:
                > Here's a proposed diff for the POSTSCREEN_README:
                >
                > rob0@harrier:~/stuff/postscreen.dnswl$ diff -Nru POSTSCREEN_README*
                > --- POSTSCREEN_README 2013-04-12 03:34:16.000000000 +0000
                > +++ POSTSCREEN_README.new 2013-04-24 21:04:06.155395154 +0000
                > @@ -245,6 +245,7 @@
                >
                > * Pregreet test
                > * DNS White/blacklist test
                > + * Skipping other tests for whitelisted clients

                This will not "skip" (or otherwise overrule) the *static* access
                list that is queried before DNS lookup.

                Normally a client must "pass *dynamic* test X" to become temporarily
                whitelisted for test X. Instead of passing test X, the new feature
                temporarily whitelists the client for test X based on DNSBL score.

                In theory there could be different DNSBL thresholds for different
                tests. But I expect that no sane person would use Postfix that
                way.

                Wietse
              Your message has been successfully submitted and would be delivered to recipients shortly.