Loading ...
Sorry, an error occurred while loading the content.
 

smtpd restriction order, rbl dnsbl rhsbl usage -- WAS: Re: Three trivial filtering questions

Expand Messages
  • Stan Hoeppner
    ... Fair enough. In the mean time I ll share my experience with dnsbl and rhsbl restrictions. YMMV. I use the following *BL restrictions after all other
    Message 1 of 18 , Aug 6, 2013
      On 8/5/2013 2:52 AM, Ronald F. Guilmette wrote:

      > Actually, having adjusted my smtpd_recipient_restrictions rather
      > dramatically today, and looking now at the day's maillog file,
      > I think that I am entirely less sure that the problem is what
      > I said it was earlier. I am now getting at least _some_
      > rejects based on SURBL and URIBL, whereas earlier I had not
      > yet seen any at all in the maillog.
      >
      > I think that I should ask everyone to ignore my earlier question
      > about this until I have time to gather more data and see if I can
      > see what's actually going on. Perhaps there's nothing wrong at
      > all!

      Fair enough. In the mean time I'll share my experience with dnsbl and
      rhsbl restrictions. YMMV. I use the following *BL restrictions after
      all other main.cf restrictions, which is why the absolute numbers are so
      low. RBL checks are the most latency expensive built-in smtpd
      restrictions so I evaluate them last.

      reject_rbl_client zen.spamhaus.org
      reject_rbl_client b.barracudacentral.org
      reject_rhsbl_reverse_client dbl.spamhaus.org
      reject_rhsbl_sender dbl.spamhaus.org
      reject_rhsbl_helo dbl.spamhaus.org

      'gcml' is my simple keystroke saving script that greps the mail log for
      an input string.

      ~$ gcml reject|wc -l
      9957
      ~$ gcml reject|grep zen|wc -l
      149
      ~$ gcml reject|grep barracudacentral|wc -l
      118
      ~$ gcml reject|grep dbl|grep Unverified|wc -l
      15
      ~$ gcml reject|grep dbl|grep 'Sender address'|wc -l
      30

      Here we see the relative effectiveness of *BLs in general on this MX,
      accounting for only 3% of rejections. rhsbls account for 0.3% of
      rejections. 45 of 9957 messages may seem not worth the effort, but
      that's 45 fewer msgs going through Spamassassin spamd. Body scanning
      those 45 eats far more resources than rejecting the other 9912
      connections combined, so for me it's worth the extra 3 lines in main.cf.


      On 8/5/2013 3:11 AM, Ronald F. Guilmette wrote:

      > Last, first, does the order make any difference in the end?


      I'm responding a bit out of context to what you were asking with this in
      your other post, but the question is a perfect lead-in to demonstrate
      why main.cf restriction order matters.

      Note above that reject_rhsbl_helo didn't reject a single connection.
      This is most likely due to the fact that snowshoe spammers using $domain
      in HELO also use $domain in the rDNS string and/or the envelop sender
      address. So if we rearrange the restriction order to something like this

      reject_rhsbl_helo dbl.spamhaus.org
      reject_rhsbl_reverse_client dbl.spamhaus.org
      reject_rhsbl_sender dbl.spamhaus.org
      reject_rbl_client zen.spamhaus.org
      reject_rbl_client b.barracudacentral.org

      then we'll likely see the rhsbl checks rejecting many more connections.
      If I put this entire block near the top of my restrictions the numbers
      would skyrocket with a much higher percent of that 9957 being rejected
      by *BL Restrictions.

      With all restrictions under smtpd_recipient_restrictions they are
      processed in strict order. The first match wins, so if the rhsbl_helo
      check evaluates to true the msg is rejected and no other smtpd
      restrictions below it are processed. This strict ordering currently
      allows 97% of the spam rejected in smtpd to be killed off with the
      fewest resources consumed, first via inbuilt Postfix checks, then by my
      various custom tables, and finally by *BL restrictions. Any spam that
      runs this smtpd restriction gauntlet is then rejected by header_checks
      and if not enters the queue and is piped to spamd for tagging.

      This strict evaluation ordering allows one to optimize resource
      consumption per spam connection by using the least expensive
      restrictions first and most expensive last. This is not only critical
      for very high volume MTAs, but also for low power machines or those that
      serve multiple functions such as SOHO servers. *BL checks won't
      introduce sufficient latency to cause serious delays on the latter class
      of machines, but they certainly can on high volume MTAs, even if using a
      local resolver and/or rbldnsd. Obviously it's best to reject spam
      connections with the least number of DNS queries possible, from a
      latency/throughput and/or resource consumption perspective.

      I'm sure already know most of this Ron. I'm using this opportunity to
      get a recent post into the archives covering these topics, hence the
      subject change with added search keywords.

      --
      Stan
    • Stan Hoeppner
      ... PCRE tables don t run externally . They re simply tables. Postfix smtpd processes them by calling libpcre to which it is linked, just as it is linked
      Message 2 of 18 , Aug 6, 2013
        On 8/5/2013 6:16 PM, Ronald F. Guilmette wrote:
        > In message <520023B2.1070709@...>,
        > Noel Jones <njones@...> wrote:

        >>> OK. Works for me! I just wish that it wasn't necessary to
        >>> have to run an external PCRE to catch it, and that the

        PCRE tables don't "run externally". They're simply tables. Postfix
        smtpd processes them by calling libpcre to which it is linked, just as
        it is linked with other libraries including libc, libresolv, libssl, etc
        that handle other functions:

        ~$ pmap 10296
        10296: smtpd -n smtp -t inet -u -c -o stress= -s 2 ...
        ...
        b7104000 200K r-x-- /lib/libpcre.so.3.12.1
        b7136000 4K rw--- /lib/libpcre.so.3.12.1
        b7137000 12K r-x-- /usr/lib/postfix/dict_pcre.so
        b713a000 4K r---- /usr/lib/postfix/dict_pcre.so
        b713b000 4K rw--- /usr/lib/postfix/dict_pcre.so
        ...
        b74f3000 88K r-x-- /usr/lib/libsasl2.so.2.0.23
        b7509000 4K rw--- /usr/lib/libsasl2.so.2.0.23
        ...
        b773e000 16K r-x-- /var/spool/postfix/lib/libnss_dns-2.11.3.so
        b7742000 4K r---- /var/spool/postfix/lib/libnss_dns-2.11.3.so
        b7743000 4K rw--- /var/spool/postfix/lib/libnss_dns-2.11.3.so
        ...
        b7763000 184K r-x-- /usr/lib/postfix/smtpd
        b7792000 8K r---- /usr/lib/postfix/smtpd
        b7794000 4K rw--- /usr/lib/postfix/smtpd
        b9178000 264K rw--- [ anon ]
        bfd19000 132K rw--- [ stack ]
        total 7664K

        >> I can see where one might get confused. I'll submit a one-line doc
        >> patch rather than argue the point.
        >
        > I have no wish to belabor the point either, however for the record
        > I am not confused. An FQDN is an FQDN, and [A.B.C.D] quite certainly
        > isn't one.

        Indeed. You're not the first to be stumped by the discrepancy between
        this parameter's behavior and that described in the docs. I commend
        Noel for submitting this long overdue patch.

        --
        Stan
      Your message has been successfully submitted and would be delivered to recipients shortly.