smtpd restriction order, rbl dnsbl rhsbl usage -- WAS: Re: Three trivial filtering questions
- On 8/5/2013 2:52 AM, Ronald F. Guilmette wrote:
> Actually, having adjusted my smtpd_recipient_restrictions ratherFair enough. In the mean time I'll share my experience with dnsbl and
> dramatically today, and looking now at the day's maillog file,
> I think that I am entirely less sure that the problem is what
> I said it was earlier. I am now getting at least _some_
> rejects based on SURBL and URIBL, whereas earlier I had not
> yet seen any at all in the maillog.
> I think that I should ask everyone to ignore my earlier question
> about this until I have time to gather more data and see if I can
> see what's actually going on. Perhaps there's nothing wrong at
rhsbl restrictions. YMMV. I use the following *BL restrictions after
all other main.cf restrictions, which is why the absolute numbers are so
low. RBL checks are the most latency expensive built-in smtpd
restrictions so I evaluate them last.
'gcml' is my simple keystroke saving script that greps the mail log for
an input string.
~$ gcml reject|wc -l
~$ gcml reject|grep zen|wc -l
~$ gcml reject|grep barracudacentral|wc -l
~$ gcml reject|grep dbl|grep Unverified|wc -l
~$ gcml reject|grep dbl|grep 'Sender address'|wc -l
Here we see the relative effectiveness of *BLs in general on this MX,
accounting for only 3% of rejections. rhsbls account for 0.3% of
rejections. 45 of 9957 messages may seem not worth the effort, but
that's 45 fewer msgs going through Spamassassin spamd. Body scanning
those 45 eats far more resources than rejecting the other 9912
connections combined, so for me it's worth the extra 3 lines in main.cf.
On 8/5/2013 3:11 AM, Ronald F. Guilmette wrote:
> Last, first, does the order make any difference in the end?
I'm responding a bit out of context to what you were asking with this in
your other post, but the question is a perfect lead-in to demonstrate
why main.cf restriction order matters.
Note above that reject_rhsbl_helo didn't reject a single connection.
This is most likely due to the fact that snowshoe spammers using $domain
in HELO also use $domain in the rDNS string and/or the envelop sender
address. So if we rearrange the restriction order to something like this
then we'll likely see the rhsbl checks rejecting many more connections.
If I put this entire block near the top of my restrictions the numbers
would skyrocket with a much higher percent of that 9957 being rejected
by *BL Restrictions.
With all restrictions under smtpd_recipient_restrictions they are
processed in strict order. The first match wins, so if the rhsbl_helo
check evaluates to true the msg is rejected and no other smtpd
restrictions below it are processed. This strict ordering currently
allows 97% of the spam rejected in smtpd to be killed off with the
fewest resources consumed, first via inbuilt Postfix checks, then by my
various custom tables, and finally by *BL restrictions. Any spam that
runs this smtpd restriction gauntlet is then rejected by header_checks
and if not enters the queue and is piped to spamd for tagging.
This strict evaluation ordering allows one to optimize resource
consumption per spam connection by using the least expensive
restrictions first and most expensive last. This is not only critical
for very high volume MTAs, but also for low power machines or those that
serve multiple functions such as SOHO servers. *BL checks won't
introduce sufficient latency to cause serious delays on the latter class
of machines, but they certainly can on high volume MTAs, even if using a
local resolver and/or rbldnsd. Obviously it's best to reject spam
connections with the least number of DNS queries possible, from a
latency/throughput and/or resource consumption perspective.
I'm sure already know most of this Ron. I'm using this opportunity to
get a recent post into the archives covering these topics, hence the
subject change with added search keywords.
- On 8/5/2013 6:16 PM, Ronald F. Guilmette wrote:
> In message <520023B2.1070709@...>,PCRE tables don't "run externally". They're simply tables. Postfix
> Noel Jones <njones@...> wrote:
>>> OK. Works for me! I just wish that it wasn't necessary to
>>> have to run an external PCRE to catch it, and that the
smtpd processes them by calling libpcre to which it is linked, just as
it is linked with other libraries including libc, libresolv, libssl, etc
that handle other functions:
~$ pmap 10296
10296: smtpd -n smtp -t inet -u -c -o stress= -s 2 ...
b7104000 200K r-x-- /lib/libpcre.so.3.12.1
b7136000 4K rw--- /lib/libpcre.so.3.12.1
b7137000 12K r-x-- /usr/lib/postfix/dict_pcre.so
b713a000 4K r---- /usr/lib/postfix/dict_pcre.so
b713b000 4K rw--- /usr/lib/postfix/dict_pcre.so
b74f3000 88K r-x-- /usr/lib/libsasl2.so.2.0.23
b7509000 4K rw--- /usr/lib/libsasl2.so.2.0.23
b773e000 16K r-x-- /var/spool/postfix/lib/libnss_dns-2.11.3.so
b7742000 4K r---- /var/spool/postfix/lib/libnss_dns-2.11.3.so
b7743000 4K rw--- /var/spool/postfix/lib/libnss_dns-2.11.3.so
b7763000 184K r-x-- /usr/lib/postfix/smtpd
b7792000 8K r---- /usr/lib/postfix/smtpd
b7794000 4K rw--- /usr/lib/postfix/smtpd
b9178000 264K rw--- [ anon ]
bfd19000 132K rw--- [ stack ]
>> I can see where one might get confused. I'll submit a one-line docIndeed. You're not the first to be stumped by the discrepancy between
>> patch rather than argue the point.
> I have no wish to belabor the point either, however for the record
> I am not confused. An FQDN is an FQDN, and [A.B.C.D] quite certainly
> isn't one.
this parameter's behavior and that described in the docs. I commend
Noel for submitting this long overdue patch.