Loading ...
Sorry, an error occurred while loading the content.

iptables based spam prevention

Expand Messages
  • Niclas Arndt
    Hi, Sorry if this is slightly off-topic, but at least a bunch of experts are listening. I am using Spamhaus (and other methods) and over time I have amassed a
    Message 1 of 7 , Aug 25 11:11 AM
    • 0 Attachment
      Hi,

      Sorry if this is slightly off-topic, but at least a bunch of experts are listening.

      I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.

      On the other hand, I would like to place as little a load as possible on Spamhaus.

      Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

      Any other related response is of course welcome.

      Thanks in advance.

      Niclas
    • Glenn English
      ... It seems to work well for me. I have a small number of users, though, so I can pretty safely block things without loosing stuff. My mail server s iptables
      Message 2 of 7 , Aug 25 12:42 PM
      • 0 Attachment
        On Aug 25, 2013, at 12:11 PM, Niclas Arndt wrote:

        > I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.
        >
        > On the other hand, I would like to place as little a load as possible on Spamhaus.
        >
        > Here are my questions: Is the iptables approach at all viable in the long run?

        It seems to work well for me. I have a small number of users, though, so I can pretty safely block things without loosing stuff.

        My mail server's iptables INPUT chain jumps to an "SMTP_BLK" chain when the destination is an SMTP port (actually, it splits on TCP/UDP first, but it eventually ends up in a chain for only SMTP hits). It runs through a little whitelisting, then comes to a long list of IPs that are DROPped.

        Those never get to Postfix or Spamhaus, so they don't load anything significantly. The downside is that I have to manually enter the IPs and remove them after a while (I have a shell script to help with this, but it's still a PITA daily duty).

        Fail2ban helps too, but it blocks IPs completely instead of just SMTP hits. And it blocks after a number of hits. Manually is more accurate...

        > Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

        I'm not sure what you mean here. Iptables won't enter an invalid IP. But if you're asking for something that checks whether the IP is active, the only way I know of to do this would involve a lot of parsing whois answers. And that'd be pretty tough.

        If you're asking about something to verify that an IP is indeed a spammer, that's called Spamhaus...

        As for just loading a list of IPs, that's easy. If your iptables packet filter is split into chains, like mine is, a simple shell script does the job -- that's one of the reasons I made my PF so complex.

        --
        Glenn English
      • Noel Jones
        ... I use a postfix check_client_access table that I use as a whitelist/blacklist before the spamhaus lookup and other anti-spam checks. A hash (or even
        Message 3 of 7 , Aug 25 1:42 PM
        • 0 Attachment
          On 8/25/2013 1:11 PM, Niclas Arndt wrote:
          > Hi,
          >
          > Sorry if this is slightly off-topic, but at least a bunch of experts
          > are listening.
          >
          > I am using Spamhaus (and other methods) and over time I have amassed
          > a list of IP ranges that (according to Spamhaus) shouldn't be
          > sending e-mail at all. One problem is that this list tends to become
          > quite long and another is that I would like to verify it so that I
          > don't eventually block legitimate e-mail.
          >
          > On the other hand, I would like to place as little a load as
          > possible on Spamhaus.
          >
          > Here are my questions: Is the iptables approach at all viable in the
          > long run? Is there any non-commercial way to upload a text file
          > containing spamming IP addresses and have it verified for correctness?
          >
          > Any other related response is of course welcome.

          I use a postfix check_client_access table that I use as a
          whitelist/blacklist before the spamhaus lookup and other anti-spam
          checks.

          A hash (or even better, cdb) table can grow to millions of entries
          before it will place any significant load on postfix, so table size
          isn't really a concern.

          Since false positives are always concern, doing the rejects in
          postfix lets me see the sender and recipient before the mail is
          rejected.

          Something like:
          # main.cf
          smtpd_recipient_restrictions =
          permit_mynetworks
          reject_unauth_destination
          check_client_access hash:/etc/postfix/whitelist-blacklist
          ... other anti-spam checks ...




          -- Noel Jones
        • LuKreme
          ... No. This is why RBLS use DNS, because DNS is cheap and it caches automatically. If you are blocking a few sites, (even a few thousand) that is one thing,
          Message 4 of 7 , Aug 25 9:36 PM
          • 0 Attachment
            On 25 Aug 2013, at 12:11 , Niclas Arndt <niclas_arndt@...> wrote:

            > Here are my questions: Is the iptables approach at all viable in the long run?

            No. This is why RBLS use DNS, because DNS is cheap and it caches automatically. If you are blocking a few sites, (even a few thousand) that is one thing, but when you are trying to block millions? That is something else. Do you want IPTables to have millions of IPs?

            --
            and I lift my glass to the Awful Truth / which you can't reveal to the
            Ears of Youth / except to say it isn't worth a dime
          • DTNX Postmaster
            ... Set up your own private rbldnsd; forward requests to it via your local caching resolver. It scales easily to millions of entries, it does not require a
            Message 5 of 7 , Aug 26 1:22 AM
            • 0 Attachment
              On Aug 26, 2013, at 06:20, peter@... wrote:

              > On Aug/25.20:11:49, Niclas Arndt wrote:
              >> Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?
              >
              > Your IP tables will get scary quite rapidly, possibly without bounds.
              > More so if you do not expire old records.
              >
              > The XBL component alone should make IP tables faint.
              >
              > DNS is almost certainly a saner way. In your case, shove your records in
              > your own local DNS server and make a private block list. If you have a fit
              > of insanity, allow other people to query it too...

              Set up your own private rbldnsd; forward requests to it via your local
              caching resolver. It scales easily to millions of entries, it does not
              require a reload of Postfix for updates, the file format is very simple
              and therefore easy to build automatically, and so on.

              Also, you get the benefit of the RBL features of Postfix. You can have
              different replies for different types of IP addresses; have one type be
              rejected directly by postscreen and another type by your recipient
              restrictions, for example.

              Once you have it running it is trivial to add domain based rejections
              as well, and reject HELO hostnames, reverse DNS results, sender domains
              and such.

              HTH,
              Joni
            • Jeroen Geilman
              ... postfix 2.8 and later offer the postscreen(8) triage service, which deals very efficiently with large amounts of DNSBL lookups. Run a local DNS cache on
              Message 6 of 7 , Aug 27 3:01 PM
              • 0 Attachment
                On 08/25/2013 08:11 PM, Niclas Arndt wrote:
                Hi,

                Sorry if this is slightly off-topic, but at least a bunch of experts are listening.

                I am using Spamhaus (and other methods) and over time I have amassed a list of IP ranges that (according to Spamhaus) shouldn't be sending e-mail at all. One problem is that this list tends to become quite long and another is that I would like to verify it so that I don't eventually block legitimate e-mail.

                On the other hand, I would like to place as little a load as possible on Spamhaus.

                Here are my questions: Is the iptables approach at all viable in the long run? Is there any non-commercial way to upload a text file containing spamming IP addresses and have it verified for correctness?

                postfix 2.8 and later offer the postscreen(8) triage service, which deals very efficiently with large amounts of DNSBL lookups.
                Run a local DNS cache on the postfix machine and point postscreen at zen.
                You'll be hitting the spamhaus non-commercial limit long before you hit the local cache's limits.

                This automatically adds and expires DNSBL entries without any effort from you, as a free bonus (this is the biggest problem with your iptables approach.)

                -- 
                J.
                
              • Stan Hoeppner
                ... Given your description this would be the PBL list, typically filled with consumer broadband IP ranges. ... This suggests you ve collected these IPs from
                Message 7 of 7 , Aug 28 9:17 AM
                • 0 Attachment
                  On 8/27/2013 5:01 PM, Jeroen Geilman wrote:
                  > On 08/25/2013 08:11 PM, Niclas Arndt wrote:

                  >> Sorry if this is slightly off-topic, but at least a bunch of experts
                  >> are listening.
                  >>
                  >> I am using Spamhaus (and other methods) and over time I have amassed a
                  >> list of IP ranges that (according to Spamhaus) shouldn't be sending
                  >> e-mail at all.

                  Given your description this would be the PBL list, typically filled with
                  consumer broadband IP ranges.

                  >> One problem is that this list tends to become quite
                  >> long and another is that I would like to verify it so that I don't
                  >> eventually block legitimate e-mail.

                  This suggests you've collected these IPs from your logs and have built a
                  local table. Why? I don't understand what you mean by "verify it".
                  WRT blocking legitimate email that's always a risk with dnsbls, though
                  in the case of the Spamhaus lists a very low risk.

                  >> On the other hand, I would like to place as little a load as possible
                  >> on Spamhaus.

                  See below. But note you are allowed 300K queries/day to Zen.

                  >> Here are my questions: Is the iptables approach at all viable in the
                  >> long run? Is there any non-commercial way to upload a text file
                  >> containing spamming IP addresses and have it verified for correctness?
                  >
                  > postfix 2.8 and later offer the postscreen(8) triage service, which
                  > deals very efficiently with large amounts of DNSBL lookups.
                  > Run a local DNS cache on the postfix machine and point postscreen at zen.
                  > You'll be hitting the spamhaus non-commercial limit long before you hit
                  > the local cache's limits.
                  >
                  > This automatically adds and expires DNSBL entries without any effort
                  > from you, as a free bonus (this is the biggest problem with your
                  > iptables approach.)

                  A local caching resolver is a good start. I use pdns_recursor.

                  If you're concerned about your query load to Spamhaus public Zen servers
                  then block spam connections without using dnsbls, or more precisely,
                  block as much as possible before querying dnsbls. This is what myself
                  and many others do. I.e. use the least expensive tools first and the
                  most expensive last. Expense being defined as time to completion and
                  resources consumed (net/CPU/RAM/disk IO).

                  This not only decreases the load you put on dnsbls, but it also
                  increases throughput due to decreasing remote lookup latency per
                  message. Judicious use of inbuilt Postfix features and local tables can
                  cut per msg latency from a few dozen milliseconds down to a few
                  microseconds.

                  To do this, enable Postcreen in its standard mode. Do not configure
                  dnsbl block lists in postscreen. This will stop nearly all bot traffic.
                  Next, use the "everything under smtpd_recipient_restrictions" main.cf
                  model so you can precisely control the order in which restrictions are
                  processed. Inbuilt Postfix restrictions are the least expensive, then
                  CDB, hash, and CIDR tables. REGEXP and PCRE tables are a little more
                  expensive, possibly much more if tables are large (and without loops)
                  and expressions complex. Header checks are normally next in expense
                  followed by body checks, and local policy daemons and content filters
                  can be more expensive still.

                  So as a general starting point you'd want something similar to this.
                  This configuration assumes the Postfix server is an MX as well as a
                  submission relay.

                  smtpd_recipient_restrictions =

                  # These are inbuilt Postfix restrictions

                  permit_mynetworks
                  permit_sasl_authenticated
                  reject_unauth_destination
                  reject_unknown_reverse_client_hostname
                  reject_non_fqdn_sender
                  reject_non_fqdn_helo_hostname
                  reject_invalid_helo_hostname
                  reject_unlisted_recipient

                  # These check a hash table of domains which have sent spam here
                  # or that have been obtained via intelligence (A/S mailing lists)
                  # and check dnswl.org for sender that should be allowed through

                  check_reverse_client_hostname_access hash:/etc/postfix/blacklist
                  check_helo_access hash:/etc/postfix/blacklist
                  check_sender_access hash:/etc/postfix/blacklist
                  check_client_access hash:/etc/postfix/whitelist
                  permit_dnswl_client list.dnswl.org=127.0.[2..14].[2..3]

                  # This checks a rather larger CIDR table of snowshoe spammer netblocks
                  # that I've been building for ~4 years now

                  check_client_access /etc/postfix/spammer

                  # If none of the previous restrictions reject the spam I then query
                  # various [dns|rhs]bls.

                  reject_rbl_client zen.spamhaus.org
                  reject_rbl_client b.barracudacentral.org
                  reject_rbl_client psbl.surriel.com
                  reject_rhsbl_reverse_client dbl.spamhaus.org
                  reject_rhsbl_sender dbl.spamhaus.org
                  reject_rhsbl_helo dbl.spamhaus.org

                  If you want to be able to score [dns|rhs]bl results you can use a policy
                  filter here such as postfwd or policyd instead of the simple
                  reject/allow offered by these Postfix restrictions.

                  Combining this with a local caching DNS resolver as Jeroen suggested you
                  should cut down on your dnsbl query load pretty substantially.

                  Now, regarding the list you've already built, if you would like I can
                  provide you a tool that will automatically convert it into a Postfix
                  CIDR table that you can use with check_client_access. Though if you
                  built that list in the manner I'm guessing, then I'd recommend against
                  it. Some of the IPs may be legit systems that were temporarily listed
                  due to malware outbreak or similar, and are permanently in your file.

                  --
                  Stan
                Your message has been successfully submitted and would be delivered to recipients shortly.