Loading ...
Sorry, an error occurred while loading the content.
 

Milter to block registrars

Expand Messages
  • James B. Byrne
    Without going into a lot of detail and without naming names I wish to know if, at the time of connection to Postfix, there exists any feasible means of
    Message 1 of 12 , May 27, 2014
      Without going into a lot of detail and without naming names I wish to know if,
      at the time of connection to Postfix, there exists any feasible means of
      determining the registrar used by the connecting domain? As well, I would
      like to know is there any practical means of determining at the time of smtp
      connection by direct enquiry of a registrar when the connecting domain was
      registered and block all connections from all non-whitelisted domains
      registered within the past N days?

      I am aware of the 'Day Old Bread' RBL / Greylist is used by SpamAssassin but
      after some investigation I have come to the belief that a registrar is in fact
      behind the latest spam attack we have encountered. Our experience is that by
      the time DOB is updated the domain is no longer generating mail at all. Given
      the remote possibility that any domain registered with that registrar would
      ever have a legitimate reason to contact us I wish to simply deny access to
      our servers from any domain registered with them. Given the equal
      implausibility of a newly registered domain having any legitimate need I wish
      also to block these.

      Does anyone know of any milter projects usable by Postfix that address either
      of these desires?


      --
      *** E-Mail is NOT a SECURE channel ***
      James B. Byrne mailto:ByrneJB@...
      Harte & Lyne Limited http://www.harte-lyne.ca
      9 Brockley Drive vox: +1 905 561 1241
      Hamilton, Ontario fax: +1 905 561 0757
      Canada L8E 3C3
    • Bennett Todd
      Two thoughts. I ve received legitimate email from a registrar where I was listed as a contact for a domain. If no one uses an email address in your domain to
      Message 2 of 12 , May 27, 2014
        Two thoughts.

        I've received legitimate email from a registrar where I was listed as a contact for a domain. If no one uses an email address in your domain to register, that's not a problem.

        And second, whois is the way I query to find out about a domain, answers to questions like who registered it, with which registrar, and when.
        It may be safe to query it directly, but I'm a chicken, when I decided to include whois reports for each distinct IP address that directed Heartbleed attacks at our servers, I rate limited the queries and cached the results.
      • Wietse Venema
        ... Beware, some whois servers enforce rate limits, so this is likely to cause problems for a busy mail server. On the other hand, Postfix has support to block
        Message 3 of 12 , May 27, 2014
          James B. Byrne:
          > Without going into a lot of detail and without naming names I wish to know if,
          > at the time of connection to Postfix, there exists any feasible means of
          > determining the registrar used by the connecting domain? As well, I would

          Beware, some whois servers enforce rate limits, so this is likely
          to cause problems for a busy mail server.

          On the other hand, Postfix has support to block domains by their
          DNS (or MX) service provider: check_mumble_{ns,mx}_access for mumble
          in client, reverse_client, helo, sender, recipient.

          I have used that in the past to block snowshoe spam.

          Wietse
        • LuKreme
          ... Not really. Even if you wrote a milter or local rbl and used something like greylisting to give you time to do the lookups, you would probably run into
          Message 4 of 12 , May 27, 2014
            On 27 May 2014, at 13:19 , James B. Byrne <byrnejb@...> wrote:

            > Without going into a lot of detail and without naming names I wish to know if,
            > at the time of connection to Postfix, there exists any feasible means of
            > determining the registrar used by the connecting domain?

            Not really.

            Even if you wrote a milter or local rbl and used something like greylisting to give you time to do the lookups, you would probably run into problems with making many whois queries. Not only that, but every registrar seems to return info in different forms, so parsing the data will be difficult.

            > As well, I would like to know is there any practical means of determining at the time of smtp connection by direct enquiry of a registrar when the connecting domain was registered and block all connections from all non-whitelisted domains registered within the past N days?

            Same problem.

            > I am aware of the 'Day Old Bread' RBL / Greylist is used by SpamAssassin but
            > after some investigation I have come to the belief that a registrar is in fact
            > behind the latest spam attack we have encountered. Our experience is that by
            > the time DOB is updated the domain is no longer generating mail at all. Given
            > the remote possibility that any domain registered with that registrar would
            > ever have a legitimate reason to contact us I wish to simply deny access to
            > our servers from any domain registered with them. Given the equal
            > implausibility of a newly registered domain having any legitimate need I wish
            > also to block these.

            You could modify a greylist setup to keep track of domains sending you mail and not allow a new domain for x days if you've never seen them before. I doubt this is a good idea, but you could certainly investigate it.

            --
            Dinosaurs are attacking! Throw a barrel!
          • Robert Schetterer
            ... perhaps this helps http://rss.uribl.com/nic/ Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15,
            Message 5 of 12 , May 27, 2014
              Am 27.05.2014 21:19, schrieb James B. Byrne:
              > Without going into a lot of detail and without naming names I wish to know if,
              > at the time of connection to Postfix, there exists any feasible means of
              > determining the registrar used by the connecting domain? As well, I would
              > like to know is there any practical means of determining at the time of smtp
              > connection by direct enquiry of a registrar when the connecting domain was
              > registered and block all connections from all non-whitelisted domains
              > registered within the past N days?
              >
              > I am aware of the 'Day Old Bread' RBL / Greylist is used by SpamAssassin but
              > after some investigation I have come to the belief that a registrar is in fact
              > behind the latest spam attack we have encountered. Our experience is that by
              > the time DOB is updated the domain is no longer generating mail at all. Given
              > the remote possibility that any domain registered with that registrar would
              > ever have a legitimate reason to contact us I wish to simply deny access to
              > our servers from any domain registered with them. Given the equal
              > implausibility of a newly registered domain having any legitimate need I wish
              > also to block these.
              >
              > Does anyone know of any milter projects usable by Postfix that address either
              > of these desires?
              >
              >

              perhaps this helps

              http://rss.uribl.com/nic/


              Best Regards
              MfG Robert Schetterer

              --
              [*] sys4 AG

              http://sys4.de, +49 (89) 30 90 46 64
              Franziskanerstraße 15, 81669 München

              Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
              Vorstand: Patrick Ben Koetter, Marc Schiffbauer
              Aufsichtsratsvorsitzender: Florian Kirstein
            • Marius Gologan
              Whois should definitely not be implemented in automated systems - read ToS of RIPE, ARIN, LACNIC etc. A special-made milter that will dig for details during
              Message 6 of 12 , May 27, 2014
                Whois should definitely not be implemented in automated systems - read ToS
                of RIPE, ARIN, LACNIC etc.
                A special-made milter that will dig for details during the connection time
                is not applicable.
                A secondary benefit of greylist is IP rotation. That will provide you an
                insight about some networks , IP ranges and ISPs.
                Registrars or hosting providers are not behind attacks, but they play a key
                role in providing resources and delisting - notice the delisting rules of
                some popular RBLs for IP classes. Now are there, next day are gone despite
                their own retention policy.

                I would go with reputation (mine or a third-party - the decision depends on
                the messages volume) since some registrars are less tolerable than others,
                volume and percentage are important too.
                For example, you don't want to block domains registered with godaddy just
                because they might have some spamming domains there.

                You can adjust the scoring in spamassassin for uribl.com if you want to be
                more aggressive.
                As a fast doable solution, I would prefer a custom meta rule (uribl.com &
                bayes_90+ & pyzor - maybe) and a shortcircut rule to reduce resources and
                time.
                Plus a script that will collect all those IPs/Domains and put them into
                postfix or rbldnsd to reject next connections more efficiently.

                Geographic rules may help reducing spam, but not in all cases.
                Too many non-existing recipients is also a sign of spam. You can turn some
                of them into spam traps.

                Marius.

                -----Original Message-----
                From: owner-postfix-users@...
                [mailto:owner-postfix-users@...] On Behalf Of James B. Byrne
                Sent: Tuesday, May 27, 2014 10:20 PM
                To: postfix-users-digest@...
                Subject: Milter to block registrars

                Without going into a lot of detail and without naming names I wish to know
                if,
                at the time of connection to Postfix, there exists any feasible means of
                determining the registrar used by the connecting domain? As well, I would
                like to know is there any practical means of determining at the time of smtp
                connection by direct enquiry of a registrar when the connecting domain was
                registered and block all connections from all non-whitelisted domains
                registered within the past N days?

                I am aware of the 'Day Old Bread' RBL / Greylist is used by SpamAssassin but
                after some investigation I have come to the belief that a registrar is in
                fact
                behind the latest spam attack we have encountered. Our experience is that by
                the time DOB is updated the domain is no longer generating mail at all.
                Given
                the remote possibility that any domain registered with that registrar would
                ever have a legitimate reason to contact us I wish to simply deny access to
                our servers from any domain registered with them. Given the equal
                implausibility of a newly registered domain having any legitimate need I
                wish
                also to block these.

                Does anyone know of any milter projects usable by Postfix that address
                either
                of these desires?


                --
                *** E-Mail is NOT a SECURE channel ***
                James B. Byrne mailto:ByrneJB@...
                Harte & Lyne Limited http://www.harte-lyne.ca
                9 Brockley Drive vox: +1 905 561 1241
                Hamilton, Ontario fax: +1 905 561 0757
                Canada L8E 3C3
              • James B. Byrne
                ... I am attempting to be circumspect with respect to this situation. Briefly, I have formed the belief -- based upon evidence of mass changes to the
                Message 7 of 12 , May 27, 2014
                  On Tue, May 27, 2014 15:32, Bennett Todd wrote:
                  > Two thoughts.
                  >
                  > I've received legitimate email from a registrar where I was listed as a
                  > contact for a domain. If no one uses an email address in your domain to
                  > register, that's not a problem.

                  I am attempting to be circumspect with respect to this situation. Briefly, I
                  have formed the belief -- based upon evidence of mass changes to the ownership
                  and contact information of domains I reported as providing fraudulent contact
                  information and acting as ephemeral sources of mass UCE mailings -- that the
                  registrar of these domains is directly involved in the underlying SPAM
                  utterances and is not simply incompetent, inefficient, disinterested or simply
                  lazy. As I say, this is my belief and I do not ask any to share it. However,
                  I am prepared to act upon it in my own interests.

                  Presently I am not in the position to develop my own milter although that may
                  be exactly what I will have to do in the end. I am seeking some sort of
                  add-in to Postfix that will take the incoming email domain and do a check of
                  some local cache acting thereafter according to the contents thereof. If the
                  domain is not already tagged then the milter does a whois on the domain and
                  determines the registrar. If the registrar is blacklisted in our
                  configuration then that domain is tagged, cached, the connection dropped and
                  further connections from the same domain are found in the cache and
                  effectively ignored.

                  Obviously, once the basics for such a milter are in place it is no great
                  stretch to check the registration date and tag those domains as well
                  regardless of registrar. There seem to me no difficulty having uncached,
                  non-whitelisted, domains given a temporary refusal while the whois lookup
                  takes place, thus allowing reasonably spaced whois lookups. If the whois
                  query is not returned in a timely manner (read blocked) then email from the
                  subject domain is refused on a temporary basis until the information is
                  obtained and a determination made.

                  I am fairly confident that in our circumstance the number of whois queries
                  will be rather small and the great majority of domains checked will prove to
                  be spam sources. My enquiry here is aimed at establishing whether this is
                  already done and if not then the feasibility of doing it at all. I am aware
                  of the Prefix WhoIs Milter project but that simply provides additional
                  headers. I am looking for something a little more interventionist. Of
                  course, if I have to do this myself then that would be a good place to start.

                  --
                  *** E-Mail is NOT a SECURE channel ***
                  James B. Byrne mailto:ByrneJB@...
                  Harte & Lyne Limited http://www.harte-lyne.ca
                  9 Brockley Drive vox: +1 905 561 1241
                  Hamilton, Ontario fax: +1 905 561 0757
                  Canada L8E 3C3
                • Eliezer Croitoru
                  ... Well you can use a domain blacklist which a addon will probe any domain you get emails from and verify their registrar and if found out then added to the
                  Message 8 of 12 , May 27, 2014
                    On 05/27/2014 11:33 PM, James B. Byrne wrote:
                    > On Tue, May 27, 2014 15:32, Bennett Todd wrote:
                    >> >Two thoughts.
                    >> >
                    >> >I've received legitimate email from a registrar where I was listed as a
                    >> >contact for a domain. If no one uses an email address in your domain to
                    >> >register, that's not a problem.
                    Well you can use a domain blacklist which a addon will probe any domain
                    you get emails from and verify their registrar and if found out then
                    added to the domain blacklist.
                    Or you can get a daily\hourly report and report them to them main
                    blacklists and by that block them and help the internet be a better
                    place for emails.

                    But notice that a blacklisted domain might not like it.

                    Eliezer
                  • James B. Byrne
                    ... In my case I have a reasonable doubt that the registrar involved is entirely innocent. In fact, on the balance of probabilities I rather think not.
                    Message 9 of 12 , May 27, 2014
                      On Tue, May 27, 2014 16:26, Marius Gologan wrote:
                      >
                      > Whois should definitely not be implemented in automated systems - read ToS
                      > of RIPE, ARIN, LACNIC etc.
                      > A special-made milter that will dig for details during the connection time
                      > is not applicable.
                      > A secondary benefit of greylist is IP rotation. That will provide you an
                      > insight about some networks , IP ranges and ISPs.
                      > Registrars or hosting providers are not behind attacks, but they play a key
                      > role in providing resources and delisting - notice the delisting rules of
                      > some popular RBLs for IP classes. Now are there, next day are gone despite
                      > their own retention policy.

                      In my case I have a reasonable doubt that the registrar involved is entirely
                      innocent. In fact, on the balance of probabilities I rather think not.
                      However, the technique used in these recent spam attacks is that the domains
                      are registered the same day, in some cases the same hour, that the UCE arrives
                      and they are discarded within hours of their first use. It seems that most
                      greylist/blackhole lists are incapable of reacting in such a brief window.

                      >
                      > I would go with reputation (mine or a third-party - the decision depends on
                      > the messages volume) since some registrars are less tolerable than others,
                      > volume and percentage are important too.

                      I am not sure what this means so I have to ask you to explain it to me. I
                      apologise in advance if I appear thick.

                      > For example, you don't want to block domains registered with godaddy just
                      > because they might have some spamming domains there.

                      I am not out to block every registrar, or even most, and hopefully not even a
                      considerable number. Right now it is only one. Based on recent experience I
                      would settle for a timely entry in DOB, but it is not reasonable to expect
                      them to add newly minted domain names within minutes of their registration.

                      >
                      > You can adjust the scoring in spamassassin for uribl.com if you want
                      > to be more aggressive. As a fast doable solution, I would prefer a
                      > custom meta rule (uribl.com & bayes_90+ & pyzor - maybe) and a
                      > shortcircut rule to reduce resources and time.

                      I have examined the messages and actually followed the links contained in
                      some. What is happening is that the same fresh domain is used throughout the
                      UCE and when one follows the message links then javascript is used to redirect
                      one to the desired end address. uribl is no more likely to have the URIs
                      contained in the messages than DOB and for the same reason. The URIs simply
                      did not exist four to eight hours ago, were never used before and will not be
                      used again, at least not for UCE. One at least has been re-purposed as a
                      watering-hole trap.


                      > Plus a script that will collect all those IPs/Domains and put them into
                      > postfix or rbldnsd to reject next connections more efficiently.

                      Yes, I suppose the easiest thing is to simply count the connections and after
                      N then block further receipts from these domains. However, I have observed by
                      visual inspection of the maillog that the UCE originating domains are rotated
                      while sending so that one might have difficulty in picking a suitable time
                      period to match domain connections within.

                      >
                      > Geographic rules may help reducing spam, but not in all cases.
                      > Too many non-existing recipients is also a sign of spam. You can turn some
                      > of them into spam traps.
                      >

                      The domains that I bothered to trace had mail sent from servers in the Czech
                      Republic, Columbia, the USA, Canada, Taiwan, Vietnam, and Mexico. I presume
                      that the dozens that I did not check originate from an equally diffuse
                      collection of places.

                      The non existent addresses is likely a fruitful line to pursue. Is it
                      possible to configure Postfix as shipped to automatically add a connecting IP
                      address to a block list based upon the address that it is attempting delivery
                      to? And then thereafter simply disregard connection attempts from the same
                      source?

                      Thanks,

                      --
                      *** E-Mail is NOT a SECURE channel ***
                      James B. Byrne mailto:ByrneJB@...
                      Harte & Lyne Limited http://www.harte-lyne.ca
                      9 Brockley Drive vox: +1 905 561 1241
                      Hamilton, Ontario fax: +1 905 561 0757
                      Canada L8E 3C3
                    • Bennett Todd
                      Given the situation, perhaps you could set up a resolver that blocks, or that s behind a packet filter that blocks, the IPs of the name servers they re using.
                      Message 10 of 12 , May 27, 2014

                        Given the situation, perhaps you could set up a resolver that blocks, or that's behind a packet filter that blocks, the IPs of the name servers they're using. That would catch it at the NS lookup, and would be no extra traffic, unlike whois.

                      • Patrick Ben Koetter
                        ... If they also run the NS or MX services, you might be able to block them using check_sender_ns_access and/or check_sender_mx_access in Postfix. p@rick --
                        Message 11 of 12 , May 27, 2014
                          * James B. Byrne <byrnejb@...>:
                          > Without going into a lot of detail and without naming names I wish to know if,
                          > at the time of connection to Postfix, there exists any feasible means of
                          > determining the registrar used by the connecting domain? As well, I would
                          > like to know is there any practical means of determining at the time of smtp
                          > connection by direct enquiry of a registrar when the connecting domain was
                          > registered and block all connections from all non-whitelisted domains
                          > registered within the past N days?

                          If they also run the NS or MX services, you might be able to block them using
                          check_sender_ns_access and/or check_sender_mx_access in Postfix.

                          p@rick

                          --
                          [*] sys4 AG

                          https://sys4.de, +49 (89) 30 90 46 64
                          Franziskanerstraße 15, 81669 München

                          Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
                          Vorstand: Patrick Ben Koetter, Marc Schiffbauer
                          Aufsichtsratsvorsitzender: Florian Kirstein
                        • Marius Gologan
                          There are RBLs for domains (aka DBL) that block recent domains (
                          Message 12 of 12 , May 27, 2014
                            There are RBLs for domains (aka DBL) that block recent domains (<= 5 or 7
                            days).
                            Indeed they need some time after "first-seen". Large RBL providers may rely
                            on their own old/large database. If the domain is not there, it must be new
                            (1). If they see multiple queries from different places (reliable), the
                            domain is sending mass mailing (2). Lookup individual sending IP to
                            determine if the domain is not hosted by a large hosting provider (3).
                            Lookup all sending IPs (4).
                            1+2+3 = (new) spamming domain (DBL).
                            1+2+3+4 = snow shoe, block CIDR (along with domain).

                            Unlike blacklist/whitelist, reputational list is based on percentage and
                            time frame.
                            You record not only bad domains, but you track also legit domains, for 1
                            month (time frame). The percentage between good and legit will be the
                            reputation of the registrar (1).
                            Once you start collecting and playing with these details, you will discover
                            more useful clues such as privacy-on/off (2) which can validate reputation
                            (3).

                            The project will take some effort and will not solve spam coming from
                            hijacked accounts hosted by innocent ISPs/ESPs - bayes is more useful here.
                            Some years ago I was tracking IPs that appeared nowhere. After days where
                            registered in South America (lacnic). The only way of protecting my network
                            in this particular case was to block connections from any unassigned IP.

                            As you say, "email is not a secure channel", but it can be tricky for all
                            parties. Personally, I prefer to let them deliver in quarantine and block
                            everything they have later, except 1 case where they use (or he uses) 1 IP
                            for each message per week totaling few thousands (hijacked MTA systems). I
                            admit, I'm a bit out of imagination and only bayes saves the day. Blocking
                            those IPs/Domains will be useless since they never occur second time.


                            -----Original Message-----
                            From: James B. Byrne [mailto:byrnejb@...]
                            Sent: Wednesday, May 28, 2014 12:14 AM
                            To: Marius Gologan
                            Cc: postfix-users-digest@...
                            Subject: RE: Milter to block registrars


                            On Tue, May 27, 2014 16:26, Marius Gologan wrote:
                            >
                            > Whois should definitely not be implemented in automated systems - read
                            > ToS of RIPE, ARIN, LACNIC etc.
                            > A special-made milter that will dig for details during the connection
                            > time is not applicable.
                            > A secondary benefit of greylist is IP rotation. That will provide you
                            > an insight about some networks , IP ranges and ISPs.
                            > Registrars or hosting providers are not behind attacks, but they play
                            > a key role in providing resources and delisting - notice the delisting
                            > rules of some popular RBLs for IP classes. Now are there, next day are
                            > gone despite their own retention policy.

                            In my case I have a reasonable doubt that the registrar involved is entirely
                            innocent. In fact, on the balance of probabilities I rather think not.
                            However, the technique used in these recent spam attacks is that the domains
                            are registered the same day, in some cases the same hour, that the UCE
                            arrives and they are discarded within hours of their first use. It seems
                            that most greylist/blackhole lists are incapable of reacting in such a brief
                            window.

                            >
                            > I would go with reputation (mine or a third-party - the decision
                            > depends on the messages volume) since some registrars are less
                            > tolerable than others, volume and percentage are important too.

                            I am not sure what this means so I have to ask you to explain it to me. I
                            apologise in advance if I appear thick.

                            > For example, you don't want to block domains registered with godaddy
                            > just because they might have some spamming domains there.

                            I am not out to block every registrar, or even most, and hopefully not even
                            a considerable number. Right now it is only one. Based on recent
                            experience I would settle for a timely entry in DOB, but it is not
                            reasonable to expect them to add newly minted domain names within minutes of
                            their registration.

                            >
                            > You can adjust the scoring in spamassassin for uribl.com if you want
                            > to be more aggressive. As a fast doable solution, I would prefer a
                            > custom meta rule (uribl.com & bayes_90+ & pyzor - maybe) and a
                            > shortcircut rule to reduce resources and time.

                            I have examined the messages and actually followed the links contained in
                            some. What is happening is that the same fresh domain is used throughout
                            the UCE and when one follows the message links then javascript is used to
                            redirect one to the desired end address. uribl is no more likely to have
                            the URIs contained in the messages than DOB and for the same reason. The
                            URIs simply did not exist four to eight hours ago, were never used before
                            and will not be used again, at least not for UCE. One at least has been
                            re-purposed as a watering-hole trap.


                            > Plus a script that will collect all those IPs/Domains and put them
                            > into postfix or rbldnsd to reject next connections more efficiently.

                            Yes, I suppose the easiest thing is to simply count the connections and
                            after N then block further receipts from these domains. However, I have
                            observed by visual inspection of the maillog that the UCE originating
                            domains are rotated while sending so that one might have difficulty in
                            picking a suitable time period to match domain connections within.

                            >
                            > Geographic rules may help reducing spam, but not in all cases.
                            > Too many non-existing recipients is also a sign of spam. You can turn
                            > some of them into spam traps.
                            >

                            The domains that I bothered to trace had mail sent from servers in the Czech
                            Republic, Columbia, the USA, Canada, Taiwan, Vietnam, and Mexico. I presume
                            that the dozens that I did not check originate from an equally diffuse
                            collection of places.

                            The non existent addresses is likely a fruitful line to pursue. Is it
                            possible to configure Postfix as shipped to automatically add a connecting
                            IP address to a block list based upon the address that it is attempting
                            delivery to? And then thereafter simply disregard connection attempts from
                            the same source?

                            Thanks,

                            --
                            *** E-Mail is NOT a SECURE channel ***
                            James B. Byrne mailto:ByrneJB@...
                            Harte & Lyne Limited http://www.harte-lyne.ca
                            9 Brockley Drive vox: +1 905 561 1241
                            Hamilton, Ontario fax: +1 905 561 0757
                            Canada L8E 3C3
                          Your message has been successfully submitted and would be delivered to recipients shortly.