Loading ...
Sorry, an error occurred while loading the content.

introducing mopher, the mail gopher

Expand Messages
  • Manuel Badzong
    Hi, I would like to introduce mail gopher, a new all-in-one, MIT-licensed mail filter. Mopher is designed to be lightweight, modular and extensible, has
    Message 1 of 8 , Jun 13, 2013
    View Source
    • 0 Attachment
      Hi,

      I would like to introduce mail gopher, a new all-in-one, MIT-licensed
      mail filter.

      Mopher is designed to be lightweight, modular and extensible, has
      several unique features and uses a very flexible and customizable
      configuration syntax that is very similar to the common firewall
      rule-lists some of us are already familiar with.

      Mopher can:

      + tarpit hosts
      + greylist hosts
      + greylist based on sender/recipient tuples
      + greylist based on sender-domain/recipient tuples
      + auto-whitelist hosts
      + auto-whitelist based on sender/recipient tuples
      + auto-whitelist based on sender-domain/recipient tuples
      + query black- and whitelists
      + query for SPF records
      + speak with spamassassin (through spamd)
      + reject during any protocol stage
      + act on body-size
      + count connections by hosts
      + count failed/successful delivery attempts by hosts
      + inject headers with all available information
      + log all available information (in a format of choice)
      + archive mails

      Mopher has:

      + a db-independent data backend
      + dynamically loadable modules
      + extensible syntax (by modules)
      + well structured default logging

      Mopher supports:

      + most libmilter features
      + Berkeley DB
      + MySQL
      + libspf2
      + PSL (by Mozilla, see http://publicsuffix.org/)

      Mopher compiles and runs on:

      + GNU/Linux
      + NetBSD
      + FreeBSD

      Mopher runs on a couple of production servers, I ran the daemon
      (mopherd) extensively through valgrind and tested it on several
      occasions with smtp-source.

      Due to its modular design, package maintainers can split up a large
      build into several packages and therefore avoid unwanted dependencies.
      A pkgsrc-package already exists (see pkgsrc-wip) and I'll probably
      create a Debian-/RPM-package if nobody else does it in the meantime.

      Mopher ist extensible, hence there are several things that could be
      added to mopher in the near future:

      + legacy BDB support
      + SQLite support
      + PostgreSQL support
      + NoSQL archiving support (any backend possible)
      + DCC support
      + DKIM verification support
      + ...

      Mopher is hosted on GitHub, has a Mailing-List (with some useful
      configuration examples) and could some day also get a Wiki:

      + https://github.com/badzong/mopher
      + https://groups.google.com/group/mopher


      Thank you all very much for reading and I hope some of you will give it
      a shot.

      Cheers,

      Manuel


      P.S. Feedback is always welcome; either public or private.
    • postfix
      forgot LDAP support? suomi
      Message 2 of 8 , Jun 14, 2013
      View Source
      • 0 Attachment
        forgot LDAP support?

        suomi

        On 2013-06-14 08:50, Manuel Badzong wrote:
        > Hi,
        >
        > I would like to introduce mail gopher, a new all-in-one, MIT-licensed
        > mail filter.
        >
        > Mopher is designed to be lightweight, modular and extensible, has
        > several unique features and uses a very flexible and customizable
        > configuration syntax that is very similar to the common firewall
        > rule-lists some of us are already familiar with.
        >
        > Mopher can:
        >
        > + tarpit hosts
        > + greylist hosts
        > + greylist based on sender/recipient tuples
        > + greylist based on sender-domain/recipient tuples
        > + auto-whitelist hosts
        > + auto-whitelist based on sender/recipient tuples
        > + auto-whitelist based on sender-domain/recipient tuples
        > + query black- and whitelists
        > + query for SPF records
        > + speak with spamassassin (through spamd)
        > + reject during any protocol stage
        > + act on body-size
        > + count connections by hosts
        > + count failed/successful delivery attempts by hosts
        > + inject headers with all available information
        > + log all available information (in a format of choice)
        > + archive mails
        >
        > Mopher has:
        >
        > + a db-independent data backend
        > + dynamically loadable modules
        > + extensible syntax (by modules)
        > + well structured default logging
        >
        > Mopher supports:
        >
        > + most libmilter features
        > + Berkeley DB
        > + MySQL
        > + libspf2
        > + PSL (by Mozilla, see http://publicsuffix.org/)
        >
        > Mopher compiles and runs on:
        >
        > + GNU/Linux
        > + NetBSD
        > + FreeBSD
        >
        > Mopher runs on a couple of production servers, I ran the daemon
        > (mopherd) extensively through valgrind and tested it on several
        > occasions with smtp-source.
        >
        > Due to its modular design, package maintainers can split up a large
        > build into several packages and therefore avoid unwanted dependencies.
        > A pkgsrc-package already exists (see pkgsrc-wip) and I'll probably
        > create a Debian-/RPM-package if nobody else does it in the meantime.
        >
        > Mopher ist extensible, hence there are several things that could be
        > added to mopher in the near future:
        >
        > + legacy BDB support
        > + SQLite support
        > + PostgreSQL support
        > + NoSQL archiving support (any backend possible)
        > + DCC support
        > + DKIM verification support
        > + ...
        >
        > Mopher is hosted on GitHub, has a Mailing-List (with some useful
        > configuration examples) and could some day also get a Wiki:
        >
        > + https://github.com/badzong/mopher
        > + https://groups.google.com/group/mopher
        >
        >
        > Thank you all very much for reading and I hope some of you will give it
        > a shot.
        >
        > Cheers,
        >
        > Manuel
        >
        >
        > P.S. Feedback is always welcome; either public or private.
        >
      • Bastian Blank
        ... How does it relate to Postfix? Postfix already does this with a bit of help. ... Bad idea in userspace. Bad idea in practice, you want to get rid of them
        Message 3 of 8 , Jun 14, 2013
        View Source
        • 0 Attachment
          On Fri, Jun 14, 2013 at 08:50:42AM +0200, Manuel Badzong wrote:
          > I would like to introduce mail gopher, a new all-in-one, MIT-licensed
          > mail filter.

          How does it relate to Postfix? Postfix already does this with a bit of
          help.

          > Mopher can:
          > + tarpit hosts

          Bad idea in userspace. Bad idea in practice, you want to get rid of them
          as fast as possible.

          > + greylist hosts
          > + greylist based on sender/recipient tuples
          > + greylist based on sender-domain/recipient tuples
          > + auto-whitelist hosts
          > + auto-whitelist based on sender/recipient tuples
          > + auto-whitelist based on sender-domain/recipient tuples
          > + query black- and whitelists
          > + query for SPF records

          Normal policy protocol, properly used and tested since years.

          > + speak with spamassassin (through spamd)

          SMTP/LMTP proxy.

          > + reject during any protocol stage

          Postfix.

          > + act on body-size

          Policy.

          > + count connections by hosts

          Built-in.

          > + count failed/successful delivery attempts by hosts

          fail2ban. What do you want to do with this information?

          > + inject headers with all available information

          PREPEND.

          > + log all available information (in a format of choice)

          syslog.

          > + archive mails

          Not the purpose of a MTA, the MDA is properly capable of doing so.

          > Mopher has:
          > + a db-independent data backend

          Unlikely. The filesystem is a DB, a directory to be exact.

          > + dynamically loadable modules

          Exists as patch for Postfix, but not portable enough.

          > + extensible syntax (by modules)

          Urgs.

          > + well structured default logging

          Postfix does this.

          > Mopher supports:
          > + most libmilter features

          Aha, so no MTA at all.

          > + Berkeley DB
          > + MySQL

          I see no "real" DB.

          > + libspf2

          Nice try.

          > + PSL (by Mozilla, see http://publicsuffix.org/)

          What is the use for this? This all is focused on web.

          > Mopher compiles and runs on:
          > + GNU/Linux
          > + NetBSD
          > + FreeBSD

          Impressive, not.

          Bastian

          --
          You canna change the laws of physics, Captain; I've got to have thirty minutes!
        • Petar Bogdanovic
          ... It s a milter that some people on this list might find useful. ... So kernel space then? ... Whitelisting based on the amount of successfully delivered
          Message 4 of 8 , Jun 14, 2013
          View Source
          • 0 Attachment
            On Fri, Jun 14, 2013 at 12:08:00PM +0200, Bastian Blank wrote:
            > On Fri, Jun 14, 2013 at 08:50:42AM +0200, Manuel Badzong wrote:
            > > I would like to introduce mail gopher, a new all-in-one, MIT-licensed
            > > mail filter.
            >
            > How does it relate to Postfix?

            It's a milter that some people on this list might find useful.


            > > Mopher can:
            > > + tarpit hosts
            >
            > Bad idea in userspace.

            So kernel space then?


            > > + count failed/successful delivery attempts by hosts
            >
            > What do you want to do with this information?

            Whitelisting based on the amount of successfully delivered mails is
            probably the best example.


            > > + PSL (by Mozilla, see http://publicsuffix.org/)
            >
            > What is the use for this?

            It helps with domain-based greylisting. There are no simple rules when
            figuring out the registered part of a fqdn.


            Petar Bogdanovic
          • Petar Bogdanovic
            ... Yes. And probably other items too. It s really an open-end list.. Petar Bogdanovic
            Message 5 of 8 , Jun 14, 2013
            View Source
            • 0 Attachment
              On Fri, Jun 14, 2013 at 11:55:27AM +0200, postfix wrote:
              > forgot LDAP support?

              Yes. And probably other items too. It's really an open-end list..

              Petar Bogdanovic
            • Bastian Blank
              ... So it only supports what the milter server can do. ... A milter can t do that anyway, the communication is controlled by the other side of the milter
              Message 6 of 8 , Jun 14, 2013
              View Source
              • 0 Attachment
                On Fri, Jun 14, 2013 at 12:37:11PM +0200, Petar Bogdanovic wrote:
                > On Fri, Jun 14, 2013 at 12:08:00PM +0200, Bastian Blank wrote:
                > > On Fri, Jun 14, 2013 at 08:50:42AM +0200, Manuel Badzong wrote:
                > > > I would like to introduce mail gopher, a new all-in-one, MIT-licensed
                > > > mail filter.
                > > How does it relate to Postfix?
                > It's a milter that some people on this list might find useful.

                So it only supports what the milter server can do.

                > > > Mopher can:
                > > > + tarpit hosts
                > > Bad idea in userspace.
                > So kernel space then?

                A milter can't do that anyway, the communication is controlled by the
                other side of the milter connection. You can wait a long time for each
                response, but this does not get you anything.

                > > > + count failed/successful delivery attempts by hosts
                > > What do you want to do with this information?
                > Whitelisting based on the amount of successfully delivered mails is
                > probably the best example.

                You need whitelisting for high volume senders, because they split stuff
                over larger address ranges. Those also produce a high rejection rate
                (not: ratio).

                > > > + PSL (by Mozilla, see http://publicsuffix.org/)
                > > What is the use for this?
                > It helps with domain-based greylisting. There are no simple rules when
                > figuring out the registered part of a fqdn.

                So you do greylisting based on DNS reverse lookups?

                Bastian

                --
                Extreme feminine beauty is always disturbing.
                -- Spock, "The Cloud Minders", stardate 5818.4
              • Benny Pedersen
                ... patch postfix to not accept mails with dns A/AAAA records, there is ignorants everywhere -- senders that put my email into body content will deliver it to
                Message 7 of 8 , Jun 14, 2013
                View Source
                • 0 Attachment
                  Bastian Blank skrev den 2013-06-14 12:08:

                  >> + PSL (by Mozilla, see http://publicsuffix.org/)
                  > What is the use for this? This all is focused on web.

                  patch postfix to not accept mails with dns A/AAAA records, there is
                  ignorants everywhere

                  --
                  senders that put my email into body content will deliver it to my own
                  trashcan, so if you like to get reply, dont do it
                • Petar Bogdanovic
                  ... Mopher is a milter (or mail filter) and the original mail never described it as being anything else than a milter (e.g. an MTA). ... It was a rhetorical
                  Message 8 of 8 , Jun 14, 2013
                  View Source
                  • 0 Attachment
                    On Fri, Jun 14, 2013 at 12:48:51PM +0200, Bastian Blank wrote:
                    > On Fri, Jun 14, 2013 at 12:37:11PM +0200, Petar Bogdanovic wrote:
                    > > It's a milter that some people on this list might find useful.
                    >
                    > So it only supports what the milter server can do.

                    Mopher is a milter (or mail filter) and the original mail never
                    described it as being anything else than a milter (e.g. an MTA).


                    > > So kernel space then?
                    >
                    > A milter can't do that anyway,

                    It was a rhetorical question since I wasn't sure what you were saying.


                    > > It helps with domain-based greylisting. There are no simple rules when
                    > > figuring out the registered part of a fqdn.
                    >
                    > So you do greylisting based on DNS reverse lookups?

                    In that particular case, mopher doesn't need to do any lookups but uses
                    whatever libmilter provides as sender hostname (which, at least in case
                    of Postfix, should be a PTR RR that matches with the according A RR).
                    It then extracts the registered part with the help of the PSL rule-set.

                    The default greylisting is therefore based on the following triplet:
                    sender domain (the registered part), envelope from and recipient.

                    Petar Bogdanovic
                  Your message has been successfully submitted and would be delivered to recipients shortly.