Loading ...
Sorry, an error occurred while loading the content.

Could you help me with Postfix + MimeDefang?

Expand Messages
  • Mark Alan
    Hello list, I would like to use MimeDefang to sanitize the emails that arrive at one of our 3 mailing lists, i.e., to convert html- text, remove unsafe
    Message 1 of 10 , Feb 4, 2013
    • 0 Attachment
      Hello list,

      I would like to use MimeDefang to sanitize the emails that arrive
      at one of our 3 mailing lists, i.e., to convert html->text, remove
      unsafe attachments, and remove+webserve file attachments
      larger than 500KB.

      There are few tutorials on this subject and most, like Mickey Hill's
      http://www.mickeyhill.com/mimedefang-howto , ask for the installation
      of sendmail and present a config tightly coupled with sendmail
      internals (the real sendmail, not postfix's sendmail).

      Could you please provide (or point to) a couple of working examples on
      how to setup mimedefang with postfix (would it be better done as a
      transport+filter, or as milter?) and, if possible, throw some light
      on the advantages/disadvantages of each alternative?

      Thank you,

      Mark
      PS: My current setup uses postfix 2.9.3 + mlmmj 1.2.18 + ubuntu
      12.04, to manage 2 low volume mailing lists (< 300 subscribers and
      less than 50 emails/month), as follows:

      MAILING_LIST_1 <mailing_list_1@...>
      MAILING_LIST_2 <mailing_list_2@...>

      postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
      # /etc/postfix/virtual-alias-maps
      mailing_list_1@... mailing_list_1@...
      mailing_list_2@... mailing_list_2@...

      postconf -e 'transport_maps = /etc/postfix/virtual-transport'
      # /etc/postfix/virtual-transport
      mailing_list_1@... mlmmj:mailing_list_1
      mailing_list_2@... mlmmj:mailing_list_2

      # /etc/postfix/master.cf
      # transport for the mlmmj mailing list manager
      mlmmj unix - n n - - pipe
      flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
      -L /var/spool/mlmmj/$nexthop
    • Noel Jones
      ... mimedefang works as a milter, so that s how you must interface it with postfix. All mimedefang configuration is done within mimedefang, not within the host
      Message 2 of 10 , Feb 4, 2013
      • 0 Attachment
        On 2/4/2013 4:14 AM, Mark Alan wrote:
        > Hello list,
        >
        > I would like to use MimeDefang to sanitize the emails that arrive
        > at one of our 3 mailing lists, i.e., to convert html->text, remove
        > unsafe attachments, and remove+webserve file attachments
        > larger than 500KB.
        >
        > There are few tutorials on this subject and most, like Mickey Hill's
        > http://www.mickeyhill.com/mimedefang-howto , ask for the installation
        > of sendmail and present a config tightly coupled with sendmail
        > internals (the real sendmail, not postfix's sendmail).
        >
        > Could you please provide (or point to) a couple of working examples on
        > how to setup mimedefang with postfix (would it be better done as a
        > transport+filter, or as milter?) and, if possible, throw some light
        > on the advantages/disadvantages of each alternative?
        >
        > Thank you,
        >
        > Mark


        mimedefang works as a milter, so that's how you must interface it
        with postfix.

        All mimedefang configuration is done within mimedefang, not within
        the host mailer. So other than the few milter interface details, it
        doesn't really matter if you're using sendmail(TM) or postfix.

        Configuring the postfix milter interface is very easy. Although
        there are a bunch of postfix milter options, most of them have
        sensible defaults and will not need to be changed.

        With a typical milter listening on a local TCP port, all you need is
        a postfix main.cf line something like:
        # main.cf
        smtpd_milters = inet:127.0.0.1:portnum
        where portnum matches where the milter application is configured to
        listen. Using a local TCP port is the easiest and simplest way to
        configure a milter.

        Configuring postfix for a milter listening on a unix socket is similar:
        # main.cf
        smtpd_milters = unix:/path/to/socket
        Figuring out the proper path and permissions can be a little tricky,
        especially if postfix is running with the chroot flag set. Using a
        local socket can be a little more secure, but has no performance
        benefits.

        Full docs are here:
        http://www.postfix.org/MILTER_README.html
        the config details you will mostly be interested in:
        http://www.postfix.org/MILTER_README.html#config



        -- Noel Jones
      • Mark Alan
        On Mon, 04 Feb 2013 09:40:41 -0600, Noel Jones ... First, thank you Noel for sharing your experience and spending your time at trying
        Message 3 of 10 , Feb 4, 2013
        • 0 Attachment
          On Mon, 04 Feb 2013 09:40:41 -0600, Noel Jones <njones@...>
          wrote:
          > On 2/4/2013 4:14 AM, Mark Alan wrote:

          > > I would like to use MimeDefang to sanitize the emails that arrive
          > > at one of our 3 mailing lists, i.e., to convert html->text, remove
          > > unsafe attachments, and remove+webserve file attachments
          > > larger than 500KB.
          > >
          > > There are few tutorials on this subject and most, like Mickey Hill's
          > > http://www.mickeyhill.com/mimedefang-howto , ask for the
          > > installation of sendmail and present a config tightly coupled with
          > > sendmail internals (the real sendmail, not postfix's sendmail).
          > >
          > > Could you please provide (or point to) a couple of working examples
          > > on how to setup mimedefang with postfix (would it be better done as
          > > a transport+filter, or as milter?) and, if possible, throw some
          > > light on the advantages/disadvantages of each alternative?
          > >
          > > Thank you,
          > >
          > > Mark
          >
          >
          > mimedefang works as a milter, so that's how you must interface it
          > with postfix.
          > .../...
          > the config details you will mostly be interested in:
          > http://www.postfix.org/MILTER_README.html#config

          First, thank you Noel for sharing your experience and spending your time
          at trying to help.

          Regarding mimedefang, and its ability to work as a milter, and the
          general setup of a milter under postfix, well... I have been there and
          done that (in due time, I even described in this list, the config that
          we use to run opendkim as a milter, communicating by a unix socket with
          a chrooted postfix).

          But the question here was entirely different: "... to use MimeDefang
          to sanitize the emails that arrive at ONE of our 3 mailing lists"

          The problem was not to apply mimedefang to all incoming mail (like a
          milter base config usually does).
          The problem is how to do it in order to to process a SINGLE target
          email address (the address of a given mailing list), without consuming
          unnecessary machine resources, i.e., without "miltering" all the email
          that arrives at the postfix server.
          That was why I also attached the main.cf/master.cf filter+transport
          config that we use to pipe the emails addressed to mailing lists into
          the mailing list management software.

          Thank you,

          Mark
        • Noel Jones
          ... Sorry, missed that part. A milter applies to all mail. If you want the milter to only process some mail, hopefully there are controls within the milter
          Message 4 of 10 , Feb 4, 2013
          • 0 Attachment
            On 2/4/2013 2:26 PM, Mark Alan wrote:

            >
            > But the question here was entirely different: "... to use MimeDefang
            > to sanitize the emails that arrive at ONE of our 3 mailing lists"
            >
            > The problem was not to apply mimedefang to all incoming mail (like a
            > milter base config usually does).
            > The problem is how to do it in order to to process a SINGLE target
            > email address (the address of a given mailing list), without consuming
            > unnecessary machine resources, i.e., without "miltering" all the email
            > that arrives at the postfix server.

            Sorry, missed that part.

            A milter applies to all mail. If you want the milter to only
            process some mail, hopefully there are controls within the milter
            application for that.




            -- Noel Jones
          • Bill Cole
            ... As Noel said, Postfix offers no way to do that. You could do it inside MD, but it won t be very resource-sparing to do so because MD will build its
            Message 5 of 10 , Feb 4, 2013
            • 0 Attachment
              On 4 Feb 2013, at 15:26, Mark Alan wrote:

              > The problem is how to do it in order to to process a SINGLE target
              > email address (the address of a given mailing list), without consuming
              > unnecessary machine resources, i.e., without "miltering" all the email
              > that arrives at the postfix server.

              As Noel said, Postfix offers no way to do that. You could do it inside
              MD, but it won't be very resource-sparing to do so because MD will build
              its per-message working directory for you before your filter can tell it
              to pass the message along unmolested. An alternative to hooking the MD
              milter into your main smtpd would be to define a transport in master.cf
              running smtpd with MD as a milter, and use postfix's transport map to
              route just the one address there. This would also allow you to avoid the
              ugly problem of envelope recipient splitting inside MD. You can probably
              get a more complete answer on the MD mailing list. Also note that
              configuring MD means writing a collection of Perl functions with
              predefined interfaces to implement the message filtering. If you are not
              comfortable writing Perl, MD may not be the right tool for you.
            • Noel Jones
              ... Good suggestion, Bill. If mimedefang is intended to only mangle messages, a per-user transport into a different postfix instance would work fine. If
              Message 6 of 10 , Feb 4, 2013
              • 0 Attachment
                On 2/4/2013 9:13 PM, Bill Cole wrote:
                > On 4 Feb 2013, at 15:26, Mark Alan wrote:
                >
                >> The problem is how to do it in order to to process a SINGLE target
                >> email address (the address of a given mailing list), without
                >> consuming
                >> unnecessary machine resources, i.e., without "miltering" all the
                >> email
                >> that arrives at the postfix server.
                >
                > As Noel said, Postfix offers no way to do that. You could do it
                > inside MD, but it won't be very resource-sparing to do so because MD
                > will build its per-message working directory for you before your
                > filter can tell it to pass the message along unmolested. An
                > alternative to hooking the MD milter into your main smtpd would be
                > to define a transport in master.cf running smtpd with MD as a
                > milter, and use postfix's transport map to route just the one
                > address there. This would also allow you to avoid the ugly problem
                > of envelope recipient splitting inside MD. You can probably get a
                > more complete answer on the MD mailing list. Also note that
                > configuring MD means writing a collection of Perl functions with
                > predefined interfaces to implement the message filtering. If you are
                > not comfortable writing Perl, MD may not be the right tool for you.

                Good suggestion, Bill.

                If mimedefang is intended to only "mangle" messages, a per-user
                transport into a different postfix instance would work fine.

                If mimedefang might sometimes reject a message, a transport would
                not be acceptable as it would cause backscatter by bouncing the
                message to a possibly forged sender address.



                -- Noel Jones
              • Mark Alan
                On Mon, 04 Feb 2013 22:13:14 -0500, Bill Cole ... Well, that was also my gut feeling, that was why I posted here to try to find some (solid) evidence. So,
                Message 7 of 10 , Feb 5, 2013
                • 0 Attachment
                  On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                  <postfixlists-070913@...> wrote:

                  > alternative to hooking the MD milter into your main smtpd would be to
                  > define a transport in master.cf running smtpd with MD as a milter,
                  > and use postfix's transport map to route just the one address there.
                  > This would also allow you to avoid the ugly problem of envelope
                  > recipient splitting inside MD.

                  Well, that was also my gut feeling, that was why I posted here to try
                  to find some (solid) evidence.

                  So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                  SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock', would the
                  following do the job?

                  postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                  # /etc/postfix/virtual-alias-maps
                  mailing_list_1@... mailing_list_1@...
                  ...
                  postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                  # /etc/postfix/virtual-transport
                  mailing_list_1@... filteredmlmmj:mailing_list_1
                  ...
                  # /etc/postfix/master.cf
                  # transport for the mlmmj mailing lists
                  mlmmj unix - n n - - pipe
                  flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                  -L /var/spool/mlmmj/$nexthop
                  # filtered transport for the mlmmj mailing list manager
                  filteredmlmmj unix - - - - - mlmmj
                  -o smtpd_milters = unix:mimedefang/mimedefang.sock

                  Please note, in this last statement, 'unix', 'mlmmj' and '-o
                  smtpd_milters' nad the 5 dashes.

                  > You can probably get a more complete answer on the MD mailing list.

                  Not at all. The stated problem is an old problem. I have researched
                  extensively a lot of discussions about this subject both in MD list and
                  in the postfix list (and a lot of useless 'recipes' too). People tend
                  to see this issue as some sort of magically solved hit or miss issue.
                  And the people that develop MD seems to be more in the business of
                  selling "canned" solutions (pun intended), than into producing good
                  and clear documentation.

                  > Also note that configuring MD means
                  > writing a collection of Perl functions with predefined interfaces to
                  > implement the message filtering. If you are not comfortable writing
                  > Perl,

                  No problem with the needed Perl functions.

                  > MD may not be the right tool for you.

                  MD is certainly resources hungry. But I do not know any other app
                  that meets the specs:
                  convert html->text,
                  remove unsafe attachments (offenders with known ext's),
                  remove+webserve file attachments larger than 500KB

                  Right now we are piping email into "altermime --input=- --removeall",
                  but altermime is orphaned/abandonware and it does not do that file
                  attachment remove+webserve job.

                  Thank you,

                  Mark.
                • Bill Cole
                  ... Bad idea. Don t put non-postfix stuff in /var/spool/postfix/. The default SPOOLDIR= /var/spool/MIMEDefang and
                  Message 8 of 10 , Feb 5, 2013
                  • 0 Attachment
                    On 5 Feb 2013, at 4:46, Mark Alan wrote:

                    > On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                    > <postfixlists-070913@...> wrote:
                    >
                    >> alternative to hooking the MD milter into your main smtpd would be to
                    >> define a transport in master.cf running smtpd with MD as a milter,
                    >> and use postfix's transport map to route just the one address there.
                    >> This would also allow you to avoid the ugly problem of envelope
                    >> recipient splitting inside MD.
                    >
                    > Well, that was also my gut feeling, that was why I posted here to try
                    > to find some (solid) evidence.
                    >
                    > So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                    > SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',

                    Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
                    default
                    SPOOLDIR='/var/spool/MIMEDefang' and
                    SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.

                    > would the
                    > following do the job?
                    >
                    > postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                    > # /etc/postfix/virtual-alias-maps
                    > mailing_list_1@... mailing_list_1@...
                    > ...
                    > postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                    > # /etc/postfix/virtual-transport
                    > mailing_list_1@... filteredmlmmj:mailing_list_1
                    > ...
                    > # /etc/postfix/master.cf
                    > # transport for the mlmmj mailing lists
                    > mlmmj unix - n n - - pipe
                    > flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                    > -L /var/spool/mlmmj/$nexthop
                    > # filtered transport for the mlmmj mailing list manager
                    > filteredmlmmj unix - - - - - mlmmj
                    > -o smtpd_milters = unix:mimedefang/mimedefang.sock
                    >
                    > Please note, in this last statement, 'unix', 'mlmmj' and '-o
                    > smtpd_milters' nad the 5 dashes.

                    Substantially wrong. You would need to define a new transport in
                    master.cf running *smtpd* in the manner used for a submission daemon or
                    the transport used for amavisd output. e.g. something like this:

                    localhost:10025 inet n - n - - smtpd
                    -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
                    -o transport_maps=
                    (maybe other -o lines to override main.cf)

                    I don't use MD or Postfix that way myself, so I'm a bit leery of cooking
                    up anything that looks like a complete example. The concept is that you
                    are routing some mail through another postfix SMTP daemon instance with
                    its own independent configuration.

                    >> You can probably get a more complete answer on the MD mailing list.
                    >
                    > Not at all. The stated problem is an old problem. I have researched
                    > extensively a lot of discussions about this subject both in MD list
                    > and
                    > in the postfix list (and a lot of useless 'recipes' too). People tend
                    > to see this issue as some sort of magically solved hit or miss issue.

                    Or irrelevant to them. What you want to do isn't something most people
                    using MD or Postfix want to do, and most people using one or the other
                    are not using them together. It seems to me that the best way to do this
                    would be to let MD see everything rather than to try to route around it,
                    but in any case most of your work will be in writing the filter
                    routines, not doing the plumbing. The experts in that aspect will be on
                    the MD list more than they will be here.

                    > And the people that develop MD seems to be more in the business of
                    > selling "canned" solutions (pun intended), than into producing good
                    > and clear documentation.

                    There used to be a wiki, but it got overrun with spam. The docs in the
                    distribution are pretty solid and the code is not hard to follow. The
                    LISA '04 presentation on the website is actually quite good at providing
                    a high-level view. If you're looking for 'recipes' that will work for
                    your particular need you aren't going to find them because that's just
                    not the sort of tool MD is. It is written for Perl coders who also run
                    mail systems, not mail admins who can read Perl. Manipulating mail
                    content has become increasingly unpopular over the past decade for
                    mostly good reasons and it's relatively rare to use MD solely for that
                    purpose or to only deploy it for some addresses. It's not remarkable
                    that no one has a setup for addressing your issue that they feel
                    comfortable sharing. I've been using MD for a decade with both Sendmail
                    and Postfix and only have a general idea of how to approach it.


                    >> Also note that configuring MD means
                    >> writing a collection of Perl functions with predefined interfaces to
                    >> implement the message filtering. If you are not comfortable writing
                    >> Perl,
                    >
                    > No problem with the needed Perl functions.
                    >
                    >> MD may not be the right tool for you.
                    >
                    > MD is certainly resources hungry. But I do not know any other app
                    > that meets the specs:
                    > convert html->text,
                    > remove unsafe attachments (offenders with known ext's),
                    > remove+webserve file attachments larger than 500KB
                    >
                    > Right now we are piping email into "altermime --input=- --removeall",
                    > but altermime is orphaned/abandonware and it does not do that file
                    > attachment remove+webserve job.

                    You might want to consider just writing your own pipe-suitable delivery
                    filter using the MIME-Tools suite rather than loading up all the
                    superstructure of MD. It seems like you would be using a small subset of
                    MD itself and coding most of what you actually want done in your filter
                    and filter_multipart functions, so why bother with MD?
                  • James Griffin
                    ... I believe demime would/could achieve what you re looking for. It is used on a few mailing lists i m subscribed to and it s quite easy to set up. Have you
                    Message 9 of 10 , Feb 5, 2013
                    • 0 Attachment
                      --> Noel Jones <njones@...> [2013-02-04 14:56:23 -0600]:

                      > On 2/4/2013 2:26 PM, Mark Alan wrote:
                      >
                      > >
                      > > But the question here was entirely different: "... to use MimeDefang
                      > > to sanitize the emails that arrive at ONE of our 3 mailing lists"
                      > >
                      > > The problem was not to apply mimedefang to all incoming mail (like a
                      > > milter base config usually does).
                      > > The problem is how to do it in order to to process a SINGLE target
                      > > email address (the address of a given mailing list), without consuming
                      > > unnecessary machine resources, i.e., without "miltering" all the email
                      > > that arrives at the postfix server.
                      >
                      > Sorry, missed that part.
                      >
                      > A milter applies to all mail. If you want the milter to only
                      > process some mail, hopefully there are controls within the milter
                      > application for that.

                      I believe demime would/could achieve what you're looking for. It
                      is used on a few mailing lists i'm subscribed to and it's quite
                      easy to set up. Have you considered that or other alternatives?


                      --
                      Primary Key: 4096R/1D31DC38 2011-12-03
                      Key Fingerprint: A4B9 E875 A18C 6E11 F46D B788 BEE6 1251 1D31 DC38
                    • Noel Jones
                      ... You re on the right track here. The transport table cannot be overridden with a -o option, so this would need to go through a separate postfix instance,
                      Message 10 of 10 , Feb 5, 2013
                      • 0 Attachment
                        On 2/5/2013 10:47 AM, Bill Cole wrote:
                        > On 5 Feb 2013, at 4:46, Mark Alan wrote:
                        >
                        >> On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                        >> <postfixlists-070913@...> wrote:
                        >>
                        >>> alternative to hooking the MD milter into your main smtpd would
                        >>> be to
                        >>> define a transport in master.cf running smtpd with MD as a milter,
                        >>> and use postfix's transport map to route just the one address there.
                        >>> This would also allow you to avoid the ugly problem of envelope
                        >>> recipient splitting inside MD.
                        >>
                        >> Well, that was also my gut feeling, that was why I posted here to try
                        >> to find some (solid) evidence.
                        >>
                        >> So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                        >> SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',
                        >
                        > Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
                        > default
                        > SPOOLDIR='/var/spool/MIMEDefang' and
                        > SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.
                        >
                        >> would the
                        >> following do the job?
                        >>
                        >> postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                        >> # /etc/postfix/virtual-alias-maps
                        >> mailing_list_1@... mailing_list_1@...
                        >> ...
                        >> postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                        >> # /etc/postfix/virtual-transport
                        >> mailing_list_1@... filteredmlmmj:mailing_list_1
                        >> ...
                        >> # /etc/postfix/master.cf
                        >> # transport for the mlmmj mailing lists
                        >> mlmmj unix - n n - - pipe
                        >> flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                        >> -L /var/spool/mlmmj/$nexthop
                        >> # filtered transport for the mlmmj mailing list manager
                        >> filteredmlmmj unix - - - - - mlmmj
                        >> -o smtpd_milters = unix:mimedefang/mimedefang.sock
                        >>
                        >> Please note, in this last statement, 'unix', 'mlmmj' and '-o
                        >> smtpd_milters' nad the 5 dashes.
                        >
                        > Substantially wrong. You would need to define a new transport in
                        > master.cf running *smtpd* in the manner used for a submission daemon
                        > or the transport used for amavisd output. e.g. something like this:
                        >
                        > localhost:10025 inet n - n - - smtpd
                        > -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
                        > -o transport_maps=
                        > (maybe other -o lines to override main.cf)


                        You're on the right track here.

                        The transport table cannot be overridden with a -o option, so this
                        would need to go through a separate postfix instance, not just a
                        different smtpd listener.
                        http://www.postfix.org/MULTI_INSTANCE_README.html


                        On 2/5/2013 11:23 AM, James Griffin wrote:
                        > I believe demime would/could achieve what you're looking for. It
                        > is used on a few mailing lists i'm subscribed to and it's quite
                        > easy to set up. Have you considered that or other alternatives?

                        That sounds like a much easier solution.





                        -- Noel Jones
                      Your message has been successfully submitted and would be delivered to recipients shortly.