Loading ...
Sorry, an error occurred while loading the content.

Re: Could you help me with Postfix + MimeDefang?

Expand Messages
  • Mark Alan
    On Mon, 04 Feb 2013 09:40:41 -0600, Noel Jones ... First, thank you Noel for sharing your experience and spending your time at trying
    Message 1 of 10 , Feb 4, 2013
    • 0 Attachment
      On Mon, 04 Feb 2013 09:40:41 -0600, Noel Jones <njones@...>
      wrote:
      > On 2/4/2013 4:14 AM, Mark Alan wrote:

      > > I would like to use MimeDefang to sanitize the emails that arrive
      > > at one of our 3 mailing lists, i.e., to convert html->text, remove
      > > unsafe attachments, and remove+webserve file attachments
      > > larger than 500KB.
      > >
      > > There are few tutorials on this subject and most, like Mickey Hill's
      > > http://www.mickeyhill.com/mimedefang-howto , ask for the
      > > installation of sendmail and present a config tightly coupled with
      > > sendmail internals (the real sendmail, not postfix's sendmail).
      > >
      > > Could you please provide (or point to) a couple of working examples
      > > on how to setup mimedefang with postfix (would it be better done as
      > > a transport+filter, or as milter?) and, if possible, throw some
      > > light on the advantages/disadvantages of each alternative?
      > >
      > > Thank you,
      > >
      > > Mark
      >
      >
      > mimedefang works as a milter, so that's how you must interface it
      > with postfix.
      > .../...
      > the config details you will mostly be interested in:
      > http://www.postfix.org/MILTER_README.html#config

      First, thank you Noel for sharing your experience and spending your time
      at trying to help.

      Regarding mimedefang, and its ability to work as a milter, and the
      general setup of a milter under postfix, well... I have been there and
      done that (in due time, I even described in this list, the config that
      we use to run opendkim as a milter, communicating by a unix socket with
      a chrooted postfix).

      But the question here was entirely different: "... to use MimeDefang
      to sanitize the emails that arrive at ONE of our 3 mailing lists"

      The problem was not to apply mimedefang to all incoming mail (like a
      milter base config usually does).
      The problem is how to do it in order to to process a SINGLE target
      email address (the address of a given mailing list), without consuming
      unnecessary machine resources, i.e., without "miltering" all the email
      that arrives at the postfix server.
      That was why I also attached the main.cf/master.cf filter+transport
      config that we use to pipe the emails addressed to mailing lists into
      the mailing list management software.

      Thank you,

      Mark
    • Noel Jones
      ... Sorry, missed that part. A milter applies to all mail. If you want the milter to only process some mail, hopefully there are controls within the milter
      Message 2 of 10 , Feb 4, 2013
      • 0 Attachment
        On 2/4/2013 2:26 PM, Mark Alan wrote:

        >
        > But the question here was entirely different: "... to use MimeDefang
        > to sanitize the emails that arrive at ONE of our 3 mailing lists"
        >
        > The problem was not to apply mimedefang to all incoming mail (like a
        > milter base config usually does).
        > The problem is how to do it in order to to process a SINGLE target
        > email address (the address of a given mailing list), without consuming
        > unnecessary machine resources, i.e., without "miltering" all the email
        > that arrives at the postfix server.

        Sorry, missed that part.

        A milter applies to all mail. If you want the milter to only
        process some mail, hopefully there are controls within the milter
        application for that.




        -- Noel Jones
      • Bill Cole
        ... As Noel said, Postfix offers no way to do that. You could do it inside MD, but it won t be very resource-sparing to do so because MD will build its
        Message 3 of 10 , Feb 4, 2013
        • 0 Attachment
          On 4 Feb 2013, at 15:26, Mark Alan wrote:

          > The problem is how to do it in order to to process a SINGLE target
          > email address (the address of a given mailing list), without consuming
          > unnecessary machine resources, i.e., without "miltering" all the email
          > that arrives at the postfix server.

          As Noel said, Postfix offers no way to do that. You could do it inside
          MD, but it won't be very resource-sparing to do so because MD will build
          its per-message working directory for you before your filter can tell it
          to pass the message along unmolested. An alternative to hooking the MD
          milter into your main smtpd would be to define a transport in master.cf
          running smtpd with MD as a milter, and use postfix's transport map to
          route just the one address there. This would also allow you to avoid the
          ugly problem of envelope recipient splitting inside MD. You can probably
          get a more complete answer on the MD mailing list. Also note that
          configuring MD means writing a collection of Perl functions with
          predefined interfaces to implement the message filtering. If you are not
          comfortable writing Perl, MD may not be the right tool for you.
        • Noel Jones
          ... Good suggestion, Bill. If mimedefang is intended to only mangle messages, a per-user transport into a different postfix instance would work fine. If
          Message 4 of 10 , Feb 4, 2013
          • 0 Attachment
            On 2/4/2013 9:13 PM, Bill Cole wrote:
            > On 4 Feb 2013, at 15:26, Mark Alan wrote:
            >
            >> The problem is how to do it in order to to process a SINGLE target
            >> email address (the address of a given mailing list), without
            >> consuming
            >> unnecessary machine resources, i.e., without "miltering" all the
            >> email
            >> that arrives at the postfix server.
            >
            > As Noel said, Postfix offers no way to do that. You could do it
            > inside MD, but it won't be very resource-sparing to do so because MD
            > will build its per-message working directory for you before your
            > filter can tell it to pass the message along unmolested. An
            > alternative to hooking the MD milter into your main smtpd would be
            > to define a transport in master.cf running smtpd with MD as a
            > milter, and use postfix's transport map to route just the one
            > address there. This would also allow you to avoid the ugly problem
            > of envelope recipient splitting inside MD. You can probably get a
            > more complete answer on the MD mailing list. Also note that
            > configuring MD means writing a collection of Perl functions with
            > predefined interfaces to implement the message filtering. If you are
            > not comfortable writing Perl, MD may not be the right tool for you.

            Good suggestion, Bill.

            If mimedefang is intended to only "mangle" messages, a per-user
            transport into a different postfix instance would work fine.

            If mimedefang might sometimes reject a message, a transport would
            not be acceptable as it would cause backscatter by bouncing the
            message to a possibly forged sender address.



            -- Noel Jones
          • Mark Alan
            On Mon, 04 Feb 2013 22:13:14 -0500, Bill Cole ... Well, that was also my gut feeling, that was why I posted here to try to find some (solid) evidence. So,
            Message 5 of 10 , Feb 5, 2013
            • 0 Attachment
              On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
              <postfixlists-070913@...> wrote:

              > alternative to hooking the MD milter into your main smtpd would be to
              > define a transport in master.cf running smtpd with MD as a milter,
              > and use postfix's transport map to route just the one address there.
              > This would also allow you to avoid the ugly problem of envelope
              > recipient splitting inside MD.

              Well, that was also my gut feeling, that was why I posted here to try
              to find some (solid) evidence.

              So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
              SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock', would the
              following do the job?

              postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
              # /etc/postfix/virtual-alias-maps
              mailing_list_1@... mailing_list_1@...
              ...
              postconf -e 'transport_maps = /etc/postfix/virtual-transport'
              # /etc/postfix/virtual-transport
              mailing_list_1@... filteredmlmmj:mailing_list_1
              ...
              # /etc/postfix/master.cf
              # transport for the mlmmj mailing lists
              mlmmj unix - n n - - pipe
              flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
              -L /var/spool/mlmmj/$nexthop
              # filtered transport for the mlmmj mailing list manager
              filteredmlmmj unix - - - - - mlmmj
              -o smtpd_milters = unix:mimedefang/mimedefang.sock

              Please note, in this last statement, 'unix', 'mlmmj' and '-o
              smtpd_milters' nad the 5 dashes.

              > You can probably get a more complete answer on the MD mailing list.

              Not at all. The stated problem is an old problem. I have researched
              extensively a lot of discussions about this subject both in MD list and
              in the postfix list (and a lot of useless 'recipes' too). People tend
              to see this issue as some sort of magically solved hit or miss issue.
              And the people that develop MD seems to be more in the business of
              selling "canned" solutions (pun intended), than into producing good
              and clear documentation.

              > Also note that configuring MD means
              > writing a collection of Perl functions with predefined interfaces to
              > implement the message filtering. If you are not comfortable writing
              > Perl,

              No problem with the needed Perl functions.

              > MD may not be the right tool for you.

              MD is certainly resources hungry. But I do not know any other app
              that meets the specs:
              convert html->text,
              remove unsafe attachments (offenders with known ext's),
              remove+webserve file attachments larger than 500KB

              Right now we are piping email into "altermime --input=- --removeall",
              but altermime is orphaned/abandonware and it does not do that file
              attachment remove+webserve job.

              Thank you,

              Mark.
            • Bill Cole
              ... Bad idea. Don t put non-postfix stuff in /var/spool/postfix/. The default SPOOLDIR= /var/spool/MIMEDefang and
              Message 6 of 10 , Feb 5, 2013
              • 0 Attachment
                On 5 Feb 2013, at 4:46, Mark Alan wrote:

                > On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                > <postfixlists-070913@...> wrote:
                >
                >> alternative to hooking the MD milter into your main smtpd would be to
                >> define a transport in master.cf running smtpd with MD as a milter,
                >> and use postfix's transport map to route just the one address there.
                >> This would also allow you to avoid the ugly problem of envelope
                >> recipient splitting inside MD.
                >
                > Well, that was also my gut feeling, that was why I posted here to try
                > to find some (solid) evidence.
                >
                > So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                > SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',

                Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
                default
                SPOOLDIR='/var/spool/MIMEDefang' and
                SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.

                > would the
                > following do the job?
                >
                > postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                > # /etc/postfix/virtual-alias-maps
                > mailing_list_1@... mailing_list_1@...
                > ...
                > postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                > # /etc/postfix/virtual-transport
                > mailing_list_1@... filteredmlmmj:mailing_list_1
                > ...
                > # /etc/postfix/master.cf
                > # transport for the mlmmj mailing lists
                > mlmmj unix - n n - - pipe
                > flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                > -L /var/spool/mlmmj/$nexthop
                > # filtered transport for the mlmmj mailing list manager
                > filteredmlmmj unix - - - - - mlmmj
                > -o smtpd_milters = unix:mimedefang/mimedefang.sock
                >
                > Please note, in this last statement, 'unix', 'mlmmj' and '-o
                > smtpd_milters' nad the 5 dashes.

                Substantially wrong. You would need to define a new transport in
                master.cf running *smtpd* in the manner used for a submission daemon or
                the transport used for amavisd output. e.g. something like this:

                localhost:10025 inet n - n - - smtpd
                -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
                -o transport_maps=
                (maybe other -o lines to override main.cf)

                I don't use MD or Postfix that way myself, so I'm a bit leery of cooking
                up anything that looks like a complete example. The concept is that you
                are routing some mail through another postfix SMTP daemon instance with
                its own independent configuration.

                >> You can probably get a more complete answer on the MD mailing list.
                >
                > Not at all. The stated problem is an old problem. I have researched
                > extensively a lot of discussions about this subject both in MD list
                > and
                > in the postfix list (and a lot of useless 'recipes' too). People tend
                > to see this issue as some sort of magically solved hit or miss issue.

                Or irrelevant to them. What you want to do isn't something most people
                using MD or Postfix want to do, and most people using one or the other
                are not using them together. It seems to me that the best way to do this
                would be to let MD see everything rather than to try to route around it,
                but in any case most of your work will be in writing the filter
                routines, not doing the plumbing. The experts in that aspect will be on
                the MD list more than they will be here.

                > And the people that develop MD seems to be more in the business of
                > selling "canned" solutions (pun intended), than into producing good
                > and clear documentation.

                There used to be a wiki, but it got overrun with spam. The docs in the
                distribution are pretty solid and the code is not hard to follow. The
                LISA '04 presentation on the website is actually quite good at providing
                a high-level view. If you're looking for 'recipes' that will work for
                your particular need you aren't going to find them because that's just
                not the sort of tool MD is. It is written for Perl coders who also run
                mail systems, not mail admins who can read Perl. Manipulating mail
                content has become increasingly unpopular over the past decade for
                mostly good reasons and it's relatively rare to use MD solely for that
                purpose or to only deploy it for some addresses. It's not remarkable
                that no one has a setup for addressing your issue that they feel
                comfortable sharing. I've been using MD for a decade with both Sendmail
                and Postfix and only have a general idea of how to approach it.


                >> Also note that configuring MD means
                >> writing a collection of Perl functions with predefined interfaces to
                >> implement the message filtering. If you are not comfortable writing
                >> Perl,
                >
                > No problem with the needed Perl functions.
                >
                >> MD may not be the right tool for you.
                >
                > MD is certainly resources hungry. But I do not know any other app
                > that meets the specs:
                > convert html->text,
                > remove unsafe attachments (offenders with known ext's),
                > remove+webserve file attachments larger than 500KB
                >
                > Right now we are piping email into "altermime --input=- --removeall",
                > but altermime is orphaned/abandonware and it does not do that file
                > attachment remove+webserve job.

                You might want to consider just writing your own pipe-suitable delivery
                filter using the MIME-Tools suite rather than loading up all the
                superstructure of MD. It seems like you would be using a small subset of
                MD itself and coding most of what you actually want done in your filter
                and filter_multipart functions, so why bother with MD?
              • James Griffin
                ... I believe demime would/could achieve what you re looking for. It is used on a few mailing lists i m subscribed to and it s quite easy to set up. Have you
                Message 7 of 10 , Feb 5, 2013
                • 0 Attachment
                  --> Noel Jones <njones@...> [2013-02-04 14:56:23 -0600]:

                  > On 2/4/2013 2:26 PM, Mark Alan wrote:
                  >
                  > >
                  > > But the question here was entirely different: "... to use MimeDefang
                  > > to sanitize the emails that arrive at ONE of our 3 mailing lists"
                  > >
                  > > The problem was not to apply mimedefang to all incoming mail (like a
                  > > milter base config usually does).
                  > > The problem is how to do it in order to to process a SINGLE target
                  > > email address (the address of a given mailing list), without consuming
                  > > unnecessary machine resources, i.e., without "miltering" all the email
                  > > that arrives at the postfix server.
                  >
                  > Sorry, missed that part.
                  >
                  > A milter applies to all mail. If you want the milter to only
                  > process some mail, hopefully there are controls within the milter
                  > application for that.

                  I believe demime would/could achieve what you're looking for. It
                  is used on a few mailing lists i'm subscribed to and it's quite
                  easy to set up. Have you considered that or other alternatives?


                  --
                  Primary Key: 4096R/1D31DC38 2011-12-03
                  Key Fingerprint: A4B9 E875 A18C 6E11 F46D B788 BEE6 1251 1D31 DC38
                • Noel Jones
                  ... You re on the right track here. The transport table cannot be overridden with a -o option, so this would need to go through a separate postfix instance,
                  Message 8 of 10 , Feb 5, 2013
                  • 0 Attachment
                    On 2/5/2013 10:47 AM, Bill Cole wrote:
                    > On 5 Feb 2013, at 4:46, Mark Alan wrote:
                    >
                    >> On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                    >> <postfixlists-070913@...> wrote:
                    >>
                    >>> alternative to hooking the MD milter into your main smtpd would
                    >>> be to
                    >>> define a transport in master.cf running smtpd with MD as a milter,
                    >>> and use postfix's transport map to route just the one address there.
                    >>> This would also allow you to avoid the ugly problem of envelope
                    >>> recipient splitting inside MD.
                    >>
                    >> Well, that was also my gut feeling, that was why I posted here to try
                    >> to find some (solid) evidence.
                    >>
                    >> So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                    >> SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',
                    >
                    > Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
                    > default
                    > SPOOLDIR='/var/spool/MIMEDefang' and
                    > SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.
                    >
                    >> would the
                    >> following do the job?
                    >>
                    >> postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                    >> # /etc/postfix/virtual-alias-maps
                    >> mailing_list_1@... mailing_list_1@...
                    >> ...
                    >> postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                    >> # /etc/postfix/virtual-transport
                    >> mailing_list_1@... filteredmlmmj:mailing_list_1
                    >> ...
                    >> # /etc/postfix/master.cf
                    >> # transport for the mlmmj mailing lists
                    >> mlmmj unix - n n - - pipe
                    >> flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                    >> -L /var/spool/mlmmj/$nexthop
                    >> # filtered transport for the mlmmj mailing list manager
                    >> filteredmlmmj unix - - - - - mlmmj
                    >> -o smtpd_milters = unix:mimedefang/mimedefang.sock
                    >>
                    >> Please note, in this last statement, 'unix', 'mlmmj' and '-o
                    >> smtpd_milters' nad the 5 dashes.
                    >
                    > Substantially wrong. You would need to define a new transport in
                    > master.cf running *smtpd* in the manner used for a submission daemon
                    > or the transport used for amavisd output. e.g. something like this:
                    >
                    > localhost:10025 inet n - n - - smtpd
                    > -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
                    > -o transport_maps=
                    > (maybe other -o lines to override main.cf)


                    You're on the right track here.

                    The transport table cannot be overridden with a -o option, so this
                    would need to go through a separate postfix instance, not just a
                    different smtpd listener.
                    http://www.postfix.org/MULTI_INSTANCE_README.html


                    On 2/5/2013 11:23 AM, James Griffin wrote:
                    > I believe demime would/could achieve what you're looking for. It
                    > is used on a few mailing lists i'm subscribed to and it's quite
                    > easy to set up. Have you considered that or other alternatives?

                    That sounds like a much easier solution.





                    -- Noel Jones
                  Your message has been successfully submitted and would be delivered to recipients shortly.