Loading ...
Sorry, an error occurred while loading the content.
 

Re: Could you help me with Postfix + MimeDefang?

Expand Messages
  • Noel Jones
    ... Sorry, missed that part. A milter applies to all mail. If you want the milter to only process some mail, hopefully there are controls within the milter
    Message 1 of 10 , Feb 4, 2013
      On 2/4/2013 2:26 PM, Mark Alan wrote:

      >
      > But the question here was entirely different: "... to use MimeDefang
      > to sanitize the emails that arrive at ONE of our 3 mailing lists"
      >
      > The problem was not to apply mimedefang to all incoming mail (like a
      > milter base config usually does).
      > The problem is how to do it in order to to process a SINGLE target
      > email address (the address of a given mailing list), without consuming
      > unnecessary machine resources, i.e., without "miltering" all the email
      > that arrives at the postfix server.

      Sorry, missed that part.

      A milter applies to all mail. If you want the milter to only
      process some mail, hopefully there are controls within the milter
      application for that.




      -- Noel Jones
    • Bill Cole
      ... As Noel said, Postfix offers no way to do that. You could do it inside MD, but it won t be very resource-sparing to do so because MD will build its
      Message 2 of 10 , Feb 4, 2013
        On 4 Feb 2013, at 15:26, Mark Alan wrote:

        > The problem is how to do it in order to to process a SINGLE target
        > email address (the address of a given mailing list), without consuming
        > unnecessary machine resources, i.e., without "miltering" all the email
        > that arrives at the postfix server.

        As Noel said, Postfix offers no way to do that. You could do it inside
        MD, but it won't be very resource-sparing to do so because MD will build
        its per-message working directory for you before your filter can tell it
        to pass the message along unmolested. An alternative to hooking the MD
        milter into your main smtpd would be to define a transport in master.cf
        running smtpd with MD as a milter, and use postfix's transport map to
        route just the one address there. This would also allow you to avoid the
        ugly problem of envelope recipient splitting inside MD. You can probably
        get a more complete answer on the MD mailing list. Also note that
        configuring MD means writing a collection of Perl functions with
        predefined interfaces to implement the message filtering. If you are not
        comfortable writing Perl, MD may not be the right tool for you.
      • Noel Jones
        ... Good suggestion, Bill. If mimedefang is intended to only mangle messages, a per-user transport into a different postfix instance would work fine. If
        Message 3 of 10 , Feb 4, 2013
          On 2/4/2013 9:13 PM, Bill Cole wrote:
          > On 4 Feb 2013, at 15:26, Mark Alan wrote:
          >
          >> The problem is how to do it in order to to process a SINGLE target
          >> email address (the address of a given mailing list), without
          >> consuming
          >> unnecessary machine resources, i.e., without "miltering" all the
          >> email
          >> that arrives at the postfix server.
          >
          > As Noel said, Postfix offers no way to do that. You could do it
          > inside MD, but it won't be very resource-sparing to do so because MD
          > will build its per-message working directory for you before your
          > filter can tell it to pass the message along unmolested. An
          > alternative to hooking the MD milter into your main smtpd would be
          > to define a transport in master.cf running smtpd with MD as a
          > milter, and use postfix's transport map to route just the one
          > address there. This would also allow you to avoid the ugly problem
          > of envelope recipient splitting inside MD. You can probably get a
          > more complete answer on the MD mailing list. Also note that
          > configuring MD means writing a collection of Perl functions with
          > predefined interfaces to implement the message filtering. If you are
          > not comfortable writing Perl, MD may not be the right tool for you.

          Good suggestion, Bill.

          If mimedefang is intended to only "mangle" messages, a per-user
          transport into a different postfix instance would work fine.

          If mimedefang might sometimes reject a message, a transport would
          not be acceptable as it would cause backscatter by bouncing the
          message to a possibly forged sender address.



          -- Noel Jones
        • Mark Alan
          On Mon, 04 Feb 2013 22:13:14 -0500, Bill Cole ... Well, that was also my gut feeling, that was why I posted here to try to find some (solid) evidence. So,
          Message 4 of 10 , Feb 5, 2013
            On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
            <postfixlists-070913@...> wrote:

            > alternative to hooking the MD milter into your main smtpd would be to
            > define a transport in master.cf running smtpd with MD as a milter,
            > and use postfix's transport map to route just the one address there.
            > This would also allow you to avoid the ugly problem of envelope
            > recipient splitting inside MD.

            Well, that was also my gut feeling, that was why I posted here to try
            to find some (solid) evidence.

            So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
            SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock', would the
            following do the job?

            postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
            # /etc/postfix/virtual-alias-maps
            mailing_list_1@... mailing_list_1@...
            ...
            postconf -e 'transport_maps = /etc/postfix/virtual-transport'
            # /etc/postfix/virtual-transport
            mailing_list_1@... filteredmlmmj:mailing_list_1
            ...
            # /etc/postfix/master.cf
            # transport for the mlmmj mailing lists
            mlmmj unix - n n - - pipe
            flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
            -L /var/spool/mlmmj/$nexthop
            # filtered transport for the mlmmj mailing list manager
            filteredmlmmj unix - - - - - mlmmj
            -o smtpd_milters = unix:mimedefang/mimedefang.sock

            Please note, in this last statement, 'unix', 'mlmmj' and '-o
            smtpd_milters' nad the 5 dashes.

            > You can probably get a more complete answer on the MD mailing list.

            Not at all. The stated problem is an old problem. I have researched
            extensively a lot of discussions about this subject both in MD list and
            in the postfix list (and a lot of useless 'recipes' too). People tend
            to see this issue as some sort of magically solved hit or miss issue.
            And the people that develop MD seems to be more in the business of
            selling "canned" solutions (pun intended), than into producing good
            and clear documentation.

            > Also note that configuring MD means
            > writing a collection of Perl functions with predefined interfaces to
            > implement the message filtering. If you are not comfortable writing
            > Perl,

            No problem with the needed Perl functions.

            > MD may not be the right tool for you.

            MD is certainly resources hungry. But I do not know any other app
            that meets the specs:
            convert html->text,
            remove unsafe attachments (offenders with known ext's),
            remove+webserve file attachments larger than 500KB

            Right now we are piping email into "altermime --input=- --removeall",
            but altermime is orphaned/abandonware and it does not do that file
            attachment remove+webserve job.

            Thank you,

            Mark.
          • Bill Cole
            ... Bad idea. Don t put non-postfix stuff in /var/spool/postfix/. The default SPOOLDIR= /var/spool/MIMEDefang and
            Message 5 of 10 , Feb 5, 2013
              On 5 Feb 2013, at 4:46, Mark Alan wrote:

              > On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
              > <postfixlists-070913@...> wrote:
              >
              >> alternative to hooking the MD milter into your main smtpd would be to
              >> define a transport in master.cf running smtpd with MD as a milter,
              >> and use postfix's transport map to route just the one address there.
              >> This would also allow you to avoid the ugly problem of envelope
              >> recipient splitting inside MD.
              >
              > Well, that was also my gut feeling, that was why I posted here to try
              > to find some (solid) evidence.
              >
              > So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
              > SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',

              Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
              default
              SPOOLDIR='/var/spool/MIMEDefang' and
              SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.

              > would the
              > following do the job?
              >
              > postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
              > # /etc/postfix/virtual-alias-maps
              > mailing_list_1@... mailing_list_1@...
              > ...
              > postconf -e 'transport_maps = /etc/postfix/virtual-transport'
              > # /etc/postfix/virtual-transport
              > mailing_list_1@... filteredmlmmj:mailing_list_1
              > ...
              > # /etc/postfix/master.cf
              > # transport for the mlmmj mailing lists
              > mlmmj unix - n n - - pipe
              > flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
              > -L /var/spool/mlmmj/$nexthop
              > # filtered transport for the mlmmj mailing list manager
              > filteredmlmmj unix - - - - - mlmmj
              > -o smtpd_milters = unix:mimedefang/mimedefang.sock
              >
              > Please note, in this last statement, 'unix', 'mlmmj' and '-o
              > smtpd_milters' nad the 5 dashes.

              Substantially wrong. You would need to define a new transport in
              master.cf running *smtpd* in the manner used for a submission daemon or
              the transport used for amavisd output. e.g. something like this:

              localhost:10025 inet n - n - - smtpd
              -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
              -o transport_maps=
              (maybe other -o lines to override main.cf)

              I don't use MD or Postfix that way myself, so I'm a bit leery of cooking
              up anything that looks like a complete example. The concept is that you
              are routing some mail through another postfix SMTP daemon instance with
              its own independent configuration.

              >> You can probably get a more complete answer on the MD mailing list.
              >
              > Not at all. The stated problem is an old problem. I have researched
              > extensively a lot of discussions about this subject both in MD list
              > and
              > in the postfix list (and a lot of useless 'recipes' too). People tend
              > to see this issue as some sort of magically solved hit or miss issue.

              Or irrelevant to them. What you want to do isn't something most people
              using MD or Postfix want to do, and most people using one or the other
              are not using them together. It seems to me that the best way to do this
              would be to let MD see everything rather than to try to route around it,
              but in any case most of your work will be in writing the filter
              routines, not doing the plumbing. The experts in that aspect will be on
              the MD list more than they will be here.

              > And the people that develop MD seems to be more in the business of
              > selling "canned" solutions (pun intended), than into producing good
              > and clear documentation.

              There used to be a wiki, but it got overrun with spam. The docs in the
              distribution are pretty solid and the code is not hard to follow. The
              LISA '04 presentation on the website is actually quite good at providing
              a high-level view. If you're looking for 'recipes' that will work for
              your particular need you aren't going to find them because that's just
              not the sort of tool MD is. It is written for Perl coders who also run
              mail systems, not mail admins who can read Perl. Manipulating mail
              content has become increasingly unpopular over the past decade for
              mostly good reasons and it's relatively rare to use MD solely for that
              purpose or to only deploy it for some addresses. It's not remarkable
              that no one has a setup for addressing your issue that they feel
              comfortable sharing. I've been using MD for a decade with both Sendmail
              and Postfix and only have a general idea of how to approach it.


              >> Also note that configuring MD means
              >> writing a collection of Perl functions with predefined interfaces to
              >> implement the message filtering. If you are not comfortable writing
              >> Perl,
              >
              > No problem with the needed Perl functions.
              >
              >> MD may not be the right tool for you.
              >
              > MD is certainly resources hungry. But I do not know any other app
              > that meets the specs:
              > convert html->text,
              > remove unsafe attachments (offenders with known ext's),
              > remove+webserve file attachments larger than 500KB
              >
              > Right now we are piping email into "altermime --input=- --removeall",
              > but altermime is orphaned/abandonware and it does not do that file
              > attachment remove+webserve job.

              You might want to consider just writing your own pipe-suitable delivery
              filter using the MIME-Tools suite rather than loading up all the
              superstructure of MD. It seems like you would be using a small subset of
              MD itself and coding most of what you actually want done in your filter
              and filter_multipart functions, so why bother with MD?
            • James Griffin
              ... I believe demime would/could achieve what you re looking for. It is used on a few mailing lists i m subscribed to and it s quite easy to set up. Have you
              Message 6 of 10 , Feb 5, 2013
                --> Noel Jones <njones@...> [2013-02-04 14:56:23 -0600]:

                > On 2/4/2013 2:26 PM, Mark Alan wrote:
                >
                > >
                > > But the question here was entirely different: "... to use MimeDefang
                > > to sanitize the emails that arrive at ONE of our 3 mailing lists"
                > >
                > > The problem was not to apply mimedefang to all incoming mail (like a
                > > milter base config usually does).
                > > The problem is how to do it in order to to process a SINGLE target
                > > email address (the address of a given mailing list), without consuming
                > > unnecessary machine resources, i.e., without "miltering" all the email
                > > that arrives at the postfix server.
                >
                > Sorry, missed that part.
                >
                > A milter applies to all mail. If you want the milter to only
                > process some mail, hopefully there are controls within the milter
                > application for that.

                I believe demime would/could achieve what you're looking for. It
                is used on a few mailing lists i'm subscribed to and it's quite
                easy to set up. Have you considered that or other alternatives?


                --
                Primary Key: 4096R/1D31DC38 2011-12-03
                Key Fingerprint: A4B9 E875 A18C 6E11 F46D B788 BEE6 1251 1D31 DC38
              • Noel Jones
                ... You re on the right track here. The transport table cannot be overridden with a -o option, so this would need to go through a separate postfix instance,
                Message 7 of 10 , Feb 5, 2013
                  On 2/5/2013 10:47 AM, Bill Cole wrote:
                  > On 5 Feb 2013, at 4:46, Mark Alan wrote:
                  >
                  >> On Mon, 04 Feb 2013 22:13:14 -0500, "Bill Cole"
                  >> <postfixlists-070913@...> wrote:
                  >>
                  >>> alternative to hooking the MD milter into your main smtpd would
                  >>> be to
                  >>> define a transport in master.cf running smtpd with MD as a milter,
                  >>> and use postfix's transport map to route just the one address there.
                  >>> This would also allow you to avoid the ugly problem of envelope
                  >>> recipient splitting inside MD.
                  >>
                  >> Well, that was also my gut feeling, that was why I posted here to try
                  >> to find some (solid) evidence.
                  >>
                  >> So, assuming MD SPOOLDIR='/var/spool/postfix/mimedefang' and
                  >> SOCKET='/var/spool/postfix/mimedefang/mimedefang.sock',
                  >
                  > Bad idea. Don't put non-postfix stuff in /var/spool/postfix/. The
                  > default
                  > SPOOLDIR='/var/spool/MIMEDefang' and
                  > SOCKET='/var/spool/MIMEDefang/mimedefang.sock' are fine.
                  >
                  >> would the
                  >> following do the job?
                  >>
                  >> postconf -e 'virtual_alias_maps = /etc/postfix/virtual-alias-maps
                  >> # /etc/postfix/virtual-alias-maps
                  >> mailing_list_1@... mailing_list_1@...
                  >> ...
                  >> postconf -e 'transport_maps = /etc/postfix/virtual-transport'
                  >> # /etc/postfix/virtual-transport
                  >> mailing_list_1@... filteredmlmmj:mailing_list_1
                  >> ...
                  >> # /etc/postfix/master.cf
                  >> # transport for the mlmmj mailing lists
                  >> mlmmj unix - n n - - pipe
                  >> flags=ORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F
                  >> -L /var/spool/mlmmj/$nexthop
                  >> # filtered transport for the mlmmj mailing list manager
                  >> filteredmlmmj unix - - - - - mlmmj
                  >> -o smtpd_milters = unix:mimedefang/mimedefang.sock
                  >>
                  >> Please note, in this last statement, 'unix', 'mlmmj' and '-o
                  >> smtpd_milters' nad the 5 dashes.
                  >
                  > Substantially wrong. You would need to define a new transport in
                  > master.cf running *smtpd* in the manner used for a submission daemon
                  > or the transport used for amavisd output. e.g. something like this:
                  >
                  > localhost:10025 inet n - n - - smtpd
                  > -o smtpd_milters=unix:/var/spool/MIMEDefang/mimedefang.sock
                  > -o transport_maps=
                  > (maybe other -o lines to override main.cf)


                  You're on the right track here.

                  The transport table cannot be overridden with a -o option, so this
                  would need to go through a separate postfix instance, not just a
                  different smtpd listener.
                  http://www.postfix.org/MULTI_INSTANCE_README.html


                  On 2/5/2013 11:23 AM, James Griffin wrote:
                  > I believe demime would/could achieve what you're looking for. It
                  > is used on a few mailing lists i'm subscribed to and it's quite
                  > easy to set up. Have you considered that or other alternatives?

                  That sounds like a much easier solution.





                  -- Noel Jones
                Your message has been successfully submitted and would be delivered to recipients shortly.