Loading ...
Sorry, an error occurred while loading the content.

Re: How to copy all incoming and outgoing messages

Expand Messages
  • Victor Duchovni
    ... You can get a fair wait with regexp based recipient_bcc_maps, which allow you to capture the original envelope recipient. Merge message copies are not
    Message 1 of 14 , May 1, 2008
    • 0 Attachment
      On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:

      > This is great question, as I've been wondering how people meet gov.
      > requirements on storing corp. emails with Postfix.
      > As it is our problem is that some people use POP3 and delete as they
      > download (I've been fighting this for years, but I can't control the
      > foreign office!).
      > Anyhow.... Even with IMAP, although most mail is saved, still there is
      > they can delete emails from Trash and then those emails are lost.

      You can get a fair wait with regexp based recipient_bcc_maps, which
      allow you to capture the original envelope recipient. Merge message
      copies are not sufficient IMHO as headers are not accurate/complete.

      I usee a "tee" proxy that sends an encapsulated archive copy and the real
      message in parallel (archive "." immediately precedes message "." and
      blocks message delivery on failure).

      --
      Viktor.

      Disclaimer: off-list followups get on-list replies or get ignored.
      Please do not ignore the "Reply-To" header.

      To unsubscribe from the postfix-users list, visit
      http://www.postfix.org/lists.html or click the link below:
      <mailto:majordomo@...?body=unsubscribe%20postfix-users>

      If my response solves your problem, the best way to thank me is to not
      send an "it worked, thanks" follow-up. If you must respond, please put
      "It worked, thanks" in the "Subject" so I can delete these quickly.
    • Sahil Tandon
      ... Is there publicly available documentation on how to implement the tee proxy backup solution in Postfix? -- Sahil Tandon
      Message 2 of 14 , May 1, 2008
      • 0 Attachment
        * Victor Duchovni <Victor.Duchovni@...> [2008-05-01 19:24:17 -0400]:

        > On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:
        >
        > > This is great question, as I've been wondering how people meet gov.
        > > requirements on storing corp. emails with Postfix.
        > > As it is our problem is that some people use POP3 and delete as they
        > > download (I've been fighting this for years, but I can't control the
        > > foreign office!).
        > > Anyhow.... Even with IMAP, although most mail is saved, still there is
        > > they can delete emails from Trash and then those emails are lost.
        >
        > You can get a fair wait with regexp based recipient_bcc_maps, which
        > allow you to capture the original envelope recipient. Merge message
        > copies are not sufficient IMHO as headers are not accurate/complete.
        >
        > I usee a "tee" proxy that sends an encapsulated archive copy and the real
        > message in parallel (archive "." immediately precedes message "." and
        > blocks message delivery on failure).

        Is there publicly available documentation on how to implement the tee proxy
        backup solution in Postfix?

        --
        Sahil Tandon <sahil@...>
      • Victor Duchovni
        ... 1. Write a tee proxy 2. Set content_filter to the tee proxy -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not
        Message 3 of 14 , May 1, 2008
        • 0 Attachment
          On Thu, May 01, 2008 at 08:04:17PM -0400, Sahil Tandon wrote:

          > Is there publicly available documentation on how to implement the tee proxy
          > backup solution in Postfix?

          1. Write a tee proxy
          2. Set content_filter to the tee proxy

          --
          Viktor.

          Disclaimer: off-list followups get on-list replies or get ignored.
          Please do not ignore the "Reply-To" header.

          To unsubscribe from the postfix-users list, visit
          http://www.postfix.org/lists.html or click the link below:
          <mailto:majordomo@...?body=unsubscribe%20postfix-users>

          If my response solves your problem, the best way to thank me is to not
          send an "it worked, thanks" follow-up. If you must respond, please put
          "It worked, thanks" in the "Subject" so I can delete these quickly.
        • Tandon, Sahil (IM)
          ... Thanks, that is very helpful. Sahil ... NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or
          Message 4 of 14 , May 2, 2008
          • 0 Attachment
            > > Is there publicly available documentation on how to
            > implement the tee
            > > proxy backup solution in Postfix?
            >
            > 1. Write a tee proxy
            > 2. Set content_filter to the tee proxy

            Thanks, that is very helpful.

            Sahil
            --------------------------------------------------------

            NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
          • Victor Duchovni
            ... It s the best I can do until I find the cycles to fully document and then release the tee proxy I am using. I don t think it is appropriate to release it
            Message 5 of 14 , May 2, 2008
            • 0 Attachment
              On Fri, May 02, 2008 at 09:19:34AM -0400, Tandon, Sahil (IM) wrote:

              > > > Is there publicly available documentation on how to
              > > implement the tee
              > > > proxy backup solution in Postfix?
              > >
              > > 1. Write a tee proxy
              > > 2. Set content_filter to the tee proxy
              >
              > Thanks, that is very helpful.

              It's the best I can do until I find the cycles to fully document and
              then release the tee proxy I am using. I don't think it is appropriate
              to release it in an undocumented state.

              --
              Viktor.

              Disclaimer: off-list followups get on-list replies or get ignored.
              Please do not ignore the "Reply-To" header.

              To unsubscribe from the postfix-users list, visit
              http://www.postfix.org/lists.html or click the link below:
              <mailto:majordomo@...?body=unsubscribe%20postfix-users>

              If my response solves your problem, the best way to thank me is to not
              send an "it worked, thanks" follow-up. If you must respond, please put
              "It worked, thanks" in the "Subject" so I can delete these quickly.
            • Charles Marcus
              ... Understandable, and I m sure I and many others look forward to this... With the advancing requirements of burdensome regulations, this (some kind of basic
              Message 6 of 14 , May 2, 2008
              • 0 Attachment
                On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                > It's the best I can do until I find the cycles to fully document and
                > then release the tee proxy I am using. I don't think it is appropriate
                > to release it in an undocumented state.

                Understandable, and I'm sure I and many others look forward to this...

                With the advancing requirements of burdensome regulations, this (some
                kind of basic archiving capability) will become more and more important,
                and in my opinion, it is not unreasonable for postfix to provide an
                integrated/built-in method of performing this function, to ensure a
                robust and safe - ie, don't bounce if there is a problem with the
                mirror, but queue until it is back up, etc - functionality.

                It would also be nice if it could easily deliver to an appropriate
                sub-folder - ie, one named after the local address part of the original
                envelope recipient (the one(s) being tested for during recipient
                validation stage when the primary server accepted the message for final
                delivery)...

                --

                Best regards,

                Charles
              • Victor Duchovni
                ... I don t expect the proxy in question to ever be part of Postfix. Postfix supports at least 4 extension mechanisms: - Post-queue content filters - Pre-queue
                Message 7 of 14 , May 2, 2008
                • 0 Attachment
                  On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:

                  > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                  > >It's the best I can do until I find the cycles to fully document and
                  > >then release the tee proxy I am using. I don't think it is appropriate
                  > >to release it in an undocumented state.
                  >
                  > Understandable, and I'm sure I and many others look forward to this...
                  >
                  > With the advancing requirements of burdensome regulations, this (some
                  > kind of basic archiving capability) will become more and more important,
                  > and in my opinion, it is not unreasonable for postfix to provide an
                  > integrated/built-in method of performing this function, to ensure a
                  > robust and safe - ie, don't bounce if there is a problem with the
                  > mirror, but queue until it is back up, etc - functionality.

                  I don't expect the proxy in question to ever be part of Postfix. Postfix
                  supports at least 4 extension mechanisms:

                  - Post-queue content filters
                  - Pre-queue proxy filters
                  - Milters
                  - Policy servers

                  It is up to the Postfix community and vendors to create add-on tools
                  that make use of these features. So I don't see a "buit-in" archive
                  feature any time soon.

                  > It would also be nice if it could easily deliver to an appropriate
                  > sub-folder - ie, one named after the local address part of the original
                  > envelope recipient (the one(s) being tested for during recipient
                  > validation stage when the primary server accepted the message for final
                  > delivery)...

                  The right mechanism leaves this choice to the administrator, the
                  archive copy is created and queued, after that you can deliver it
                  where-ever you want (configure the archive Postfix instance transport
                  rules accoringly). It would be wrong to make delivery decisions in the
                  archive module, they would never be sufficiently comprehensive.

                  --
                  Viktor.

                  Disclaimer: off-list followups get on-list replies or get ignored.
                  Please do not ignore the "Reply-To" header.

                  To unsubscribe from the postfix-users list, visit
                  http://www.postfix.org/lists.html or click the link below:
                  <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                  If my response solves your problem, the best way to thank me is to not
                  send an "it worked, thanks" follow-up. If you must respond, please put
                  "It worked, thanks" in the "Subject" so I can delete these quickly.
                • Michael Katz
                  ... You can do exactly this with MPP. I get a million lashes for mentioning our solution on the list, but until someone funds my life we have to charge for
                  Message 8 of 14 , May 2, 2008
                  • 0 Attachment
                    Forums wrote:
                    > Hi Everyone,
                    >
                    > How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.
                    >
                    > So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.
                    >
                    > Peter

                    You can do exactly this with MPP. I get a million lashes for mentioning
                    our solution on the list, but until someone funds my life we have to
                    charge for the software. messagepartners.com .

                    M Katz

                    >
                    >
                    >
                  • Wietse Venema
                    ... If we can agree on a usable MIME encapsulation, then it should be possible to spawn off a message in the cleanup server, after the Milter processing has
                    Message 9 of 14 , May 2, 2008
                    • 0 Attachment
                      Victor Duchovni:
                      > On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:
                      >
                      > > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                      > > >It's the best I can do until I find the cycles to fully document and
                      > > >then release the tee proxy I am using. I don't think it is appropriate
                      > > >to release it in an undocumented state.
                      > >
                      > > Understandable, and I'm sure I and many others look forward to this...
                      > >
                      > > With the advancing requirements of burdensome regulations, this (some
                      > > kind of basic archiving capability) will become more and more important,
                      > > and in my opinion, it is not unreasonable for postfix to provide an
                      > > integrated/built-in method of performing this function, to ensure a
                      > > robust and safe - ie, don't bounce if there is a problem with the
                      > > mirror, but queue until it is back up, etc - functionality.
                      >
                      > I don't expect the proxy in question to ever be part of Postfix. Postfix
                      > supports at least 4 extension mechanisms:
                      >
                      > - Post-queue content filters
                      > - Pre-queue proxy filters
                      > - Milters
                      > - Policy servers
                      >
                      > It is up to the Postfix community and vendors to create add-on tools
                      > that make use of these features. So I don't see a "buit-in" archive
                      > feature any time soon.
                      >
                      > > It would also be nice if it could easily deliver to an appropriate
                      > > sub-folder - ie, one named after the local address part of the original
                      > > envelope recipient (the one(s) being tested for during recipient
                      > > validation stage when the primary server accepted the message for final
                      > > delivery)...
                      >
                      > The right mechanism leaves this choice to the administrator, the
                      > archive copy is created and queued, after that you can deliver it
                      > where-ever you want (configure the archive Postfix instance transport
                      > rules accoringly). It would be wrong to make delivery decisions in the
                      > archive module, they would never be sufficiently comprehensive.

                      If we can agree on a usable MIME encapsulation, then it should be
                      possible to spawn off a message in the cleanup server, after the
                      Milter processing has happened, and before the (SMTP) client is
                      notified that the mail transaction is complete.

                      However, Postfix is a general-purpose MTA, and you can already
                      configure dedicated delivery channels (with transport maps and
                      master.cf) that have soft-bounce turned on, so there is no need
                      for built-in special delivery modes that never bounce. Just set
                      the maximal queue time large enough.

                      Wietse
                    • Victor Duchovni
                      ... This is a bit tricky, because the is not necessarily a right answer. - IMHO, The natural format for an archive message is a success DSN, with the
                      Message 10 of 14 , May 2, 2008
                      • 0 Attachment
                        On Fri, May 02, 2008 at 01:04:16PM -0400, Wietse Venema wrote:

                        > If we can agree on a usable MIME encapsulation, then it should be
                        > possible to spawn off a message in the cleanup server, after the
                        > Milter processing has happened, and before the (SMTP) client is
                        > notified that the mail transaction is complete.

                        This is a bit tricky, because the is not necessarily a "right" answer.

                        - IMHO, The natural format for an archive message is a success DSN,
                        with the original message attached in full (not just headers). Having
                        the format defined by a standard is IMHO rather attractive.

                        - Sendmail have for many years been supplying "copier" milter that
                        is perhaps a defacto standard for such encapsulation. This is a
                        multipart/mixed, with the envelope in the first part and the message
                        in the second. The first part encodes the sender and recipients one
                        per line as follows:

                        --boundary
                        Content-Type: text/plain
                        Content-Transfer-Encoding: 7bit

                        Attached is a copy of a message being sent by Sendmail Message Copier
                        Sendmail Copier.
                        via the proxy on <hostname.without.the.angle.brackets>.

                        Original sender: sender@...
                        Original recipient(s): rcpt1@...
                        rcpt2@...
                        rcpt3@...

                        Original message is attached.

                        --boundary

                        It would be far more natural to drop the verbiage and prefix each
                        address with a type:

                        Sender: <address>
                        Recipient: <address>
                        ...

                        The second part is the attached message.

                        - Microsoft Exchange has an archive format called
                        Exchange Envelope Journalling. This too is not ideal, but is
                        broadly implemented.

                        Any format we would choose would either be ugly (ad-hoc format from
                        existing vendor products) or Postfix-specific (at least initially).

                        The archive module for my 'tee' proxy punts the issue by using
                        format templates for the envelope part.

                        --boundary
                        Content-Type: text/plain
                        Content-Transfer-Encoding: 7bit

                        $topmatter
                        `printf "$senderfmt", $sender`
                        `printf "$1strcptfmt", $rcpt1`
                        `printf "$nxtrcptfmt", $rcpt2`
                        ...
                        $footer

                        --boundary

                        So I can generate the Sendmail compatible format without hard-coding
                        it.

                        This can generate any 2-part envelope/message encapsulation with some
                        stuff above the envelope, then the sender address in some form, then
                        the recipients with the 1st formatted differently if need be, a footer
                        and a separately attached message.

                        I also capture "ORCPT" values, but don't currently distinguish between
                        these and real recipients, because I don't accept "ORCPT" from outside,
                        rather ORCPT is the recipient just before my own virtual address
                        expansion. So capture of ORCPT would need to be optional and probably
                        deserves its own format template.

                        $topmatter
                        `printf "$senderfmt", $sender`
                        `printf "$1strcptfmt", $rcpt1`
                        [`printf "$orcptfmt", $orcpt1`]
                        `printf "$nxtrcptfmt", $rcpt2`
                        [`printf "$orcptfmt", $orcpt2`]
                        ...
                        $footer

                        With so much rope, is this still a reasonable Postfix feature?

                        --
                        Viktor.

                        Disclaimer: off-list followups get on-list replies or get ignored.
                        Please do not ignore the "Reply-To" header.

                        To unsubscribe from the postfix-users list, visit
                        http://www.postfix.org/lists.html or click the link below:
                        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                        If my response solves your problem, the best way to thank me is to not
                        send an "it worked, thanks" follow-up. If you must respond, please put
                        "It worked, thanks" in the "Subject" so I can delete these quickly.
                      • Wietse Venema
                        [About forking off an acrhive-copy message just before the cleanup server commits the queue file transaction] ... It should be able to provide one text/plain
                        Message 11 of 14 , May 2, 2008
                        • 0 Attachment
                          [About forking off an acrhive-copy message just before the cleanup
                          server commits the queue file transaction]

                          Victor Duchovni:
                          > With so much rope, is this still a reasonable Postfix feature?

                          It should be able to provide one text/plain MIME segment with
                          original sender, sender, dsn original recipient, Postfix original
                          recipient; and one message/rfc822 MIME segment with the content.

                          As long as the fields in the first MIME segment have distinct
                          labels, I don't see this as particularly challenging. Given the
                          bounce templates as an example, the first MIME segment could even
                          be made configurable. But it would in all likelihood be a separate
                          daemon process.

                          Wietse
                        Your message has been successfully submitted and would be delivered to recipients shortly.