Loading ...
Sorry, an error occurred while loading the content.

How to copy all incoming and outgoing messages

Expand Messages
  • Forums
    Hi Everyone, How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It s a SOX requirement actually. I have to save
    Message 1 of 14 , May 1, 2008
    • 0 Attachment
      Hi Everyone,

      How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.

      So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.

      Peter
    • D Hill
      ... Postfix configuration parameters: always_bcc, sender_bcc_maps, recipient_bcc_maps control what you are asking for. You can do a search on this page:
      Message 2 of 14 , May 1, 2008
      • 0 Attachment
        On Thu, 1 May 2008 at 22:22 -0000, forums@... confabulated:

        > Hi Everyone,
        >
        > How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.
        >
        > So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.

        Postfix configuration parameters:

        always_bcc, sender_bcc_maps, recipient_bcc_maps

        control what you are asking for. You can do a search on this page:

        http://www.postfix.org/postconf.5.html

        for further explination.
      • Curtis Vaughan
        ... This is great question, as I ve been wondering how people meet gov. requirements on storing corp. emails with Postfix. As it is our problem is that some
        Message 3 of 14 , May 1, 2008
        • 0 Attachment
          D Hill wrote:
          > On Thu, 1 May 2008 at 22:22 -0000, forums@...
          > confabulated:
          >
          >> Hi Everyone,
          >>
          >> How do I make it so postfix will copy all incoming and outgoing
          >> emails to another email invisibly? It's a SOX requirement actually. I
          >> have to save all the emails.
          >>
          >> So, I create one account called "log". Every email that goes through
          >> the mail server has to be invisibly copied to that user whether
          >> incoming or outgoing. Thanks for the help. I'm new to postfix.
          >
          > Postfix configuration parameters:
          >
          > always_bcc, sender_bcc_maps, recipient_bcc_maps
          >
          > control what you are asking for. You can do a search on this page:
          >
          > http://www.postfix.org/postconf.5.html
          >
          > for further explination.

          This is great question, as I've been wondering how people meet gov.
          requirements on storing corp. emails with Postfix.
          As it is our problem is that some people use POP3 and delete as they
          download (I've been fighting this for years, but I can't control the
          foreign office!).
          Anyhow.... Even with IMAP, although most mail is saved, still there is
          they can delete emails from Trash and then those emails are lost.

          If all mail were just sent to another address, then it would make it
          difficult to find emails in the event we need to recover specific ones.
          It seems like there should be a way to say, regardless of whether
          someone wants to delete -- well, don't delete it from the server. Maybe
          do something else with it within the same users profile. I don't know.
          I should note that we are not a large company. Only 60-odd employees
          worldwide. And the budget for me to have continual backups doesn't
          exist. We do nightly backups using BackupPC. So we have about a weeks
          retention.

          Any ideas are most welcome, but please take into consideration our
          constrictions (budgetary).
        • Victor Duchovni
          ... You can get a fair wait with regexp based recipient_bcc_maps, which allow you to capture the original envelope recipient. Merge message copies are not
          Message 4 of 14 , May 1, 2008
          • 0 Attachment
            On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:

            > This is great question, as I've been wondering how people meet gov.
            > requirements on storing corp. emails with Postfix.
            > As it is our problem is that some people use POP3 and delete as they
            > download (I've been fighting this for years, but I can't control the
            > foreign office!).
            > Anyhow.... Even with IMAP, although most mail is saved, still there is
            > they can delete emails from Trash and then those emails are lost.

            You can get a fair wait with regexp based recipient_bcc_maps, which
            allow you to capture the original envelope recipient. Merge message
            copies are not sufficient IMHO as headers are not accurate/complete.

            I usee a "tee" proxy that sends an encapsulated archive copy and the real
            message in parallel (archive "." immediately precedes message "." and
            blocks message delivery on failure).

            --
            Viktor.

            Disclaimer: off-list followups get on-list replies or get ignored.
            Please do not ignore the "Reply-To" header.

            To unsubscribe from the postfix-users list, visit
            http://www.postfix.org/lists.html or click the link below:
            <mailto:majordomo@...?body=unsubscribe%20postfix-users>

            If my response solves your problem, the best way to thank me is to not
            send an "it worked, thanks" follow-up. If you must respond, please put
            "It worked, thanks" in the "Subject" so I can delete these quickly.
          • Sahil Tandon
            ... Is there publicly available documentation on how to implement the tee proxy backup solution in Postfix? -- Sahil Tandon
            Message 5 of 14 , May 1, 2008
            • 0 Attachment
              * Victor Duchovni <Victor.Duchovni@...> [2008-05-01 19:24:17 -0400]:

              > On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:
              >
              > > This is great question, as I've been wondering how people meet gov.
              > > requirements on storing corp. emails with Postfix.
              > > As it is our problem is that some people use POP3 and delete as they
              > > download (I've been fighting this for years, but I can't control the
              > > foreign office!).
              > > Anyhow.... Even with IMAP, although most mail is saved, still there is
              > > they can delete emails from Trash and then those emails are lost.
              >
              > You can get a fair wait with regexp based recipient_bcc_maps, which
              > allow you to capture the original envelope recipient. Merge message
              > copies are not sufficient IMHO as headers are not accurate/complete.
              >
              > I usee a "tee" proxy that sends an encapsulated archive copy and the real
              > message in parallel (archive "." immediately precedes message "." and
              > blocks message delivery on failure).

              Is there publicly available documentation on how to implement the tee proxy
              backup solution in Postfix?

              --
              Sahil Tandon <sahil@...>
            • Victor Duchovni
              ... 1. Write a tee proxy 2. Set content_filter to the tee proxy -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not
              Message 6 of 14 , May 1, 2008
              • 0 Attachment
                On Thu, May 01, 2008 at 08:04:17PM -0400, Sahil Tandon wrote:

                > Is there publicly available documentation on how to implement the tee proxy
                > backup solution in Postfix?

                1. Write a tee proxy
                2. Set content_filter to the tee proxy

                --
                Viktor.

                Disclaimer: off-list followups get on-list replies or get ignored.
                Please do not ignore the "Reply-To" header.

                To unsubscribe from the postfix-users list, visit
                http://www.postfix.org/lists.html or click the link below:
                <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                If my response solves your problem, the best way to thank me is to not
                send an "it worked, thanks" follow-up. If you must respond, please put
                "It worked, thanks" in the "Subject" so I can delete these quickly.
              • Tandon, Sahil (IM)
                ... Thanks, that is very helpful. Sahil ... NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or
                Message 7 of 14 , May 2, 2008
                • 0 Attachment
                  > > Is there publicly available documentation on how to
                  > implement the tee
                  > > proxy backup solution in Postfix?
                  >
                  > 1. Write a tee proxy
                  > 2. Set content_filter to the tee proxy

                  Thanks, that is very helpful.

                  Sahil
                  --------------------------------------------------------

                  NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
                • Victor Duchovni
                  ... It s the best I can do until I find the cycles to fully document and then release the tee proxy I am using. I don t think it is appropriate to release it
                  Message 8 of 14 , May 2, 2008
                  • 0 Attachment
                    On Fri, May 02, 2008 at 09:19:34AM -0400, Tandon, Sahil (IM) wrote:

                    > > > Is there publicly available documentation on how to
                    > > implement the tee
                    > > > proxy backup solution in Postfix?
                    > >
                    > > 1. Write a tee proxy
                    > > 2. Set content_filter to the tee proxy
                    >
                    > Thanks, that is very helpful.

                    It's the best I can do until I find the cycles to fully document and
                    then release the tee proxy I am using. I don't think it is appropriate
                    to release it in an undocumented state.

                    --
                    Viktor.

                    Disclaimer: off-list followups get on-list replies or get ignored.
                    Please do not ignore the "Reply-To" header.

                    To unsubscribe from the postfix-users list, visit
                    http://www.postfix.org/lists.html or click the link below:
                    <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                    If my response solves your problem, the best way to thank me is to not
                    send an "it worked, thanks" follow-up. If you must respond, please put
                    "It worked, thanks" in the "Subject" so I can delete these quickly.
                  • Charles Marcus
                    ... Understandable, and I m sure I and many others look forward to this... With the advancing requirements of burdensome regulations, this (some kind of basic
                    Message 9 of 14 , May 2, 2008
                    • 0 Attachment
                      On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                      > It's the best I can do until I find the cycles to fully document and
                      > then release the tee proxy I am using. I don't think it is appropriate
                      > to release it in an undocumented state.

                      Understandable, and I'm sure I and many others look forward to this...

                      With the advancing requirements of burdensome regulations, this (some
                      kind of basic archiving capability) will become more and more important,
                      and in my opinion, it is not unreasonable for postfix to provide an
                      integrated/built-in method of performing this function, to ensure a
                      robust and safe - ie, don't bounce if there is a problem with the
                      mirror, but queue until it is back up, etc - functionality.

                      It would also be nice if it could easily deliver to an appropriate
                      sub-folder - ie, one named after the local address part of the original
                      envelope recipient (the one(s) being tested for during recipient
                      validation stage when the primary server accepted the message for final
                      delivery)...

                      --

                      Best regards,

                      Charles
                    • Victor Duchovni
                      ... I don t expect the proxy in question to ever be part of Postfix. Postfix supports at least 4 extension mechanisms: - Post-queue content filters - Pre-queue
                      Message 10 of 14 , May 2, 2008
                      • 0 Attachment
                        On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:

                        > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                        > >It's the best I can do until I find the cycles to fully document and
                        > >then release the tee proxy I am using. I don't think it is appropriate
                        > >to release it in an undocumented state.
                        >
                        > Understandable, and I'm sure I and many others look forward to this...
                        >
                        > With the advancing requirements of burdensome regulations, this (some
                        > kind of basic archiving capability) will become more and more important,
                        > and in my opinion, it is not unreasonable for postfix to provide an
                        > integrated/built-in method of performing this function, to ensure a
                        > robust and safe - ie, don't bounce if there is a problem with the
                        > mirror, but queue until it is back up, etc - functionality.

                        I don't expect the proxy in question to ever be part of Postfix. Postfix
                        supports at least 4 extension mechanisms:

                        - Post-queue content filters
                        - Pre-queue proxy filters
                        - Milters
                        - Policy servers

                        It is up to the Postfix community and vendors to create add-on tools
                        that make use of these features. So I don't see a "buit-in" archive
                        feature any time soon.

                        > It would also be nice if it could easily deliver to an appropriate
                        > sub-folder - ie, one named after the local address part of the original
                        > envelope recipient (the one(s) being tested for during recipient
                        > validation stage when the primary server accepted the message for final
                        > delivery)...

                        The right mechanism leaves this choice to the administrator, the
                        archive copy is created and queued, after that you can deliver it
                        where-ever you want (configure the archive Postfix instance transport
                        rules accoringly). It would be wrong to make delivery decisions in the
                        archive module, they would never be sufficiently comprehensive.

                        --
                        Viktor.

                        Disclaimer: off-list followups get on-list replies or get ignored.
                        Please do not ignore the "Reply-To" header.

                        To unsubscribe from the postfix-users list, visit
                        http://www.postfix.org/lists.html or click the link below:
                        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                        If my response solves your problem, the best way to thank me is to not
                        send an "it worked, thanks" follow-up. If you must respond, please put
                        "It worked, thanks" in the "Subject" so I can delete these quickly.
                      • Michael Katz
                        ... You can do exactly this with MPP. I get a million lashes for mentioning our solution on the list, but until someone funds my life we have to charge for
                        Message 11 of 14 , May 2, 2008
                        • 0 Attachment
                          Forums wrote:
                          > Hi Everyone,
                          >
                          > How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.
                          >
                          > So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.
                          >
                          > Peter

                          You can do exactly this with MPP. I get a million lashes for mentioning
                          our solution on the list, but until someone funds my life we have to
                          charge for the software. messagepartners.com .

                          M Katz

                          >
                          >
                          >
                        • Wietse Venema
                          ... If we can agree on a usable MIME encapsulation, then it should be possible to spawn off a message in the cleanup server, after the Milter processing has
                          Message 12 of 14 , May 2, 2008
                          • 0 Attachment
                            Victor Duchovni:
                            > On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:
                            >
                            > > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                            > > >It's the best I can do until I find the cycles to fully document and
                            > > >then release the tee proxy I am using. I don't think it is appropriate
                            > > >to release it in an undocumented state.
                            > >
                            > > Understandable, and I'm sure I and many others look forward to this...
                            > >
                            > > With the advancing requirements of burdensome regulations, this (some
                            > > kind of basic archiving capability) will become more and more important,
                            > > and in my opinion, it is not unreasonable for postfix to provide an
                            > > integrated/built-in method of performing this function, to ensure a
                            > > robust and safe - ie, don't bounce if there is a problem with the
                            > > mirror, but queue until it is back up, etc - functionality.
                            >
                            > I don't expect the proxy in question to ever be part of Postfix. Postfix
                            > supports at least 4 extension mechanisms:
                            >
                            > - Post-queue content filters
                            > - Pre-queue proxy filters
                            > - Milters
                            > - Policy servers
                            >
                            > It is up to the Postfix community and vendors to create add-on tools
                            > that make use of these features. So I don't see a "buit-in" archive
                            > feature any time soon.
                            >
                            > > It would also be nice if it could easily deliver to an appropriate
                            > > sub-folder - ie, one named after the local address part of the original
                            > > envelope recipient (the one(s) being tested for during recipient
                            > > validation stage when the primary server accepted the message for final
                            > > delivery)...
                            >
                            > The right mechanism leaves this choice to the administrator, the
                            > archive copy is created and queued, after that you can deliver it
                            > where-ever you want (configure the archive Postfix instance transport
                            > rules accoringly). It would be wrong to make delivery decisions in the
                            > archive module, they would never be sufficiently comprehensive.

                            If we can agree on a usable MIME encapsulation, then it should be
                            possible to spawn off a message in the cleanup server, after the
                            Milter processing has happened, and before the (SMTP) client is
                            notified that the mail transaction is complete.

                            However, Postfix is a general-purpose MTA, and you can already
                            configure dedicated delivery channels (with transport maps and
                            master.cf) that have soft-bounce turned on, so there is no need
                            for built-in special delivery modes that never bounce. Just set
                            the maximal queue time large enough.

                            Wietse
                          • Victor Duchovni
                            ... This is a bit tricky, because the is not necessarily a right answer. - IMHO, The natural format for an archive message is a success DSN, with the
                            Message 13 of 14 , May 2, 2008
                            • 0 Attachment
                              On Fri, May 02, 2008 at 01:04:16PM -0400, Wietse Venema wrote:

                              > If we can agree on a usable MIME encapsulation, then it should be
                              > possible to spawn off a message in the cleanup server, after the
                              > Milter processing has happened, and before the (SMTP) client is
                              > notified that the mail transaction is complete.

                              This is a bit tricky, because the is not necessarily a "right" answer.

                              - IMHO, The natural format for an archive message is a success DSN,
                              with the original message attached in full (not just headers). Having
                              the format defined by a standard is IMHO rather attractive.

                              - Sendmail have for many years been supplying "copier" milter that
                              is perhaps a defacto standard for such encapsulation. This is a
                              multipart/mixed, with the envelope in the first part and the message
                              in the second. The first part encodes the sender and recipients one
                              per line as follows:

                              --boundary
                              Content-Type: text/plain
                              Content-Transfer-Encoding: 7bit

                              Attached is a copy of a message being sent by Sendmail Message Copier
                              Sendmail Copier.
                              via the proxy on <hostname.without.the.angle.brackets>.

                              Original sender: sender@...
                              Original recipient(s): rcpt1@...
                              rcpt2@...
                              rcpt3@...

                              Original message is attached.

                              --boundary

                              It would be far more natural to drop the verbiage and prefix each
                              address with a type:

                              Sender: <address>
                              Recipient: <address>
                              ...

                              The second part is the attached message.

                              - Microsoft Exchange has an archive format called
                              Exchange Envelope Journalling. This too is not ideal, but is
                              broadly implemented.

                              Any format we would choose would either be ugly (ad-hoc format from
                              existing vendor products) or Postfix-specific (at least initially).

                              The archive module for my 'tee' proxy punts the issue by using
                              format templates for the envelope part.

                              --boundary
                              Content-Type: text/plain
                              Content-Transfer-Encoding: 7bit

                              $topmatter
                              `printf "$senderfmt", $sender`
                              `printf "$1strcptfmt", $rcpt1`
                              `printf "$nxtrcptfmt", $rcpt2`
                              ...
                              $footer

                              --boundary

                              So I can generate the Sendmail compatible format without hard-coding
                              it.

                              This can generate any 2-part envelope/message encapsulation with some
                              stuff above the envelope, then the sender address in some form, then
                              the recipients with the 1st formatted differently if need be, a footer
                              and a separately attached message.

                              I also capture "ORCPT" values, but don't currently distinguish between
                              these and real recipients, because I don't accept "ORCPT" from outside,
                              rather ORCPT is the recipient just before my own virtual address
                              expansion. So capture of ORCPT would need to be optional and probably
                              deserves its own format template.

                              $topmatter
                              `printf "$senderfmt", $sender`
                              `printf "$1strcptfmt", $rcpt1`
                              [`printf "$orcptfmt", $orcpt1`]
                              `printf "$nxtrcptfmt", $rcpt2`
                              [`printf "$orcptfmt", $orcpt2`]
                              ...
                              $footer

                              With so much rope, is this still a reasonable Postfix feature?

                              --
                              Viktor.

                              Disclaimer: off-list followups get on-list replies or get ignored.
                              Please do not ignore the "Reply-To" header.

                              To unsubscribe from the postfix-users list, visit
                              http://www.postfix.org/lists.html or click the link below:
                              <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                              If my response solves your problem, the best way to thank me is to not
                              send an "it worked, thanks" follow-up. If you must respond, please put
                              "It worked, thanks" in the "Subject" so I can delete these quickly.
                            • Wietse Venema
                              [About forking off an acrhive-copy message just before the cleanup server commits the queue file transaction] ... It should be able to provide one text/plain
                              Message 14 of 14 , May 2, 2008
                              • 0 Attachment
                                [About forking off an acrhive-copy message just before the cleanup
                                server commits the queue file transaction]

                                Victor Duchovni:
                                > With so much rope, is this still a reasonable Postfix feature?

                                It should be able to provide one text/plain MIME segment with
                                original sender, sender, dsn original recipient, Postfix original
                                recipient; and one message/rfc822 MIME segment with the content.

                                As long as the fields in the first MIME segment have distinct
                                labels, I don't see this as particularly challenging. Given the
                                bounce templates as an example, the first MIME segment could even
                                be made configurable. But it would in all likelihood be a separate
                                daemon process.

                                Wietse
                              Your message has been successfully submitted and would be delivered to recipients shortly.