Loading ...
Sorry, an error occurred while loading the content.

Re: How to copy all incoming and outgoing messages

Expand Messages
  • D Hill
    ... Postfix configuration parameters: always_bcc, sender_bcc_maps, recipient_bcc_maps control what you are asking for. You can do a search on this page:
    Message 1 of 14 , May 1, 2008
    • 0 Attachment
      On Thu, 1 May 2008 at 22:22 -0000, forums@... confabulated:

      > Hi Everyone,
      >
      > How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.
      >
      > So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.

      Postfix configuration parameters:

      always_bcc, sender_bcc_maps, recipient_bcc_maps

      control what you are asking for. You can do a search on this page:

      http://www.postfix.org/postconf.5.html

      for further explination.
    • Curtis Vaughan
      ... This is great question, as I ve been wondering how people meet gov. requirements on storing corp. emails with Postfix. As it is our problem is that some
      Message 2 of 14 , May 1, 2008
      • 0 Attachment
        D Hill wrote:
        > On Thu, 1 May 2008 at 22:22 -0000, forums@...
        > confabulated:
        >
        >> Hi Everyone,
        >>
        >> How do I make it so postfix will copy all incoming and outgoing
        >> emails to another email invisibly? It's a SOX requirement actually. I
        >> have to save all the emails.
        >>
        >> So, I create one account called "log". Every email that goes through
        >> the mail server has to be invisibly copied to that user whether
        >> incoming or outgoing. Thanks for the help. I'm new to postfix.
        >
        > Postfix configuration parameters:
        >
        > always_bcc, sender_bcc_maps, recipient_bcc_maps
        >
        > control what you are asking for. You can do a search on this page:
        >
        > http://www.postfix.org/postconf.5.html
        >
        > for further explination.

        This is great question, as I've been wondering how people meet gov.
        requirements on storing corp. emails with Postfix.
        As it is our problem is that some people use POP3 and delete as they
        download (I've been fighting this for years, but I can't control the
        foreign office!).
        Anyhow.... Even with IMAP, although most mail is saved, still there is
        they can delete emails from Trash and then those emails are lost.

        If all mail were just sent to another address, then it would make it
        difficult to find emails in the event we need to recover specific ones.
        It seems like there should be a way to say, regardless of whether
        someone wants to delete -- well, don't delete it from the server. Maybe
        do something else with it within the same users profile. I don't know.
        I should note that we are not a large company. Only 60-odd employees
        worldwide. And the budget for me to have continual backups doesn't
        exist. We do nightly backups using BackupPC. So we have about a weeks
        retention.

        Any ideas are most welcome, but please take into consideration our
        constrictions (budgetary).
      • Victor Duchovni
        ... You can get a fair wait with regexp based recipient_bcc_maps, which allow you to capture the original envelope recipient. Merge message copies are not
        Message 3 of 14 , May 1, 2008
        • 0 Attachment
          On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:

          > This is great question, as I've been wondering how people meet gov.
          > requirements on storing corp. emails with Postfix.
          > As it is our problem is that some people use POP3 and delete as they
          > download (I've been fighting this for years, but I can't control the
          > foreign office!).
          > Anyhow.... Even with IMAP, although most mail is saved, still there is
          > they can delete emails from Trash and then those emails are lost.

          You can get a fair wait with regexp based recipient_bcc_maps, which
          allow you to capture the original envelope recipient. Merge message
          copies are not sufficient IMHO as headers are not accurate/complete.

          I usee a "tee" proxy that sends an encapsulated archive copy and the real
          message in parallel (archive "." immediately precedes message "." and
          blocks message delivery on failure).

          --
          Viktor.

          Disclaimer: off-list followups get on-list replies or get ignored.
          Please do not ignore the "Reply-To" header.

          To unsubscribe from the postfix-users list, visit
          http://www.postfix.org/lists.html or click the link below:
          <mailto:majordomo@...?body=unsubscribe%20postfix-users>

          If my response solves your problem, the best way to thank me is to not
          send an "it worked, thanks" follow-up. If you must respond, please put
          "It worked, thanks" in the "Subject" so I can delete these quickly.
        • Sahil Tandon
          ... Is there publicly available documentation on how to implement the tee proxy backup solution in Postfix? -- Sahil Tandon
          Message 4 of 14 , May 1, 2008
          • 0 Attachment
            * Victor Duchovni <Victor.Duchovni@...> [2008-05-01 19:24:17 -0400]:

            > On Thu, May 01, 2008 at 04:17:26PM -0700, Curtis Vaughan wrote:
            >
            > > This is great question, as I've been wondering how people meet gov.
            > > requirements on storing corp. emails with Postfix.
            > > As it is our problem is that some people use POP3 and delete as they
            > > download (I've been fighting this for years, but I can't control the
            > > foreign office!).
            > > Anyhow.... Even with IMAP, although most mail is saved, still there is
            > > they can delete emails from Trash and then those emails are lost.
            >
            > You can get a fair wait with regexp based recipient_bcc_maps, which
            > allow you to capture the original envelope recipient. Merge message
            > copies are not sufficient IMHO as headers are not accurate/complete.
            >
            > I usee a "tee" proxy that sends an encapsulated archive copy and the real
            > message in parallel (archive "." immediately precedes message "." and
            > blocks message delivery on failure).

            Is there publicly available documentation on how to implement the tee proxy
            backup solution in Postfix?

            --
            Sahil Tandon <sahil@...>
          • Victor Duchovni
            ... 1. Write a tee proxy 2. Set content_filter to the tee proxy -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not
            Message 5 of 14 , May 1, 2008
            • 0 Attachment
              On Thu, May 01, 2008 at 08:04:17PM -0400, Sahil Tandon wrote:

              > Is there publicly available documentation on how to implement the tee proxy
              > backup solution in Postfix?

              1. Write a tee proxy
              2. Set content_filter to the tee proxy

              --
              Viktor.

              Disclaimer: off-list followups get on-list replies or get ignored.
              Please do not ignore the "Reply-To" header.

              To unsubscribe from the postfix-users list, visit
              http://www.postfix.org/lists.html or click the link below:
              <mailto:majordomo@...?body=unsubscribe%20postfix-users>

              If my response solves your problem, the best way to thank me is to not
              send an "it worked, thanks" follow-up. If you must respond, please put
              "It worked, thanks" in the "Subject" so I can delete these quickly.
            • Tandon, Sahil (IM)
              ... Thanks, that is very helpful. Sahil ... NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or
              Message 6 of 14 , May 2, 2008
              • 0 Attachment
                > > Is there publicly available documentation on how to
                > implement the tee
                > > proxy backup solution in Postfix?
                >
                > 1. Write a tee proxy
                > 2. Set content_filter to the tee proxy

                Thanks, that is very helpful.

                Sahil
                --------------------------------------------------------

                NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.
              • Victor Duchovni
                ... It s the best I can do until I find the cycles to fully document and then release the tee proxy I am using. I don t think it is appropriate to release it
                Message 7 of 14 , May 2, 2008
                • 0 Attachment
                  On Fri, May 02, 2008 at 09:19:34AM -0400, Tandon, Sahil (IM) wrote:

                  > > > Is there publicly available documentation on how to
                  > > implement the tee
                  > > > proxy backup solution in Postfix?
                  > >
                  > > 1. Write a tee proxy
                  > > 2. Set content_filter to the tee proxy
                  >
                  > Thanks, that is very helpful.

                  It's the best I can do until I find the cycles to fully document and
                  then release the tee proxy I am using. I don't think it is appropriate
                  to release it in an undocumented state.

                  --
                  Viktor.

                  Disclaimer: off-list followups get on-list replies or get ignored.
                  Please do not ignore the "Reply-To" header.

                  To unsubscribe from the postfix-users list, visit
                  http://www.postfix.org/lists.html or click the link below:
                  <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                  If my response solves your problem, the best way to thank me is to not
                  send an "it worked, thanks" follow-up. If you must respond, please put
                  "It worked, thanks" in the "Subject" so I can delete these quickly.
                • Charles Marcus
                  ... Understandable, and I m sure I and many others look forward to this... With the advancing requirements of burdensome regulations, this (some kind of basic
                  Message 8 of 14 , May 2, 2008
                  • 0 Attachment
                    On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                    > It's the best I can do until I find the cycles to fully document and
                    > then release the tee proxy I am using. I don't think it is appropriate
                    > to release it in an undocumented state.

                    Understandable, and I'm sure I and many others look forward to this...

                    With the advancing requirements of burdensome regulations, this (some
                    kind of basic archiving capability) will become more and more important,
                    and in my opinion, it is not unreasonable for postfix to provide an
                    integrated/built-in method of performing this function, to ensure a
                    robust and safe - ie, don't bounce if there is a problem with the
                    mirror, but queue until it is back up, etc - functionality.

                    It would also be nice if it could easily deliver to an appropriate
                    sub-folder - ie, one named after the local address part of the original
                    envelope recipient (the one(s) being tested for during recipient
                    validation stage when the primary server accepted the message for final
                    delivery)...

                    --

                    Best regards,

                    Charles
                  • Victor Duchovni
                    ... I don t expect the proxy in question to ever be part of Postfix. Postfix supports at least 4 extension mechanisms: - Post-queue content filters - Pre-queue
                    Message 9 of 14 , May 2, 2008
                    • 0 Attachment
                      On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:

                      > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                      > >It's the best I can do until I find the cycles to fully document and
                      > >then release the tee proxy I am using. I don't think it is appropriate
                      > >to release it in an undocumented state.
                      >
                      > Understandable, and I'm sure I and many others look forward to this...
                      >
                      > With the advancing requirements of burdensome regulations, this (some
                      > kind of basic archiving capability) will become more and more important,
                      > and in my opinion, it is not unreasonable for postfix to provide an
                      > integrated/built-in method of performing this function, to ensure a
                      > robust and safe - ie, don't bounce if there is a problem with the
                      > mirror, but queue until it is back up, etc - functionality.

                      I don't expect the proxy in question to ever be part of Postfix. Postfix
                      supports at least 4 extension mechanisms:

                      - Post-queue content filters
                      - Pre-queue proxy filters
                      - Milters
                      - Policy servers

                      It is up to the Postfix community and vendors to create add-on tools
                      that make use of these features. So I don't see a "buit-in" archive
                      feature any time soon.

                      > It would also be nice if it could easily deliver to an appropriate
                      > sub-folder - ie, one named after the local address part of the original
                      > envelope recipient (the one(s) being tested for during recipient
                      > validation stage when the primary server accepted the message for final
                      > delivery)...

                      The right mechanism leaves this choice to the administrator, the
                      archive copy is created and queued, after that you can deliver it
                      where-ever you want (configure the archive Postfix instance transport
                      rules accoringly). It would be wrong to make delivery decisions in the
                      archive module, they would never be sufficiently comprehensive.

                      --
                      Viktor.

                      Disclaimer: off-list followups get on-list replies or get ignored.
                      Please do not ignore the "Reply-To" header.

                      To unsubscribe from the postfix-users list, visit
                      http://www.postfix.org/lists.html or click the link below:
                      <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                      If my response solves your problem, the best way to thank me is to not
                      send an "it worked, thanks" follow-up. If you must respond, please put
                      "It worked, thanks" in the "Subject" so I can delete these quickly.
                    • Michael Katz
                      ... You can do exactly this with MPP. I get a million lashes for mentioning our solution on the list, but until someone funds my life we have to charge for
                      Message 10 of 14 , May 2, 2008
                      • 0 Attachment
                        Forums wrote:
                        > Hi Everyone,
                        >
                        > How do I make it so postfix will copy all incoming and outgoing emails to another email invisibly? It's a SOX requirement actually. I have to save all the emails.
                        >
                        > So, I create one account called "log". Every email that goes through the mail server has to be invisibly copied to that user whether incoming or outgoing. Thanks for the help. I'm new to postfix.
                        >
                        > Peter

                        You can do exactly this with MPP. I get a million lashes for mentioning
                        our solution on the list, but until someone funds my life we have to
                        charge for the software. messagepartners.com .

                        M Katz

                        >
                        >
                        >
                      • Wietse Venema
                        ... If we can agree on a usable MIME encapsulation, then it should be possible to spawn off a message in the cleanup server, after the Milter processing has
                        Message 11 of 14 , May 2, 2008
                        • 0 Attachment
                          Victor Duchovni:
                          > On Fri, May 02, 2008 at 10:41:40AM -0400, Charles Marcus wrote:
                          >
                          > > On 5/2/2008, Victor Duchovni (Victor.Duchovni@...) wrote:
                          > > >It's the best I can do until I find the cycles to fully document and
                          > > >then release the tee proxy I am using. I don't think it is appropriate
                          > > >to release it in an undocumented state.
                          > >
                          > > Understandable, and I'm sure I and many others look forward to this...
                          > >
                          > > With the advancing requirements of burdensome regulations, this (some
                          > > kind of basic archiving capability) will become more and more important,
                          > > and in my opinion, it is not unreasonable for postfix to provide an
                          > > integrated/built-in method of performing this function, to ensure a
                          > > robust and safe - ie, don't bounce if there is a problem with the
                          > > mirror, but queue until it is back up, etc - functionality.
                          >
                          > I don't expect the proxy in question to ever be part of Postfix. Postfix
                          > supports at least 4 extension mechanisms:
                          >
                          > - Post-queue content filters
                          > - Pre-queue proxy filters
                          > - Milters
                          > - Policy servers
                          >
                          > It is up to the Postfix community and vendors to create add-on tools
                          > that make use of these features. So I don't see a "buit-in" archive
                          > feature any time soon.
                          >
                          > > It would also be nice if it could easily deliver to an appropriate
                          > > sub-folder - ie, one named after the local address part of the original
                          > > envelope recipient (the one(s) being tested for during recipient
                          > > validation stage when the primary server accepted the message for final
                          > > delivery)...
                          >
                          > The right mechanism leaves this choice to the administrator, the
                          > archive copy is created and queued, after that you can deliver it
                          > where-ever you want (configure the archive Postfix instance transport
                          > rules accoringly). It would be wrong to make delivery decisions in the
                          > archive module, they would never be sufficiently comprehensive.

                          If we can agree on a usable MIME encapsulation, then it should be
                          possible to spawn off a message in the cleanup server, after the
                          Milter processing has happened, and before the (SMTP) client is
                          notified that the mail transaction is complete.

                          However, Postfix is a general-purpose MTA, and you can already
                          configure dedicated delivery channels (with transport maps and
                          master.cf) that have soft-bounce turned on, so there is no need
                          for built-in special delivery modes that never bounce. Just set
                          the maximal queue time large enough.

                          Wietse
                        • Victor Duchovni
                          ... This is a bit tricky, because the is not necessarily a right answer. - IMHO, The natural format for an archive message is a success DSN, with the
                          Message 12 of 14 , May 2, 2008
                          • 0 Attachment
                            On Fri, May 02, 2008 at 01:04:16PM -0400, Wietse Venema wrote:

                            > If we can agree on a usable MIME encapsulation, then it should be
                            > possible to spawn off a message in the cleanup server, after the
                            > Milter processing has happened, and before the (SMTP) client is
                            > notified that the mail transaction is complete.

                            This is a bit tricky, because the is not necessarily a "right" answer.

                            - IMHO, The natural format for an archive message is a success DSN,
                            with the original message attached in full (not just headers). Having
                            the format defined by a standard is IMHO rather attractive.

                            - Sendmail have for many years been supplying "copier" milter that
                            is perhaps a defacto standard for such encapsulation. This is a
                            multipart/mixed, with the envelope in the first part and the message
                            in the second. The first part encodes the sender and recipients one
                            per line as follows:

                            --boundary
                            Content-Type: text/plain
                            Content-Transfer-Encoding: 7bit

                            Attached is a copy of a message being sent by Sendmail Message Copier
                            Sendmail Copier.
                            via the proxy on <hostname.without.the.angle.brackets>.

                            Original sender: sender@...
                            Original recipient(s): rcpt1@...
                            rcpt2@...
                            rcpt3@...

                            Original message is attached.

                            --boundary

                            It would be far more natural to drop the verbiage and prefix each
                            address with a type:

                            Sender: <address>
                            Recipient: <address>
                            ...

                            The second part is the attached message.

                            - Microsoft Exchange has an archive format called
                            Exchange Envelope Journalling. This too is not ideal, but is
                            broadly implemented.

                            Any format we would choose would either be ugly (ad-hoc format from
                            existing vendor products) or Postfix-specific (at least initially).

                            The archive module for my 'tee' proxy punts the issue by using
                            format templates for the envelope part.

                            --boundary
                            Content-Type: text/plain
                            Content-Transfer-Encoding: 7bit

                            $topmatter
                            `printf "$senderfmt", $sender`
                            `printf "$1strcptfmt", $rcpt1`
                            `printf "$nxtrcptfmt", $rcpt2`
                            ...
                            $footer

                            --boundary

                            So I can generate the Sendmail compatible format without hard-coding
                            it.

                            This can generate any 2-part envelope/message encapsulation with some
                            stuff above the envelope, then the sender address in some form, then
                            the recipients with the 1st formatted differently if need be, a footer
                            and a separately attached message.

                            I also capture "ORCPT" values, but don't currently distinguish between
                            these and real recipients, because I don't accept "ORCPT" from outside,
                            rather ORCPT is the recipient just before my own virtual address
                            expansion. So capture of ORCPT would need to be optional and probably
                            deserves its own format template.

                            $topmatter
                            `printf "$senderfmt", $sender`
                            `printf "$1strcptfmt", $rcpt1`
                            [`printf "$orcptfmt", $orcpt1`]
                            `printf "$nxtrcptfmt", $rcpt2`
                            [`printf "$orcptfmt", $orcpt2`]
                            ...
                            $footer

                            With so much rope, is this still a reasonable Postfix feature?

                            --
                            Viktor.

                            Disclaimer: off-list followups get on-list replies or get ignored.
                            Please do not ignore the "Reply-To" header.

                            To unsubscribe from the postfix-users list, visit
                            http://www.postfix.org/lists.html or click the link below:
                            <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                            If my response solves your problem, the best way to thank me is to not
                            send an "it worked, thanks" follow-up. If you must respond, please put
                            "It worked, thanks" in the "Subject" so I can delete these quickly.
                          • Wietse Venema
                            [About forking off an acrhive-copy message just before the cleanup server commits the queue file transaction] ... It should be able to provide one text/plain
                            Message 13 of 14 , May 2, 2008
                            • 0 Attachment
                              [About forking off an acrhive-copy message just before the cleanup
                              server commits the queue file transaction]

                              Victor Duchovni:
                              > With so much rope, is this still a reasonable Postfix feature?

                              It should be able to provide one text/plain MIME segment with
                              original sender, sender, dsn original recipient, Postfix original
                              recipient; and one message/rfc822 MIME segment with the content.

                              As long as the fields in the first MIME segment have distinct
                              labels, I don't see this as particularly challenging. Given the
                              bounce templates as an example, the first MIME segment could even
                              be made configurable. But it would in all likelihood be a separate
                              daemon process.

                              Wietse
                            Your message has been successfully submitted and would be delivered to recipients shortly.