Loading ...
Sorry, an error occurred while loading the content.
 

Stripping Received: headers

Expand Messages
  • Geoff Shang
    Hi, I m trying to strip Received: headers from mail at various parts of our processing, for security reasons. I m starting with mail that comes in from
    Message 1 of 16 , Apr 11, 2013
      Hi,

      I'm trying to strip Received: headers from mail at various parts of our
      processing, for security reasons.

      I'm starting with mail that comes in from authenticated clients. I tried
      doing the following:

      master.cf:

      submission inet n - - - - smtpd
      -o smtpd_enforce_tls=yes
      -o smtpd_sasl_auth_enable=yes
      -o smtpd_client_restrictions=permit_sasl_authenticated,reject
      -o milter_macro_daemon_name=ORIGINATING
      -o header_checks=pcre:/etc/postfix/header_checks

      /etc/postfix/header_checks:

      /^Received:/ IGNORE

      I ran this through Postmap with a query from a message I sent myself, and
      the IGNORE key is correctly returned. But if I actually send myself a
      message, it comes through with the Received: line intact.

      I did some searching and found
      http://marc.info/?l=postfix-users&m=122106227124195&w=2

      I'm curious to know why this would work and the above wouldn't. Am I just
      trying to do it too early in the process?

      A related question, is it possible to prevent Postfix from generating
      lines like this?

      Geoff.
    • Wietse Venema
      ... As documented header_checks is not an smtpd(8) feature, it is a cleanup(8) feature. The easiest way to give separate treatment to mail from the internal
      Message 2 of 16 , Apr 11, 2013
        Geoff Shang:
        > submission inet n - - - - smtpd
        > -o smtpd_enforce_tls=yes
        > -o smtpd_sasl_auth_enable=yes
        > -o smtpd_client_restrictions=permit_sasl_authenticated,reject
        > -o milter_macro_daemon_name=ORIGINATING
        > -o header_checks=pcre:/etc/postfix/header_checks

        As documented header_checks is not an smtpd(8) feature, it is
        a cleanup(8) feature.

        The easiest way to give separate treatment to mail from the
        internal network versus mail from outside is to use separate
        Postfix instances.

        Otherwise,

        submission inet n - - - - smtpd
        -o cleanup_service=submission_cleanup

        submission_cleanup unix n ............................ cleanup
        -o header_checks=pcre:/etc/postfix/header_checks

        would do the job.

        Wietse
      • Benny Pedersen
        ... header_checks is incomming on smtpd, but you use submission so you must change to to smtp_header_checks http://www.postfix.org/header_checks.5.html ...
        Message 3 of 16 , Apr 11, 2013
          Geoff Shang skrev den 2013-04-11 16:33:
          > Hi,
          >
          > I'm trying to strip Received: headers from mail at various parts of
          > our processing, for security reasons.
          >
          > I'm starting with mail that comes in from authenticated clients. I
          > tried doing the following:
          >
          > master.cf:
          >
          > submission inet n - - - - smtpd
          > -o smtpd_enforce_tls=yes
          > -o smtpd_sasl_auth_enable=yes
          > -o smtpd_client_restrictions=permit_sasl_authenticated,reject
          > -o milter_macro_daemon_name=ORIGINATING
          > -o header_checks=pcre:/etc/postfix/header_checks

          header_checks is incomming on smtpd, but you use submission

          so you must change to to smtp_header_checks

          http://www.postfix.org/header_checks.5.html

          >
          > /etc/postfix/header_checks:
          >
          > /^Received:/ IGNORE

          this one is to gready, dont use it on header_checks

          >
          > I ran this through Postmap with a query from a message I sent myself,
          > and the IGNORE key is correctly returned. But if I actually send
          > myself a message, it comes through with the Received: line intact.
          >
          > I did some searching and found
          > http://marc.info/?l=postfix-users&m=122106227124195&w=2
          >
          > I'm curious to know why this would work and the above wouldn't. Am I
          > just trying to do it too early in the process?
          >
          > A related question, is it possible to prevent Postfix from generating
          > lines like this?

          what problems do you like to resolve ?

          >
          > Geoff.

          --
          senders that put my email into body content will deliver it to my own
          trashcan, so if you like to get reply, dont do it
        • Noel Jones
          ... No, header_checks are performed on all incoming mail. As already explained, the problem above is that -o header_checks=... has no effect on smtpd(5). ...
          Message 4 of 16 , Apr 11, 2013
            On 4/11/2013 10:05 AM, Benny Pedersen wrote:
            > Geoff Shang skrev den 2013-04-11 16:33:
            >> Hi,
            >>
            >> I'm trying to strip Received: headers from mail at various parts of
            >> our processing, for security reasons.
            >>
            >> I'm starting with mail that comes in from authenticated clients. I
            >> tried doing the following:
            >>
            >> master.cf:
            >>
            >> submission inet n - - - - smtpd
            >> -o smtpd_enforce_tls=yes
            >> -o smtpd_sasl_auth_enable=yes
            >> -o smtpd_client_restrictions=permit_sasl_authenticated,reject
            >> -o milter_macro_daemon_name=ORIGINATING
            >> -o header_checks=pcre:/etc/postfix/header_checks
            >
            > header_checks is incomming on smtpd, but you use submission

            No, header_checks are performed on all incoming mail.

            As already explained, the problem above is that "-o
            header_checks=..." has no effect on smtpd(5).

            >
            > so you must change to to smtp_header_checks

            smtp_header_checks are performed on outgoing mail during smtp(5)
            delivery.

            But you're sort of on the right track. You can use
            smtp_header_checks to remove the Received: headers from
            authenticated mail before external delivery with something like:
            /^Received: .*by myserver.example.com \(Postfix\) with ESMTPS?A
            id.*$/ IGNORE

            Geoff, please note I've seen some overanxious anti-spam systems that
            consider mail with no Received: headers as spam.



            -- Noel Jones




            >
            > http://www.postfix.org/header_checks.5.html
            >
            >>
            >> /etc/postfix/header_checks:
            >>
            >> /^Received:/ IGNORE
            >
            > this one is to gready, dont use it on header_checks
            >
            >>
            >> I ran this through Postmap with a query from a message I sent myself,
            >> and the IGNORE key is correctly returned. But if I actually send
            >> myself a message, it comes through with the Received: line intact.
            >>
            >> I did some searching and found
            >> http://marc.info/?l=postfix-users&m=122106227124195&w=2
            >>
            >> I'm curious to know why this would work and the above wouldn't. Am I
            >> just trying to do it too early in the process?
            >>
            >> A related question, is it possible to prevent Postfix from generating
            >> lines like this?
            >
            > what problems do you like to resolve ?
            >
            >>
            >> Geoff.
            >
          • Benny Pedersen
            ... +1 ... yes it included as it used all incomming, but not directly with smtpd ... is submission not using smtp_header_checks ? ... yep will soon try to
            Message 5 of 16 , Apr 11, 2013
              Noel Jones skrev den 2013-04-11 18:29:

              > No, header_checks are performed on all incoming mail.

              +1

              > As already explained, the problem above is that "-o
              > header_checks=..." has no effect on smtpd(5).

              yes it included as it used all incomming, but not directly with smtpd

              >> so you must change to to smtp_header_checks
              >
              > smtp_header_checks are performed on outgoing mail during smtp(5)
              > delivery.

              is submission not using smtp_header_checks ?

              > But you're sort of on the right track. You can use
              > smtp_header_checks to remove the Received: headers from
              > authenticated mail before external delivery with something like:
              > /^Received: .*by myserver.example.com \(Postfix\) with ESMTPS?A
              > id.*$/ IGNORE

              yep will soon try to apply it here

              > Geoff, please note I've seen some overanxious anti-spam systems that
              > consider mail with no Received: headers as spam.

              can one show an example main.cf that remove all recieved headers on
              remote senders ?, there would allways be one last hop imho

              --
              senders that put my email into body content will deliver it to my own
              trashcan, so if you like to get reply, dont do it
            • DTNX Postmaster
              ... In our case, the problem was with overzealous ones that filter on all Received: headers, and therefore block legitimate mail because the authenticated
              Message 6 of 16 , Apr 11, 2013
                On Apr 11, 2013, at 18:29, Noel Jones <njones@...> wrote:

                >> so you must change to to smtp_header_checks
                >
                > smtp_header_checks are performed on outgoing mail during smtp(5)
                > delivery.
                >
                > But you're sort of on the right track. You can use
                > smtp_header_checks to remove the Received: headers from
                > authenticated mail before external delivery with something like:
                > /^Received: .*by myserver.example.com \(Postfix\) with ESMTPS?A
                > id.*$/ IGNORE
                >
                > Geoff, please note I've seen some overanxious anti-spam systems that
                > consider mail with no Received: headers as spam.

                In our case, the problem was with overzealous ones that filter on all
                Received: headers, and therefore block legitimate mail because the
                authenticated client is connecting from an access provider range listed
                by Spamhaus, or something similar.

                Our solution so far is to strip a few of the internal Received:
                headers, and 'REPLACE' the one that contains the connecting IP with a
                'Received: by hostname.domain.tld (from authenticated client)' header.
                Since the submission hosts never send directly, it will always have at
                least three or four Received: headers when offered to the destination
                MX.

                Since the regular expression is fairly specific, this is done with
                'header_checks' in our case.

                HTH,
                Jona
              • Reindl Harald
                ... has your submission service smtp or smtpd in master.cf? mine has smtpd as all other working ones out there
                Message 7 of 16 , Apr 11, 2013
                  Am 11.04.2013 18:55, schrieb Benny Pedersen:
                  >> smtp_header_checks are performed on outgoing mail during smtp(5)
                  >> delivery.
                  >
                  > is submission not using smtp_header_checks?

                  has your submission service smtp or smtpd in master.cf?
                  mine has smtpd as all other working ones out there
                • Reindl Harald
                  ... to make it clear: submission is nothing else as smtpd on port 587 and if you want not rely on /etc/services you would even write 587 instead submission the
                  Message 8 of 16 , Apr 11, 2013
                    Am 11.04.2013 19:20, schrieb Reindl Harald:
                    >
                    >
                    > Am 11.04.2013 18:55, schrieb Benny Pedersen:
                    >>> smtp_header_checks are performed on outgoing mail during smtp(5)
                    >>> delivery.
                    >>
                    >> is submission not using smtp_header_checks?
                    >
                    > has your submission service smtp or smtpd in master.cf?
                    > mine has smtpd as all other working ones out there

                    to make it clear:

                    submission is nothing else as smtpd on port 587
                    and if you want not rely on /etc/services you would
                    even write 587 instead submission

                    the only difference between port 25 and 587 is
                    usually that you require authentication on 587

                    [harry@srv-rhsoft:~]$ cat /etc/services | grep submission
                    submission 587/tcp msa # mail message submission
                    submission 587/udp msa # mail message submission
                  • Noel Jones
                    ... No. submission uses the smtpd(5) service to receive mail, which uses header_checks (indirectly, through the cleanup service). smtp_header_checks are used
                    Message 9 of 16 , Apr 11, 2013
                      On 4/11/2013 11:55 AM, Benny Pedersen wrote:
                      > Noel Jones skrev den 2013-04-11 18:29:

                      >> smtp_header_checks are performed on outgoing mail during smtp(5)
                      >> delivery.
                      >
                      > is submission not using smtp_header_checks ?

                      No.

                      submission uses the smtpd(5) service to receive mail, which uses
                      header_checks (indirectly, through the cleanup service).

                      smtp_header_checks are used by the smtp(5) transport when sending
                      mail to remote systems.


                      http://www.postfix.org/OVERVIEW.html


                      -- Noel Jones
                    • Geoff Shang
                      ... Oh duh! Thanks for pointing this out. ... Thanks Wietse. I think I will opt for this latter option. Some have suggested smtp_header_checks, and I may use
                      Message 10 of 16 , Apr 12, 2013
                        On Thu, 11 Apr 2013, Wietse Venema wrote:

                        > Geoff Shang:
                        >> submission inet n - - - - smtpd
                        >> -o smtpd_enforce_tls=yes
                        >> -o smtpd_sasl_auth_enable=yes
                        >> -o smtpd_client_restrictions=permit_sasl_authenticated,reject
                        >> -o milter_macro_daemon_name=ORIGINATING
                        >> -o header_checks=pcre:/etc/postfix/header_checks
                        >
                        > As documented header_checks is not an smtpd(8) feature, it is
                        > a cleanup(8) feature.

                        Oh duh! Thanks for pointing this out.

                        > The easiest way to give separate treatment to mail from the
                        > internal network versus mail from outside is to use separate
                        > Postfix instances.
                        >
                        > Otherwise,
                        >
                        > submission inet n - - - - smtpd
                        > -o cleanup_service=submission_cleanup
                        >
                        > submission_cleanup unix n ............................ cleanup
                        > -o header_checks=pcre:/etc/postfix/header_checks
                        >
                        > would do the job.

                        Thanks Wietse. I think I will opt for this latter option.

                        Some have suggested smtp_header_checks, and I may use this in some places.
                        But since this box will deliver some mail locally as well as externally, I
                        think I will implement the separate cleanup process.

                        Thanks everyone for your input.

                        Geoff.
                      • Geoff Shang
                        ... hmm. This didn t work. I m a bit stuck as to why. I thought that perhaps it might be running before the Received: header is created, but in that case, I
                        Message 11 of 16 , Apr 12, 2013
                          On Fri, 12 Apr 2013, Geoff Shang wrote:

                          >> submission inet n - - - - smtpd
                          >> -o cleanup_service=submission_cleanup
                          >>
                          >> submission_cleanup unix n ............................ cleanup
                          >> -o header_checks=pcre:/etc/postfix/header_checks
                          >>
                          >> would do the job.
                          >
                          > Thanks Wietse. I think I will opt for this latter option.

                          hmm. This didn't work. I'm a bit stuck as to why. I thought that
                          perhaps it might be running before the Received: header is created, but in
                          that case, I don't know why the example I linked to earlier that searches
                          for an authenticated header would work, while this would not.

                          Geoff.
                        • Wietse Venema
                          ... You need to do postfix reload after editing master.cf. The submission_cleanup service will see the Received: header that was prepended by the submission
                          Message 12 of 16 , Apr 12, 2013
                            Geoff Shang:
                            > On Fri, 12 Apr 2013, Geoff Shang wrote:
                            >
                            > >> submission inet n - - - - smtpd
                            > >> -o cleanup_service=submission_cleanup
                            > >>
                            > >> submission_cleanup unix n ............................ cleanup
                            > >> -o header_checks=pcre:/etc/postfix/header_checks
                            > >>
                            > >> would do the job.
                            > >
                            > > Thanks Wietse. I think I will opt for this latter option.
                            >
                            > hmm. This didn't work. I'm a bit stuck as to why. I thought that
                            > perhaps it might be running before the Received: header is created, but in
                            > that case, I don't know why the example I linked to earlier that searches
                            > for an authenticated header would work, while this would not.

                            You need to do "postfix reload" after editing master.cf.

                            The submission_cleanup service will see the Received: header that
                            was prepended by the submission server.

                            However, if your Milter adds headers then those aren't seen by
                            header_checks; you would need to use milter_header_checks.

                            Wietse
                          • Geoff Shang
                            ... I did. I did it again for good measure - no difference. ... Is there any way I can be sure that the special cleanup agent is running? I see the socket
                            Message 13 of 16 , Apr 12, 2013
                              On Fri, 12 Apr 2013, Wietse Venema wrote:

                              > You need to do "postfix reload" after editing master.cf.

                              I did. I did it again for good measure - no difference.

                              > The submission_cleanup service will see the Received: header that
                              > was prepended by the submission server.

                              Is there any way I can be sure that the special cleanup agent is running?
                              I see the socket /var/spool/postfix/public/submission_cleanup

                              > However, if your Milter adds headers then those aren't seen by
                              > header_checks; you would need to use milter_header_checks.

                              We don't appear to be using any milters, despite the
                              'milter_macro_daemon_name=ORIGINATING'

                              Here's what I did in case I messed up:

                              master.cf:

                              # service type private unpriv chroot wakeup maxproc command + args
                              # (yes) (yes) (yes) (never) (100)
                              #
                              ==========================================================================
                              smtp inet n - - - - smtpd
                              submission inet n - - - - smtpd
                              -o smtpd_enforce_tls=yes
                              -o smtpd_sasl_auth_enable=yes
                              -o smtpd_client_restrictions=permit_sasl_authenticated,reject
                              -o milter_macro_daemon_name=ORIGINATING
                              # Use a special cleanup service so we can strip headers.
                              -o cleanup_service=submission_cleanup

                              smtps inet n - - - - smtpd
                              -o smtpd_tls_wrappermode=yes
                              -o smtpd_sasl_auth_enable=yes
                              -o smtpd_client_restrictions=permit_sasl_authenticated,reject
                              -o milter_macro_daemon_name=ORIGINATING
                              # Use a special cleanup service so we can strip headers.
                              -o cleanup_service=submission_cleanup

                              submission_cleanup unix n - - - - cleanup
                              # Strip Received: lines from authenticated mail
                              -o header_checks=pcre:/etc/postfix/header_checks



                              /etc/postfix/header_checks:

                              # Remove any Received: headers from authenticated mail.
                              /^Received:/ IGNORE



                              An example message. The line is matched if I run it through postmap.
                              Some details have to be obscured, sorry. I'm on holiday so I'm not
                              worried about letting the hostname through, you can all get it from my
                              headers anyway. Obviously I'm not posting from my work address.

                              Return-Path: <my.address@...>
                              X-Original-To: my.address@...
                              Delivered-To: my.address@...
                              Received: from [192.168.0.20] (dsl-mlibrasgw2-50de1c-161.dhcp.inet.fi
                              [80.222.28.161])
                              by mail.example.com (Postfix) with ESMTPSA id DED281C40E9
                              for <my.address@...>; Fri, 12 Apr 2013 14:35:47
                              +0000 (UTC)
                              Date: Fri, 12 Apr 2013 17:35:44 +0300 (EEST)
                              From: Geoff Shang <my.address@...>
                              X-X-Sender: geoff@...
                              To: my.address@...
                              Subject: test
                              Message-ID: <alpine.DEB.2.02.1304121735310.14582@...>
                              User-Agent: Alpine 2.02 (DEB 1266 2009-07-14)
                              MIME-Version: 1.0
                              Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII

                              Geoff.\
                            • /dev/rob0
                              A word at the outset here: I predict this will come back to bite you in a most painful way. As Noel suggested, you re going to run afoul of some clueless spam
                              Message 14 of 16 , Apr 12, 2013
                                A word at the outset here: I predict this will come back to bite you
                                in a most painful way. As Noel suggested, you're going to run afoul
                                of some clueless spam checks. Some years back I know that Hotmail/MSN
                                actually *discarded* such mail silently!

                                Note also that Postfix itself uses Received: headers as a protection
                                against mail loops. Let's hope you don't get a loop going!

                                On Fri, Apr 12, 2013 at 05:49:47PM +0300, Geoff Shang wrote:
                                > Is there any way I can be sure that the special cleanup agent
                                > is running? I see the socket
                                > /var/spool/postfix/public/submission_cleanup

                                It's running. To see what it does:

                                > master.cf:

                                > submission_cleanup unix n - - - - cleanup
                                > # Strip Received: lines from authenticated mail
                                > -o header_checks=pcre:/etc/postfix/header_checks
                                -o syslog_name=postfix/submission/cleanup

                                Every non-default service should have its own syslog_name to enhance
                                your log searches.

                                > /etc/postfix/header_checks:
                                >
                                > # Remove any Received: headers from authenticated mail.
                                > /^Received:/ IGNORE
                                /./ WARN

                                That might get too noisy in the logs, but at least you will know your
                                alternate cleanup service is being used.
                                --
                                http://rob0.nodns4.us/ -- system administration and consulting
                                Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                              • Wietse Venema
                                ... Are you using receive_override_options? in main.cf or master.cf? Wietse
                                Message 15 of 16 , Apr 12, 2013
                                  Geoff Shang:
                                  > On Fri, 12 Apr 2013, Wietse Venema wrote:
                                  >
                                  > > You need to do "postfix reload" after editing master.cf.
                                  >
                                  > I did. I did it again for good measure - no difference.

                                  Are you using receive_override_options? in main.cf or master.cf?

                                  Wietse
                                • Geoff Shang
                                  ... No. Geoff.
                                  Message 16 of 16 , Apr 15, 2013
                                    On Fri, 12 Apr 2013, Wietse Venema wrote:

                                    > Geoff Shang:
                                    >> On Fri, 12 Apr 2013, Wietse Venema wrote:
                                    >>
                                    >>> You need to do "postfix reload" after editing master.cf.
                                    >>
                                    >> I did. I did it again for good measure - no difference.
                                    >
                                    > Are you using receive_override_options? in main.cf or master.cf?

                                    No.

                                    Geoff.
                                  Your message has been successfully submitted and would be delivered to recipients shortly.