Loading ...
Sorry, an error occurred while loading the content.

Re: Finding the envelope-sender after always_bcc?

Expand Messages
  • Jeff Weinberger
    ... Thank you! I will ask on the dspam list about this. I ve been reading the dspam docs this morning and the project owner states fairly strongly that
    Message 1 of 28 , Jan 1, 2009
    • 0 Attachment
      On Jan 1, 2009, at 9:49 AM, mouss wrote:

      > Jeff Weinberger a écrit :
      >>
      >> Thank you! That helps clarify what I was missing in this - so clearly
      >> dspam is not passing the envelope sender back to postfix.
      >>
      >> I suspect it doesn't know how via SMTP, but it can also use the
      >> sendmail command to do this. I will experiment with this.
      >>
      >
      > no, keep using smtp. just check your setup (and see why dspam barks
      > about sql errors). ask on the dspam list.
      >
      > If you wnat to keep using dspam, you should follow the (recent) fork:
      > http://www.dspam-community.org
      >
      >> would the correct command be "/usr/sbin/sendmail -f" ?
      >>
      >
      > yes, but smtp is better.

      Thank you!

      I will ask on the dspam list about this.

      I've been reading the dspam docs this morning and the project owner
      states fairly strongly that sendmail is far preferable to SMTP re-
      injection (in the postfix setup docs), but doesn't say why. I don't
      know enough about the workings of dspam (yet) to know, but I'd tend to
      agree with you on the use of SMTP as more desirable.

      dspam has always had this mysql issue, which is related to its
      database of spam tokens, and I'll look into whether it might be related.

      Thank you for your help!
    • mouss
      ... if he doesn t say why, then we don t know whether there was a problem at the time or if there is another reason. People have been using smtp mode without
      Message 2 of 28 , Jan 1, 2009
      • 0 Attachment
        Jeff Weinberger a écrit :
        >
        > I've been reading the dspam docs this morning and the project owner
        > states fairly strongly that sendmail is far preferable to SMTP
        > re-injection (in the postfix setup docs), but doesn't say why.

        if he doesn't say why, then we don't know whether there was a problem at
        the time or if there is another reason. People have been using smtp mode
        without problems.

        > I don't
        > know enough about the workings of dspam (yet) to know, but I'd tend to
        > agree with you on the use of SMTP as more desirable.
        >
        > dspam has always had this mysql issue, which is related to its database
        > of spam tokens, and I'll look into whether it might be related.
        >

        I don't think it is related, but check it though...
      • Jeff Weinberger
        ... First let me thank mouss for all his generous help on this - as it turned out, I was able to solve it with the information in this thread and
        Message 3 of 28 , Jan 1, 2009
        • 0 Attachment
          On Jan 1, 2009, at 12:58 PM, mouss wrote:

          > Jeff Weinberger a écrit :
          >>
          >> I've been reading the dspam docs this morning and the project owner
          >> states fairly strongly that sendmail is far preferable to SMTP
          >> re-injection (in the postfix setup docs), but doesn't say why.
          >
          > if he doesn't say why, then we don't know whether there was a
          > problem at
          > the time or if there is another reason. People have been using smtp
          > mode
          > without problems.
          >
          >> I don't
          >> know enough about the workings of dspam (yet) to know, but I'd tend
          >> to
          >> agree with you on the use of SMTP as more desirable.
          >>
          >> dspam has always had this mysql issue, which is related to its
          >> database
          >> of spam tokens, and I'll look into whether it might be related.
          >>
          >
          > I don't think it is related, but check it though...
          >

          First let me thank mouss for all his generous help on this - as it
          turned out, I was able to solve it with the information in this thread
          and http://www.pubbs.net/postfix/200811/3719/ from Victor but it
          required some trickery to get the envelope-sender past dspam.

          I'm posting my solution here in the hope that it might help someone
          else.

          I used a pcre: table for smtpd_sender_restrictions and the PREPEND
          action as follows:

          main.cf:
          smtpd_sender_restrictions = check_sender_access pcre:/etc/
          postfix/smtpd_sender_restrictions.pcre

          smtpd_sender_restrictions.pcre
          /^(.*)/ PREPEND X-Envelope-Sender: ${1}

          this will insert into every message a header "X-Envelope-Sender: "
          followed by the envelope sender value. It won't pass the envelope-
          sender as SMTP MAIL FROM (dspam wasn't designed to do that, and until
          I take the step to a better before-queue filter or something that
          does, this will work, since all I needed was to capture the envelope-
          sender).

          Limited testing shows this to work. There might be cases beyond what I
          tested that will behave differently than I expect or very oddly.

          Thanks again!
        • mouss
          ... dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix with SMT MAIL FROM. I have used this in the past and I m sure others are using it now
          Message 4 of 28 , Jan 2, 2009
          • 0 Attachment
            Jeff Weinberger a écrit :
            >
            > I used a pcre: table for smtpd_sender_restrictions and the PREPEND
            > action as follows:
            >
            > main.cf:
            > smtpd_sender_restrictions = check_sender_access
            > pcre:/etc/postfix/smtpd_sender_restrictions.pcre
            >
            > smtpd_sender_restrictions.pcre
            > /^(.*)/ PREPEND X-Envelope-Sender: ${1}
            >
            > this will insert into every message a header "X-Envelope-Sender: "
            > followed by the envelope sender value. It won't pass the
            > envelope-sender as SMTP MAIL FROM (dspam wasn't designed to do that,

            dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix
            with SMT MAIL FROM. I have used this in the past and I'm sure others are
            using it now (ping steve?). something in your setup prevents this from
            working but I don' think it is a dspam limitation.

            > and
            > until I take the step to a better before-queue filter or something that
            > does, this will work, since all I needed was to capture the
            > envelope-sender).
            >
            > Limited testing shows this to work. There might be cases beyond what I
            > tested that will behave differently than I expect or very oddly.
            >
            > Thanks again!
          • Steve
            ... Hallo Jeff, Salut Mouss ... I hear you :) Have not followed the discussion. So I don t know exactly what the problem is. I will need to read the thread.
            Message 5 of 28 , Jan 2, 2009
            • 0 Attachment
              -------- Original-Nachricht --------
              > Datum: Fri, 02 Jan 2009 11:30:18 +0100
              > Von: mouss <mouss@...>
              > An: Jeff Weinberger <jeff@...>
              > CC: postfix-users@...
              > Betreff: Re: Finding the envelope-sender after always_bcc? (SOLVED)

              > Jeff Weinberger a écrit :
              > >
              Hallo Jeff, Salut Mouss


              > > I used a pcre: table for smtpd_sender_restrictions and the PREPEND
              > > action as follows:
              > >
              > > main.cf:
              > > smtpd_sender_restrictions = check_sender_access
              > > pcre:/etc/postfix/smtpd_sender_restrictions.pcre
              > >
              > > smtpd_sender_restrictions.pcre
              > > /^(.*)/ PREPEND X-Envelope-Sender: ${1}
              > >
              > > this will insert into every message a header "X-Envelope-Sender: "
              > > followed by the envelope sender value. It won't pass the
              > > envelope-sender as SMTP MAIL FROM (dspam wasn't designed to do that,
              >
              > dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix
              > with SMT MAIL FROM. I have used this in the past and I'm sure others are
              > using it now (ping steve?).
              >
              I hear you :) Have not followed the discussion. So I don't know exactly what the problem is. I will need to read the thread. However... I have the same setup as you mouss. I get the mail from Postfix into DSPAM with LMTP and then inject the message back to Postfix with SMTP. Has worked that way for the last years. Never had I any issue with that. In the beginning I was using pipe and then later SMTP for transporting mail to DSPAM but after having issues with pipe/smtp I switched to LMTP and have not had any issues since then.

              @Jeff: If you want, you could quickly address me directly with your DSPAM issue and save me some time to read the whole thread.


              > something in your setup prevents this from
              > working but I don' think it is a dspam limitation.
              >
              > > and
              > > until I take the step to a better before-queue filter or something that
              > > does, this will work, since all I needed was to capture the
              > > envelope-sender).
              > >
              > > Limited testing shows this to work. There might be cases beyond what I
              > > tested that will behave differently than I expect or very oddly.
              > >
              > > Thanks again!

              // Steve
              --
              Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
            • Jeff Weinberger
              ... It s definitely my set up. I don t use LMTP to pass the message to dspam, I use a transport called dspam that uses pipe. That means there s no S/LMTP
              Message 6 of 28 , Jan 2, 2009
              • 0 Attachment
                On Jan 2, 2009, at 2:30 AM, mouss wrote:

                > Jeff Weinberger a écrit :
                >>
                >> I used a pcre: table for smtpd_sender_restrictions and the PREPEND
                >> action as follows:
                >>
                >> main.cf:
                >> smtpd_sender_restrictions = check_sender_access
                >> pcre:/etc/postfix/smtpd_sender_restrictions.pcre
                >>
                >> smtpd_sender_restrictions.pcre
                >> /^(.*)/ PREPEND X-Envelope-Sender: ${1}
                >>
                >> this will insert into every message a header "X-Envelope-Sender: "
                >> followed by the envelope sender value. It won't pass the
                >> envelope-sender as SMTP MAIL FROM (dspam wasn't designed to do that,
                >
                > dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix
                > with SMT MAIL FROM. I have used this in the past and I'm sure others
                > are
                > using it now (ping steve?). something in your setup prevents this from
                > working but I don' think it is a dspam limitation.

                It's definitely my set up. I don't use LMTP to pass the message to
                dspam, I use a transport called "dspam" that uses pipe. That means
                there's no S/LMTP dialog, just the message itself passed as STDIN.

                I have to move dspam to use LMTP and then move it to a before-queue
                content filter so that this workaround becomes unnecessary, but until
                I go to make those changes, this will suffice.

                I'm not completely convinced that dspam will work seamlessly as a
                before-queue content filter, so I'll have to do some testing to see
                how well that works and whether it can do what I need and hand fully
                formed messages with SMTP dialog information back to postfix.

                Thank you!!

                >
                >
                >> and
                >> until I take the step to a better before-queue filter or something
                >> that
                >> does, this will work, since all I needed was to capture the
                >> envelope-sender).
                >>
                >> Limited testing shows this to work. There might be cases beyond
                >> what I
                >> tested that will behave differently than I expect or very oddly.
                >>
                >> Thanks again!
                >
                >
              • Wietse Venema
                Jeff Weinberger: [ Charset ISO-8859-1 unsupported, converting... ] ... If you can show what you ve configured to pass mail into dspam, then perhaps someone can
                Message 7 of 28 , Jan 2, 2009
                • 0 Attachment
                  Jeff Weinberger:
                  [ Charset ISO-8859-1 unsupported, converting... ]
                  > On Jan 2, 2009, at 2:30 AM, mouss wrote:
                  >
                  > > Jeff Weinberger a ?crit :
                  > >>
                  > >> I used a pcre: table for smtpd_sender_restrictions and the PREPEND
                  > >> action as follows:
                  > >>
                  > >> main.cf:
                  > >> smtpd_sender_restrictions = check_sender_access
                  > >> pcre:/etc/postfix/smtpd_sender_restrictions.pcre
                  > >>
                  > >> smtpd_sender_restrictions.pcre
                  > >> /^(.*)/ PREPEND X-Envelope-Sender: ${1}
                  > >>
                  > >> this will insert into every message a header "X-Envelope-Sender: "
                  > >> followed by the envelope sender value. It won't pass the
                  > >> envelope-sender as SMTP MAIL FROM (dspam wasn't designed to do that,
                  > >
                  > > dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix
                  > > with SMT MAIL FROM. I have used this in the past and I'm sure others
                  > > are
                  > > using it now (ping steve?). something in your setup prevents this from
                  > > working but I don' think it is a dspam limitation.
                  >
                  > It's definitely my set up. I don't use LMTP to pass the message to
                  > dspam, I use a transport called "dspam" that uses pipe. That means
                  > there's no S/LMTP dialog, just the message itself passed as STDIN.

                  If you can show what you've configured to pass mail into dspam,
                  then perhaps someone can tell you how to get that envelope sender
                  address into dspam, too.

                  Wietse
                • mouss
                  ... so _you_ are not passing the envelope sender to dspam. Consider running dspam in relay mode : postfix --(LMTP)-- dspam --(SMTP)-- postfix ... why do you
                  Message 8 of 28 , Jan 2, 2009
                  • 0 Attachment
                    Jeff Weinberger a écrit :
                    >
                    > It's definitely my set up. I don't use LMTP to pass the message to
                    > dspam, I use a transport called "dspam" that uses pipe. That means
                    > there's no S/LMTP dialog, just the message itself passed as STDIN.
                    >

                    so _you_ are not passing the envelope sender to dspam.

                    Consider running dspam in "relay mode":
                    postfix --(LMTP)--> dspam --(SMTP)--> postfix

                    > I have to move dspam to use LMTP and then move it to a before-queue

                    why do you want to run it in pre-queue mode? This is not needed and is
                    not simple to setup.

                    > content filter so that this workaround becomes unnecessary, but until I
                    > go to make those changes, this will suffice.
                    >
                    > I'm not completely convinced that dspam will work seamlessly as a
                    > before-queue content filter, so I'll have to do some testing to see how
                    > well that works and whether it can do what I need and hand fully formed
                    > messages with SMTP dialog information back to postfix.
                    >
                  • Jeff Weinberger
                    ... If I understand your diagram, then the content_filter would look like: content_filter=lmtp:unix:/path/to/dspam args and that might pass through the
                    Message 9 of 28 , Jan 2, 2009
                    • 0 Attachment
                      On Jan 2, 2009, at 9:20 AM, mouss wrote:

                      > Jeff Weinberger a écrit :
                      >>
                      >> It's definitely my set up. I don't use LMTP to pass the message to
                      >> dspam, I use a transport called "dspam" that uses pipe. That means
                      >> there's no S/LMTP dialog, just the message itself passed as STDIN.
                      >>
                      >
                      > so _you_ are not passing the envelope sender to dspam.
                      >
                      > Consider running dspam in "relay mode":
                      > postfix --(LMTP)--> dspam --(SMTP)--> postfix
                      >
                      >> I have to move dspam to use LMTP and then move it to a before-queue
                      >
                      > why do you want to run it in pre-queue mode? This is not needed and is
                      > not simple to setup.

                      If I understand your diagram, then the content_filter would look like:

                      content_filter=lmtp:unix:/path/to/dspam args

                      and that might pass through the envelope information (I'm not
                      convinced, but if dspam can do it, that would be how).

                      But since dspam can speak LMTP and SMTP why would an smtpd proxy be
                      hard to set up? It would certainly avoid the bcc issues, etc. that I"
                      experiences by having the message run through postfix twice. After
                      reading through SMTPD_PROXY_README, it seems like a bit of a challenge
                      to make it work, but not that hard...what do you think might be
                      difficult?

                      Thanks for all your help - over the course of thi dialog I've learned
                      a lot about postfix and have become more aware of and proficient with
                      parts I knew little about. This has been very helpful.

                      >
                      >
                      >> content filter so that this workaround becomes unnecessary, but
                      >> until I
                      >> go to make those changes, this will suffice.
                      >>
                      >> I'm not completely convinced that dspam will work seamlessly as a
                      >> before-queue content filter, so I'll have to do some testing to see
                      >> how
                      >> well that works and whether it can do what I need and hand fully
                      >> formed
                      >> messages with SMTP dialog information back to postfix.
                      >>
                      >
                    • Jeff Weinberger
                      ... I thought I had a while back, but I m not sure it matters. Right now I am passing mail to dspam via pipe, which means dspam does not know how to handle the
                      Message 10 of 28 , Jan 2, 2009
                      • 0 Attachment
                        Wietse Venema wrote:
                        Jeff Weinberger:
                        [ Charset ISO-8859-1 unsupported, converting.. . ]
                        > On Jan 2, 2009, at 2:30 AM, mouss wrote:

                        > > Jeff Weinberger a ?crit :
                        > >>
                        > >> I used a pcre: table for smtpd_sender_ restrictions and the PREPEND
                        > >> action as follows:
                        > >>
                        > >> main.cf:
                        > >> smtpd_sender_ restrictions = check_sender_ access
                        > >> pcre:/etc/postfix/ smtpd_sender_ restrictions. pcre
                        > >>
                        > >> smtpd_sender_ restrictions. pcre
                        > >> /^(.*)/ PREPEND X-Envelope-Sender: ${1}
                        > >>
                        > >> this will insert into every message a header "X-Envelope- Sender: "
                        > >> followed by the envelope sender value. It won't pass the
                        > >> envelope-sender as SMTP MAIL FROM (dspam wasn't designed to do that,
                        > >
                        > > dspam gets it via LMTP MAIL FROM, and it will pass it back to postfix
                        > > with SMT MAIL FROM. I have used this in the past and I'm sure others 
                        > > are
                        > > using it now (ping steve?). something in your setup prevents this from
                        > > working but I don' think it is a dspam limitation.

                        > It's definitely my set up. I don't use LMTP to pass the message to 
                        > dspam, I use a transport called "dspam" that uses pipe. That means 
                        > there's no S/LMTP dialog, just the message itself passed as STDIN.

                        If you can show what you've configured to pass mail into dspam,
                        then perhaps someone can tell you how to get that envelope sender
                        address into dspam, too.

                        Wietse

                        I thought I had a while back, but I"m not sure it matters. Right now I am passing mail to dspam via pipe, which means dspam does not know how to handle the envelope-sender as a command-line argument. I need to change this to passing mail to dspam via LMTP.

                        I don't know how exactly, but I hope with some help, I can make it work!

                        That said, here's the current configuration:

                             content_filter=dspam:dspam

                        and in master.cf:

                             dspam     unix  -       n       n       -       10      pipe
                                flags=Ru user=_dspam argv=/usr/local/bin/dspam --deliver=innocent --user ${recipient} -i - f $sender -- $recipient

                        dspam is running as a daemon .and re-injects the mail as SMTP to an alternate port.


                        I assume the LMTP setup should look something like:

                        main.cf:

                             content_filter=lmtp:unix:/path/to/dspam  --deliver=innocent --user ${recipient} -i - f $sender -- $recipient

                        and the transport in master.cf becomes unnecessary.

                        I don't know how to handle the command line arguments (or even if I have to), so suggestions, help and especially examples would be most helpful.

                        Thank you!




                      • mouss
                        ... Is there a reason why you keep adding yahoo groups after I remove them fro CC? This is starting to annoy me... and by the way, disable the X-DSPAM-Factors
                        Message 11 of 28 , Jan 2, 2009
                        • 0 Attachment
                          Jeff Weinberger a écrit :
                          > On Jan 2, 2009, at 9:20 AM, mouss wrote:
                          >
                          >> Jeff Weinberger a écrit :
                          >>>
                          >>> It's definitely my set up. I don't use LMTP to pass the message to
                          >>> dspam, I use a transport called "dspam" that uses pipe. That means
                          >>> there's no S/LMTP dialog, just the message itself passed as STDIN.
                          >>>
                          >>
                          >> so _you_ are not passing the envelope sender to dspam.
                          >>
                          >> Consider running dspam in "relay mode":
                          >> postfix --(LMTP)--> dspam --(SMTP)--> postfix
                          >>
                          >>> I have to move dspam to use LMTP and then move it to a before-queue
                          >>
                          >> why do you want to run it in pre-queue mode? This is not needed and is
                          >> not simple to setup.
                          >

                          Is there a reason why you keep adding yahoo groups after I remove them
                          fro CC? This is starting to annoy me...

                          and by the way, disable the X-DSPAM-Factors header. dspam doesn't encode
                          it, which results in things like:

                          X-DSPAM-Factors: 27,
                          ...
                          a+écrit, 0.01000,

                          and this is not a valid header.

                          > If I understand your diagram, then the content_filter would look like:
                          >
                          > content_filter=lmtp:unix:/path/to/dspam args

                          No.
                          content_filter=lmtp:inet:127.0.0.1:10024

                          where the 10024 is the same port used in dspam.conf:
                          ServerPort 10024

                          of course, dspam must be running in daemon mode.

                          >
                          > and that might pass through the envelope information (I'm not convinced,
                          > but if dspam can do it, that would be how).

                          LMTP is similar to SMTP, and dspam can run as an LMTP server (this is
                          configured in dspam.conf).

                          >
                          > But since dspam can speak LMTP and SMTP

                          dspam has a server and a client. so which speaks what?

                          if we are talking about the server part:

                          $ cd dspam-3.8.0/src
                          $ cat daemon.c
                          ...
                          input = daemon_expect(TTX, "LHLO");
                          if (input == NULL)
                          goto CLOSE;
                          ...

                          it wants LHLO (which is for LMTP), not HELO or EHLO. so no smtp there.


                          > why would an smtpd proxy be hard
                          > to set up? It would certainly avoid the bcc issues, etc.

                          I don't understand why you mix pre-queue and bcc. maybe you confuse
                          pre-queue with a "not simple content filter"?

                          > that I"
                          > experiences by having the message run through postfix twice. After
                          > reading through SMTPD_PROXY_README, it seems like a bit of a challenge
                          > to make it work, but not that hard...what do you think might be difficult?
                          >



                          > Thanks for all your help - over the course of thi dialog I've learned a
                          > lot about postfix and have become more aware of and proficient with
                          > parts I knew little about. This has been very helpful.
                          >
                          >>
                          >>
                          >>> content filter so that this workaround becomes unnecessary, but until I
                          >>> go to make those changes, this will suffice.
                          >>>
                          >>> I'm not completely convinced that dspam will work seamlessly as a
                          >>> before-queue content filter, so I'll have to do some testing to see how
                          >>> well that works and whether it can do what I need and hand fully formed
                          >>> messages with SMTP dialog information back to postfix.
                          >>>
                          >>
                          >
                          >
                        • Wietse Venema
                          ... Hmm, so dspam is getting the envelope recipent. ... With lmtp, the information is NOT passed on the command line but instead is passed via the LMTP
                          Message 12 of 28 , Jan 2, 2009
                          • 0 Attachment
                            Jeff Weinberger:
                            > That said, here's the current configuration:
                            >
                            > content_filter=dspam:dspam
                            >
                            > and in master.cf:
                            >
                            > dspam unix - n n - 10 pipe
                            > flags=Ru user=_dspam argv=/usr/local/bin/dspam --
                            > deliver=innocent --user ${recipient} -i - f $sender -- $recipient
                            >

                            Hmm, so dspam is getting the envelope recipent.

                            > I assume the LMTP setup should look something like:
                            >
                            > main.cf:
                            >
                            > content_filter=lmtp:unix:/path/to/dspam --deliver=innocent --
                            > user ${recipient} -i - f $sender -- $recipient
                            >
                            > and the transport in master.cf becomes unnecessary.
                            >
                            > I don't know how to handle the command line arguments (or even if I
                            > have to), so suggestions, help and especially examples would be most
                            > helpful.

                            With lmtp, the information is NOT passed on the command line
                            but instead is passed via the LMTP protocol.

                            Wietse
                          • Victor Duchovni
                            ... Why is there a space between - and f in the above recipe? -- Viktor. Disclaimer: off-list followups get on-list replies or get ignored. Please do not
                            Message 13 of 28 , Jan 2, 2009
                            • 0 Attachment
                              On Fri, Jan 02, 2009 at 05:20:16PM -0500, Wietse Venema wrote:

                              > Jeff Weinberger:
                              > > That said, here's the current configuration:
                              > >
                              > > content_filter=dspam:dspam
                              > >
                              > > and in master.cf:
                              > >
                              > > dspam unix - n n - 10 pipe
                              > > flags=Ru user=_dspam argv=/usr/local/bin/dspam --
                              > > deliver=innocent --user ${recipient} -i - f $sender -- $recipient
                              > >
                              >
                              > Hmm, so dspam is getting the envelope recipent.

                              Why is there a space between "-" and "f" in the above recipe?

                              --
                              Viktor.

                              Disclaimer: off-list followups get on-list replies or get ignored.
                              Please do not ignore the "Reply-To" header.

                              To unsubscribe from the postfix-users list, visit
                              http://www.postfix.org/lists.html or click the link below:
                              <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                              If my response solves your problem, the best way to thank me is to not
                              send an "it worked, thanks" follow-up. If you must respond, please put
                              "It worked, thanks" in the "Subject" so I can delete these quickly.
                            • Jeff Weinberger
                              ... Sorry!! I was having problems with my messages not posting. I will stop adding :) ... OK thanks! ... dspam is running in daemon mode. This makes sense as a
                              Message 14 of 28 , Jan 2, 2009
                              • 0 Attachment
                                On Jan 2, 2009, at 2:17 PM, mouss wrote:

                                > Jeff Weinberger a écrit :
                                >> On Jan 2, 2009, at 9:20 AM, mouss wrote:
                                >>
                                >>> Jeff Weinberger a écrit :
                                >>>>
                                >>>> It's definitely my set up. I don't use LMTP to pass the message to
                                >>>> dspam, I use a transport called "dspam" that uses pipe. That means
                                >>>> there's no S/LMTP dialog, just the message itself passed as STDIN.
                                >>>>
                                >>>
                                >>> so _you_ are not passing the envelope sender to dspam.
                                >>>
                                >>> Consider running dspam in "relay mode":
                                >>> postfix --(LMTP)--> dspam --(SMTP)--> postfix
                                >>>
                                >>>> I have to move dspam to use LMTP and then move it to a before-queue
                                >>>
                                >>> why do you want to run it in pre-queue mode? This is not needed
                                >>> and is
                                >>> not simple to setup.
                                >>
                                >
                                > Is there a reason why you keep adding yahoo groups after I remove them
                                > fro CC? This is starting to annoy me...

                                Sorry!! I was having problems with my messages not posting. I will
                                stop adding :)
                                >
                                >
                                > and by the way, disable the X-DSPAM-Factors header. dspam doesn't
                                > encode
                                > it, which results in things like:
                                >
                                > X-DSPAM-Factors: 27,
                                > ...
                                > a+écrit, 0.01000,
                                >
                                > and this is not a valid header.

                                OK thanks!

                                >
                                >
                                >> If I understand your diagram, then the content_filter would look
                                >> like:
                                >>
                                >> content_filter=lmtp:unix:/path/to/dspam args
                                >
                                > No.
                                > content_filter=lmtp:inet:127.0.0.1:10024
                                >
                                > where the 10024 is the same port used in dspam.conf:
                                > ServerPort 10024
                                >
                                > of course, dspam must be running in daemon mode.

                                dspam is running in daemon mode. This makes sense as a setup.

                                The example in the dspam docs for postfix shows

                                content_filter=lmtp:unix:/path/to/dspam.sock

                                which is why I thought unix: instead of inet:

                                is there any difference, other than performance?

                                On a related question (if more broad): some content_filter examples I
                                see use content_filter in main.cf and some as "-o content_filter=..."
                                in master.cf.

                                I understand from prior conversations here that you can't override
                                content filters, they are global. (yes?)

                                So is there an advantage/disadvantage to specifying the content filter
                                in main.cf vs. master.cf?

                                >
                                >
                                >>
                                >> and that might pass through the envelope information (I'm not
                                >> convinced,
                                >> but if dspam can do it, that would be how).
                                >
                                > LMTP is similar to SMTP, and dspam can run as an LMTP server (this is
                                > configured in dspam.conf).
                                >
                                >>
                                >> But since dspam can speak LMTP and SMTP
                                >
                                > dspam has a server and a client. so which speaks what?
                                >
                                > if we are talking about the server part:
                                >
                                > $ cd dspam-3.8.0/src
                                > $ cat daemon.c
                                > ..
                                > input = daemon_expect(TTX, "LHLO");
                                > if (input == NULL)
                                > goto CLOSE;
                                > ..
                                >
                                > it wants LHLO (which is for LMTP), not HELO or EHLO. so no smtp there.
                                >
                                >
                                >> why would an smtpd proxy be hard
                                >> to set up? It would certainly avoid the bcc issues, etc.
                                >
                                > I don't understand why you mix pre-queue and bcc. maybe you confuse
                                > pre-queue with a "not simple content filter"?

                                I am new to pre-queue filtering and am clearly still confused :)

                                you had asked:

                                >>> why do you want to run it in pre-queue mode? This is not needed
                                >>> and is
                                >>> not simple to setu

                                so I thought it might be hard.

                                It's not important for now...getting dspam working as LMTP will be
                                fine (assuming it passes along all the right information once it works)

                                Thank you!!

                                >
                                >
                                >> that I"
                                >> experiences by having the message run through postfix twice. After
                                >> reading through SMTPD_PROXY_README, it seems like a bit of a
                                >> challenge
                                >> to make it work, but not that hard...what do you think might be
                                >> difficult?
                                >>
                                >
                                >
                                >
                                >> Thanks for all your help - over the course of thi dialog I've
                                >> learned a
                                >> lot about postfix and have become more aware of and proficient with
                                >> parts I knew little about. This has been very helpful.
                                >>
                                >>>
                                >>>
                                >>>> content filter so that this workaround becomes unnecessary, but
                                >>>> until I
                                >>>> go to make those changes, this will suffice.
                                >>>>
                                >>>> I'm not completely convinced that dspam will work seamlessly as a
                                >>>> before-queue content filter, so I'll have to do some testing to
                                >>>> see how
                                >>>> well that works and whether it can do what I need and hand fully
                                >>>> formed
                                >>>> messages with SMTP dialog information back to postfix.
                                >>>>
                                >>>
                                >>
                                >>
                                >
                                >
                              • mouss
                                ... it s ok to use a unix socket, but it s a socket, not the dspam binary with args. ... I wouldn t put performances into the equation without measurements
                                Message 15 of 28 , Jan 2, 2009
                                • 0 Attachment
                                  Jeff Weinberger a écrit :

                                  >>> content_filter=lmtp:unix:/path/to/dspam args
                                  >>
                                  >> No.
                                  >> content_filter=lmtp:inet:127.0.0.1:10024
                                  >>
                                  >> where the 10024 is the same port used in dspam.conf:
                                  >> ServerPort 10024
                                  >>
                                  >> of course, dspam must be running in daemon mode.
                                  >
                                  > dspam is running in daemon mode. This makes sense as a setup.
                                  >
                                  > The example in the dspam docs for postfix shows
                                  >
                                  > content_filter=lmtp:unix:/path/to/dspam.sock
                                  >
                                  > which is why I thought unix: instead of inet:
                                  >

                                  it's ok to use a unix socket, but it's a socket, not the dspam binary
                                  with args.

                                  > is there any difference, other than performance?
                                  >

                                  I wouldn't put performances into the equation without measurements (one
                                  should tune where the bottleneck is, not in every small piece).

                                  if you use a unix socket, you'll have to make sure it is accessible. In
                                  particular, if someday you decide to chroot postfix, you'll need to put
                                  the unix socket in the chroot jail.

                                  > On a related question (if more broad): some content_filter examples I
                                  > see use content_filter in main.cf and some as "-o content_filter=..." in
                                  > master.cf.
                                  >
                                  > I understand from prior conversations here that you can't override
                                  > content filters, they are global. (yes?)
                                  >

                                  no, they are not global. each smtpd may have its own content_filter.
                                  (don't confuse with: "one message, one filter". The latter simply means
                                  that postfix won't split a single message into one message per recipient
                                  before passing it to the content filter).

                                  > So is there an advantage/disadvantage to specifying the content filter
                                  > in main.cf vs. master.cf?
                                  >

                                  I guess an example is better than literature, no?

                                  here is a "not uncommon setup":
                                  - port 25 is used for "MX" mail (aka inbound mail). it uses the
                                  content_filter defined in main.cf

                                  - port 587 is used for "submission" (authenticated, ...). such mail is
                                  scanned for viruses but not for spam (there's not much things a bayesian
                                  filter could do here, except in simple setups with a site-wide bayes).
                                  so -o is used to set the filter for this service

                                  - sendmail mail is not filtered, because we "trust" the box (there's no
                                  user, ... etc) and we don't want anything blocking such mail. or we use
                                  sendmail to reinject mail after filtering, so we don't want to create a
                                  loop. for this, we set "-o content_filter=".


                                  >
                                  > I am new to pre-queue filtering and am clearly still confused :)
                                  >

                                  - content_filter refers to after-the-queue filtering. This means postfix
                                  saves the mail on disk, says "ok" to the client, and sometimes after
                                  that, postfix passes the message to the filter

                                  - proxy_filter refers to pre-queue: postfix keeps the client connected,
                                  passed the message to the filter and waits for the filter response
                                  before responding to the client. This assumes SMTP, because the message
                                  was received via SMTP, so the client wants a single response for the
                                  whole message, and not one response per recipient.

                                  the pipe method that you were using is an example of a content_filter,
                                  and is referred to as a "simple filter" in postfix docs. but a
                                  content_filter need not be a pipe.


                                  > you had asked:
                                  >
                                  >>>> why do you want to run it in pre-queue mode? This is not needed and is
                                  >>>> not simple to setu
                                  >
                                  > so I thought it might be hard.
                                  >

                                  and it is. reread what I said about dspam and lmtp...

                                  > It's not important for now...getting dspam working as LMTP will be fine
                                  > (assuming it passes along all the right information once it works)
                                  >
                                • Jeff Weinberger
                                  ... OK, thanks. I will set up dspam to listen on port 10024 - seems to make the most sense. I don t need a localhost:10024 entry in master.cf then? right? ...
                                  Message 16 of 28 , Jan 2, 2009
                                  • 0 Attachment
                                    On Jan 2, 2009, at 3:20 PM, mouss wrote:

                                    > Jeff Weinberger a écrit :
                                    >
                                    >>>> content_filter=lmtp:unix:/path/to/dspam args
                                    >>>
                                    >>> No.
                                    >>> content_filter=lmtp:inet:127.0.0.1:10024
                                    >>>
                                    >>> where the 10024 is the same port used in dspam.conf:
                                    >>> ServerPort 10024
                                    >>>
                                    >>> of course, dspam must be running in daemon mode.
                                    >>
                                    >> dspam is running in daemon mode. This makes sense as a setup.
                                    >>
                                    >> The example in the dspam docs for postfix shows
                                    >>
                                    >> content_filter=lmtp:unix:/path/to/dspam.sock
                                    >>
                                    >> which is why I thought unix: instead of inet:
                                    >>
                                    >
                                    > it's ok to use a unix socket, but it's a socket, not the dspam binary
                                    > with args.

                                    OK, thanks. I will set up dspam to listen on port 10024 - seems to
                                    make the most sense. I don't need a localhost:10024 entry in master.cf
                                    then? right?

                                    >
                                    >
                                    >> is there any difference, other than performance?
                                    >>
                                    >
                                    > I wouldn't put performances into the equation without measurements
                                    > (one
                                    > should tune where the bottleneck is, not in every small piece).
                                    >
                                    > if you use a unix socket, you'll have to make sure it is accessible.
                                    > In
                                    > particular, if someday you decide to chroot postfix, you'll need to
                                    > put
                                    > the unix socket in the chroot jail.

                                    exactly. Thanks

                                    >
                                    >
                                    >> On a related question (if more broad): some content_filter examples I
                                    >> see use content_filter in main.cf and some as "-o
                                    >> content_filter=..." in
                                    >> master.cf.
                                    >>
                                    >> I understand from prior conversations here that you can't override
                                    >> content filters, they are global. (yes?)
                                    >>
                                    >
                                    > no, they are not global. each smtpd may have its own content_filter.
                                    > (don't confuse with: "one message, one filter". The latter simply
                                    > means
                                    > that postfix won't split a single message into one message per
                                    > recipient
                                    > before passing it to the content filter).

                                    That makes sense. thank you.

                                    >
                                    >
                                    >> So is there an advantage/disadvantage to specifying the content
                                    >> filter
                                    >> in main.cf vs. master.cf?
                                    >>
                                    >
                                    > I guess an example is better than literature, no?
                                    >
                                    > here is a "not uncommon setup":
                                    > - port 25 is used for "MX" mail (aka inbound mail). it uses the
                                    > content_filter defined in main.cf

                                    right, as I do right now.

                                    >
                                    >
                                    > - port 587 is used for "submission" (authenticated, ...). such mail is
                                    > scanned for viruses but not for spam (there's not much things a
                                    > bayesian
                                    > filter could do here, except in simple setups with a site-wide bayes).
                                    > so -o is used to set the filter for this service

                                    I need to set this up also - seems easy, but is there an example of
                                    the localhost:587 master.cf entry somewhere I could start with?

                                    >
                                    >
                                    > - sendmail mail is not filtered, because we "trust" the box (there's
                                    > no
                                    > user, ... etc) and we don't want anything blocking such mail. or we
                                    > use
                                    > sendmail to reinject mail after filtering, so we don't want to
                                    > create a
                                    > loop. for this, we set "-o content_filter=".
                                    >

                                    do you mean the re-injection into postfix? I have "-o content_filter="
                                    there already. I'm re-injecting mail via SMTP, not sendmail...

                                    >
                                    >>
                                    >> I am new to pre-queue filtering and am clearly still confused :)
                                    >>
                                    >
                                    > - content_filter refers to after-the-queue filtering. This means
                                    > postfix
                                    > saves the mail on disk, says "ok" to the client, and sometimes after
                                    > that, postfix passes the message to the filter
                                    >
                                    > - proxy_filter refers to pre-queue: postfix keeps the client
                                    > connected,
                                    > passed the message to the filter and waits for the filter response
                                    > before responding to the client. This assumes SMTP, because the
                                    > message
                                    > was received via SMTP, so the client wants a single response for the
                                    > whole message, and not one response per recipient.
                                    >
                                    > the pipe method that you were using is an example of a content_filter,
                                    > and is referred to as a "simple filter" in postfix docs. but a
                                    > content_filter need not be a pipe.
                                    >
                                    >
                                    >> you had asked:
                                    >>
                                    >>>>> why do you want to run it in pre-queue mode? This is not needed
                                    >>>>> and is
                                    >>>>> not simple to setu
                                    >>
                                    >> so I thought it might be hard.
                                    >>
                                    >
                                    > and it is. reread what I said about dspam and lmtp...

                                    Thank you!


                                    >
                                    >
                                    >> It's not important for now...getting dspam working as LMTP will be
                                    >> fine
                                    >> (assuming it passes along all the right information once it works)
                                    >>
                                    >
                                  • mouss
                                    ... no, 10024 will be used by dspam. your postfix should have a 127.0.0.1:10025 to get mail back. ... your master.cf should already contain a submission
                                    Message 17 of 28 , Jan 2, 2009
                                    • 0 Attachment
                                      Jeff Weinberger a écrit :
                                      >
                                      > OK, thanks. I will set up dspam to listen on port 10024 - seems to make
                                      > the most sense. I don't need a localhost:10024 entry in master.cf then?
                                      > right?
                                      >

                                      no, 10024 will be used by dspam. your postfix should have a
                                      127.0.0.1:10025 to get mail back.


                                      >>
                                      >>> So is there an advantage/disadvantage to specifying the content filter
                                      >>> in main.cf vs. master.cf?
                                      >>>
                                      >>
                                      >> I guess an example is better than literature, no?
                                      >>
                                      >> here is a "not uncommon setup":
                                      >> - port 25 is used for "MX" mail (aka inbound mail). it uses the
                                      >> content_filter defined in main.cf
                                      >
                                      > right, as I do right now.
                                      >
                                      >>
                                      >>
                                      >> - port 587 is used for "submission" (authenticated, ...). such mail is
                                      >> scanned for viruses but not for spam (there's not much things a bayesian
                                      >> filter could do here, except in simple setups with a site-wide bayes).
                                      >> so -o is used to set the filter for this service
                                      >
                                      > I need to set this up also - seems easy, but is there an example of the
                                      > localhost:587 master.cf entry somewhere I could start with?
                                      >

                                      your master.cf should already contain a "submission" service (it's
                                      commented out by default). you can add -o conten_filter and other
                                      parameters.


                                      >>
                                      >> - sendmail mail is not filtered, because we "trust" the box (there's no
                                      >> user, ... etc) and we don't want anything blocking such mail. or we use
                                      >> sendmail to reinject mail after filtering, so we don't want to create a
                                      >> loop. for this, we set "-o content_filter=".
                                      >>
                                      >
                                      > do you mean the re-injection into postfix? I have "-o content_filter="
                                      > there already. I'm re-injecting mail via SMTP, not sendmail...
                                      >

                                      so you already had an example that overrides the content_filter in
                                      master.cf ;-)
                                    • Jeff Weinberger
                                      ... perfect thanks! ... it didn t - but that s probably because apple modified the example file and took out the submission entry. I found it in the
                                      Message 18 of 28 , Jan 2, 2009
                                      • 0 Attachment
                                        On Jan 2, 2009, at 4:03 PM, mouss wrote:

                                        > Jeff Weinberger a écrit :
                                        >>
                                        >> OK, thanks. I will set up dspam to listen on port 10024 - seems to
                                        >> make
                                        >> the most sense. I don't need a localhost:10024 entry in master.cf
                                        >> then?
                                        >> right?
                                        >>
                                        >
                                        > no, 10024 will be used by dspam. your postfix should have a
                                        > 127.0.0.1:10025 to get mail back.

                                        perfect thanks!

                                        >
                                        >
                                        >
                                        >>>
                                        >>>> So is there an advantage/disadvantage to specifying the content
                                        >>>> filter
                                        >>>> in main.cf vs. master.cf?
                                        >>>>
                                        >>>
                                        >>> I guess an example is better than literature, no?
                                        >>>
                                        >>> here is a "not uncommon setup":
                                        >>> - port 25 is used for "MX" mail (aka inbound mail). it uses the
                                        >>> content_filter defined in main.cf
                                        >>
                                        >> right, as I do right now.
                                        >>
                                        >>>
                                        >>>
                                        >>> - port 587 is used for "submission" (authenticated, ...). such
                                        >>> mail is
                                        >>> scanned for viruses but not for spam (there's not much things a
                                        >>> bayesian
                                        >>> filter could do here, except in simple setups with a site-wide
                                        >>> bayes).
                                        >>> so -o is used to set the filter for this service
                                        >>
                                        >> I need to set this up also - seems easy, but is there an example of
                                        >> the
                                        >> localhost:587 master.cf entry somewhere I could start with?
                                        >>
                                        >
                                        > your master.cf should already contain a "submission" service (it's
                                        > commented out by default). you can add -o conten_filter and other
                                        > parameters.

                                        it didn't - but that's probably because apple modified the example
                                        file and took out the submission entry. I found it in the distribution.

                                        I'm guessing I should leave my port 25 (smtp) entry to allow
                                        submission on that port with authentication (leaving
                                        "smtpd_client_restrictions=permit_sasl_authenticated...") - is there
                                        any harm in that?

                                        >
                                        >
                                        >
                                        >>>
                                        >>> - sendmail mail is not filtered, because we "trust" the box
                                        >>> (there's no
                                        >>> user, ... etc) and we don't want anything blocking such mail. or
                                        >>> we use
                                        >>> sendmail to reinject mail after filtering, so we don't want to
                                        >>> create a
                                        >>> loop. for this, we set "-o content_filter=".
                                        >>>
                                        >>
                                        >> do you mean the re-injection into postfix? I have "-o
                                        >> content_filter="
                                        >> there already. I'm re-injecting mail via SMTP, not sendmail...
                                        >>
                                        >
                                        > so you already had an example that overrides the content_filter in
                                        > master.cf ;-)
                                        >

                                        I see that I did... oops ;)

                                        >
                                      • mouss
                                        ... no. but separating roles makes configuration easier. so encourage users to use 587.
                                        Message 19 of 28 , Jan 2, 2009
                                        • 0 Attachment
                                          Jeff Weinberger a écrit :
                                          > I'm guessing I should leave my port 25 (smtp) entry to allow submission
                                          > on that port with authentication (leaving
                                          > "smtpd_client_restrictions=permit_sasl_authenticated...") - is there any
                                          > harm in that?
                                          >

                                          no. but separating roles makes configuration easier. so encourage users
                                          to use 587.
                                        • Jeff Weinberger
                                          ... ok, thanks - that makes sense and it seems to be working fine for me. dspam isn t working fine accepting messages via LMTP, so I m taking that over to the
                                          Message 20 of 28 , Jan 2, 2009
                                          • 0 Attachment
                                            On Jan 2, 2009, at 5:32 PM, mouss wrote:

                                            > Jeff Weinberger a écrit :
                                            >> I'm guessing I should leave my port 25 (smtp) entry to allow
                                            >> submission
                                            >> on that port with authentication (leaving
                                            >> "smtpd_client_restrictions=permit_sasl_authenticated...") - is
                                            >> there any
                                            >> harm in that?
                                            >>
                                            >
                                            > no. but separating roles makes configuration easier. so encourage
                                            > users
                                            > to use 587.
                                            >
                                            >

                                            ok, thanks - that makes sense and it seems to be working fine for me.

                                            dspam isn't working fine accepting messages via LMTP, so I'm taking
                                            that over to the dspam list

                                            Thank you for your help!
                                          Your message has been successfully submitted and would be delivered to recipients shortly.