Loading ...
Sorry, an error occurred while loading the content.

sender_bcc_maps, virtual_alias_maps, no_address_mappings

Expand Messages
  • Rich Wales
    I m running Postfix 2.3.5 and amavisd-new 2.5.0 on a FreeBSD 6.2 system at my home. My current setup has been running very stably for a long time. Recently, I
    Message 1 of 13 , Sep 26, 2007
    • 0 Attachment
      I'm running Postfix 2.3.5 and amavisd-new 2.5.0 on a FreeBSD 6.2 system
      at my home. My current setup has been running very stably for a long
      time.

      Recently, I decided to try supporting a new domain via virtual aliases.
      However, I seem to be having a conflict between "virtual_alias_maps" and
      "sender_bcc_maps" -- I can't get both of these features to work properly
      as part of the same configuration. (I'm using sender_bcc_maps to create
      automatic copies of selected users' mail.)

      When I first tried adding virtual_alias_maps info to my configuration,
      the problem I saw was that messages for the virtual-alias domain were
      accepted (via SMTP) and queued, but they would stay stuck in the queue
      with "User unknown in virtual alias table" errors. (Note that I had
      turned on "soft_bounce" in my configuration until I knew everything was
      working correctly after adding the new feature.)

      I went through my master.cf and discovered that if I modified the
      "receive_override_options" parameter in my post-Amavis listener daemon
      on port 10025 -- by removing "no_address_mappings" -- the virtual alias
      mappings started working again. This seemed to make sense, since the
      documentation says that "no_address_mappings" is typically specified
      BEFORE (not after) an external content filter. Indeed, I wondered why
      I had ever included no_address_mappings in the receive_override_options
      for the post-Amavis listener in the first place.

      Later, however, I discovered that when I sent out mail, I was getting
      two (or even more) copies back, rather than just one copy (as I had
      previously been getting via sender_bcc_maps). I reverted my change to
      master.cf (by putting "no_address_mappings" back into the port-10025
      listener options), and the extraneous multiple copies stopped -- though
      at the expense of the virtual aliases, which are once again getting
      stuck in the queue with "user unknown in virtual alias table" errors.

      So, what I believe I need to do is figure out what settings I need to
      use so that BOTH virtual_alias_maps AND sender_bcc_maps will work. I
      only want the "sender BCC" processing to happen once -- and, I believe,
      as late as possible in the overall mail handling process -- but I also
      need the "virtual alias" processing to happen AT LEAST once.

      I'm confused, BTW, as to why removing "no_address_mappings" from the
      receive_override_options in the port-10025 listener should have caused
      multiple copies of my mail to be sent back to myself -- given that I'm
      also explicitly disabling "sender_bcc_maps" for the service on port
      10025. Is it possible that what's broken is something else entirely,
      and not the "sender BCC" processing?

      I'm attaching the output of "postconf -n", as well as my "master.cf"
      file (the version where sender_bcc_maps work but virtual_alias_maps
      doesn't work).

      --
      Rich Wales === Palo Alto, CA, USA === richw@...
      http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
      "The difference between theory and practice is that, in theory,
      theory and practice are identical -- whereas in practice, they aren't."
    • Rich Wales
      Hi. Regarding the problem I asked about yesterday (trying to get virtual aliases to work, but I started getting extra copies of mail when I removed
      Message 2 of 13 , Sep 27, 2007
      • 0 Attachment
        Hi. Regarding the problem I asked about yesterday (trying to get
        virtual aliases to work, but I started getting extra copies of mail
        when I removed "no_address_mappings" from the "receive_override_options"
        in the post-amavisd-new SMTP listener on port 10025:

        Since no one has ventured any guesses yet as to what was going on, let
        me try thinking along slightly different lines.

        Perhaps my real question should be, Why do I need "no_address_mappings"
        specified for the port-10025 listener, in order to stop duplicate
        message delivery? (The amavisd-new how-to material I've found doesn't
        say to use "no_address_mappings" on the port-10025 listener, but the
        facts on the ground are that -- whether it makes sense or not -- I find
        myself needing to use this parameter in this spot, or else I start
        getting duplicates.)

        Again, I'm attaching the results of "postconf -n", plus my master.cf
        (the version that has the seemingly extraneous, but in fact needed,
        "no_address_mappings" parameter as part of the port-10025 listener
        configuration).

        Any comments welcome -- including suggestions of additional tests I
        should run, or additional output which I should provide in order to
        help any interested person to debug this problem.

        --
        Rich Wales === Palo Alto, CA, USA === richw@...
        http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
        "The difference between theory and practice is that, in theory,
        theory and practice are identical -- whereas in practice, they aren't."
      • Noel Jones
        ... When you get no responses, usually either no one understood what you were talking about or your assumptions are so wrong that it will take a huge effort to
        Message 3 of 13 , Sep 28, 2007
        • 0 Attachment
          At 12:24 AM 9/28/2007, Rich Wales wrote:
          >Hi. Regarding the problem I asked about yesterday (trying to get
          >virtual aliases to work, but I started getting extra copies of mail
          >when I removed "no_address_mappings" from the "receive_override_options"
          >in the post-amavisd-new SMTP listener on port 10025:
          >
          >Since no one has ventured any guesses yet as to what was going on, let
          >me try thinking along slightly different lines.

          When you get no responses, usually either no one understood what you
          were talking about or your assumptions are so wrong that it will take
          a huge effort to get you back on track. Or maybe the question was
          simply overlooked. I don't remember which...

          >Perhaps my real question should be, Why do I need "no_address_mappings"
          >specified for the port-10025 listener, in order to stop duplicate
          >message delivery?

          Because that's what no_address_mappings is for. When you use a
          content_filter such as amavisd-new mail passes through postfix
          twice. If postfix is configured to do address expansion on both
          passes, then mail will be duplicated. Hence the introduction of
          no_address_mappings to conveniently disable either the pre-filter or
          post-filter address expansion. (Prior to no_address_mappings, one
          was forced to use a less convenient and even more confusing method to
          disable address expansion).

          >(The amavisd-new how-to material I've found doesn't
          >say to use "no_address_mappings" on the port-10025 listener, but the
          >facts on the ground are that -- whether it makes sense or not -- I find
          >myself needing to use this parameter in this spot, or else I start
          >getting duplicates.)

          You're looking at very old how-to's that predate the introduction of
          no_address_mappings.

          >Again, I'm attaching the results of "postconf -n", plus my master.cf
          >(the version that has the seemingly extraneous, but in fact needed,
          >"no_address_mappings" parameter as part of the port-10025 listener
          >configuration).

          See the README.postfix included with amavisd-new, and stop referring
          to outdated information.
          http://www.ijs.si/software/amavisd/README.postfix.html

          --
          Noel Jones
        • Rich Wales
          ... OK, thanks. Right now, I ve managed to get a virtual alias domain to work IF the message is originally submitted to my server -- but NOT if the message
          Message 4 of 13 , Sep 28, 2007
          • 0 Attachment
            Noel Jones wrote:

            > You're looking at very old how-to's that predate the introduction
            > of no_address_mappings. . . . See the README.postfix included with
            > amavisd-new, and stop referring to outdated information.

            OK, thanks.

            Right now, I've managed to get a virtual alias domain to work IF the
            message is originally submitted to my server -- but NOT if the message
            comes in via the regular SMTP server.

            I'm assuming this difference is due to the fact that I have specified
            "no_address_mappings" for my SMTP daemon -- but not for the Submission
            daemon.

            But if I remove "no_address_mappings" from my SMTP daemon configuration
            (in order to make virtual alias domain processing work for mail relayed
            in from the outside), I start getting extraneous sender_bcc processing
            on messages originally from me that get relayed onto my server (e.g.,
            messages I send to a mailing list, where the list server is sending me
            a copy of my own message).

            Is there any way to configure the SMTP daemon to do virtual alias
            processing, but NOT also do sender_bcc processing? Or should I be
            looking at something else (not the SMTP daemon) in order to enable
            virtual alias handling?

            Apologies in advance if the answer is somewhere in the docs, but I'm
            unable to find it.

            --
            Rich Wales === Palo Alto, CA, USA === richw@...
            http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
            "The difference between theory and practice is that, in theory,
            theory and practice are identical -- whereas in practice, they aren't."
          • mouss
            ... then you have disabled rewrite or virtual_alias_maps in the after the filter smtpd. ... master.cf will tell. ... you must disable address rewrite in all
            Message 5 of 13 , Sep 29, 2007
            • 0 Attachment
              Rich Wales wrote:
              > Noel Jones wrote:
              >
              >> You're looking at very old how-to's that predate the introduction
              >> of no_address_mappings. . . . See the README.postfix included with
              >> amavisd-new, and stop referring to outdated information.
              >
              > OK, thanks.
              >
              > Right now, I've managed to get a virtual alias domain to work IF the
              > message is originally submitted to my server -- but NOT if the message
              > comes in via the regular SMTP server.
              >

              then you have disabled rewrite or virtual_alias_maps in the after the
              filter smtpd.

              > [snip]
              >
              > Is there any way to configure the SMTP daemon to do virtual alias
              > processing, but NOT also do sender_bcc processing? Or should I be
              > looking at something else (not the SMTP daemon) in order to enable
              > virtual alias handling?
              >

              master.cf will tell.

              > Apologies in advance if the answer is somewhere in the docs, but I'm
              > unable to find it.
              >

              you must disable address rewrite in all but _ONE_ smtpd. The common way
              is to disable it before amavis, and enable it after amavis. and don't
              play with virtual_alias_maps in your master.cf....

              post your master.cf for more infos.
            • Rich Wales
              ... Thanks. I ve reinstated my configuration to the way it was before I started messing around with virtual alias domains. See the attached copy of my
              Message 6 of 13 , Sep 29, 2007
              • 0 Attachment
                mouss wrote:

                > you must disable address rewrite in all but _ONE_ smtpd. The common
                > way is to disable it before amavis, and enable it after amavis. and
                > don't play with virtual_alias_maps in your master.cf.... post your
                > master.cf for more infos.

                Thanks. I've reinstated my configuration to the way it was before I
                started messing around with virtual alias domains. See the attached
                copy of my "postconf -n" output plus my master.cf file.

                How should I modify this configuration so that address rewriting will
                be done in the right place?

                --
                Rich Wales === Palo Alto, CA, USA === richw@...
                http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
                "The difference between theory and practice is that, in theory,
                theory and practice are identical -- whereas in practice, they aren't."
              • mouss
                ... see comments below. I recommend that you take a look at http://www.ijs.si/software/amavisd/README.postfix.html and/or
                Message 7 of 13 , Sep 29, 2007
                • 0 Attachment
                  Rich Wales wrote:
                  > mouss wrote:
                  >
                  >> you must disable address rewrite in all but _ONE_ smtpd. The common
                  >> way is to disable it before amavis, and enable it after amavis. and
                  >> don't play with virtual_alias_maps in your master.cf.... post your
                  >> master.cf for more infos.
                  >
                  > Thanks. I've reinstated my configuration to the way it was before I
                  > started messing around with virtual alias domains. See the attached
                  > copy of my "postconf -n" output plus my master.cf file.
                  >
                  > How should I modify this configuration so that address rewriting will
                  > be done in the right place?
                  >
                  >

                  see comments below. I recommend that you take a look at
                  http://www.ijs.si/software/amavisd/README.postfix.html
                  and/or
                  http://www.ijs.si/software/amavisd/README.postfix.old
                  and copy-paste the configuration parameters there. don't add any other
                  setting unless you know why.


                  > smtp inet n - n - - smtpd
                  > -o sender_bcc_maps=
                  remove this. no_address_mapping disables rewrite, so why unset the map?

                  > -o smtpd_client_restrictions=
                  > -o smtpd_helo_restrictions=
                  > -o
                  >smtpd_recipient_restrictions=check_client_access,hash:/usr/local/etc/postfix/submission_access,permit_mynetworks,sleep,1,reject_unauth_pipelining,reject_invalid_helo_hostname,reject_unauth_destination,reject_unlisted_recipient,check_client_access,pcre:/usr/local/etc/postfix/client_greylist_check,check_helo_access,pcre:/usr/local/etc/postfix/helo_greylist_check,reject_rbl_client,zen.spamhaus.org
                  > -o smtpd_delay_reject=yes
                  > -o >receive_override_options=no_header_body_checks,no_address_mappings
                  >#
                  >[snip]
                  >smtp-amavis unix - - n - 2 smtp
                  > -o sender_bcc_maps=

                  remove this. don't add random settings. Note this is an "smtp"
                  transport, not an "smtpd" service.

                  > -o receive_override_options=no_address_mappings

                  remove this. don't add random settings.

                  > -o smtp_data_done_timeout=1200
                  > -o smtp_send_xforward_command=yes
                  > -o disable_dns_lookups=yes
                  > -o max_use=20

                  >127.0.0.1:10025 inet n - n - - smtpd
                  > -o content_filter=
                  > -o local_recipient_maps=
                  > -o relay_recipient_maps=

                  why do you unset recipient maps? just disable recipient validation
                  (which you already do xwith receive_overrid...

                  > [snip]
                  > -o
                  >
                  receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_address_mappings


                  duh. don't disable address rewrite again.

                  > -o sender_bcc_maps=

                  so you disable sender_bcc_maps?
                • Rich Wales
                  Thanks, mouss. I took out the configuration lines which you objected to in my master.cf file, and I copied/pasted the dedicated smtp-client and dedicated
                  Message 8 of 13 , Sep 29, 2007
                  • 0 Attachment
                    Thanks, mouss.

                    I took out the configuration lines which you objected to in my
                    master.cf file, and I copied/pasted the "dedicated smtp-client"
                    and "dedicated SMTP-server" blocks from README.postfix.html in the
                    amavisd-new documentation (I'm running amavisd-new 2.5.0, dated
                    2007-04-23).

                    The resulting configuration did NOT work.

                    Specifically, I started getting multiple, extraneous copies of
                    test mail submitted into my server. These extra copies stopped
                    (and everything started working OK again) only after I added
                    "no_address_mappings" back into the reinjection SMTP server
                    listening on port 10025. Yes, I know you (and README.postfix.html)
                    say NOT to block address mapping at this point, but all I can tell
                    you is that my setup currently works ONLY if I add this extra
                    configuration item in this particular place.

                    And yes, I did try adding "no_address_mappings" in my submission
                    daemon (and retaining a vanilla post-Amavis reinjection daemon
                    configuration), but that didn't work properly either.

                    As best I can tell from my current configuration -- the one that
                    appears to do everything right EXCEPT it won't do virtual alias
                    domains -- address mapping is currently happening ONLY in the
                    "submission" daemon and nowhere else (because the regular SMTP daemon
                    and the post-Amavis reinjection daemon both have no_address_mappings
                    specified right now). I suppose this is OK if the only address
                    mapping I'm doing is sender BCC, but I need to start doing some
                    virtual alias processing, and this current configuration won't do
                    that for mail coming in to my SMTP server from random originators.

                    As I suggested earlier, I think I could be happy IF there were some
                    way I could enable virtual alias processing of mail arriving via the
                    regular SMTP daemon (the one listening on port 25), but WITHOUT also
                    enabling sender BCC processing of mail arriving via SMTP. But I'm
                    not aware, at the moment, of any way to do such a thing.

                    I'm attaching my current master.cf and my "postconf -n" output.

                    --
                    Rich Wales === Palo Alto, CA, USA === richw@...
                    http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
                    "The difference between theory and practice is that, in theory,
                    theory and practice are identical -- whereas in practice, they aren't."
                  • Rich Wales
                    Elaborating on part of what I said earlier: The reason why I want to have sender BCC processing ONLY for mail which is first being submitted -- as opposed to
                    Message 9 of 13 , Sep 30, 2007
                    • 0 Attachment
                      Elaborating on part of what I said earlier:

                      The reason why I want to have sender BCC processing ONLY for mail
                      which is first being submitted -- as opposed to mail arriving via
                      the SMTP daemon -- is that I'm using sender_bcc_maps as a means
                      of generating "file copies" of email I (and others in my family)
                      compose and send out. I'm doing it this way in lieu of making
                      people use a default BCC: option in their mail configurations --
                      either for cases when people are using clients (such as Outlook)
                      that don't support such an option, or in some cases when users
                      would be confused and object to an unsightly BCC: address showing
                      up every time they compose a message (don't laugh, I've had this
                      problem with some of my users).

                      Again, though, it appears that I can't restrict sender BCC rewriting
                      to happen only during submission without ALSO restricting virtual
                      alias rewriting to happen only during submission -- a restriction
                      which makes virtual alias domains essentially useless.

                      Any tips as to how to do what I'm trying to do (both with virtual
                      alias domains and sender BCC processing) would be gratefully
                      appreciated. Thanks.

                      --
                      Rich Wales === Palo Alto, CA, USA === richw@...
                      http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
                      "The difference between theory and practice is that, in theory,
                      theory and practice are identical -- whereas in practice, they aren't."
                    • mouss
                      ... How could we guess that you want bcc for the submission port only? Next time, state your _goal_ precisely and consiely. if you need to add details, put
                      Message 10 of 13 , Sep 30, 2007
                      • 0 Attachment
                        Rich Wales wrote:
                        > Elaborating on part of what I said earlier:
                        >
                        > The reason why I want to have sender BCC processing ONLY for mail
                        > which is first being submitted -- as opposed to mail arriving via
                        > the SMTP daemon -- is that I'm using sender_bcc_maps as a means
                        > of generating "file copies" of email I (and others in my family)
                        > compose and send out. I'm doing it this way in lieu of making
                        > people use a default BCC: option in their mail configurations --
                        > either for cases when people are using clients (such as Outlook)
                        > that don't support such an option, or in some cases when users
                        > would be confused and object to an unsightly BCC: address showing
                        > up every time they compose a message (don't laugh, I've had this
                        > problem with some of my users).

                        How could we guess that you want bcc for the submission port only?
                        Next time, state your _goal_ precisely and consiely. if you need to add
                        details, put them at last, otherwise, we just get lost.

                        > [snip]


                        with a single instance, edit master.cf to have something like this:

                        submission ... smtpd
                        ...
                        -o cleanup_service_name=msa-cleanup

                        msa-cleanup ... cleanup
                        -o sender_bcc_maps=hash:/etc/postfix/msa_sender_bcc
                        ...

                        and remove all other references to sender_bcc_maps (in main.cf and
                        master.cf). for the rest, configure your services as recommended in the
                        amavisd-new documentation (already suggested in this thread).
                      • Rich Wales
                        ... Thanks. I ll try this and let people know how it works. Sorry I didn t say anything earlier about the restricted use to which I was putting the sender BCC
                        Message 11 of 13 , Sep 30, 2007
                        • 0 Attachment
                          mouss wrote:

                          > with a single instance, edit master.cf to have something like this:
                          >
                          > submission ... smtpd
                          > ...
                          > -o cleanup_service_name=msa-cleanup
                          >
                          > msa-cleanup ... cleanup
                          > -o sender_bcc_maps=hash:/etc/postfix/msa_sender_bcc
                          > ...
                          >
                          > and remove all other references to sender_bcc_maps (in main.cf and
                          > master.cf). for the rest, configure your services as recommended in
                          > the amavisd-new documentation (already suggested in this thread).

                          Thanks. I'll try this and let people know how it works.

                          Sorry I didn't say anything earlier about the restricted use to which
                          I was putting the sender BCC facility, but it didn't occur to me until
                          earlier today that what I was doing might not be transparently obvious.

                          I definitely had never thought of setting up a separate cleanup service
                          (with different parameters) for submission as opposed to other functions.
                          The "cleanup_service_name" parameter is, of course, mentioned in the
                          postconf.5.html page, but I didn't note any mention of the possibilities
                          for being creative with this configuration option (until you mentioned
                          it yourself just now). Perhaps this idea deserves some discussion in a
                          how-to somewhere.

                          --
                          Rich Wales === Palo Alto, CA, USA === richw@...
                          http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
                          "The difference between theory and practice is that, in theory,
                          theory and practice are identical -- whereas in practice, they aren't."
                        • mouss
                          ... In a previous mail, I sugested http://www.ijs.si/software/amavisd/README.postfix.old which describes the use of cleanup_service_name. before
                          Message 12 of 13 , Sep 30, 2007
                          • 0 Attachment
                            Rich Wales wrote:
                            > Thanks. I'll try this and let people know how it works.
                            >
                            > Sorry I didn't say anything earlier about the restricted use to which
                            > I was putting the sender BCC facility, but it didn't occur to me until
                            > earlier today that what I was doing might not be transparently obvious.
                            >
                            > I definitely had never thought of setting up a separate cleanup service
                            > (with different parameters) for submission as opposed to other functions.
                            > The "cleanup_service_name" parameter is, of course, mentioned in the
                            > postconf.5.html page, but I didn't note any mention of the possibilities
                            > for being creative with this configuration option (until you mentioned
                            > it yourself just now). Perhaps this idea deserves some discussion in a
                            > how-to somewhere.
                            >


                            In a previous mail, I sugested
                            http://www.ijs.si/software/amavisd/README.postfix.old
                            which describes the use of cleanup_service_name. before
                            override_received_mapping support, that was the way to disable address
                            rewrite. it is still the way to change configuration parameters that are
                            handled by cleanup.

                            a "cleaner" approach is to use multiple postfix instances. This has
                            other advantages.
                          • Rich Wales
                            Just to confirm (as I said I would do earlier) that mouss s idea -- ... -- appears to be working just fine for my situation (in which I want to use
                            Message 13 of 13 , Oct 2, 2007
                            • 0 Attachment
                              Just to confirm (as I said I would do earlier) that mouss's idea --

                              > with a single instance, edit master.cf to have something like this:
                              >
                              > submission ... smtpd
                              > ...
                              > -o cleanup_service_name=msa-cleanup
                              >
                              > msa-cleanup ... cleanup
                              > -o sender_bcc_maps=hash:/etc/postfix/msa_sender_bcc
                              > ...
                              >
                              > and remove all other references to sender_bcc_maps (in main.cf and
                              > master.cf). for the rest, configure your services as recommended in
                              > the amavisd-new documentation (already suggested in this thread).

                              -- appears to be working just fine for my situation (in which I want to
                              use sender_bcc_maps as a "file copy" functionality for initial message
                              submission only, but also want to use virtual alias domains for incoming
                              mail from all sources).

                              --
                              Rich Wales === Palo Alto, CA, USA === richw@...
                              http://www.richw.org === http://en.wikipedia.org/wiki/User:Richwales
                              "The difference between theory and practice is that, in theory,
                              theory and practice are identical -- whereas in practice, they aren't."
                            Your message has been successfully submitted and would be delivered to recipients shortly.