Loading ...
Sorry, an error occurred while loading the content.

spamassassin (spamd/spamc) duplicating messages in alias/forward+mailbox situation

Expand Messages
  • Johannes Jakob
    Greetings, I m nearly finished, setting up a postfix installation to coexist with qmail based servers with plesk 7.5.4. Let s assume userA@domainA.com wants to
    Message 1 of 7 , Sep 10, 2013
    • 0 Attachment
      Greetings,

      I'm nearly finished, setting up a postfix installation to coexist with
      qmail based servers with plesk 7.5.4.

      Let's assume userA@... wants to have his incoming mails:
      - stored locally in a mailbox (maildir format) and
      - forwarded to an other address

      since all domains on this box are virtual, I'm using
      virtual_mailbox_maps and virtual_alias_maps as well as
      virtual_mailbox_domains:

      ---
      virtual_alias_maps:
      userA@... userA@...,userA@...

      virtual_mailbox_maps:
      userA@... /var/qmail/mailnames/domainA.com/userA/Maildir/

      virtual_mailbox_domains:
      domainA.com ok
      ---

      so far everything is working just fine.

      To complete my setup, I only need to throw in spamassassin for spam tagging.
      Since spam-tagging will be done on a cluster of spam checking servers,
      I need to do this vis spamd/spamc config and can't just use the amavis
      internal spamassassin methods. So I went along this howto, adding
      spamassassin as content-filter:

      http://wiki.apache.org/spamassassin/IntegratedSpamdInPostfix

      (just added "-d server1,server2" as spamassassin args).



      Well, even this is working to a degree: incoming mail passes the
      spamchecks, is being tagged and handed over to postfix's sendmail,
      BUT: because of the situation with local mailbox AND forward, every
      mail gets _duplicated_!


      Can somebody please point out to me, how to fix this? ;-)


      Any help is appreciated! ;-)

      Best Regards,

      John
    • Wietse Venema
      ... You created a mail filter loop. How to fix: please see the mailing list welcome message below. I.e., describe what you have, we already know what you want.
      Message 2 of 7 , Sep 10, 2013
      • 0 Attachment
        Johannes Jakob:
        > Well, even this is working to a degree: incoming mail passes the
        > spamchecks, is being tagged and handed over to postfix's sendmail,
        > BUT: because of the situation with local mailbox AND forward, every
        > mail gets _duplicated_!
        >
        > Can somebody please point out to me, how to fix this? ;-)

        You created a mail filter loop. How to fix: please see the mailing
        list welcome message below. I.e., describe what you have, we already
        know what you want.

        Wietse

        TO REPORT A PROBLEM see http://www.postfix.org/DEBUG_README.html#mail

        TO (UN)SUBSCRIBE see http://www.postfix.org/lists.html

        Thank you for using Postfix.
      • Johannes Jakob
        Hello Wietse, Hi List, Thanks for the quick response and sorry for not posting the complete configuration, I thought the linked tutorial would be sufficient.
        Message 3 of 7 , Sep 11, 2013
        • 0 Attachment
          Hello Wietse,
          Hi List,

          Thanks for the quick response and sorry for not posting the complete
          configuration, I thought the linked tutorial would be sufficient.

          On Tue, Sep 10, 2013 at 12:38 PM, Wietse Venema <wietse@...> wrote:
          > You created a mail filter loop. How to fix: please see the mailing
          > list welcome message below. I.e., describe what you have, we already
          > know what you want.

          I solved the duplication problem by adding ":dummy" to the
          content_filter definition.

          There are two aspects of this configuration bothering me:

          1) Every incoming email will be scanned n times, n being the number
          of recipients or alias targets
          2) Logfiles are getting harder to read because of the after-queue filtering


          Are there any before-queue integration methos for spamassassin and
          postfix that can be used in a spamd/spamc setup like ours?
          Neither amavis-new, nor the spampd method described here
          (http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd) seem
          to support being client for remote spamd server(s).
          I would love to be able to grep for the queued-as-id in the logs to
          get the full processing log of those emails ;)
          Having to deal with qmail logs for the last 5 years was driving me nuts...


          Thanks again for your patiance!

          John


          Now my configuration looks like this:

          main.cf:
          ----------
          alias_maps = hash:/etc/aliases
          append_dot_mydomain = no
          autoresponder_destination_recipient_limit = 1
          biff = no
          broken_sasl_auth_clients = yes
          config_directory = /etc/postfix
          content_filter = smtp-amavis:[127.0.0.1]:10024
          delay_warning_time = 4h
          inet_interfaces = all
          mailbox_size_limit = 0
          message_size_limit = 51200000
          mydestination = mx10.domain.net
          myhostname = mx10.domain.net
          mynetworks = /etc/postfix/mynetworks
          policy-spf_time_limit = 3600s
          readme_directory = no
          recipient_delimiter = +
          smtp_tls_note_starttls_offer = yes
          smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
          smtp_use_tls = yes
          smtpd_client_restrictions = reject_rbl_client dnsbl.sorbs.net
          smtpd_data_restrictions = reject_unauth_pipelining
          smtpd_helo_restrictions = permit_mynetworks,
          reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname,
          reject_unknown_helo_hostname
          smtpd_recipient_restrictions = permit_mynetworks,
          reject_invalid_hostname, reject_unknown_recipient_domain,
          reject_unverified_recipient, permit_sasl_authenticated,
          reject_unauth_destination, reject_rbl_client zen.spamhaus.org,
          reject_rbl_client sbl.spamhaus.org, reject_rhsbl_helo
          dbl.spamhaus.org, reject_rhsbl_sender dbl.spamhaus.org,
          check_policy_service unix:private/policy-spf, permit
          smtpd_sasl_auth_enable = yes
          smtpd_sasl_local_domain =
          smtpd_sasl_tls_security_options = noanonymous
          smtpd_sasl_type = cyrus
          smtpd_tls_cert_file = /etc/ssl/private/smtpd.pem
          smtpd_tls_key_file = /etc/ssl/private/smtpd.pem
          smtpd_tls_loglevel = 1
          smtpd_tls_received_header = yes
          smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
          smtpd_use_tls = yes
          tls_random_source = dev:/dev/urandom
          transport_maps = hash:/etc/postfix/transport_maps
          unknown_local_recipient_reject_code = 550
          virtual_alias_domains = hash:/etc/postfix/virtual_alias_domains
          virtual_alias_maps = hash:/etc/postfix/virtual_alias_maps
          virtual_gid_maps = static:101
          virtual_mailbox_base = /var/qmail/mailnames/
          virtual_mailbox_domains = hash:/etc/postfix/virtual_mailbox_domains
          virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox_maps
          virtual_minimum_uid = 100
          virtual_uid_maps = static:110
          ----------

          master.cf:
          ----------
          smtp inet n - - - - smtpd
          -o content_filter=spamassassin:dummy
          submission inet n - - - - smtpd
          -o syslog_name=postfix/submission
          -o smtpd_tls_security_level=encrypt
          -o smtpd_sasl_auth_enable=yes
          -o smtpd_client_restrictions=permit_sasl_authenticated,reject
          -o milter_macro_daemon_name=ORIGINATING
          smtps inet n - - - - smtpd
          -o syslog_name=postfix/smtps
          -o smtpd_tls_wrappermode=yes
          -o smtpd_sasl_auth_enable=yes
          -o smtpd_client_restrictions=permit_sasl_authenticated,reject
          -o milter_macro_daemon_name=ORIGINATING
          pickup fifo n - - 60 1 pickup
          cleanup unix n - - - 0 cleanup
          qmgr fifo n - n 300 1 qmgr
          tlsmgr unix - - - 1000? 1 tlsmgr
          rewrite unix - - - - - trivial-rewrite
          bounce unix - - - - 0 bounce
          defer unix - - - - 0 bounce
          trace unix - - - - 0 bounce
          verify unix - - - - 1 verify
          flush unix n - - 1000? 0 flush
          proxymap unix - - n - - proxymap
          proxywrite unix - - n - 1 proxymap
          smtp unix - - - - - smtp
          relay unix - - - - - smtp
          showq unix n - - - - showq
          error unix - - - - - error
          retry unix - - - - - error
          discard unix - - - - - discard
          local unix - n n - - local
          virtual unix - n n - - virtual
          lmtp unix - - - - - lmtp
          anvil unix - - - - 1 anvil
          scache unix - - - - 1 scache
          maildrop unix - n n - - pipe
          flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
          uucp unix - n n - - pipe
          flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
          ifmail unix - n n - - pipe
          flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
          bsmtp unix - n n - - pipe
          flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
          scalemail-backend unix - n n - 2 pipe
          flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
          ${nexthop} ${user} ${extension}
          mailman unix - n n - - pipe
          flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
          ${nexthop} ${user}
          policy-spf unix - n n - 0 spawn
          user=nobody argv=/usr/bin/policyd-spf
          smtp-amavis unix - - - - 2 smtp
          -o smtp_data_done_timeout=1200
          -o smtp_send_xforward_command=yes
          -o smtp_tls_note_starttls_offer=no
          127.0.0.1:10025 inet n - - - - smtpd
          -o content_filter=
          -o smtpd_delay_reject=no
          -o smtpd_client_restrictions=permit_mynetworks,reject
          -o smtpd_helo_restrictions=
          -o smtpd_sender_restrictions=
          -o smtpd_recipient_restrictions=permit_mynetworks,reject
          -o smtpd_data_restrictions=reject_unauth_pipelining
          -o smtpd_end_of_data_restrictions=
          -o smtpd_restriction_classes=
          -o mynetworks=127.0.0.0/8
          -o smtpd_error_sleep_time=0
          -o smtpd_soft_error_limit=1001
          -o smtpd_hard_error_limit=1000
          -o smtpd_client_connection_count_limit=0
          -o smtpd_client_connection_rate_limit=0
          -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
          -o local_header_rewrite_clients=
          -o smtpd_milters=
          -o local_recipient_maps=
          -o relay_recipient_maps=
          autoresponder unix - n n - - pipe
          flags=F user=nobody argv=/usr/bin/perl
          /usr/local/bin/postfix-autoresponder.pl ${sender} ${size}
          ${original_recipient}
          spamassassin unix - n n - - pipe
          flags=Rq user=nobody argv=/usr/local/bin/spamassassin.sh -oi -f
          ${sender} ${recipient}
          ----------

          cat /usr/local/bin/spamassassin.sh
          ----------
          #!/bin/bash

          SENDMAIL=/usr/sbin/sendmail
          SPAMASSASSIN=/usr/bin/spamc
          SPAMASSASSINARGS="-d server1,server2 -u $4"

          logger <<<"Spam filter piping to ${SPAMASSASSIN} ${SPAMASSASSINARGS},
          then to: $SENDMAIL $@"
          ${SPAMASSASSIN} ${SPAMASSASSINARGS} | ${SENDMAIL} "$@"

          exit $?
          ----------


          ---
          virtual_alias_maps:
          userA@... userA@...,userA@...
          ---
          virtual_mailbox_maps:
          userA@... /var/qmail/mailnames/domainA.com/userA/Maildir/
          ---
          virtual_mailbox_domains:
          domainA.com ok
          ---
        • Wietse Venema
          ... You appear to believe that you implemented the cookbook recipe correctly. I think that is too optimistic. In my experience, people often don t see the
          Message 4 of 7 , Sep 11, 2013
          • 0 Attachment
            Johannes Jakob:
            > Hello Wietse,
            > Hi List,
            >
            > Thanks for the quick response and sorry for not posting the complete
            > configuration, I thought the linked tutorial would be sufficient.

            You appear to believe that you implemented the cookbook recipe
            correctly. I think that is too optimistic. In my experience, people
            often don't see the difference between what they want to have and
            what they actually have.

            Wietse
          • Johannes Jakob
            ... I m seriously confused... Did I miss something obvious? I just checked again, not seeing anything relevant (besides from the additional spamassassin
            Message 5 of 7 , Sep 11, 2013
            • 0 Attachment
              >> Thanks for the quick response and sorry for not posting the complete
              >> configuration, I thought the linked tutorial would be sufficient.
              >
              > You appear to believe that you implemented the cookbook recipe
              > correctly. I think that is too optimistic. In my experience, people
              > often don't see the difference between what they want to have and
              > what they actually have.

              I'm seriously confused...
              Did I miss something obvious? I just checked again, not seeing
              anything relevant (besides from the additional spamassassin arguments
              and the changed transport name) different from the original tutorial.
              I read some parts of the content_filter documentation where it says

              "The "-o content_filter" line causes Postfix to add one content filter
              request record to each incoming mail message, with content
              "filter:dummy". This record overrides the normal mail routing and
              causes mail to be given to the content filter instead."

              so I added it ;-)

              But the rest of my setups should be quite similar to the original tutorial's.

              Could you please point out to me, where you see incorrect changes in
              my given configuration snippets?


              Nevertheless my question stays: is there a clean way to add
              spamassassin before-queue and keep the possibility to use spamd/spamc
              setup?

              Thanks again for your time ;-)

              John
            • Noel Jones
              ... There are some milters that use spamc/spamd, and should work well with recent postfix versions. Google is your friend. -- Noel Jones
              Message 6 of 7 , Sep 11, 2013
              • 0 Attachment
                On 9/11/2013 9:53 AM, Johannes Jakob wrote:
                >
                > Nevertheless my question stays: is there a clean way to add
                > spamassassin before-queue and keep the possibility to use spamd/spamc
                > setup?

                There are some milters that use spamc/spamd, and should work well
                with recent postfix versions. Google is your friend.


                -- Noel Jones
              • Johannes Jakob
                ... Thanks Noel... milter was the keyword I needed... google d a lot, but thought, postfix and spamassassin sites would be the best source of howtos.
                Message 7 of 7 , Sep 11, 2013
                • 0 Attachment
                  On Wed, Sep 11, 2013 at 5:03 PM, Noel Jones <njones@...> wrote:
                  > There are some milters that use spamc/spamd, and should work well
                  > with recent postfix versions. Google is your friend.

                  Thanks Noel... "milter" was the keyword I needed... google'd a lot,
                  but thought, postfix and spamassassin sites would be the best source
                  of howtos.

                  spamass-milter is working perfectly and all my problems are solved.

                  Thanks again,

                  John
                Your message has been successfully submitted and would be delivered to recipients shortly.