Loading ...
Sorry, an error occurred while loading the content.

Sometimes loosing mails when using Procmail as content filter

Expand Messages
  • Michael Neurohr
    Hi, I m using Procmail to filter incoming messages with Spamassassin and ClamAV. Today I recognized that mails are getting lost when a single mail contains
    Message 1 of 6 , Jun 11, 2014
    • 0 Attachment
      Hi,
      I'm using Procmail to filter incoming messages with Spamassassin and ClamAV.
      Today I recognized that mails are getting lost when a single mail
      contains multiple recipients that are hosted on this server. I've no
      clue what I'm doing wrong. Filtering the mails if only one of the
      recipients is hosted on this server works fine.

      I'm using Postfix version 2.6.6.

      It looks like the mail is lost directly during the first Procmail rule.
      I already tried to define a lock for the first rule, but it does not help.

      The first mail (out of multiple recipients in that single mail) arrives
      correctly at its recipient, all other are getting lost.

      Can someone tell me what I'm doing wrong?

      Thanks,
      Michael

      The corresponding log lines are:

      maillog
      =====================================================================
      Jun 11 08:52:30 mx0 postfix/smtpd[31969]: connect from
      testrelay.tu-graz.ac.at[129.27.2.201]
      Jun 11 08:52:30 mx0 postfix/smtpd[31969]: 8581482404B8:
      client=testrelay.tu-graz.ac.at[129.27.2.201]
      Jun 11 08:52:30 mx0 postfix/cleanup[31977]: 8581482404B8:
      message-id=<5397FCAA.8060603@...>
      Jun 11 08:52:30 mx0 postfix/qmgr[31967]: 8581482404B8:
      from=<michael.neurohr@...>, size=2241, nrcpt=4 (queue active)
      Jun 11 08:52:30 mx0 postfix/smtpd[31969]: disconnect from
      testrelay.tu-graz.ac.at[129.27.2.201]
      Jun 11 08:52:35 mx0 postfix/pickup[31968]: D3AC582437EE: uid=5000
      from=<michael.neurohr@...>
      Jun 11 08:52:35 mx0 postfix/pipe[31978]: 8581482404B8:
      to=<gerhard@...>, relay=procmail, delay=5.4,
      delays=0.13/0/0/5.3, dsn=2.0.0, status=sent (delivered via procmail service)
      Jun 11 08:52:35 mx0 postfix/pipe[31978]: 8581482404B8:
      to=<michael@...>, relay=procmail, delay=5.4,
      delays=0.13/0/0/5.3, dsn=2.0.0, status=sent (delivered via procmail service)
      Jun 11 08:52:35 mx0 postfix/pipe[31978]: 8581482404B8:
      to=<susanna@...>, relay=procmail, delay=5.4,
      delays=0.13/0/0/5.3, dsn=2.0.0, status=sent (delivered via procmail service)
      Jun 11 08:52:35 mx0 postfix/pipe[31978]: 8581482404B8:
      to=<veronika@...>, relay=procmail, delay=5.4,
      delays=0.13/0/0/5.3, dsn=2.0.0, status=sent (delivered via procmail service)
      Jun 11 08:52:35 mx0 postfix/qmgr[31967]: 8581482404B8: removed
      Jun 11 08:52:35 mx0 postfix/cleanup[31977]: D3AC582437EE:
      message-id=<5397FCAA.8060603@...>
      Jun 11 08:52:35 mx0 postfix/qmgr[31967]: D3AC582437EE:
      from=<michael.neurohr@...>, size=2659, nrcpt=1 (queue active)
      Jun 11 08:52:35 mx0 dovecot: lmtp(32093): Connect from local
      Jun 11 08:52:35 mx0 dovecot: lmtp(32093, veronika@...):
      HyvbNLP8l1NdfQAAtCQUIQ: sieve: msgid=<5397FCAA.8060603@...>:
      stored mail into mailbox 'INBOX'
      Jun 11 08:52:35 mx0 postfix/lmtp[32003]: D3AC582437EE:
      to=<veronika@...>,
      relay=mx0.filestore.su[private/dovecot-lmtp], delay=0.13,
      delays=0.02/0/0.01/0.1, dsn=2.0.0, status=sent (250 2.0.0
      <veronika@...> HyvbNLP8l1NdfQAAtCQUIQ Saved)
      Jun 11 08:52:35 mx0 dovecot: lmtp(32093): Disconnect from local:
      Successful quit
      Jun 11 08:52:35 mx0 postfix/qmgr[31967]: D3AC582437EE: removed
      =====================================================================

      procmail.log
      =====================================================================
      procmail: [32072] Wed Jun 11 08:52:30 2014
      procmail: Locking "/var/mail/vhosts/backup-mail/veronika@.../.lock"
      procmail: Assigning
      "LASTFOLDER=/var/mail/vhosts/backup-mail/veronika@.../new/1402469550.32072_1.mx0.ovz"
      procmail: Unlocking
      "/var/mail/vhosts/backup-mail/veronika@.../.lock"
      procmail: Executing "/usr/local/bin/clamassassin"
      procmail: Match on "^X-Virus-Status: No"
      procmail: Executing "/usr/bin/spamassassin"
      procmail: [32072] Wed Jun 11 08:52:35 2014
      procmail: Executing
      "/usr/sbin/sendmail,-f,michael.neurohr@...,veronika@..."
      procmail: [32072] Wed Jun 11 08:52:35 2014
      procmail: Assigning "LASTFOLDER=/usr/sbin/sendmail -f
      michael.neurohr@... veronika@..."
      Subject: Re: test
      Folder: /usr/sbin/sendmail -f michael.neurohr@... vero 2532
      =====================================================================

      My procmail script is:
      =====================================================================
      SHELL=/bin/sh
      SENDMAILFLAGS="-f $E_SENDER $E_RECIPIENT"
      LOGFILE="/var/log/procmail"
      VERBOSE=on

      # First make a backup of the mail
      :0c:
      /var/mail/vhosts/backup-mail/$E_RECIPIENT/

      # Scan for viruses
      :0fw
      | /usr/local/bin/clamassassin

      # If no virus has been found, scan for Spam
      :0fw
      * ^X-Virus-Status: No
      | /usr/bin/spamassassin

      # Last action: inject back to Postfix
      :0w
      | /usr/sbin/sendmail $SENDMAILFLAGS
      =====================================================================

      It is connected to Postfix as follows:
      =====================================================================
      smtp inet n - n - - smtpd
      -o content_filter=procmail:dummy
      -o receive_override_options=no_address_mappings
      -o smtpd_recipient_restrictions=reject_unauth_destination
      =====================================================================
      =====================================================================
      procmail unix - n n - 10 pipe
      flags=Rq user=vmail null_sender= argv=/usr/bin/procmail -m
      E_SENDER=$sender E_RECIPIENT=$recipient /etc/procmailrc
      =====================================================================

      postconf -n produces:
      =====================================================================
      alias_database = hash:/etc/aliases
      alias_maps = hash:/etc/aliases
      append_dot_mydomain = no
      biff = no
      command_directory = /usr/sbin
      config_directory = /etc/postfix
      daemon_directory = /usr/libexec/postfix
      data_directory = /var/lib/postfix
      html_directory = no
      inet_interfaces = all
      inet_protocols = ipv4, ipv6
      mail_owner = postfix
      mailbox_size_limit = 0
      mailq_path = /usr/bin/mailq
      manpage_directory = /usr/share/man
      message_size_limit = 26214400
      mydestination = localhost, $myhostname
      myhostname = mx0.filestore.su
      mynetworks = 127.0.0.0/8 [::1]/128 [fe80::]/10
      myorigin = $mydomain
      newaliases_path = /usr/bin/newaliases
      queue_directory = /var/spool/postfix
      readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES
      recipient_delimiter = +
      relayhost =
      sample_directory = /usr/share/doc/postfix-2.6.6/samples
      sendmail_path = /usr/sbin/sendmail
      setgid_group = postdrop
      smtp_bind_address6 = 2a04:f81:783f:783f:43e1:787a:4be1:29a8
      smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
      smtpd_client_restrictions = reject_rbl_client ix.dnsbl.manitu.net
      smtpd_recipient_restrictions = permit_sasl_authenticated,
      reject_non_fqdn_sender, reject_non_fqdn_recipient,
      reject_unknown_sender_domain, reject_unknown_recipient_domain,
      permit_mynetworks, reject_unauth_destination, permit
      smtpd_sasl_auth_enable = yes
      smtpd_sasl_path = private/auth
      smtpd_sasl_type = dovecot
      smtpd_tls_CAfile = /var/mail/ssl/ca-bundle.crt
      smtpd_tls_auth_only = yes
      smtpd_tls_cert_file = /var/mail/ssl/mx0.filestore.su.pem
      smtpd_tls_key_file = /var/mail/ssl/mx0.filestore.su.key
      smtpd_use_tls = yes
      virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf
      virtual_mailbox_domains =
      mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
      virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf
      virtual_transport = lmtp:unix:private/dovecot-lmtp
      =====================================================================
    • D'Arcy J.M. Cain
      On Wed, 11 Jun 2014 09:37:44 +0200 ... You post a lot of information but I don t see /var/log/procmail. Perhaps it has a clue. I also use procmail to run
      Message 2 of 6 , Jun 11, 2014
      • 0 Attachment
        On Wed, 11 Jun 2014 09:37:44 +0200
        Michael Neurohr <mnewpipe@...> wrote:
        > I'm using Procmail to filter incoming messages with Spamassassin and
        > ClamAV. Today I recognized that mails are getting lost when a single
        > mail contains multiple recipients that are hosted on this server.
        > I've no clue what I'm doing wrong. Filtering the mails if only one of
        > the recipients is hosted on this server works fine.

        You post a lot of information but I don't see /var/log/procmail.
        Perhaps it has a clue.

        I also use procmail to run spamassassin but I do it from individual
        user .procmailrc files (generated from their preferences) and I wonder
        if that has something to do with it. What about SA and CAV system
        configs? Could there be something in there that uses a common file to
        check for dups? I have this in my .procmailrc files:

        :0 Wh: msgid.lock
        | formail -D 65536 $HOME/.msgid.cache

        If something similar is happening with those programs but with a common
        database (e.g. /var/db/msgid.cache) that might have the effect that you
        are seeing.

        Just spitballing here. The procmail log might shed more light.

        --
        D'Arcy J.M. Cain
        System Administrator, Vex.Net
        http://www.Vex.Net/ IM:darcy@...
        VoIP: sip:darcy@...
      • Wietse Venema
        ... Configure Postfix to make single-recipient deliveries. Set procmail_destination_recipient_limit = 1 which is required just like it is required for
        Message 3 of 6 , Jun 11, 2014
        • 0 Attachment
          Michael Neurohr:
          > Hi,
          > I'm using Procmail to filter incoming messages with Spamassassin and ClamAV.
          > Today I recognized that mails are getting lost when a single mail
          > contains multiple recipients that are hosted on this server. I've no
          > clue what I'm doing wrong. Filtering the mails if only one of the
          > recipients is hosted on this server works fine.

          Configure Postfix to make single-recipient deliveries.

          Set "procmail_destination_recipient_limit = 1" which is required
          just like it is required for local(8) deliveries.

          Wietse
        • Michael Neurohr
          ... Yes, that did it. Many thanks! Is that setting needed for all filters that are used like unix pipes? Michael
          Message 4 of 6 , Jun 11, 2014
          • 0 Attachment
            On 11.06.2014 14:17, Wietse Venema wrote:
            > Configure Postfix to make single-recipient deliveries.
            >
            > Set "procmail_destination_recipient_limit = 1" which is required
            > just like it is required for local(8) deliveries.

            Yes, that did it. Many thanks!
            Is that setting needed for all filters that are used like unix pipes?

            Michael
          • Michael Neurohr
            ... I wrote a wrong info before the procmail log. All I m seeing in /var/log/procmail is: =====================================================================
            Message 5 of 6 , Jun 11, 2014
            • 0 Attachment
              On 11.06.2014 13:02, D'Arcy J.M. Cain wrote:
              > You post a lot of information but I don't see /var/log/procmail.
              > Perhaps it has a clue.

              I wrote a wrong info before the procmail log. All I'm seeing in
              /var/log/procmail is:

              =====================================================================
              procmail: [32072] Wed Jun 11 08:52:30 2014
              procmail: Locking "/var/mail/vhosts/backup-mail/veronika@.../.lock"
              procmail: Assigning
              "LASTFOLDER=/var/mail/vhosts/backup-mail/veronika@.../new/1402469550.32072_1.mx0.ovz"
              procmail: Unlocking
              "/var/mail/vhosts/backup-mail/veronika@.../.lock"
              procmail: Executing "/usr/local/bin/clamassassin"
              procmail: Match on "^X-Virus-Status: No"
              procmail: Executing "/usr/bin/spamassassin"
              procmail: [32072] Wed Jun 11 08:52:35 2014
              procmail: Executing
              "/usr/sbin/sendmail,-f,michael.neurohr@...,veronika@..."
              procmail: [32072] Wed Jun 11 08:52:35 2014
              procmail: Assigning "LASTFOLDER=/usr/sbin/sendmail -f
              michael.neurohr@... veronika@..."
              Subject: Re: test
              Folder: /usr/sbin/sendmail -f michael.neurohr@... vero 2532
              =====================================================================

              The processing of the other users do not produce any log entries. That
              fact is really wired...


              But as Wietse wrote, the problem was the multiple recipient deliveries.
              The problem is solved now :-)

              Michael
            • Noel Jones
              ... That setting is needed for any transport or filter that expects only one recipient at a time. Very generally, this means final delivery, but there are
              Message 6 of 6 , Jun 11, 2014
              • 0 Attachment
                On 6/11/2014 9:10 AM, Michael Neurohr wrote:
                > On 11.06.2014 14:17, Wietse Venema wrote:
                >> Configure Postfix to make single-recipient deliveries.
                >>
                >> Set "procmail_destination_recipient_limit = 1" which is required
                >> just like it is required for local(8) deliveries.
                >
                > Yes, that did it. Many thanks!
                > Is that setting needed for all filters that are used like unix pipes?
                >
                > Michael
                >

                That setting is needed for any transport or filter that expects only
                one recipient at a time.

                Very generally, this means final delivery, but there are exceptions.



                -- Noel Jones
              Your message has been successfully submitted and would be delivered to recipients shortly.