Loading ...
Sorry, an error occurred while loading the content.

Re: FILTER_README suggestions

Expand Messages
  • /dev/rob0
    ... Interesting. As readily-available as your address is, I would think spam would be a major problem for you. ... Amavisd-new with local(8) and address
    Message 1 of 16 , Sep 1, 2006
    • 0 Attachment
      On Friday 01 September 2006 09:53, Wietse Venema wrote:
      > Postfix's FILTER_README was written long before backscatter became
      > a problem. The first example (see below signature) has a warning
      > not to reject mail:
      >
      > Note: in this time of mail worms and spam, it is a BAD IDEA to
      > send known viruses or spam back to the sender, because that
      > address is likely to be forged. It is safer to discard known
      > to be bad content and to quarantine suspicious content so that
      > it can be inspected by a human being.
      >
      > Unfortunately, the text gives no example of how one would implement
      > this advice. Personally, I use no external filter so I have a hard
      > time coming up with field-tested examples.

      Interesting. As readily-available as your address is, I would think
      spam would be a major problem for you.

      > What do people use:

      Amavisd-new with local(8) and address tagging, with ~/.forward+spam
      redirecting to a user-accessible Quarantine folder. I'm very pleased
      with the results. It's smooth and simple, no arcane and difficult
      procmail syntax. It's something I could put within the reach of an
      ordinary user, with a simple example in "/etc/skel/.forward+spam".

      I do change the amavisd-new defaults to D_PASS for all categories but
      virus (and I have not yet had a virus pass through Postfix's defenses.)

      > - Have the filter return a distinct exit status that says "discard"?
      >
      > - Have the filter insert a "badness" indicator in a message header,
      > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
      > rules, cyrus sieves, or procmail filters?
      >
      > - Something completely different? Maybe no-one uses the pipe+sendmail
      > example and we can drop it from the documentation.

      In actual practice I bet few sites are using the simple content filter
      example. However, it is potentially useful as a starting point for one
      who wants to roll his/her own filter.

      > 24 # Specify your content filter here.
      > 25 # filter <in.$$ || {
      > 26 # echo Message content rejected; exit $EX_UNAVAILABLE; }

      You could just expand the comments in the script to list the filter's
      choices. :)
      --
      Offlist mail to this address is discarded unless
      "/dev/rob0" or "not-spam" is in Subject: header
    • Darron Froese
      ... We reject some mail directly at SMTP (reject_unknown_sender_domain, reject_non_fqdn_sender, check_helo_access, reject_unknown_recipient_domain,
      Message 2 of 16 , Sep 1, 2006
      • 0 Attachment
        On 1-Sep-06, at 8:53 AM, Wietse Venema wrote:

        > Unfortunately, the text gives no example of how one would implement
        > this advice. Personally, I use no external filter so I have a hard
        > time coming up with field-tested examples.
        >
        > What do people use:

        We reject some mail directly at SMTP (reject_unknown_sender_domain,
        reject_non_fqdn_sender, check_helo_access,
        reject_unknown_recipient_domain, reject_unauth_destination,
        reject_unlisted_recipient and a manual domain name blacklist) but
        after that:

        1. Everything that matches a cable, DSL or dialup connection gets
        greylisted with tumgreyspf - configurable with a manual whitelist for
        broken mail servers.
        2. All mail with known viruses get silently discarded (amavisd-new
        and clamav)
        3. Mail gets tagged with SpamAssassin (through amavisd-new and a
        bunch of extra SA plugins)
        4. Clients have the option to discard above a certain SA score or
        filter through a web interface .

        I did up a diagram of it to figure it all out in my mind while I was
        building:

        http://nonfiction.ca/mail-arch.jpg

        Works great for us - brought down my spam levels from 300 / day with
        old Postfix and SA 2.6 down to about 2 or 3 / day.
        --
        darron froese
        principal
        nonfiction studios inc.
        t 403.686.8887
        c 403.819.7887
        f 403.313.9233
        w http://nonfiction.ca/
        e darron@...
      • Victor Duchovni
        ... I use badness headers, then pump all labeled mail into the hold queue of a downstream Postfix instance. From the hold queue messages are moved into
        Message 3 of 16 , Sep 1, 2006
        • 0 Attachment
          On Fri, Sep 01, 2006 at 10:53:49AM -0400, Wietse Venema wrote:

          > - Have the filter insert a "badness" indicator in a message header,
          > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
          > rules, cyrus sieves, or procmail filters?

          I use "badness" headers, then pump all labeled mail into the "hold" queue
          of a downstream Postfix instance. From the "hold" queue messages are
          moved into medium term storage (quarantine) and users receive periodic
          notices of quarantined mail, and are able to click on links to release
          selected messages. The tools in question are not publically available,
          perhaps someday...

          --
          Viktor.

          Disclaimer: off-list followups get on-list replies or get ignored.
          Please do not ignore the "Reply-To" header.

          To unsubscribe from the postfix-users list, visit
          http://www.postfix.org/lists.html or click the link below:
          <mailto:majordomo@...?body=unsubscribe%20postfix-users>

          If my response solves your problem, the best way to thank me is to not
          send an "it worked, thanks" follow-up. If you must respond, please put
          "It worked, thanks" in the "Subject" so I can delete these quickly.
        • Tony Earnshaw
          fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema: [...] ... A couple of dead simple stanzas from a maildrop/dspam setup: in /etc/maildroprc ($VHOME
          Message 4 of 16 , Sep 1, 2006
          • 0 Attachment
            fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema:

            [...]

            > > We use "tag & deliver" a.k.a second option. For some clients, maildrop
            > > places the mail in appropriate folder. For others, we let the MUA decide
            > > what to do with tagged mail.
            >
            > Thanks to all who recommended the "tag and deliver" approach.
            >
            > > Basically, quarantining is too much trouble for our end users and we don't
            > > discard email for fear of false positives.
            > >
            > > I suggest adding some simple maildrop/procmail recipies as examples to
            > > help the uninitiated.
            >
            > If you have examples to share, it will save me time.

            A couple of dead simple stanzas from a maildrop/dspam setup:

            in /etc/maildroprc ($VHOME is a variable defined at the head of
            maildroprc):

            `test -d $VHOME/Maildir/.dspam-quarantine/`
            if( $RETURNCODE == 1 )
            {
            `/usr/bin/maildirmake "$VHOME/Maildir/.dspam-quarantine"/`
            }
            __________________________

            if ( /^X-DSPAM-Result:[:space:]+Spam$/ )
            {
            to "$VHOME/Maildir/.dspam-quarantine"
            }

            --Tonni

            --
            Tony Earnshaw
            reservebergenser
          • Darron Froese
            ... This is a Sieve recipe that is available to our users: # Delete SpamAssassin blacklisted mails. if header :comparator i;ascii-casemap :contains
            Message 5 of 16 , Sep 1, 2006
            • 0 Attachment
              On 1-Sep-06, at 10:40 AM, Wietse Venema wrote:

              > If you have examples to share, it will save me time.

              This is a Sieve recipe that is available to our users:

              # Delete SpamAssassin blacklisted mails.
              if header :comparator "i;ascii-casemap" :contains "X-Spam-Status"
              "BLACKLISTED" {
              discard;
              stop;
              }
              # Delete high score SPAM above 14 SA score (configurable).
              if allof ( header :comparator "i;ascii-casemap" :matches "Subject"
              "[SPAM]*", header :value "gt" :comparator "i;ascii-numeric" "X-Spam-
              Score" ["14"] ) {
              discard;
              stop;
              }
              # Filter Lower Scored Spam for checking.
              if header :comparator "i;ascii-casemap" :matches "Subject" "[SPAM]*" {
              fileinto "INBOX.spam";
              stop;
              }

              It's their choice what they want to do for their own accounts - this
              is my personal setup - all managed via a web interface.
              --
              darron froese
              principal
              nonfiction studios inc.
              t 403.686.8887
              c 403.819.7887
              f 403.313.9233
              w http://nonfiction.ca/
              e darron@...
            • Eray Aslan
              ... Here is a sample maildroprc file: maildirmake=/path/to/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/ #automatically
              Message 6 of 16 , Sep 1, 2006
              • 0 Attachment
                On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
                > If you have examples to share, it will save me time.

                Here is a sample maildroprc file:

                maildirmake=/path/to/maildirmake
                MAILDIR=$DEFAULT
                JUNK_FOLDER=.Spam

                _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


                #automatically create the Junk folder
                `test -d $_JUNK_DEST`
                if ($RETURNCODE != 0 )
                {
                `$maildirmake $_JUNK_DEST`
                # auto subscribe. the following works for courier-imap
                `echo $_JUNK_DEST >> $MAILDIR/courierimapsubscribed`
                }

                # Spam gets tagged with X-Spam-Flag
                if ( /^X-Spam-Flag: YES/:h )
                {
                exception {
                to "$_JUNK_DEST"
                }
                }
                else
                {
                exception {
                to "$MAILDIR/"
                }
                }

                --
                Eray
              • Aaron Bennett
                ... I can t speak highly enough of Maia Mailguard. It s based on amavisd-new but allows per-user bayes training, white and blacklists. We re a middling-volume
                Message 7 of 16 , Sep 1, 2006
                • 0 Attachment
                  Wietse Venema wrote:
                  > What do people use:
                  >
                  >

                  I can't speak highly enough of Maia Mailguard. It's based on
                  amavisd-new but allows per-user bayes training, white and blacklists.
                  We're a middling-volume site (about 85K messages per day) and have two
                  dual-xeon mail relays running Maia and uvscan/clamd. The database
                  component of Maia sits on a third box. In our older environment, we had
                  amavisd-new + spamassassin running without per-user training or
                  quarantining or the other benefits of Maia and as a result, we had to be
                  conservative with our SA thresholds to avoid false-positives. Our
                  number 1 complaint was spam -- since we list all or most staff and
                  faculty email on our web page. Now with Maia, we are heroes. Since
                  mid-july when we rolled it out, we've blocked over 600,000 spams with a
                  false positive rate of 0.06% and a false-negative rate of 6.57%. The
                  power is the user-training; it's easy and simple and lets users take
                  effective action against spam. It really lets SpamAssassin shine.



                  --
                  Aaron Bennett
                  Sr. Unix Systems Administrator
                  Clark University ITS
                  abennett@... | 508.781.7315
                • Eray Aslan
                  ... For the record, here is a working maildroprc file. Please disregard the previous one. maildirmake=/usr/bin/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam
                  Message 8 of 16 , Sep 1, 2006
                  • 0 Attachment
                    On Fri, September 1, 2006 8:30 pm, Eray Aslan wrote:
                    > On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
                    >> If you have examples to share, it will save me time.
                    >
                    > Here is a sample maildroprc file:

                    For the record, here is a working maildroprc file. Please disregard the
                    previous one.


                    maildirmake=/usr/bin/maildirmake
                    MAILDIR=$DEFAULT
                    JUNK_FOLDER=.Spam

                    _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


                    #automatically create the Junk folder
                    `test -d $_JUNK_DEST`
                    if ($RETURNCODE != 0 )
                    {
                    `$maildirmake $_JUNK_DEST`
                    # auto subscribe. the following works for courier-imap
                    `echo INBOX$JUNK_FOLDER >> $MAILDIR/courierimapsubscribed`
                    }
                    # Spam gets tagged with X-Spam-Flag
                    if ( /^X-Spam-Flag: YES/:h )
                    {
                    exception {
                    to "$_JUNK_DEST"
                    }
                    }
                    else
                    {
                    exception {
                    to "$MAILDIR/"
                    }
                    }

                    --
                    Eray
                  • mouss
                    ... I favour tag & deliver, be that to a Junk folder or somewhere else. So I configure the filters to tag all mail (spam or not), and use these in maildrop to
                    Message 9 of 16 , Sep 1, 2006
                    • 0 Attachment
                      Wietse Venema wrote:
                      > Postfix's FILTER_README was written long before backscatter became
                      > a problem. The first example (see below signature) has a warning
                      > not to reject mail:
                      >
                      > Note: in this time of mail worms and spam, it is a BAD IDEA to
                      > send known viruses or spam back to the sender, because that
                      > address is likely to be forged. It is safer to discard known
                      > to be bad content and to quarantine suspicious content so that
                      > it can be inspected by a human being.
                      >
                      > Unfortunately, the text gives no example of how one would implement
                      > this advice. Personally, I use no external filter so I have a hard
                      > time coming up with field-tested examples.
                      >
                      > What do people use:
                      >
                      > - Have the filter return a distinct exit status that says "discard"?
                      >
                      > - Have the filter insert a "badness" indicator in a message header,
                      > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                      > rules, cyrus sieves, or procmail filters?
                      >
                      > - Something completely different? Maybe no-one uses the pipe+sendmail
                      > example and we can drop it from the documentation.
                      >
                      >

                      I favour tag & deliver, be that to a Junk folder or somewhere else. So I
                      configure the filters to tag all mail (spam or not), and use these in
                      maildrop to deliver to a Junk folder.

                      here is an example with spamassassin + courier-imap (the .folder
                      notation below) + maildrop 2.x

                      if (/^X-Spam-Flag:\s*YES/)
                      {
                      exception {
                      to "$DEFAULT/.Junk/";
                      }
                      }

                      with dspam, this would be
                      if (/^X-DSPAM-Result: Spam/)
                      ...

                      for amavisd-new banned attachments:
                      if (/^X-Amavis-Alert:\s*BANNED/)
                      ...

                      for bogofilter, one would use
                      /^X-Bogosity:\s*(\S+),.*\s+spamicity=([\d\.]+)/
                      BOGO_STATUS="${MATCH1}"
                      BOGO_SCORE="${MATCH2}"

                      and decide based on these vars (bogofilter has "unsure" result, so the
                      decision here is not binary).

                      ...

                      This may be either per-site (maildroprc) or per-user (.mailfilter).

                      PS. One can implement a "commitee" where the final disposition is a
                      function of the individual filter tags (if SA and bogo agree, you have
                      more confidence in the result, ... etc). This however needs real
                      measurements to get any justification...
                    • Jorey Bump
                      ... I tag, then reject/hold/deliver based on the score in a message header (rejecting only in a before-queue content filter). I prefer not to alter anything
                      Message 10 of 16 , Sep 1, 2006
                      • 0 Attachment
                        Wietse Venema wrote:

                        > - Have the filter insert a "badness" indicator in a message header,
                        > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                        > rules, cyrus sieves, or procmail filters?

                        I tag, then reject/hold/deliver based on the score in a message header
                        (rejecting only in a before-queue content filter). I prefer not to alter
                        anything normally visible to the user (like the subject), but they are
                        able to act on the header information of anything that gets through.

                        I use header_checks for filtering, selectively uncommenting and
                        adjusting the following:

                        # HOLD messages marked as spam by SpamAssassin, for later inspection
                        #/^X-Spam-Flag: YES/ HOLD Identified as spam by SpamAssassin.

                        # REJECT messages marked as spam by SpamAssassin
                        # Use this with a before-queue content filter, only!
                        #/^X-Spam-Flag: YES/ REJECT Identified as spam by SpamAssassin.

                        # REJECT only high scores
                        # Use this with a before-queue content filter, only!
                        /X-Spam-Level: \*{7,}/ REJECT Identified as spam by SpamAssassin.

                        # HOLD messages with score in specified range for inspection
                        /X-Spam-Level: \*{5,6}$/ HOLD Identified as spam by SpamAssassin.


                        Only the HOLD rules are relevant to FILTER_README, but I include the
                        rest because I will typically set up an after-queue filter first, then
                        configure the before-queue filter. This way, I can easily switch back to
                        the after-queue filter if there are any problems.
                      • o2 - Marcin Wasilewski
                        ... From: Wietse Venema To: Postfix users Sent: Friday, September 01, 2006 4:53 PM Subject:
                        Message 11 of 16 , Sep 4, 2006
                        • 0 Attachment
                          ----- Original Message -----
                          From: "Wietse Venema" <wietse@...>
                          To: "Postfix users" <postfix-users@...>
                          Sent: Friday, September 01, 2006 4:53 PM
                          Subject: FILTER_README suggestions


                          > Postfix's FILTER_README was written long before backscatter became
                          > a problem. The first example (see below signature) has a warning
                          > not to reject mail:
                          >
                          > Note: in this time of mail worms and spam, it is a BAD IDEA to
                          > send known viruses or spam back to the sender, because that
                          > address is likely to be forged. It is safer to discard known
                          > to be bad content and to quarantine suspicious content so that
                          > it can be inspected by a human being.
                          >
                          > Unfortunately, the text gives no example of how one would implement
                          > this advice. Personally, I use no external filter so I have a hard
                          > time coming up with field-tested examples.
                          >
                          > What do people use:
                          >
                          > - Have the filter return a distinct exit status that says "discard"?
                          >
                          > - Have the filter insert a "badness" indicator in a message header,
                          > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                          > rules, cyrus sieves, or procmail filters?
                          >
                          > - Something completely different? Maybe no-one uses the pipe+sendmail
                          > example and we can drop it from the documentation.
                          >
                          > Wietse
                          >
                          > 1 #!/bin/sh
                          > 2
                          > 3 # Simple shell-based filter. It is meant to be invoked as follows:
                          > 4 # /path/to/script -f sender recipients...
                          > 5
                          > 6 # Localize these. The -G option does nothing before Postfix 2.3.
                          > 7 INSPECT_DIR=/var/spool/filter
                          > 8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
                          > 9
                          > 10 # Exit codes from <sysexits.h>
                          > 11 EX_TEMPFAIL=75
                          > 12 EX_UNAVAILABLE=69
                          > 13
                          > 14 # Clean up when done or when aborting.
                          > 15 trap "rm -f in.$$" 0 1 2 3 15
                          > 16
                          > 17 # Start processing.
                          > 18 cd $INSPECT_DIR || {
                          > 19 echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
                          > 20
                          > 21 cat >in.$$ || {
                          > 22 echo Cannot save mail to file; exit $EX_TEMPFAIL; }
                          > 23
                          > 24 # Specify your content filter here.
                          > 25 # filter <in.$$ || {
                          > 26 # echo Message content rejected; exit $EX_UNAVAILABLE; }
                          > 27
                          > 28 $SENDMAIL "$@" <in.$$
                          > 29
                          > 30 exit $?
                          >

                          Hello,

                          what do You think about this example: sa_quarantine.sh
                          #!/bin/bash
                          #Marcin Wasilewski, 20060904
                          QUARANTINE_ABOVE=6
                          REJECT_ABOVE=15
                          SCORE=0

                          INSPECT_DIR=/proxsmtp
                          QUARANTINE_DIR=/proxsmtp/QUARANTINE
                          EX_TEMPFAIL=75

                          # Start processing.
                          cat | spamc | tee $INSPECT_DIR/in.$$ || {
                          echo -e "Cannot save mail to file"; exit $EX_TEMPFAIL; }

                          SCORE=`grep '^X-Spam-Status' $INSPECT_DIR/in.$$ | sed 's/.* score=//' | sed
                          's/ .*//' |sed 's/\..*//'`

                          if [ "$SCORE" -ge $REJECT_ABOVE ]; then
                          echo "550 Sorry, your message was flagged as spam and rejected!" >&2
                          rm -f $INSPECT_DIR/in.$$
                          exit 1
                          fi

                          if [ "$SCORE" -ge $QUARANTINE_ABOVE ]; then
                          echo "550 Sorry, your message was flagged as spam and quarantined!" >&2
                          mv $INSPECT_DIR/in.$$ $QUARANTINE_DIR/`date +%Y%m%d-%H:%M:%S`_$RANDOM.eml
                          exit 1
                          fi

                          exit 0
                          #######################
                          it is a beta verison that I just create and it works.
                          It requires: http://memberwebs.com/nielsen/software/proxsmtp/ to run
                          proxsmtpd -f proxsmtpd.conf -d 4
                          where proxsmtpd.conf is:
                          OutAddress: 10025
                          FilterCommand: /proxsmtp/sa_quarantine.sh
                          FilterType: pipe
                          Listen: 127.0.0.1:10024

                          and Postfix's master.cf should be:
                          smtp inet n - - - - smtpd
                          -o smtpd_proxy_filter=127.0.0.1:10024

                          # Re-injection after content filter
                          127.0.0.1:10025 inet n - n - - smtpd
                          -o smtpd_authorized_xforward_hosts=127.0.0.0/8
                          -o smtpd_client_restrictions=
                          -o smtpd_helo_restrictions=
                          -o smtpd_sender_restrictions=
                          -o smtpd_recipient_restrictions=permit_mynetworks,reject
                          -o smtpd_data_restrictions=
                          -o receive_override_options=no_unknown_recipient_checks

                          I didn't test it strong enough to move it to production environment but I
                          think it may be a good example.
                          If someone could look at sa_quarantine.sh and correct my sed exp. and other
                          errors - because I'm sure it can be done in better way :)

                          Best regards
                          Marcin
                        Your message has been successfully submitted and would be delivered to recipients shortly.