Loading ...
Sorry, an error occurred while loading the content.

Re: FILTER_README suggestions

Expand Messages
  • Darron Froese
    ... We reject some mail directly at SMTP (reject_unknown_sender_domain, reject_non_fqdn_sender, check_helo_access, reject_unknown_recipient_domain,
    Message 1 of 16 , Sep 1, 2006
    • 0 Attachment
      On 1-Sep-06, at 8:53 AM, Wietse Venema wrote:

      > Unfortunately, the text gives no example of how one would implement
      > this advice. Personally, I use no external filter so I have a hard
      > time coming up with field-tested examples.
      >
      > What do people use:

      We reject some mail directly at SMTP (reject_unknown_sender_domain,
      reject_non_fqdn_sender, check_helo_access,
      reject_unknown_recipient_domain, reject_unauth_destination,
      reject_unlisted_recipient and a manual domain name blacklist) but
      after that:

      1. Everything that matches a cable, DSL or dialup connection gets
      greylisted with tumgreyspf - configurable with a manual whitelist for
      broken mail servers.
      2. All mail with known viruses get silently discarded (amavisd-new
      and clamav)
      3. Mail gets tagged with SpamAssassin (through amavisd-new and a
      bunch of extra SA plugins)
      4. Clients have the option to discard above a certain SA score or
      filter through a web interface .

      I did up a diagram of it to figure it all out in my mind while I was
      building:

      http://nonfiction.ca/mail-arch.jpg

      Works great for us - brought down my spam levels from 300 / day with
      old Postfix and SA 2.6 down to about 2 or 3 / day.
      --
      darron froese
      principal
      nonfiction studios inc.
      t 403.686.8887
      c 403.819.7887
      f 403.313.9233
      w http://nonfiction.ca/
      e darron@...
    • Tony Earnshaw
      fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema: [...] ... A couple of dead simple stanzas from a maildrop/dspam setup: in /etc/maildroprc ($VHOME
      Message 2 of 16 , Sep 1, 2006
      • 0 Attachment
        fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema:

        [...]

        > > We use "tag & deliver" a.k.a second option. For some clients, maildrop
        > > places the mail in appropriate folder. For others, we let the MUA decide
        > > what to do with tagged mail.
        >
        > Thanks to all who recommended the "tag and deliver" approach.
        >
        > > Basically, quarantining is too much trouble for our end users and we don't
        > > discard email for fear of false positives.
        > >
        > > I suggest adding some simple maildrop/procmail recipies as examples to
        > > help the uninitiated.
        >
        > If you have examples to share, it will save me time.

        A couple of dead simple stanzas from a maildrop/dspam setup:

        in /etc/maildroprc ($VHOME is a variable defined at the head of
        maildroprc):

        `test -d $VHOME/Maildir/.dspam-quarantine/`
        if( $RETURNCODE == 1 )
        {
        `/usr/bin/maildirmake "$VHOME/Maildir/.dspam-quarantine"/`
        }
        __________________________

        if ( /^X-DSPAM-Result:[:space:]+Spam$/ )
        {
        to "$VHOME/Maildir/.dspam-quarantine"
        }

        --Tonni

        --
        Tony Earnshaw
        reservebergenser
      • Darron Froese
        ... This is a Sieve recipe that is available to our users: # Delete SpamAssassin blacklisted mails. if header :comparator i;ascii-casemap :contains
        Message 3 of 16 , Sep 1, 2006
        • 0 Attachment
          On 1-Sep-06, at 10:40 AM, Wietse Venema wrote:

          > If you have examples to share, it will save me time.

          This is a Sieve recipe that is available to our users:

          # Delete SpamAssassin blacklisted mails.
          if header :comparator "i;ascii-casemap" :contains "X-Spam-Status"
          "BLACKLISTED" {
          discard;
          stop;
          }
          # Delete high score SPAM above 14 SA score (configurable).
          if allof ( header :comparator "i;ascii-casemap" :matches "Subject"
          "[SPAM]*", header :value "gt" :comparator "i;ascii-numeric" "X-Spam-
          Score" ["14"] ) {
          discard;
          stop;
          }
          # Filter Lower Scored Spam for checking.
          if header :comparator "i;ascii-casemap" :matches "Subject" "[SPAM]*" {
          fileinto "INBOX.spam";
          stop;
          }

          It's their choice what they want to do for their own accounts - this
          is my personal setup - all managed via a web interface.
          --
          darron froese
          principal
          nonfiction studios inc.
          t 403.686.8887
          c 403.819.7887
          f 403.313.9233
          w http://nonfiction.ca/
          e darron@...
        • Eray Aslan
          ... Here is a sample maildroprc file: maildirmake=/path/to/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/ #automatically
          Message 4 of 16 , Sep 1, 2006
          • 0 Attachment
            On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
            > If you have examples to share, it will save me time.

            Here is a sample maildroprc file:

            maildirmake=/path/to/maildirmake
            MAILDIR=$DEFAULT
            JUNK_FOLDER=.Spam

            _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


            #automatically create the Junk folder
            `test -d $_JUNK_DEST`
            if ($RETURNCODE != 0 )
            {
            `$maildirmake $_JUNK_DEST`
            # auto subscribe. the following works for courier-imap
            `echo $_JUNK_DEST >> $MAILDIR/courierimapsubscribed`
            }

            # Spam gets tagged with X-Spam-Flag
            if ( /^X-Spam-Flag: YES/:h )
            {
            exception {
            to "$_JUNK_DEST"
            }
            }
            else
            {
            exception {
            to "$MAILDIR/"
            }
            }

            --
            Eray
          • Aaron Bennett
            ... I can t speak highly enough of Maia Mailguard. It s based on amavisd-new but allows per-user bayes training, white and blacklists. We re a middling-volume
            Message 5 of 16 , Sep 1, 2006
            • 0 Attachment
              Wietse Venema wrote:
              > What do people use:
              >
              >

              I can't speak highly enough of Maia Mailguard. It's based on
              amavisd-new but allows per-user bayes training, white and blacklists.
              We're a middling-volume site (about 85K messages per day) and have two
              dual-xeon mail relays running Maia and uvscan/clamd. The database
              component of Maia sits on a third box. In our older environment, we had
              amavisd-new + spamassassin running without per-user training or
              quarantining or the other benefits of Maia and as a result, we had to be
              conservative with our SA thresholds to avoid false-positives. Our
              number 1 complaint was spam -- since we list all or most staff and
              faculty email on our web page. Now with Maia, we are heroes. Since
              mid-july when we rolled it out, we've blocked over 600,000 spams with a
              false positive rate of 0.06% and a false-negative rate of 6.57%. The
              power is the user-training; it's easy and simple and lets users take
              effective action against spam. It really lets SpamAssassin shine.



              --
              Aaron Bennett
              Sr. Unix Systems Administrator
              Clark University ITS
              abennett@... | 508.781.7315
            • Eray Aslan
              ... For the record, here is a working maildroprc file. Please disregard the previous one. maildirmake=/usr/bin/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam
              Message 6 of 16 , Sep 1, 2006
              • 0 Attachment
                On Fri, September 1, 2006 8:30 pm, Eray Aslan wrote:
                > On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
                >> If you have examples to share, it will save me time.
                >
                > Here is a sample maildroprc file:

                For the record, here is a working maildroprc file. Please disregard the
                previous one.


                maildirmake=/usr/bin/maildirmake
                MAILDIR=$DEFAULT
                JUNK_FOLDER=.Spam

                _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


                #automatically create the Junk folder
                `test -d $_JUNK_DEST`
                if ($RETURNCODE != 0 )
                {
                `$maildirmake $_JUNK_DEST`
                # auto subscribe. the following works for courier-imap
                `echo INBOX$JUNK_FOLDER >> $MAILDIR/courierimapsubscribed`
                }
                # Spam gets tagged with X-Spam-Flag
                if ( /^X-Spam-Flag: YES/:h )
                {
                exception {
                to "$_JUNK_DEST"
                }
                }
                else
                {
                exception {
                to "$MAILDIR/"
                }
                }

                --
                Eray
              • mouss
                ... I favour tag & deliver, be that to a Junk folder or somewhere else. So I configure the filters to tag all mail (spam or not), and use these in maildrop to
                Message 7 of 16 , Sep 1, 2006
                • 0 Attachment
                  Wietse Venema wrote:
                  > Postfix's FILTER_README was written long before backscatter became
                  > a problem. The first example (see below signature) has a warning
                  > not to reject mail:
                  >
                  > Note: in this time of mail worms and spam, it is a BAD IDEA to
                  > send known viruses or spam back to the sender, because that
                  > address is likely to be forged. It is safer to discard known
                  > to be bad content and to quarantine suspicious content so that
                  > it can be inspected by a human being.
                  >
                  > Unfortunately, the text gives no example of how one would implement
                  > this advice. Personally, I use no external filter so I have a hard
                  > time coming up with field-tested examples.
                  >
                  > What do people use:
                  >
                  > - Have the filter return a distinct exit status that says "discard"?
                  >
                  > - Have the filter insert a "badness" indicator in a message header,
                  > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                  > rules, cyrus sieves, or procmail filters?
                  >
                  > - Something completely different? Maybe no-one uses the pipe+sendmail
                  > example and we can drop it from the documentation.
                  >
                  >

                  I favour tag & deliver, be that to a Junk folder or somewhere else. So I
                  configure the filters to tag all mail (spam or not), and use these in
                  maildrop to deliver to a Junk folder.

                  here is an example with spamassassin + courier-imap (the .folder
                  notation below) + maildrop 2.x

                  if (/^X-Spam-Flag:\s*YES/)
                  {
                  exception {
                  to "$DEFAULT/.Junk/";
                  }
                  }

                  with dspam, this would be
                  if (/^X-DSPAM-Result: Spam/)
                  ...

                  for amavisd-new banned attachments:
                  if (/^X-Amavis-Alert:\s*BANNED/)
                  ...

                  for bogofilter, one would use
                  /^X-Bogosity:\s*(\S+),.*\s+spamicity=([\d\.]+)/
                  BOGO_STATUS="${MATCH1}"
                  BOGO_SCORE="${MATCH2}"

                  and decide based on these vars (bogofilter has "unsure" result, so the
                  decision here is not binary).

                  ...

                  This may be either per-site (maildroprc) or per-user (.mailfilter).

                  PS. One can implement a "commitee" where the final disposition is a
                  function of the individual filter tags (if SA and bogo agree, you have
                  more confidence in the result, ... etc). This however needs real
                  measurements to get any justification...
                • Jorey Bump
                  ... I tag, then reject/hold/deliver based on the score in a message header (rejecting only in a before-queue content filter). I prefer not to alter anything
                  Message 8 of 16 , Sep 1, 2006
                  • 0 Attachment
                    Wietse Venema wrote:

                    > - Have the filter insert a "badness" indicator in a message header,
                    > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                    > rules, cyrus sieves, or procmail filters?

                    I tag, then reject/hold/deliver based on the score in a message header
                    (rejecting only in a before-queue content filter). I prefer not to alter
                    anything normally visible to the user (like the subject), but they are
                    able to act on the header information of anything that gets through.

                    I use header_checks for filtering, selectively uncommenting and
                    adjusting the following:

                    # HOLD messages marked as spam by SpamAssassin, for later inspection
                    #/^X-Spam-Flag: YES/ HOLD Identified as spam by SpamAssassin.

                    # REJECT messages marked as spam by SpamAssassin
                    # Use this with a before-queue content filter, only!
                    #/^X-Spam-Flag: YES/ REJECT Identified as spam by SpamAssassin.

                    # REJECT only high scores
                    # Use this with a before-queue content filter, only!
                    /X-Spam-Level: \*{7,}/ REJECT Identified as spam by SpamAssassin.

                    # HOLD messages with score in specified range for inspection
                    /X-Spam-Level: \*{5,6}$/ HOLD Identified as spam by SpamAssassin.


                    Only the HOLD rules are relevant to FILTER_README, but I include the
                    rest because I will typically set up an after-queue filter first, then
                    configure the before-queue filter. This way, I can easily switch back to
                    the after-queue filter if there are any problems.
                  • o2 - Marcin Wasilewski
                    ... From: Wietse Venema To: Postfix users Sent: Friday, September 01, 2006 4:53 PM Subject:
                    Message 9 of 16 , Sep 4, 2006
                    • 0 Attachment
                      ----- Original Message -----
                      From: "Wietse Venema" <wietse@...>
                      To: "Postfix users" <postfix-users@...>
                      Sent: Friday, September 01, 2006 4:53 PM
                      Subject: FILTER_README suggestions


                      > Postfix's FILTER_README was written long before backscatter became
                      > a problem. The first example (see below signature) has a warning
                      > not to reject mail:
                      >
                      > Note: in this time of mail worms and spam, it is a BAD IDEA to
                      > send known viruses or spam back to the sender, because that
                      > address is likely to be forged. It is safer to discard known
                      > to be bad content and to quarantine suspicious content so that
                      > it can be inspected by a human being.
                      >
                      > Unfortunately, the text gives no example of how one would implement
                      > this advice. Personally, I use no external filter so I have a hard
                      > time coming up with field-tested examples.
                      >
                      > What do people use:
                      >
                      > - Have the filter return a distinct exit status that says "discard"?
                      >
                      > - Have the filter insert a "badness" indicator in a message header,
                      > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                      > rules, cyrus sieves, or procmail filters?
                      >
                      > - Something completely different? Maybe no-one uses the pipe+sendmail
                      > example and we can drop it from the documentation.
                      >
                      > Wietse
                      >
                      > 1 #!/bin/sh
                      > 2
                      > 3 # Simple shell-based filter. It is meant to be invoked as follows:
                      > 4 # /path/to/script -f sender recipients...
                      > 5
                      > 6 # Localize these. The -G option does nothing before Postfix 2.3.
                      > 7 INSPECT_DIR=/var/spool/filter
                      > 8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
                      > 9
                      > 10 # Exit codes from <sysexits.h>
                      > 11 EX_TEMPFAIL=75
                      > 12 EX_UNAVAILABLE=69
                      > 13
                      > 14 # Clean up when done or when aborting.
                      > 15 trap "rm -f in.$$" 0 1 2 3 15
                      > 16
                      > 17 # Start processing.
                      > 18 cd $INSPECT_DIR || {
                      > 19 echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
                      > 20
                      > 21 cat >in.$$ || {
                      > 22 echo Cannot save mail to file; exit $EX_TEMPFAIL; }
                      > 23
                      > 24 # Specify your content filter here.
                      > 25 # filter <in.$$ || {
                      > 26 # echo Message content rejected; exit $EX_UNAVAILABLE; }
                      > 27
                      > 28 $SENDMAIL "$@" <in.$$
                      > 29
                      > 30 exit $?
                      >

                      Hello,

                      what do You think about this example: sa_quarantine.sh
                      #!/bin/bash
                      #Marcin Wasilewski, 20060904
                      QUARANTINE_ABOVE=6
                      REJECT_ABOVE=15
                      SCORE=0

                      INSPECT_DIR=/proxsmtp
                      QUARANTINE_DIR=/proxsmtp/QUARANTINE
                      EX_TEMPFAIL=75

                      # Start processing.
                      cat | spamc | tee $INSPECT_DIR/in.$$ || {
                      echo -e "Cannot save mail to file"; exit $EX_TEMPFAIL; }

                      SCORE=`grep '^X-Spam-Status' $INSPECT_DIR/in.$$ | sed 's/.* score=//' | sed
                      's/ .*//' |sed 's/\..*//'`

                      if [ "$SCORE" -ge $REJECT_ABOVE ]; then
                      echo "550 Sorry, your message was flagged as spam and rejected!" >&2
                      rm -f $INSPECT_DIR/in.$$
                      exit 1
                      fi

                      if [ "$SCORE" -ge $QUARANTINE_ABOVE ]; then
                      echo "550 Sorry, your message was flagged as spam and quarantined!" >&2
                      mv $INSPECT_DIR/in.$$ $QUARANTINE_DIR/`date +%Y%m%d-%H:%M:%S`_$RANDOM.eml
                      exit 1
                      fi

                      exit 0
                      #######################
                      it is a beta verison that I just create and it works.
                      It requires: http://memberwebs.com/nielsen/software/proxsmtp/ to run
                      proxsmtpd -f proxsmtpd.conf -d 4
                      where proxsmtpd.conf is:
                      OutAddress: 10025
                      FilterCommand: /proxsmtp/sa_quarantine.sh
                      FilterType: pipe
                      Listen: 127.0.0.1:10024

                      and Postfix's master.cf should be:
                      smtp inet n - - - - smtpd
                      -o smtpd_proxy_filter=127.0.0.1:10024

                      # Re-injection after content filter
                      127.0.0.1:10025 inet n - n - - smtpd
                      -o smtpd_authorized_xforward_hosts=127.0.0.0/8
                      -o smtpd_client_restrictions=
                      -o smtpd_helo_restrictions=
                      -o smtpd_sender_restrictions=
                      -o smtpd_recipient_restrictions=permit_mynetworks,reject
                      -o smtpd_data_restrictions=
                      -o receive_override_options=no_unknown_recipient_checks

                      I didn't test it strong enough to move it to production environment but I
                      think it may be a good example.
                      If someone could look at sa_quarantine.sh and correct my sed exp. and other
                      errors - because I'm sure it can be done in better way :)

                      Best regards
                      Marcin
                    Your message has been successfully submitted and would be delivered to recipients shortly.