Loading ...
Sorry, an error occurred while loading the content.

Re: FILTER_README suggestions

Expand Messages
  • Darron Froese
    ... We reject some mail directly at SMTP (reject_unknown_sender_domain, reject_non_fqdn_sender, check_helo_access, reject_unknown_recipient_domain,
    Message 1 of 16 , Sep 1, 2006
    • 0 Attachment
      On 1-Sep-06, at 8:53 AM, Wietse Venema wrote:

      > Unfortunately, the text gives no example of how one would implement
      > this advice. Personally, I use no external filter so I have a hard
      > time coming up with field-tested examples.
      >
      > What do people use:

      We reject some mail directly at SMTP (reject_unknown_sender_domain,
      reject_non_fqdn_sender, check_helo_access,
      reject_unknown_recipient_domain, reject_unauth_destination,
      reject_unlisted_recipient and a manual domain name blacklist) but
      after that:

      1. Everything that matches a cable, DSL or dialup connection gets
      greylisted with tumgreyspf - configurable with a manual whitelist for
      broken mail servers.
      2. All mail with known viruses get silently discarded (amavisd-new
      and clamav)
      3. Mail gets tagged with SpamAssassin (through amavisd-new and a
      bunch of extra SA plugins)
      4. Clients have the option to discard above a certain SA score or
      filter through a web interface .

      I did up a diagram of it to figure it all out in my mind while I was
      building:

      http://nonfiction.ca/mail-arch.jpg

      Works great for us - brought down my spam levels from 300 / day with
      old Postfix and SA 2.6 down to about 2 or 3 / day.
      --
      darron froese
      principal
      nonfiction studios inc.
      t 403.686.8887
      c 403.819.7887
      f 403.313.9233
      w http://nonfiction.ca/
      e darron@...
    • Victor Duchovni
      ... I use badness headers, then pump all labeled mail into the hold queue of a downstream Postfix instance. From the hold queue messages are moved into
      Message 2 of 16 , Sep 1, 2006
      • 0 Attachment
        On Fri, Sep 01, 2006 at 10:53:49AM -0400, Wietse Venema wrote:

        > - Have the filter insert a "badness" indicator in a message header,
        > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
        > rules, cyrus sieves, or procmail filters?

        I use "badness" headers, then pump all labeled mail into the "hold" queue
        of a downstream Postfix instance. From the "hold" queue messages are
        moved into medium term storage (quarantine) and users receive periodic
        notices of quarantined mail, and are able to click on links to release
        selected messages. The tools in question are not publically available,
        perhaps someday...

        --
        Viktor.

        Disclaimer: off-list followups get on-list replies or get ignored.
        Please do not ignore the "Reply-To" header.

        To unsubscribe from the postfix-users list, visit
        http://www.postfix.org/lists.html or click the link below:
        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

        If my response solves your problem, the best way to thank me is to not
        send an "it worked, thanks" follow-up. If you must respond, please put
        "It worked, thanks" in the "Subject" so I can delete these quickly.
      • Tony Earnshaw
        fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema: [...] ... A couple of dead simple stanzas from a maildrop/dspam setup: in /etc/maildroprc ($VHOME
        Message 3 of 16 , Sep 1, 2006
        • 0 Attachment
          fr den 01.09.2006 Klokka 12:40 (-0400) skreiv Wietse Venema:

          [...]

          > > We use "tag & deliver" a.k.a second option. For some clients, maildrop
          > > places the mail in appropriate folder. For others, we let the MUA decide
          > > what to do with tagged mail.
          >
          > Thanks to all who recommended the "tag and deliver" approach.
          >
          > > Basically, quarantining is too much trouble for our end users and we don't
          > > discard email for fear of false positives.
          > >
          > > I suggest adding some simple maildrop/procmail recipies as examples to
          > > help the uninitiated.
          >
          > If you have examples to share, it will save me time.

          A couple of dead simple stanzas from a maildrop/dspam setup:

          in /etc/maildroprc ($VHOME is a variable defined at the head of
          maildroprc):

          `test -d $VHOME/Maildir/.dspam-quarantine/`
          if( $RETURNCODE == 1 )
          {
          `/usr/bin/maildirmake "$VHOME/Maildir/.dspam-quarantine"/`
          }
          __________________________

          if ( /^X-DSPAM-Result:[:space:]+Spam$/ )
          {
          to "$VHOME/Maildir/.dspam-quarantine"
          }

          --Tonni

          --
          Tony Earnshaw
          reservebergenser
        • Darron Froese
          ... This is a Sieve recipe that is available to our users: # Delete SpamAssassin blacklisted mails. if header :comparator i;ascii-casemap :contains
          Message 4 of 16 , Sep 1, 2006
          • 0 Attachment
            On 1-Sep-06, at 10:40 AM, Wietse Venema wrote:

            > If you have examples to share, it will save me time.

            This is a Sieve recipe that is available to our users:

            # Delete SpamAssassin blacklisted mails.
            if header :comparator "i;ascii-casemap" :contains "X-Spam-Status"
            "BLACKLISTED" {
            discard;
            stop;
            }
            # Delete high score SPAM above 14 SA score (configurable).
            if allof ( header :comparator "i;ascii-casemap" :matches "Subject"
            "[SPAM]*", header :value "gt" :comparator "i;ascii-numeric" "X-Spam-
            Score" ["14"] ) {
            discard;
            stop;
            }
            # Filter Lower Scored Spam for checking.
            if header :comparator "i;ascii-casemap" :matches "Subject" "[SPAM]*" {
            fileinto "INBOX.spam";
            stop;
            }

            It's their choice what they want to do for their own accounts - this
            is my personal setup - all managed via a web interface.
            --
            darron froese
            principal
            nonfiction studios inc.
            t 403.686.8887
            c 403.819.7887
            f 403.313.9233
            w http://nonfiction.ca/
            e darron@...
          • Eray Aslan
            ... Here is a sample maildroprc file: maildirmake=/path/to/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/ #automatically
            Message 5 of 16 , Sep 1, 2006
            • 0 Attachment
              On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
              > If you have examples to share, it will save me time.

              Here is a sample maildroprc file:

              maildirmake=/path/to/maildirmake
              MAILDIR=$DEFAULT
              JUNK_FOLDER=.Spam

              _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


              #automatically create the Junk folder
              `test -d $_JUNK_DEST`
              if ($RETURNCODE != 0 )
              {
              `$maildirmake $_JUNK_DEST`
              # auto subscribe. the following works for courier-imap
              `echo $_JUNK_DEST >> $MAILDIR/courierimapsubscribed`
              }

              # Spam gets tagged with X-Spam-Flag
              if ( /^X-Spam-Flag: YES/:h )
              {
              exception {
              to "$_JUNK_DEST"
              }
              }
              else
              {
              exception {
              to "$MAILDIR/"
              }
              }

              --
              Eray
            • Aaron Bennett
              ... I can t speak highly enough of Maia Mailguard. It s based on amavisd-new but allows per-user bayes training, white and blacklists. We re a middling-volume
              Message 6 of 16 , Sep 1, 2006
              • 0 Attachment
                Wietse Venema wrote:
                > What do people use:
                >
                >

                I can't speak highly enough of Maia Mailguard. It's based on
                amavisd-new but allows per-user bayes training, white and blacklists.
                We're a middling-volume site (about 85K messages per day) and have two
                dual-xeon mail relays running Maia and uvscan/clamd. The database
                component of Maia sits on a third box. In our older environment, we had
                amavisd-new + spamassassin running without per-user training or
                quarantining or the other benefits of Maia and as a result, we had to be
                conservative with our SA thresholds to avoid false-positives. Our
                number 1 complaint was spam -- since we list all or most staff and
                faculty email on our web page. Now with Maia, we are heroes. Since
                mid-july when we rolled it out, we've blocked over 600,000 spams with a
                false positive rate of 0.06% and a false-negative rate of 6.57%. The
                power is the user-training; it's easy and simple and lets users take
                effective action against spam. It really lets SpamAssassin shine.



                --
                Aaron Bennett
                Sr. Unix Systems Administrator
                Clark University ITS
                abennett@... | 508.781.7315
              • Eray Aslan
                ... For the record, here is a working maildroprc file. Please disregard the previous one. maildirmake=/usr/bin/maildirmake MAILDIR=$DEFAULT JUNK_FOLDER=.Spam
                Message 7 of 16 , Sep 1, 2006
                • 0 Attachment
                  On Fri, September 1, 2006 8:30 pm, Eray Aslan wrote:
                  > On Fri, September 1, 2006 7:40 pm, Wietse Venema wrote:
                  >> If you have examples to share, it will save me time.
                  >
                  > Here is a sample maildroprc file:

                  For the record, here is a working maildroprc file. Please disregard the
                  previous one.


                  maildirmake=/usr/bin/maildirmake
                  MAILDIR=$DEFAULT
                  JUNK_FOLDER=.Spam

                  _JUNK_DEST=$MAILDIR/$JUNK_FOLDER/


                  #automatically create the Junk folder
                  `test -d $_JUNK_DEST`
                  if ($RETURNCODE != 0 )
                  {
                  `$maildirmake $_JUNK_DEST`
                  # auto subscribe. the following works for courier-imap
                  `echo INBOX$JUNK_FOLDER >> $MAILDIR/courierimapsubscribed`
                  }
                  # Spam gets tagged with X-Spam-Flag
                  if ( /^X-Spam-Flag: YES/:h )
                  {
                  exception {
                  to "$_JUNK_DEST"
                  }
                  }
                  else
                  {
                  exception {
                  to "$MAILDIR/"
                  }
                  }

                  --
                  Eray
                • mouss
                  ... I favour tag & deliver, be that to a Junk folder or somewhere else. So I configure the filters to tag all mail (spam or not), and use these in maildrop to
                  Message 8 of 16 , Sep 1, 2006
                  • 0 Attachment
                    Wietse Venema wrote:
                    > Postfix's FILTER_README was written long before backscatter became
                    > a problem. The first example (see below signature) has a warning
                    > not to reject mail:
                    >
                    > Note: in this time of mail worms and spam, it is a BAD IDEA to
                    > send known viruses or spam back to the sender, because that
                    > address is likely to be forged. It is safer to discard known
                    > to be bad content and to quarantine suspicious content so that
                    > it can be inspected by a human being.
                    >
                    > Unfortunately, the text gives no example of how one would implement
                    > this advice. Personally, I use no external filter so I have a hard
                    > time coming up with field-tested examples.
                    >
                    > What do people use:
                    >
                    > - Have the filter return a distinct exit status that says "discard"?
                    >
                    > - Have the filter insert a "badness" indicator in a message header,
                    > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                    > rules, cyrus sieves, or procmail filters?
                    >
                    > - Something completely different? Maybe no-one uses the pipe+sendmail
                    > example and we can drop it from the documentation.
                    >
                    >

                    I favour tag & deliver, be that to a Junk folder or somewhere else. So I
                    configure the filters to tag all mail (spam or not), and use these in
                    maildrop to deliver to a Junk folder.

                    here is an example with spamassassin + courier-imap (the .folder
                    notation below) + maildrop 2.x

                    if (/^X-Spam-Flag:\s*YES/)
                    {
                    exception {
                    to "$DEFAULT/.Junk/";
                    }
                    }

                    with dspam, this would be
                    if (/^X-DSPAM-Result: Spam/)
                    ...

                    for amavisd-new banned attachments:
                    if (/^X-Amavis-Alert:\s*BANNED/)
                    ...

                    for bogofilter, one would use
                    /^X-Bogosity:\s*(\S+),.*\s+spamicity=([\d\.]+)/
                    BOGO_STATUS="${MATCH1}"
                    BOGO_SCORE="${MATCH2}"

                    and decide based on these vars (bogofilter has "unsure" result, so the
                    decision here is not binary).

                    ...

                    This may be either per-site (maildroprc) or per-user (.mailfilter).

                    PS. One can implement a "commitee" where the final disposition is a
                    function of the individual filter tags (if SA and bogo agree, you have
                    more confidence in the result, ... etc). This however needs real
                    measurements to get any justification...
                  • Jorey Bump
                    ... I tag, then reject/hold/deliver based on the score in a message header (rejecting only in a before-queue content filter). I prefer not to alter anything
                    Message 9 of 16 , Sep 1, 2006
                    • 0 Attachment
                      Wietse Venema wrote:

                      > - Have the filter insert a "badness" indicator in a message header,
                      > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                      > rules, cyrus sieves, or procmail filters?

                      I tag, then reject/hold/deliver based on the score in a message header
                      (rejecting only in a before-queue content filter). I prefer not to alter
                      anything normally visible to the user (like the subject), but they are
                      able to act on the header information of anything that gets through.

                      I use header_checks for filtering, selectively uncommenting and
                      adjusting the following:

                      # HOLD messages marked as spam by SpamAssassin, for later inspection
                      #/^X-Spam-Flag: YES/ HOLD Identified as spam by SpamAssassin.

                      # REJECT messages marked as spam by SpamAssassin
                      # Use this with a before-queue content filter, only!
                      #/^X-Spam-Flag: YES/ REJECT Identified as spam by SpamAssassin.

                      # REJECT only high scores
                      # Use this with a before-queue content filter, only!
                      /X-Spam-Level: \*{7,}/ REJECT Identified as spam by SpamAssassin.

                      # HOLD messages with score in specified range for inspection
                      /X-Spam-Level: \*{5,6}$/ HOLD Identified as spam by SpamAssassin.


                      Only the HOLD rules are relevant to FILTER_README, but I include the
                      rest because I will typically set up an after-queue filter first, then
                      configure the before-queue filter. This way, I can easily switch back to
                      the after-queue filter if there are any problems.
                    • o2 - Marcin Wasilewski
                      ... From: Wietse Venema To: Postfix users Sent: Friday, September 01, 2006 4:53 PM Subject:
                      Message 10 of 16 , Sep 4, 2006
                      • 0 Attachment
                        ----- Original Message -----
                        From: "Wietse Venema" <wietse@...>
                        To: "Postfix users" <postfix-users@...>
                        Sent: Friday, September 01, 2006 4:53 PM
                        Subject: FILTER_README suggestions


                        > Postfix's FILTER_README was written long before backscatter became
                        > a problem. The first example (see below signature) has a warning
                        > not to reject mail:
                        >
                        > Note: in this time of mail worms and spam, it is a BAD IDEA to
                        > send known viruses or spam back to the sender, because that
                        > address is likely to be forged. It is safer to discard known
                        > to be bad content and to quarantine suspicious content so that
                        > it can be inspected by a human being.
                        >
                        > Unfortunately, the text gives no example of how one would implement
                        > this advice. Personally, I use no external filter so I have a hard
                        > time coming up with field-tested examples.
                        >
                        > What do people use:
                        >
                        > - Have the filter return a distinct exit status that says "discard"?
                        >
                        > - Have the filter insert a "badness" indicator in a message header,
                        > and dispose of bad mail with Postfix HOLD/DISCARD actions, maildrop
                        > rules, cyrus sieves, or procmail filters?
                        >
                        > - Something completely different? Maybe no-one uses the pipe+sendmail
                        > example and we can drop it from the documentation.
                        >
                        > Wietse
                        >
                        > 1 #!/bin/sh
                        > 2
                        > 3 # Simple shell-based filter. It is meant to be invoked as follows:
                        > 4 # /path/to/script -f sender recipients...
                        > 5
                        > 6 # Localize these. The -G option does nothing before Postfix 2.3.
                        > 7 INSPECT_DIR=/var/spool/filter
                        > 8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
                        > 9
                        > 10 # Exit codes from <sysexits.h>
                        > 11 EX_TEMPFAIL=75
                        > 12 EX_UNAVAILABLE=69
                        > 13
                        > 14 # Clean up when done or when aborting.
                        > 15 trap "rm -f in.$$" 0 1 2 3 15
                        > 16
                        > 17 # Start processing.
                        > 18 cd $INSPECT_DIR || {
                        > 19 echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
                        > 20
                        > 21 cat >in.$$ || {
                        > 22 echo Cannot save mail to file; exit $EX_TEMPFAIL; }
                        > 23
                        > 24 # Specify your content filter here.
                        > 25 # filter <in.$$ || {
                        > 26 # echo Message content rejected; exit $EX_UNAVAILABLE; }
                        > 27
                        > 28 $SENDMAIL "$@" <in.$$
                        > 29
                        > 30 exit $?
                        >

                        Hello,

                        what do You think about this example: sa_quarantine.sh
                        #!/bin/bash
                        #Marcin Wasilewski, 20060904
                        QUARANTINE_ABOVE=6
                        REJECT_ABOVE=15
                        SCORE=0

                        INSPECT_DIR=/proxsmtp
                        QUARANTINE_DIR=/proxsmtp/QUARANTINE
                        EX_TEMPFAIL=75

                        # Start processing.
                        cat | spamc | tee $INSPECT_DIR/in.$$ || {
                        echo -e "Cannot save mail to file"; exit $EX_TEMPFAIL; }

                        SCORE=`grep '^X-Spam-Status' $INSPECT_DIR/in.$$ | sed 's/.* score=//' | sed
                        's/ .*//' |sed 's/\..*//'`

                        if [ "$SCORE" -ge $REJECT_ABOVE ]; then
                        echo "550 Sorry, your message was flagged as spam and rejected!" >&2
                        rm -f $INSPECT_DIR/in.$$
                        exit 1
                        fi

                        if [ "$SCORE" -ge $QUARANTINE_ABOVE ]; then
                        echo "550 Sorry, your message was flagged as spam and quarantined!" >&2
                        mv $INSPECT_DIR/in.$$ $QUARANTINE_DIR/`date +%Y%m%d-%H:%M:%S`_$RANDOM.eml
                        exit 1
                        fi

                        exit 0
                        #######################
                        it is a beta verison that I just create and it works.
                        It requires: http://memberwebs.com/nielsen/software/proxsmtp/ to run
                        proxsmtpd -f proxsmtpd.conf -d 4
                        where proxsmtpd.conf is:
                        OutAddress: 10025
                        FilterCommand: /proxsmtp/sa_quarantine.sh
                        FilterType: pipe
                        Listen: 127.0.0.1:10024

                        and Postfix's master.cf should be:
                        smtp inet n - - - - smtpd
                        -o smtpd_proxy_filter=127.0.0.1:10024

                        # Re-injection after content filter
                        127.0.0.1:10025 inet n - n - - smtpd
                        -o smtpd_authorized_xforward_hosts=127.0.0.0/8
                        -o smtpd_client_restrictions=
                        -o smtpd_helo_restrictions=
                        -o smtpd_sender_restrictions=
                        -o smtpd_recipient_restrictions=permit_mynetworks,reject
                        -o smtpd_data_restrictions=
                        -o receive_override_options=no_unknown_recipient_checks

                        I didn't test it strong enough to move it to production environment but I
                        think it may be a good example.
                        If someone could look at sa_quarantine.sh and correct my sed exp. and other
                        errors - because I'm sure it can be done in better way :)

                        Best regards
                        Marcin
                      Your message has been successfully submitted and would be delivered to recipients shortly.