Loading ...
Sorry, an error occurred while loading the content.
 

pcre problem

Expand Messages
  • Jürgen Herrmann
    Hi there! I have a problem with the following pcre table: if !/X-Spam-Level:.* *{7,}/ /X-Spam-Flag:.*YES/ HOLD endif /X-Spam-Level:.* *{7,}/ DISCARD the HOLD
    Message 1 of 11 , Jun 18, 2014
      Hi there!

      I have a problem with the following pcre table:

      if !/X-Spam-Level:.*\*{7,}/
      /X-Spam-Flag:.*YES/ HOLD
      endif
      /X-Spam-Level:.*\*{7,}/ DISCARD

      the HOLD part of it is executed. the DISCARD part never matches.

      I also had a version with
      /X-Spam-Level: \*{7,}/
      instead of
      /X-Spam-Level:.*\*{7,}/

      and also the following version does never discard any mails:

      /X-Spam-Flag:.*YES/ HOLD
      /X-Spam-Level:.*\*{7,}/ DISCARD

      (I added the if/endif in case HOLD takes precedence over DISCARD)

      As i mentioned, I'm sure the table itself is matched against, as
      mails land on the HOLD queue. But mails with spamassassin score >= 7
      do not get discarded.

      I'm pretty lost at this point :) any hints, anybody?

      Best regards and thanks in advance,
      Jürgen Herrmann
      --
      >> XLhost.de ® - Webhosting von supersmall bis eXtra Large <<

      XLhost.de GmbH
      Jürgen Herrmann, Geschäftsführer
      Boelckestrasse 21, 93051 Regensburg, Germany

      Geschäftsführer: Jürgen Herrmann
      Registriert unter: HRB9918
      Umsatzsteuer-Identifikationsnummer: DE245931218

      Fon: +49 (0)800 XLHOSTDE [0800 95467833]
      Fax: +49 (0)800 95467830
      Web: http://www.XLhost.de
    • Wietse Venema
      ... As documented in pcre_table(5) and other places: if /pattern/flags endif Match the input string against the patterns between if and endif, if and
      Message 2 of 11 , Jun 18, 2014
        J?rgen Herrmann:
        > Hi there!
        >
        > I have a problem with the following pcre table:
        >
        > if !/X-Spam-Level:.*\*{7,}/
        > /X-Spam-Flag:.*YES/ HOLD
        > endif

        As documented in pcre_table(5) and other places:

        if /pattern/flags

        endif Match the input string against the patterns between if and
        endif, if and only if THAT SAME INPUT STRING also matches pat-
        tern. The if..endif can nest.

        In other words, "/X-Spam-Flag:.*YES/ HOLD" executes always.

        Second, your patterns will match "X-Spam-Flag:.*YES" in the middle
        of a line. use /^X-Spam.../ instead.

        Wietse
      • lists@rhsoft.net
        ... are you aware that you *must not* answer with 250 OK and the silently discard messages? the spamfilter belongs *pre-queue* and you have to REJECT spam,
        Message 3 of 11 , Jun 18, 2014
          Am 18.06.2014 15:49, schrieb Jürgen Herrmann:
          > I have a problem with the following pcre table:
          >
          > if !/X-Spam-Level:.*\*{7,}/
          > /X-Spam-Flag:.*YES/ HOLD
          > endif
          > /X-Spam-Level:.*\*{7,}/ DISCARD
          >
          > the HOLD part of it is executed. the DISCARD part never matches.
          >
          > I also had a version with
          > /X-Spam-Level: \*{7,}/
          > instead of
          > /X-Spam-Level:.*\*{7,}/
          >
          > and also the following version does never discard any mails:
          >
          > /X-Spam-Flag:.*YES/ HOLD
          > /X-Spam-Level:.*\*{7,}/ DISCARD

          are you aware that you *must not* answer with 250 OK and the
          silently discard messages? the spamfilter belongs *pre-queue*
          and you have to REJECT spam, after you receive it and answer
          with "250 OK" you have to deliver the message - period

          http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
          "and the ability to use before-queue content filtering"

          fix your setup and get rid of the broken pcre-after-queue idea
        • Jürgen Herrmann
          ... Hmm, i have a ! before the if-pattern. if !/pattern/flags endif Match the input string against the patterns between if and endif, if and only if that
          Message 4 of 11 , Jun 18, 2014
            Am 18.06.2014 15:58, schrieb wietse@...:
            > J?rgen Herrmann:
            >> Hi there!
            >>
            >> I have a problem with the following pcre table:
            >>
            >> if !/X-Spam-Level:.*\*{7,}/
            >> /X-Spam-Flag:.*YES/ HOLD
            >> endif
            >
            > As documented in pcre_table(5) and other places:
            >
            > if /pattern/flags
            >
            > endif Match the input string against the patterns between if
            > and
            > endif, if and only if THAT SAME INPUT STRING also matches pat-
            > tern. The if..endif can nest.
            >
            > In other words, "/X-Spam-Flag:.*YES/ HOLD" executes always.

            Hmm, i have a ! before the if-pattern.

            if !/pattern/flags

            endif Match the input string against the patterns between
            if and
            endif, if and only if that same input string does not
            match pat-
            tern. The if..endif can nest.

            I read that like this:
            /X-Spam-Flag:.*YES/ HOLD
            is only executed if
            /X-Spam-Level:.*\*{7,}/
            DOS NOT MATCH - correct?

            > Second, your patterns will match "X-Spam-Flag:.*YES" in the middle
            > of a line. use /^X-Spam.../ instead.
            >
            > Wietse

            OK, will do that.

            Jürgen
            --
            >> XLhost.de ® - Webhosting von supersmall bis eXtra Large <<

            XLhost.de GmbH
            Jürgen Herrmann, Geschäftsführer
            Boelckestrasse 21, 93051 Regensburg, Germany

            Geschäftsführer: Jürgen Herrmann
            Registriert unter: HRB9918
            Umsatzsteuer-Identifikationsnummer: DE245931218

            Fon: +49 (0)800 XLHOSTDE [0800 95467833]
            Fax: +49 (0)800 95467830
            Web: http://www.XLhost.de
          • Jürgen Herrmann
            ... this is in a milter_header_checks and on a host used as relay host. so if i reject mail there it would create backscatter via the relaying host. why not
            Message 5 of 11 , Jun 18, 2014
              Am 18.06.2014 15:59, schrieb lists@...:
              > Am 18.06.2014 15:49, schrieb Jürgen Herrmann:
              >> I have a problem with the following pcre table:
              >>
              >> if !/X-Spam-Level:.*\*{7,}/
              >> /X-Spam-Flag:.*YES/ HOLD
              >> endif
              >> /X-Spam-Level:.*\*{7,}/ DISCARD
              >>
              >> the HOLD part of it is executed. the DISCARD part never matches.
              >>
              >> I also had a version with
              >> /X-Spam-Level: \*{7,}/
              >> instead of
              >> /X-Spam-Level:.*\*{7,}/
              >>
              >> and also the following version does never discard any mails:
              >>
              >> /X-Spam-Flag:.*YES/ HOLD
              >> /X-Spam-Level:.*\*{7,}/ DISCARD
              >
              > are you aware that you *must not* answer with 250 OK and the
              > silently discard messages? the spamfilter belongs *pre-queue*
              > and you have to REJECT spam, after you receive it and answer
              > with "250 OK" you have to deliver the message - period
              >
              > http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
              > "and the ability to use before-queue content filtering"
              >
              > fix your setup and get rid of the broken pcre-after-queue idea

              this is in a milter_header_checks and on a host used as relay
              host. so if i reject mail there it would create backscatter via
              the relaying host. why not discard mails that are surely spam?

              i can only see a solution to this problem when installing SA
              on every mailing host, which is what i wanted to avoid.

              Jürgen
              --
              >> XLhost.de ® - Webhosting von supersmall bis eXtra Large <<

              XLhost.de GmbH
              Jürgen Herrmann, Geschäftsführer
              Boelckestrasse 21, 93051 Regensburg, Germany

              Geschäftsführer: Jürgen Herrmann
              Registriert unter: HRB9918
              Umsatzsteuer-Identifikationsnummer: DE245931218

              Fon: +49 (0)800 XLHOSTDE [0800 95467833]
              Fax: +49 (0)800 95467830
              Web: http://www.XLhost.de
            • Wietse Venema
              ... You are NOT READING my reply. The if !/X-Spam-Level/ always succeeds for headers with /X-Spam-Flag/ Wietse
              Message 6 of 11 , Jun 18, 2014
                J?rgen Herrmann:
                > I have a problem with the following pcre table:
                >
                > if !/X-Spam-Level:.*\*{7,}/
                > /X-Spam-Flag:.*YES/ HOLD
                > endif

                Wietse:
                > > As documented in pcre_table(5) and other places:
                > >
                > > if /pattern/flags
                > >
                > > endif Match the input string against the patterns between if
                > > and
                > > endif, if and only if THAT SAME INPUT STRING also matches pat-
                > > tern. The if..endif can nest.
                > >
                > > In other words, "/X-Spam-Flag:.*YES/ HOLD" executes always.
                >
                > Hmm, i have a ! before the if-pattern.
                >
                > if !/pattern/flags

                You are NOT READING my reply. The "if !/X-Spam-Level/"
                always succeeds for headers with /X-Spam-Flag/

                Wietse
              • lists@rhsoft.net
                ... why not discard mails which are surely spam? because you can not say surely , a filter without false positives is just a dream and because the sender has
                Message 7 of 11 , Jun 18, 2014
                  Am 18.06.2014 16:17, schrieb Jürgen Herrmann:
                  > Am 18.06.2014 15:59, schrieb lists@...:
                  >> Am 18.06.2014 15:49, schrieb Jürgen Herrmann:
                  >>> I have a problem with the following pcre table:
                  >>>
                  >>> if !/X-Spam-Level:.*\*{7,}/
                  >>> /X-Spam-Flag:.*YES/ HOLD
                  >>> endif
                  >>> /X-Spam-Level:.*\*{7,}/ DISCARD
                  >>>
                  >>> the HOLD part of it is executed. the DISCARD part never matches.
                  >>>
                  >>> I also had a version with
                  >>> /X-Spam-Level: \*{7,}/
                  >>> instead of
                  >>> /X-Spam-Level:.*\*{7,}/
                  >>>
                  >>> and also the following version does never discard any mails:
                  >>>
                  >>> /X-Spam-Flag:.*YES/ HOLD
                  >>> /X-Spam-Level:.*\*{7,}/ DISCARD
                  >>
                  >> are you aware that you *must not* answer with 250 OK and the
                  >> silently discard messages? the spamfilter belongs *pre-queue*
                  >> and you have to REJECT spam, after you receive it and answer
                  >> with "250 OK" you have to deliver the message - period
                  >>
                  >> http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
                  >> "and the ability to use before-queue content filtering"
                  >>
                  >> fix your setup and get rid of the broken pcre-after-queue idea
                  >
                  > this is in a milter_header_checks and on a host used as relay
                  > host. so if i reject mail there it would create backscatter via
                  > the relaying host. why not discard mails that are surely spam?

                  why not discard mails which are surely spam?

                  because you can not say "surely", a filter without false positives
                  is just a dream and because the sender has to know that the are
                  not accepted to have a chance re-send, a spammer won't do that

                  you bring anybody in big trouble debugging mailproblems

                  one complains why a message was not answered
                  the other says "i never got a email"
                  the sender says his operator said the message was accepted

                  *always* the first station has to check and reject mails
                  and after accepting it it must not be rejected on the
                  next hop while discard is always a very bad idea

                  > i can only see a solution to this problem when installing SA
                  > on every mailing host, which is what i wanted to avoid.

                  every first hop in your own network has to make the decision
                  if a mail is accepted or rejected
                • Jürgen Herrmann
                  ... Sorry, i was READING, but not UNDERSTANDING i fear :) and i m still not understanding. do you say that FIRST the body of the if-clause (/X-Spam-Flag/) is
                  Message 8 of 11 , Jun 18, 2014
                    Am 18.06.2014 16:19, schrieb wietse@...:
                    > J?rgen Herrmann:
                    >> I have a problem with the following pcre table:
                    >>
                    >> if !/X-Spam-Level:.*\*{7,}/
                    >> /X-Spam-Flag:.*YES/ HOLD
                    >> endif
                    >
                    > Wietse:
                    >> > As documented in pcre_table(5) and other places:
                    >> >
                    >> > if /pattern/flags
                    >> >
                    >> > endif Match the input string against the patterns between if
                    >> > and
                    >> > endif, if and only if THAT SAME INPUT STRING also matches pat-
                    >> > tern. The if..endif can nest.
                    >> >
                    >> > In other words, "/X-Spam-Flag:.*YES/ HOLD" executes always.
                    >>
                    >> Hmm, i have a ! before the if-pattern.
                    >>
                    >> if !/pattern/flags
                    >
                    > You are NOT READING my reply. The "if !/X-Spam-Level/"
                    > always succeeds for headers with /X-Spam-Flag/
                    >
                    > Wietse

                    Sorry, i was READING, but not UNDERSTANDING i fear :)
                    and i'm still not understanding. do you say that FIRST the
                    body of the if-clause (/X-Spam-Flag/) is evaluated and if that matches,
                    THEN the /X-Spam-Level:/ is evaluated?

                    maybe you can answer this also: does HOLD take precedence over DISCARD?`
                    is the precedence of results of table lookups described somewhere?
                    or does it depend on the order of lines in the pcre table an the forst
                    non-OK/DUNNO outcome is respected?

                    Jürgen
                    --
                    >> XLhost.de ® - Webhosting von supersmall bis eXtra Large <<

                    XLhost.de GmbH
                    Jürgen Herrmann, Geschäftsführer
                    Boelckestrasse 21, 93051 Regensburg, Germany

                    Geschäftsführer: Jürgen Herrmann
                    Registriert unter: HRB9918
                    Umsatzsteuer-Identifikationsnummer: DE245931218

                    Fon: +49 (0)800 XLHOSTDE [0800 95467833]
                    Fax: +49 (0)800 95467830
                    Web: http://www.XLhost.de
                  • Wietse Venema
                    ... As documented, this processes input one line at a time. As documented, actions between if !/pattern/...end execute only for input lines that match if
                    Message 9 of 11 , Jun 18, 2014
                      J?rgen Herrmann:
                      > Am 18.06.2014 16:19, schrieb wietse@...:
                      > > J?rgen Herrmann:
                      > >> I have a problem with the following pcre table:
                      > >>
                      > >> if !/X-Spam-Level:.*\*{7,}/
                      > >> /X-Spam-Flag:.*YES/ HOLD
                      > >> endif

                      As documented, this processes input one line at a time.

                      As documented, actions between "if !/pattern/...end" execute
                      only for input lines that match "if !/pattern/".

                      If the current header is X-Spam-Flag:, then the "if !/X-Spam-Level:/"
                      matches, and the /X-Spam-Flag:/ pattern executes the HOLD action.

                      Really, people should learn to read what the text says, instead
                      of reading what they want to see.

                      Wietse
                    • Jürgen Herrmann
                      ... hi! mind you this is for _outgoing_ spam protection only for a farm of webservers. we had so many problems regarding outgoing spam in the last few months
                      Message 10 of 11 , Jun 20, 2014
                        Am 18.06.2014 16:24, schrieb lists@...:
                        > Am 18.06.2014 16:17, schrieb Jürgen Herrmann:
                        >> Am 18.06.2014 15:59, schrieb lists@...:
                        >>> Am 18.06.2014 15:49, schrieb Jürgen Herrmann:
                        >>>> I have a problem with the following pcre table:
                        >>>>
                        >>>> if !/X-Spam-Level:.*\*{7,}/
                        >>>> /X-Spam-Flag:.*YES/ HOLD
                        >>>> endif
                        >>>> /X-Spam-Level:.*\*{7,}/ DISCARD
                        >>>>
                        >>>> the HOLD part of it is executed. the DISCARD part never matches.
                        >>>>
                        >>>> I also had a version with
                        >>>> /X-Spam-Level: \*{7,}/
                        >>>> instead of
                        >>>> /X-Spam-Level:.*\*{7,}/
                        >>>>
                        >>>> and also the following version does never discard any mails:
                        >>>>
                        >>>> /X-Spam-Flag:.*YES/ HOLD
                        >>>> /X-Spam-Level:.*\*{7,}/ DISCARD
                        >>>
                        >>> are you aware that you *must not* answer with 250 OK and the
                        >>> silently discard messages? the spamfilter belongs *pre-queue*
                        >>> and you have to REJECT spam, after you receive it and answer
                        >>> with "250 OK" you have to deliver the message - period
                        >>>
                        >>> http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
                        >>> "and the ability to use before-queue content filtering"
                        >>>
                        >>> fix your setup and get rid of the broken pcre-after-queue idea
                        >>
                        >> this is in a milter_header_checks and on a host used as relay
                        >> host. so if i reject mail there it would create backscatter via
                        >> the relaying host. why not discard mails that are surely spam?
                        >
                        > why not discard mails which are surely spam?
                        >
                        > because you can not say "surely", a filter without false positives
                        > is just a dream and because the sender has to know that the are
                        > not accepted to have a chance re-send, a spammer won't do that
                        >
                        > you bring anybody in big trouble debugging mailproblems
                        >
                        > one complains why a message was not answered
                        > the other says "i never got a email"
                        > the sender says his operator said the message was accepted
                        >
                        > *always* the first station has to check and reject mails
                        > and after accepting it it must not be rejected on the
                        > next hop while discard is always a very bad idea

                        hi!

                        mind you this is for _outgoing_ spam protection only for a farm
                        of webservers. we had so many problems regarding outgoing spam
                        in the last few months that i'm willing to take a little risk...
                        as mail here is 90% of the time queued via the sendmail command
                        you'd probably agree that i should not reject mails in this case
                        and cause backscatter?

                        thanks for your input!

                        best regards,
                        jürgen herrmann
                        --
                        >> XLhost.de ® - Webhosting von supersmall bis eXtra Large <<

                        XLhost.de GmbH
                        Jürgen Herrmann, Geschäftsführer
                        Boelckestrasse 21, 93051 Regensburg, Germany

                        Geschäftsführer: Jürgen Herrmann
                        Registriert unter: HRB9918
                        Umsatzsteuer-Identifikationsnummer: DE245931218

                        Fon: +49 (0)800 XLHOSTDE [0800 95467833]
                        Fax: +49 (0)800 95467830
                        Web: http://www.XLhost.de
                      • lists@rhsoft.net
                        ... uhm if the message is rejected directly on the MTA on the webserver you just reject it for the web-application given the webservers have a sane setup and
                        Message 11 of 11 , Jun 20, 2014
                          Am 20.06.2014 10:52, schrieb Jürgen Herrmann:
                          > Am 18.06.2014 16:24, schrieb lists@...:
                          >> Am 18.06.2014 16:17, schrieb Jürgen Herrmann:
                          >>> Am 18.06.2014 15:59, schrieb lists@...:
                          >>>> Am 18.06.2014 15:49, schrieb Jürgen Herrmann:
                          >>>>> I have a problem with the following pcre table:
                          >>>>>
                          >>>>> if !/X-Spam-Level:.*\*{7,}/
                          >>>>> /X-Spam-Flag:.*YES/ HOLD
                          >>>>> endif
                          >>>>> /X-Spam-Level:.*\*{7,}/ DISCARD
                          >>>>>
                          >>>>> the HOLD part of it is executed. the DISCARD part never matches.
                          >>>>>
                          >>>>> I also had a version with
                          >>>>> /X-Spam-Level: \*{7,}/
                          >>>>> instead of
                          >>>>> /X-Spam-Level:.*\*{7,}/
                          >>>>>
                          >>>>> and also the following version does never discard any mails:
                          >>>>>
                          >>>>> /X-Spam-Flag:.*YES/ HOLD
                          >>>>> /X-Spam-Level:.*\*{7,}/ DISCARD
                          >>>>
                          >>>> are you aware that you *must not* answer with 250 OK and the
                          >>>> silently discard messages? the spamfilter belongs *pre-queue*
                          >>>> and you have to REJECT spam, after you receive it and answer
                          >>>> with "250 OK" you have to deliver the message - period
                          >>>>
                          >>>> http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd
                          >>>> "and the ability to use before-queue content filtering"
                          >>>>
                          >>>> fix your setup and get rid of the broken pcre-after-queue idea
                          >>>
                          >>> this is in a milter_header_checks and on a host used as relay
                          >>> host. so if i reject mail there it would create backscatter via
                          >>> the relaying host. why not discard mails that are surely spam?
                          >>
                          >> why not discard mails which are surely spam?
                          >>
                          >> because you can not say "surely", a filter without false positives
                          >> is just a dream and because the sender has to know that the are
                          >> not accepted to have a chance re-send, a spammer won't do that
                          >>
                          >> you bring anybody in big trouble debugging mailproblems
                          >>
                          >> one complains why a message was not answered
                          >> the other says "i never got a email"
                          >> the sender says his operator said the message was accepted
                          >>
                          >> *always* the first station has to check and reject mails
                          >> and after accepting it it must not be rejected on the
                          >> next hop while discard is always a very bad idea
                          >
                          > hi!
                          >
                          > mind you this is for _outgoing_ spam protection only for a farm
                          > of webservers. we had so many problems regarding outgoing spam
                          > in the last few months that i'm willing to take a little risk...
                          > as mail here is 90% of the time queued via the sendmail command
                          > you'd probably agree that i should not reject mails in this case
                          > and cause backscatter?
                          >
                          > thanks for your input!

                          uhm if the message is rejected directly on the MTA on the webserver
                          you just reject it for the web-application given the webservers
                          have a sane setup and enforce SMTP instead sendmail

                          our webservers dont allow mail() function and limit senders
                          to adresses for which we would also receive mail, that should
                          be done anyways because sooner or later your farm will land
                          on blacklists if your customers are using random senders from
                          domains with SPF/DKIM/DMARC
                          _____________________

                          that's on any webserver here and the mysql-user for mysql-senderaccess.cf
                          has a limited view on the main mailserer to only get valid addresses

                          local_recipient_maps = proxy:mysql:/etc/postfix/mysql-recipients.cf
                          smtpd_sender_restrictions = check_sender_access proxy:mysql:/etc/postfix/mysql-senderaccess.cf
                        Your message has been successfully submitted and would be delivered to recipients shortly.