Loading ...
Sorry, an error occurred while loading the content.

Re: Postscreen status script, take two

Expand Messages
  • Sahil Tandon
    ... Be careful with grep(1) patterns. You overstate CONNECTs by including NOQUEUE: reject: CONNECT in the count. Meanwhile, the script ... That bracket
    Message 1 of 6 , Feb 2, 2013
    • 0 Attachment
      On Wed, 2013-01-30 at 14:23:19 -0500, Mike. wrote:

      > I made some changes to the script based upon the excellent feedback I
      > received here.
      >
      > The script no longer wanders beyond the postscreen log records in
      > order to gather the information needed to determine the postscreen
      > rejection rate. So that removes the problems caused by
      > multiple-recipient messages.
      > ...

      Be careful with grep(1) patterns. You overstate CONNECTs by including
      'NOQUEUE: reject: CONNECT' in the count. Meanwhile, the script
      understates total DNSBL rejections, which you measure with:

      | grep -c "DNSBL rank [3-99]"

      That bracket expression matches on a _single_ character, and does not
      capture double-digit ranks. A similar mistake occurs in the attempt to
      aggregate 9+ ranks:

      | grep -c "DNSBL rank [9-99] "

      This only counts appearances of "DNSBL rank 9" in the log, as
      illustrated below:

      | % grep -c "DNSBL rank [9-99] " maillog
      | 4494

      | % grep -c "DNSBL rank 9 " maillog
      | 4494

      Review the re_format(7) and grep(1) manuals to improve understanding of
      regular expressions. In case it helps you, last year I had cobbled
      together a slower (it is Python rather than a set of grep(1)
      expressions) script[1] to collect similar statistics. No promises that
      it is error-free.

      [1] http://people.freebsd.org/~sahil/scripts/mailstats.py.txt

      --
      Sahil Tandon
    • Mike.
      ... I ... to ... of ... that ... ============= Thanks for the feedback.
      Message 2 of 6 , Feb 2, 2013
      • 0 Attachment
        On 2/2/2013 at 9:52 AM Sahil Tandon wrote:

        |On Wed, 2013-01-30 at 14:23:19 -0500, Mike. wrote:
        |
        |> I made some changes to the script based upon the excellent feedback
        I
        |> received here.
        |>
        |> The script no longer wanders beyond the postscreen log records in
        |> order to gather the information needed to determine the postscreen
        |> rejection rate. So that removes the problems caused by
        |> multiple-recipient messages.
        |> ...
        |
        |Be careful with grep(1) patterns. You overstate CONNECTs by including
        |'NOQUEUE: reject: CONNECT' in the count. Meanwhile, the script
        |understates total DNSBL rejections, which you measure with:
        |
        || grep -c "DNSBL rank [3-99]"
        |
        |That bracket expression matches on a _single_ character, and does not
        |capture double-digit ranks. A similar mistake occurs in the attempt
        to
        |aggregate 9+ ranks:
        |
        || grep -c "DNSBL rank [9-99] "
        |
        |This only counts appearances of "DNSBL rank 9" in the log, as
        |illustrated below:
        |
        || % grep -c "DNSBL rank [9-99] " maillog
        || 4494
        |
        || % grep -c "DNSBL rank 9 " maillog
        || 4494
        |
        |Review the re_format(7) and grep(1) manuals to improve understanding
        of
        |regular expressions. In case it helps you, last year I had cobbled
        |together a slower (it is Python rather than a set of grep(1)
        |expressions) script[1] to collect similar statistics. No promises
        that
        |it is error-free.
        |
        |[1] http://people.freebsd.org/~sahil/scripts/mailstats.py.txt
        |
        |--
        |Sahil Tandon

        =============


        Thanks for the feedback.
      • Mike.
        ... I tightened up that regex to include only the CONNECT occurrences I want. ... ============= I fixed both of those. Version 1.4 of the pslogscan.sh script,
        Message 3 of 6 , Feb 3, 2013
        • 0 Attachment
          On 2/2/2013 at 9:52 AM Sahil Tandon wrote:

          |Be careful with grep(1) patterns. You overstate CONNECTs by
          |including 'NOQUEUE: reject: CONNECT' in the count.

          I tightened up that regex to include only the CONNECT occurrences I
          want.



          | Meanwhile, the script
          | understates total DNSBL rejections,...
          |
          |A similar mistake occurs in the attempt to
          |aggregate 9+ ranks:
          =============

          I fixed both of those.


          Version 1.4 of the pslogscan.sh script, incorporating the above fixes,
          is available at:
          http://archive.mgm51.com/sources/pslogscan.html


          Thanks again for your feedback.
        • Steve Jenkins
          ... Hey, Mike. It s a cool idea - and almost works on a CentOS 6 box, but I m getting ambiguous redirect errors in a couple of cases. When it s first run
          Message 4 of 6 , Feb 6, 2013
          • 0 Attachment
            On Sun, Feb 3, 2013 at 9:06 AM, Mike. <the.lists@...> wrote:
            Version 1.4 of the pslogscan.sh script, incorporating the above fixes,
            is available at:
            http://archive.mgm51.com/sources/pslogscan.html

            Hey, Mike. It's a cool idea - and almost works on a CentOS 6 box, but I'm getting "ambiguous redirect" errors in a couple of cases. When it's first run (/tmp/pslogscan does not exist yet) I get:

            # ./pslogscan.sh /var/log/maillog
            Scanning /var/log/maillog
            mktemp: cannot create temp file /tmp/pslogscan: File exists

              CONNECT log records:      1106
              PASS NEW log records:     50
              PASS OLD log records:     25
              WHITELISTED log records:  717
              BLACKLISTED log records:  0

                      rejected:         314  (28%)


              Protocol errors:
                            HANGUP log records:  223
                          PREGREET log records:  62
                      BARE NEWLINE log records:  0
                COMMAND TIME LIMIT log records:  0
                COMMAND PIPELINING log records:  0

              DNS black lists log records:
            ./pslogscan.sh: line 140: ${TmpFile}: ambiguous redirect
                          zen.spamhaus.org:

            Then it freezes and I have to CTRL+C out.

            On all subsequent attempts (if /tmp/pslogscan already exists) I get:

            # ./pslogscan.sh /var/log/maillog
            Scanning /var/log/maillog
            mktemp: cannot create temp file /tmp/pslogscan: File exists
            mktemp: cannot create temp file /tmp/pslogscan: File exists

            ./pslogscan.sh: line 78: ${PostscreenLog}: ambiguous redirect

            Thanks for your efforts! I'm happy to test out future versions on my system. Feel free to email me directly and I'll test them out.

            SteveJ 
          • Steve Jenkins
            ... And.... ignore all that. Commenting mktempTemplate=pslogscan and uncommenting #mktempTemplate=pslogscan.XXX did the trick. :) Thx again! SteveJ
            Message 5 of 6 , Feb 6, 2013
            • 0 Attachment
              On Wed, Feb 6, 2013 at 9:41 AM, Steve Jenkins <stevejenkins@...> wrote:
              Hey, Mike. It's a cool idea - and almost works on a CentOS 6 box, but I'm getting "ambiguous redirect" errors in a couple of cases. When it's first run (/tmp/pslogscan does not exist yet) I get:

              # ./pslogscan.sh /var/log/maillog
              Scanning /var/log/maillog
              mktemp: cannot create temp file /tmp/pslogscan: File exists

                CONNECT log records:      1106
                PASS NEW log records:     50
                PASS OLD log records:     25
                WHITELISTED log records:  717
                BLACKLISTED log records:  0

                        rejected:         314  (28%)


                Protocol errors:
                              HANGUP log records:  223
                            PREGREET log records:  62
                        BARE NEWLINE log records:  0
                  COMMAND TIME LIMIT log records:  0
                  COMMAND PIPELINING log records:  0

                DNS black lists log records:
              ./pslogscan.sh: line 140: ${TmpFile}: ambiguous redirect
                            zen.spamhaus.org:

              Then it freezes and I have to CTRL+C out.

              On all subsequent attempts (if /tmp/pslogscan already exists) I get:

              # ./pslogscan.sh /var/log/maillog
              Scanning /var/log/maillog
              mktemp: cannot create temp file /tmp/pslogscan: File exists
              mktemp: cannot create temp file /tmp/pslogscan: File exists

              ./pslogscan.sh: line 78: ${PostscreenLog}: ambiguous redirect

              Thanks for your efforts! I'm happy to test out future versions on my system. Feel free to email me directly and I'll test them out.

              And.... ignore all that. Commenting mktempTemplate=pslogscan and uncommenting #mktempTemplate=pslogscan.XXX did the trick. :)

              Thx again!

              SteveJ

            Your message has been successfully submitted and would be delivered to recipients shortly.