Loading ...
Sorry, an error occurred while loading the content.

how can I tweak the logging?

Expand Messages
  • Rob Tanner
    Hi, I m trying to come up with mechanisms to catch compromised accounts sending SPAM. Since spammers don t necessarily have all good addresses a large number
    Message 1 of 6 , Jun 13, 2013
    • 0 Attachment
      Hi,

      I'm trying to come up with mechanisms to catch compromised accounts sending SPAM.  Since spammers don't necessarily have all good addresses a large number of their SPAM messages bounce with 550 errors (mailbox unavailable or doesn't even exist).  I would like to monitor men logs and catch that pattern.  The problem is that the log entry that includes the 550 error only shows where the message was intended to go and not where it came from.  That's found on another log entry line.  Is there anyway to tweak the logging mechanism so both bits of data appear on the same log line?

      Thanks.


      Rob Tanner
      UNIX Services Manager
      Linfield College, McMinnville Oregon

      ITS will never ask you for your password.  Please don’t share yours with anyone!

    • Newton Pasqualini Filho
      Can you cut part of you log file and send to the list? I am able to detect in a single line when I find NOQUEUE in log. Regards, Newton Pasqualini Filho
      Message 2 of 6 , Jun 13, 2013
      • 0 Attachment
        Can you cut part of you log file and send to the list?

        I am able to detect in a single line when I find "NOQUEUE" in log.

        Regards,
        Newton Pasqualini Filho



        Em 13/06/2013, às 18:34, Rob Tanner <rtanner@...> escreveu:

        Hi,

        I'm trying to come up with mechanisms to catch compromised accounts sending SPAM.  Since spammers don't necessarily have all good addresses a large number of their SPAM messages bounce with 550 errors (mailbox unavailable or doesn't even exist).  I would like to monitor men logs and catch that pattern.  The problem is that the log entry that includes the 550 error only shows where the message was intended to go and not where it came from.  That's found on another log entry line.  Is there anyway to tweak the logging mechanism so both bits of data appear on the same log line?

        Thanks.


        Rob Tanner
        UNIX Services Manager
        Linfield College, McMinnville Oregon

        ITS will never ask you for your password.  Please don’t share yours with anyone!


      • Rob Tanner
        As requested. I suppose I could grab the queue ID and back track to the sender but when the logs get long (which they do, half a million or more lines) these
        Message 3 of 6 , Jun 13, 2013
        • 0 Attachment
          As requested.  I suppose I could grab the queue ID and back track to the sender but when the logs get long (which they do, half a million or more lines) these scans can take a while and I'm trying to capture this info in real time (more or less):

          Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: from=<rtanner@...>, size=3993, nrcpt=1 (queue active)
          Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 767641453B: skipped, still being delivered
          Jun 13 15:10:47 neskowin postfix/smtpd[23646]: disconnect from mail.wfo.linfield.edu[10.170.131.75]
          Jun 13 15:10:47 neskowin postfix/smtpd[22320]: connect from localhost.localdomain[127.0.0.1]
          Jun 13 15:10:47 neskowin postfix/smtpd[22320]: 7F7AF77C96: client=localhost.localdomain[127.0.0.1]
          Jun 13 15:10:47 neskowin postfix/cleanup[23328]: 7F7AF77C96: message-id=<71DA23E7-A7FB-4409-962A-A4B31DBBC3CE@...>
          Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 7F7AF77C96: from=<rtanner@...>, size=4190, nrcpt=1 (queue active)
          Jun 13 15:10:47 neskowin postfix/smtp[23326]: 6D97E7778E: to=<sillyputty25x@...>, relay=localhost.linfield.edu[127.0.0.1], delay=0, status=sent (250 OK, sent 51BA4367_13111_1998_1 250 Ok: queued as 7F7AF77C96)
          Jun 13 15:10:47 neskowin postfix/smtpd[22320]: disconnect from localhost.localdomain[127.0.0.1]
          Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: removed
          Jun 13 15:10:47 neskowin postfix/smtp[23198]: 7F7AF77C96: to=<sillyputty25x@...>, relay=gmail-smtp-in.l.google.com[173.194.79.27], delay=0, status=bounced (host gmail-smtp-in.l.google.com[173.194.79.27] said: 550-5.1.1 The email account that you tried to reach does not exist. Please try 550-5.1.1 double-checking the recipient's email address for typos or 550-5.1.1 unnecessary spaces. Learn more at 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 ol10si12569562pbb.214 - gsmtp (in reply to RCPT TO command))

          Thanks,
          Rob


          On Jun 13, 2013, at 2:44 PM, Newton Pasqualini Filho <newtonpasqualini@...>
           wrote:

          Can you cut part of you log file and send to the list?

          I am able to detect in a single line when I find "NOQUEUE" in log.

          Regards,
          Newton Pasqualini Filho



          Em 13/06/2013, às 18:34, Rob Tanner <rtanner@...> escreveu:

          Hi,

          I'm trying to come up with mechanisms to catch compromised accounts sending SPAM.  Since spammers don't necessarily have all good addresses a large number of their SPAM messages bounce with 550 errors (mailbox unavailable or doesn't even exist).  I would like to monitor men logs and catch that pattern.  The problem is that the log entry that includes the 550 error only shows where the message was intended to go and not where it came from.  That's found on another log entry line.  Is there anyway to tweak the logging mechanism so both bits of data appear on the same log line?

          Thanks.


          Rob Tanner
          UNIX Services Manager
          Linfield College, McMinnville Oregon

          ITS will never ask you for your password.  Please don’t share yours with anyone!



        • Newton Pasqualini Filho
          Wow, So this error messages are not yours, this comes from the external side. There is no way to catch this arg in same line as from. You can do a script that
          Message 4 of 6 , Jun 13, 2013
          • 0 Attachment
            Wow,

            So this error messages are not yours, this comes from the external side.

            There is no way to catch this arg in same line as from.

            You can do a script that can handle the log and store in memory to run in realtime, or you can create a cronjob.

            I can help you with the cronjob script to handle who are sending spam to Gmail for example.

            Setup a bash script with these two lines bellow:
            #!/bin/bash
            for mid in `cat /var/log/maillog | grep answer=6596 | awk '{print $6}'`; do cat /var/log/maillog | grep $mid | grep "from" | awk '{print $7}' | awk -F "<" '{print $2}' | awk -F ">" '{print $1}'; done

            Regards
            Newton Pasqualini Filho



            Em 13/06/2013, às 19:18, Rob Tanner <rtanner@...> escreveu:

            As requested.  I suppose I could grab the queue ID and back track to the sender but when the logs get long (which they do, half a million or more lines) these scans can take a while and I'm trying to capture this info in real time (more or less):

            Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: from=<rtanner@...>, size=3993, nrcpt=1 (queue active)
            Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 767641453B: skipped, still being delivered
            Jun 13 15:10:47 neskowin postfix/smtpd[23646]: disconnect from mail.wfo.linfield.edu[10.170.131.75]
            Jun 13 15:10:47 neskowin postfix/smtpd[22320]: connect from localhost.localdomain[127.0.0.1]
            Jun 13 15:10:47 neskowin postfix/smtpd[22320]: 7F7AF77C96: client=localhost.localdomain[127.0.0.1]
            Jun 13 15:10:47 neskowin postfix/cleanup[23328]: 7F7AF77C96: message-id=<71DA23E7-A7FB-4409-962A-A4B31DBBC3CE@...>
            Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 7F7AF77C96: from=<rtanner@...>, size=4190, nrcpt=1 (queue active)
            Jun 13 15:10:47 neskowin postfix/smtp[23326]: 6D97E7778E: to=<sillyputty25x@...>, relay=localhost.linfield.edu[127.0.0.1], delay=0, status=sent (250 OK, sent 51BA4367_13111_1998_1 250 Ok: queued as 7F7AF77C96)
            Jun 13 15:10:47 neskowin postfix/smtpd[22320]: disconnect from localhost.localdomain[127.0.0.1]
            Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: removed
            Jun 13 15:10:47 neskowin postfix/smtp[23198]: 7F7AF77C96: to=<sillyputty25x@...>, relay=gmail-smtp-in.l.google.com[173.194.79.27], delay=0, status=bounced (host gmail-smtp-in.l.google.com[173.194.79.27] said: 550-5.1.1 The email account that you tried to reach does not exist. Please try 550-5.1.1 double-checking the recipient's email address for typos or 550-5.1.1 unnecessary spaces. Learn more at 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 ol10si12569562pbb.214 - gsmtp (in reply to RCPT TO command))

            Thanks,
            Rob


            On Jun 13, 2013, at 2:44 PM, Newton Pasqualini Filho <newtonpasqualini@...>
             wrote:

            Can you cut part of you log file and send to the list?

            I am able to detect in a single line when I find "NOQUEUE" in log.

            Regards,
            Newton Pasqualini Filho



            Em 13/06/2013, às 18:34, Rob Tanner <rtanner@...> escreveu:

            Hi,

            I'm trying to come up with mechanisms to catch compromised accounts sending SPAM.  Since spammers don't necessarily have all good addresses a large number of their SPAM messages bounce with 550 errors (mailbox unavailable or doesn't even exist).  I would like to monitor men logs and catch that pattern.  The problem is that the log entry that includes the 550 error only shows where the message was intended to go and not where it came from.  That's found on another log entry line.  Is there anyway to tweak the logging mechanism so both bits of data appear on the same log line?

            Thanks.


            Rob Tanner
            UNIX Services Manager
            Linfield College, McMinnville Oregon

            ITS will never ask you for your password.  Please don’t share yours with anyone!




          • Newton Pasqualini Filho
            Check if you can do a early logrotate, this will help you with this problem when running scripts. You can every hour rotate the log file and then run this
            Message 5 of 6 , Jun 13, 2013
            • 0 Attachment
              Check if you can do a early logrotate, this will help you with this problem when running scripts.

              You can every hour rotate the log file and then run this script into the old log.

              Newton Pasqualini Filho



              Em 13/06/2013, às 19:28, Newton Pasqualini Filho <newtonpasqualini@...> escreveu:

              Wow,

              So this error messages are not yours, this comes from the external side.

              There is no way to catch this arg in same line as from.

              You can do a script that can handle the log and store in memory to run in realtime, or you can create a cronjob.

              I can help you with the cronjob script to handle who are sending spam to Gmail for example.

              Setup a bash script with these two lines bellow:
              #!/bin/bash
              for mid in `cat /var/log/maillog | grep answer=6596 | awk '{print $6}'`; do cat /var/log/maillog | grep $mid | grep "from" | awk '{print $7}' | awk -F "<" '{print $2}' | awk -F ">" '{print $1}'; done

              Regards
              Newton Pasqualini Filho



              Em 13/06/2013, às 19:18, Rob Tanner <rtanner@...> escreveu:

              As requested.  I suppose I could grab the queue ID and back track to the sender but when the logs get long (which they do, half a million or more lines) these scans can take a while and I'm trying to capture this info in real time (more or less):

              Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: from=<rtanner@...>, size=3993, nrcpt=1 (queue active)
              Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 767641453B: skipped, still being delivered
              Jun 13 15:10:47 neskowin postfix/smtpd[23646]: disconnect from mail.wfo.linfield.edu[10.170.131.75]
              Jun 13 15:10:47 neskowin postfix/smtpd[22320]: connect from localhost.localdomain[127.0.0.1]
              Jun 13 15:10:47 neskowin postfix/smtpd[22320]: 7F7AF77C96: client=localhost.localdomain[127.0.0.1]
              Jun 13 15:10:47 neskowin postfix/cleanup[23328]: 7F7AF77C96: message-id=<71DA23E7-A7FB-4409-962A-A4B31DBBC3CE@...>
              Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 7F7AF77C96: from=<rtanner@...>, size=4190, nrcpt=1 (queue active)
              Jun 13 15:10:47 neskowin postfix/smtp[23326]: 6D97E7778E: to=<sillyputty25x@...>, relay=localhost.linfield.edu[127.0.0.1], delay=0, status=sent (250 OK, sent 51BA4367_13111_1998_1 250 Ok: queued as 7F7AF77C96)
              Jun 13 15:10:47 neskowin postfix/smtpd[22320]: disconnect from localhost.localdomain[127.0.0.1]
              Jun 13 15:10:47 neskowin postfix/qmgr[13765]: 6D97E7778E: removed
              Jun 13 15:10:47 neskowin postfix/smtp[23198]: 7F7AF77C96: to=<sillyputty25x@...>, relay=gmail-smtp-in.l.google.com[173.194.79.27], delay=0, status=bounced (host gmail-smtp-in.l.google.com[173.194.79.27] said: 550-5.1.1 The email account that you tried to reach does not exist. Please try 550-5.1.1 double-checking the recipient's email address for typos or 550-5.1.1 unnecessary spaces. Learn more at 550 5.1.1 http://support.google.com/mail/bin/answer.py?answer=6596 ol10si12569562pbb.214 - gsmtp (in reply to RCPT TO command))

              Thanks,
              Rob


              On Jun 13, 2013, at 2:44 PM, Newton Pasqualini Filho <newtonpasqualini@...>
               wrote:

              Can you cut part of you log file and send to the list?

              I am able to detect in a single line when I find "NOQUEUE" in log.

              Regards,
              Newton Pasqualini Filho



              Em 13/06/2013, às 18:34, Rob Tanner <rtanner@...> escreveu:

              Hi,

              I'm trying to come up with mechanisms to catch compromised accounts sending SPAM.  Since spammers don't necessarily have all good addresses a large number of their SPAM messages bounce with 550 errors (mailbox unavailable or doesn't even exist).  I would like to monitor men logs and catch that pattern.  The problem is that the log entry that includes the 550 error only shows where the message was intended to go and not where it came from.  That's found on another log entry line.  Is there anyway to tweak the logging mechanism so both bits of data appear on the same log line?

              Thanks.


              Rob Tanner
              UNIX Services Manager
              Linfield College, McMinnville Oregon

              ITS will never ask you for your password.  Please don’t share yours with anyone!





            • Benny Pedersen
              ... big logs can still be grepped, it works well for postfix-logwatch and pflogsumm if you tweek the logs its pointless to grep info from it later, if logs are
              Message 6 of 6 , Jun 14, 2013
              • 0 Attachment
                Rob Tanner skrev den 2013-06-14 00:18:
                > As requested. I suppose I could grab the queue ID and back track to
                > the sender but when the logs get long (which they do, half a million
                > or more lines) these scans can take a while and I'm trying to capture
                > this info in real time (more or less):

                big logs can still be grepped, it works well for postfix-logwatch and
                pflogsumm

                if you tweek the logs its pointless to grep info from it later, if logs
                are big, rotate more, eg rotate hourly ?

                --
                senders that put my email into body content will deliver it to my own
                trashcan, so if you like to get reply, dont do it
              Your message has been successfully submitted and would be delivered to recipients shortly.