Loading ...
Sorry, an error occurred while loading the content.

Re: New document: STRESS_README

Expand Messages
  • Wietse Venema
    ... The above are logged within the same second, as it is should be. The anvil server does nothing else between logging these messages, so there never should
    Message 1 of 38 , Nov 1, 2007
    • 0 Attachment
      This is the last normal activity logging from the anvil server:

      > Nov 1 10:13:03 mx-01 postfix/anvil[14283]: statistics: max connection rate 200/60s for (smtp:80.253.80.28) at Nov 1 10:04:11
      > Nov 1 10:13:03 mx-01 postfix/anvil[14283]: statistics: max connection count 37 for (smtp:80.253.80.28) at Nov 1 10:04:17
      > Nov 1 10:13:03 mx-01 postfix/anvil[14283]: statistics: max cache size 2003 at Nov 1 10:07:43

      The above are logged within the same second, as it is should be.
      The anvil server does nothing else between logging these messages,
      so there never should be any delay between them.

      > Nov 1 10:14:52 mx-01 postfix/anvil[14283]: statistics: max connection rate 12/60s for (smtp:66.56.131.32) at Nov 1 10:14:25
      > Nov 1 10:14:56 mx-01 postfix/anvil[14283]: statistics: max connection count 13 for (smtp:220.224.9.191) at Nov 1 10:13:49
      > Nov 1 10:14:57 mx-01 postfix/anvil[14283]: statistics: max cache size 1974 at Nov 1 10:14:15

      And these are the correspondnig strace records. They confirm that
      the time stamps are supplied by postfix/anvil.

      > send(7, "<22>Nov 1 10:14:52 postfix/anvi"..., 123, MSG_NOSIGNAL) = 123
      > send(7, "<22>Nov 1 10:14:56 postfix/anvi"..., 121, MSG_NOSIGNAL) = 121
      > send(7, "<22>Nov 1 10:14:57 postfix/anvi"..., 92, MSG_NOSIGNAL) = 92

      Note that these records are logged several seconds apart, even if
      the anvil server does nothing in the mean time. Something is
      dramatically slowing down the anvil server or the syslogd server

      I suspect that it is the latter. On Linux, UNIX-domain sockets are
      always reliable (both stream and datagram). This has serious
      consequences for syslogd and its clients.

      IMPORTANT: on Linux, Postfix/syslogd performance sucks horribly
      under load, unless you put '-' before the /var/log/maillog name.
      Be sure to "kill -HUP" the syslogd after editing syslogd.conf.

      This is especially important when Postfix is experiencing a high
      load (either on the incoming side or while delivering mail).

      Wietse
    • Victor Duchovni
      ... Why not just a single mail log file? How does mail.info differ from mail ? I am guessing you are writing each info (majority) log message to disk
      Message 38 of 38 , Nov 1, 2007
      • 0 Attachment
        On Thu, Nov 01, 2007 at 10:10:30AM -0700, Andreas Grimm wrote:

        > Hello,
        >
        > no, destinations are local files:
        > destination mailinfo { file("/var/log/mail.info"); };
        > destination mailwarn { file("/var/log/mail.warn"); };
        > destination mailerr { file("/var/log/mail.err" fsync(yes)); };
        > destination mail { file("/var/log/mail"); };

        Why not just a single mail log file? How does "mail.info" differ from
        "mail"? I am guessing you are writing each "info" (majority) log message
        to disk twice. This is wasteful.

        > Messages in the fifo should be dropped, thats right. Maybe the release
        > comes from the simple restart of syslog-ng (cleaning up garbage?). It's
        > confusing.

        Either the log socket was actually unix-stream, or the sync flags were
        wrong, or syslog-ng was unwell after running a long time, but that should
        not happen.

        Perhaps there were other settings in syslog-ng.conf that had not yet
        taken effect? Also, you will lose some messages when you rotate
        log files by renaming, and send SIGHUP to the log server. It rebinds
        the /dev/log unix-domain socket, and messages are lost (briefly)
        while the socket is rebuilt.

        I avoid this, by not using fixed file names, and not sending SIGHUP.

        destination mail {
        file("/d/d1/log/raw/mail/$YEAR/$MONTH/$DAY/$HOUR"
        template("$ISODATE $HOST $MSG\n") template_escape(no)
        owner("root") group("log") perm(0640)
        dir_owner("root") dir_group("log") dir_perm(02750)
        create_dirs(yes));
        };

        These log files don't require external rotation, it is enough to just
        delete sufficiently old ones periodically. We compress them first
        (writing compressed data to a parallel directory tree) and move
        sync the compressed files to a log aggregation server.

        --
        Viktor.

        Disclaimer: off-list followups get on-list replies or get ignored.
        Please do not ignore the "Reply-To" header.

        To unsubscribe from the postfix-users list, visit
        http://www.postfix.org/lists.html or click the link below:
        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

        If my response solves your problem, the best way to thank me is to not
        send an "it worked, thanks" follow-up. If you must respond, please put
        "It worked, thanks" in the "Subject" so I can delete these quickly.
      Your message has been successfully submitted and would be delivered to recipients shortly.