Loading ...
Sorry, an error occurred while loading the content.

Re: Message_size_limit issue with postfix v 2.8.8-1 on RHEL 6

Expand Messages
  • Nicolas HAHN
    ... We do it as any NDR is captured by the open source tool I m currently coding (I sent a mail there yesterday about version 0.9.13). That s a feature of the
    Message 1 of 29 , Apr 24, 2013
    • 0 Attachment

      > 1.  Don't send bounces to postmaster, just generate and read log summaries
      >     that may highlight aggregate problems with your mail stream.
      >
      >         notify_classes =
      >
      >     This applies to any MTA handing mail for a large number of users,
      >     it is fine to have postmaster notices for a machine with a small
      >     handful of users, but after than postmaster notices are just a waste
      >     of time and focus your attention on the wrong things (reading bounces
      >     of other people's mail).
      >
      >     The logs not the postmaster mailbox are your notices of trouble.

      We do it as any NDR is captured by the open source tool I'm currently coding (I sent a mail there yesterday about version 0.9.13). That's a feature of the tool to allow any SMTP admins in United Nations to access any NDR generated. That's the UN policy as UN Internet Service Provider. This is, again, politic (this kind of reason is given for a lot of things in UN :-)
      Furthermore, that's a really good think to have complete generated NDR in UN because with that, internal UN customers cannot say "Hey! We haven't received any NDR!". And customers themselves have access to the tool to have the confirmation NDRs are generated, and to see their content directly in the tool. Again, politic... Protection... blablabla

      But we are not there to discuss UN political choices and features about our messaging services :x

      >
      > 2.  When problems happen. READ THE LOGS.

      That's what I did (especially as coder of my real time postfix logging tool), but I think I might need some holidays after seeing Gb of logs each day since a lot of years... Finally, I start to miss things :-p

      On the other hand, any software shouldn't allow such situation to happen: conflict between two settings that don't match then qmgr killed... Any software should prevent that and validate any settings for acceptance, conformity, dependencies, ... That's why my feature request :)

      >
      >> Postfix feature request: that would be nice that Postfix be able
      >> to do this kind of basic checks by itself when starting (or when
      >> configuration is reloaded) between various inter-dependent
      >> configuration settings, and display in the logs at least some
      >> warnings when such kind of issue is detected :)
      >
      >     http://www.postfix.org/DEBUG_README.html#logging
      >
      > --
      >         Viktor.
      >


      ----------------------------------------------------------------
      This message was sent using IMP, the Internet Messaging Program.

    • Wietse Venema
      ... This is already in the logfile. Learn to read it. Wietse
      Message 2 of 29 , Apr 24, 2013
      • 0 Attachment
        Nicolas HAHN:
        > Postfix feature request: that would be nice that Postfix be able
        > to do this kind of basic checks by itself when starting (or when
        > configuration is reloaded) between various inter-dependent
        > configuration settings, and display in the logs at least some
        > warnings when such kind of issue is detected :)

        This is already in the logfile. Learn to read it.

        Wietse
      • Nicolas HAHN
        Yes, but if I m right, the log message is emitted at the time there is an e-mail processed (by postfix/local for my issue). What I m speaking about there is
        Message 3 of 29 , Apr 24, 2013
        • 0 Attachment

          Yes, but if I'm right, the log message is emitted at the time there is an e-mail processed (by postfix/local for my issue).

          What I'm speaking about there is that postfix should check the configuration for this issue (for example) at the time it is started or its configuration is reloaded, and REFUSE to start because of "fatal: main.cf configuration error: mailbox_size_limit is smaller than message_size_limit".

          Don't you think?

          But I learn I learn (well... I try)


          Wietse Venema <wietse@...> a écrit :

          > Nicolas HAHN:
          >> Postfix feature request: that would be nice that Postfix be able
          >> to do this kind of basic checks by itself when starting (or when
          >> configuration is reloaded) between various inter-dependent
          >> configuration settings, and display in the logs at least some
          >> warnings when such kind of issue is detected :)
          >
          > This is already in the logfile. Learn to read it.
          >
          >         Wietse
          >


          ----------------------------------------------------------------
          This message was sent using IMP, the Internet Messaging Program.

        • Nicolas HAHN
          This story also makes me think, suddenly, that I should integrate in my Log Search Tool a feature allowing real time fatal error catching (and not only fatal)
          Message 4 of 29 , Apr 24, 2013
          • 0 Attachment

            This story also makes me think, suddenly, that I should integrate in my Log Search Tool a feature allowing real time fatal error catching (and not only fatal) form postfix logs and real time alerting of the users using the tool in case a fatal comes during e-mail procesing. Will see that for versions 0.9.14 or 0.9.15.

            There is always something positive to take somewhere :)

            BR.
            nicolas

            Wietse Venema <wietse@...> a écrit :

            > Nicolas HAHN:
            >> Postfix feature request: that would be nice that Postfix be able
            >> to do this kind of basic checks by itself when starting (or when
            >> configuration is reloaded) between various inter-dependent
            >> configuration settings, and display in the logs at least some
            >> warnings when such kind of issue is detected :)
            >
            > This is already in the logfile. Learn to read it.
            >
            >         Wietse
            >


            ----------------------------------------------------------------
            This message was sent using IMP, the Internet Messaging Program.

          • Wietse Venema
            ... If you think about duplicating all configuration tests that are inside Postfix, into a program that can be run *before* Postfix, then you can forget about
            Message 5 of 29 , Apr 24, 2013
            • 0 Attachment
              Nicolas HAHN:
              > Yes, but if I'm right, the log message is emitted at the time there
              > is an e-mail processed (by postfix/local for my issue).
              >
              > What I'm speaking about there is that postfix should check the
              > configuration for this issue (for example) at the time it is started
              > or its configuration is reloaded, and REFUSE to start because of
              > "fatal: main.cf configuration error: mailbox_size_limit is smaller
              > than message_size_limit".
              >
              > Don't you think?

              If you think about duplicating all configuration tests that are
              inside Postfix, into a program that can be run *before* Postfix,
              then you can forget about that idea.

              Configuration tests will not be duplicated by hand, and they will
              not duplicated by some automated program that extracts them from
              Postfix source code. It is not going to happen.

              You will have to learn to read logfiles. Get used to it.

              Wietse
            • Nicolas HAHN
              This is a reply to myself because I m reviewing the way it works. ... In fact, the fatal is written in the logs each minute and 1 second for this issue in my
              Message 6 of 29 , Apr 24, 2013
              • 0 Attachment

                This is a reply to myself because I'm reviewing the way it works.

                > Yes, but if I'm right, the log message is emitted at the time there
                > is an e-mail processed (by postfix/local for my issue).

                In fact, the fatal is written in the logs each minute and 1 second  for this issue in my case by the postfix/local daemon



                ----------------------------------------------------------------
                This message was sent using IMP, the Internet Messaging Program.

              • Viktor Dukhovni
                ... Can you post the fatal error messages you found, especially the messages that log why the queue manager was restarting, as that is the real problem. It is
                Message 7 of 29 , Apr 24, 2013
                • 0 Attachment
                  On Wed, Apr 24, 2013 at 04:47:26PM +0200, Nicolas HAHN wrote:

                  > This is a reply to myself because I'm reviewing the way it works.
                  >
                  > > Yes, but if I'm right, the log message is emitted at the time there
                  > > is an e-mail processed (by postfix/local for my issue).
                  >
                  > In fact, the fatal is written in the logs each minute and 1 second
                  > for this issue in my case by the postfix/local daemon

                  Can you post the fatal error messages you found, especially the
                  messages that log why the queue manager was restarting, as that
                  is the real problem.

                  It is clear why local(8) was having issues, it writes mailbox files.
                  It is not yet clear why qmgr(8) was having issues. Though the need
                  to generate bounces into a non-working postmaster mailbox is unfortunate,
                  when the postmaster mailbox can't be written, that should just lead to
                  double bounces, which then get thrown away, so I don't see why the
                  queue-manager would restart...

                  --
                  Viktor.
                • Nicolas HAHN
                  ... Here is what I found: 2013-04-24T10:04:38.005665+00:00 iccpfxor04 postfix/local[9370]: fatal: main.cf configuration error: mailbox_size_limit is smaller
                  Message 8 of 29 , Apr 24, 2013
                  • 0 Attachment

                    > Can you post the fatal error messages you found, especially the
                    > messages that log why the queue manager was restarting, as that
                    > is the real problem.

                    Here is what I found:

                    2013-04-24T10:04:38.005665+00:00 iccpfxor04 postfix/local[9370]: fatal: main.cf configuration error: mailbox_size_limit is smaller than message_size_limit
                    2013-04-24T10:04:39.006185+00:00 iccpfxor04 postfix/master[26402]: warning: process /usr/libexec/postfix/local pid 9370 exit status 1
                    2013-04-24T10:04:39.006209+00:00 iccpfxor04 postfix/master[26402]: warning: /usr/libexec/postfix/local: bad command startup -- throttling

                    For the qmgr, except what I've posted previously, there is nothing else.

                    BUT that's normal if you consider this scenario:

                    When I've seen the emails for local delivery were growing in active queue, I've updated several times the message_size_limit and reloaded postfix configuration each time.
                    The fact qmgr shows a different PID can then considered being normal each time the configuration is reloaded (processes are stopped then restarted certainly).

                    The lesson for me in fact is that even if you code a log search engine tool, even the coder should not rely ONLY on it for anything (especially when this is an ongoing work and the tool in question is not parsing fatal or warnings from postfix logs for now). I then did some searches directly in the logs based on the QIDs of the emails that stayed in the active queue. And as the warning and fatal don't write the QID in the logs (which is normal because the fatal was written each minute and not related to any email processed finally), they were not catched by my simple grep filter when investigating the logs. That's a conjunction of things that lead me to write my first email there about this issue. It's then true that I should have investigated the logs more carefully :-X

                    So the conclusion is that there is nothing wrong with the QMGR.

                    What I consider just abnormal as already written is that for me (so it's my opinion), Postfix should refuse to start when it detects a fatal about a configuration issue in the config files. But it starts any way and display each minute the fatal in the log file. It should display also a message on the console just before to exit. But there are probably good reasons it is actually designed like that.

                    That's for this reason I'm going to integrate warning and fatal real time parsing in my tool for next coming version. That should help to prevent this kind of behavior from lazzy SMTP admins :-D

                    >
                    > It is clear why local(8) was having issues, it writes mailbox files.
                    > It is not yet clear why qmgr(8) was having issues.  Though the need
                    > to generate bounces into a non-working postmaster mailbox is unfortunate,
                    > when the postmaster mailbox can't be written, that should just lead to
                    > double bounces, which then get thrown away, so I don't see why the
                    > queue-manager would restart...
                    >
                    > --
                    >         Viktor.
                    >


                    ----------------------------------------------------------------
                    This message was sent using IMP, the Internet Messaging Program.

                  • Wietse Venema
                    ... If you think about duplicating all configuration tests that are inside Postfix, into a program that can be run *before* Postfix, then you can forget about
                    Message 9 of 29 , Apr 24, 2013
                    • 0 Attachment
                      Nicolas HAHN:
                      > What I consider just abnormal as already written is that for me
                      > (so it's my opinion), Postfix should refuse to start when it detects
                      > a fatal about a configuration issue in the config files. But it
                      > starts any way and display each minute the fatal in the log file.

                      If you think about duplicating all configuration tests that are
                      inside Postfix, into a program that can be run *before* Postfix,
                      then you can forget about that idea.

                      Configuration tests will not be duplicated by hand, and they will
                      not duplicated by some automated program that extracts them from
                      Postfix source code. It is not going to happen.

                      You will have to learn to read logfiles. Get used to it.

                      Wietse
                    • Nicolas HAHN
                      Yea. Thanks, i ve seen it the first time you posted it. But that s not for this reason I ll change my mind about this. BR. nicolas ... This message was sent
                      Message 10 of 29 , Apr 24, 2013
                      • 0 Attachment

                        Yea. Thanks, i've seen it the first time you posted it.

                        But that's not for this reason I'll change my mind about this.

                        BR.
                        nicolas

                        Wietse Venema <wietse@...> a écrit :

                        > Nicolas HAHN:
                        >> What I consider just abnormal as already written is that for me
                        >> (so it's my opinion), Postfix should refuse to start when it detects
                        >> a fatal about a configuration issue in the config files. But it
                        >> starts any way and display each minute the fatal in the log file.
                        >
                        > If you think about duplicating all configuration tests that are
                        > inside Postfix, into a program that can be run *before* Postfix,
                        > then you can forget about that idea.
                        >
                        > Configuration tests will not be duplicated by hand, and they will
                        > not duplicated by some automated program that extracts them from
                        > Postfix source code. It is not going to happen.
                        >
                        > You will have to learn to read logfiles. Get used to it.
                        >
                        >         Wietse
                        >


                        ----------------------------------------------------------------
                        This message was sent using IMP, the Internet Messaging Program.

                      • /dev/rob0
                        ... Here, local(8) is having a fatal error. This error occurs whenever local (and probably virtual(8) as well) is invoked for delivery. Conversely, this error
                        Message 11 of 29 , Apr 24, 2013
                        • 0 Attachment
                          On Wed, Apr 24, 2013 at 05:23:09PM +0200, Nicolas HAHN wrote:
                          > > Can you post the fatal error messages you found, especially the
                          > > messages that log why the queue manager was restarting, as that
                          > > is the real problem.
                          >
                          > Here is what I found:
                          >
                          > 2013-04-24T10:04:38.005665+00:00 iccpfxor04 postfix/local[9370]:
                          > fatal: main.cf configuration error: mailbox_size_limit is smaller
                          > than message_size_limit

                          Here, local(8) is having a fatal error. This error occurs whenever
                          local (and probably virtual(8) as well) is invoked for delivery.
                          Conversely, this error does NOT occur otherwise. The rest of the
                          system might be fine. (Postfix has a modular design, BTW.)

                          > 2013-04-24T10:04:39.006185+00:00 iccpfxor04 postfix/master[26402]:
                          > warning: process /usr/libexec/postfix/local pid 9370 exit status 1
                          > 2013-04-24T10:04:39.006209+00:00 iccpfxor04 postfix/master[26402]:
                          > warning: /usr/libexec/postfix/local: bad command startup --
                          > throttling

                          And see, master(8) is doing fine, letting you know that there is a
                          problem with local.

                          [snip]
                          > What I consider just abnormal as already written is that for me (so
                          > it's my opinion), Postfix should refuse to start when it detects a
                          > fatal about a configuration issue in the config files. But it

                          This seems to betray a lack of understanding of the architecture. I
                          think at this point you'd benefit from further study:

                          http://www.postfix.org/OVERVIEW.html

                          What about a system with no local/virtual delivery? Suppose delivery
                          is via pipe(8) or lmtp(8)? Or suppose there are no hosted domains at
                          all, merely a MSA/relay for other hosts? This fatal error in local
                          won't ever affect such a system.

                          > starts any way and display each minute the fatal in the log file.

                          Every time local attempts to deliver.

                          > It should display also a message on the console just before to

                          There is no console, this is a daemon process detached from the
                          terminal.

                          > exit. But there are probably good reasons it is actually designed
                          > like that.

                          Yes.

                          > That's for this reason I'm going to integrate warning and fatal
                          > real time parsing in my tool for next coming version. That should
                          > help to prevent this kind of behavior from lazzy SMTP admins :-D

                          It's probably impossible to compensate for a sysadmin who lacks
                          understanding of the system s/he is running.
                          --
                          http://rob0.nodns4.us/ -- system administration and consulting
                          Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                        • Nicolas HAHN
                          The archietcture is not a good excuse for me, I m sorry. As a coder, allowing a software to start despite the fact there is a FATAL is a total non-sens. And
                          Message 12 of 29 , Apr 24, 2013
                          • 0 Attachment

                            The "archietcture" is not a good excuse for me, I'm sorry. As a coder, allowing a software to start despite the fact there is a FATAL is a total non-sens. And saying finally that is just a daemon which will not run but the others will, I really don't know how to take it...

                            And you can daemonize later. This also is not a good excuse.

                            I take the Nagios example:

                            [root@server nagios]# service nagios restart
                            Running configuration check... CONFIG ERROR!  Restart aborted.  Check your Nagios configuration.
                            [root@server nagios]#

                            And after having fixed the configuration files:

                            [root@server nagios]# service nagios restart
                            Running configuration check...done.
                            Stopping nagios: ...done.
                            Starting nagios: done.
                            [root@server nagios]#


                            But OK I understand your points and will stop to post my blabla.


                            /dev/rob0 <rob0@...> a écrit :

                            > On Wed, Apr 24, 2013 at 05:23:09PM +0200, Nicolas HAHN wrote:
                            >> > Can you post the fatal error messages you found, especially the
                            >> > messages that log why the queue manager was restarting, as that
                            >> > is the real problem.
                            >>
                            >> Here is what I found:
                            >>
                            >> 2013-04-24T10:04:38.005665+00:00 iccpfxor04 postfix/local[9370]:
                            >> fatal: main.cf configuration error: mailbox_size_limit is smaller
                            >> than message_size_limit
                            >
                            > Here, local(8) is having a fatal error. This error occurs whenever
                            > local (and probably virtual(8) as well) is invoked for delivery.
                            > Conversely, this error does NOT occur otherwise. The rest of the
                            > system might be fine. (Postfix has a modular design, BTW.)
                            >
                            >> 2013-04-24T10:04:39.006185+00:00 iccpfxor04 postfix/master[26402]:
                            >> warning: process /usr/libexec/postfix/local pid 9370 exit status 1
                            >> 2013-04-24T10:04:39.006209+00:00 iccpfxor04 postfix/master[26402]:
                            >> warning: /usr/libexec/postfix/local: bad command startup --
                            >> throttling
                            >
                            > And see, master(8) is doing fine, letting you know that there is a
                            > problem with local.
                            >
                            > [snip]
                            >> What I consider just abnormal as already written is that for me (so
                            >> it's my opinion), Postfix should refuse to start when it detects a
                            >> fatal about a configuration issue in the config files. But it
                            >
                            > This seems to betray a lack of understanding of the architecture. I
                            > think at this point you'd benefit from further study:
                            >
                            > http://www.postfix.org/OVERVIEW.html
                            >
                            > What about a system with no local/virtual delivery? Suppose delivery
                            > is via pipe(8) or lmtp(8)? Or suppose there are no hosted domains at
                            > all, merely a MSA/relay for other hosts? This fatal error in local
                            > won't ever affect such a system.
                            >
                            >> starts any way and display each minute the fatal in the log file.
                            >
                            > Every time local attempts to deliver.
                            >
                            >> It should display also a message on the console just before to
                            >
                            > There is no console, this is a daemon process detached from the
                            > terminal.
                            >
                            >> exit. But there are probably good reasons it is actually designed
                            >> like that.
                            >
                            > Yes.
                            >
                            >> That's for this reason I'm going to integrate warning and fatal
                            >> real time parsing in my tool for next coming version. That should
                            >> help to prevent this kind of behavior from lazzy SMTP admins :-D
                            >
                            > It's probably impossible to compensate for a sysadmin who lacks
                            > understanding of the system s/he is running.
                            > --
                            >   http://rob0.nodns4.us/ -- system administration and consulting
                            >   Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
                            >


                            ----------------------------------------------------------------
                            This message was sent using IMP, the Internet Messaging Program.

                          • Viktor Dukhovni
                            ... You don t yet known enough about Postfix to appreciate the answer, this is not your fault, but the design is fine. -- Viktor.
                            Message 13 of 29 , Apr 24, 2013
                            • 0 Attachment
                              On Wed, Apr 24, 2013 at 07:45:52PM +0200, Nicolas HAHN wrote:

                              > The "archietcture" is not a good excuse for me, I'm sorry.

                              You don't yet known enough about Postfix to appreciate the answer,
                              this is not your fault, but the design is fine.

                              --
                              Viktor.
                            • Wietse Venema
                              ... OK, before things get even more embarassing, maybe you should come back when you have some experience with a multi-programmed system of 200k lines of code
                              Message 14 of 29 , Apr 24, 2013
                              • 0 Attachment
                                Nicolas HAHN:
                                > The "archietcture" is not a good excuse for me, I'm sorry. As a
                                > coder, allowing a software to start despite the fact there is a
                                > FATAL is a total non-sens. And saying finally that is just a daemon
                                > which will not run but the others will, I really don't know how
                                > to take it...

                                OK, before things get even more embarassing, maybe you should come
                                back when you have some experience with a multi-programmed system
                                of 200k lines of code scattered over 40 different programs.

                                Let's not waste more time debating this.

                                Wietse
                              • Reindl Harald
                                ... well, that s the difference between coder and delevoper a coder writes something which works for now and every few years all is thrown away because
                                Message 15 of 29 , Apr 24, 2013
                                • 0 Attachment
                                  Am 24.04.2013 19:45, schrieb Nicolas HAHN:
                                  > The "archietcture" is not a good excuse for me, I'm sorry. As a coder

                                  well, that's the difference between "coder" and "delevoper"

                                  a "coder" writes something which works for now and every
                                  few years all is thrown away because the architecture
                                  and software-design does not fit in growing needs

                                  look back how many years postfix is perfectly maintained
                                  AND documentaed like no other software with nearly zero
                                  breakages of existung setups while it is as scaleable
                                  as possible for near to any environment

                                  sorry, but after following the thread you are not qualified
                                  enough to judge design-patterns of a software you do not
                                  understand enough
                                • Nicolas Hahn
                                  It s true I don t have your experience as you are the postfix coders. But it s also true you don t know what can be my expericence. Considering your political
                                  Message 16 of 29 , Apr 24, 2013
                                  • 0 Attachment
                                    It's true I don't have your experience as you are the postfix coders.

                                    But it's also true you don't know what can be my expericence.

                                    Considering your political answers since the beginning, embarassing for who? You're right that's enough. I'm going to answer you in private.

                                    Envoyé de mon iPad

                                    Le 24 avr. 2013 à 20:00, Wietse Venema <wietse@...> a écrit :

                                    > Nicolas HAHN:
                                    >> The "archietcture" is not a good excuse for me, I'm sorry. As a
                                    >> coder, allowing a software to start despite the fact there is a
                                    >> FATAL is a total non-sens. And saying finally that is just a daemon
                                    >> which will not run but the others will, I really don't know how
                                    >> to take it...
                                    >
                                    > OK, before things get even more embarassing, maybe you should come
                                    > back when you have some experience with a multi-programmed system
                                    > of 200k lines of code scattered over 40 different programs.
                                    >
                                    > Let's not waste more time debating this.
                                    >
                                    > Wietse
                                  • Nicolas HAHN
                                    You re right that s enough and I ll answer you in private. ... This message was sent using IMP, the Internet Messaging Program. You re right that s enough and
                                    Message 17 of 29 , Apr 24, 2013
                                    • 0 Attachment

                                      You're right that's enough and I'll answer you in private.


                                      Wietse Venema <wietse@...> a écrit :

                                      > Nicolas HAHN:
                                      >> The "archietcture" is not a good excuse for me, I'm sorry. As a
                                      >> coder, allowing a software to start despite the fact there is a
                                      >> FATAL is a total non-sens. And saying finally that is just a daemon
                                      >> which will not run but the others will, I really don't know how
                                      >> to take it...
                                      >
                                      > OK, before things get even more embarassing, maybe you should come
                                      > back when you have some experience with a multi-programmed system
                                      > of 200k lines of code scattered over 40 different programs.
                                      >
                                      > Let's not waste more time debating this.
                                      >
                                      >         Wietse
                                      >


                                      ----------------------------------------------------------------
                                      This message was sent using IMP, the Internet Messaging Program.

                                    • Nicolas HAHN
                                      ... I agree totally on that. That s why I write in the users mailing list, not in the developpers mailing list. To stop this thread that is borring for
                                      Message 18 of 29 , Apr 24, 2013
                                      • 0 Attachment

                                        > sorry, but after following the thread you are not qualified
                                        > enough to judge design-patterns of a software you do not
                                        > understand enough

                                        I agree totally on that. That's why I write in the users mailing list, not in the developpers mailing list.

                                        To stop this thread that is borring for everybody now, the next part of my answer in private.



                                        ----------------------------------------------------------------
                                        This message was sent using IMP, the Internet Messaging Program.

                                      Your message has been successfully submitted and would be delivered to recipients shortly.