Loading ...
Sorry, an error occurred while loading the content.

clock problem and postfix behaviour

Expand Messages
  • cbaegert@europeanservers.net
    Hi, we had a big problem yesterday : a power loss in our datacenter. Our mail server restarted with a clock set to 2003, although it s a brand new server (and
    Message 1 of 18 , Mar 1, 2006
    • 0 Attachment
      Hi,

      we had a big problem yesterday : a power loss in our datacenter.

      Our mail server restarted with a clock set to 2003, although it's a brand new
      server (and I supposed it was a brand new battery !)

      When I saw that, I hurried to set the clock to 2006. Then the mail server
      rejected all the messages it received but not delivered during the time it
      was set to 2003, with "status=expired, returned to sender"

      I'm wondering if I could have done something differently to avoid this
      problem, but I don't have the answer for now. Any idea ?

      Or maybe something in Postfix could be implemented to avoid this annoying
      behaviour ? I imagine it's not the first time that this problem occurs, and
      it certainly won't be the last time ! After a power loss, we don't have to
      time to read the whole Postfix manual, so I think it will be better if
      Postfix could handle this problem automatically whatever the admin mistakes !

      Christophe
    • Patrick Ben Koetter
      ... Postfix uses the time the system provides. Use NTP or a hardware clock to set the system time correctly. p@rick -- The Book of
      Message 2 of 18 , Mar 1, 2006
      • 0 Attachment
        * cbaegert@... <cbaegert@...>:
        > Hi,
        >
        > we had a big problem yesterday : a power loss in our datacenter.
        >
        > Our mail server restarted with a clock set to 2003, although it's a brand new
        > server (and I supposed it was a brand new battery !)
        >
        > When I saw that, I hurried to set the clock to 2006. Then the mail server
        > rejected all the messages it received but not delivered during the time it
        > was set to 2003, with "status=expired, returned to sender"
        >
        > I'm wondering if I could have done something differently to avoid this
        > problem, but I don't have the answer for now. Any idea ?
        >
        > Or maybe something in Postfix could be implemented to avoid this annoying
        > behaviour ? I imagine it's not the first time that this problem occurs, and
        > it certainly won't be the last time ! After a power loss, we don't have to
        > time to read the whole Postfix manual, so I think it will be better if
        > Postfix could handle this problem automatically whatever the admin mistakes !

        Postfix uses the time the system provides. Use NTP or a hardware clock to set
        the system time correctly.

        <http://www.ntp.org/>

        p@rick

        --
        The Book of Postfix
        <http://www.postfix-book.com>
        saslfinger (debugging SMTP AUTH):
        <http://postfix.state-of-mind.de/patrick.koetter/saslfinger/>
      • Keith Matthews
        On Wed, 1 Mar 2006 12:58:45 +0100 ... How would postfix know that the time had changed ? It s a lot easier (and safer) to ensure the time is set correctly at
        Message 3 of 18 , Mar 1, 2006
        • 0 Attachment
          On Wed, 1 Mar 2006 12:58:45 +0100
          cbaegert@... wrote:

          > Hi,
          >
          > we had a big problem yesterday : a power loss in our datacenter.
          >
          > Our mail server restarted with a clock set to 2003, although it's a
          > brand new server (and I supposed it was a brand new battery !)
          >
          > When I saw that, I hurried to set the clock to 2006. Then the mail
          > server rejected all the messages it received but not delivered during
          > the time it was set to 2003, with "status=expired, returned to sender"
          >
          > I'm wondering if I could have done something differently to avoid this
          >
          > problem, but I don't have the answer for now. Any idea ?
          >
          > Or maybe something in Postfix could be implemented to avoid this
          > annoying behaviour ? I imagine it's not the first time that this
          > problem occurs, and it certainly won't be the last time ! After a
          > power loss, we don't have to time to read the whole Postfix manual, so
          > I think it will be better if Postfix could handle this problem
          > automatically whatever the admin mistakes !
          >

          How would postfix know that the time had changed ?

          It's a lot easier (and safer) to ensure the time is set correctly at
          startup. I have calls to rdate (or whatever) in rc.local plus an
          invocation of ntpd.

          --
          Due to excessive spam as a result of archiving of this list I only
          accept mail through the list server.
        • lst_hoe01@kwsoft.de
          ... Postfix has no other way to get the date/time than to ask the system clock and rely on it. If you receive mail in 2003 and flush the queue in 2006 you have
          Message 4 of 18 , Mar 1, 2006
          • 0 Attachment
            Zitat von cbaegert@...:

            > Hi,
            >
            > we had a big problem yesterday : a power loss in our datacenter.
            >
            > Our mail server restarted with a clock set to 2003, although it's a brand new
            > server (and I supposed it was a brand new battery !)
            >
            > When I saw that, I hurried to set the clock to 2006. Then the mail server
            > rejected all the messages it received but not delivered during the time it
            > was set to 2003, with "status=expired, returned to sender"
            >
            > I'm wondering if I could have done something differently to avoid this
            > problem, but I don't have the answer for now. Any idea ?
            >
            > Or maybe something in Postfix could be implemented to avoid this annoying
            > behaviour ? I imagine it's not the first time that this problem occurs, and
            > it certainly won't be the last time ! After a power loss, we don't have to
            > time to read the whole Postfix manual, so I think it will be better if
            > Postfix could handle this problem automatically whatever the admin mistakes !

            Postfix has no other way to get the date/time than to ask the system
            clock and rely on it. If you receive mail in 2003 and flush the queue
            in 2006 you have only one attempt before the mail is expired.
            The correct way to solve the problem is to not accept new mail and
            flush the queue. If all mail is out stop postfix, set the clock and
            start postfix again.

            Regards

            Andreas
          • Amol Hatwar
            *Chop* ... Yes. You could have run ntpd before the fiasco happened. ... Keeping accurate time is the system s responsibility, not the mail servers. If you
            Message 5 of 18 , Mar 1, 2006
            • 0 Attachment
              *Chop*
              > I'm wondering if I could have done something differently to avoid this
              > problem, but I don't have the answer for now. Any idea ?
              >

              Yes. You could have run ntpd before the fiasco happened.


              > Or maybe something in Postfix could be implemented to avoid this annoying
              > behaviour ? I imagine it's not the first time that this problem occurs, and
              > it certainly won't be the last time ! After a power loss, we don't have to
              > time to read the whole Postfix manual, so I think it will be better if
              > Postfix could handle this problem automatically whatever the admin mistakes !

              Keeping accurate time is the system's responsibility, not the mail
              servers. If you computer has a defective system clock or suffers with
              serious clock drifts, you _should_ install ntpd. In your case you should
              use a combination of ntpdate and ntpd at startup.

              Regards,

              ah
            • Dhawal Doshy
              ... I doubt running ntp is a solution.. ntp will sync the time and NOT the date, though i could be mistaken. - dhawal
              Message 6 of 18 , Mar 1, 2006
              • 0 Attachment
                Amol Hatwar wrote:
                > *Chop*
                >> I'm wondering if I could have done something differently to avoid this
                >> problem, but I don't have the answer for now. Any idea ?
                >>
                >
                > Yes. You could have run ntpd before the fiasco happened.

                I doubt running ntp is a solution.. ntp will sync the time and NOT the
                date, though i could be mistaken.

                - dhawal

                >> Or maybe something in Postfix could be implemented to avoid this annoying
                >> behaviour ? I imagine it's not the first time that this problem occurs, and
                >> it certainly won't be the last time ! After a power loss, we don't have to
                >> time to read the whole Postfix manual, so I think it will be better if
                >> Postfix could handle this problem automatically whatever the admin mistakes !
                >
                > Keeping accurate time is the system's responsibility, not the mail
                > servers. If you computer has a defective system clock or suffers with
                > serious clock drifts, you _should_ install ntpd. In your case you should
                > use a combination of ntpdate and ntpd at startup.
                >
                > Regards,
                >
                > ah
                >
              • Ralf Hildebrandt
                ... Of course one runs ntpdate BEFORE starting ntpd. Our book mentions the need for a stable clock on a mail server -- Ralf Hildebrandt
                Message 7 of 18 , Mar 1, 2006
                • 0 Attachment
                  * Dhawal Doshy <dhawal@...>:

                  > >Yes. You could have run ntpd before the fiasco happened.
                  >
                  > I doubt running ntp is a solution.. ntp will sync the time and NOT the
                  > date, though i could be mistaken.

                  Of course one runs "ntpdate" BEFORE starting ntpd. Our book mentions
                  the need for a stable clock on a mail server
                  --
                  Ralf Hildebrandt (Ralf.Hildebrandt@...) spamtrap@...
                  Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155
                  http://www.postfix-buch.com
                  A computer scientist is someone who fixes things that aren't broken.
                • Simon Waters
                  ... Run ntpdate on the way up before running Postfix, no guarantee as ntpdate can fail due to networking issue, if the box is up before the routers, but
                  Message 8 of 18 , Mar 1, 2006
                  • 0 Attachment
                    On Wednesday 01 Mar 2006 11:58, cbaegert@... wrote:
                    >
                    > I'm wondering if I could have done something differently to avoid this
                    > problem, but I don't have the answer for now. Any idea ?

                    Run "ntpdate" on the way up before running Postfix, no guarantee as ntpdate
                    can fail due to networking issue, if the box is up before the routers, but
                    worth trying, we do it on all the GNU/Linux boxes here.

                    > so I think it will be better if
                    > Postfix could handle this problem automatically whatever the admin mistakes
                    > !

                    I think a general sanity check on date might be useful.

                    I was stunned to discover Lotus Notes didn't have one of these for replication
                    when I started using it, even those it did date based document replication,
                    it only needs a few replications with weird dates before I started losing
                    hair and will to live, and I was just an end user. It was quite happy to
                    receive updates that were older than the originals.

                    Problem is figuring out what test - at least in the current behaviour email
                    was returned to sender - so nothing was lost or held in an unacknowledged
                    state. Where as what to do with email that is much older in queue than the
                    queue expiry when Postfix starts, perhaps one last ditch delivery attempt
                    before bouncing?!
                  • cbaegert@europeanservers.net
                    Hi Peter, ... It s a good idea... I didn t know ntpdate, thank you, it s more useful for my problem than ntpd that I already use. Maybe this should be added in
                    Message 9 of 18 , Mar 1, 2006
                    • 0 Attachment
                      Hi Peter,

                      Le Mercredi 01 Mars 2006 12:59, Peter Kunst a écrit :
                      > Hi Christophe,
                      >
                      > it's a good idea to run "ntpdate" against a valid NTP source on all
                      > critical servers at __bootup__ time, and later on, ntpd. postfix isn't
                      > responsible for setting global system time.

                      It's a good idea... I didn't know ntpdate, thank you, it's more useful for my
                      problem than ntpd that I already use.

                      Maybe this should be added in the postfix documentation... sure, it's the
                      admin responsibility to make sure the time is OK, but if the documentation
                      says that hundred of mails can be rejected by postfix in a few seconds if the
                      time is wrong at startup, and that ntpdate can prevent this problem, the
                      admin will certainly install it instead of thinking it's OK with ntpd only.

                      Regards,
                      --
                      Christophe
                    • cbaegert@europeanservers.net
                      Hi, ... ntpd was launched, but did you ever try to launch ntpd when the clock is 3 years late ? ntpdate, that s the software I needed (thanks Peter) Regards,
                      Message 10 of 18 , Mar 1, 2006
                      • 0 Attachment
                        Hi,

                        Le Mercredi 01 Mars 2006 13:27, Amol Hatwar a écrit :
                        > *Chop*
                        >
                        > > I'm wondering if I could have done something differently to avoid this
                        > > problem, but I don't have the answer for now. Any idea ?
                        >
                        > Yes. You could have run ntpd before the fiasco happened.

                        ntpd was launched, but did you ever try to launch ntpd when the clock is 3
                        years late ?

                        ntpdate, that's the software I needed (thanks Peter)

                        Regards,
                        --
                        Christophe
                      • Ralf Hildebrandt
                        ... Yes, nothing will happen :) Thus: The problem won t occur. -- Ralf Hildebrandt (Ralf.Hildebrandt@charite.de) spamtrap@charite.de Postfix -
                        Message 11 of 18 , Mar 1, 2006
                        • 0 Attachment
                          * cbaegert@... <cbaegert@...>:

                          > ntpd was launched, but did you ever try to launch ntpd when the clock is 3
                          > years late ?

                          Yes, nothing will happen :)
                          Thus: The problem won't occur.

                          --
                          Ralf Hildebrandt (Ralf.Hildebrandt@...) spamtrap@...
                          Postfix - Einrichtung, Betrieb und Wartung Tel. +49 (0)30-450 570-155
                          http://www.postfix-buch.com
                          "Mac users swear by their Mac, PC users swear at their PC."
                        • Amol Hatwar
                          *chop* ... ^^^^^^^ I guess people should put a habit of reading the entire mail before replying. Regards, ah
                          Message 12 of 18 , Mar 1, 2006
                          • 0 Attachment
                            *chop*
                            > > Keeping accurate time is the system's responsibility, not the mail
                            > > servers. If you computer has a defective system clock or suffers with
                            > > serious clock drifts, you _should_ install ntpd. In your case you should
                            > > use a combination of ntpdate and ntpd at startup.
                            > >
                            ^^^^^^^ I guess people should put a habit of
                            reading the entire mail before replying.

                            Regards,

                            ah
                          • cbaegert@europeanservers.net
                            Hi Simon, ... yes, because you re right, even ntpdate isn t enough if it s launched before the routers are up ! Regards, -- Christophe
                            Message 13 of 18 , Mar 1, 2006
                            • 0 Attachment
                              Hi Simon,

                              Le Mercredi 01 Mars 2006 13:36, Simon Waters a écrit :
                              > On Wednesday 01 Mar 2006 11:58, cbaegert@... wrote:
                              > > I'm wondering if I could have done something differently to avoid this
                              > > problem, but I don't have the answer for now. Any idea ?
                              >
                              > Run "ntpdate" on the way up before running Postfix, no guarantee as ntpdate
                              > can fail due to networking issue, if the box is up before the routers, but
                              > worth trying, we do it on all the GNU/Linux boxes here.
                              >
                              > > so I think it will be better if
                              > > Postfix could handle this problem automatically whatever the admin
                              > > mistakes !
                              >
                              > I think a general sanity check on date might be useful.

                              yes, because you're right, even ntpdate isn't enough if it's launched before
                              the routers are up !

                              Regards,
                              --
                              Christophe
                            • Mark Martinec
                              dhawal, ... You are mistaken. Christophe, ... ntpdate is being slowly phased out. Using ntpd with option -g is now officially the preferred way for a reliable
                              Message 14 of 18 , Mar 1, 2006
                              • 0 Attachment
                                dhawal,
                                | I doubt running ntp is a solution.. ntp will sync the time and NOT the
                                | date, though i could be mistaken.

                                You are mistaken.


                                Christophe,
                                > It's a good idea... I didn't know ntpdate, thank you, it's more useful for
                                > my problem than ntpd that I already use.

                                ntpdate is being slowly phased out.
                                Using ntpd with option -g is now officially the preferred way
                                for a reliable start (although some people still tend to favor
                                calling ntpdate during startup).

                                Mark
                              • Keith Matthews
                                On Wed, 1 Mar 2006 14:40:29 +0100 ... Not every platform implements that yet. Nor do all ship ntpdate. OpenBSD has never (AFAIK) shipped ntpdate, preferring
                                Message 15 of 18 , Mar 1, 2006
                                • 0 Attachment
                                  On Wed, 1 Mar 2006 14:40:29 +0100
                                  Mark Martinec <Mark.Martinec+postfix@...> wrote:

                                  > dhawal,
                                  > | I doubt running ntp is a solution.. ntp will sync the time and NOT
                                  > | the date, though i could be mistaken.
                                  >
                                  > You are mistaken.
                                  >
                                  >
                                  > Christophe,
                                  > > It's a good idea... I didn't know ntpdate, thank you, it's more
                                  > > useful for my problem than ntpd that I already use.
                                  >
                                  > ntpdate is being slowly phased out.
                                  > Using ntpd with option -g is now officially the preferred way
                                  > for a reliable start (although some people still tend to favor
                                  > calling ntpdate during startup).
                                  >

                                  Not every platform implements that yet. Nor do all ship ntpdate.

                                  OpenBSD has never (AFAIK) shipped ntpdate, preferring rdate. OBSD's
                                  OpenNTPD has the '-s' flag which says "Set the time immediately at
                                  startup if the local clock is off by more than 180 seconds. Allows for
                                  a large time correction, eliminating the need to run rdate(8) before
                                  starting ntpd"

                                  --
                                  Due to excessive spam as a result of archiving of this list I only
                                  accept mail through the list server.
                                • Victor Duchovni
                                  ... The message arrival time is stored in the queue file, so when the clock moves forward significantly, messages may appear to be old. Postfix never expires a
                                  Message 16 of 18 , Mar 1, 2006
                                  • 0 Attachment
                                    On Wed, Mar 01, 2006 at 12:58:45PM +0100, cbaegert@... wrote:

                                    > Our mail server restarted with a clock set to 2003, although it's a brand new
                                    > server (and I supposed it was a brand new battery !)
                                    >
                                    > When I saw that, I hurried to set the clock to 2006. Then the mail server
                                    > rejected all the messages it received but not delivered during the time it
                                    > was set to 2003, with "status=expired, returned to sender"

                                    The message arrival time is stored in the queue file, so when the clock
                                    moves forward significantly, messages may appear to be old.

                                    Postfix never expires a message before a delivery attempt, expiration only
                                    happens after a failed delivery attempt, so the expired messages all failed
                                    to be delivered on the most recent delivery attempt.

                                    > I'm wondering if I could have done something differently to avoid this
                                    > problem, but I don't have the answer for now. Any idea ?
                                    >

                                    If you run "postsuper -r ALL" before starting postfix, the timestamps
                                    on all messages are reset when they are re-injected into the queue
                                    via pickup. This is true for Postfix releases prior to version 2.3.
                                    There is not an obvious solution for 2.3. History snippet from the
                                    postfix-2.3-20051105 snapshot.

                                    20051105

                                    All delay logging now has sub-second resolution. This means
                                    updating all code that reads or updates the records that
                                    specify when mail arrived, and ensuring that mail submitted
                                    with older Postfix versions produces sensible results.
                                    Files: global/post_mail.c, global/mail_timeofday.[hc],
                                    global/log_adhoc.c, postdrop/postdrop.c, pickup/pickup.c,
                                    cleanup/cleanup_envelope.c, cleanup/cleanup_message.c,
                                    smtpd/smtpd.c, qmqpd/qmqpd.c, *qmgr/qmgr_message.c,
                                    *qmgr/qmgr_active.c, local/forward.c.

                                    Open problems:

                                    "postsuper -r" no longer resets the message arrival time,
                                    because pickup(8) no longer overrides queue file time stamp
                                    information. This can be a problem when mail "on hold" is
                                    released after a long time.

                                    --
                                    Viktor.

                                    Disclaimer: off-list followups get on-list replies or get ignored.
                                    Please do not ignore the "Reply-To" header.

                                    To unsubscribe from the postfix-users list, visit
                                    http://www.postfix.org/lists.html or click the link below:
                                    <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                                    If my response solves your problem, the best way to thank me is to not
                                    send an "it worked, thanks" follow-up. If you must respond, please put
                                    "It worked, thanks" in the "Subject" so I can delete these quickly.
                                  • cbaegert@europeanservers.net
                                    Hi Victor, ... Very interesting ! I hope it will be available again. Regards, -- Christophe
                                    Message 17 of 18 , Mar 1, 2006
                                    • 0 Attachment
                                      Hi Victor,

                                      Le Mercredi 01 Mars 2006 14:57, Victor Duchovni a écrit :
                                      > If you run "postsuper -r ALL" before starting postfix, the timestamps
                                      > on all messages are reset when they are re-injected into the queue
                                      > via pickup. This is true for Postfix releases prior to version 2.3.
                                      > There is not an obvious solution for 2.3. History snippet from the
                                      > postfix-2.3-20051105 snapshot.

                                      Very interesting ! I hope it will be available again.

                                      Regards,
                                      --
                                      Christophe
                                    • Wietse Venema
                                      ... Postfix bounces too old mail only after it tries to deliver it. Wietse
                                      Message 18 of 18 , Mar 1, 2006
                                      • 0 Attachment
                                        Simon Waters:
                                        > state. Where as what to do with email that is much older in queue than the
                                        > queue expiry when Postfix starts, perhaps one last ditch delivery attempt
                                        > before bouncing?!

                                        Postfix bounces too old mail only after it tries to deliver it.

                                        Wietse
                                      Your message has been successfully submitted and would be delivered to recipients shortly.