Loading ...
Sorry, an error occurred while loading the content.

Re: Support for MDB in postfix 2.10?

Expand Messages
  • Wietse Venema
    ... I have updated the Postfix patch for lmdb databases. This is now included in snapshot 20130315. However, this code has several unexpected limitations. I
    Message 1 of 20 , Mar 15, 2013
    • 0 Attachment
      Quanah Gibson-Mount:
      > <wietse@...> wrote:
      >
      > > Please keep me posted when there is a package that *other* people
      > > can install on a main-stream OS distribution, so that *I* know what
      > > *their* Postfix will need work with.
      >
      > FreeBSD now has an official lmdb ports package.
      > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.

      I have updated the Postfix patch for "lmdb" databases. This is now
      included in snapshot 20130315.

      However, this code has several unexpected limitations. I documented
      the ones that I discovered today in LMDB_README. Because of the
      limitations, the code is "snapshot only", i.e. it will not be part
      of the stable release without major changes.

      I suggest that people do not abandon CDB databases just yet.

      Wietse
    • Wietse Venema
      ... Snaphot 20130317 addresses sub-optimal behavior in the LMDB client code that affected tlsmgr and postmap -i , and it makes the code more resilient. In
      Message 2 of 20 , Mar 17, 2013
      • 0 Attachment
        Wietse Venema:
        > Quanah Gibson-Mount:
        > > <wietse@...> wrote:
        > >
        > > > Please keep me posted when there is a package that *other* people
        > > > can install on a main-stream OS distribution, so that *I* know what
        > > > *their* Postfix will need work with.
        > >
        > > FreeBSD now has an official lmdb ports package.
        > > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.
        >
        > I have updated the Postfix patch for "lmdb" databases. This is now
        > included in snapshot 20130315.

        Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
        code that affected tlsmgr and "postmap -i", and it makes the code
        more resilient.

        In particular, the Postfix LMDB client will no longer keep crashing
        on a "database full" error. Instead, Postfix can now recover without
        immediately requiring human intervention. This is important because
        many Postfix databases contain data that is maintained by a Postfix
        daemon process, and the size of the data is not known in advance.

        With this, LMDB no longer requires constant watching for "database
        full" errors. As the system recovers from an error, it logs a warning
        that humans can take care of the next day.

        Wietse
      • Quanah Gibson-Mount
        --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema ... Excellent news, thank you Wietse! Does this mean it may be included in a future release? --Quanah
        Message 3 of 20 , Mar 18, 2013
        • 0 Attachment
          --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
          <wietse@...> wrote:

          > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
          > code that affected tlsmgr and "postmap -i", and it makes the code
          > more resilient.
          >
          > In particular, the Postfix LMDB client will no longer keep crashing
          > on a "database full" error. Instead, Postfix can now recover without
          > immediately requiring human intervention. This is important because
          > many Postfix databases contain data that is maintained by a Postfix
          > daemon process, and the size of the data is not known in advance.
          >
          > With this, LMDB no longer requires constant watching for "database
          > full" errors. As the system recovers from an error, it logs a warning
          > that humans can take care of the next day.

          Excellent news, thank you Wietse! Does this mean it may be included in a
          future release?

          --Quanah


          --

          Quanah Gibson-Mount
          Sr. Member of Technical Staff
          Zimbra, Inc
          A Division of VMware, Inc.
          --------------------
          Zimbra :: the leader in open source messaging and collaboration
        • Quanah Gibson-Mount
          --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema ... These sound like really excellent improvements. I ve been using MDB with OpenLDAP for a while, and
          Message 4 of 20 , Mar 18, 2013
          • 0 Attachment
            --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema
            <wietse@...> wrote:

            > Quanah Gibson-Mount:
            >> --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
            >> <wietse@...> wrote:
            >>
            >> > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
            >> > code that affected tlsmgr and "postmap -i", and it makes the code
            >> > more resilient.
            >> >
            >> > In particular, the Postfix LMDB client will no longer keep crashing
            >> > on a "database full" error. Instead, Postfix can now recover without
            >> > immediately requiring human intervention. This is important because
            >> > many Postfix databases contain data that is maintained by a Postfix
            >> > daemon process, and the size of the data is not known in advance.
            >> >
            >> > With this, LMDB no longer requires constant watching for "database
            >> > full" errors. As the system recovers from an error, it logs a warning
            >> > that humans can take care of the next day.
            >>
            >> Excellent news, thank you Wietse! Does this mean it may be included in
            >> a future release?
            >
            > Yes. And with these failure modes eliminated, it becomes worthwhile
            > to explore new opportunities, such as updating a shared table.
            >
            > For example, multiple postscreen(8) or tlsmgr(8) or verify(8) daemons
            > should be able to update a shared LMDB database, something that
            > cannot safely be done with Berkeley DB. Occasionally some daemon
            > may restart to automatically recover from an LMDB "table full"
            > condition, but the system won't come to a halt. You can postpone
            > these restarts by increasing lmdb_map_size in main.cf before the
            > table reaches that limit.

            These sound like really excellent improvements. I've been using MDB with
            OpenLDAP for a while, and we're working hard on removing our reliance on
            BDB everywhere, so now that postfix has support for it, it is a major win
            in my book. Thank you!

            --Quanah

            --

            Quanah Gibson-Mount
            Sr. Member of Technical Staff
            Zimbra, Inc
            A Division of VMware, Inc.
            --------------------
            Zimbra :: the leader in open source messaging and collaboration
          Your message has been successfully submitted and would be delivered to recipients shortly.