Loading ...
Sorry, an error occurred while loading the content.
 

Re: Support for MDB in postfix 2.10?

Expand Messages
  • Quanah Gibson-Mount
    --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists ... Specifically because there is a package on Debian that already provided libmdb.so for some
    Message 1 of 20 , Feb 25, 2013
      --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
      <postfix-list@...> wrote:

      > Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
      > a naming conflict with other mdb* packages. See commit:
      >
      > http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
      > f3bcd59c2055d53e4759b0c3356b001eca44b0
      >
      > So the correct name is lmdb (liblmdb.so and lmdb.h).

      Specifically because there is a package on Debian that already provided
      "libmdb.so" for some software from Microsoft. We couldn't find any
      conflict with liblmdb.so.

      --Quanah

      --

      Quanah Gibson-Mount
      Sr. Member of Technical Staff
      Zimbra, Inc
      A Division of VMware, Inc.
      --------------------
      Zimbra :: the leader in open source messaging and collaboration
    • Quanah Gibson-Mount
      --On Monday, February 25, 2013 11:10 AM -0800 Quanah Gibson-Mount ... Debian packaging issue: FreeBSD
      Message 2 of 20 , Feb 25, 2013
        --On Monday, February 25, 2013 11:10 AM -0800 Quanah Gibson-Mount
        <quanah@...> wrote:

        > --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
        > <postfix-list@...> wrote:
        >
        >> Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
        >> a naming conflict with other mdb* packages. See commit:
        >>
        >> http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
        >> f3bcd59c2055d53e4759b0c3356b001eca44b0
        >>
        >> So the correct name is lmdb (liblmdb.so and lmdb.h).
        >
        > Specifically because there is a package on Debian that already provided
        > "libmdb.so" for some software from Microsoft. We couldn't find any
        > conflict with liblmdb.so.


        Debian packaging issue:
        <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=694757>
        FreeBSD packaging issue: <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007>


        --Quanah


        --

        Quanah Gibson-Mount
        Sr. Member of Technical Staff
        Zimbra, Inc
        A Division of VMware, Inc.
        --------------------
        Zimbra :: the leader in open source messaging and collaboration
      • Wietse Venema
        ... I figured this was mdbtools (for Microsoft Access databases) which is available for Linux and BSD systems among others. Please keep me posted when there is
        Message 3 of 20 , Feb 25, 2013
          Quanah Gibson-Mount:
          > --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
          > <postfix-list@...> wrote:
          >
          > > Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
          > > a naming conflict with other mdb* packages. See commit:
          > >
          > > http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
          > > f3bcd59c2055d53e4759b0c3356b001eca44b0
          > >
          > > So the correct name is lmdb (liblmdb.so and lmdb.h).
          >
          > Specifically because there is a package on Debian that already provided
          > "libmdb.so" for some software from Microsoft. We couldn't find any
          > conflict with liblmdb.so.

          I figured this was mdbtools (for Microsoft Access databases) which
          is available for Linux and BSD systems among others.

          Please keep me posted when there is a package that *other* people
          can install on a main-stream OS distribution, so that *I* know what
          *their* Postfix will need work with.

          Wietse
        • Quanah Gibson-Mount
          --On Monday, February 25, 2013 2:29 PM -0500 Wietse Venema ... FreeBSD now has an official lmdb ports package.
          Message 4 of 20 , Mar 11, 2013
            --On Monday, February 25, 2013 2:29 PM -0500 Wietse Venema
            <wietse@...> wrote:

            > Please keep me posted when there is a package that *other* people
            > can install on a main-stream OS distribution, so that *I* know what
            > *their* Postfix will need work with.

            FreeBSD now has an official lmdb ports package.
            <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.

            --Quanah

            --

            Quanah Gibson-Mount
            Sr. Member of Technical Staff
            Zimbra, Inc
            A Division of VMware, Inc.
            --------------------
            Zimbra :: the leader in open source messaging and collaboration
          • Wietse Venema
            ... I have updated the Postfix patch for lmdb databases. This is now included in snapshot 20130315. However, this code has several unexpected limitations. I
            Message 5 of 20 , Mar 15, 2013
              Quanah Gibson-Mount:
              > <wietse@...> wrote:
              >
              > > Please keep me posted when there is a package that *other* people
              > > can install on a main-stream OS distribution, so that *I* know what
              > > *their* Postfix will need work with.
              >
              > FreeBSD now has an official lmdb ports package.
              > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.

              I have updated the Postfix patch for "lmdb" databases. This is now
              included in snapshot 20130315.

              However, this code has several unexpected limitations. I documented
              the ones that I discovered today in LMDB_README. Because of the
              limitations, the code is "snapshot only", i.e. it will not be part
              of the stable release without major changes.

              I suggest that people do not abandon CDB databases just yet.

              Wietse
            • Wietse Venema
              ... Snaphot 20130317 addresses sub-optimal behavior in the LMDB client code that affected tlsmgr and postmap -i , and it makes the code more resilient. In
              Message 6 of 20 , Mar 17, 2013
                Wietse Venema:
                > Quanah Gibson-Mount:
                > > <wietse@...> wrote:
                > >
                > > > Please keep me posted when there is a package that *other* people
                > > > can install on a main-stream OS distribution, so that *I* know what
                > > > *their* Postfix will need work with.
                > >
                > > FreeBSD now has an official lmdb ports package.
                > > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.
                >
                > I have updated the Postfix patch for "lmdb" databases. This is now
                > included in snapshot 20130315.

                Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                code that affected tlsmgr and "postmap -i", and it makes the code
                more resilient.

                In particular, the Postfix LMDB client will no longer keep crashing
                on a "database full" error. Instead, Postfix can now recover without
                immediately requiring human intervention. This is important because
                many Postfix databases contain data that is maintained by a Postfix
                daemon process, and the size of the data is not known in advance.

                With this, LMDB no longer requires constant watching for "database
                full" errors. As the system recovers from an error, it logs a warning
                that humans can take care of the next day.

                Wietse
              • Quanah Gibson-Mount
                --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema ... Excellent news, thank you Wietse! Does this mean it may be included in a future release? --Quanah
                Message 7 of 20 , Mar 18, 2013
                  --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
                  <wietse@...> wrote:

                  > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                  > code that affected tlsmgr and "postmap -i", and it makes the code
                  > more resilient.
                  >
                  > In particular, the Postfix LMDB client will no longer keep crashing
                  > on a "database full" error. Instead, Postfix can now recover without
                  > immediately requiring human intervention. This is important because
                  > many Postfix databases contain data that is maintained by a Postfix
                  > daemon process, and the size of the data is not known in advance.
                  >
                  > With this, LMDB no longer requires constant watching for "database
                  > full" errors. As the system recovers from an error, it logs a warning
                  > that humans can take care of the next day.

                  Excellent news, thank you Wietse! Does this mean it may be included in a
                  future release?

                  --Quanah


                  --

                  Quanah Gibson-Mount
                  Sr. Member of Technical Staff
                  Zimbra, Inc
                  A Division of VMware, Inc.
                  --------------------
                  Zimbra :: the leader in open source messaging and collaboration
                • Quanah Gibson-Mount
                  --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema ... These sound like really excellent improvements. I ve been using MDB with OpenLDAP for a while, and
                  Message 8 of 20 , Mar 18, 2013
                    --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema
                    <wietse@...> wrote:

                    > Quanah Gibson-Mount:
                    >> --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
                    >> <wietse@...> wrote:
                    >>
                    >> > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                    >> > code that affected tlsmgr and "postmap -i", and it makes the code
                    >> > more resilient.
                    >> >
                    >> > In particular, the Postfix LMDB client will no longer keep crashing
                    >> > on a "database full" error. Instead, Postfix can now recover without
                    >> > immediately requiring human intervention. This is important because
                    >> > many Postfix databases contain data that is maintained by a Postfix
                    >> > daemon process, and the size of the data is not known in advance.
                    >> >
                    >> > With this, LMDB no longer requires constant watching for "database
                    >> > full" errors. As the system recovers from an error, it logs a warning
                    >> > that humans can take care of the next day.
                    >>
                    >> Excellent news, thank you Wietse! Does this mean it may be included in
                    >> a future release?
                    >
                    > Yes. And with these failure modes eliminated, it becomes worthwhile
                    > to explore new opportunities, such as updating a shared table.
                    >
                    > For example, multiple postscreen(8) or tlsmgr(8) or verify(8) daemons
                    > should be able to update a shared LMDB database, something that
                    > cannot safely be done with Berkeley DB. Occasionally some daemon
                    > may restart to automatically recover from an LMDB "table full"
                    > condition, but the system won't come to a halt. You can postpone
                    > these restarts by increasing lmdb_map_size in main.cf before the
                    > table reaches that limit.

                    These sound like really excellent improvements. I've been using MDB with
                    OpenLDAP for a while, and we're working hard on removing our reliance on
                    BDB everywhere, so now that postfix has support for it, it is a major win
                    in my book. Thank you!

                    --Quanah

                    --

                    Quanah Gibson-Mount
                    Sr. Member of Technical Staff
                    Zimbra, Inc
                    A Division of VMware, Inc.
                    --------------------
                    Zimbra :: the leader in open source messaging and collaboration
                  Your message has been successfully submitted and would be delivered to recipients shortly.