Loading ...
Sorry, an error occurred while loading the content.

Re: Support for MDB in postfix 2.10?

Expand Messages
  • Patrick Lists
    ... Got it. I ll see what it takes to get this RPM included in Fedora and EPEL. If accepted that would cover Fedora, RHEL and CentOS. ... Judging from the git
    Message 1 of 20 , Feb 25 10:36 AM
    • 0 Attachment
      On 02/25/2013 07:19 PM, Wietse Venema wrote:
      > FYI, in order to maintain Postfix in a meaningful manner I need to
      > know what the library will look like on OTHER people's systems.
      > Therefore I will wait until there is a package for a mainstream
      > LINUX or *BSD distribution that OTHER people can install by typing
      > one or two commands.

      Got it. I'll see what it takes to get this RPM included in Fedora and
      EPEL. If accepted that would cover Fedora, RHEL and CentOS.

      > Howard's Postfix client code from a few months ago requires mdb.h
      > and libmdb.*. On the other hand your SRPM appears to install lmdb.h
      > and liblmdb.*. Is this the beginning of a Babylonian confusion?
      > What names will real systems use?

      Judging from the git commit log, on Nov 12, 2012 the "l" was added due
      to a naming conflict with other mdb* packages. See commit:

      http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13f3bcd59c2055d53e4759b0c3356b001eca44b0


      So the correct name is lmdb (liblmdb.so and lmdb.h).

      Regards,
      Patrick
    • Quanah Gibson-Mount
      --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists ... Specifically because there is a package on Debian that already provided libmdb.so for some
      Message 2 of 20 , Feb 25 11:10 AM
      • 0 Attachment
        --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
        <postfix-list@...> wrote:

        > Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
        > a naming conflict with other mdb* packages. See commit:
        >
        > http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
        > f3bcd59c2055d53e4759b0c3356b001eca44b0
        >
        > So the correct name is lmdb (liblmdb.so and lmdb.h).

        Specifically because there is a package on Debian that already provided
        "libmdb.so" for some software from Microsoft. We couldn't find any
        conflict with liblmdb.so.

        --Quanah

        --

        Quanah Gibson-Mount
        Sr. Member of Technical Staff
        Zimbra, Inc
        A Division of VMware, Inc.
        --------------------
        Zimbra :: the leader in open source messaging and collaboration
      • Quanah Gibson-Mount
        --On Monday, February 25, 2013 11:10 AM -0800 Quanah Gibson-Mount ... Debian packaging issue: FreeBSD
        Message 3 of 20 , Feb 25 11:25 AM
        • 0 Attachment
          --On Monday, February 25, 2013 11:10 AM -0800 Quanah Gibson-Mount
          <quanah@...> wrote:

          > --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
          > <postfix-list@...> wrote:
          >
          >> Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
          >> a naming conflict with other mdb* packages. See commit:
          >>
          >> http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
          >> f3bcd59c2055d53e4759b0c3356b001eca44b0
          >>
          >> So the correct name is lmdb (liblmdb.so and lmdb.h).
          >
          > Specifically because there is a package on Debian that already provided
          > "libmdb.so" for some software from Microsoft. We couldn't find any
          > conflict with liblmdb.so.


          Debian packaging issue:
          <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=694757>
          FreeBSD packaging issue: <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007>


          --Quanah


          --

          Quanah Gibson-Mount
          Sr. Member of Technical Staff
          Zimbra, Inc
          A Division of VMware, Inc.
          --------------------
          Zimbra :: the leader in open source messaging and collaboration
        • Wietse Venema
          ... I figured this was mdbtools (for Microsoft Access databases) which is available for Linux and BSD systems among others. Please keep me posted when there is
          Message 4 of 20 , Feb 25 11:29 AM
          • 0 Attachment
            Quanah Gibson-Mount:
            > --On Monday, February 25, 2013 7:36 PM +0100 Patrick Lists
            > <postfix-list@...> wrote:
            >
            > > Judging from the git commit log, on Nov 12, 2012 the "l" was added due to
            > > a naming conflict with other mdb* packages. See commit:
            > >
            > > http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=13
            > > f3bcd59c2055d53e4759b0c3356b001eca44b0
            > >
            > > So the correct name is lmdb (liblmdb.so and lmdb.h).
            >
            > Specifically because there is a package on Debian that already provided
            > "libmdb.so" for some software from Microsoft. We couldn't find any
            > conflict with liblmdb.so.

            I figured this was mdbtools (for Microsoft Access databases) which
            is available for Linux and BSD systems among others.

            Please keep me posted when there is a package that *other* people
            can install on a main-stream OS distribution, so that *I* know what
            *their* Postfix will need work with.

            Wietse
          • Quanah Gibson-Mount
            --On Monday, February 25, 2013 2:29 PM -0500 Wietse Venema ... FreeBSD now has an official lmdb ports package.
            Message 5 of 20 , Mar 11, 2013
            • 0 Attachment
              --On Monday, February 25, 2013 2:29 PM -0500 Wietse Venema
              <wietse@...> wrote:

              > Please keep me posted when there is a package that *other* people
              > can install on a main-stream OS distribution, so that *I* know what
              > *their* Postfix will need work with.

              FreeBSD now has an official lmdb ports package.
              <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.

              --Quanah

              --

              Quanah Gibson-Mount
              Sr. Member of Technical Staff
              Zimbra, Inc
              A Division of VMware, Inc.
              --------------------
              Zimbra :: the leader in open source messaging and collaboration
            • Wietse Venema
              ... I have updated the Postfix patch for lmdb databases. This is now included in snapshot 20130315. However, this code has several unexpected limitations. I
              Message 6 of 20 , Mar 15, 2013
              • 0 Attachment
                Quanah Gibson-Mount:
                > <wietse@...> wrote:
                >
                > > Please keep me posted when there is a package that *other* people
                > > can install on a main-stream OS distribution, so that *I* know what
                > > *their* Postfix will need work with.
                >
                > FreeBSD now has an official lmdb ports package.
                > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.

                I have updated the Postfix patch for "lmdb" databases. This is now
                included in snapshot 20130315.

                However, this code has several unexpected limitations. I documented
                the ones that I discovered today in LMDB_README. Because of the
                limitations, the code is "snapshot only", i.e. it will not be part
                of the stable release without major changes.

                I suggest that people do not abandon CDB databases just yet.

                Wietse
              • Wietse Venema
                ... Snaphot 20130317 addresses sub-optimal behavior in the LMDB client code that affected tlsmgr and postmap -i , and it makes the code more resilient. In
                Message 7 of 20 , Mar 17, 2013
                • 0 Attachment
                  Wietse Venema:
                  > Quanah Gibson-Mount:
                  > > <wietse@...> wrote:
                  > >
                  > > > Please keep me posted when there is a package that *other* people
                  > > > can install on a main-stream OS distribution, so that *I* know what
                  > > > *their* Postfix will need work with.
                  > >
                  > > FreeBSD now has an official lmdb ports package.
                  > > <http://www.freebsd.org/cgi/query-pr.cgi?pr=174007> has been closed.
                  >
                  > I have updated the Postfix patch for "lmdb" databases. This is now
                  > included in snapshot 20130315.

                  Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                  code that affected tlsmgr and "postmap -i", and it makes the code
                  more resilient.

                  In particular, the Postfix LMDB client will no longer keep crashing
                  on a "database full" error. Instead, Postfix can now recover without
                  immediately requiring human intervention. This is important because
                  many Postfix databases contain data that is maintained by a Postfix
                  daemon process, and the size of the data is not known in advance.

                  With this, LMDB no longer requires constant watching for "database
                  full" errors. As the system recovers from an error, it logs a warning
                  that humans can take care of the next day.

                  Wietse
                • Quanah Gibson-Mount
                  --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema ... Excellent news, thank you Wietse! Does this mean it may be included in a future release? --Quanah
                  Message 8 of 20 , Mar 18, 2013
                  • 0 Attachment
                    --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
                    <wietse@...> wrote:

                    > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                    > code that affected tlsmgr and "postmap -i", and it makes the code
                    > more resilient.
                    >
                    > In particular, the Postfix LMDB client will no longer keep crashing
                    > on a "database full" error. Instead, Postfix can now recover without
                    > immediately requiring human intervention. This is important because
                    > many Postfix databases contain data that is maintained by a Postfix
                    > daemon process, and the size of the data is not known in advance.
                    >
                    > With this, LMDB no longer requires constant watching for "database
                    > full" errors. As the system recovers from an error, it logs a warning
                    > that humans can take care of the next day.

                    Excellent news, thank you Wietse! Does this mean it may be included in a
                    future release?

                    --Quanah


                    --

                    Quanah Gibson-Mount
                    Sr. Member of Technical Staff
                    Zimbra, Inc
                    A Division of VMware, Inc.
                    --------------------
                    Zimbra :: the leader in open source messaging and collaboration
                  • Quanah Gibson-Mount
                    --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema ... These sound like really excellent improvements. I ve been using MDB with OpenLDAP for a while, and
                    Message 9 of 20 , Mar 18, 2013
                    • 0 Attachment
                      --On Monday, March 18, 2013 4:26 PM -0400 Wietse Venema
                      <wietse@...> wrote:

                      > Quanah Gibson-Mount:
                      >> --On Sunday, March 17, 2013 8:15 PM -0400 Wietse Venema
                      >> <wietse@...> wrote:
                      >>
                      >> > Snaphot 20130317 addresses sub-optimal behavior in the LMDB client
                      >> > code that affected tlsmgr and "postmap -i", and it makes the code
                      >> > more resilient.
                      >> >
                      >> > In particular, the Postfix LMDB client will no longer keep crashing
                      >> > on a "database full" error. Instead, Postfix can now recover without
                      >> > immediately requiring human intervention. This is important because
                      >> > many Postfix databases contain data that is maintained by a Postfix
                      >> > daemon process, and the size of the data is not known in advance.
                      >> >
                      >> > With this, LMDB no longer requires constant watching for "database
                      >> > full" errors. As the system recovers from an error, it logs a warning
                      >> > that humans can take care of the next day.
                      >>
                      >> Excellent news, thank you Wietse! Does this mean it may be included in
                      >> a future release?
                      >
                      > Yes. And with these failure modes eliminated, it becomes worthwhile
                      > to explore new opportunities, such as updating a shared table.
                      >
                      > For example, multiple postscreen(8) or tlsmgr(8) or verify(8) daemons
                      > should be able to update a shared LMDB database, something that
                      > cannot safely be done with Berkeley DB. Occasionally some daemon
                      > may restart to automatically recover from an LMDB "table full"
                      > condition, but the system won't come to a halt. You can postpone
                      > these restarts by increasing lmdb_map_size in main.cf before the
                      > table reaches that limit.

                      These sound like really excellent improvements. I've been using MDB with
                      OpenLDAP for a while, and we're working hard on removing our reliance on
                      BDB everywhere, so now that postfix has support for it, it is a major win
                      in my book. Thank you!

                      --Quanah

                      --

                      Quanah Gibson-Mount
                      Sr. Member of Technical Staff
                      Zimbra, Inc
                      A Division of VMware, Inc.
                      --------------------
                      Zimbra :: the leader in open source messaging and collaboration
                    Your message has been successfully submitted and would be delivered to recipients shortly.