Loading ...
Sorry, an error occurred while loading the content.

Re: Persistant LDAP connections

Expand Messages
  • Quanah Gibson-Mount
    --On Friday, March 08, 2013 3:45 PM +0200 Geoff Shang ... This is exactly what postfix does for our setup, using postfix 2.10.0. Example connection from this
    Message 1 of 15 , Mar 8, 2013
    • 0 Attachment
      --On Friday, March 08, 2013 3:45 PM +0200 Geoff Shang
      <geoff@...> wrote:

      > Hi,

      > Right now, Postfix is connecting to LDAP every time it needs to do one of
      > these lookups, then disconnects again. I thought that specifying
      > "proxy:" in the entry might deal with this, but it doesn't appear to have
      > done so.
      >
      > My question is, is it possible to get proxymap to open a persistant
      > connection for LDAP to do relay_domain and relay_recipient lookups?

      This is exactly what postfix does for our setup, using postfix 2.10.0.
      Example connection from this morning:

      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 fd=84 ACCEPT from
      IP=X.X.X.X:53128 (IP=X.X.X.X:389)
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=0 EXT
      oid=1.3.6.1.4.1.1466.20037
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=0 STARTTLS
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=0 RESULT oid= err=0
      text=
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 fd=84 TLS established
      tls_ssf=256 ssf=256
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=1 BIND
      dn="uid=zmpostfix,cn=appaccts,cn=zimbra" method=128
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=1 BIND
      dn="uid=zmpostfix,cn=appaccts,cn=zimbra" mech=SIMPLE ssf=0
      Mar 8 08:01:49 ldap01-zcs slapd[7644]: conn=29791 op=1 RESULT tag=97 err=0
      text=


      [... lots of queries ...]

      Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SRCH
      attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress
      Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SEARCH RESULT
      tag=101 err=0 nentries=0 text=
      Mar 8 08:12:32 ldap01-zcs slapd[7644]: conn=29791 fd=84 closed (connection
      lost)


      Or as you can see, it stayed connected from 08:01:49 to 08:12:32, during
      which time it performed 7,800 operations (all mail delivery lookups)
      outside of the initial bind before disconnecting.

      --Quanah

      --

      Quanah Gibson-Mount
      Sr. Member of Technical Staff
      Zimbra, Inc
      A Division of VMware, Inc.
      --------------------
      Zimbra :: the leader in open source messaging and collaboration
    • Viktor Dukhovni
      ... No Postfix release has ever done that. LDAP connections have always been cached by the process that makes the queries. The first query triggers a
      Message 2 of 15 , Mar 8, 2013
      • 0 Attachment
        On Fri, Mar 08, 2013 at 03:45:57PM +0200, Geoff Shang wrote:

        > Right now, Postfix is connecting to LDAP every time it needs to do
        > one of these lookups, then disconnects again.

        No Postfix release has ever done that. LDAP connections have always
        been cached by the process that makes the queries. The first query
        triggers a connection, subsequent queries re-use the connection.

        > I thought that specifying "proxy:" in the entry might deal with
        > this, but it doesn't appear to have done so.

        The effect of the "proxy:" prefix is to reduce the number of
        processes that make LDAP connections, by pooling all the connections
        via a small number of proxy processes.

        On a sufficiently idle system (say a test system which only receives
        intermittent email messages) the processes that are connected to LDAP
        may exit when idle for long enough, and then new connections will be
        made later. The same happens on systems where some misguidedly runs
        "postfix reload" frequently.

        > My question is, is it possible to get proxymap to open a persistant
        > connection for LDAP to do relay_domain and relay_recipient lookups?

        It is not possible to open non-persisten connections. If the LDAP
        server closes connections that the LDAP client did not actively
        close, that could be a reason for connections to not stay open.

        > /etc/postfix/ldap-domains.cf:
        >
        > version = 3
        > start_tls = no
        > tls_require_cert = no
        > server_host = ldap://ldap-server.ourdomain.com
        >
        > /etc/postfix/ldap-users.cf:
        >
        > version = 3
        > start_tls = no
        > tls_require_cert = no
        > server_host = ldap://ldap-server.ourdomain.com

        Furthermore, both tables have the same connection-related
        Parameters, and so as of postfix-2.0.16-2003091 both tables
        use the same LDAP connection.

        20030917

        Multiple LDAP lookup tables in the one Postfix process now
        share one LDAP connection. ...

        This snapshot eventually evolved into Postfix 2.1. So Postfix 2.1
        or newer supports both connection caching and connection consolidation
        for multiple tables that differ only in the query paramers (search base,
        scope, query, returned attributes).

        The proxymap service was introduced in postfix-2.0.0-20030103.

        --
        Viktor.
      • Quanah Gibson-Mount
        --On Friday, March 08, 2013 5:29 PM +0000 Viktor Dukhovni ... This is not really the behavior I see from proxy:. For example, the connection I pasted from
        Message 3 of 15 , Mar 8, 2013
        • 0 Attachment
          --On Friday, March 08, 2013 5:29 PM +0000 Viktor Dukhovni
          <postfix-users@...> wrote:

          > The effect of the "proxy:" prefix is to reduce the number of
          > processes that make LDAP connections, by pooling all the connections
          > via a small number of proxy processes.
          >
          > On a sufficiently idle system (say a test system which only receives
          > intermittent email messages) the processes that are connected to LDAP
          > may exit when idle for long enough, and then new connections will be
          > made later. The same happens on systems where some misguidedly runs
          > "postfix reload" frequently.

          This is not really the behavior I see from proxy:. For example, the
          connection I pasted from this morning:

          Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SRCH
          attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress
          Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SEARCH RESULT
          tag=101 err=0 nentries=0 text=
          Mar 8 08:12:32 ldap01-zcs slapd[7644]: conn=29791 fd=84 closed (connection
          lost)


          You can see that postfix closed the connection (without sending an unbind)
          5 seconds after operation 7801. Postfix has established 258 connections
          since 4 am this morning, all of which it initiates a close on after some
          amount of time (i.e., the LDAP server is not the one closing the
          connection).

          --Quanah


          --

          Quanah Gibson-Mount
          Sr. Member of Technical Staff
          Zimbra, Inc
          A Division of VMware, Inc.
          --------------------
          Zimbra :: the leader in open source messaging and collaboration
        • Quanah Gibson-Mount
          --On Friday, March 08, 2013 9:41 AM -0800 Quanah Gibson-Mount ... Vs, for example, our OpenDKIM setup, which has used a persistent connection to our ldap
          Message 4 of 15 , Mar 8, 2013
          • 0 Attachment
            --On Friday, March 08, 2013 9:41 AM -0800 Quanah Gibson-Mount
            <quanah@...> wrote:

            > --On Friday, March 08, 2013 5:29 PM +0000 Viktor Dukhovni
            > <postfix-users@...> wrote:
            >
            >> The effect of the "proxy:" prefix is to reduce the number of
            >> processes that make LDAP connections, by pooling all the connections
            >> via a small number of proxy processes.
            >>
            >> On a sufficiently idle system (say a test system which only receives
            >> intermittent email messages) the processes that are connected to LDAP
            >> may exit when idle for long enough, and then new connections will be
            >> made later. The same happens on systems where some misguidedly runs
            >> "postfix reload" frequently.
            >
            > This is not really the behavior I see from proxy:. For example, the
            > connection I pasted from this morning:
            >
            > Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SRCH
            > attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress
            > Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SEARCH RESULT
            > tag=101 err=0 nentries=0 text=
            > Mar 8 08:12:32 ldap01-zcs slapd[7644]: conn=29791 fd=84 closed
            > (connection lost)
            >
            >
            > You can see that postfix closed the connection (without sending an
            > unbind) 5 seconds after operation 7801. Postfix has established 258
            > connections since 4 am this morning, all of which it initiates a close on
            > after some amount of time (i.e., the LDAP server is not the one closing
            > the connection).

            Vs, for example, our OpenDKIM setup, which has used a persistent connection
            to our ldap server for a few days, and is now on op=137251.

            --Quanah

            --

            Quanah Gibson-Mount
            Sr. Member of Technical Staff
            Zimbra, Inc
            A Division of VMware, Inc.
            --------------------
            Zimbra :: the leader in open source messaging and collaboration
          • Viktor Dukhovni
            ... Not all connections are necessarily from proxy processes, and not all proxy processes are eternal. ... Postfix does not do graceful shutdown of database
            Message 5 of 15 , Mar 8, 2013
            • 0 Attachment
              On Fri, Mar 08, 2013 at 09:41:10AM -0800, Quanah Gibson-Mount wrote:

              > >On a sufficiently idle system (say a test system which only receives
              > >intermittent email messages) the processes that are connected to LDAP
              > >may exit when idle for long enough, and then new connections will be
              > >made later. The same happens on systems where some misguidedly runs
              > >"postfix reload" frequently.
              >
              > This is not really the behavior I see from proxy:. For example, the
              > connection I pasted from this morning:

              Not all connections are necessarily from proxy processes, and not
              all proxy processes are eternal.

              > Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SRCH
              > attr=zimbraMailCanonicalAddress zimbraMailCatchAllCanonicalAddress
              > Mar 8 08:12:27 ldap01-zcs slapd[7644]: conn=29791 op=7801 SEARCH
              > RESULT tag=101 err=0 nentries=0 text=
              > Mar 8 08:12:32 ldap01-zcs slapd[7644]: conn=29791 fd=84 closed
              > (connection lost)
              >
              > You can see that postfix closed the connection (without sending an
              > unbind) 5 seconds after operation 7801. Postfix has established 258
              > connections since 4 am this morning, all of which it initiates a
              > close on after some amount of time (i.e., the LDAP server is not the
              > one closing the connection).

              Postfix does not do "graceful shutdown" of database tables on exit.
              When a process is done, it exits. What is your max_idle set to?

              --
              Viktor.
            • Quanah Gibson-Mount
              --On Friday, March 08, 2013 5:48 PM +0000 Viktor Dukhovni ... Ok. And I m not complaining, really. ;) The proxy: functionality is definitely much more robust
              Message 6 of 15 , Mar 8, 2013
              • 0 Attachment
                --On Friday, March 08, 2013 5:48 PM +0000 Viktor Dukhovni
                <postfix-users@...> wrote:

                > On Fri, Mar 08, 2013 at 09:41:10AM -0800, Quanah Gibson-Mount wrote:
                >
                >> > On a sufficiently idle system (say a test system which only receives
                >> > intermittent email messages) the processes that are connected to LDAP
                >> > may exit when idle for long enough, and then new connections will be
                >> > made later. The same happens on systems where some misguidedly runs
                >> > "postfix reload" frequently.
                >>
                >> This is not really the behavior I see from proxy:. For example, the
                >> connection I pasted from this morning:
                >
                > Not all connections are necessarily from proxy processes, and not
                > all proxy processes are eternal.

                Ok. And I'm not complaining, really. ;) The proxy: functionality is
                definitely much more robust than without it.

                > Postfix does not do "graceful shutdown" of database tables on exit.
                > When a process is done, it exits. What is your max_idle set to?

                [zimbra@edge01-zcs ~]$ postconf | grep max_idle
                max_idle = 100s
                smtpd_policy_service_max_idle = 300s


                --Quanah

                --

                Quanah Gibson-Mount
                Sr. Member of Technical Staff
                Zimbra, Inc
                A Division of VMware, Inc.
                --------------------
                Zimbra :: the leader in open source messaging and collaboration
              • Viktor Dukhovni
                ... Postfix is robust without proxy: , but LDAP servers sometimes don t like hundreds of connections from Postfix when each smtpd(8) and each cleanup(8) makes
                Message 7 of 15 , Mar 8, 2013
                • 0 Attachment
                  On Fri, Mar 08, 2013 at 09:57:49AM -0800, Quanah Gibson-Mount wrote:

                  > >Not all connections are necessarily from proxy processes, and not
                  > >all proxy processes are eternal.
                  >
                  > Ok. And I'm not complaining, really. ;) The proxy: functionality
                  > is definitely much more robust than without it.

                  Postfix is robust without "proxy:", but LDAP servers sometimes
                  don't like hundreds of connections from Postfix when each smtpd(8)
                  and each cleanup(8) makes its own connection. Proxymap is kinder
                  and gentler to LDAP, it makes little difference to Postfix.

                  > >Postfix does not do "graceful shutdown" of database tables on exit.
                  > >When a process is done, it exits. What is your max_idle set to?
                  >
                  > [zimbra@edge01-zcs ~]$ postconf | grep max_idle
                  > max_idle = 100s
                  > smtpd_policy_service_max_idle = 300s

                  Processes that accept multiple client connections at the same time
                  like proxymap and trivial-rewrite will also exit after handling at
                  least 100 ($max_use) client connections if at some point their
                  client connection count drops to zero. You can also check whether
                  there are any max_idle overrides in master.cf.

                  Anyway your observations in no way contradict what I said, I don't
                  know why you thought they did.

                  --
                  Viktor.
                • Quanah Gibson-Mount
                  --On Friday, March 08, 2013 6:13 PM +0000 Viktor Dukhovni ... I ve bumped max_use to 5000 to see what happens. My point is that the connections are not as
                  Message 8 of 15 , Mar 8, 2013
                  • 0 Attachment
                    --On Friday, March 08, 2013 6:13 PM +0000 Viktor Dukhovni
                    <postfix-users@...> wrote:


                    > Processes that accept multiple client connections at the same time
                    > like proxymap and trivial-rewrite will also exit after handling at
                    > least 100 ($max_use) client connections if at some point their
                    > client connection count drops to zero. You can also check whether
                    > there are any max_idle overrides in master.cf.
                    >
                    > Anyway your observations in no way contradict what I said, I don't
                    > know why you thought they did.

                    I've bumped max_use to 5000 to see what happens. My point is that the
                    connections are not as persistent as one may desire. ;) I.e., OpenDKIM
                    stays connected forever until the server closes. Postfix is not
                    (currently) doing that for me, but as you note, may well be related to the
                    max_use setting.

                    --Quanah

                    --

                    Quanah Gibson-Mount
                    Sr. Member of Technical Staff
                    Zimbra, Inc
                    A Division of VMware, Inc.
                    --------------------
                    Zimbra :: the leader in open source messaging and collaboration
                  • Viktor Dukhovni
                    ... This is not a feature, it is a bug. OpenDKIM is a multi-threaded process that does not periodically exit to be replaced by a fresh process. As such it
                    Message 9 of 15 , Mar 8, 2013
                    • 0 Attachment
                      On Fri, Mar 08, 2013 at 10:20:20AM -0800, Quanah Gibson-Mount wrote:

                      > My point is that
                      > the connections are not as persistent as one may desire. ;) I.e.,
                      > OpenDKIM stays connected forever until the server closes.

                      This is not a feature, it is a bug. OpenDKIM is a multi-threaded
                      process that does not periodically exit to be replaced by a fresh
                      process. As such it does not tolerate memory leaks in its own code
                      or in the libraries it uses.

                      Postfix avoids this design pattern as much as possible. Other than
                      the tiny master server, only the queue manager (which does no table
                      lookups directly, and does not use SSL, GSSAPI, LDAP, ...), the
                      pickup server and tlsmgr run indefinitely. All three are simple and
                      have minimal interactions with non-Postfix resources.

                      > Postfix
                      > is not (currently) doing that for me, but as you note, may well be
                      > related to the max_use setting.

                      This is a feature. Also this keeps the load on your LDAP servers
                      more balanced, connections don't stick to one server forever.

                      --
                      Viktor.
                    • Quanah Gibson-Mount
                      --On Friday, March 08, 2013 7:05 PM +0000 Viktor Dukhovni ... OpenDKIM does what I ask. It makes a persistent connection and cuts out the overhead of
                      Message 10 of 15 , Mar 8, 2013
                      • 0 Attachment
                        --On Friday, March 08, 2013 7:05 PM +0000 Viktor Dukhovni
                        <postfix-users@...> wrote:

                        > On Fri, Mar 08, 2013 at 10:20:20AM -0800, Quanah Gibson-Mount wrote:
                        >
                        >> My point is that
                        >> the connections are not as persistent as one may desire. ;) I.e.,
                        >> OpenDKIM stays connected forever until the server closes.
                        >
                        > This is not a feature, it is a bug. OpenDKIM is a multi-threaded
                        > process that does not periodically exit to be replaced by a fresh
                        > process. As such it does not tolerate memory leaks in its own code
                        > or in the libraries it uses.

                        OpenDKIM does what I ask. It makes a persistent connection and cuts out
                        the overhead of persistent rebinding.

                        > Postfix avoids this design pattern as much as possible. Other than
                        > the tiny master server, only the queue manager (which does no table
                        > lookups directly, and does not use SSL, GSSAPI, LDAP, ...), the
                        > pickup server and tlsmgr run indefinitely. All three are simple and
                        > have minimal interactions with non-Postfix resources.
                        >
                        >> Postfix
                        >> is not (currently) doing that for me, but as you note, may well be
                        >> related to the max_use setting.
                        >
                        > This is a feature. Also this keeps the load on your LDAP servers
                        > more balanced, connections don't stick to one server forever.

                        I don't see an issue with them sticking to the first server in its URL
                        list, which is how postfix behaves. I organize my URLs as necessary on the
                        MTAs to distribute out the load. If I needed something more complicated
                        than that, I'd use a load balancer and load balanced name to return to
                        postfix. In any case, lookups from postfix cause an insignificant amount
                        of load, as long as they are persistent.

                        Thanks for pointing out max_use. Now instead of postfix rebinding every
                        4-5 minutes to the LDAP servers, it is at least every 20 minutes between
                        binds, significantly cutting out startTLS negotiation overhead and
                        improving performance.

                        It is trivial to see what a significant difference it makes in postfix
                        behavior to go from the default of 100 to 5000:
                        <http://www.pastebin.ca/2330089>

                        --Quanah

                        --

                        Quanah Gibson-Mount
                        Sr. Member of Technical Staff
                        Zimbra, Inc
                        A Division of VMware, Inc.
                        --------------------
                        Zimbra :: the leader in open source messaging and collaboration
                      • Viktor Dukhovni
                        ... Just because you want it, does not mean it is better. :-) ... It is surely trivial to see what an insignificant difference this makes. Between all those
                        Message 11 of 15 , Mar 8, 2013
                        • 0 Attachment
                          On Fri, Mar 08, 2013 at 11:24:25AM -0800, Quanah Gibson-Mount wrote:

                          > >This is not a feature, it is a bug. OpenDKIM is a multi-threaded
                          > >process that does not periodically exit to be replaced by a fresh
                          > >process. As such it does not tolerate memory leaks in its own code
                          > >or in the libraries it uses.
                          >
                          > OpenDKIM does what I ask. It makes a persistent connection and cuts
                          > out the overhead of persistent rebinding.

                          Just because you want it, does not mean it is better. :-)

                          > Thanks for pointing out max_use. Now instead of postfix rebinding
                          > every 4-5 minutes to the LDAP servers, it is at least every 20
                          > minutes between binds, significantly cutting out startTLS
                          > negotiation overhead and improving performance.
                          >
                          > It is trivial to see what a significant difference it makes in
                          > postfix behavior to go from the default of 100 to 5000:
                          > <http://www.pastebin.ca/2330089>

                          It is surely trivial to see what an insignificant difference this
                          makes. Between all those connections thousands of lookups are
                          made, the connection overhead is negligible.

                          The difference between a TLS handshake and LDAP bind every 4-5
                          minutes vs. every 20 minutes (or even infinity as with DKIM) is
                          negligible. Almost all the payoff from re-use is in the first
                          O(10) uses, after that it is diminishing returns all the way....

                          It is similar with max_use, it is of couse reasonably safe to have
                          it higher than 100, but the benefit is marginal at best.

                          --
                          Viktor.
                        Your message has been successfully submitted and would be delivered to recipients shortly.