Loading ...
Sorry, an error occurred while loading the content.

poor ldap load distribution

Expand Messages
  • Dan Lannom
    I m running postfix 2.4.0-2 on Debian Etch [backport] with ldap lookups for virtualaliases/relay_recipient. postfix-ldap is also version 2.4.0-2 and
    Message 1 of 4 , Jun 1, 2007
    • 0 Attachment
      I'm running postfix 2.4.0-2 on Debian Etch [backport] with ldap lookups
      for virtualaliases/relay_recipient. postfix-ldap is also version 2.4.0-2
      and ldap-utils is 2.3.30-5.

      I'm trying to use a round robin DNS entry to distribute the queries over
      multiple ldap servers, but for some reason they are all being redirected
      to the same system, which is pushing load too high.

      Ldapsearch against the same DNS entries exhibits the same behavior of
      all queries going to the same server. So it seems unlikely to be a
      postfix issue per-se.


      The current maps look like

      server_host = rrobin-dns.domain
      search_base = dc=searchbase
      query_filter =
      (&(|(mail=%s)(mailalternateaddress=%s))(maildeliveryoption=mailbox))
      result_attribute = maildrop
      bind=no

      and the queries from the DNS server return something like:

      Name: dns-a-record
      Addresses: ip1, ip2
      ..
      Name: dns-a-record
      Addresses: ip2, ip1

      in normal round-robin fashion

      Originally the ldap servers were on different subnets but moving them to
      the same subnet made no different.

      The server getting all the queries responds slightly faster due to a
      different configuration, but this is not trivial to change.

      Load does transfer to the 2nd system, if the 1rst system is disabled.

      Any suggestions?

      Thanks,

      Dan Lannom
      UM-Dearborn
    • Wietse Venema
      ... I am not aware of any Postfix control over the order of host selection. Postfix passes the LDAP server name to the ldap_init() or ldap_initialize()
      Message 2 of 4 , Jun 1, 2007
      • 0 Attachment
        Dan Lannom:
        > I'm running postfix 2.4.0-2 on Debian Etch [backport] with ldap lookups
        > for virtualaliases/relay_recipient. postfix-ldap is also version 2.4.0-2
        > and ldap-utils is 2.3.30-5.
        >
        > I'm trying to use a round robin DNS entry to distribute the queries over
        > multiple ldap servers, but for some reason they are all being redirected
        > to the same system, which is pushing load too high.
        >
        > Ldapsearch against the same DNS entries exhibits the same behavior of
        > all queries going to the same server. So it seems unlikely to be a
        > postfix issue per-se.
        >
        >
        > The current maps look like
        >
        > server_host = rrobin-dns.domain
        > search_base = dc=searchbase
        > query_filter =
        > (&(|(mail=%s)(mailalternateaddress=%s))(maildeliveryoption=mailbox))
        > result_attribute = maildrop
        > bind=no
        >
        > and the queries from the DNS server return something like:
        >
        > Name: dns-a-record
        > Addresses: ip1, ip2
        > ..
        > Name: dns-a-record
        > Addresses: ip2, ip1
        >
        > in normal round-robin fashion
        >
        > Originally the ldap servers were on different subnets but moving them to
        > the same subnet made no different.
        >
        > The server getting all the queries responds slightly faster due to a
        > different configuration, but this is not trivial to change.
        >
        > Load does transfer to the 2nd system, if the 1rst system is disabled.
        >
        > Any suggestions?

        I am not aware of any Postfix control over the order of host
        selection. Postfix passes the LDAP server name to the ldap_init()
        or ldap_initialize() function, and the LDAP library figures out
        what host to connect to.

        If your DNS server always returns the host list in the exact same
        order, or if some nasty system routine always sorts the host list
        into the same order, then the LDAP library will always try the
        hosts in the same order.

        This sounds like a job for an LD_PRELOAD shim that randomizes
        multi-address results from gethostbyname() or from getaddrinfo().
        How would one force such a shim between libldap and libc?

        Wietse
      • Victor Duchovni
        ... In what Postfix context are you using LDAP? Are you using proxy:ldap:... ? Is transport(5) lookups? Postfix keeps LDAP tables open, so each Postfix
        Message 3 of 4 , Jun 1, 2007
        • 0 Attachment
          On Fri, Jun 01, 2007 at 02:14:34PM -0400, Dan Lannom wrote:

          > I'm trying to use a round robin DNS entry to distribute the queries over
          > multiple ldap servers, but for some reason they are all being redirected
          > to the same system, which is pushing load too high.
          >

          In what Postfix context are you using LDAP? Are you using "proxy:ldap:..."?
          Is transport(5) lookups?

          Postfix keeps LDAP tables open, so each Postfix process that uses LDAP is
          going to query the same LDAP server for the process lifetime.

          - If you are doing LDAP lookups from smtpd(8) and cleanup(8), the
          LDAP connections don't last too long (max_use/max_idle), and load
          distribution may occur if the LDAP library cooperates. The downside
          is that under load you get a LOT of LDAP connections.

          - Most people who care about load, have high volume systems, where
          proxymap(8) is a must for LDAP queries from smtpd(8) and cleanup(8),
          so the picture changes. Each proxymap(8) process (many fewer of
          these) connects to an LDAP server and stays connected. These connection
          count is one or two orders of magnitude lower, but distribution may
          be uneven, and stay uneven, if you roll "heads" a few times in a row.

          - If you are using LDAP for transport(5) instead of virtual(5) (I
          recommend the latter), then proxymap is unwise, trivial-rewrite
          is already a multiplexed service, just like proxymap, and indirection
          just adds overhead. Again under load you may have more than one
          trivial-rewrite, but LDAP connections may cluster on the same server
          at times (roll "heads" ...).

          On my "busy" server using LDAP via proxymap for virtual(5) rewrites, the
          process counts (sampled just now) are:

          76 smtpd
          21 cleanup
          7 proxymap

          so 97 would-be LDAP clients are served by 7 LDAP connections from
          7 proxymap processes, which if LDAP cooperates each pick their
          own IP address for the LDAP server.

          I don't actually use multiple A records for LDAP load balancing, rather
          for machines in a cluster, I shuffle the LDAP server preference list from
          host to host, and in some cases use actual network load balancers.

          --
          Viktor.

          Disclaimer: off-list followups get on-list replies or get ignored.
          Please do not ignore the "Reply-To" header.

          To unsubscribe from the postfix-users list, visit
          http://www.postfix.org/lists.html or click the link below:
          <mailto:majordomo@...?body=unsubscribe%20postfix-users>

          If my response solves your problem, the best way to thank me is to not
          send an "it worked, thanks" follow-up. If you must respond, please put
          "It worked, thanks" in the "Subject" so I can delete these quickly.
        • Dan Lannom
          ... I m using virtual(5) and relay_recipients which is set to $virtual_alias_maps. ... I ll try out proxymap. I ve always considered our email environment
          Message 4 of 4 , Jun 1, 2007
          • 0 Attachment
            Victor Duchovni wrote:
            > On Fri, Jun 01, 2007 at 02:14:34PM -0400, Dan Lannom wrote:
            >
            >> I'm trying to use a round robin DNS entry to distribute the queries over
            >> multiple ldap servers, but for some reason they are all being redirected
            >> to the same system, which is pushing load too high.
            >>
            >
            > In what Postfix context are you using LDAP? Are you using "proxy:ldap:..."?
            > Is transport(5) lookups?

            I'm using virtual(5) and relay_recipients which is set to
            $virtual_alias_maps.

            >
            > - Most people who care about load, have high volume systems, where
            > proxymap(8) is a must for LDAP queries from smtpd(8) and cleanup(8),
            > so the picture changes. Each proxymap(8) process (many fewer of
            > these) connects to an LDAP server and stays connected. These connection
            > count is one or two orders of magnitude lower, but distribution may
            > be uneven, and stay uneven, if you roll "heads" a few times in a row.

            I'll try out proxymap. I've always considered our email environment
            small since we're only accepting ~200k messages per day

            I'm typically seeing 60-120 established LDAP sessions all to the same
            host, so its more than a few bad rolls.

            > On my "busy" server using LDAP via proxymap for virtual(5) rewrites, the
            > process counts (sampled just now) are:
            >
            > 76 smtpd
            > 21 cleanup
            > 7 proxymap
            >
            > so 97 would-be LDAP clients are served by 7 LDAP connections from
            > 7 proxymap processes, which if LDAP cooperates each pick their
            > own IP address for the LDAP server.
            >
            > I don't actually use multiple A records for LDAP load balancing, rather
            > for machines in a cluster, I shuffle the LDAP server preference list from
            > host to host, and in some cases use actual network load balancers.

            I'll switch to the shuffle method, I was probably trying to do too much
            with DNS.

            Thanks,

            Dan
          Your message has been successfully submitted and would be delivered to recipients shortly.