Loading ...
Sorry, an error occurred while loading the content.

ldap_table and insignificant spaces

Expand Messages
  • Bastian Blank
    Hi I found that one MTA bounced several mails. The mails where sent to ` test @example.com and accepted by Postfix. The backend LMTP then rejected the mails.
    Message 1 of 6 , Mar 1, 2013
    • 0 Attachment
      Hi

      I found that one MTA bounced several mails. The mails where sent to
      `" test"@...' and accepted by Postfix. The backend LMTP then
      rejected the mails.

      This is what I found out:
      - RCPT TO:<" test"@...>
      - The ldap table gets the sanitized address: ` test@...'
      (note the leading space is still there).
      - This is converted into a ldap query (mail= test@...).
      - The ldap server sanitices the query to (mail=test@...) as
      mandated by RFC 4717, 4.2.3; it removes the insignificant space.

      I found no way to supress this insignificant space removal by using
      different matches, there exists none with the requests capabilities.
      Also escaping obviously does not make a difference.

      Not sure if something should be done about it. At least it is a
      surprising outcome for a simple question; while both parties works
      perfectly fine.

      Bastian

      --
      Insults are effective only where emotion is present.
      -- Spock, "Who Mourns for Adonais?" stardate 3468.1
    • Viktor Dukhovni
      ... I see the insignificant space handling defined in 4518 (referenced from 4517). https://tools.ietf.org/html/rfc4518#section-2.6.1 It seems to suggest that
      Message 2 of 6 , Mar 5, 2013
      • 0 Attachment
        On Fri, Mar 01, 2013 at 03:19:42PM +0100, Bastian Blank wrote:

        > I found that one MTA bounced several mails. The mails where sent to
        > " test"@... and accepted by Postfix. The backend LMTP then
        > rejected the mails.
        >
        > This is what I found out:
        > - RCPT TO:<" test"@...>
        > - The ldap table gets the sanitized address: ` test@...'
        > (note the leading space is still there).
        > - This is converted into a ldap query (mail= test@...).
        > - The ldap server sanitices the query to (mail=test@...) as
        > mandated by RFC 4717, 4.2.3; it removes the insignificant space.

        I see the "insignificant space handling" defined in 4518 (referenced
        from 4517).

        https://tools.ietf.org/html/rfc4518#section-2.6.1

        It seems to suggest that exact string matches should take the form

        attr=<SPACE>value<SPACE>

        where any spaces inside the value are encoded as <SPACE><SPACE>.

        Is this backwards compatible with older LDAP servers that are not
        UTF-8 based?

        An easy way to achieve this would be:

        query_filter = (mail = %s )

        if such spaces are not removed at a higher level by the LDAP library.
        Does this help?

        > Not sure if something should be done about it. At least it is a
        > surprising outcome for a simple question; while both parties works
        > perfectly fine.

        Another thing that could help is if Postfix would use the "external"
        form of the address:

        " test"@...

        with the quotes as the query string. I seem to recall that this is
        already the case with lookup keys in virtual_alias_maps, but it
        may not be the case with other tables. Which Postfix "mumble_maps"
        parameter are you using with LDAP?

        Arguably, all lookup keys in tables should be in "external" (RFC-5322)
        form. The suggested doubling of internal spaces is far less important
        in practice that avoiding the loss of leading spaces.

        --
        Viktor.
      • Wietse Venema
        ... Which external form? - RFC 5321 and 5322 have different rules. - Worse, The mappings are not one-to-one. That is, there are multiple correct
        Message 3 of 6 , Mar 5, 2013
        • 0 Attachment
          Viktor Dukhovni:
          > Arguably, all lookup keys in tables should be in "external" (RFC-5322)
          > form. The suggested doubling of internal spaces is far less important
          > in practice that avoiding the loss of leading spaces.

          Which external form?

          - RFC 5321 and 5322 have different rules.

          - Worse, The mappings are not one-to-one. That is, there are multiple
          correct implementations for quote_821_local() and quote_822_local().

          Wietse
        • Viktor Dukhovni
          ... I was a bit sloppy in my wording, I guess 5321 is the right one for processing envelope addresses, and perhaps 5322 for processing header addresses (with
          Message 4 of 6 , Mar 5, 2013
          • 0 Attachment
            On Tue, Mar 05, 2013 at 11:16:18PM -0500, Wietse Venema wrote:

            > Viktor Dukhovni:
            > > Arguably, all lookup keys in tables should be in "external" (RFC-5322)
            > > form. The suggested doubling of internal spaces is far less important
            > > in practice that avoiding the loss of leading spaces.
            >
            > Which external form?
            >
            > - RFC 5321 and 5322 have different rules.

            I was a bit sloppy in my wording, I guess 5321 is the right one
            for processing envelope addresses, and perhaps 5322 for processing
            header addresses (with canonical and generic rewrites for example).

            This complicates recipient extension support, since to compute an
            address sans extension we'd need to dequote, lose the extension
            and then requote (using the right quoting style). This is not
            trivial.

            > - Worse, The mappings are not one-to-one. That is, there are multiple
            > correct implementations for quote_821_local() and quote_822_local().

            So long as we pick something consistent and documented, users can
            likely adapt their table keys. In most cases the external form
            and the internal form are identical, but the exceptions are a pain.

            For this specific LDAP issue, if changing the query works that's
            the fastest path to success, and we can adjust the ldap_table(5)
            document with the latest best practice syntax.

            The more things change, the less they stay the same. :-)

            --
            Viktor.
          • Bastian Blank
            ... I forgot that, yes. ... This stunt is done to support working substring matches. It does not help here and you can t trick it by adding more spaces. ...
            Message 5 of 6 , Mar 6, 2013
            • 0 Attachment
              On Wed, Mar 06, 2013 at 03:51:28AM +0000, Viktor Dukhovni wrote:
              > On Fri, Mar 01, 2013 at 03:19:42PM +0100, Bastian Blank wrote:
              > > - The ldap server sanitices the query to (mail=test@...) as
              > > mandated by RFC 4717, 4.2.3; it removes the insignificant space.
              > I see the "insignificant space handling" defined in 4518 (referenced
              > from 4517).
              > https://tools.ietf.org/html/rfc4518#section-2.6.1

              I forgot that, yes.

              > It seems to suggest that exact string matches should take the form
              > attr=<SPACE>value<SPACE>
              > where any spaces inside the value are encoded as <SPACE><SPACE>.

              This stunt is done to support working substring matches. It does not
              help here and you can't trick it by adding more spaces.

              > Is this backwards compatible with older LDAP servers that are not
              > UTF-8 based?

              LDAPv3 was always UTF-8 based.

              > An easy way to achieve this would be:
              > query_filter = (mail = %s )
              > if such spaces are not removed at a higher level by the LDAP library.
              > Does this help?

              No, this won't help, as the syntax rules will sanitize also the
              additional spaces.

              > > Not sure if something should be done about it. At least it is a
              > > surprising outcome for a simple question; while both parties works
              > > perfectly fine.
              > Another thing that could help is if Postfix would use the "external"
              > form of the address:
              > " test"@...
              > with the quotes as the query string.

              I don't think this would be a good idea. It still includes a N-to-1
              mapping of spaces.

              I think in this case all queries including spaces should be ignored by
              ldap_table, because of the way LDAP works with them.

              > I seem to recall that this is
              > already the case with lookup keys in virtual_alias_maps, but it
              > may not be the case with other tables.

              This makes sense. In my tests such addresses where not rewritten as
              expected.

              > Which Postfix "mumble_maps"
              > parameter are you using with LDAP?

              transport, virtual_alias, virtual_alias_domains, virtual_mailbox,
              virtual_mailbox_domains, relay_recipient, relay_domains

              Bastian

              --
              All your people must learn before you can reach for the stars.
              -- Kirk, "The Gamesters of Triskelion", stardate 3259.2
            • Wietse Venema
              ... Oh, and did I mention that some lookup tables are used for both headers and envelopes, which have different quoting rules and therefore different
              Message 6 of 6 , Mar 6, 2013
              • 0 Attachment
                Viktor Dukhovni:
                > On Tue, Mar 05, 2013 at 11:16:18PM -0500, Wietse Venema wrote:
                >
                > > Viktor Dukhovni:
                > > > Arguably, all lookup keys in tables should be in "external" (RFC-5322)
                > > > form. The suggested doubling of internal spaces is far less important
                > > > in practice that avoiding the loss of leading spaces.
                > >
                > > Which external form?
                > >
                > > - RFC 5321 and 5322 have different rules.
                >
                > I was a bit sloppy in my wording, I guess 5321 is the right one
                > for processing envelope addresses, and perhaps 5322 for processing
                > header addresses (with canonical and generic rewrites for example).

                Oh, and did I mention that some lookup tables are used for both
                headers and envelopes, which have different quoting rules and
                therefore different "external" forms?

                > This complicates recipient extension support, since to compute an
                > address sans extension we'd need to dequote, lose the extension
                > and then requote (using the right quoting style). This is not
                > trivial.

                It may be more productive to add a dbcommon etc. flag that says
                query with the external form of an address (but keep using internal
                forms in Postfix itself).

                Wietse

                > > - Worse, The mappings are not one-to-one. That is, there are multiple
                > > correct implementations for quote_821_local() and quote_822_local().
                >
                > So long as we pick something consistent and documented, users can
                > likely adapt their table keys. In most cases the external form
                > and the internal form are identical, but the exceptions are a pain.
                >
                > For this specific LDAP issue, if changing the query works that's
                > the fastest path to success, and we can adjust the ldap_table(5)
                > document with the latest best practice syntax.
                >
                > The more things change, the less they stay the same. :-)
                >
                > --
                > Viktor.
                >
              Your message has been successfully submitted and would be delivered to recipients shortly.