Loading ...
Sorry, an error occurred while loading the content.

[offtopic] When is STARTTLS allowed?

Expand Messages
  • Pedro Lamarão
    First off, I m sorry to post this rather offtopic question. I searched Google Groups for a USENET group specific to SMTP and found none. I ve just read RFC
    Message 1 of 12 , Jan 31, 2007
    • 0 Attachment
      First off, I'm sorry to post this rather offtopic question.
      I searched Google Groups for a USENET group specific to SMTP and found none.

      I've just read RFC 2487 and was left wondering exactly when is STARTTLS
      allowed.
      The obvious use case is to use it just after the TCP connection was
      established, when the session is at the "initial" state.
      But is it allowed after MAIL, when the session is at the "envelope" state?
      Is it allowed after RCPT during the "envelope" state?
      (This state terminology is mine, sorry if it is too confusing; I'm
      trying to build a "state machine" picture out of the protocol.)

      The RFC merely states that, after STARTTLS handshaking completes, the
      connection goes back to the "initial" state and a HELO or EHLO must be
      issued.

      Any help is appreciated.

      --
      Pedro Lamarão
      Desenvolvimento

      Intersix Technologies S.A.
      SP: (55 11 3803-9300)
      RJ: (55 21 3852-3240)
      www.intersix.com.br

      Your Security is our Business
    • Victor Duchovni
      ... With SMTP, STARTTLS is allowed only between EHLO and MAIL , and only if the server s ESMTP EHLO response includes 250-STARTTLS (or ends with 250
      Message 2 of 12 , Jan 31, 2007
      • 0 Attachment
        On Wed, Jan 31, 2007 at 04:14:52PM -0200, Pedro Lamar?o wrote:

        > First off, I'm sorry to post this rather offtopic question.
        > I searched Google Groups for a USENET group specific to SMTP and found none.
        >
        > I've just read RFC 2487 and was left wondering exactly when is STARTTLS
        > allowed.
        > The obvious use case is to use it just after the TCP connection was
        > established, when the session is at the "initial" state.
        > But is it allowed after MAIL, when the session is at the "envelope" state?
        > Is it allowed after RCPT during the "envelope" state?
        > (This state terminology is mine, sorry if it is too confusing; I'm
        > trying to build a "state machine" picture out of the protocol.)
        >
        > The RFC merely states that, after STARTTLS handshaking completes, the
        > connection goes back to the "initial" state and a HELO or EHLO must be
        > issued.

        With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
        and only if the server's ESMTP EHLO response includes "250-STARTTLS"
        (or ends with "250 STARTTLS").

        --
        Viktor.

        Disclaimer: off-list followups get on-list replies or get ignored.
        Please do not ignore the "Reply-To" header.

        To unsubscribe from the postfix-users list, visit
        http://www.postfix.org/lists.html or click the link below:
        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

        If my response solves your problem, the best way to thank me is to not
        send an "it worked, thanks" follow-up. If you must respond, please put
        "It worked, thanks" in the "Subject" so I can delete these quickly.
      • Wietse Venema
        ... A specific pointer would help. If you look at Postfix source, then you will see that it accepts STARTTLS at any protocol stage, except after STARTTLS .
        Message 3 of 12 , Jan 31, 2007
        • 0 Attachment
          Victor Duchovni:
          > On Wed, Jan 31, 2007 at 04:14:52PM -0200, Pedro Lamar?o wrote:
          >
          > > First off, I'm sorry to post this rather offtopic question.
          > > I searched Google Groups for a USENET group specific to SMTP and found none.
          > >
          > > I've just read RFC 2487 and was left wondering exactly when is STARTTLS
          > > allowed.
          > > The obvious use case is to use it just after the TCP connection was
          > > established, when the session is at the "initial" state.
          > > But is it allowed after MAIL, when the session is at the "envelope" state?
          > > Is it allowed after RCPT during the "envelope" state?
          > > (This state terminology is mine, sorry if it is too confusing; I'm
          > > trying to build a "state machine" picture out of the protocol.)
          > >
          > > The RFC merely states that, after STARTTLS handshaking completes, the
          > > connection goes back to the "initial" state and a HELO or EHLO must be
          > > issued.
          >
          > With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
          > and only if the server's ESMTP EHLO response includes "250-STARTTLS"
          > (or ends with "250 STARTTLS").

          A specific pointer would help. If you look at Postfix source, then
          you will see that it accepts "STARTTLS" at any protocol stage,
          except after "STARTTLS". Postfix accepts "STARTTLS" is accepted
          after MAIL FROM, just like RSET, HELO or EHLO, because like Pedro
          I could not find a statement to the contrary.

          Wietse
        • Pedro Lamarão
          ... By contrast, RFC 2554 explicitly states in section 4: The AUTH command is not permitted during a mail transaction. My research produced the following
          Message 4 of 12 , Jan 31, 2007
          • 0 Attachment
            Wietse Venema escreveu:
            > Victor Duchovni:
            >
            >> On Wed, Jan 31, 2007 at 04:14:52PM -0200, Pedro Lamar?o wrote:
            >>
            >>
            >>> First off, I'm sorry to post this rather offtopic question.
            >>> I searched Google Groups for a USENET group specific to SMTP and found none.
            >>>
            >>> I've just read RFC 2487 and was left wondering exactly when is STARTTLS
            >>> allowed.
            >>> The obvious use case is to use it just after the TCP connection was
            >>> established, when the session is at the "initial" state.
            >>> But is it allowed after MAIL, when the session is at the "envelope" state?
            >>> Is it allowed after RCPT during the "envelope" state?
            >>> (This state terminology is mine, sorry if it is too confusing; I'm
            >>> trying to build a "state machine" picture out of the protocol.)
            >>>
            >>> The RFC merely states that, after STARTTLS handshaking completes, the
            >>> connection goes back to the "initial" state and a HELO or EHLO must be
            >>> issued.
            >>>
            >> With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
            >> and only if the server's ESMTP EHLO response includes "250-STARTTLS"
            >> (or ends with "250 STARTTLS").
            >>
            >
            > A specific pointer would help. If you look at Postfix source, then
            > you will see that it accepts "STARTTLS" at any protocol stage,
            > except after "STARTTLS". Postfix accepts "STARTTLS" is accepted
            > after MAIL FROM, just like RSET, HELO or EHLO, because like Pedro
            > I could not find a statement to the contrary.
            >

            By contrast, RFC 2554 explicitly states in section 4:

            "The AUTH command is not permitted during a mail transaction."

            My research produced the following chart for the "SMTP State Machine":

            http://mndfck.org/~pedro.lamarao/stuff/SMTP_StateChart.png

            (It is an optimistic chart and contains only one "error" case.)

            Thank you for your help!

            --
            Pedro Lamarão
            Desenvolvimento

            Intersix Technologies S.A.
            SP: (55 11 3803-9300)
            RJ: (55 21 3852-3240)
            www.intersix.com.br

            Your Security is our Business
          • Wietse Venema
            ... HELO, EHLO, RSET and NOOP are recognized and allowed everywhere except with data in progress , and except with NOOP, they may result in a server state
            Message 5 of 12 , Jan 31, 2007
            • 0 Attachment
              Pedro Lamar?o:
              > Wietse Venema escreveu:
              > > A specific pointer would help. If you look at Postfix source, then
              > > you will see that it accepts "STARTTLS" at any protocol stage,
              > > except after "STARTTLS". Postfix accepts "STARTTLS" is accepted
              > > after MAIL FROM, just like RSET, HELO or EHLO, because like Pedro
              > > I could not find a statement to the contrary.
              > >
              >
              > By contrast, RFC 2554 explicitly states in section 4:
              >
              > "The AUTH command is not permitted during a mail transaction."
              >
              > My research produced the following chart for the "SMTP State Machine":
              >
              > http://mndfck.org/~pedro.lamarao/stuff/SMTP_StateChart.png
              >
              > (It is an optimistic chart and contains only one "error" case.)
              >
              > Thank you for your help!

              HELO, EHLO, RSET and NOOP are recognized and allowed everywhere
              except with "data in progress", and except with NOOP, they may
              result in a server state change.

              Wietse
            • Victor Duchovni
              ... 5.2 Result of the STARTTLS Command Upon completion of the TLS handshake, the SMTP protocol is reset to the initial state (the state in SMTP after a server
              Message 6 of 12 , Jan 31, 2007
              • 0 Attachment
                On Wed, Jan 31, 2007 at 01:32:41PM -0500, Wietse Venema wrote:

                > > With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
                > > and only if the server's ESMTP EHLO response includes "250-STARTTLS"
                > > (or ends with "250 STARTTLS").
                >
                > A specific pointer would help. If you look at Postfix source, then
                > you will see that it accepts "STARTTLS" at any protocol stage,
                > except after "STARTTLS". Postfix accepts "STARTTLS" is accepted
                > after MAIL FROM, just like RSET, HELO or EHLO, because like Pedro
                > I could not find a statement to the contrary.

                5.2 Result of the STARTTLS Command

                Upon completion of the TLS handshake, the SMTP protocol is reset to
                the initial state (the state in SMTP after a server issues a 220
                service ready greeting). The server MUST discard any knowledge
                obtained from the client, such as the argument to the EHLO command,
                which was not obtained from the TLS negotiation itself. The client
                MUST discard any knowledge obtained from the server, such as the list
                of SMTP service extensions, which was not obtained from the TLS
                negotiation itself. The client SHOULD send an EHLO command as the
                first command after a successful TLS negotiation.

                So we have:

                - Can't STARTTLS *before* EHLO, because the server's ESMTP feature-set
                is unknown.

                - Silly to "STARTTLS" between "MAIL" and "DATA", because "STARTTLS"
                implies an "EHLO reset". So "STARTTLS" is equivalent to:

                RSET
                STARTTLS

                and so the sender is always at the beginning of a transaction after
                STARTTLS. If it is legal to use "STARTTLS" after MAIL, it is still
                wrong :-)

                - The client should "EHLO" after STARTTLS, which again resets any
                transaction in progress.


                --
                Viktor.

                Disclaimer: off-list followups get on-list replies or get ignored.
                Please do not ignore the "Reply-To" header.

                To unsubscribe from the postfix-users list, visit
                http://www.postfix.org/lists.html or click the link below:
                <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                If my response solves your problem, the best way to thank me is to not
                send an "it worked, thanks" follow-up. If you must respond, please put
                "It worked, thanks" in the "Subject" so I can delete these quickly.
              • Wietse Venema
                ... This can be enforced with smtpd_helo_required = yes . ... There is prior art: RFC 2821 already allows EHLO/HELO in the middle of a MAIL transaction. This
                Message 7 of 12 , Jan 31, 2007
                • 0 Attachment
                  Victor Duchovni:
                  > 5.2 Result of the STARTTLS Command
                  >
                  > Upon completion of the TLS handshake, the SMTP protocol is reset to
                  > the initial state (the state in SMTP after a server issues a 220
                  > service ready greeting). The server MUST discard any knowledge
                  > obtained from the client, such as the argument to the EHLO command,
                  > which was not obtained from the TLS negotiation itself. The client
                  > MUST discard any knowledge obtained from the server, such as the list
                  > of SMTP service extensions, which was not obtained from the TLS
                  > negotiation itself. The client SHOULD send an EHLO command as the
                  > first command after a successful TLS negotiation.
                  >
                  > So we have:
                  >
                  > - Can't STARTTLS *before* EHLO, because the server's ESMTP feature-set
                  > is unknown.

                  This can be enforced with "smtpd_helo_required = yes".

                  > - Silly to "STARTTLS" between "MAIL" and "DATA", because "STARTTLS"
                  > implies an "EHLO reset".

                  There is prior art: RFC 2821 already allows EHLO/HELO in the middle
                  of a MAIL transaction. This among others is discussed in the source
                  with "XXX 2821".

                  The Postfix SMTP server checks whether the server state allows a
                  given command. In the case of STARTTLS I was unable to discover
                  explicit restrictions except that STARTTLS recursion is not allowed.

                  Wietse
                • mouss
                  ... While I agree on the principle, that is still theoritically possible: the client starts a transaction without TLS (because it s less expensive), gets
                  Message 8 of 12 , Jan 31, 2007
                  • 0 Attachment
                    Victor Duchovni wrote:
                    > - Silly to "STARTTLS" between "MAIL" and "DATA", because "STARTTLS"
                    > implies an "EHLO reset". So "STARTTLS" is equivalent to:
                    >
                    > RSET
                    > STARTTLS
                    >
                    > and so the sender is always at the beginning of a transaction after
                    > STARTTLS. If it is legal to use "STARTTLS" after MAIL, it is still
                    > wrong :-)
                    >

                    While I agree on the principle, that is still theoritically possible:
                    the client starts a transaction without TLS (because it's less
                    expensive), gets rejected (access denied or so), and tries TLS. Of
                    course, it may be simpler to close the connection and start another one...

                    > - The client should "EHLO" after STARTTLS, which again resets any
                    > transaction in progress.
                    >
                    >
                    >
                  • Lutz Jaenicke
                    ... STARTTLS is allowed even if no 250 STARTTLS was sent. A man in the middle might have modified the EHLO response sent by the remote MTA. Thats one of
                    Message 9 of 12 , Feb 1, 2007
                    • 0 Attachment
                      Victor Duchovni wrote:
                      > On Wed, Jan 31, 2007 at 04:14:52PM -0200, Pedro Lamar?o wrote:
                      >
                      >
                      >> First off, I'm sorry to post this rather offtopic question.
                      >> I searched Google Groups for a USENET group specific to SMTP and found none.
                      >>
                      >> I've just read RFC 2487 and was left wondering exactly when is STARTTLS
                      >> allowed.
                      >> The obvious use case is to use it just after the TCP connection was
                      >> established, when the session is at the "initial" state.
                      >> But is it allowed after MAIL, when the session is at the "envelope" state?
                      >> Is it allowed after RCPT during the "envelope" state?
                      >> (This state terminology is mine, sorry if it is too confusing; I'm
                      >> trying to build a "state machine" picture out of the protocol.)
                      >>
                      >> The RFC merely states that, after STARTTLS handshaking completes, the
                      >> connection goes back to the "initial" state and a HELO or EHLO must be
                      >> issued.
                      >>
                      >
                      > With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
                      > and only if the server's ESMTP EHLO response includes "250-STARTTLS"
                      > (or ends with "250 STARTTLS").
                      >
                      >
                      STARTTLS is allowed even if no 250 STARTTLS was sent. A "man in the middle"
                      might have modified the EHLO response sent by the remote MTA.
                      Thats one of reasons why the EHLO response MUST be discarded after
                      STARTTLS (the other one being that a different feature set may be valid
                      now.

                      Best regards,
                      Lutz
                    • Victor Duchovni
                      ... The MITM can also return 450 in response to STARTTTLS . If there a man in the middle, queue and retry. I see no justitification for STARTTLS without a
                      Message 10 of 12 , Feb 1, 2007
                      • 0 Attachment
                        On Thu, Feb 01, 2007 at 11:51:03AM +0100, Lutz Jaenicke wrote:

                        > > With SMTP, "STARTTLS" is allowed only between "EHLO" and "MAIL",
                        > > and only if the server's ESMTP EHLO response includes "250-STARTTLS"
                        > > (or ends with "250 STARTTLS").
                        > >
                        > STARTTLS is allowed even if no 250 STARTTLS was sent. A "man in the middle"
                        > might have modified the EHLO response sent by the remote MTA.
                        > Thats one of reasons why the EHLO response MUST be discarded after
                        > STARTTLS (the other one being that a different feature set may be valid
                        > now.

                        The MITM can also return 450 in response to "STARTTTLS". If there a
                        man in the middle, queue and retry. I see no justitification for
                        STARTTLS without a server announcement. Furthermore, section 7 of
                        RFC 2487 includes this text:

                        A man-in-the-middle attack can be launched by deleting the "250
                        STARTTLS" response from the server. This would cause the client not
                        to try to start a TLS session. An SMTP client can protect against
                        this attack by recording the fact that a particular SMTP server
                        offers TLS during one session and generating an alarm if it does not
                        appear in the EHLO response for a later session. The lack of TLS
                        during a session SHOULD NOT result in the bouncing of email, although
                        it could result in delayed processing

                        Nothing here about sending "STARTTLS" when it is not offered, rather the
                        opposite is implicit.

                        --
                        Viktor.

                        Disclaimer: off-list followups get on-list replies or get ignored.
                        Please do not ignore the "Reply-To" header.

                        To unsubscribe from the postfix-users list, visit
                        http://www.postfix.org/lists.html or click the link below:
                        <mailto:majordomo@...?body=unsubscribe%20postfix-users>

                        If my response solves your problem, the best way to thank me is to not
                        send an "it worked, thanks" follow-up. If you must respond, please put
                        "It worked, thanks" in the "Subject" so I can delete these quickly.
                      • Pedro Lamarão
                        ... Thanks for your feedback. I ve updated the chart in the URL above to include NOOP in Start , Reset and Envelope in Progress , and HELO/EHLO in
                        Message 11 of 12 , Feb 1, 2007
                        • 0 Attachment
                          Wietse Venema escreveu:
                          > Pedro Lamar?o:
                          >
                          >> My research produced the following chart for the "SMTP State Machine":
                          >>
                          >> http://mndfck.org/~pedro.lamarao/stuff/SMTP_StateChart.png
                          >>
                          >> (It is an optimistic chart and contains only one "error" case.)
                          >>
                          >> Thank you for your help!
                          >>
                          >
                          > HELO, EHLO, RSET and NOOP are recognized and allowed everywhere
                          > except with "data in progress", and except with NOOP, they may
                          > result in a server state change.
                          >

                          Thanks for your feedback.

                          I've updated the chart in the URL above to include NOOP in "Start",
                          "Reset" and "Envelope in Progress", and HELO/EHLO in "Envelope in Progress".

                          But I may be still missing something. HELO, EHLO and RSET, when issued
                          in the "Envelope in Progress" state, _may_ result in a state change? Is
                          it permitted, then, to _not_ reset the mail transaction if these
                          commands are issued after MAIL?

                          This is a complex protocol indeed.

                          --
                          Pedro Lamarão

                          --
                          Pedro Lamarão
                          Desenvolvimento

                          Intersix Technologies S.A.
                          SP: (55 11 3803-9300)
                          RJ: (55 21 3852-3240)
                          www.intersix.com.br

                          Your Security is our Business
                        • Mark Martinec
                          Pedro, ... As a sidenote, you may rename a state envelope in progress to a transaction in progress or in transaction . RFC 2821 terminology: 3.3 Mail
                          Message 12 of 12 , Feb 1, 2007
                          • 0 Attachment
                            Pedro,

                            > >> http://mndfck.org/~pedro.lamarao/stuff/SMTP_StateChart.png

                            > I've updated the chart in the URL above to include NOOP in "Start",
                            > "Reset" and "Envelope in Progress", and HELO/EHLO in "Envelope in
                            > Progress".

                            As a sidenote, you may rename a state 'envelope in progress'
                            to a 'transaction in progress' or 'in transaction'.

                            RFC 2821 terminology:


                            3.3 Mail Transactions

                            There are three steps to SMTP mail transactions. The transaction
                            starts with a MAIL command which gives the sender identification.
                            (In general, the MAIL command may be sent only when no mail
                            transaction is in progress; see section 4.1.4.) A series of one or
                            more RCPT commands follows giving the receiver information. Then a
                            DATA command initiates transfer of the mail data and is terminated by
                            the "end of mail" data indicator, which also confirms the
                            transaction.

                            The first step in the procedure is the MAIL command.
                            MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>

                            This command tells the SMTP-receiver that a new mail transaction is
                            starting and to reset all its state tables and buffers, including any
                            recipients or mail data. ...

                            Mark
                          Your message has been successfully submitted and would be delivered to recipients shortly.