Loading ...
Sorry, an error occurred while loading the content.

Syntax matching. extend inside normal inside keepend

Expand Messages
  • Ilya
    Hello.I have a question regarding syntax matching. I have some kind of syntax and I have some solution to highlight it, but it does not work the way I
    Message 1 of 12 , Sep 1, 2006
    • 0 Attachment
      Hello.

      I have a question regarding syntax matching. I have some kind of syntax
      and I have some solution to highlight it, but it does not work the way I
      expect it to.

      What I want: match syntax that consists of blocks (enclosed in {}),
      strings (enclosed in "") and identifiers (starts with ${ and ends with
      }). Block end should also end any string that starts inside this block.

      My solution:

      :syn cluster Top contains=Block,String,Identifier
      :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
      :syn region String start=+"+ end=+"+ contains=Identifier
      :syn region Identifier start=+\${+ end=+}+ extend

      Here is a sample that shows that this solution does not work:

      {
      "string ${var} string"
      }

      For some reason an Identifier region ends a String region and a second
      word 'string' is not highlighted as a string (and Vim says that it has
      Block match there).

      Question: why the String region is not restored after the Identifier
      region ends?

      P.S. Removing keepend from a Block makes highlighting right in this
      example but then an important point that Block should end unmatched
      String is lost.

      P.P.S. Solution code is in 1.vim and test code is in 1.vim.test so you
      may try it on your vim with

      vim -s 1.vim 1.vim.test

      Ilya
    • Yakov Lerner
      ... When I removed both keepend and extend , it started to work as expected. With kepend extend , it apparently takes identifier-closing brace for a
      Message 2 of 12 , Sep 2, 2006
      • 0 Attachment
        On 9/2/06, Ilya <ilya@...> wrote:
        > Hello.
        >
        > I have a question regarding syntax matching. I have some kind of syntax
        > and I have some solution to highlight it, but it does not work the way I
        > expect it to.
        >
        > What I want: match syntax that consists of blocks (enclosed in {}),
        > strings (enclosed in "") and identifiers (starts with ${ and ends with
        > }). Block end should also end any string that starts inside this block.
        >
        > My solution:
        >
        > :syn cluster Top contains=Block,String,Identifier
        > :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
        > :syn region String start=+"+ end=+"+ contains=Identifier
        > :syn region Identifier start=+\${+ end=+}+ extend
        >
        > Here is a sample that shows that this solution does not work:
        >
        > {
        > "string ${var} string"
        > }
        >
        > For some reason an Identifier region ends a String region and a second
        > word 'string' is not highlighted as a string (and Vim says that it has
        > Block match there).
        >
        > Question: why the String region is not restored after the Identifier
        > region ends?

        When I removed both 'keepend' and 'extend', it
        started to work as expected.

        With ''kepend extend', it
        apparently takes identifier-closing brace for a
        block-closing brace. I don't know why.

        You might also try another solution: the 'syn match'
        for identifier instead of region.

        Yakov
      • Ilya
        ... No, it would not work as I want it to. Block would not end unclosed String. Like this: { Some string with a quote ( ) inside } ... second word string
        Message 3 of 12 , Sep 3, 2006
        • 0 Attachment
          Yakov Lerner wrote:
          > When I removed both 'keepend' and 'extend', it
          > started to work as expected.
          No, it would not work as I want it to. Block would not end unclosed
          String. Like this:
          { Some string with a quote ( " ) inside }
          >
          > With ''kepend extend', it
          > apparently takes identifier-closing brace for a
          > block-closing brace. I don't know why.
          Again no. I've wrote wrote about it in my original email:
          > For some reason an Identifier region ends a String region and a
          second word 'string' is not highlighted as a string (and Vim says that
          it has Block match there).
          So Block is not ended, but String is.
          >
          > You might also try another solution: the 'syn match'
          > for identifier instead of region.
          I've tried using match. And there is not difference to using region.
          >
          > Yakov
          >
          >
        • Yakov Lerner
          ... Ah, I have a guess what can be the problem. Maybe region does not like it when ending pattern ([ ]) is same as beginning pattern. It does not know then
          Message 4 of 12 , Sep 3, 2006
          • 0 Attachment
            On 9/3/06, Ilya <ilya@...> wrote:
            > Yakov Lerner wrote:
            > > When I removed both 'keepend' and 'extend', it
            > > started to work as expected.
            > No, it would not work as I want it to. Block would not end unclosed
            > String. Like this:
            > { Some string with a quote ( " ) inside }

            Ah, I have a guess what can be the problem. Maybe
            region does not like it when ending pattern (["]) is same as
            beginning pattern. It does not know then whether to treat 2nd
            double-quote as a beginning of new nested
            region (string), or end of previous block (string).

            Maybe you should try
            'syn match' for both strings and identifiers.
            (nested 'syn match'). Maybe you should look into perl.vim
            how it does it.

            Yakov
          • Ilya
            ... I do not understand how ending pattern for the String region could be connected to my case, could you please explain in more detail what is your point?
            Message 5 of 12 , Sep 3, 2006
            • 0 Attachment
              Yakov Lerner wrote:
              > On 9/3/06, Ilya <ilya@...> wrote:
              >> Yakov Lerner wrote:
              >> > When I removed both 'keepend' and 'extend', it
              >> > started to work as expected.
              >> No, it would not work as I want it to. Block would not end unclosed
              >> String. Like this:
              >> { Some string with a quote ( " ) inside }
              >
              > Ah, I have a guess what can be the problem. Maybe
              > region does not like it when ending pattern (["]) is same as
              > beginning pattern. It does not know then whether to treat 2nd
              > double-quote as a beginning of new nested
              > region (string), or end of previous block (string).
              I do not understand how ending pattern for the String region could be
              connected to my case, could you please explain in more detail what is
              your point?

              Note that I want `" ) inside ' to be highlighted as a String, but with
              `}' String highlight should stop. And this could be easily achieved
              with proposed solution. Problem arises only when String contains
              Identifier. In this case Identifier region ends also enclosing a String
              region, but not enclosing Block region. Here is picture:
              { "string ${var} string" }
              BBBBBBBBBBBBBBBBBBBBBBBBBB
              SSSSSSSSSSSSSS SS
              IIIIII
              B - Block region
              S - String region
              I - Identifier region

              I believe that matching could be as follows:
              { "string ${var} string" }
              BBBBBBBBBBBBBBBBBBBBBBBBBB
              SSSSSSSSSSSSSSSSSSSSSS
              IIIIII

              Here is a picture for Block that contains only one quote. It is
              perfectly legal and match my expectations:
              { string with a quote ( " ) inside }
              BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
              SSSSSSSSSSS
              >
              >
              > Maybe you should try
              > 'syn match' for both strings and identifiers.
              I've tried using match for both the String and the Identifier - there is
              absolutely no difference from using two regions or using one region and
              one match.
              > (nested 'syn match'). Maybe you should look into perl.vim
              > how it does it.
              I'll gladly look there, but could you please be more specific? Where in
              Perl syntax there is such a situation when outer block ends all
              contained blocks of different type? I've used perl syntax highlight for
              some time and I cannot remember if I thought something similar there and
              rereading and understanding whole syntax highlight scheme, I believe, is
              quite a bit of work...

              In Perl "everything" could be understood from syntax. There is no need
              to "force" some region to end "unexpectedly". I'm trying to improve Tcl
              highlight - there you could not tell whether some Block contains code
              and thus should be highlighted appropriately or it contains some
              arbitrary data that does not follow any of the Tcl code rules or follow
              some "other code" rules (like been SQL code), without evaluation program
              itself.
              >
              > Yakov
              >
              >
            • Ilya
              Sorry for duplicating, but my mailer for some reason removed several spaces from my pictures . ... I do not understand how ending pattern for the String
              Message 6 of 12 , Sep 3, 2006
              • 0 Attachment
                Sorry for duplicating, but my mailer for some reason removed several
                spaces from my "pictures".

                Yakov Lerner wrote:
                > On 9/3/06, Ilya <ilya@...> wrote:
                >> Yakov Lerner wrote:
                >> > When I removed both 'keepend' and 'extend', it
                >> > started to work as expected.
                >> No, it would not work as I want it to. Block would not end unclosed
                >> String. Like this:
                >> { Some string with a quote ( " ) inside }
                >
                > Ah, I have a guess what can be the problem. Maybe
                > region does not like it when ending pattern (["]) is same as
                > beginning pattern. It does not know then whether to treat 2nd
                > double-quote as a beginning of new nested
                > region (string), or end of previous block (string).
                I do not understand how ending pattern for the String region could be
                connected to my case, could you please explain in more detail what is
                your point?

                Note that I want `" ) inside ' to be highlighted as a String, but with
                `}' String highlight should stop. And this could be easily achieved
                with proposed solution. Problem arises only when String contains
                Identifier. In this case Identifier region ends also enclosing a String
                region, but not enclosing Block region. Here is picture:
                { "string ${var} string" }
                BBBBBBBBBBBBBBBBBBBBBBBBBB
                SSSSSSSSSSSSSS SS
                IIIIII
                B - Block region
                S - String region
                I - Identifier region

                I believe that matching could be as follows:
                { "string ${var} string" }
                BBBBBBBBBBBBBBBBBBBBBBBBBB
                SSSSSSSSSSSSSSSSSSSSSS
                IIIIII

                Here is a picture for Block that contains only one quote. It is
                perfectly legal and match my expectations:
                { string with a quote ( " ) inside }
                BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
                SSSSSSSSSSS
                >
                >
                > Maybe you should try
                > 'syn match' for both strings and identifiers.
                I've tried using match for both the String and the Identifier - there is
                absolutely no difference from using two regions or using one region and
                one match.
                > (nested 'syn match'). Maybe you should look into perl.vim
                > how it does it.
                I'll gladly look there, but could you please be more specific? Where in
                Perl syntax there is such a situation when outer block ends all
                contained blocks of different type? I've used perl syntax highlight for
                some time and I cannot remember if I thought something similar there and
                rereading and understanding whole syntax highlight scheme, I believe, is
                quite a bit of work...

                In Perl "everything" could be understood from syntax. There is no need
                to "force" some region to end "unexpectedly". I'm trying to improve Tcl
                highlight - there you could not tell whether some Block contains code
                and thus should be highlighted appropriately or it contains some
                arbitrary data that does not follow any of the Tcl code rules or follow
                some "other code" rules (like been SQL code), without evaluation program
                itself.
                >
                > Yakov
                >
                >
              • Yakov Lerner
                ... Your keepend option clearly causes } which is end of identifier to terminate the enclosing block, in full accordance with :help syn-keepend. If I read
                Message 7 of 12 , Sep 3, 2006
                • 0 Attachment
                  On 9/2/06, Ilya <ilya@...> wrote:
                  > Hello.
                  >
                  > I have a question regarding syntax matching. I have some kind of syntax
                  > and I have some solution to highlight it, but it does not work the way I
                  > expect it to.
                  >
                  > What I want: match syntax that consists of blocks (enclosed in {}),
                  > strings (enclosed in "") and identifiers (starts with ${ and ends with
                  > }). Block end should also end any string that starts inside this block.
                  >
                  > My solution:
                  >
                  > :syn cluster Top contains=Block,String,Identifier
                  > :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
                  > :syn region String start=+"+ end=+"+ contains=Identifier
                  > :syn region Identifier start=+\${+ end=+}+ extend

                  Your 'keepend' option clearly causes '}' which is end of
                  identifier to terminate the enclosing block, in full accordance
                  with :help syn-keepend.
                  If I read ':help syn-keepend' right, then 'keepend' does not
                  allow a region on which it is used to have proper subregions.

                  Yakov
                • Ilya
                  ... No, it does not because of an extend option. Here is a picture (I ve already included it in my previous mail): { string ${var} string }
                  Message 8 of 12 , Sep 3, 2006
                  • 0 Attachment
                    Yakov Lerner wrote:
                    > On 9/2/06, Ilya <ilya@...> wrote:
                    >> Hello.
                    >>
                    >> I have a question regarding syntax matching. I have some kind of syntax
                    >> and I have some solution to highlight it, but it does not work the way I
                    >> expect it to.
                    >>
                    >> What I want: match syntax that consists of blocks (enclosed in {}),
                    >> strings (enclosed in "") and identifiers (starts with ${ and ends with
                    >> }). Block end should also end any string that starts inside this block.
                    >>
                    >> My solution:
                    >>
                    >> :syn cluster Top contains=Block,String,Identifier
                    >> :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
                    >> :syn region String start=+"+ end=+"+ contains=Identifier
                    >> :syn region Identifier start=+\${+ end=+}+ extend
                    >
                    > Your 'keepend' option clearly causes '}' which is end of
                    > identifier to terminate the enclosing block, in full accordance
                    > with :help syn-keepend.
                    No, it does not because of an extend option. Here is a picture (I've
                    already included it in my previous mail):
                    { "string ${var} string" }
                    BBBBBBBBBBBBBBBBBBBBBBBBBB
                    SSSSSSSSSSSSSS SS
                    IIIIII
                    B - Block region
                    S - String region
                    I - Identifier region

                    I got this information using synID function.
                    > If I read ':help syn-keepend' right, then 'keepend' does not
                    > allow a region on which it is used to have proper subregions.
                    Except for 'extend' regions. That is why Block is 'keepend' and 'extend'.
                    >
                    > Yakov
                    >
                    >
                  • Yakov Lerner
                    ... What about the following example: { 11111 } 2222 { } Do you want to distinguish mismatched double-quotes within one line only, or across multiple lines
                    Message 9 of 12 , Sep 3, 2006
                    • 0 Attachment
                      On 9/3/06, Ilya <ilya@...> wrote:
                      > Yakov Lerner wrote:
                      > > When I removed both 'keepend' and 'extend', it
                      > > started to work as expected.
                      > No, it would not work as I want it to. Block would not end unclosed
                      > String. Like this:
                      > { Some string with a quote ( " ) inside }

                      What about the following example:
                      {
                      " 11111 }
                      2222 { "
                      }
                      Do you want to distinguish mismatched double-quotes
                      within one line only, or across multiple lines ? As you
                      syntax is currently written, your intention is ambiguoius.
                      On one hand, your 'syntax region String' can span multiple lines.
                      On other hand, you want that '}' inside quoted string terminate a block.

                      Do you want to consider line '" 11111 }' a line with
                      mismatched quote, or it's supposed to match quote on
                      after '2222 {' ? NB that your 'syntax region String' does allow
                      strings spanning multilpe lines.

                      Yakov
                    • Yakov Lerner
                      ... What if extend can override only the keepend of only immediately enclosing region ? The example in :help syn-extend only has examlpe of overriding
                      Message 10 of 12 , Sep 3, 2006
                      • 0 Attachment
                        On 9/3/06, Ilya <ilya@...> wrote:
                        > Yakov Lerner wrote:
                        > > On 9/2/06, Ilya <ilya@...> wrote:
                        > >> Hello.
                        > >>
                        > >> I have a question regarding syntax matching. I have some kind of syntax
                        > >> and I have some solution to highlight it, but it does not work the way I
                        > >> expect it to.
                        > >>
                        > >> What I want: match syntax that consists of blocks (enclosed in {}),
                        > >> strings (enclosed in "") and identifiers (starts with ${ and ends with
                        > >> }). Block end should also end any string that starts inside this block.
                        > >>
                        > >> My solution:
                        > >>
                        > >> :syn cluster Top contains=Block,String,Identifier
                        > >> :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
                        > >> :syn region String start=+"+ end=+"+ contains=Identifier
                        > >> :syn region Identifier start=+\${+ end=+}+ extend
                        > >
                        > > Your 'keepend' option clearly causes '}' which is end of
                        > > identifier to terminate the enclosing block, in full accordance
                        > > with :help syn-keepend.
                        > No, it does not because of an extend option. Here is a picture (I've
                        > already included it in my previous mail):
                        > { "string ${var} string" }
                        > BBBBBBBBBBBBBBBBBBBBBBBBBB
                        > SSSSSSSSSSSSSS SS
                        > IIIIII
                        > B - Block region
                        > S - String region
                        > I - Identifier region
                        >
                        > I got this information using synID function.
                        > > If I read ':help syn-keepend' right, then 'keepend' does not
                        > > allow a region on which it is used to have proper subregions.
                        > Except for 'extend' regions. That is why Block is 'keepend' and 'extend'.

                        What if 'extend' can override only the 'keepend' of only
                        immediately enclosing region ? The example in :help syn-extend
                        only has examlpe of overriding 'keepend' of immediately containing
                        region. Does not say whether 'extend' acts all the way up
                        the stack of enclosing regions.

                        Yakov
                      • Ilya
                        ... That is nearly my question (look at the topic). But, if you consider the way Vim currently takes an extend region that is inside a normal region that is
                        Message 11 of 12 , Sep 4, 2006
                        • 0 Attachment
                          Yakov Lerner wrote:
                          > On 9/3/06, Ilya <ilya@...> wrote:
                          >> Yakov Lerner wrote:
                          >> > On 9/2/06, Ilya <ilya@...> wrote:
                          >> >> Hello.
                          >> >>
                          >> >> I have a question regarding syntax matching. I have some kind of
                          >> syntax
                          >> >> and I have some solution to highlight it, but it does not work the
                          >> way I
                          >> >> expect it to.
                          >> >>
                          >> >> What I want: match syntax that consists of blocks (enclosed in {}),
                          >> >> strings (enclosed in "") and identifiers (starts with ${ and ends
                          >> with
                          >> >> }). Block end should also end any string that starts inside this
                          >> block.
                          >> >>
                          >> >> My solution:
                          >> >>
                          >> >> :syn cluster Top contains=Block,String,Identifier
                          >> >> :syn region Block start=+{+ end=+}+ keepend extend contains=@Top
                          >> >> :syn region String start=+"+ end=+"+ contains=Identifier
                          >> >> :syn region Identifier start=+\${+ end=+}+ extend
                          >> >
                          >> > Your 'keepend' option clearly causes '}' which is end of
                          >> > identifier to terminate the enclosing block, in full accordance
                          >> > with :help syn-keepend.
                          >> No, it does not because of an extend option. Here is a picture (I've
                          >> already included it in my previous mail):
                          >> { "string ${var} string" }
                          >> BBBBBBBBBBBBBBBBBBBBBBBBBB
                          >> SSSSSSSSSSSSSS SS
                          >> IIIIII
                          >> B - Block region
                          >> S - String region
                          >> I - Identifier region
                          >>
                          >> I got this information using synID function.
                          >> > If I read ':help syn-keepend' right, then 'keepend' does not
                          >> > allow a region on which it is used to have proper subregions.
                          >> Except for 'extend' regions. That is why Block is 'keepend' and
                          >> 'extend'.
                          >
                          > What if 'extend' can override only the 'keepend' of only
                          > immediately enclosing region ? The example in :help syn-extend
                          > only has examlpe of overriding 'keepend' of immediately containing
                          > region. Does not say whether 'extend' acts all the way up
                          > the stack of enclosing regions.
                          That is nearly my question (look at the topic). But, if you consider
                          the way Vim currently takes an extend region that is inside a normal
                          region that is inside a keeped you can see (as I've pictured) that
                          external keepend region is not ended but middle normal region is. And
                          this is strange to me. I was expecting someone to comment on this
                          matter. Is there any sense in this approach or it is kind of a bug?
                          >
                          > Yakov
                          >
                          >
                        • Ilya
                          ... In your example, according to the rules in my solution, there should be 2 Blocks with 2 Strings inside. And both Strings would be without closing . Like
                          Message 12 of 12 , Sep 4, 2006
                          • 0 Attachment
                            Yakov Lerner wrote:
                            > On 9/3/06, Ilya <ilya@...> wrote:
                            >> Yakov Lerner wrote:
                            >> > When I removed both 'keepend' and 'extend', it
                            >> > started to work as expected.
                            >> No, it would not work as I want it to. Block would not end unclosed
                            >> String. Like this:
                            >> { Some string with a quote ( " ) inside }
                            >
                            > What about the following example:
                            > {
                            > " 11111 }
                            > 2222 { "
                            > }
                            > Do you want to distinguish mismatched double-quotes
                            > within one line only, or across multiple lines ? As you
                            > syntax is currently written, your intention is ambiguoius.
                            > On one hand, your 'syntax region String' can span multiple lines.
                            > On other hand, you want that '}' inside quoted string terminate a block.
                            >
                            > Do you want to consider line '" 11111 }' a line with
                            > mismatched quote, or it's supposed to match quote on
                            > after '2222 {' ? NB that your 'syntax region String' does allow
                            > strings spanning multilpe lines.

                            In your example, according to the rules in my solution, there should be
                            2 Blocks with 2 Strings inside. And both Strings would be without
                            closing ".
                            Like this:

                            {
                            BB
                            " 11111 }
                            BBBBBBBBBBBBBBBBBSSSSSSSSB
                            2222 { "
                            BBSS
                            }
                            SSSSSSB

                            And my solution work just as I expect it to on this code.

                            >
                            > Yakov
                            >
                            >
                          Your message has been successfully submitted and would be delivered to recipients shortly.