Loading ...
Sorry, an error occurred while loading the content.

help on syntax file for fixed-width record files

Expand Messages
  • Dan Wierenga
    I m trying to create a syntax file for a file containing fixed-width records [1], so that whenever I edit a file, the records are highlighted so that each
    Message 1 of 10 , Jan 7, 2013
    • 0 Attachment
      I'm trying to create a syntax file for a file containing fixed-width records [1], so that whenever I edit a file, the records are highlighted so that each field is easily distinguishable by color.

      My file looks like this:

      A000000001IMFOO23456
      C000000002IMFOO23456
      Z000000003IMFOO23456


      The file has several record types.  A record type is declared by the first character of the line i.e. A, C, Z, etc.  The lines are 1464 characters long (!) so I included just the start of them for brevity.  Each record contains different elements, i.e. an A record has an element from position 2 through 10, 11 through 20, etc., but a C Record has elements from 2 through 10 and 11 through 24.  So, the syntax file has to account for the first character of each line.
      I'm starting on the A records, and here's what I have so far:


      :syntax region ARecord_OriginatorID start="^A" end=".{20}"
      :syntax region ARecord_RecordCount start="^A" end=".{10}"
      :syntax match ARecord_RecordType "^A" oneline



      :hi ARecord_RecordOriginatorID ctermfg=yellow
      :hi ARecord_RecordCount ctermfg=red
      :hi ARecord_RecordType ctermfg=blue


      However, this results in only the first character (i.e. "A") being blue.  My goal is that the A is blue, the next nine characters at positions 2 through 10 are red, and the next 10 characters are yellow.  

      Any help on this would be appreciated.  I feel like I'm missing something fundamental to syntax patterns, but I'm not sure what.  This is my first attempt at a syntax file of any kind.

      Thanks,
      Dan


      [1] Specifically, a Canadian Payments Association Standard 005 AFT File.  https://www.rbcroyalbank.com/ach/file-451771.pdf

      --
      You received this message from the "vim_use" maillist.
      Do not top-post! Type your reply below the text you are replying to.
      For more information, visit http://www.vim.org/maillist.php
    • Vlad Irnov
      ... syntax match ARecord_RecordType /^A/ contains=ALL syntax match ARecord_RecordCount /^A. {9}/ contains=ALL syntax match ARecord_RecordOriginatorID
      Message 2 of 10 , Jan 7, 2013
      • 0 Attachment
        On 1/7/13, Dan Wierenga <dwierenga@...> wrote:
        > I'm trying to create a syntax file for a file containing fixed-width
        > records [1], so that whenever I edit a file, the records are highlighted so
        > that each field is easily distinguishable by color.
        >
        > My file looks like this:
        >
        > A000000001IMFOO23456
        > C000000002IMFOO23456
        > Z000000003IMFOO23456
        >
        >
        > The file has several record types. A record type is declared by the first
        > character of the line i.e. A, C, Z, etc. The lines are 1464 characters
        > long (!) so I included just the start of them for brevity. Each record
        > contains different elements, i.e. an A record has an element from position
        > 2 through 10, 11 through 20, etc., but a C Record has elements from 2
        > through 10 and 11 through 24. So, the syntax file has to account for the
        > first character of each line.
        > I'm starting on the A records, and here's what I have so far:
        >
        >
        > :syntax region ARecord_OriginatorID start="^A" end=".{20}"
        > :syntax region ARecord_RecordCount start="^A" end=".{10}"
        > :syntax match ARecord_RecordType "^A" oneline
        >
        >
        >
        > :hi ARecord_RecordOriginatorID ctermfg=yellow
        > :hi ARecord_RecordCount ctermfg=red
        > :hi ARecord_RecordType ctermfg=blue
        >
        >
        > However, this results in only the first character (i.e. "A") being blue.
        > My goal is that the A is blue, the next nine characters at positions 2
        > through 10 are red, and the next 10 characters are yellow.


        syntax match ARecord_RecordType /^A/ contains=ALL
        syntax match ARecord_RecordCount /^A.\{9}/ contains=ALL
        syntax match ARecord_RecordOriginatorID /^A.\{19}/ contains=ALL

        hi ARecord_RecordType ctermfg=blue
        hi ARecord_RecordCount ctermfg=red
        hi ARecord_RecordOriginatorID ctermfg=yellow


        HTH,
        Vlad

        --
        You received this message from the "vim_use" maillist.
        Do not top-post! Type your reply below the text you are replying to.
        For more information, visit http://www.vim.org/maillist.php
      • Boris Danilov
        Dan, Your solution isn t working because you re tryting to match 3 items starting with ^A . When there are several match or region patterns only the one
        Message 3 of 10 , Jan 8, 2013
        • 0 Attachment
          Dan,

          Your solution isn't working because you're tryting to match 3 items
          starting with "^A". When there are several match or region patterns
          only the one defined last will match (look :help syn-priority),
          therefore ARecord_RecordType matches for you. Once again, when
          something matches in current position, this segment normally isn't
          tried for the matches with other syntax elements.

          Have a look at :help syn-nextgroup.

          :syntax match ARecord_RecordType "^A" nextgroup=ARecord_RecordCount
          :syntax match ARecord_RecordCount ".\{9}" nextgroup=ARecord_OriginatorID
          :syntax match ARecord_OriginatorID ".\{10}"

          :syntax match CRecord_RecordType "^C" nextgroup=CRecord_RecordCount
          :syntax match CRecord_RecordCount ".\{9}" nextgroup=CRecord_OriginatorID
          :syntax match CRecord_OriginatorID ".\{14}"

          :highlight link ARecord_RecordType RecordType
          :highlight link CRecord_RecordType RecordType

          :highlight link ARecord_RecordCount RecordCount
          :highlight link CRecord_RecordCount RecordCount

          :highlight link ARecord_RecordOriginatorID RecordOriginatorID
          :highlight link CRecord_RecordOriginatorID RecordOriginatorID

          :highlight RecordType ctermfg=blue
          :highlight RecordCount ctermfg=red
          :highlight RecordOriginatorID ctermfg=yellow

          etc.

          I suggest reading :help usr_44.txt to better understand syntax basics.
          Also there is a good habbit of prepending a language suffix to all
          names of your syntactical groups like the following
          langARecord_RecordType.

          Regards,
          Boris

          On Tue, Jan 8, 2013 at 8:33 AM, Vlad Irnov <vlad.irnov@...> wrote:
          > On 1/7/13, Dan Wierenga <dwierenga@...> wrote:
          >> I'm trying to create a syntax file for a file containing fixed-width
          >> records [1], so that whenever I edit a file, the records are highlighted so
          >> that each field is easily distinguishable by color.
          >>
          >> My file looks like this:
          >>
          >> A000000001IMFOO23456
          >> C000000002IMFOO23456
          >> Z000000003IMFOO23456
          >>
          >>
          >> The file has several record types. A record type is declared by the first
          >> character of the line i.e. A, C, Z, etc. The lines are 1464 characters
          >> long (!) so I included just the start of them for brevity. Each record
          >> contains different elements, i.e. an A record has an element from position
          >> 2 through 10, 11 through 20, etc., but a C Record has elements from 2
          >> through 10 and 11 through 24. So, the syntax file has to account for the
          >> first character of each line.
          >> I'm starting on the A records, and here's what I have so far:
          >>
          >>
          >> :syntax region ARecord_OriginatorID start="^A" end=".{20}"
          >> :syntax region ARecord_RecordCount start="^A" end=".{10}"
          >> :syntax match ARecord_RecordType "^A" oneline
          >>
          >>
          >>
          >> :hi ARecord_RecordOriginatorID ctermfg=yellow
          >> :hi ARecord_RecordCount ctermfg=red
          >> :hi ARecord_RecordType ctermfg=blue
          >>
          >>
          >> However, this results in only the first character (i.e. "A") being blue.
          >> My goal is that the A is blue, the next nine characters at positions 2
          >> through 10 are red, and the next 10 characters are yellow.
          >
          >
          > syntax match ARecord_RecordType /^A/ contains=ALL
          > syntax match ARecord_RecordCount /^A.\{9}/ contains=ALL
          > syntax match ARecord_RecordOriginatorID /^A.\{19}/ contains=ALL
          >
          > hi ARecord_RecordType ctermfg=blue
          > hi ARecord_RecordCount ctermfg=red
          > hi ARecord_RecordOriginatorID ctermfg=yellow
          >
          >
          > HTH,
          > Vlad
          >
          > --
          > You received this message from the "vim_use" maillist.
          > Do not top-post! Type your reply below the text you are replying to.
          > For more information, visit http://www.vim.org/maillist.php

          --
          You received this message from the "vim_use" maillist.
          Do not top-post! Type your reply below the text you are replying to.
          For more information, visit http://www.vim.org/maillist.php
        • Boris Danilov
          ... I m sorry, there is a mistake. To prevent XRecord_RecordOriginatorID and XRecord_RecordCount match anywhere I believe you should mark them ...
          Message 4 of 10 , Jan 8, 2013
          • 0 Attachment
            On Tue, Jan 8, 2013 at 12:10 PM, Boris Danilov <brdanilov@...> wrote:
            > :syntax match ARecord_RecordType "^A" nextgroup=ARecord_RecordCount
            > :syntax match ARecord_RecordCount ".\{9}" nextgroup=ARecord_OriginatorID
            > :syntax match ARecord_OriginatorID ".\{10}"
            >
            > :syntax match CRecord_RecordType "^C" nextgroup=CRecord_RecordCount
            > :syntax match CRecord_RecordCount ".\{9}" nextgroup=CRecord_OriginatorID
            > :syntax match CRecord_OriginatorID ".\{14}"
            >
            > :highlight link ARecord_RecordType RecordType
            > :highlight link CRecord_RecordType RecordType
            >
            > :highlight link ARecord_RecordCount RecordCount
            > :highlight link CRecord_RecordCount RecordCount
            >
            > :highlight link ARecord_RecordOriginatorID RecordOriginatorID
            > :highlight link CRecord_RecordOriginatorID RecordOriginatorID
            >
            > :highlight RecordType ctermfg=blue
            > :highlight RecordCount ctermfg=red
            > :highlight RecordOriginatorID ctermfg=yellow

            I'm sorry, there is a mistake. To prevent XRecord_RecordOriginatorID
            and XRecord_RecordCount match anywhere I believe you should mark them
            as contained elements like this:
            :syntax match ARecord_RecordCount ".\{9}"
            nextgroup=ARecord_OriginatorID contained
            :syntax match ARecord_OriginatorID ".\{10}" contained

            --
            You received this message from the "vim_use" maillist.
            Do not top-post! Type your reply below the text you are replying to.
            For more information, visit http://www.vim.org/maillist.php
          • Dan Wierenga
            ... This works perfectly, thanks! -- You received this message from the vim_use maillist. Do not top-post! Type your reply below the text you are replying
            Message 5 of 10 , Jan 8, 2013
            • 0 Attachment
              On Mon, Jan 7, 2013 at 8:33 PM, Vlad Irnov <vlad.irnov@...> wrote:

              syntax match ARecord_RecordType /^A/ contains=ALL
              syntax match ARecord_RecordCount /^A.\{9}/ contains=ALL
              syntax match ARecord_RecordOriginatorID /^A.\{19}/ contains=ALL

               
              This works perfectly, thanks! 

              --
              You received this message from the "vim_use" maillist.
              Do not top-post! Type your reply below the text you are replying to.
              For more information, visit http://www.vim.org/maillist.php
            • Dan Wierenga
              ... I mostly understood *what* was happening, just not how to fix it. Unfortunately your solution didn t work for me. ... I read that, but I don t think I
              Message 6 of 10 , Jan 8, 2013
              • 0 Attachment
                On Tue, Jan 8, 2013 at 12:10 AM, Boris Danilov <brdanilov@...> wrote:
                Dan,

                Your solution isn't working because you're tryting to match 3 items
                starting with "^A". When there are several match or region patterns
                only the one defined last will match (look :help syn-priority),
                therefore ARecord_RecordType matches for you. Once again, when
                something matches in current position, this segment normally isn't
                tried for the matches with other syntax elements.

                I mostly understood *what* was happening, just not how to fix it.  Unfortunately your solution didn't work for me.
                 
                I suggest reading :help usr_44.txt to better understand syntax basics.
                 

                I read that, but I don't think I understood most of it.  I'll continue on with Vlad's solution and come back to usr_44.txt afterwards to see if it makes more sense then.

                Thanks for the help,
                Dan 

                --
                You received this message from the "vim_use" maillist.
                Do not top-post! Type your reply below the text you are replying to.
                For more information, visit http://www.vim.org/maillist.php
              • Boris Danilov
                Dan, ... This solution is potentially slower because it rematches everything several times. If your strings are really that long on some stage you might
                Message 7 of 10 , Jan 8, 2013
                • 0 Attachment
                  Dan,

                  > I mostly understood *what* was happening, just not how to fix it.
                  > Unfortunately your solution didn't work for me.

                  This solution is potentially slower because it rematches everything
                  several times. If your strings are really that long on some stage you
                  might encounter a lag. My idea was to just tell vim what syntax group
                  comes next, not sure what was wrong.

                  Regards,
                  Boris

                  --
                  You received this message from the "vim_use" maillist.
                  Do not top-post! Type your reply below the text you are replying to.
                  For more information, visit http://www.vim.org/maillist.php
                • Vlad Irnov
                  ... Hi Boris, ... nextgroup=ARecord_OriginatorID contained ... It is not obvious to me why this should be (much) faster than the method with CONTAINS. That
                  Message 8 of 10 , Jan 9, 2013
                  • 0 Attachment
                    On 1/9/13, Boris Danilov <brdanilov@...> wrote:
                    > Dan,
                    >
                    >> I mostly understood *what* was happening, just not how to fix it.
                    >> Unfortunately your solution didn't work for me.
                    >
                    > This solution is potentially slower because it rematches everything
                    > several times. If your strings are really that long on some stage you
                    > might encounter a lag. My idea was to just tell vim what syntax group
                    > comes next, not sure what was wrong.

                    Hi Boris,
                    your method does work when I try it like this:

                    :syntax match ARecord_RecordType "^A" nextgroup=ARecord_RecordCount
                    :syntax match ARecord_RecordCount ".\{9}"
                    nextgroup=ARecord_OriginatorID contained
                    :syntax match ARecord_OriginatorID ".\{10}" contained

                    :highlight ARecord_RecordType ctermfg=blue guifg=blue
                    :highlight ARecord_RecordCount ctermfg=red guifg=red
                    :highlight ARecord_OriginatorID ctermfg=yellow guifg=yellow


                    It is not obvious to me why this should be (much) faster than the
                    method with CONTAINS. That needs to be tested by profiling.

                    Regards,
                    Vlad

                    --
                    You received this message from the "vim_use" maillist.
                    Do not top-post! Type your reply below the text you are replying to.
                    For more information, visit http://www.vim.org/maillist.php
                  • Boris Danilov
                    Hello Vlad, ... I tried to show a good way to link highlight groups to one, but failed. In any way, my reasoning is that ... It checks all patterns one by one:
                    Message 9 of 10 , Jan 9, 2013
                    • 0 Attachment
                      Hello Vlad,

                      > Hi Boris,
                      > your method does work when I try it like this:
                      >
                      > :syntax match ARecord_RecordType "^A" nextgroup=ARecord_RecordCount
                      > :syntax match ARecord_RecordCount ".\{9}"
                      > nextgroup=ARecord_OriginatorID contained
                      > :syntax match ARecord_OriginatorID ".\{10}" contained
                      >
                      > :highlight ARecord_RecordType ctermfg=blue guifg=blue
                      > :highlight ARecord_RecordCount ctermfg=red guifg=red
                      > :highlight ARecord_OriginatorID ctermfg=yellow guifg=yellow
                      >
                      >
                      > It is not obvious to me why this should be (much) faster than the
                      > method with CONTAINS. That needs to be tested by profiling.

                      I tried to show a good way to link highlight groups to one, but
                      failed. In any way, my reasoning is that
                      when you have this code:

                      > syntax match ARecord_RecordType /^A/ contains=ALL
                      > syntax match ARecord_RecordCount /^A.\{9}/ contains=ALL
                      > syntax match ARecord_RecordOriginatorID /^A.\{19}/ contains=ALL

                      It checks all patterns one by one: the first pattern does match, then
                      the second one matches and finally the third does match. Nothing
                      matches anymore so the last pattern is used. Then again it has to
                      start all over again from "^A" because the third pattern contains all
                      patterns so it tries first and second once again. I don't know how
                      it's implemented and really hope the implementation is *much* smarter
                      that I described now. Maybe implementation uses theory of finite
                      automata, FIRST and LAST sets and other good optimization stuff... but
                      some 7th sense suggests me that when only one pattern (which doesn't
                      contain anything) matches and it immediatelly tells what is the next
                      pattern that matches too (and doesn't contain unnesesary patterns
                      inside) and finally it tells vim the third pattern that matches
                      flawlessly and so on. This way all vim must do is just to consume
                      input chars.

                      Regards,
                      Boris

                      --
                      You received this message from the "vim_use" maillist.
                      Do not top-post! Type your reply below the text you are replying to.
                      For more information, visit http://www.vim.org/maillist.php
                    • Boris Danilov
                      P.S. Also, I believe vim doesn t even try to check patterns that labled as contained outside of groups that contain them. Major number of patterns leaving the
                      Message 10 of 10 , Jan 9, 2013
                      • 0 Attachment
                        P.S. Also, I believe vim doesn't even try to check patterns that
                        labled as contained outside of groups that contain them. Major number
                        of patterns leaving the scene right away.

                        P.P.S. By the way, if you will need speed, you can add display to the
                        each pattern of this kind (they don't extend the line anyways and
                        doesn't hide patterns that can spread the line). And set the following
                        option in the end:

                        syntax sync maxlines=10

                        If the file is large and really doesn't contain anything else it will
                        many times speed your syntax highlighting.

                        Regards,
                        Boris


                        On Thu, Jan 10, 2013 at 2:38 AM, Boris Danilov <brdanilov@...> wrote:
                        > Hello Vlad,
                        >
                        >> Hi Boris,
                        >> your method does work when I try it like this:
                        >>
                        >> :syntax match ARecord_RecordType "^A" nextgroup=ARecord_RecordCount
                        >> :syntax match ARecord_RecordCount ".\{9}"
                        >> nextgroup=ARecord_OriginatorID contained
                        >> :syntax match ARecord_OriginatorID ".\{10}" contained
                        >>
                        >> :highlight ARecord_RecordType ctermfg=blue guifg=blue
                        >> :highlight ARecord_RecordCount ctermfg=red guifg=red
                        >> :highlight ARecord_OriginatorID ctermfg=yellow guifg=yellow
                        >>
                        >>
                        >> It is not obvious to me why this should be (much) faster than the
                        >> method with CONTAINS. That needs to be tested by profiling.
                        >
                        > I tried to show a good way to link highlight groups to one, but
                        > failed. In any way, my reasoning is that
                        > when you have this code:
                        >
                        >> syntax match ARecord_RecordType /^A/ contains=ALL
                        >> syntax match ARecord_RecordCount /^A.\{9}/ contains=ALL
                        >> syntax match ARecord_RecordOriginatorID /^A.\{19}/ contains=ALL
                        >
                        > It checks all patterns one by one: the first pattern does match, then
                        > the second one matches and finally the third does match. Nothing
                        > matches anymore so the last pattern is used. Then again it has to
                        > start all over again from "^A" because the third pattern contains all
                        > patterns so it tries first and second once again. I don't know how
                        > it's implemented and really hope the implementation is *much* smarter
                        > that I described now. Maybe implementation uses theory of finite
                        > automata, FIRST and LAST sets and other good optimization stuff... but
                        > some 7th sense suggests me that when only one pattern (which doesn't
                        > contain anything) matches and it immediatelly tells what is the next
                        > pattern that matches too (and doesn't contain unnesesary patterns
                        > inside) and finally it tells vim the third pattern that matches
                        > flawlessly and so on. This way all vim must do is just to consume
                        > input chars.
                        >
                        > Regards,
                        > Boris

                        --
                        You received this message from the "vim_use" maillist.
                        Do not top-post! Type your reply below the text you are replying to.
                        For more information, visit http://www.vim.org/maillist.php
                      Your message has been successfully submitted and would be delivered to recipients shortly.