Loading ...
Sorry, an error occurred while loading the content.

raw fixed width data

Expand Messages
  • Wes Potts
    Is it possible to configure vim such that it can distinguish records (and treat them as lines) in a fixed-width data file (without newlines)? For example: if I
    Message 1 of 10 , Aug 2, 2005
    • 0 Attachment
      Is it possible to configure vim such that it can distinguish records
      (and treat them as lines) in a fixed-width data file (without
      newlines)?

      For example: if I have a data file containing a sequence of characters like:

      123456789012345678901

      Can I set a "record length" of say... 3, such that vim displays lines
      of 3 characters:

      123
      456
      789
      012
      345
      678
      901

      I deal with very large data files (fixed width) on a regular basis.
      Some have Unix EOLs, some DOS EOLs, and some have no EOLs. I
      frequently need to edit data to test programs and the data files with
      no EOLs are becoming a nuisance.

      Thanks,
      Wes
    • Tim Chase
      ... If you don t mind actually editing your data (or a copy thereof), its something I ve done before with simply ... which will insert a newline after every
      Message 2 of 10 , Aug 2, 2005
      • 0 Attachment
        > For example: if I have a data file containing a sequence of
        > characters like:
        >
        > 123456789012345678901
        >
        > Can I set a "record length" of say... 3, such that vim
        > displays lines of 3 characters:
        >
        > 123
        > 456
        > 789

        If you don't mind actually editing your data (or a copy thereof),
        its something I've done before with simply

        :%s/.\{3}/&\r/g

        which will insert a newline after every third character. Adjust
        accordingly for your favorite record-length :)

        If the files are huge and slow Vim down, you can do something
        similar with sed, if you've got access to it:

        sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

        > I frequently need to edit data to test programs and the data
        > files with no EOLs are becoming a nuisance.

        So the solution is to put EOLs in where you want 'em :)

        Hope this helps,

        -tim
      • Wes Potts
        ... I ve added newlines with perl before but, I m hoping to find a better solution as these files are sometimes HUGE (Gigabytes) and when making edits
        Message 3 of 10 , Aug 2, 2005
        • 0 Attachment
          On 8/2/05, Tim Chase <vim@...> wrote:
          > > For example: if I have a data file containing a sequence of
          > > characters like:
          > >
          > > 123456789012345678901
          > >
          > > Can I set a "record length" of say... 3, such that vim
          > > displays lines of 3 characters:
          > >
          > > 123
          > > 456
          > > 789
          >
          > If you don't mind actually editing your data (or a copy thereof),
          > its something I've done before with simply
          >
          > :%s/.\{3}/&\r/g
          >
          > which will insert a newline after every third character. Adjust
          > accordingly for your favorite record-length :)
          >
          > If the files are huge and slow Vim down, you can do something
          > similar with sed, if you've got access to it:
          >
          > sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

          I've added newlines with perl before but, I'm hoping to find a
          "better" solution as these files are sometimes HUGE (Gigabytes) and
          when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
          edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
          solution that would allow "in-place" editing. What I'm looking for is
          a "record" editor.

          >
          > > I frequently need to edit data to test programs and the data
          > > files with no EOLs are becoming a nuisance.
          >
          > So the solution is to put EOLs in where you want 'em :)
          >
          > Hope this helps,
          >
          > -tim
          >

          Thanks for the input.

          Wes
        • jose isaias cabrera
          If you have programmed before, I would suggest to write a small d program for it. http://www.digitalmars.com/ D is a like a combination of c, c++, java and c#,
          Message 4 of 10 , Aug 2, 2005
          • 0 Attachment
            If you have programmed before, I would suggest to write a small d program
            for it.

            http://www.digitalmars.com/

            D is a like a combination of c, c++, java and c#, but better and faster.
            :-) It's a compiled language, so... The string capabilities are amazing.
            You could write your own record editor or creator. I have changed my
            programming language to D.

            Just a thought...


            ----- Original Message -----
            From: "Wes Potts" <wespotts@...>
            To: "Tim Chase" <vim@...>
            Cc: <vim@...>
            Sent: Tuesday, August 02, 2005 4:52 PM
            Subject: Re: raw fixed width data


            On 8/2/05, Tim Chase <vim@...> wrote:
            > > For example: if I have a data file containing a sequence of
            > > characters like:
            > >
            > > 123456789012345678901
            > >
            > > Can I set a "record length" of say... 3, such that vim
            > > displays lines of 3 characters:
            > >
            > > 123
            > > 456
            > > 789
            >
            > If you don't mind actually editing your data (or a copy thereof),
            > its something I've done before with simply
            >
            > :%s/.\{3}/&\r/g
            >
            > which will insert a newline after every third character. Adjust
            > accordingly for your favorite record-length :)
            >
            > If the files are huge and slow Vim down, you can do something
            > similar with sed, if you've got access to it:
            >
            > sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

            I've added newlines with perl before but, I'm hoping to find a
            "better" solution as these files are sometimes HUGE (Gigabytes) and
            when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
            edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
            solution that would allow "in-place" editing. What I'm looking for is
            a "record" editor.

            >
            > > I frequently need to edit data to test programs and the data
            > > files with no EOLs are becoming a nuisance.
            >
            > So the solution is to put EOLs in where you want 'em :)
            >
            > Hope this helps,
            >
            > -tim
            >

            Thanks for the input.

            Wes
          • Tony Mechelynck
            ... From: Wes Potts To: Sent: Tuesday, August 02, 2005 10:15 PM Subject: raw fixed width data ... WARNING: the following
            Message 5 of 10 , Aug 2, 2005
            • 0 Attachment
              ----- Original Message -----
              From: "Wes Potts" <wespotts@...>
              To: <vim@...>
              Sent: Tuesday, August 02, 2005 10:15 PM
              Subject: raw fixed width data


              > Is it possible to configure vim such that it can distinguish records
              > (and treat them as lines) in a fixed-width data file (without
              > newlines)?
              >
              > For example: if I have a data file containing a sequence of characters
              > like:
              >
              > 123456789012345678901
              >
              > Can I set a "record length" of say... 3, such that vim displays lines
              > of 3 characters:
              >
              > 123
              > 456
              > 789
              > 012
              > 345
              > 678
              > 901
              >
              > I deal with very large data files (fixed width) on a regular basis.
              > Some have Unix EOLs, some DOS EOLs, and some have no EOLs. I
              > frequently need to edit data to test programs and the data files with
              > no EOLs are becoming a nuisance.
              >
              > Thanks,
              > Wes
              >
              >
              >

              WARNING: the following is untested.

              At start of edit, let's say you want a fixed length of 50 characters:

              :%s/\.{50}/\0\r/g

              This will break lines after every 50 characters not including ends-of-lines.

              At end of edit (if required)

              :%join!

              This removes all ends-of-lines.


              HTH,
              Tony.
            • Tofer Chagnon
              ... You could try doing a vertical split, and then adjusting the window width so that the desired number of columns show. You might want to do :nnoremap j gj
              Message 6 of 10 , Aug 2, 2005
              • 0 Attachment
                On 8/2/05, Wes Potts <wespotts@...> wrote:
                > Is it possible to configure vim such that it can distinguish records
                > (and treat them as lines) in a fixed-width data file (without
                > newlines)?
                >
                > For example: if I have a data file containing a sequence of characters like:
                >
                > 123456789012345678901
                >
                > Can I set a "record length" of say... 3, such that vim displays lines
                > of 3 characters:
                >
                > 123
                > 456
                > 789
                > 012
                > 345
                > 678
                > 901
                >

                You could try doing a vertical split, and then adjusting the window
                width so that the desired number of columns show. You might want to
                do :nnoremap j gj :nnoremap k gk for this editing style.

                Tofer
              • Tim Chase
                ... Ah...I didn t gather the frustration of the multiple steps to begin with :) Vim does allow you to automate some of this, as you can see in the ... This
                Message 7 of 10 , Aug 2, 2005
                • 0 Attachment
                  >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
                  >
                  > I've added newlines with perl before but, I'm hoping to find a
                  > "better" solution as these files are sometimes HUGE (Gigabytes) and
                  > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
                  > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
                  > solution that would allow "in-place" editing. What I'm looking for is
                  > a "record" editor.

                  Ah...I didn't gather the frustration of the multiple steps to
                  begin with :)

                  Vim does allow you to automate some of this, as you can see in the

                  :help hex-editing

                  This does presume you have access to "xxd". You may be able to
                  tweak it to use "xxd -c 3" to spit out 3-column format. however,
                  I don't know whether it requires you to edit both the binary and
                  the visual goods.

                  Alternatively, if you've got your perl script, or the previous
                  sed command, or even the previous vim command, you can do the
                  same sort of exercise using autocommands:

                  augroup Binary
                  au!
                  au BufReadPost *.bin %s/.\{3}/&\r/g
                  au BufWritePre *.bin %j!
                  au BufWritePost *.bin set nomod
                  augroup END

                  Seems to do the trick for me, and in a slightly more automated
                  fashion than hand cranking it.

                  Hope this helps (more), :)

                  -tim
                • Marian Csontos
                  ... Try ... You can t move using standard keys - up, down arrows, jk, ^, 0, $, home, end,... - instead you have to use gj, gk, g^, g0, g$,... But You can map
                  Message 8 of 10 , Aug 2, 2005
                  • 0 Attachment
                    On Tue, 02 Aug 2005 22:52:40 +0200, Wes Potts <wespotts@...> wrote:

                    > On 8/2/05, Tim Chase <vim@...> wrote:
                    >> > For example: if I have a data file containing a sequence of
                    >> > characters like:
                    >> >
                    >> > 123456789012345678901
                    >> >
                    >> > Can I set a "record length" of say... 3, such that vim
                    >> > displays lines of 3 characters:
                    >> >
                    >> > 123
                    >> > 456
                    >> > 789
                    >>
                    >> If you don't mind actually editing your data (or a copy thereof),
                    >> its something I've done before with simply
                    >>
                    >> :%s/.\{3}/&\r/g
                    >>
                    >> which will insert a newline after every third character. Adjust
                    >> accordingly for your favorite record-length :)
                    >>
                    >> If the files are huge and slow Vim down, you can do something
                    >> similar with sed, if you've got access to it:
                    >>
                    >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
                    >
                    > I've added newlines with perl before but, I'm hoping to find a
                    > "better" solution as these files are sometimes HUGE (Gigabytes) and
                    > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
                    > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
                    > solution that would allow "in-place" editing. What I'm looking for is
                    > a "record" editor.
                    >

                    Try
                    :set wrap
                    :set columns=<record-width>

                    You can't move using standard keys - up, down arrows, jk, ^, 0, $, home,
                    end,... - instead you have to use gj, gk, g^, g0, g$,...
                    But You can map them:
                    nmap <down> gj
                    imap <down> <C-O>gj
                    nmap <up> gk
                    imap <up> <C-O>gk
                    ...

                    Regards

                    -- Marian


                    ________ Information from NOD32 ________
                    This message was checked by NOD32 Antivirus System for Linux Mail Server.
                    http://www.nod32.com
                  • Steve Vance
                    ... Careful, the %j! will also get rid of newlines which were in the file to begin with. Maybe something better like %s/ (. {3} ) n/ 1/g but I m not sure if
                    Message 9 of 10 , Aug 4, 2005
                    • 0 Attachment
                      --- In vim@yahoogroups.com, Tim Chase <vim@t...> wrote:
                      > augroup Binary
                      > au!
                      > au BufReadPost *.bin %s/.\{3}/&\r/g
                      > au BufWritePre *.bin %j!

                      Careful, the %j! will also get rid of newlines which were in the file to
                      begin with.
                      Maybe something better like
                      %s/\(.\{3}\)\n/\1/g
                      but I'm not sure if that does it, either.

                      stv
                    • Vigil
                      ... I have no idea if this would be any use, but it may be just the thing for you: http://record-editor.sourceforge.net/ -- .
                      Message 10 of 10 , Aug 6, 2005
                      • 0 Attachment
                        On Tue, 2 Aug 2005, Wes Potts wrote:

                        > solution that would allow "in-place" editing. What I'm looking for is
                        > a "record" editor.

                        I have no idea if this would be any use, but it may be just the thing for you:

                        http://record-editor.sourceforge.net/

                        --

                        .
                      Your message has been successfully submitted and would be delivered to recipients shortly.