Loading ...
Sorry, an error occurred while loading the content.

Re: raw fixed width data

Expand Messages
  • Wes Potts
    ... I ve added newlines with perl before but, I m hoping to find a better solution as these files are sometimes HUGE (Gigabytes) and when making edits
    Message 1 of 10 , Aug 2 1:52 PM
    • 0 Attachment
      On 8/2/05, Tim Chase <vim@...> wrote:
      > > For example: if I have a data file containing a sequence of
      > > characters like:
      > >
      > > 123456789012345678901
      > >
      > > Can I set a "record length" of say... 3, such that vim
      > > displays lines of 3 characters:
      > >
      > > 123
      > > 456
      > > 789
      >
      > If you don't mind actually editing your data (or a copy thereof),
      > its something I've done before with simply
      >
      > :%s/.\{3}/&\r/g
      >
      > which will insert a newline after every third character. Adjust
      > accordingly for your favorite record-length :)
      >
      > If the files are huge and slow Vim down, you can do something
      > similar with sed, if you've got access to it:
      >
      > sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

      I've added newlines with perl before but, I'm hoping to find a
      "better" solution as these files are sometimes HUGE (Gigabytes) and
      when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
      edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
      solution that would allow "in-place" editing. What I'm looking for is
      a "record" editor.

      >
      > > I frequently need to edit data to test programs and the data
      > > files with no EOLs are becoming a nuisance.
      >
      > So the solution is to put EOLs in where you want 'em :)
      >
      > Hope this helps,
      >
      > -tim
      >

      Thanks for the input.

      Wes
    • jose isaias cabrera
      If you have programmed before, I would suggest to write a small d program for it. http://www.digitalmars.com/ D is a like a combination of c, c++, java and c#,
      Message 2 of 10 , Aug 2 2:04 PM
      • 0 Attachment
        If you have programmed before, I would suggest to write a small d program
        for it.

        http://www.digitalmars.com/

        D is a like a combination of c, c++, java and c#, but better and faster.
        :-) It's a compiled language, so... The string capabilities are amazing.
        You could write your own record editor or creator. I have changed my
        programming language to D.

        Just a thought...


        ----- Original Message -----
        From: "Wes Potts" <wespotts@...>
        To: "Tim Chase" <vim@...>
        Cc: <vim@...>
        Sent: Tuesday, August 02, 2005 4:52 PM
        Subject: Re: raw fixed width data


        On 8/2/05, Tim Chase <vim@...> wrote:
        > > For example: if I have a data file containing a sequence of
        > > characters like:
        > >
        > > 123456789012345678901
        > >
        > > Can I set a "record length" of say... 3, such that vim
        > > displays lines of 3 characters:
        > >
        > > 123
        > > 456
        > > 789
        >
        > If you don't mind actually editing your data (or a copy thereof),
        > its something I've done before with simply
        >
        > :%s/.\{3}/&\r/g
        >
        > which will insert a newline after every third character. Adjust
        > accordingly for your favorite record-length :)
        >
        > If the files are huge and slow Vim down, you can do something
        > similar with sed, if you've got access to it:
        >
        > sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

        I've added newlines with perl before but, I'm hoping to find a
        "better" solution as these files are sometimes HUGE (Gigabytes) and
        when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
        edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
        solution that would allow "in-place" editing. What I'm looking for is
        a "record" editor.

        >
        > > I frequently need to edit data to test programs and the data
        > > files with no EOLs are becoming a nuisance.
        >
        > So the solution is to put EOLs in where you want 'em :)
        >
        > Hope this helps,
        >
        > -tim
        >

        Thanks for the input.

        Wes
      • Tony Mechelynck
        ... From: Wes Potts To: Sent: Tuesday, August 02, 2005 10:15 PM Subject: raw fixed width data ... WARNING: the following
        Message 3 of 10 , Aug 2 2:08 PM
        • 0 Attachment
          ----- Original Message -----
          From: "Wes Potts" <wespotts@...>
          To: <vim@...>
          Sent: Tuesday, August 02, 2005 10:15 PM
          Subject: raw fixed width data


          > Is it possible to configure vim such that it can distinguish records
          > (and treat them as lines) in a fixed-width data file (without
          > newlines)?
          >
          > For example: if I have a data file containing a sequence of characters
          > like:
          >
          > 123456789012345678901
          >
          > Can I set a "record length" of say... 3, such that vim displays lines
          > of 3 characters:
          >
          > 123
          > 456
          > 789
          > 012
          > 345
          > 678
          > 901
          >
          > I deal with very large data files (fixed width) on a regular basis.
          > Some have Unix EOLs, some DOS EOLs, and some have no EOLs. I
          > frequently need to edit data to test programs and the data files with
          > no EOLs are becoming a nuisance.
          >
          > Thanks,
          > Wes
          >
          >
          >

          WARNING: the following is untested.

          At start of edit, let's say you want a fixed length of 50 characters:

          :%s/\.{50}/\0\r/g

          This will break lines after every 50 characters not including ends-of-lines.

          At end of edit (if required)

          :%join!

          This removes all ends-of-lines.


          HTH,
          Tony.
        • Tofer Chagnon
          ... You could try doing a vertical split, and then adjusting the window width so that the desired number of columns show. You might want to do :nnoremap j gj
          Message 4 of 10 , Aug 2 2:13 PM
          • 0 Attachment
            On 8/2/05, Wes Potts <wespotts@...> wrote:
            > Is it possible to configure vim such that it can distinguish records
            > (and treat them as lines) in a fixed-width data file (without
            > newlines)?
            >
            > For example: if I have a data file containing a sequence of characters like:
            >
            > 123456789012345678901
            >
            > Can I set a "record length" of say... 3, such that vim displays lines
            > of 3 characters:
            >
            > 123
            > 456
            > 789
            > 012
            > 345
            > 678
            > 901
            >

            You could try doing a vertical split, and then adjusting the window
            width so that the desired number of columns show. You might want to
            do :nnoremap j gj :nnoremap k gk for this editing style.

            Tofer
          • Tim Chase
            ... Ah...I didn t gather the frustration of the multiple steps to begin with :) Vim does allow you to automate some of this, as you can see in the ... This
            Message 5 of 10 , Aug 2 2:31 PM
            • 0 Attachment
              >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
              >
              > I've added newlines with perl before but, I'm hoping to find a
              > "better" solution as these files are sometimes HUGE (Gigabytes) and
              > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
              > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
              > solution that would allow "in-place" editing. What I'm looking for is
              > a "record" editor.

              Ah...I didn't gather the frustration of the multiple steps to
              begin with :)

              Vim does allow you to automate some of this, as you can see in the

              :help hex-editing

              This does presume you have access to "xxd". You may be able to
              tweak it to use "xxd -c 3" to spit out 3-column format. however,
              I don't know whether it requires you to edit both the binary and
              the visual goods.

              Alternatively, if you've got your perl script, or the previous
              sed command, or even the previous vim command, you can do the
              same sort of exercise using autocommands:

              augroup Binary
              au!
              au BufReadPost *.bin %s/.\{3}/&\r/g
              au BufWritePre *.bin %j!
              au BufWritePost *.bin set nomod
              augroup END

              Seems to do the trick for me, and in a slightly more automated
              fashion than hand cranking it.

              Hope this helps (more), :)

              -tim
            • Marian Csontos
              ... Try ... You can t move using standard keys - up, down arrows, jk, ^, 0, $, home, end,... - instead you have to use gj, gk, g^, g0, g$,... But You can map
              Message 6 of 10 , Aug 2 11:34 PM
              • 0 Attachment
                On Tue, 02 Aug 2005 22:52:40 +0200, Wes Potts <wespotts@...> wrote:

                > On 8/2/05, Tim Chase <vim@...> wrote:
                >> > For example: if I have a data file containing a sequence of
                >> > characters like:
                >> >
                >> > 123456789012345678901
                >> >
                >> > Can I set a "record length" of say... 3, such that vim
                >> > displays lines of 3 characters:
                >> >
                >> > 123
                >> > 456
                >> > 789
                >>
                >> If you don't mind actually editing your data (or a copy thereof),
                >> its something I've done before with simply
                >>
                >> :%s/.\{3}/&\r/g
                >>
                >> which will insert a newline after every third character. Adjust
                >> accordingly for your favorite record-length :)
                >>
                >> If the files are huge and slow Vim down, you can do something
                >> similar with sed, if you've got access to it:
                >>
                >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
                >
                > I've added newlines with perl before but, I'm hoping to find a
                > "better" solution as these files are sometimes HUGE (Gigabytes) and
                > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
                > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
                > solution that would allow "in-place" editing. What I'm looking for is
                > a "record" editor.
                >

                Try
                :set wrap
                :set columns=<record-width>

                You can't move using standard keys - up, down arrows, jk, ^, 0, $, home,
                end,... - instead you have to use gj, gk, g^, g0, g$,...
                But You can map them:
                nmap <down> gj
                imap <down> <C-O>gj
                nmap <up> gk
                imap <up> <C-O>gk
                ...

                Regards

                -- Marian


                ________ Information from NOD32 ________
                This message was checked by NOD32 Antivirus System for Linux Mail Server.
                http://www.nod32.com
              • Steve Vance
                ... Careful, the %j! will also get rid of newlines which were in the file to begin with. Maybe something better like %s/ (. {3} ) n/ 1/g but I m not sure if
                Message 7 of 10 , Aug 4 12:59 PM
                • 0 Attachment
                  --- In vim@yahoogroups.com, Tim Chase <vim@t...> wrote:
                  > augroup Binary
                  > au!
                  > au BufReadPost *.bin %s/.\{3}/&\r/g
                  > au BufWritePre *.bin %j!

                  Careful, the %j! will also get rid of newlines which were in the file to
                  begin with.
                  Maybe something better like
                  %s/\(.\{3}\)\n/\1/g
                  but I'm not sure if that does it, either.

                  stv
                • Vigil
                  ... I have no idea if this would be any use, but it may be just the thing for you: http://record-editor.sourceforge.net/ -- .
                  Message 8 of 10 , Aug 6 8:41 AM
                  • 0 Attachment
                    On Tue, 2 Aug 2005, Wes Potts wrote:

                    > solution that would allow "in-place" editing. What I'm looking for is
                    > a "record" editor.

                    I have no idea if this would be any use, but it may be just the thing for you:

                    http://record-editor.sourceforge.net/

                    --

                    .
                  Your message has been successfully submitted and would be delivered to recipients shortly.