Loading ...
Sorry, an error occurred while loading the content.

Re: raw fixed width data

Expand Messages
  • jose isaias cabrera
    If you have programmed before, I would suggest to write a small d program for it. http://www.digitalmars.com/ D is a like a combination of c, c++, java and c#,
    Message 1 of 10 , Aug 2, 2005
    • 0 Attachment
      If you have programmed before, I would suggest to write a small d program
      for it.

      http://www.digitalmars.com/

      D is a like a combination of c, c++, java and c#, but better and faster.
      :-) It's a compiled language, so... The string capabilities are amazing.
      You could write your own record editor or creator. I have changed my
      programming language to D.

      Just a thought...


      ----- Original Message -----
      From: "Wes Potts" <wespotts@...>
      To: "Tim Chase" <vim@...>
      Cc: <vim@...>
      Sent: Tuesday, August 02, 2005 4:52 PM
      Subject: Re: raw fixed width data


      On 8/2/05, Tim Chase <vim@...> wrote:
      > > For example: if I have a data file containing a sequence of
      > > characters like:
      > >
      > > 123456789012345678901
      > >
      > > Can I set a "record length" of say... 3, such that vim
      > > displays lines of 3 characters:
      > >
      > > 123
      > > 456
      > > 789
      >
      > If you don't mind actually editing your data (or a copy thereof),
      > its something I've done before with simply
      >
      > :%s/.\{3}/&\r/g
      >
      > which will insert a newline after every third character. Adjust
      > accordingly for your favorite record-length :)
      >
      > If the files are huge and slow Vim down, you can do something
      > similar with sed, if you've got access to it:
      >
      > sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt

      I've added newlines with perl before but, I'm hoping to find a
      "better" solution as these files are sometimes HUGE (Gigabytes) and
      when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
      edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
      solution that would allow "in-place" editing. What I'm looking for is
      a "record" editor.

      >
      > > I frequently need to edit data to test programs and the data
      > > files with no EOLs are becoming a nuisance.
      >
      > So the solution is to put EOLs in where you want 'em :)
      >
      > Hope this helps,
      >
      > -tim
      >

      Thanks for the input.

      Wes
    • Tony Mechelynck
      ... From: Wes Potts To: Sent: Tuesday, August 02, 2005 10:15 PM Subject: raw fixed width data ... WARNING: the following
      Message 2 of 10 , Aug 2, 2005
      • 0 Attachment
        ----- Original Message -----
        From: "Wes Potts" <wespotts@...>
        To: <vim@...>
        Sent: Tuesday, August 02, 2005 10:15 PM
        Subject: raw fixed width data


        > Is it possible to configure vim such that it can distinguish records
        > (and treat them as lines) in a fixed-width data file (without
        > newlines)?
        >
        > For example: if I have a data file containing a sequence of characters
        > like:
        >
        > 123456789012345678901
        >
        > Can I set a "record length" of say... 3, such that vim displays lines
        > of 3 characters:
        >
        > 123
        > 456
        > 789
        > 012
        > 345
        > 678
        > 901
        >
        > I deal with very large data files (fixed width) on a regular basis.
        > Some have Unix EOLs, some DOS EOLs, and some have no EOLs. I
        > frequently need to edit data to test programs and the data files with
        > no EOLs are becoming a nuisance.
        >
        > Thanks,
        > Wes
        >
        >
        >

        WARNING: the following is untested.

        At start of edit, let's say you want a fixed length of 50 characters:

        :%s/\.{50}/\0\r/g

        This will break lines after every 50 characters not including ends-of-lines.

        At end of edit (if required)

        :%join!

        This removes all ends-of-lines.


        HTH,
        Tony.
      • Tofer Chagnon
        ... You could try doing a vertical split, and then adjusting the window width so that the desired number of columns show. You might want to do :nnoremap j gj
        Message 3 of 10 , Aug 2, 2005
        • 0 Attachment
          On 8/2/05, Wes Potts <wespotts@...> wrote:
          > Is it possible to configure vim such that it can distinguish records
          > (and treat them as lines) in a fixed-width data file (without
          > newlines)?
          >
          > For example: if I have a data file containing a sequence of characters like:
          >
          > 123456789012345678901
          >
          > Can I set a "record length" of say... 3, such that vim displays lines
          > of 3 characters:
          >
          > 123
          > 456
          > 789
          > 012
          > 345
          > 678
          > 901
          >

          You could try doing a vertical split, and then adjusting the window
          width so that the desired number of columns show. You might want to
          do :nnoremap j gj :nnoremap k gk for this editing style.

          Tofer
        • Tim Chase
          ... Ah...I didn t gather the frustration of the multiple steps to begin with :) Vim does allow you to automate some of this, as you can see in the ... This
          Message 4 of 10 , Aug 2, 2005
          • 0 Attachment
            >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
            >
            > I've added newlines with perl before but, I'm hoping to find a
            > "better" solution as these files are sometimes HUGE (Gigabytes) and
            > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
            > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
            > solution that would allow "in-place" editing. What I'm looking for is
            > a "record" editor.

            Ah...I didn't gather the frustration of the multiple steps to
            begin with :)

            Vim does allow you to automate some of this, as you can see in the

            :help hex-editing

            This does presume you have access to "xxd". You may be able to
            tweak it to use "xxd -c 3" to spit out 3-column format. however,
            I don't know whether it requires you to edit both the binary and
            the visual goods.

            Alternatively, if you've got your perl script, or the previous
            sed command, or even the previous vim command, you can do the
            same sort of exercise using autocommands:

            augroup Binary
            au!
            au BufReadPost *.bin %s/.\{3}/&\r/g
            au BufWritePre *.bin %j!
            au BufWritePost *.bin set nomod
            augroup END

            Seems to do the trick for me, and in a slightly more automated
            fashion than hand cranking it.

            Hope this helps (more), :)

            -tim
          • Marian Csontos
            ... Try ... You can t move using standard keys - up, down arrows, jk, ^, 0, $, home, end,... - instead you have to use gj, gk, g^, g0, g$,... But You can map
            Message 5 of 10 , Aug 2, 2005
            • 0 Attachment
              On Tue, 02 Aug 2005 22:52:40 +0200, Wes Potts <wespotts@...> wrote:

              > On 8/2/05, Tim Chase <vim@...> wrote:
              >> > For example: if I have a data file containing a sequence of
              >> > characters like:
              >> >
              >> > 123456789012345678901
              >> >
              >> > Can I set a "record length" of say... 3, such that vim
              >> > displays lines of 3 characters:
              >> >
              >> > 123
              >> > 456
              >> > 789
              >>
              >> If you don't mind actually editing your data (or a copy thereof),
              >> its something I've done before with simply
              >>
              >> :%s/.\{3}/&\r/g
              >>
              >> which will insert a newline after every third character. Adjust
              >> accordingly for your favorite record-length :)
              >>
              >> If the files are huge and slow Vim down, you can do something
              >> similar with sed, if you've got access to it:
              >>
              >> sed 's/.\{3\}/&\n/g' infile.txt > outfile.txt
              >
              > I've added newlines with perl before but, I'm hoping to find a
              > "better" solution as these files are sometimes HUGE (Gigabytes) and
              > when making edits repeatedly, a 4 or 5 step process ( copy, add EOLs,
              > edit, remove EOLs, copy back ) can be tedious. So, I'd prefer a
              > solution that would allow "in-place" editing. What I'm looking for is
              > a "record" editor.
              >

              Try
              :set wrap
              :set columns=<record-width>

              You can't move using standard keys - up, down arrows, jk, ^, 0, $, home,
              end,... - instead you have to use gj, gk, g^, g0, g$,...
              But You can map them:
              nmap <down> gj
              imap <down> <C-O>gj
              nmap <up> gk
              imap <up> <C-O>gk
              ...

              Regards

              -- Marian


              ________ Information from NOD32 ________
              This message was checked by NOD32 Antivirus System for Linux Mail Server.
              http://www.nod32.com
            • Steve Vance
              ... Careful, the %j! will also get rid of newlines which were in the file to begin with. Maybe something better like %s/ (. {3} ) n/ 1/g but I m not sure if
              Message 6 of 10 , Aug 4, 2005
              • 0 Attachment
                --- In vim@yahoogroups.com, Tim Chase <vim@t...> wrote:
                > augroup Binary
                > au!
                > au BufReadPost *.bin %s/.\{3}/&\r/g
                > au BufWritePre *.bin %j!

                Careful, the %j! will also get rid of newlines which were in the file to
                begin with.
                Maybe something better like
                %s/\(.\{3}\)\n/\1/g
                but I'm not sure if that does it, either.

                stv
              • Vigil
                ... I have no idea if this would be any use, but it may be just the thing for you: http://record-editor.sourceforge.net/ -- .
                Message 7 of 10 , Aug 6, 2005
                • 0 Attachment
                  On Tue, 2 Aug 2005, Wes Potts wrote:

                  > solution that would allow "in-place" editing. What I'm looking for is
                  > a "record" editor.

                  I have no idea if this would be any use, but it may be just the thing for you:

                  http://record-editor.sourceforge.net/

                  --

                  .
                Your message has been successfully submitted and would be delivered to recipients shortly.