Loading ...
Sorry, an error occurred while loading the content.

Suggestion about the line-ending detection without the last CRLF

Expand Messages
  • Wu Yongwei
    Hi Bram, When a Windows text file does not end with CRLF, vim will consider it a NOEOL UNIX file, displaying all the CRs. I think this could be improved.
    Message 1 of 5 , Apr 2, 2006
    • 0 Attachment
      Hi Bram,

      When a Windows text file does not end with CRLF, vim will consider it
      a NOEOL UNIX file, displaying all the CRs. I think this could be
      improved. (Though in the Vim way text file should always end with an
      EOL, in the Windows world it is often not the case. And it is arguable
      whether text files really should end with an EOL on non-POSIX worlds.
      BTW, I found the problem because pages on http://www.cnet.com.au/ does
      not end with an EOL; I do not think it a good idea to educate the Web
      masters.)

      Currently I am using Vim 6.4.10.

      Best regards,

      Yongwei
      --
      Wu Yongwei
      URL: http://wyw.dcweb.cn/
    • panshizhu@routon.com
      ... I had tested with vim 6.4 but it is not as you d described. The file not end with the CR/LF at will be recognized as NOEOL DOS file not as you d
      Message 2 of 5 , Apr 2, 2006
      • 0 Attachment
        "Wu Yongwei" <wuyongwei@...> wrote on 2006.04.03 10:25:28:

        > Hi Bram,
        >
        > When a Windows text file does not end with CRLF, vim will consider it
        > a NOEOL UNIX file, displaying all the CRs. I think this could be
        > improved. (Though in the Vim way text file should always end with an
        > EOL, in the Windows world it is often not the case. And it is arguable
        > whether text files really should end with an EOL on non-POSIX worlds.
        > BTW, I found the problem because pages on http://www.cnet.com.au/ does
        > not end with an EOL; I do not think it a good idea to educate the Web
        > masters.)
        >
        > Currently I am using Vim 6.4.10.
        >
        > Best regards,
        >
        > Yongwei
        > --
        > Wu Yongwei
        > URL: http://wyw.dcweb.cn/

        I had tested with vim 6.4 but it is not as you'd described. The file not
        end with the CR/LF at will be recognized as "NOEOL DOS" file not as you'd
        described "NOEOL UNIX" file.

        I suggest you check your 'fileformats' settings. My settings were:
        set fileformats=unix,mac,dos

        Perhaps some other related settings help... Try launch with vim -u NONE and
        see if it happens again.

        --
        Sincerely
        Pan, Shizhu. ext: 2221
      • Wu Yongwei
        ... OK, I see that you are right. Check this page: http://www.cnet.com.au/ Although I guess it intends to have Windows line endings, some of its lines (8 out
        Message 3 of 5 , Apr 2, 2006
        • 0 Attachment
          On 4/3/06, panshizhu@... <panshizhu@...> wrote:
          > "Wu Yongwei" <wuyongwei@...> wrote on 2006.04.03 10:25:28:
          >
          > > Hi Bram,
          > >
          > > When a Windows text file does not end with CRLF, vim will consider it
          > > a NOEOL UNIX file, displaying all the CRs. I think this could be
          > > improved. (Though in the Vim way text file should always end with an
          > > EOL, in the Windows world it is often not the case. And it is arguable
          > > whether text files really should end with an EOL on non-POSIX worlds.
          > > BTW, I found the problem because pages on http://www.cnet.com.au/ does
          > > not end with an EOL; I do not think it a good idea to educate the Web
          > > masters.)
          > >
          > > Currently I am using Vim 6.4.10.
          > >
          > > Best regards,
          > >
          > > Yongwei
          > > --
          > > Wu Yongwei
          > > URL: http://wyw.dcweb.cn/
          >
          > I had tested with vim 6.4 but it is not as you'd described. The file not
          > end with the CR/LF at will be recognized as "NOEOL DOS" file not as you'd
          > described "NOEOL UNIX" file.
          >
          > I suggest you check your 'fileformats' settings. My settings were:
          > set fileformats=unix,mac,dos
          >
          > Perhaps some other related settings help... Try launch with vim -u NONE and
          > see if it happens again.

          OK, I see that you are right. Check this page: http://www.cnet.com.au/

          Although I guess it intends to have Windows line endings, some of its
          lines (8 out of 973) end with only LF. Sorry that I did not find it
          earlier and not check carefully enough.

          If Vim is designed for the mass, I would suggest to add some code to
          handle this case. Since I do not think Bram will sacrifice consistency
          for this trivia (though it happens a lot), it seems I have to give up
          now.

          However, I still wonder if there is any room for improvement....

          Best regards,

          Yongwei
        • panshizhu@routon.com
          ... IMO the improvement may be: write autocommand script to cope with this? i.e. automatically remove all the ^M if a buffer opend with ff=unix ? In the Unix
          Message 4 of 5 , Apr 2, 2006
          • 0 Attachment
            "Wu Yongwei" <wuyongwei@...> wrote on 2006.04.03 12:49:31:
            > OK, I see that you are right. Check this page: http://www.cnet.com.au/
            >
            > Although I guess it intends to have Windows line endings, some of its
            > lines (8 out of 973) end with only LF. Sorry that I did not find it
            > earlier and not check carefully enough.
            >
            > If Vim is designed for the mass, I would suggest to add some code to
            > handle this case. Since I do not think Bram will sacrifice consistency
            > for this trivia (though it happens a lot), it seems I have to give up
            > now.
            >
            > However, I still wonder if there is any room for improvement....
            >
            > Best regards,
            >
            > Yongwei


            IMO the improvement may be: write autocommand script to cope with this?

            i.e. automatically remove all the ^M if a buffer opend with ff=unix ?

            In the Unix world, the program should provide mechanism, not policy. So it
            is unlikely that the feature be supplied inside vim executable, but the
            run-control scripts could do the job if you wish.

            --
            Sincerely
            Pan, Shizhu. ext: 2221
          • Vigil
            In fact, it is not in the Windows world , but rather in the badly behaved editors world . You see, Openoffice, MS Office, and I think even Notepad all
            Message 5 of 5 , Apr 3, 2006
            • 0 Attachment
              In fact, it is not "in the Windows world", but rather "in the badly behaved
              editors world". You see, Openoffice, MS Office, and I think even Notepad all
              correctly use EOLs where they should be, and Eclipse, ZDE, and I think Scipe
              (on all platforms - at least linux) incorrectly leave out the last EOL.

              Traditionally, each line in a text file is terminated by the EOL sequence, no
              matter the platform, and I think older (perhaps current?) versions of fgets in
              C do not recognise a full line until it sees the EOL sequence (I could be wrong
              about the specifics - it's years since I did any C programming - but I did read
              that people encountered problems when processing 'incorrectly' terminated text
              files). There are other problems with leaving out the last EOL in text files.
              As well as concatenation, imagine you are processing a stream, line by line.
              How do you know whether the last line is valid without an EOL? Maybe the stream
              was interrupted or terminated by or for an error, etc. or maybe just that line
              was interrupted.

              If a text file does not end with EOL, how is vim to know whether this really is
              a text file or just a binary file that happens to contain a suspicious amount
              of ASCII and CR or CRLFs? It is best that vim tell you what is going on, rather
              than assume one way or the other, the way the 'file' command does on a single
              unterminated line ("ASCII text, with no line terminators").

              See http://slashdot.org/comments.pl?sid=165492&cid=13808398 for more.

              So you see, vim is doing exactly as it should when it encounters these corrupt
              'text' files :)

              You encountered this issue when viewing the source of a web page. I believe
              this is a non-issue. You are opening up a file in your text editor that was
              meant to be parsed by web browsers, which happen to ignore whitespace and other
              formatting. Indeed you should not "educate the web masters". They have their
              own editors and IDEs. Telling them that you have a problem when opening their
              web server's output in your editor will, I believe, only serve to provide a
              source of amusement.

              On Mon, 3 Apr 2006, Wu Yongwei wrote:

              > Hi Bram,
              >
              > When a Windows text file does not end with CRLF, vim will consider it
              > a NOEOL UNIX file, displaying all the CRs. I think this could be
              > improved. (Though in the Vim way text file should always end with an
              > EOL, in the Windows world it is often not the case. And it is arguable
              > whether text files really should end with an EOL on non-POSIX worlds.
              > BTW, I found the problem because pages on http://www.cnet.com.au/ does
              > not end with an EOL; I do not think it a good idea to educate the Web
              > masters.)
              >
              > Currently I am using Vim 6.4.10.
              >
              > Best regards,
              >
              > Yongwei
              > --
              > Wu Yongwei
              > URL: http://wyw.dcweb.cn/
              >

              --
              .
            Your message has been successfully submitted and would be delivered to recipients shortly.