Loading ...
Sorry, an error occurred while loading the content.

Re: VIM-6 - rightleft redisplay

Expand Messages
  • Nadim Shaikli
    On Tue, 04 Dec 2001 12:37:33 -0500 ... Yes I did, but it unfortunately it wasn t what I was looking for; thanks for responding non-the-less. ... Oops, yah it
    Message 1 of 18 , Dec 4, 2001
    • 0 Attachment
      On Tue, 04 Dec 2001 12:37:33 -0500
      "David Harrison Jr." <david at motionrealityinc dot com> wrote:
      >
      > Did you get my previous post about calling "screen_update()"?

      Yes I did, but it unfortunately it wasn't what I was looking for;
      thanks for responding non-the-less.

      > >With the current code in search.c (function 'screen_line' line #3953)
      > >
      >
      > You mean in the file "screen.c"?

      Oops, yah it should have been "screen.c (function 'screen_line' line #3953)".

      > > there
      > >is a 'while' which in essence redisplays everything (which seems to always
      > >happen upon entering a new character). Here's my question - in "rightleft"
      > >mode the redrawing is still done from left-to-right instead of right-to-left
      > >and that is playing havoc with the new Arabic shaping code
      > >
      >
      > >(Arabic as you
      > >might know or have guessed by now is a right-to-left language).
      > >
      >
      > What about numbers? Are they drawn in the same direction? I seem to
      > remember there was one language that had its numbers drawn in the
      > opposite direction from its words, thus making it a combination of
      > "rightleft" and "leftright" type of writing. If this is the case, then
      > drawing everything in one direction may not be flexible enough, no
      > matter which direction you are drawing.

      Numbers aside (and we can live with them not being processed correctly
      until Bidi is adopted). The question still stands on how to modify the
      above noted logic/code to redisplay the characters in "rightleft" mode.

      > > It seems
      > >only logical that in "rightleft" mode that the redrawing be done in the
      > >same direction (ie. from right to left).
      > >
      >
      > That does seem logical. However, I think Vim is assuming that the
      > characters can be laid out in a regular grid.
      > If this is the case, then it shouldn't matter which direction you draw
      > your characters because the glyphs that are drawn will be self-contained
      > (each to it's own cell).
      >
      > Actually, I think that the final solution will be something like "draw a
      > line at a time". This way, you can account for the direction and draw
      > it appropriately. When I say "draw a line", I'm thinking of one line of
      > text on the screen.... and drawing it as one long string, possibly. The
      > X-Windows FontSet handling/drawing routines should take care of the rest.
      >
      > I don't know enough about Unicode (specifically UTF-8) and other
      > encodings to really know what the best solution is.

      As noted, for the new Arabic code to work properly and for it to do
      the shaping correctly, the redisplay needs to be done in a right-to-left
      manner. Let me tell you why - I intercept the displaying of a character
      and have to look at what's before and what's after a certain character for
      me to make a decision on how to process (or shape) that character. In
      ":set rightleft" mode the processing is done correctly when a character is
      entered, but when the redisplay happens - the direction is reversed and the
      new Arabic code is thrown for a loop (instead of it processing characters
      from right-to-left, it gets characters from left-to-right thus rendering
      the previous and next characters all wrong).

      > >Is someone out there familiar enough with the code to offer an ifdef'ed
      > >solution that would redraw and traverse the characters from right-to-left
      > >in "rightleft" mode ? I've attempted to do that myself, but there are bits
      > >and pieces of the code that I don't understand and can't seem to get right,
      > >thus rendering my subtle attempts non-functional.
      > >
      >
      > Unfortunately, I'm not that familiar with the code to offer a solution
      > (I need to see Arabic examples.... lots of them).

      Well, I would suggest downloading our work-in-progress code,

      http://www.arabeyes.org/cgi-bin/cvsweb/arabeyes/vim
      -or-
      http://www.arabeyes.org/~nadim/vim/vim-shaping.tar.gz

      Please CC those that are already included.

      - Nadim


      __________________________________________________
      Do You Yahoo!?
      Buy the perfect holiday gifts at Yahoo! Shopping.
      http://shopping.yahoo.com
    • Nadim Shaikli
      On Tue, 4 Dec 2001 13:25:36 -0500 (EST) ... And other languages as well. ... I agree with you 100% -- Bidi is a MUST for any right-to-left language, as you
      Message 2 of 18 , Dec 4, 2001
      • 0 Attachment
        On Tue, 4 Dec 2001 13:25:36 -0500 (EST)
        "Yousof Mortazavi" <yousof at engsoc dot org> wrote:
        >
        > On Tue, 4 Dec 2001, David Harrison Jr. wrote:
        >
        > >
        > > What about numbers? Are they drawn in the same direction? I seem to
        > > remember there was one language that had its numbers drawn in the
        > > opposite direction from its words, thus making it a combination of
        > > "rightleft" and "leftright" type of writing. If this is the case, then
        > > drawing everything in one direction may not be flexible enough, no
        > > matter which direction you are drawing.
        >
        > Yes, you are right about numbers rendered as left-to-right. This
        > whole handling of right-to-left and left-to-right gets pretty
        > complicated especially when you start embedding texts from
        > left-to-right scripts within right-to-left scripts. It is,
        > however, clearly outlined in the Unicode Standard Annex #9:
        > "The Bidirectonal Algorithm" available at
        >
        > http://www.unicode.org/unicode/reports/tr9/
        >
        > As well libraries have been written, the most complete of which
        > is "fribidi" available from
        >
        > http://fribidi.sourceforge.net
        >
        > Noteworthy to say, that implementing BiDi for Vim will ease the
        > implementation of right-to-left scripts such as Arabic and
        > Hebrew.

        And other languages as well.

        > Here I'd like to propose a "BiDi" mode in Vim (just as the
        > current RL mode) that when active, characters entered get
        > shuffled according to the BiDi Algorithm as described by the
        > Unicode Standard. This mode should of course coexist with the
        > current RL mode such that when this mode is active, the base
        > direction is right-to-left, nevertheless, characters that possess
        > left-to-right behaviour are rendered as such.
        >
        > I would like to work on this project in conjunction with any
        > group that feels the need for proper Bidirectional behaviour. It
        > is, after all, part of the Unicode Standard, and the current
        > Unicode implementation of Vim would be incomplete without it.

        I agree with you 100% -- Bidi is a MUST for any right-to-left
        language, as you note above, if one were to intermix languages.
        I would like to stress, though, that Bidi is an ambitious plan
        that will undoubtedly mean lots of code change (do you have an
        estimate of the work required ?). As such, I hope that shaping
        will not take a back-seat since its almost complete and requires,
        as far as we can see, a potential fix/enhancement. Shaping, like
        Bidi, is also required by most (if not all) right-left languages
        (Arabic, Hebrew, Farsi, etc).

        I am glad to see that both Bidi and Shaping are finally being looked
        into seriously in regards to VIM.

        > Bram, do you approve?

        - Nadim


        __________________________________________________
        Do You Yahoo!?
        Buy the perfect holiday gifts at Yahoo! Shopping.
        http://shopping.yahoo.com
      • Ron Aaron
        ... Just a small correction - Hebrew only has five letters which are differently shaped (at the end of words, only) so people inputting Hebrew just input those
        Message 3 of 18 , Dec 4, 2001
        • 0 Attachment
          Nadim Shaikli <shaikli@...> writes:
          >as far as we can see, a potential fix/enhancement. Shaping, like
          >Bidi, is also required by most (if not all) right-left languages
          >(Arabic, Hebrew, Farsi, etc).

          Just a small correction - Hebrew only has five letters which are differently
          shaped (at the end of words, only) so people inputting Hebrew just input those
          characters directly, rather than relying on the underlying software to reshape
          the input letters. For Arabic, it is almost required to shape, because there
          are so many possibilities.

          Best regards,
          Ron
        • David Harrison Jr.
          ... Agreed. Now that I m thinking about these things, I wonder if certain Asian texts need to be displayed vertically? (topbottom mode?) ... Ok. I ve used it
          Message 4 of 18 , Dec 7, 2001
          • 0 Attachment
            Nadim Shaikli wrote:

            >Numbers aside (and we can live with them not being processed correctly
            >until Bidi is adopted). The question still stands on how to modify the
            >above noted logic/code to redisplay the characters in "rightleft" mode.
            >
            >

            Agreed. Now that I'm thinking about these things, I wonder if certain
            Asian texts need to be displayed vertically? (topbottom mode?)

            >>Unfortunately, I'm not that familiar with the code to offer a solution
            >>(I need to see Arabic examples.... lots of them).
            >>
            >
            >Well, I would suggest downloading our work-in-progress code,
            >
            > http://www.arabeyes.org/cgi-bin/cvsweb/arabeyes/vim
            >-or-
            > http://www.arabeyes.org/~nadim/vim/vim-shaping.tar.gz
            >


            Ok. I've used it awhile now. Lots of screen refresh problems when you
            are in an ISO-8859-x type of setup.

            The Arabic input is interesting. I press 'q','w' which show a slightly
            different set of characters than typing 'q', ' ', 'w'. I press CTRL-L
            and the "combined" 'q''w' sequence is indeed drawn differently.

            Hmmm, I'll take a closer look...

            --David H. Jr.
          • Nadim Shaikli
            On Fri, 07 Dec 2001 08:40:03 -0500 ... Actually seems like the refresh is there irrespective of which mode or setting you are in (ie. screen_line seems to
            Message 5 of 18 , Dec 7, 2001
            • 0 Attachment
              On Fri, 07 Dec 2001 08:40:03 -0500
              "David Harrison Jr." <david at motionrealityinc dot com> wrote:
              >
              > Nadim Shaikli wrote:
              >
              > >Well, I would suggest downloading our work-in-progress code,
              > >
              > > http://www.arabeyes.org/cgi-bin/cvsweb/arabeyes/vim
              > >-or-
              > > http://www.arabeyes.org/~nadim/vim/vim-shaping.tar.gz
              > >
              >
              > Ok. I've used it awhile now. Lots of screen refresh problems when
              > you are in an ISO-8859-x type of setup.

              Actually seems like the refresh is there irrespective of which mode or
              setting you are in (ie. screen_line seems to redisplay with each new
              character even in "default" english).

              > The Arabic input is interesting. I press 'q','w' which show a slightly
              > different set of characters than typing 'q', ' ', 'w'. I press CTRL-L
              > and the "combined" 'q''w' sequence is indeed drawn differently.
              >
              > Hmmm, I'll take a closer look...

              I've solved most of the known problems, shaping is pretty functional
              in the various different settings (including CTRL-L). As of now, there
              are no known issues regarding shaping stand-alone (note - "as of now" :-)

              Here's what's left (as I see it) regarding Arabic Shaping and support:

              1. Get Bram's blessings/comments :-)
              2. Fix the screen_line (in screen.c) to redisplay characters from
              right-to-left in rightleft mode.
              3. Add 'search' (ie. /) abilities in rightleft mode
              a. Add rightleft-type of directionality in command-line
              b. Add Arabic shaping to the command-line
              c. Search non-shaped characters (ie. 'off_from')
              4. Add Bidi abilities

              Please note that the links noted above (in previous post) have been
              updated and I highly recommend and encourage re-syncing.

              Thanks for your help and interest.

              - Nadim


              __________________________________________________
              Do You Yahoo!?
              Send your FREE holiday greetings online!
              http://greetings.yahoo.com
            • David Harrison Jr.
              ... This has me thinking that other languages could benefit from a generalization of this part of the code. Certain Asian texts are shown vertically, thus we
              Message 6 of 18 , Dec 7, 2001
              • 0 Attachment
                Nadim Shaikli wrote:

                >On Fri, 07 Dec 2001 08:40:03 -0500
                > "David Harrison Jr." <david at motionrealityinc dot com> wrote:
                >
                >>Nadim Shaikli wrote:
                >>
                >>>Well, I would suggest downloading our work-in-progress code,
                >>>
                >>> http://www.arabeyes.org/cgi-bin/cvsweb/arabeyes/vim
                >>>-or-
                >>> http://www.arabeyes.org/~nadim/vim/vim-shaping.tar.gz
                >>>
                >>Ok. I've used it awhile now. Lots of screen refresh problems when
                >>you are in an ISO-8859-x type of setup.
                >>
                >
                >Actually seems like the refresh is there irrespective of which mode or
                >setting you are in (ie. screen_line seems to redisplay with each new
                >character even in "default" english).
                >
                >>The Arabic input is interesting. I press 'q','w' which show a slightly
                >>different set of characters than typing 'q', ' ', 'w'. I press CTRL-L
                >>and the "combined" 'q''w' sequence is indeed drawn differently.
                >>
                >>Hmmm, I'll take a closer look...
                >>
                >
                >I've solved most of the known problems, shaping is pretty functional
                >in the various different settings (including CTRL-L). As of now, there
                >are no known issues regarding shaping stand-alone (note - "as of now" :-)
                >
                >Here's what's left (as I see it) regarding Arabic Shaping and support:
                >
                ...

                >
                > 2. Fix the screen_line (in screen.c) to redisplay characters from
                > right-to-left in rightleft mode.
                >

                This has me thinking that other languages could benefit from a
                generalization of this part of the code. Certain Asian texts are shown
                vertically, thus we may (eventually) have a set of options that allow us
                to have the main flow of text be either vertical or horizontal. Within
                each of these two modes the text could be shown one way, or the other
                "reversed" way (i.e. top-to-bottom or bottom-to-top).

                I can't even imagine how frustrating it must be for a Vim user that
                reads vertical text to be forced to read in a horizontal direction (from
                within Vim). Maybe rotating the screen 90 degrees would be the easiest
                thing to do... problem solved! :-)

                >
                >Please note that the links noted above (in previous post) have been
                >updated and I highly recommend and encourage re-syncing.
                >

                I re-synced. The main redraw problems are now gone.
                I notice that when I switch back and forth between rightleft and
                norightleft mode, different Arabic "shaped words" will be displayed
                (they are the same word, but displayed differently). I assume that they
                are only correct when in rightleft mode.

                >Thanks for your help and interest.
                >

                I don't think I was of any help, but I'm definitely interested in
                generalizing Vim's code to handle new languages.

                --David H. Jr.
              • Nadim Shaikli
                On Fri, 07 Dec 2001 16:43:12 -0500 ... BTW: any comment on that list sent as it relates to the tasks at hand ? ... This is WAY beyond what I ve set to do and
                Message 7 of 18 , Dec 7, 2001
                • 0 Attachment
                  On Fri, 07 Dec 2001 16:43:12 -0500
                  "David Harrison Jr." <david at motionrealityinc dot com> wrote:
                  >
                  > Nadim Shaikli wrote:
                  > >
                  > > 2. Fix the screen_line (in screen.c) to redisplay characters from
                  > > right-to-left in rightleft mode.
                  > >

                  BTW: any comment on that list sent as it relates to the tasks at hand ?

                  > This has me thinking that other languages could benefit from a
                  > generalization of this part of the code. Certain Asian texts are shown
                  > vertically, thus we may (eventually) have a set of options that allow us
                  > to have the main flow of text be either vertical or horizontal. Within
                  > each of these two modes the text could be shown one way, or the other
                  > "reversed" way (i.e. top-to-bottom or bottom-to-top).
                  >
                  > I can't even imagine how frustrating it must be for a Vim user that
                  > reads vertical text to be forced to read in a horizontal direction (from
                  > within Vim). Maybe rotating the screen 90 degrees would be the easiest
                  > thing to do... problem solved! :-)

                  This is WAY beyond what I've set to do and I'd humbly be way overwhelmed
                  especially with Bidi on the way. Furthermore, I personally have no
                  immediate interest in "topbottom" mode :-) I'm certainly no expert
                  on top-bottom or bottom-top languages, but most allow the text to be
                  read in left-to-right mode (I think Chinese/Mandarin is that way).

                  > I notice that when I switch back and forth between rightleft and
                  > norightleft mode, different Arabic "shaped words" will be displayed
                  > (they are the same word, but displayed differently). I assume that they
                  > are only correct when in rightleft mode.

                  Yes, the only correct form is the one in "rightleft" mode.

                  BTW: just an FYI in case you were curious to delve into an unexplored area.

                  If you enter any shaped Arabic char and try to convert to HTML
                  a core-dump results. For what its worth the same thing happens
                  with hebrew in rightleft mode (with or without the new Arabic code).
                  So it doesn't sound like anything that was recently added or
                  introduced - thought I'd mention it.

                  - Nadim


                  __________________________________________________
                  Do You Yahoo!?
                  Send your FREE holiday greetings online!
                  http://greetings.yahoo.com
                • Bram Moolenaar
                  ... I have no idea what the impact of implementing BiDi would be. It sounds complicated. In many places in the code the relation between the bytes of text in
                  Message 8 of 18 , Dec 14, 2001
                  • 0 Attachment
                    Yousof Mortazavi wrote:

                    > Noteworthy to say, that implementing BiDi for Vim will ease the
                    > implementation of right-to-left scripts such as Arabic and
                    > Hebrew.
                    >
                    > Here I'd like to propose a "BiDi" mode in Vim (just as the
                    > current RL mode) that when active, characters entered get
                    > shuffled according to the BiDi Algorithm as described by the
                    > Unicode Standard. This mode should of course coexist with the
                    > current RL mode such that when this mode is active, the base
                    > direction is right-to-left, nevertheless, characters that possess
                    > left-to-right behaviour are rendered as such.
                    >
                    > I would like to work on this project in conjunction with any
                    > group that feels the need for proper Bidirectional behaviour. It
                    > is, after all, part of the Unicode Standard, and the current
                    > Unicode implementation of Vim would be incomplete without it.
                    >
                    > Bram, do you approve?

                    I have no idea what the impact of implementing BiDi would be. It sounds
                    complicated. In many places in the code the relation between the bytes
                    of text in a line and the position on the display is used. The simplest
                    example is that the cursor position is on a character in the text, and
                    it's displayed where that character appears on screen. Thus a
                    computation needs to be done from byte number to column number. It gets
                    more complicated when using Tabs and unprintable characters. And the
                    reverse compuation is also needed (e.g., when clicking with the mouse).

                    The best way to find out how to do it is to attempt a few ways to
                    implement it and decide which one works best. Yes, that's a lot of
                    work...

                    --
                    From "know your smileys":
                    % Bike accident (a bit far-fetched, I suppose)

                    /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                    ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
                    \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
                  • Yousof Mortazavi
                    ... Luckily, this sort of code has been written and there s a library (fribidi, http://fribidi.sourceforge.net) that handles all such issues involving BiDi.
                    Message 9 of 18 , Dec 15, 2001
                    • 0 Attachment
                      On Fri, 14 Dec 2001, Bram Moolenaar wrote:

                      >
                      > Yousof Mortazavi wrote:
                      >
                      > > Noteworthy to say, that implementing BiDi for Vim will ease the
                      > > implementation of right-to-left scripts such as Arabic and
                      > > Hebrew.
                      > >
                      > > Here I'd like to propose a "BiDi" mode in Vim (just as the
                      > > current RL mode) that when active, characters entered get
                      > > shuffled according to the BiDi Algorithm as described by the
                      > > Unicode Standard. This mode should of course coexist with the
                      > > current RL mode such that when this mode is active, the base
                      > > direction is right-to-left, nevertheless, characters that possess
                      > > left-to-right behaviour are rendered as such.
                      > >
                      > > I would like to work on this project in conjunction with any
                      > > group that feels the need for proper Bidirectional behaviour. It
                      > > is, after all, part of the Unicode Standard, and the current
                      > > Unicode implementation of Vim would be incomplete without it.
                      > >
                      > > Bram, do you approve?
                      >
                      > I have no idea what the impact of implementing BiDi would be. It sounds
                      > complicated. In many places in the code the relation between the bytes
                      > of text in a line and the position on the display is used. The simplest
                      > example is that the cursor position is on a character in the text, and
                      > it's displayed where that character appears on screen. Thus a
                      > computation needs to be done from byte number to column number. It gets
                      > more complicated when using Tabs and unprintable characters. And the
                      > reverse compuation is also needed (e.g., when clicking with the mouse).

                      Luckily, this sort of code has been written and there's a library
                      (fribidi, http://fribidi.sourceforge.net) that handles all such
                      issues involving BiDi. The main task is finding the right spots
                      to insert the code. What's your policy on using libraries?

                      Also, is there any sort of documentation on the internals of Vim,
                      besides the code itself?

                      > The best way to find out how to do it is to attempt a few ways to
                      > implement it and decide which one works best. Yes, that's a lot of
                      > work...
                      >

                      It's an effort worth making.

                      --
                      -= Yousof Mortazavi <yousof@...> | Electrical Engineering =-
                      -= WWW: http://www.engsoc.org/~yousof/ | Carleton University =-
                    • Bram Moolenaar
                      ... I suppose that library only provides the algorithm(s). The integration into Vim is probably still a lot of work. There are many places that assume text
                      Message 10 of 18 , Dec 15, 2001
                      • 0 Attachment
                        > > I have no idea what the impact of implementing BiDi would be. It sounds
                        > > complicated. In many places in the code the relation between the bytes
                        > > of text in a line and the position on the display is used. The simplest
                        > > example is that the cursor position is on a character in the text, and
                        > > it's displayed where that character appears on screen. Thus a
                        > > computation needs to be done from byte number to column number. It gets
                        > > more complicated when using Tabs and unprintable characters. And the
                        > > reverse compuation is also needed (e.g., when clicking with the mouse).
                        >
                        > Luckily, this sort of code has been written and there's a library
                        > (fribidi, http://fribidi.sourceforge.net) that handles all such
                        > issues involving BiDi. The main task is finding the right spots
                        > to insert the code. What's your policy on using libraries?

                        I suppose that library only provides the algorithm(s). The integration
                        into Vim is probably still a lot of work. There are many places that
                        assume text is displayed in one direction only, it's unpredicatable what
                        needs to be changed.

                        I wonder, does the reordering need to be done when displaying text? Why
                        not do it when typing the text and store it like how it's displayed?

                        > Also, is there any sort of documentation on the internals of Vim,
                        > besides the code itself?

                        src/README.txt

                        --
                        In a world without walls and borders, who needs windows and gates?

                        /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                        ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
                        \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
                      • Yousof Mortazavi
                        ... That is true, but I m optimistic that with the help of Vim developers and yourself, I can code the fundamental aspects of BiDi. Then, some time is needed
                        Message 11 of 18 , Dec 15, 2001
                        • 0 Attachment
                          On Sat, 15 Dec 2001, Bram Moolenaar wrote:

                          > > Luckily, this sort of code has been written and there's a library
                          > > (fribidi, http://fribidi.sourceforge.net) that handles all such
                          > > issues involving BiDi. The main task is finding the right spots
                          > > to insert the code. What's your policy on using libraries?
                          >
                          > I suppose that library only provides the algorithm(s). The integration
                          > into Vim is probably still a lot of work. There are many places that
                          > assume text is displayed in one direction only, it's unpredicatable what
                          > needs to be changed.

                          That is true, but I'm optimistic that with the help of Vim
                          developers and yourself, I can code the fundamental aspects of
                          BiDi. Then, some time is needed until users submit bugs and we go
                          towards a complete implementation.

                          > I wonder, does the reordering need to be done when displaying text? Why
                          > not do it when typing the text and store it like how it's displayed?

                          Yes, it's absolutely critical.

                          The way text should be stored in memory is in a logical manner,
                          that is characters that come first must be stored first. As an
                          example, assume capital letters represent a right-to-left script,
                          and small letters a left-to-right script.
                          Then the string of text:
                          car is THE CAR in arabic
                          should be displayed as
                          car is RAC EHT in arabic

                          The former string is the logical order, while the latter is the
                          visual. These two are completely different. Searching and parsing
                          is impossible with visual strings.

                          --
                          -= Yousof Mortazavi <yousof@...> | Electrical Engineering =-
                          -= WWW: http://www.engsoc.org/~yousof/ | Carleton University =-
                        • Bram Moolenaar
                          ... This sounds extremely complicated. You are probably better off ignoring the left-right code, since it doesn t handle this kind of text at all. Suppose
                          Message 12 of 18 , Dec 15, 2001
                          • 0 Attachment
                            Yousof Mortazavi wrote:

                            > > I wonder, does the reordering need to be done when displaying text? Why
                            > > not do it when typing the text and store it like how it's displayed?
                            >
                            > Yes, it's absolutely critical.
                            >
                            > The way text should be stored in memory is in a logical manner,
                            > that is characters that come first must be stored first. As an
                            > example, assume capital letters represent a right-to-left script,
                            > and small letters a left-to-right script.
                            > Then the string of text:
                            > car is THE CAR in arabic
                            > should be displayed as
                            > car is RAC EHT in arabic
                            >
                            > The former string is the logical order, while the latter is the
                            > visual. These two are completely different. Searching and parsing
                            > is impossible with visual strings.

                            This sounds extremely complicated. You are probably better off ignoring
                            the left-right code, since it doesn't handle this kind of text at all.

                            Suppose that when the cursor is moved with "l", the user expects it to
                            go from "car is" to the 'R' of "RAC". The using "l" will move
                            rightwards on the display, but downwards in text columns. I'm sure this
                            is going to cause a lot of trouble. Would it be better to have "l" work
                            like before (move in text columns), with the result that it moves from
                            "car is" to the 'T' of "RAC EHT"? And then moves to the left over this
                            Arabic text?

                            Just one command and already we have a dilemma...

                            --
                            hundred-and-one symptoms of being an internet addict:
                            112. You are amazed that anyone uses a phone without a modem on it...let
                            alone hear actual voices.

                            /// Bram Moolenaar -- Bram@... -- http://www.moolenaar.net \\\
                            ((( Creator of Vim -- http://vim.sf.net -- ftp://ftp.vim.org/pub/vim )))
                            \\\ Help me helping AIDS orphans in Uganda - http://iccf-holland.org ///
                          • Yousof Mortazavi
                            ... I think both the normal and left-right modes should be modified, because BiDi is all about embeding and you can embed Arabic in English, or English in
                            Message 13 of 18 , Dec 15, 2001
                            • 0 Attachment
                              On Sat, 15 Dec 2001, Bram Moolenaar wrote:

                              > > Then the string of text:
                              > > car is THE CAR in arabic
                              > > should be displayed as
                              > > car is RAC EHT in arabic

                              > This sounds extremely complicated. You are probably better off ignoring
                              > the left-right code, since it doesn't handle this kind of text at all.

                              I think both the normal and left-right modes should be modified,
                              because BiDi is all about embeding and you can embed Arabic in
                              English, or English in Arabic.

                              > Suppose that when the cursor is moved with "l", the user expects it to
                              > go from "car is" to the 'R' of "RAC". The using "l" will move
                              > rightwards on the display, but downwards in text columns. I'm sure this
                              > is going to cause a lot of trouble. Would it be better to have "l" work
                              > like before (move in text columns), with the result that it moves from
                              > "car is" to the 'T' of "RAC EHT"? And then moves to the left over this
                              > Arabic text?

                              Following the logical string would be best, so yes the second
                              case is more reasonable. This can be implemented with a map of
                              the logical string to the visual string. Fribidi has a function
                              for this.

                              for example:
                              logical: car is THE CAR in arabic
                              visual: car is RAC EHT in arabic
                              l2v map: 24 1 2 3 4 5 6 13 12 11 10 9 8 7 14 15 16 17 18 19 20 21 22 23
                              ^^^^^^^^^^^^^^^^^
                              note reverse order.

                              So advancing the cursor would be based on the l2v map not the
                              cursor position.

                              --
                              -= Yousof Mortazavi <yousof@...> | Electrical Engineering =-
                              -= WWW: http://www.engsoc.org/~yousof/ | Carleton University =-
                            • Nadim Shaikli
                              ... I m not familiar with fribidi (yet) to know where its involvement starts and where it ends -- sounds like that s the first place we need to spend some time
                              Message 14 of 18 , Dec 15, 2001
                              • 0 Attachment
                                --- Bram Moolenaar wrote:
                                >
                                > Yousof Mortazavi wrote:
                                > > The way text should be stored in memory is in a logical manner,
                                > > that is characters that come first must be stored first. As an
                                > > example, assume capital letters represent a right-to-left script,
                                > > and small letters a left-to-right script.
                                > > Then the string of text:
                                > > car is THE CAR in arabic
                                > > should be displayed as
                                > > car is RAC EHT in arabic
                                > >
                                > > The former string is the logical order, while the latter is the
                                > > visual. These two are completely different. Searching and parsing
                                > > is impossible with visual strings.
                                >
                                > This sounds extremely complicated. You are probably better off ignoring
                                > the left-right code, since it doesn't handle this kind of text at all.
                                >
                                > Suppose that when the cursor is moved with "l", the user expects it to
                                > go from "car is" to the 'R' of "RAC". The using "l" will move
                                > rightwards on the display, but downwards in text columns. I'm sure this
                                > is going to cause a lot of trouble. Would it be better to have "l" work
                                > like before (move in text columns), with the result that it moves from
                                > "car is" to the 'T' of "RAC EHT"? And then moves to the left over this
                                > Arabic text?

                                I'm not familiar with fribidi (yet) to know where its involvement starts
                                and where it ends -- sounds like that's the first place we need to spend
                                some time on to better define that boundary (of what fribidi will do vs.
                                what vim will handle).

                                But to Answer your question Bram - I'd assume an "l" move should end-up on
                                'T' -- simply put, it seems reasonable to follow the physical/logical order
                                in cursor movement (physical/logical -> what's stored on disk which
                                corresponds to the order a user enter characters).

                                > Just one command and already we have a dilemma...

                                I don't think these are dilemma, I think this is just the learning curve
                                (for you and us) to get upto speed - sounds like Yousof has done some
                                research and might be a great asset to make this a reality (any resources
                                other than fribidi that's you'd recommend ?).

                                - Nadim

                                __________________________________________________
                                Do You Yahoo!?
                                Check out Yahoo! Shopping and Yahoo! Auctions for all of
                                your unique holiday gifts! Buy at http://shopping.yahoo.com
                                or bid at http://auctions.yahoo.com
                              • Yousof Mortazavi
                                ... I agree. The best resource I know of is the Unicode Standard Annex #9 (The BiDi Algorithm) available online at http://www.unicode.org/unicode/reports/tr9/
                                Message 15 of 18 , Dec 15, 2001
                                • 0 Attachment
                                  On Sat, 15 Dec 2001, Nadim Shaikli wrote:

                                  > > Just one command and already we have a dilemma...
                                  >
                                  > I don't think these are dilemma, I think this is just the learning curve
                                  > (for you and us) to get upto speed - sounds like Yousof has done some
                                  > research and might be a great asset to make this a reality (any resources
                                  > other than fribidi that's you'd recommend ?).

                                  I agree. The best resource I know of is the Unicode Standard
                                  Annex #9 (The BiDi Algorithm) available online at

                                  http://www.unicode.org/unicode/reports/tr9/

                                  (Let me know if you find anything interesting.)
                                Your message has been successfully submitted and would be delivered to recipients shortly.